RFR: 8340297: Use-after-free recognition for metaspace and class space [v11]

Tue Jan 27 10:49:17 UTC 2026

On Thu, 4 Dec 2025 10:48:25 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> This patch will give us use-after-free recognition for Metaspace and Class space. 
>> 
>> Currently, checks for Klass validity typically only perform a variation of `Metaspace::contains` and some other basic tests. These checks won't find cases where the Klass had been prematurely freed (e.g., after class redefinition), nor cases of unloaded classes if the underlying metaspace chunks have not been uncommitted, which is quite common. 
>> 
>> The patch also provides us with improved analysis methods in case we encounter problems. E.g., answering whether the Klass had been redefined or unloaded.
>> 
>> The implementation aims to be simple, fast, and safe against false positives. There is a small but non-null chance that we could get false negatives, but that cannot be avoided.
>> 
>> How this works:
>> 
>> - In `class Metadata`, we introduce a 32-bit token that holds the type of the object (1). It replaces the old "is_valid" field of the same size. That one was of limited use since any non-null garbage in those four bytes would be read as valid.
>> - To check a Metadata for validity, the token is checked. Checks are done with SafeFetch, so they can be done with questionable pointers (e.g. into uncommitted metaspace after class unloading)
>> - When metaspace is freed (bulk free after class unloading), the released chunks are zapped, destroying all tokens in the area.
>> - When metaspace is freed (prematurely, e.g., after class redefinition), the released blocks are zapped.
>> - The new checks replace Metadata::is_valid and supplement some other metadata checks done in GCs
>> 
>> Testing: The patch has been extensively tested manually, at Oracle, and SAP. Tests were thorough to not only catch errors in the patch, but also to see if the patch would uncover a lot of existing sleeper bugs. So far, we only found a single bug in Shenandoah.
>> 
>> Note: I did not yet hook up the new test to c1/c2 compiled code (there are already unimplemented functions for that). That is possible, but left for a later RFE.
>
> Thomas Stuefe has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 21 commits:
> 
>  - Merge branch 'openjdk:master' into JDK-8340297-Metaspace-API-for-checking-if-address-is-in-use
>  - includes sorted
>  - Merge branch 'master' into JDK-8340297-Metaspace-API-for-checking-if-address-is-in-use
>  - Merge master
>  - Feedback Johan, Axel
>  - Merge branch 'master' into JDK-8340297-Metaspace-API-for-checking-if-address-is-in-use
>  - remove stray macro
>  - feedback Caspar
>  - Merge branch 'master' into JDK-8340297-Metaspace-API-for-checking-if-address-is-in-use
>  - Feedback Johan
>  - ... and 11 more: https://git.openjdk.org/jdk/compare/b5970c97...8682e3b3

Sorry for leaving this hanging. I skimmed through it, and had no major concerns. Mostly focusing on Shenandoah parts here:

src/hotspot/share/gc/g1/g1HeapRegion.cpp line 468:

> 466:   if (!Metaspace::klass_is_live(klass, true, &hint)) {
> 467:     log_error(gc, verify)("klass " PTR_FORMAT " of object " PTR_FORMAT " "
> 468:                           "is an invalid klass pointer, not in metaspace, or refers to a dead klass (Hint: %u)",

OK, here and other places: are we able to translate the hint to something human-readable?

src/hotspot/share/gc/shenandoah/shenandoahAsserts.cpp line 246:

> 244:   if (!os::is_readable_pointer(obj)) {
> 245:     print_failure(_safe_unknown, obj, interior_loc, nullptr, "Shenandoah assert_correct failed",
> 246:                   "Object klass pointer should not be null",

But that's not what the check does, does it? It literally checks for readability, not for klass pointer being nullptr. Also, unnecessary white space change on the line below.

src/hotspot/share/gc/shenandoah/shenandoahAsserts.cpp line 318:

> 316: 
> 317:   if (!Metaspace::klass_is_live(obj_klass, true, &hint)) {
> 318:     printf_failure(_safe_unknown, obj, interior_loc, nullptr, "Shenandoah assert_correct failed",

Eh... This needs to be simpler.

Instead of introducing the whole new type of failure printing -- and having bugs there on their own -- perhaps you want to amend the generic `ShenandoahAsserts::print_obj` handler so it reports the klass status there?

src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp line 166:

> 164:            "Object klass pointer unreadable or invalid");
> 165: 
> 166:     check_v(ShenandoahAsserts::_safe_unknown, obj, Metaspace::klass_is_live(obj_klass, true, &hint),

Same comment as for `ShenandoahAsserts`: once you do the klass status printouts in `ShenandoahAsserts::print_failure`, you get the verifier printouts automagically.

src/hotspot/share/memory/metaspace/blockTree.hpp line 82:

> 80: 
> 81:     // A leading canary section that is zapped with the same metaspace zap pattern
> 82:     // as the rest of the payload following the header (and all of dead/unsused metaspace).

Suggestion:

    // as the rest of the payload following the header (and all of dead/unused metaspace).

src/hotspot/share/oops/metadata.hpp line 105:

> 103: }
> 104: 
> 105: #undef BUILD32

Leftover?

-------------

PR Review: https://git.openjdk.org/jdk/pull/25891#pullrequestreview-3710341484
PR Review Comment: https://git.openjdk.org/jdk/pull/25891#discussion_r2731300420
PR Review Comment: https://git.openjdk.org/jdk/pull/25891#discussion_r2731309020
PR Review Comment: https://git.openjdk.org/jdk/pull/25891#discussion_r2731349094
PR Review Comment: https://git.openjdk.org/jdk/pull/25891#discussion_r2731355481
PR Review Comment: https://git.openjdk.org/jdk/pull/25891#discussion_r2731378965
PR Review Comment: https://git.openjdk.org/jdk/pull/25891#discussion_r2731395053