RFR: 8340297: Use-after-free recognition for metaspace and class space

Thomas Stuefe stuefe at openjdk.org
Sun Jul 20 05:30:54 UTC 2025


This patch will give us use-after-free recognition for Metaspace and Class space. 

Currently, checks for Klass validity typically only perform a variation of `Metaspace::contains` and some other basic tests. These checks won't find cases where the Klass had been prematurely freed (e.g., after class redefinition), nor cases of unloaded classes if the underlying metaspace chunks have not been uncommitted, which is quite common. 

The patch also provides us with improved analysis methods in case we encounter problems. E.g., answering whether the Klass had been redefined or unloaded.

The implementation aims to be simple, fast, and safe against false positives. There is a small but non-null chance that we could get false negatives, but that cannot be avoided.

How this works:

- In `class Metadata`, we introduce a 32-bit token that holds the type of the object (1). It replaces the old "is_valid" field of the same size. That one was of limited use since any non-null garbage in those four bytes would be read as valid.
- To check a Metadata for validity, the token is checked. Checks are done with SafeFetch, so they can be done with questionable pointers (e.g. into uncommitted metaspace after class unloading)
- When metaspace is freed (bulk free after class unloading), the released chunks are zapped, destroying all tokens in the area.
- When metaspace is freed (prematurely, e.g., after class redefinition), the released blocks are zapped.
- The new checks replace Metadata::is_valid and supplement some other metadata checks done in GCs

Testing: The patch has been extensively tested manually, at Oracle, and SAP (the latter still ongoing). Test were thorough to not only catch errors in the patch, but also to see if the patch would uncover a lot of existing bugs. So far we only found a single bug in Shenandoah.

Note: I did not yet hook up the new test to c1/c2 compiled code (there are already unimplemented functions for that). That is possible, but left for a later RFE.

-------------

Commit messages:
 - copyrights
 - fix big-endian problem on AIX
 - Update klass.cpp
 - Update metaspace.hpp
 - Update metaspace.hpp
 - Update metaspace.hpp
 - fix rebase error
 - fix mac build
 - rebase, fixes

Changes: https://git.openjdk.org/jdk/pull/25891/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=25891&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8340297
  Stats: 482 lines in 26 files changed: 398 ins; 27 del; 57 mod
  Patch: https://git.openjdk.org/jdk/pull/25891.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/25891/head:pull/25891

PR: https://git.openjdk.org/jdk/pull/25891


More information about the hotspot-dev mailing list