RFR: 8330174: Protection zone for easier detection of accidental zero-nKlass use

Thomas Stuefe stuefe at openjdk.org
Sat Jan 18 17:21:44 UTC 2025


If we wrongly decode an nKlass of `0`, and the nKlass encoding base is not NULL (typical for most cases that run with CDS enabled), the resulting pointer points to the start of the Klass encoding range. That area is readable. If CDS is enabled, it will be at the start of the CDS metadata archive. If CDS is off, it is at the start of the class space. 

Now, both CDS and class space allocate a safety buffer at the start to prevent Klass structures from being located there. However, that memory is still readable, so we can read garbage data from that area. In the case of CDS, that area is just 16 bytes, after that come real data. Since Klass is large, most accesses will read beyond the 16-byte zone.

We should protect the first page in the narrow Klass encoding range to make analysis of errors like this easier. Especially in release builds where decode_not_null does not assert. We already use a similar technique in the heap, and most OSes protect the zero page for the same reason.

This patch does that. Now, decoding an `0` nKlass and then using the result `Klass` - calling virtual functions or accessing members - crashes right away.

Additionally, the patch provides a helpful output in the register/stack section, e.g:


RDI=0x0000000800000000 points into nKlass protection zone



Testing: 
- GHAs.
- I tested the patch manually on x64 Linux for both CDS on, CDS off and zero-based encoding, CDS off and non-zero-based encoding.
- I also prepared an automatic gtest, but that needs some preparatory work on the gtest suite first to work (see https://bugs.openjdk.org/browse/JDK-8348029)

-------------

Commit messages:
 - fix 32bit build
 - fix windows crash on os::protect
 - start

Changes: https://git.openjdk.org/jdk/pull/23190/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=23190&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8330174
  Stats: 166 lines in 8 files changed: 147 ins; 16 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/23190.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/23190/head:pull/23190

PR: https://git.openjdk.org/jdk/pull/23190


More information about the hotspot-dev mailing list