Integrated: 8330174: Protection zone for easier detection of accidental zero-nKlass use

[Thomas Stuefe]

On Sat, 18 Jan 2025 11:20:00 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> If we wrongly decode an nKlass of `0`, and the nKlass encoding base is not NULL (typical for most cases that run with CDS enabled), the resulting pointer points to the start of the Klass encoding range. That area is readable. If CDS is enabled, it will be at the start of the CDS metadata archive. If CDS is off, it is at the start of the class space. 
> 
> Now, both CDS and class space allocate a safety buffer at the start to prevent Klass structures from being located there. However, that memory is still readable, so we can read garbage data from that area. In the case of CDS, that area is just 16 bytes, after that come real data. Since Klass is large, most accesses will read beyond the 16-byte zone.
> 
> We should protect the first page in the narrow Klass encoding range to make analysis of errors like this easier. Especially in release builds where decode_not_null does not assert. We already use a similar technique in the heap, and most OSes protect the zero page for the same reason.
> 
> This patch does that. Now, decoding an `0` nKlass and then using the result `Klass` - calling virtual functions or accessing members - crashes right away.
> 
> Additionally, the patch provides a helpful output in the register/stack section, e.g:
> 
> 
> RDI=0x0000000800000000 points into nKlass protection zone
> 
> 
> 
> Testing: 
> - GHAs.
> - I tested the patch manually on x64 Linux for both CDS on, CDS off and zero-based encoding, CDS off and non-zero-based encoding.
> - I tested manually on Windows x64
> - I also prepared an automatic gtest, but that needs some preparatory work on the gtest suite first to work (see https://bugs.openjdk.org/browse/JDK-8348029)
> 
> -- Update 2024-01-22 -- 
> I added a jtreg test that is more thorough than a gtest (also scans the produced hs-err file)

This pull request has now been integrated.

Changeset: a70eba8e
Author:    Thomas Stuefe <stuefe at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/a70eba8e4212c2c7125475f69b3952197e7a8ce3
Stats:     426 lines in 16 files changed: 331 ins; 29 del; 66 mod

8330174: Protection zone for easier detection of accidental zero-nKlass use

Co-authored-by: Ioi Lam <iklam at openjdk.org>
Reviewed-by: iklam, rkennke

-------------

PR: https://git.openjdk.org/jdk/pull/23190