RFR: 8346971: [ubsan] psCardTable.cpp:131:24: runtime error: large index is out of bounds

[Thomas Schatzl]

On Fri, 10 Jan 2025 11:21:31 GMT, Kim Barrett <kbarrett at openjdk.org> wrote:

> Please review this change to PSStripeShadowCardTable to avoid several examples
> of UB in it's internal calculations.  We avoid the UB by switching to the
> integer domain (using uintptr_t) for all of the internal calculations, with
> casts between pointers and uintptr_t as needed at the boundaries.
> 
> This applies not just to the various pointer adjustments, but also to pointer
> comparisons. In particular, the prior range check assertions using pointer
> comparisons could have been partially or even completely "optimized" away
> based on the no-UB assumption.
> 
> Testing: mach5 tier1-5
> local (linux-x64) tier1 with -XX:+UseParallelGC

src/hotspot/share/gc/parallel/psCardTable.cpp line 129:

> 127:   // Avoid UB pointer operations by using integers internally.
> 128: 
> 129:   static_assert(sizeof(intptr_t) == sizeof(CardValue*), "simplifying assumption");

Why check `sizeof(intptr_t)` instead of `uintptr_t` here?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/23032#discussion_r1912898906