RFR: 8370198: Test gc/arguments/TestShrinkHeapInSteps.java crashed: assert(left >= right) failed: avoid underflow [v4]

[Stefan Karlsson]

On Thu, 4 Dec 2025 19:46:35 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>> Add an early-return for outside-heap address in `CollectedHeap::is_in` API.
>> 
>> While investigating this failure (JDK-8370198), I realized that some java-threads (compiler-threads) in `native` state can invoke `CollectedHeap` APIs. Since heap-resizing occurs inside safepoint but java-threads in `native` state just ignore safepoint, I have added some assert to catch such dangerous uses, where the return value might not be stable.
>> 
>> Test: tie1-5; can't reproduce the JDK-8370198 with or without this patch for >8000 runs.
>
> Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision:
> 
>   review

Changes requested by stefank (Reviewer).

src/hotspot/share/gc/serial/serialHeap.cpp line 634:

> 632: bool SerialHeap::is_in(const void* p) const {
> 633:   // precondition
> 634:   DEBUG_ONLY(verify_not_in_native_if_java_thread();)

No need for `DEBUG_ONLY` now that you've added `NOT_DEBUG_RETURN`
Suggestion:

  verify_not_in_native_if_java_thread();

-------------

PR Review: https://git.openjdk.org/jdk/pull/28393#pullrequestreview-3543492436
PR Review Comment: https://git.openjdk.org/jdk/pull/28393#discussion_r2591730326