RFR: 8370198: Test gc/arguments/TestShrinkHeapInSteps.java crashed: assert(left >= right) failed: avoid underflow

[Stefan Karlsson]

On Thu, 20 Nov 2025 11:34:26 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

> > Could this the addition be added to a function (or macro) so that the check becomes a one-liner.
> 
> I can't come up with a good name to abstract its semantics -- its correct invocation requires quite specific calling context, so I think it's best to "inline" it to the caller.

That doesn't instill confidence. I think we might have to figure out what semantics we want the check to have. Maybe it can be fixed by the suggestion below:

> 
> > Is there a reason why the added check couldn't be done at the beginning of the function?
> 
> `MacroAssembler::set_narrow_klass` is an example that will fail the assert, if placed at the beginning of the method.
> 
> The thing is that `CollectedHeap::is_in` need to take heap-resizing into account for input address that lie within heap-reserved space, which places some extra constraint for callers. OTOH, for outside-heap-reserved addresses, heap-resizing is orthogonal to the result, so calling constraint can be relaxed.

This also means that this calling code can only use the negative version of that check:

  assert(! Universe::heap()->is_in(k), "should not be an oop");

A check like:

  assert(Universe::heap()->is_in(obj), "should be an oop");

would trigger the introduced assert that we are calling this from `native`.

I suggest that we remove this assert. It was needed when we had PermGen, but now we have other safeguards. Just a few lines below we have `CompressedKlassPointers::encode(k);`, which I think performs enough sanity check on the provided `Klass*`

-------------

PR Comment: https://git.openjdk.org/jdk/pull/28393#issuecomment-3561912427