SoftReferences not cleared before OOME

[Thomas Schatzl]

Hi,

On 11.06.24 17:52, Devin Smith wrote:
> Hi,
> 
> This is a repost of
> https://mail.openjdk.org/pipermail/hotspot-gc-use/2024-May/002938.html
> ; I'm elevating here as it seems like this list is more active.

   sorry, I meant to answer that but then got overlooked.

> 
> I'm able to reliably reproduce an issue whereby the JVM throws an
> OutOfMemoryException where the majority of heap space is reclaimable
> soft references. This can be verified with
> `-XX:+HeapDumpOnOutOfMemoryError` and analyzing the resulting dump in
> VisualVM (or other heap dump tools). The condition is triggered by
> also having concurrent computation inside JNI critical regions
> (subject to multiple attempts wrt `-XX:GCLockerRetryAllocationCount`).
> 
> ```
> [1.242s][warning][gc,alloc] main: Retried waiting for GCLocker too
> often allocating 524290 words
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>          at java.base/java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:71)
>          at java.base/java.nio.ByteBuffer.allocate(ByteBuffer.java:391)
>          at io.deephaven.example.GCLockerTooOftenAllocating.main(GCLockerTooOftenAllocating.java:45)
> ```
> 
> This behavior seems to go against one of the core tenants of memory
> management / SoftReference docs:
> 
>> All soft references to softly-reachable objects are guaranteed to have been cleared before the virtual machine throws an OutOfMemoryError
> 
> I'm able to reproduce this across all LTS versions with a variety of
> GC collectors; it appears that ZGC and Shenandoah do not exhibit this
> behavior. Possibly b/c ZGC has already fixed this bug
> https://bugs.openjdk.org/browse/JDK-8289838 (there are a number of
> OpenJDK issues with similar titles).

In JDK 22 G1 also seems to pass the application due to JEP 423. At least 
I could not see any OOMEs in several attempts with that JDK version.

> https://github.com/devinrsmith/GCLockerTooOftenAllocating provides the
> example code necessary to reproduce.
> 
> Is this an OpenJDK bug?
 >
 > If so, would somebody with permissions elevate this to 
https://bugs.openjdk.org?

Imo this is a special case of 
https://bugs.openjdk.org/browse/JDK-8192647: when exiting the JNI 
critical section, the system will automatically issue a garbage 
collection by the last thread exiting the critical section.

That particular thread will do a GC that does not clear soft references 
(because the regular conditions are not met yet), and does not know that 
other threads are starved for memory.

This GC will not free up any memory, so the memory requesting thread(s) 
will retry, but again and again some other JNI thread will block their 
GC attempts to actually free up memory until the retry count is 
exceeded, leading to the OOME.

> 
> If it's not a bug, I'm hoping to gain some clarity on SoftReference /
> OOME guarantees.
> 
> I'm happy to provide additional assistance if needed.
> 

I added a note to that issue, referencing your reproducer.

Hth,
   Thomas