RFR: 8327176: UnreferencedExecutor.java can fail on libgraal with -Xcomp

[Doug Simon]

On Sun, 3 Mar 2024 17:03:51 GMT, Doug Simon <dnsimon at openjdk.org> wrote:

>> The `java/util/concurrent/Executors/UnreferencedExecutor.java` test can fail when run on libgraal and `-Xcomp` is specified. The problem is that libgraal in `-Xcomp` temporarily causes some extra memory pressure (probably due to [JDK-8310218](https://bugs.openjdk.org/browse/JDK-8310218)) which can cause recoverable OOMEs to occur (memory is recovered when the relevant libgraal compilations complete). It also seems related to async threads used for cleaning weak references when using G1 or ZGC as I cannot reproduce the failure under `-XX:+UseSerialGC`.
>> 
>> Installing a global `Thread.UncaughtExceptionHandler` that ignores `OutOfMemoryError`s resolves the problem.
>
> I confirmed that `java/util/concurrent/Executors/UnreferencedExecutor.java` still fails if the [fix applied to ThreadContainers.java](https://github.com/openjdk/jdk/commit/ada416e66cbff6c8e631bf352acc0744c248740b#diff-1e347b9a95cc4fe81a01ca70e4122a73a65d99a69668c7567abb2a6067f8cc6dL68) is reverted.

> @dougxc I added this test when fixing JDK-8308235. The point of the test is that OOME is not thrown so adding a default UEH to ignore OOME is changing the test. I wonder if this is a test that just needed to be skipped with libgraal.

I understood the point of that test to be that an OOME does not cause the VM to exit. As i mentioned (https://github.com/openjdk/jdk/pull/18098#issuecomment-1975231258), the test still detects the memory leak plugged by JDK-8308235.

That said, I have tried to reproduce the failure on a more recent libgraal and JDK build and am having no luck. I'll close this PR and come up with alternative solutions if the failure starts reappearing.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/18098#issuecomment-2011700020