RFR: 8321400: java/foreign/TestStubAllocFailure.java fails with code cache exhaustion

[Jorn Vernee]

On Mon, 11 Dec 2023 13:01:25 GMT, Jorn Vernee <jvernee at openjdk.org> wrote:

> Improve the test by being more lenient to related code cache exhaustion errors. The important thing is that we don't terminate with a fatal error, which the new code now checks for explicitly. The check for that is based on what is done by `./test/hotspot/jtreg/runtime/ErrorHandling/HsErrFileUtils.java` .
> 
> The existing `UpcallTestHelper.Output` class that was previously used to assert on stdout/stderr contents did not have the capability to look for patterns in the output. So, I've taken the opportunity to replace it with the more canonical `OutputAnalyzer` which comes from the test library.
> 
> Finally, I've also added back the test for downcall stub allocation failure which was removed as part of the initial patch because it was too inconsistent [1]. With the now approach, it should pass reliably as well.
> 
> Testing: `jdk_foreign`  suite (which contains all the affected tests)
> 
> [1]: https://github.com/openjdk/jdk/pull/16311/commits/9a1360598a91871ce6ec48330849c0e4e0279c64

test/jdk/java/foreign/TestStubAllocFailure.java line 64:

> 62:         public static void main(String[] args) throws Throwable {
> 63:             FunctionDescriptor descriptor = FunctionDescriptor.ofVoid();
> 64:             MethodHandle target = MethodHandles.lookup().findStatic(UpcallRunner.class, "target", descriptor.toMethodType());

Here I'm extracting the method handle lookup from the loop to make it more likely that we hit the code path that we are interested in, and don't get another VirtualMachineError due to this method handle lookup.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/17056#discussion_r1422479110