RFR: 8357135: java.lang.OutOfMemoryError: Error creating or attaching to libjvmci after JDK-8356447

Doug Simon dnsimon at openjdk.org
Mon May 19 17:56:01 UTC 2025


As of [JDK-8356447](https://bugs.openjdk.org/browse/JDK-8356447), libgraal initialization happens during VM startup. If during this initialization, the libgraal heap cannot be created due to lack of virtual address space, the VM will exit with:


Error occurred during initialization of VM
java.lang.OutOfMemoryError: Error creating or attaching to libjvmci (err: -1000000801, description: Reserving address space for the new isolate failed.)


This causes problems for tests that limit the virtual address space with `ulimit -v` such as `gc/arguments/TestUseCompressedOopsFlagsWithUlimit.java` and `vmTestbase/nsk/jvmti/Allocate/alloc001/alloc001.java`.
Instead of exiting the VM, the failure should be silent (unless `-XX:+PrintCompilation` is enabled) as the VM can continue without libgraal, albeit in a crippled state. This PR implements this solution.

Alternative solutions include:
1. Trying to adjust the values used with `ulimit -v` in the tests to accommodate the [virtual address reservations](https://github.com/oracle/graal/blob/69f10d3d658a6aeca3d5ce59c64af6a18336f14c/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge/AddressRangeCommittedMemoryProvider.java#L150) needed by libgraal. This is brittle as it assumes knowledge about how much address space is needed (which is turn depends on how many libgraal compiler threads are created).
2. Add a `@requires !vm.libgraal.jit` guard to the tests so they are not run when libgraal is in use.

I think the solution in this PR is the most robust for the long term.

-------------

Commit messages:
 - do not exit VM if libjvmci env creation fails

Changes: https://git.openjdk.org/jdk/pull/25307/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=25307&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8357135
  Stats: 29 lines in 3 files changed: 9 ins; 17 del; 3 mod
  Patch: https://git.openjdk.org/jdk/pull/25307.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/25307/head:pull/25307

PR: https://git.openjdk.org/jdk/pull/25307


More information about the graal-dev mailing list