RFR: 8354119: Missing C2 proper allocation failure handling during initialization (during generate_uncommon_trap_blob)

Damon Fenacci dfenacci at openjdk.org
Tue Apr 22 12:37:22 UTC 2025


After [JDK-8347406](https://bugs.openjdk.org/browse/JDK-8347406), `OptoRuntime::generate_uncommon_trap_blob` and `OptoRuntime::generate_exception_blob` return an `UncommonTrapBlob`/`ExceptionBlob` if they succeed, `nullptr` if they don't. This is then used by the compiler to shut down gently if the code cache is full (instead of crashing).
Unfortunately if the the full code cache is reached when creating the buffer at the start of these 2 methods (when calling `CodeBuffer buffer(name, 2048, 1024);`) an empty buffer is created, which in turn prevents `masm` to be properly initialized, which then causes an access violation when writing into the blob's address when first adding `subptr` later in the method (as seen in the snippet below for `generate_uncommon_trap_blob`).

https://github.com/openjdk/jdk/blob/3cc43b3224efdf1a3f35fff58b993027a9e1f4ad/src/hotspot/cpu/x86/runtime_x86_64.cpp#L55-L72

To fix this I suggest we return immediately from `OptoRuntime::generate_uncommon_trap_blob`/`OptoRuntime::generate_exception_blob` if the `buffer` creation failed.

### Testing

Tier 1-3.
No specific regression test is added (very hard, i.a. dependent on thread scheduling. On the other hand `StartupOutput.java` might catch it rarely).

-------------

Commit messages:
 - JDK-8354119: remove unused import
 - JDK-8354119: Missing C2 proper allocation failure handling during initialization (during generate_uncommon_trap_blob)

Changes: https://git.openjdk.org/jdk/pull/24549/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=24549&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8354119
  Stats: 36 lines in 8 files changed: 36 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/24549.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/24549/head:pull/24549

PR: https://git.openjdk.org/jdk/pull/24549


More information about the hotspot-compiler-dev mailing list