[lworld] RFR: 8354404: [lworld] compiler/startup/StartupOutput.java crashes during AdapterHandlerLibrary::initialize() due to too little CodeCacheSize
Marc Chevalier
mchevalier at openjdk.org
Mon Jul 21 07:45:53 UTC 2025
Experimentally, replacing 800 with 2000 is needed to have this test pass on all platform. Somehow, while 1300 seems ok on my Linux (while having a fair share of "CodeCache is full. Compiler has been disabled." messages), Windows boxes seems to struggle even with 1800. Not sure why: isn't the size of the things in the code cache (that is, the code), depends mostly on the encoding of instructions, that is dependent on architecture, not OS? Anyway, it happens reliably, so something had to be done!
Let's compare (with debug builds so I can use `CodeCacheMinimumUseSpace`, that is 1200k by default on debug builds), for various values of `ReservedCodeCacheSize` (in k), when running `-version`. For mainline:
- [4, 28]
```
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 3000 bytes. Error detail: `CodeCache: no room for StubRoutines` (continuation stubs)
```
coming from
- `report_should_not_call(char const*, int)`
- `initialize_stubs(BlobId, int, int, char const*, char const*, char const*)`
- `StubRoutines::initialize_continuation_stubs()`
- `continuation_stubs_init()`
- `init_globals()`
with `3000` being `20000` for `ReservedCodeCacheSize` between 5 and 24, and being `500` for `ReservedCodeCacheSize` = 4 (the minimum).
- [29, 32] `fatal error: Initial size of CodeCache is too small`
https://github.com/openjdk/jdk/blob/b787ad6f690df5c82a1efc5ccac658a9238ff201/src/hotspot/share/code/codeBlob.cpp#L579
coming from
- `RuntimeStub::new_runtime_stub(char const*, CodeBuffer*, short, int, OopMapSet*, bool, bool)`
- `SharedRuntime::generate_jfr_return_lease()`
- `SharedRuntime::generate_jfr_stubs()`
- `init_globals()`
- [33, 1112]
```
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 1108928 bytes. Error detail: CodeCache: no room for Interpreter
```
- `...`
- `report_should_not_call(char const*, int)`
- `StubQueue::StubQueue(StubInterface*, int, Mutex*, char const*)`
- `TemplateInterpreter::initialize_stub()`
- `interpreter_init_stub()`
- `init_globals()`
- [1113, 1116] SIGSEGV
- `CodeSection::emit_int8(unsigned char)`
- `AbstractAssembler::emit_int8(int)`
- `Assembler::push(Register)`
- `MacroAssembler::enter()`
- `RegisterSaver::save_live_registers(MacroAssembler*, int, int*, bool)`
- `SharedRuntime::generate_resolve_blob(StubId, unsigned char*)`
- `SharedRuntime::generate_stubs()`
- `init_globals()`
- [1117, 1128] `fatal error: Initial size of CodeCache is too small`
https://github.com/openjdk/jdk/blob/b787ad6f690df5c82a1efc5ccac658a9238ff201/src/hotspot/share/code/codeBlob.cpp#L596
coming from
- `SingletonBlob::operator new(unsigned long, unsigned int, bool)`
- `DeoptimizationBlob::create(CodeBuffer*, OopMapSet*, int, int, int, int)`
- `SharedRuntime::generate_deopt_blob()`
- `SharedRuntime::generate_stubs()`
- `init_globals()`
- [1129, 1144] `assert(_buffer != nullptr) failed: should be initialized`
https://github.com/openjdk/jdk/blob/b787ad6f690df5c82a1efc5ccac658a9238ff201/src/hotspot/share/runtime/sharedRuntime.cpp#L2526
coming from
- `AdapterHandlerLibrary::generate_adapter_code(AdapterBlob*&, AdapterHandlerEntry*, int, BasicType*, bool)`
- `AdapterHandlerLibrary::create_adapter(AdapterBlob*&, int, BasicType*, bool)`
- `AdapterHandlerLibrary::initialize()`
- `SharedRuntime::init_adapter_library()`
- `init_globals()`
- [1145, 1148] `assert(no_arg_blob != nullptr && obj_arg_blob != nullptr && int_arg_blob != nullptr && obj_int_arg_blob != nullptr && obj_obj_arg_blob != nullptr) failed: Initial adapters must be properly created`
https://github.com/openjdk/jdk/blob/b787ad6f690df5c82a1efc5ccac658a9238ff201/src/hotspot/share/runtime/sharedRuntime.cpp#L2605-L2609
coming from
- `AdapterHandlerLibrary::initialize()`
- `SharedRuntime::init_adapter_library()`
- `init_globals()`
(just after the previous)
- [1149, ~1700]
```
Java HotSpot(TM) 64-Bit Server VM warning: C1 initialization failed. Shutting down all compilers
Java HotSpot(TM) 64-Bit Server VM warning: C2 initialization failed. Shutting down all compilers
```
and eventually works, but occasionally crashes with
```
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 109000 bytes. Error detail: CodeCache: no room for StubRoutines (compiler stubs)
```
from
- `report_should_not_call(char const*, int)`
- `initialize_stubs(BlobId, int, int, char const*, char const*, char const*)`
- `StubRoutines::initialize_compiler_stubs()`
- `compiler_stubs_init(bool)`
- `C2Compiler::init_c2_runtime()`
- `C2Compiler::initialize()`
especially around 1180k, still happening at 1200k, which is the default minimum.
- Then a various number of `C1 initialization failed` (and sometimes `C2 initialization failed`) but it overall works.
And now, for Valhalla:
- [4, 20] finding the first behavior `CodeCache: no room for StubRoutines`
- [21, 24] a new kind of SIGSEGV!
- `CodeSection::emit_int8(unsigned char)`
- `AbstractAssembler::emit_int8(int)`
- `Assembler::prefix(Assembler::Prefix)`
- `Assembler::prefixq_and_encode(int, bool)`
- `Assembler::subq(Register, int)`
- `MacroAssembler::subptr(Register, int)`
- `StubGenerator::generate_return_value_stub(unsigned char*, char const*, bool)`
- `StubGenerator::generate_initial_stubs()`
- `StubGenerator::StubGenerator(CodeBuffer*, StubGenBlobId)`
- `StubGenerator_generate(CodeBuffer*, StubGenBlobId)`
- `initialize_stubs(StubGenBlobId, int, int, char const*, char const*, char const*)`
- `StubRoutines::initialize_initial_stubs()`
- `initial_stubs_init()`
- `init_globals()`
- [25, 28] first behavior again `CodeCache: no room for StubRoutines`
- [29, 32] `fatal error: Initial size of CodeCache is too small`
https://github.com/openjdk/valhalla/blob/b787ad6f690df5c82a1efc5ccac658a9238ff201/src/hotspot/share/code/codeBlob.cpp#L579
like the [29, 32] range of mainline
- [33, 1164] `Native memory allocation (malloc) failed to allocate 1158080 bytes. Error detail: CodeCache: no room for Interpreter` like the [33, 1112] range of mainline
- [1165, 1172] `fatal error: Initial size of CodeCache is too small`
https://github.com/openjdk/valhalla/blob/b787ad6f690df5c82a1efc5ccac658a9238ff201/src/hotspot/share/code/codeBlob.cpp#L579
- `RuntimeStub::new_runtime_stub(char const*, CodeBuffer*, short, int, OopMapSet*, bool, bool)`
- `SharedRuntime::generate_resolve_blob(SharedStubId, unsigned char*)`
- `SharedRuntime::generate_stubs()`
- `init_globals()`
- [1173, 1220] SIGSEGV
- `CodeBlob::name() const`
- `CodeBuffer::CodeBuffer(CodeBlob*)`
- `AdapterHandlerLibrary::create_adapter(AdapterBlob*&, CompiledEntrySignature&, bool)`
- `AdapterHandlerLibrary::initialize()`
- `SharedRuntime::generate_stubs()`
- `init_globals()`
actually analogous to the range [1129, 1144] (`assert(_buffer != nullptr) failed: should be initialized`) of mainline. But Valhalla is missing this assert yet, the `_buffer` is `nullptr`, and `CodeBlob::name() const` is called on this null pointer that is provided to the ctor of `CodeBuffer`
- [1221, 1224] `assert(no_arg_blob != nullptr && obj_arg_blob != nullptr && int_arg_blob != nullptr && obj_int_arg_blob != nullptr && obj_obj_arg_blob != nullptr) failed: Initial adapters must be properly created`
https://github.com/openjdk/valhalla/blob/b787ad6f690df5c82a1efc5ccac658a9238ff201/src/hotspot/share/runtime/sharedRuntime.cpp#L2605-L2609
same as the range [1145, 1148] of mainline
- [1225, 1228] SIGSEGV
- `CodeSection::emit_int8(unsigned char)`
- `AbstractAssembler::emit_int8(int)`
- `Assembler::push(Register)`
- `SharedRuntime::generate_handler_blob(SharedStubId, unsigned char*)`
- `SharedRuntime::generate_stubs()`
- `init_globals()`
- [1229, 1232] `fatal error: Initial size of CodeCache is too small`
https://github.com/openjdk/valhalla/blob/b787ad6f690df5c82a1efc5ccac658a9238ff201/src/hotspot/share/code/codeBlob.cpp#L596
same as the [1117, 1128] range of mainline
- [1233, 1236]
```
Java HotSpot(TM) 64-Bit Server VM warning: C1 initialization failed. Shutting down all compilers
Java HotSpot(TM) 64-Bit Server VM warning: C2 initialization failed. Shutting down all compilers
Error occurred during initialization of boot layer
java.lang.InternalError: java.lang.NoSuchMethodException: no such method: java.lang.invoke.MethodHandle.linkToSpecial(Object,Object,Object,MemberName)Object/invokeStatic
Suppressed: java.lang.BootstrapMethodError: bootstrap method initialization exception
Caused by: java.lang.NoSuchMethodException: no such method: java.lang.invoke.MethodHandle.linkToSpecial(Object,Object,Object,MemberName)Object/invokeStatic
```
exit code = 1
- [1237, 1300]
```
Java HotSpot(TM) 64-Bit Server VM warning: C1 initialization failed. Shutting down all compilers
Java HotSpot(TM) 64-Bit Server VM warning: C2 initialization failed. Shutting down all compilers
Error occurred during initialization of boot layer
java.lang.OutOfMemoryError: Out of space in CodeCache for adapters
Suppressed: java.lang.OutOfMemoryError: Out of space in CodeCache for method handle intrinsic
```
exit code = 1
- [1301, 1304]
```
Java HotSpot(TM) 64-Bit Server VM warning: C1 initialization failed. Shutting down all compilers
Java HotSpot(TM) 64-Bit Server VM warning: C2 initialization failed. Shutting down all compilers
Error occurred during initialization of boot layer
java.lang.InternalError: java.lang.NoSuchMethodException: no such method: java.lang.invoke.MethodHandle.linkToSpecial(Object,MemberName)void/invokeStatic
Caused by: java.lang.NoSuchMethodException: no such method: java.lang.invoke.MethodHandle.linkToSpecial(Object,MemberName)void/invokeStatic
```
exit code = 1
- [1305, 1308]
```
Java HotSpot(TM) 64-Bit Server VM warning: C1 initialization failed. Shutting down all compilers
Java HotSpot(TM) 64-Bit Server VM warning: C2 initialization failed. Shutting down all compilers
Error occurred during initialization of boot layer
java.lang.InternalError: java.lang.NoSuchMethodException: no such method: java.lang.invoke.MethodHandle.linkToSpecial(Object,MemberName)int/invokeStatic
Caused by: java.lang.NoSuchMethodException: no such method: java.lang.invoke.MethodHandle.linkToSpecial(Object,MemberName)int/invokeStatic
```
exit code = 1
- [1309, 1792]
```
Java HotSpot(TM) 64-Bit Server VM warning: C1 initialization failed. Shutting down all compilers
Java HotSpot(TM) 64-Bit Server VM warning: C2 initialization failed. Shutting down all compilers
```
and eventually works
- [1793, 1993]
```
Error occurred during initialization of boot layer
java.lang.OutOfMemoryError: Out of space in CodeCache for adapters
```
exit code = 1
with possibly additional lines among:
```
Suppressed: java.lang.OutOfMemoryError: Out of space in CodeCache for adapters
Caused by: java.lang.NoSuchMethodException: no such method: java.lang.invoke.MethodHandle.linkToSpecial(Object,Object,MemberName)void/invokeStatic
```
(yes, an empty line)
- [1994, ...] mostly works, but sometimes the same error as above, witnessed until 3780k (and especially between 3640 and 3780). Interestingly, at 3700k, we also see some fine runs, with, or without compiler shutdown. Starting 3800k, I never saw a failure, and starting 3900k, no compiler shutdown.
So, how to pick `CodeCacheMinimumUseSpace`, that is the smaller value of `ReservedCodeCacheSize` we allow? On mainline with the default value (1200k on debug, 400k on product), we don't see any crash anymore. On Valhalla, at 2000k, it works always as far as I can see, but around 3700, it crashes in most cases (maybe two thirds or three quarters of the runs). If we are aiming for a value that would guarantee no crashes, we should pick at least 3800k, that is quite a huge increase. If we consider the weird crashes around 3700k as acceptable (and maybe fixable), 2000k is probably fine. Since that's the debug value, that is three times scaled up from the product one
https://github.com/openjdk/jdk/blob/b787ad6f690df5c82a1efc5ccac658a9238ff201/src/hotspot/share/code/codeCache.cpp#L208
that would mean taking `CodeCacheMinimumUseSpace` as 1267k (in the first case, maybe rounded to 1300k), or 667k (in the second case, maybe rounded to 700k).
That's for the minimum. As for the defaults: they still somewhat makes sense, in the meaning that we can run quite some things without filling the code cache. It's not clear to me what expectations we have and how we can decide.
I didn't change those as part of this PR, as the JBS issue is not primarily about that, and it looks like a more... subtle problem. But I can if a consensus emerges. Let the discussion begin!
Thanks,
Marc, the text wall builder (textmason?)
-------------
Commit messages:
- trying a bit more...
- Raise limit in StartupOutput.java
Changes: https://git.openjdk.org/valhalla/pull/1509/files
Webrev: https://webrevs.openjdk.org/?repo=valhalla&pr=1509&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8354404
Stats: 3 lines in 2 files changed: 0 ins; 2 del; 1 mod
Patch: https://git.openjdk.org/valhalla/pull/1509.diff
Fetch: git fetch https://git.openjdk.org/valhalla.git pull/1509/head:pull/1509
PR: https://git.openjdk.org/valhalla/pull/1509
More information about the valhalla-dev
mailing list