RFR: 8326615: C1/C2 don't handle allocation failure properly during initialization (RuntimeStub::new_runtime_stub fatal crash) [v9]

[Vladimir Kozlov]

On Thu, 29 Aug 2024 06:37:34 GMT, Damon Fenacci <dfenacci at openjdk.org> wrote:

>> # Issue
>> 
>> The test `compiler/startup/StartupOutput.java` fails intermittently due to a crash after correctly printing the error `Initial size of CodeCache is too small` (the test limits the code cache using k-XX:InitialCodeCacheSize=1024K -XX:ReservedCodeCacheSize=1200k`).
>> The appearance of the issue is very dependent on thread scheduling. The original report happens during C1 initialization but C2 initialization is affected as well.
>> 
>> # Causes
>> 
>> There is one occurrence during C1 initialization  and one during C2 initialization where a call to `RuntimeStub::new_runtime_stub` can fail fatally if there is not enough space left.
>> For C1: `Compiler::init_c1_runtime` -> `Runtime1::initialize` -> `Runtime1::generate_blob_for` -> `Runtime1::generate_blob` -> `RuntimeStub::new_runtime_stub`.
>> For C2: `C2Compiler::initialize` -> `OptoRuntime::generate` -> `OptoRuntime::generate_stub` -> `Compile::Compile` -> `Compile::Code_Gen` -> `PhaseOutput::install` -> `PhaseOutput::install_stub` -> `RuntimeStub::new_runtime_stub`.
>> 
>> # Solution
>> 
>> https://github.com/openjdk/jdk/pull/15970 introduced an optional argument to `RuntimeStub::new_runtime_stub` to determine if it fails fatally or not. We can take advantage of it to avoid crashing and instead pass the information about the success or failure of the allocation up the (C1 and C2 initialization) call stack up to where we can set the compilations as failed.
>
> Damon Fenacci has updated the pull request incrementally with one additional commit since the last revision:
> 
>   JDK-8326615: update copyright year

src/hotspot/share/compiler/compilerDefinitions.inline.hpp line 29:

> 27: 
> 28: #include "c1/c1_Compiler.hpp"
> 29: #include "opto/c2compiler.hpp"

These should be also under  COMPILER*_PRESENT macros.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/19280#discussion_r1736635992