RFR: JDK-8318016: Per-compilation memory ceiling [v3]

[Tobias Hartmann]

On Fri, 27 Oct 2023 12:57:07 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

>> This RFE introduces a new compile command to limit memory usage per compilation. It complements and is based upon the JIT memory statistic introduced with https://bugs.openjdk.org/browse/JDK-8317683.
>> 
>> This new memory limit is applied during compilation for either C1 or C2. It has one of two modes:
>> - *crash* mode (mainly for compiler devs), which aborts with hs-err file when the ceiling is reached, right in the compiler thread at the allocation point that triggered the ceiling
>> - *stop* mode, which stops the compilation similarly to how hitting the node limit in C2 would stop the compilation.
>> 
>> #### Usage
>> 
>> The compile command takes the form `-XX:CompileCommand:MemLimit,<method>,<memory size>[~<option>]`
>> 
>> The memory size that can be specified as usual. The command also takes an optional trailing option, separated from the memory size by a tilde (odd choice but needed because of https://bugs.openjdk.org/browse/JDK-8318214).
>> 
>> The `<suboption>` can be either `crash` or `stop` and distinguishes between the aforementioned modes. If omitted, the default is "stop".
>> 
>> Since `MemLimit` is based on the JIT statistic, it is implicitly enabled, so there is no need to explicitly enable it with `MemStat`.
>> 
>> 
>> Usage examples:
>> 
>> 
>> - crash when reaching 100m for a single compilation:
>>  
>> 
>> -XX:CompileCommand='MemLimit,*.*,100m~crash'
>> 
>> 
>> - limit all methods to a 2m footprint and print a trailing record at VM shutdown
>> 
>> 
>> -XX:CompileCommand='MemLimit,*.*,2m' -XX:CompileCommand='MemStat,*.*,print'
>> 
>> 
>> The JIT statistic table (printed either at VM shutdown or via `jcmd <pid> Compiler.memory`) now has a new column "result" that shows if the compilation was completed successfully or had an error or an oom:
>> 
>> 
>>                               vvvvv                                                                                                                                                                                                                                                
>> total     NA        RA        result  #nodes  time    type  #rc thread              method
>> 3036016   687328    1757456   oom     2607    0,320   c2    1   0x00007fcf1c181550  java/lang/invoke/StringConcatFactory::<clinit>(()V)
>> 
>> 
>> 
>> #### Implementation notes
>> 
>> The crash variant works immediately.
>> 
>> The stop variant works delayed: If arena memory reaches the limit in the scope of a compilation, we signal the Compile object (`Compile` fo...
>
> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix typo

That's a very useful feature, the implementation looks good to me. I've submitted some testing and will report back once it passed.

test/hotspot/jtreg/compiler/print/CompileCommandMemLimit.java line 2:

> 1: /*
> 2:  * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.

Suggestion:

 * Copyright (c) 2023, Oracle and/or its affiliates. All rights reserved.

test/hotspot/jtreg/compiler/print/CompileCommandMemLimit.java line 43:

> 41:  * @test id=c1_stop
> 42:  * @requires vm.compiler1.enabled
> 43:  * @summary Checks that -XX:CompileCommand=MemLimit,...,crash causes compilation to stop

Suggestion:

 * @summary Checks that -XX:CompileCommand=MemLimit,...,stop causes compilation to stop

test/hotspot/jtreg/compiler/print/CompileCommandMemLimit.java line 51:

> 49:  * @test id=c2_stop
> 50:  * @requires vm.compiler2.enabled
> 51:  * @summary Checks that -XX:CompileCommand=MemLimit,...,crash causes compilation to stop

Suggestion:

 * @summary Checks that -XX:CompileCommand=MemLimit,...,stop causes compilation to stop

-------------

Marked as reviewed by thartmann (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/16220#pullrequestreview-1705668002
PR Review Comment: https://git.openjdk.org/jdk/pull/16220#discussion_r1377194602
PR Review Comment: https://git.openjdk.org/jdk/pull/16220#discussion_r1377196509
PR Review Comment: https://git.openjdk.org/jdk/pull/16220#discussion_r1377196657