RFR: JDK-8317987: C2 recompilations cause high memory footprint

[Thomas Stuefe]

On Thu, 12 Oct 2023 16:20:56 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

>> While playing with the new compiler memory statistic (https://github.com/openjdk/jdk/pull/16076), I saw that +StressRecompile causes a twofold increase in memory used during compilations. 
>> 
>> #### Baseline: 
>> 
>> highest peak during compilation is ~60 MB, caused by compilation of `java/lang/invoke/LambdaForm$Kind::<clinit>` with ~18k nodes.
>> 
>> `./images/jdk/bin/java -XX:CompileCommand='collectmemstat,*.*' -XX:-StressRecompilation -Xcomp -Xbatch -cp $REPROS_JAR de.stuefe.repros.Simple`
>> 
>> 
>> total     NA        RA        #nodes  time    type  #rc thread              method
>> (3/1425)
>> 59063K    13406K    40025K    18345   6,174   c2    2   0x00007fdc2c28e7b0  java/lang/invoke/LambdaForm$Kind::<clinit>(()V) 
>> 
>> 
>> #### +StressRecompile: 
>> 
>> highest peak during compilation is ~112 MB, same method, sameish node count.
>> 
>> `./images/jdk/bin/java -XX:CompileCommand='collectmemstat,*.*' -XX:+StressRecompilation -Xcomp -Xbatch -cp $REPROS_JAR de.stuefe.repros.Simple`
>> 
>> 
>> total     NA        RA        #nodes  time    type  #rc thread              method
>> (16/1434)
>> 112M      13469K    94987K    18791   16,451  c2    2   0x00007f790428e7b0  java/lang/invoke/LambdaForm$Kind::<clinit>(()V) 
>> 
>> 
>> 60M -> 112M. In both cases, a large part of the memory was allocated in resource areas. Its relative size compared to the total spike size increased. We accrue a lot of memory in ResourceArea over recompilations.
>> 
>> The solution is to wrap the `Compile` instantiation into a resource mark. Now we are at 61 MB, which is almost baseline:
>> 
>> 
>> total     NA        RA        #nodes  time    type  #rc thread              method
>> (3/1430)
>> 61203K    13469K    40791K    18791   17,013  c2    2   0x00007fdf7428e7b0  java/lang/invoke/LambdaForm$Kind::<clinit>(()V)
>
> Nice. We added a lot of C2 recompilation cases for the past few years so it become noticeable.
> The main `ResourceMark` for all JIT compilers is located in `CompileBroker::invoke_compiler_on_method()`: [compileBroker.cpp#L2141](https://github.com/openjdk/jdk/blob/master/src/hotspot/share/compiler/compileBroker.cpp#L2141)

Thanks @vnkozlov and @adinn!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16161#issuecomment-1760986653