RFR: 8302154: Hidden classes created by LambdaMetaFactory can't be unloaded [v2]

[Thomas Stuefe]

On Fri, 10 Feb 2023 19:00:53 GMT, Volker Simonis <simonis at openjdk.org> wrote:

>> Even in JDK 11, a lambda proxy classes that's referenced by the cpCache (i.e., from a resolved invokedynamic instruction associated with a lamda expression) is always kept alive. See test below.
>> 
>> So if I understand correctly, this patch will not affect lamda expressions in Java source code. It affects only direct calls to `LambdaMetafactory.metafactory()`. Is this correct?
>> 
>> 
>> public class LambdaGC {
>>     public static void main(String[] args) throws Throwable  {
>>       System.out.println("Entering LambdaGC");
>>       doit(() -> {
>>           Thread.dumpStack();
>>       });
>>       for (int i = 0; i < 10; i++) {
>>             System.gc();
>>       }
>>       System.out.println("Finish LambdaGC");
>>     }
>>     static void doit(Runnable r) {
>>         r.run();
>>     }
>> }
>> 
>> $ java11 -cp . -XX:+UnlockDiagnosticVMOptions -XX:+ShowHiddenFrames -Xlog:class+load -Xlog:class+unload LambdaGC | grep LambdaGC
>> [0.022s][info][class,load] LambdaGC source: file:/jdk3/tmp/
>> Entering LambdaGC
>> [0.024s][info][class,load] LambdaGC$$Lambda$1/0x0000000840060840 source: LambdaGC
>> java.lang.Exception: Stack trace
>> 	at java.base/java.lang.Thread.dumpStack(Thread.java:1387)
>> 	at LambdaGC.lambda$main$0(LambdaGC.java:5)
>> 	at LambdaGC$$Lambda$1/0x0000000840060840.run(<Unknown>:1000000)
>> 	at LambdaGC.doit(LambdaGC.java:13)
>> 	at LambdaGC.main(LambdaGC.java:4)
>> Finish LambdaGC
>
> @iklam, I think your understanding is correct. While the bootstrap methods for Java Lambdas do call `LambdaMetafactory.metafactory()`, they store the resulting call site (for Lambdas a `BoundMethodHandle`) in the appendix slot of the constant pool cache entry of the invokedynamic bytecode. The `BoundMethodHandle` contains a reference to an instance of the generated lambda form (i.e. in your example `LambdaGC$$Lambda$1`). This is enough in order to keep `LambdaGC$$Lambda$1` alive and prevent its unloading.
> 
> Until now, `LambdaGC$$Lambda$1` was also strongly linked to its defining class loader, but I don't think that's necessary to keep it alive (because it is referenced from the call site which is referenced from the constant pool cache).
> 
> Running your example with `-Xlog:indy+methodhandles=debug` confirms this:
> 
> set_method_handle bc=186 appendix=0x000000062b821838 method=0x00000008000c7698 (local signature)
> {method}
>  - this oop:          0x00000008000c7698
>  - method holder:     'java/lang/invoke/Invokers$Holder'
>  ...
> appendix: java.lang.invoke.BoundMethodHandle$Species_L 
>  {0x000000062b821838} - klass: 'java/lang/invoke/BoundMethodHandle$Species_L'
>   - ---- fields (total size 5 words):
>   ...
>   - final 'argL0' 'Ljava/lang/Object;' @32  a 'LambdaGC$$Lambda$1+0x0000000801000400'{0x000000062b81e080} (0xc5703c10)

@simonis This may be a stupid question, but how exactly does your patch help with Metaspace consumption? I mean, how is the deallocation path with the patch now?

We still have the CLD loaded by the BootClassLoader, right? So it is still living in the metaspace arena of the bootclassloader, even if the Lambda class itself is collected earlier. How is this memory freed or reused?

If there were a way to prematurely collect its metadata via Metaspace::deallocate, then it could be reused at least by the bootloader itself. But I don't see it.

-----

>True but it was decided at that time for JEP 371 to make it strong so that the lambda proxy classes will share the same ClassLoaderData metaspace as other classes defined by the same loader. This reduces the memory footprint consumption.

Expanding on what @mlchung wrote, this has been a nice effect of JEP 371. Metaspace intra-chunk fragmentation went down since all these pesky little CLDs went away. It also saved some C-heap as a side effect for the CLDs itself.

Example, committed metaspace for 100k lambdas:
JDK11: 271MB
JDK15: (+JEP371): 221MB
JDK17: (+JEP371 +JEP387): 189MB

-------------

PR: https://git.openjdk.org/jdk/pull/12493