RFR: 8302154: Hidden classes created by LambdaMetaFactory can't be unloaded [v2]

[Remi Forax]

----- Original Message -----
> From: "Volker Simonis" <simonis at openjdk.org>
> To: "core-libs-dev" <core-libs-dev at openjdk.org>, "hotspot-dev" <hotspot-dev at openjdk.org>
> Sent: Friday, February 10, 2023 8:03:47 PM
> Subject: Re: RFR: 8302154: Hidden classes created by LambdaMetaFactory can't be unloaded [v2]

> On Fri, 10 Feb 2023 17:29:37 GMT, Ioi Lam <iklam at openjdk.org> wrote:
> 
>>> Volker Simonis has updated the pull request incrementally with two additional
>>> commits since the last revision:
>>> 
>>>  - Remove assertions which insist on Lambda proxy classes being strongly linked
>>>  to their class loader
>>>  - Removed unused import of STRONG und updated copyright year
>>
>> Even in JDK 11, a lambda proxy classes that's referenced by the cpCache (i.e.,
>> from a resolved invokedynamic instruction associated with a lamda expression)
>> is always kept alive. See test below.
>> 
>> So if I understand correctly, this patch will not affect lamda expressions in
>> Java source code. It affects only direct calls to
>> `LambdaMetafactory.metafactory()`. Is this correct?
>> 
>> 
>> public class LambdaGC {
>>     public static void main(String[] args) throws Throwable  {
>>       System.out.println("Entering LambdaGC");
>>       doit(() -> {
>>           Thread.dumpStack();
>>       });
>>       for (int i = 0; i < 10; i++) {
>>             System.gc();
>>       }
>>       System.out.println("Finish LambdaGC");
>>     }
>>     static void doit(Runnable r) {
>>         r.run();
>>     }
>> }
>> 
>> $ java11 -cp . -XX:+UnlockDiagnosticVMOptions -XX:+ShowHiddenFrames
>> -Xlog:class+load -Xlog:class+unload LambdaGC | grep LambdaGC
>> [0.022s][info][class,load] LambdaGC source: file:/jdk3/tmp/
>> Entering LambdaGC
>> [0.024s][info][class,load] LambdaGC$$Lambda$1/0x0000000840060840 source:
>> LambdaGC
>> java.lang.Exception: Stack trace
>> 	at java.base/java.lang.Thread.dumpStack(Thread.java:1387)
>> 	at LambdaGC.lambda$main$0(LambdaGC.java:5)
>> 	at LambdaGC$$Lambda$1/0x0000000840060840.run(<Unknown>:1000000)
>> 	at LambdaGC.doit(LambdaGC.java:13)
>> 	at LambdaGC.main(LambdaGC.java:4)
>> Finish LambdaGC
> 
> @iklam, I think your understanding is correct. While the bootstrap methods for
> Java Lambdas do call `LambdaMetafactory.metafactory()`, they store the
> resulting call site (for Lambdas a `BoundMethodHandle`) in the appendix slot of
> the constant pool cache entry of the invokedynamic bytecode. The
> `BoundMethodHandle` contains a reference to an instance of the generated lambda
> form (i.e. in your example `LambdaGC$$Lambda$1`). This is enough in order to
> keep `LambdaGC$$Lambda$1` alive and prevent its unloading.
> 
> Until now, `LambdaGC$$Lambda$1` was also strongly linked to its defining class
> loader, but I don't think that's necessary to keep it alive (because it is
> referenced from the call site which is referenced from the constant pool
> cache).
> 
> Running your example with `-Xlog:indy+methodhandles=debug` confirms this:
> 
> set_method_handle bc=186 appendix=0x000000062b821838 method=0x00000008000c7698
> (local signature)
> {method}
> - this oop:          0x00000008000c7698
> - method holder:     'java/lang/invoke/Invokers$Holder'
> ...
> appendix: java.lang.invoke.BoundMethodHandle$Species_L
> {0x000000062b821838} - klass: 'java/lang/invoke/BoundMethodHandle$Species_L'
>  - ---- fields (total size 5 words):
>  ...
>  - final 'argL0' 'Ljava/lang/Object;' @32  a
>  'LambdaGC$$Lambda$1+0x0000000801000400'{0x000000062b81e080} (0xc5703c10)

Hi Volker,
the main issue if the link is not STRONG is that the VM creates one classloader data per lambda as Mandy said, if you have a library full of lambdas, this will consume a lot of C memory which is always hard to to diagnose.

I remember people from RedHat reverting a library code that was 'lambdaified' because of that.

So it's a kind of pick your poison situation and i believe the current tradeoff, you need a classloader if you want to unload something (classes or lambdas) vs you may consume too much off heap memory is the good one.

regards,
Rémi 

> 
> -------------
> 
> PR: https://git.openjdk.org/jdk/pull/12493

Rémi