RFR: 8341587: [premain] Support Soft/Weak Reference in AOT cache [v2]

[Ioi Lam]

On Tue, 25 Feb 2025 03:53:55 GMT, Ioi Lam <iklam at openjdk.org> wrote:

>> I tested on top of the latest premain repo and all Leyden tests passed. @macarte I think this PR is ready to be integrated.
>
>> @iklam How soon can this be upstreamed? I think I might be working on MH recently and I hope we can simplify the cache and avoid conflicts down the road by a quick upstream to mainline JDK.
> 
> I think this change can be upstreamed to the mainline. @macarte, do you want to create a PR for the mainline? I can help run it through our testing pipeline, as we need to do more testing for mainline changes.

> > > @iklam How soon can this be upstreamed? I think I might be working on MH recently and I hope we can simplify the cache and avoid conflicts down the road by a quick upstream to mainline JDK.
> > 
> > 
> > I think this change can be upstreamed to the mainline. @macarte, do you want to create a PR for the mainline? I can help run it through our testing pipeline, as we need to do more testing for mainline changes.
> 
> I can create the PR for mainline, we'll need a JBS Item.

Yes, please create a new JBS issue, as the JDK-8341587 was targeted to the Leyden repo and is already marked as closed.

> One question though: this commit has no flags to disable adding WeakReferences to the cache, is this okay for tip or should we add a flag?

It would be too difficult to implement a switch to disable WeakReferences. To do that we would need to keep the side table and then dynamically choose between the two behaviors.

In the long run, our goal is to be able to store complex data structures used by the JDK library (and even applications). It's quite common to have WeakReferences in such data structures, and we don't want to require the Java code to be modified before such data structures can be stored.

BTW I think we still have a problem -- in the assembly phase, the ReferenceQueue may contain WeakReferences that are used by data structures that we don't intend to store in the AOT cache, but they may be discovered by heapShared.cpp and end up in the AOT cache. 

So @liach we may not be able to upstream this into the mainline yet.

With a simple test, it seems like we are archiving only WeakReferences used by MethodTypes, etc. I don't see anything that we don't want, so that's so far so good:


# Must use JDK from leyden/premain repo
$ java -cp HelloWorld.jar -XX:AOTMode=record -XX:AOTConfiguration=hw.aotconfig HelloWorld
$ java -Xlog:cds+map=trace,cds+map+oops=trace:file=cds.oops.txt:none:filesize=0 -cp HelloWorld.jar -XX:AOTMode=create -Xlog:cds -XX:AOTCache=hw.aot -XX:AOTConfiguration=hw.aotconfig
$ grep -n private.*referent cds.oops.txt 
452055: - private 'referent' 'Ljava/lang/Object;' @12 0x00000007ff8297f0 (0xfff052fe) java.lang.invoke.LambdaForm
524890: - private 'referent' 'Ljava/lang/Object;' @12 0x00000007ff866c10 (0xfff0cd82) java.lang.invoke.MethodType (Ljava/lang/String;)Z


The WeakReferenceKey look like this:


0x00000007ff868bf0: @@ Object (0xfff0d17e) jdk.internal.util.WeakReferenceKey
 - klass: 'jdk/internal/util/WeakReferenceKey' 0x0000000800293578
 - fields (4 words):
 - private 'referent' 'Ljava/lang/Object;' @12 0x00000007ff82db60 (0xfff05b6c) java.lang.invoke.MethodType ()Ljava/lang/Object;
 - volatile 'queue' 'Ljava/lang/ref/ReferenceQueue;' @16 0x00000007ff866988 (0xfff0cd31) java.lang.ref.ReferenceQueue
 - volatile 'next' 'Ljava/lang/ref/Reference;' @20 null
 - private transient 'discovered' 'Ljava/lang/ref/Reference;' @24 null
 - private final 'hashcode' 'I' @28  1335087215 (0x4f93d06f)


It's queue look like this (0 items!)


0x00000007ff866988: @@ Object (0xfff0cd31) java.lang.ref.ReferenceQueue
 - klass: 'java/lang/ref/ReferenceQueue' 0x000000080013d000
 - fields (4 words):
 - private volatile 'head' 'Ljava/lang/ref/Reference;' @12 null
 - private 'queueLength' 'J' @16  0 (0x0000000000000000)
 - private final 'lock' 'Ljava/lang/ref/ReferenceQueue$Lock;' @24 0x00000007ff8d8aa0 (0xfff1b154) java.lang.ref.ReferenceQueue$Lock


And all the `WeakReferenceKey::next` fields are null. I wonder why.


$ grep "volatile 'next'.*Reference" cds.oops.txt | wc
    114     684    6384
$ grep "volatile 'next'.*Reference" cds.oops.txt | grep -v null | wc
      0       0       0


So I think we need to understand this a little more before moving forward with upstreaming to mainline.

-------------

PR Comment: https://git.openjdk.org/leyden/pull/41#issuecomment-2684148819