[9] RFR (M): 8079205: CallSite dependency tracking is broken after sun.misc.Cleaner became automatically cleared

[Vladimir Ivanov]

http://cr.openjdk.java.net/~vlivanov/8079205/webrev.01
https://bugs.openjdk.java.net/browse/JDK-8079205

Recent change in sun.misc.Cleaner behavior broke CallSite context cleanup.

CallSite references context class through a Cleaner to avoid its 
unnecessary retention.

The problem is the following: to do a cleanup (invalidate all affected 
nmethods) VM needs a pointer to a context class. Until Cleaner is 
cleared (and it was a manual action, since Cleaner extends 
PhantomReference isn't automatically cleared according to the docs), VM 
can extract it from CallSite.context.referent field.

I experimented with moving cleanup logic into VM [1], but Peter Levart 
came up with a clever idea and implemented FinalReference-based 
cleaner-like Finalizator. Classes don't have finalizers, but Finalizator 
allows to attach a finalization action to them. And it is guaranteed 
that the referent is alive when finalization happens.

Also, Peter spotted another problem with Cleaner-based implementation.
Cleaner cleanup action is strongly referenced, since it is registered in 
Cleaner class. CallSite context cleanup action keeps a reference to 
CallSite class (it is passed to MHN.invalidateDependentNMethods). Users 
are free to extend CallSite and many do so. If a context class and a 
call site class are loaded by a custom class loader, such loader will 
never be unloaded, causing a memory leak.

Finalizator doesn't suffer from that, since the action is referenced 
only from Finalizator instance. The downside is that cleanup action can 
be missed if Finalizator becomes unreachable. It's not a problem for 
CallSite context, because a context is always referenced from some 
CallSite and if a CallSite becomes unreachable, there's no need to 
perform a cleanup.

Testing: jdk/test/java/lang/invoke, hotspot/test/compiler/jsr292

Contributed-by: plevart, vlivanov

Best regards,
Vladimir Ivanov

PS: frankly speaking, I would move Finalizator from java.lang.ref to 
java.lang.invoke and call it Context, if there were a way to extend 
package-private FinalReference from another package :-)

[1] http://cr.openjdk.java.net/~vlivanov/8079205/webrev.00