Question in understanding ClassValue better

[Peter Levart]

Hi Jochen,

I'll try to answer your questions as profoundly as I can...


On 05/19/2016 04:27 PM, Jochen Theodorou wrote:
> Hi,
>
> at the beginning of this year I had an exchange with Peter Lavart 
> about JDK-8136353 (see 
> http://mail.openjdk.java.net/pipermail/mlvm-dev/2016-January/006563.html), 
> and that is probably based on wrong assumptions. But I must confess I 
> still have trouble understanding ClassValue semantics. And since the 
> ClassValue problem in Groovy came up again, I though I make another 
> try based on a list of assumptions and asking if they are wrong or right
>
> 1) ClassValue can be basically understood as a strong reference of a 
> class to a class value

a ClassValue instance can be thought of as a component of a compound 
key. Together with a Class, they form a tuple (aClass, aClassValue) that 
can be associated with an "associated value", AV. And yes, the AVs 
associated with tuple containing a particular Class are strongly 
reachable from that Class.

> 2) a ClassValue associated with a system class (for example 
> Integer.TYPE) is never garbage collected

An AV associated with a tuple (Integer.TYPE, aClassValue) -> AV can be 
garbage collected. But only if aClassValue can be garbage collected 1st. 
This is the most tricky part to get right in order to prevent leaks. If 
in above example, aClassValue is reachable from the AV, then we have a 
leak. The reachability of a ClassValue instance from the associated 
value AV is not always obvious. One has to take into account the 
following non-obvious references:

1 - each object instance has an implicit reference to its implementing class
2 - each class has a reference to its defining ClassLoader
3 - each ClassLoader has a reference to all classes defined by it 
(except VM annonymous classes)
4 - each ClassLoader has a reference to all its predecessors (that it 
delegates to)

Since a ClassValue instance is typically assigned to a static final 
field, such instance is reachable from the class that declares the field.

I think you can get the picture from that.

> 3) a ClassValue from a different loader than the system loader, 
> associated with a system class, will prevent that loader to unload

You mean an instance of a ClassValue subclass loaded by a non-system 
class loader? I don't think this matters much. Any object instance with 
a runtime class that is not a system class, while being reachable, holds 
the non-system ClassLoader non-reclaimable (this follows from the 1st 
and 2nd rules of non-obvious references above)

a ClassValue instance is not associated with a Class. (aClass, 
aClassValue) tuple is associated with an associated value AV. Such 
association is implemented in a way where aClassValue is not strongly 
reachable from aClass BECAUSE OF THE ASSOCIATION ITSELF, if that is what 
you wanted to know.

> 4) a ClassValue referencing to the class it is associated with, does 
> not prevent the collection of that class

An association (aClass, aClassValue) -> AV is implemented in a way where 
aClass is not strongly reachable from aClassValue BECAUSE OF THE 
ASSOCIATION ITSELF. It can still be reachable because of other 
non-obvious references mentioned above.

>
> Point 2 and 3 are kind of problematic for me and I wish them wrong, 
> but they would follow from 1. The exchange with Peter makes me think 
> assumption 4 is wrong... just I don't understand why.
>
> If those assumptions are right, then I actually wonder in what cases I 
> should use ClassValue without causing memory leaks. What I wanted to 
> use it for is to associate a meta class with every class I need a meta 
> class for. This includes system classes. If 3 is right, then doing so 
> would prevent the Groovy runtime from being unloaded. Even if the meta 
> classes are able to unload, the implementation of the ClassValue would 
> still be there. And since that comes from the same loader, that loaded 
> the runtime, that loader will stay. Now loading and (trying to) unload 
> the Groovy runtime countless times would end up in a OOME at some 
> point (permgen problem in older JDKs). And even if I would do 
> something else for class from the standard loaders, I would still get 
> into trouble on for example Tomcat. Not to forget that having two 
> parallel structures for this raises the question as of why to use 
> ClassValue at all.
>
> I think what it boils down to in the end is: When (under what 
> conditions) for what to use ClassValue at all.
>
> bye Jochen


Ok, let's set up the stage. If I understand you correctly, then:

Groovy runtime is loaded by whatever class loader is loading the 
application (see the comment in MetaClass constructor if this is not 
true). This is either the ClassLoader.getSystemClassLoader() (the APP 
class loader) if started from command line or for example Web App class 
loader in a Web container.

MetaClass(es) are objects implemented by Groovy runtime class(es). Let's 
call them simply MetaClass.


Here's how I would do that:


public class MetaClass {

     // this list keeps MetaClass instances strongly reachable from the 
MetaClass
     // class(loader) since they are only weakly reachable from their 
associated
     // Class(es)
     private static final ArrayList<MetaClass> META_CLASS_LIST = new 
ArrayList<>();

     // this WeakReference is constructed so that it keeps a strong 
reference
     // to a referent until releaseStrong() is called
     private static final class WeakEntry extends WeakReference<MetaClass> {
         private final AtomicReference<MetaClass> strong;

         WeakEntry(MetaClass mc) {
             super(mc);
             strong = new AtomicReference<>(mc);
         }

         boolean releaseStrong() {
             MetaClass mc = strong.get();
             return mc != null && strong.compareAndSet(mc, null);
         }
     }

     private static final ClassValue<WeakEntry> WEAK_ENTRY_CV =
         new ClassValue<WeakEntry>() {
             @Override
             protected WeakEntry computeValue(Class<?> type) {
                 return new WeakEntry(new MetaClass(type));
             }
         };

     // the public API
     public MetaClass getInstanceFor(Class<?> type) {
         WeakEntry entry = WEAK_ENTRY_CV.get(type);
         MetaClass mc = entry.get();
         if (entry.releaseStrong()) {
             synchronized (META_CLASS_LIST) {
                 META_CLASS_LIST.add(mc);
             }
         }
         return mc;
     }

     MetaClass(Class<?> type) {
         // derive it from 'type', but don't reference it
         // strongly if Groovy runtime is loaded by a parent
         // class loader of the application class loader
         // that loads application classes  you want
         // MetaClass(es) to be derived from.
     }
}



This will keep MetaClass instances isolated from the associated classes 
but still keep them reachable as long as the 
MetaClass.class.getClassLoader() is alive.


Is this going to help?


Regards, Peter


> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/mlvm-dev/attachments/20160519/cec0b1b8/attachment-0001.html>