Question in understanding ClassValue better
Peter Levart
peter.levart at gmail.com
Tue May 24 11:41:26 UTC 2016
On 05/24/2016 10:26 AM, Jochen Theodorou wrote:
> Peter, I fully understand if you cannot reply to this mail easily,
> just wanted to ping to ensure this is not forgotten ;)
Sorry Jochen, I forgot about it... Thanks for reminding me!
>
> On 20.05.2016 01:33, Jochen Theodorou wrote:
>> On 19.05.2016 21:32, Peter Levart wrote:
>> [...]
>>> a ClassValue instance can be thought of as a component of a compound
>>> key. Together with a Class, they form a tuple (aClass, aClassValue)
>>> that
>>> can be associated with an "associated value", AV. And yes, the AVs
>>> associated with tuple containing a particular Class are strongly
>>> reachable from that Class.
>>
>> I see, I was mixing ClassValue and AV, I was more talking about AV, than
>> ClassValue itself, though ClassValue surely plays an important role in
>> here. Anyway, I am going for that AV is the value computed by
>> aClassValue.
>>
>> You said that the AV is strongly reachable from aClass. Can I further
>> assume, that aClassValue is not strongly reachable from aClass? And that
>> aClassValue can be collected independent of aClass? Can I further
>> assume, that aClassValue can be collected even if AVs for it continue to
>> exist?
ClassValue implementation is such that aClassValue is not strongly
reachable from aClass as a consequence of (aClassValue, aClass) -> AV
association. It can be reachable because of other unrelated references.
Therefore, the answer is YES for all above questions.
>>
>> [...]
>>> An AV associated with a tuple (Integer.TYPE, aClassValue) -> AV can be
>>> garbage collected. But only if aClassValue can be garbage collected
>>> 1st.
>>
>> hmm... so in (aClass, aClassValue)->AV if aClassValue can be collected,
>> AV can, but not the other way around...
It is a two stage process with a GC cycle between the stages. In 1st
stage aClassValue is unreferenced, then GC kicks-in, collects
aClassValue and enqueues a WeakReference that was pointing to
aClassValue, then the enqueued WeakReference triggers expunging of
associated AV on next access to some association (???, aClass) for the
same aClass. That's how it is implemented in current ClassValue. If
there's no "next access", then there's no expunging and the AV remains
reachable.
>> what about aClass? if nothing
>> but AV is referencing aClass, can AV be garbage collected, even if
>> aClassValue cannot? Can I extend your statement to AV can be collected
>> only if either aClass or aClassValue can be garbage collected first?
aClass is different from aClassValue in that aClass and all associated
AVs can be collected at the same time. AVs are reachable from associated
aClass, but if aClass is not strongly reachable from anywhere, they can
all be collected together. aClassValue reachability does not play a role
here.
>>
>> Let us assume this is the case for now.
>>
>>> This is the most tricky part to get right in order to prevent leaks. If
>>> in above example, aClassValue is reachable from the AV, then we have a
>>> leak. The reachability of a ClassValue instance from the associated
>>> value AV is not always obvious. One has to take into account the
>>> following non-obvious references:
>>>
>>> 1 - each object instance has an implicit reference to its implementing
>>> class
>>> 2 - each class has a reference to its defining ClassLoader
>>> 3 - each ClassLoader has a reference to all classes defined by it
>>> (except VM annonymous classes)
>>> 4 - each ClassLoader has a reference to all its predecessors (that it
>>> delegates to)
>>>
>>> Since a ClassValue instance is typically assigned to a static final
>>> field, such instance is reachable from the class that declares the
>>> field.
>>>
>>> I think you can get the picture from that.
>>
>> yeah... that is actually problematic. Because if I keep no hard
>> reference the ClassValue can be collected, even if the AVs still
>> exist...
you need aClassValue to lookup the AVs. Without aClassValue, they are
not retrievable. So you better keep a reference to aClassValue as long
as you need it to lookup the AVs...
>> meaning they would become unreachable. And if I keep one I have
>> a memory leak... well more about in the program you have shown me
>> later on.
>>
>> [...]
>>> Ok, let's set up the stage. If I understand you correctly, then:
>>>
>>> Groovy runtime is loaded by whatever class loader is loading the
>>> application (see the comment in MetaClass constructor if this is not
>>> true). This is either the ClassLoader.getSystemClassLoader() (the APP
>>> class loader) if started from command line or for example Web App class
>>> loader in a Web container.
>>
>> Well, actually... if you start a script on the command line, the loader
>> is a child to the app class loader, when used as library it could be the
>> app loader (for example if the groovy program is precompiled) and in a
>> tomcat like scenario it could be either the class loader for the web
>> app, or the loader for all web apps. But let's go with the cases you
>> mentioned first ;)
>>
>>> MetaClass(es) are objects implemented by Groovy runtime class(es).
>>> Let's
>>> call them simply MetaClass.
>>
>> good
>>
>>> Here's how I would do that:
>>>
>>>
>>> public class MetaClass {
>>>
>>> // this list keeps MetaClass instances strongly reachable from
>>> the MetaClass
>>> // class(loader) since they are only weakly reachable from their
>>> associated
>>> // Class(es)
>>> private static final ArrayList<MetaClass> META_CLASS_LIST = new
>>> ArrayList<>();
>>>
>>> // this WeakReference is constructed so that it keeps a strong
>>> reference
>>> // to a referent until releaseStrong() is called
>>> private static final class WeakEntry extends
>>> WeakReference<MetaClass> {
>>> private final AtomicReference<MetaClass> strong;
>>>
>>> WeakEntry(MetaClass mc) {
>>> super(mc);
>>> strong = new AtomicReference<>(mc);
>>> }
>>>
>>> boolean releaseStrong() {
>>> MetaClass mc = strong.get();
>>> return mc != null && strong.compareAndSet(mc, null);
>>> }
>>> }
>>>
>>> private static final ClassValue<WeakEntry> WEAK_ENTRY_CV =
>>> new ClassValue<WeakEntry>() {
>>> @Override
>>> protected WeakEntry computeValue(Class<?> type) {
>>> return new WeakEntry(new MetaClass(type));
>>> }
>>> };
>>>
>>> // the public API
>>> public MetaClass getInstanceFor(Class<?> type) {
>>> WeakEntry entry = WEAK_ENTRY_CV.get(type);
>>> MetaClass mc = entry.get();
>>> if (entry.releaseStrong()) {
>>> synchronized (META_CLASS_LIST) {
>>> META_CLASS_LIST.add(mc);
>>> }
>>> }
>>> return mc;
>>> }
>>>
>>> MetaClass(Class<?> type) {
>>> // derive it from 'type', but don't reference it
>>> // strongly if Groovy runtime is loaded by a parent
>>> // class loader of the application class loader
>>> // that loads application classes you want
>>> // MetaClass(es) to be derived from.
>>> }
>>> }
>>>
>>>
>>> This will keep MetaClass instances isolated from the associated classes
>>> but still keep them reachable as long as the
>>> MetaClass.class.getClassLoader() is alive.
>>>
>>> Is this going to help?
>>
>> partially... MetaClass will realistically have a strong reference to the
>> class the meta class is for and a lot of other classes. I will need to
>> reference methods and fields by reflection and at some point keep them
>> referenced. Through the declaring class I should then again have a
>> strong reference to the class.
>>
>> 1_000_000.times { Eval.me("42") }
>>
>> this will create 1 million classes, in 1 million classloaders, which are
>> a child to the class loader for the Groovy runtime. Each of them will
>> get a meta class... ignoring the list, if the strong reference to the
>> script class in MetaClass is here enough to prevent collection, then
>> this is a problem. The list of course is also a problem, since keeping
>> them alive as long as MetaClass.class.getClassLoader is alive, is
>> actually not the semantics I need.
>>
>> The semantics I need is something like lifetime(AV) is roughly
>> min(lifetime(aClass),lifetime(aClassValue)), with AV referencing aClass
>> not influencing it being collectable, as long as this association is the
>> only reference to AV. If it is the max of them I am in trouble.
>>
>> And seeing how aClass kind of has a strong reference to AV and that if
>> aClassValue and AV are from the same loader and if AV has a strong
>> reference to aClass as well, it will cause aClass not being collectable
>> for as long as the loader for aClassValue and AV exists... because
>> aClassValue is a static value in one of the classes, and AV is not
>> collected before aClassValue is collected and aClass is not not
>> collected because of the reference in AV.
>>
>> The "old" solution in Groovy is to keep a "global" map with weak class
>> keys and soft references to MetaClass. This allows in theory the class
>> being collectable, but only if the MetaClass has been collected before..
>> and some VMs really do not like soft referenced classes much (forced us
>> to use the mark and sweep gc for example). Also soft references used not
>> always to be clean up when permgen space is low. But if they are not
>> cleaned up, then MetaClass prevents the garbage collection of the class,
>> causing permgen problems. Not sure how that is with meta space, but from
>> what I have seen so far, this can still happen. And of course using
>> SoftReference means we garbage collect relatively late, and cause a big
>> garbage collection. I cannot use weak references, because tend to be
>> collected to fast and creating the meta class is not exactly a cheap
>> operation.
>>
>> bye Jochen
Hm,
It seems that my example will not help much. It also seems that the only
problem with plain:
static final ClassValue<MetaClass> META_CLASS_CV = new
ClassValue<MetaClass>() {
@Override
protected MetaClass computeValue(Class<?> type) {
return new MetaClass(type);
}
};
...is the fact that MetaClass is a class loaded by non-bootstrap class
loader and that in case this is a Web app class loader, it prevents
undeployment. Can you confirm that a MetaClass instance only references
the 'type' Class<?> it is derived from (it's Methods, Fields, etc.) and
never references objects from any child class loaders of the type's
class loader?
If that is the case, then you could replace MetaClass with a generic
data structure, composed of instances of bootstrap classes (HashMap,
ArrayList, Object[], ...). That way, Groovy runtime class loader will
not be "captured" by a reference from an aClass loaded by bootstrap
class loader.
Is MetaClass a complicated data structure?
Regards, Peter
>>
>> _______________________________________________
>> mlvm-dev mailing list
>> mlvm-dev at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>
More information about the mlvm-dev
mailing list