Question in understanding ClassValue better
Jochen Theodorou
blackdrag at gmx.org
Tue May 24 08:26:28 UTC 2016
Peter, I fully understand if you cannot reply to this mail easily, just
wanted to ping to ensure this is not forgotten ;)
On 20.05.2016 01:33, Jochen Theodorou wrote:
> On 19.05.2016 21:32, Peter Levart wrote:
> [...]
>> a ClassValue instance can be thought of as a component of a compound
>> key. Together with a Class, they form a tuple (aClass, aClassValue) that
>> can be associated with an "associated value", AV. And yes, the AVs
>> associated with tuple containing a particular Class are strongly
>> reachable from that Class.
>
> I see, I was mixing ClassValue and AV, I was more talking about AV, than
> ClassValue itself, though ClassValue surely plays an important role in
> here. Anyway, I am going for that AV is the value computed by aClassValue.
>
> You said that the AV is strongly reachable from aClass. Can I further
> assume, that aClassValue is not strongly reachable from aClass? And that
> aClassValue can be collected independent of aClass? Can I further
> assume, that aClassValue can be collected even if AVs for it continue to
> exist?
>
> [...]
>> An AV associated with a tuple (Integer.TYPE, aClassValue) -> AV can be
>> garbage collected. But only if aClassValue can be garbage collected 1st.
>
> hmm... so in (aClass, aClassValue)->AV if aClassValue can be collected,
> AV can, but not the other way around... what about aClass? if nothing
> but AV is referencing aClass, can AV be garbage collected, even if
> aClassValue cannot? Can I extend your statement to AV can be collected
> only if either aClass or aClassValue can be garbage collected first?
>
> Let us assume this is the case for now.
>
>> This is the most tricky part to get right in order to prevent leaks. If
>> in above example, aClassValue is reachable from the AV, then we have a
>> leak. The reachability of a ClassValue instance from the associated
>> value AV is not always obvious. One has to take into account the
>> following non-obvious references:
>>
>> 1 - each object instance has an implicit reference to its implementing
>> class
>> 2 - each class has a reference to its defining ClassLoader
>> 3 - each ClassLoader has a reference to all classes defined by it
>> (except VM annonymous classes)
>> 4 - each ClassLoader has a reference to all its predecessors (that it
>> delegates to)
>>
>> Since a ClassValue instance is typically assigned to a static final
>> field, such instance is reachable from the class that declares the field.
>>
>> I think you can get the picture from that.
>
> yeah... that is actually problematic. Because if I keep no hard
> reference the ClassValue can be collected, even if the AVs still
> exist... meaning they would become unreachable. And if I keep one I have
> a memory leak... well more about in the program you have shown me later on.
>
> [...]
>> Ok, let's set up the stage. If I understand you correctly, then:
>>
>> Groovy runtime is loaded by whatever class loader is loading the
>> application (see the comment in MetaClass constructor if this is not
>> true). This is either the ClassLoader.getSystemClassLoader() (the APP
>> class loader) if started from command line or for example Web App class
>> loader in a Web container.
>
> Well, actually... if you start a script on the command line, the loader
> is a child to the app class loader, when used as library it could be the
> app loader (for example if the groovy program is precompiled) and in a
> tomcat like scenario it could be either the class loader for the web
> app, or the loader for all web apps. But let's go with the cases you
> mentioned first ;)
>
>> MetaClass(es) are objects implemented by Groovy runtime class(es). Let's
>> call them simply MetaClass.
>
> good
>
>> Here's how I would do that:
>>
>>
>> public class MetaClass {
>>
>> // this list keeps MetaClass instances strongly reachable from
>> the MetaClass
>> // class(loader) since they are only weakly reachable from their
>> associated
>> // Class(es)
>> private static final ArrayList<MetaClass> META_CLASS_LIST = new
>> ArrayList<>();
>>
>> // this WeakReference is constructed so that it keeps a strong
>> reference
>> // to a referent until releaseStrong() is called
>> private static final class WeakEntry extends
>> WeakReference<MetaClass> {
>> private final AtomicReference<MetaClass> strong;
>>
>> WeakEntry(MetaClass mc) {
>> super(mc);
>> strong = new AtomicReference<>(mc);
>> }
>>
>> boolean releaseStrong() {
>> MetaClass mc = strong.get();
>> return mc != null && strong.compareAndSet(mc, null);
>> }
>> }
>>
>> private static final ClassValue<WeakEntry> WEAK_ENTRY_CV =
>> new ClassValue<WeakEntry>() {
>> @Override
>> protected WeakEntry computeValue(Class<?> type) {
>> return new WeakEntry(new MetaClass(type));
>> }
>> };
>>
>> // the public API
>> public MetaClass getInstanceFor(Class<?> type) {
>> WeakEntry entry = WEAK_ENTRY_CV.get(type);
>> MetaClass mc = entry.get();
>> if (entry.releaseStrong()) {
>> synchronized (META_CLASS_LIST) {
>> META_CLASS_LIST.add(mc);
>> }
>> }
>> return mc;
>> }
>>
>> MetaClass(Class<?> type) {
>> // derive it from 'type', but don't reference it
>> // strongly if Groovy runtime is loaded by a parent
>> // class loader of the application class loader
>> // that loads application classes you want
>> // MetaClass(es) to be derived from.
>> }
>> }
>>
>>
>> This will keep MetaClass instances isolated from the associated classes
>> but still keep them reachable as long as the
>> MetaClass.class.getClassLoader() is alive.
>>
>> Is this going to help?
>
> partially... MetaClass will realistically have a strong reference to the
> class the meta class is for and a lot of other classes. I will need to
> reference methods and fields by reflection and at some point keep them
> referenced. Through the declaring class I should then again have a
> strong reference to the class.
>
> 1_000_000.times { Eval.me("42") }
>
> this will create 1 million classes, in 1 million classloaders, which are
> a child to the class loader for the Groovy runtime. Each of them will
> get a meta class... ignoring the list, if the strong reference to the
> script class in MetaClass is here enough to prevent collection, then
> this is a problem. The list of course is also a problem, since keeping
> them alive as long as MetaClass.class.getClassLoader is alive, is
> actually not the semantics I need.
>
> The semantics I need is something like lifetime(AV) is roughly
> min(lifetime(aClass),lifetime(aClassValue)), with AV referencing aClass
> not influencing it being collectable, as long as this association is the
> only reference to AV. If it is the max of them I am in trouble.
>
> And seeing how aClass kind of has a strong reference to AV and that if
> aClassValue and AV are from the same loader and if AV has a strong
> reference to aClass as well, it will cause aClass not being collectable
> for as long as the loader for aClassValue and AV exists... because
> aClassValue is a static value in one of the classes, and AV is not
> collected before aClassValue is collected and aClass is not not
> collected because of the reference in AV.
>
> The "old" solution in Groovy is to keep a "global" map with weak class
> keys and soft references to MetaClass. This allows in theory the class
> being collectable, but only if the MetaClass has been collected before..
> and some VMs really do not like soft referenced classes much (forced us
> to use the mark and sweep gc for example). Also soft references used not
> always to be clean up when permgen space is low. But if they are not
> cleaned up, then MetaClass prevents the garbage collection of the class,
> causing permgen problems. Not sure how that is with meta space, but from
> what I have seen so far, this can still happen. And of course using
> SoftReference means we garbage collect relatively late, and cause a big
> garbage collection. I cannot use weak references, because tend to be
> collected to fast and creating the meta class is not exactly a cheap
> operation.
>
> bye Jochen
>
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
More information about the mlvm-dev
mailing list