Object arrays with no referrers.

[Remi Forax]

On 06/02/2014 12:30 PM, MacGregor, Duncan (GE Energy Management) wrote:
> Okay, I¹ve done some more digging, and although I still can¹t track down
> the source of individual referrer-less Object arrays (I¹m not sure the
> necessary information is even present in an hprof dump) I can say that
> they are mostly due to anonymous classes (based on the contents of some I
> have examined), and I do now understand what¹s going on with the large
> number of DMHs, and some other areas of memory usage.
>
> 1. Method handles and class constant pools
>
> Since we¹re operating in an OSGi environment
> java.lang.invoke.MethodHandles.Lookup.canBeCached() tends to return false,
> a lot. This is probably not a bad thing in and of itself as the
> LOOKASIDE_ASIDE could otherwise become full of DMHs which end up
> preventing large structures from being collected, but it does cause
> problems for us as almost all the DMH constants in our classes are
> pointing to the same small set of bootstrap methods. I¹ve also found it is
> horribly easy to generate a large set of DMHs for things like the fallback
> and cache invalidation methods of call sites, but this appears to be
> easily fixable by tweaking the order of binding and folding operations.

Hi Duncan,
for similar reason, i try to only bind one object by fallback and store 
the multiple values you need
as fields of that object. This object may or may not be a CallSite BTW.

> So, would it be possible to change the caching of DMHs to be based on
> ClassValues and allowing significantly more DMHs to be cached?
>
> 2. LambdaForm$Name[] memory usage
>
> Running outside of an OSGi environment our biggest uses of memory are
> still all LambdaForm related, and of that the biggest contributor is the
> Name arrays hanging off the Lfs. The majority of these are extremely small
> (26% have 0 or 1 elements, and over 60% have fewer than four) which
> suggests there is space to change small cases to instance variables and
> reserve arrays for more complex forms.
>
> I¹m not sure how much memory that will save, as I hope to reduce the
> number of LambdaForms by tweaking the way we combine method handles, but
> it is likely to remain important as customers run applications with larger
> data models.  The application I¹ve been examining has over 1.3 million
> LambdaForm$Names and arrays of same which account for about 70MB of memory
> usage without checking for retained usage, and there are over 100,000
> empty argument lists on LFNs, and another 500,000 with length 1 argument
> arrays, so even small improvement would be helpful here.
>
> Would it be possible to discuss these issues and a potential timeline for
> fixing them as part of the 8u update releases? Due to our commitments I
> doubt we¹ll be able to offer help in the development, but we can offer
> help in testing and should be able to provide sample heap dumps for
> analysis.

cheers,
Rémi

>
> On 22/05/2014 17:51, "MacGregor, Duncan (GE Energy Management)"
> <duncan.macgregor at ge.com> wrote:
>
>> I¹ve been doing some work towards reducing the memory footprint of our
>> applications under Java 8  and while picking through heap dumps have come
>> across a large number of java.lang.Object[] instances that have no
>> referrers but are not getting collected. I assume these are something to
>> do with static class data or class loader data and that these roots
>> aren¹t being represented in the heap dumps? At least one of these arrays
>> is of over 150000 elements and is entirely filled with nulls, and there
>> are a lot of smaller arrays as well so I would like to find out what is
>> generating them and why.
>>
> >From doing some comparisons with LambdaForm enabled versions of Java 7
>> which do not show this behaviour I¹m beginning to think that the
>> metaspace garbage collection is not interacting well with LambdaForm, at
>> least in our case,
>>
>> I¹m also finding a large number of DirectMethodHandles for our call site
>> bootstrap methods, all of which seem to be identical, so I assume we get
>> a new one for every class constant pool? We are loading a significant
>> number of classes in total, and using a large number of class loaders
>> (partly because of OSGi, and partly because we wanted code that is
>> immediately executed as part of the bootstrap process to be executed in
>> temporary Cls so that it and its resources could be Gced away), and I¹m
>> wondering if that is contributing to this issue.
>>
>> Any pointers you guys can give would be extremely useful.
>>
>> Thanks, Duncan.
>> _______________________________________________
>> mlvm-dev mailing list
>> mlvm-dev at openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> _______________________________________________
> mlvm-dev mailing list
> mlvm-dev at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev