Object arrays with no referrers.

[MacGregor, Duncan (GE Energy Management)]

Okay, I¹ve done some more digging, and although I still can¹t track down
the source of individual referrer-less Object arrays (I¹m not sure the
necessary information is even present in an hprof dump) I can say that
they are mostly due to anonymous classes (based on the contents of some I
have examined), and I do now understand what¹s going on with the large
number of DMHs, and some other areas of memory usage.

1. Method handles and class constant pools

Since we¹re operating in an OSGi environment
java.lang.invoke.MethodHandles.Lookup.canBeCached() tends to return false,
a lot. This is probably not a bad thing in and of itself as the
LOOKASIDE_ASIDE could otherwise become full of DMHs which end up
preventing large structures from being collected, but it does cause
problems for us as almost all the DMH constants in our classes are
pointing to the same small set of bootstrap methods. I¹ve also found it is
horribly easy to generate a large set of DMHs for things like the fallback
and cache invalidation methods of call sites, but this appears to be
easily fixable by tweaking the order of binding and folding operations.

So, would it be possible to change the caching of DMHs to be based on
ClassValues and allowing significantly more DMHs to be cached?

2. LambdaForm$Name[] memory usage

Running outside of an OSGi environment our biggest uses of memory are
still all LambdaForm related, and of that the biggest contributor is the
Name arrays hanging off the Lfs. The majority of these are extremely small
(26% have 0 or 1 elements, and over 60% have fewer than four) which
suggests there is space to change small cases to instance variables and
reserve arrays for more complex forms.

I¹m not sure how much memory that will save, as I hope to reduce the
number of LambdaForms by tweaking the way we combine method handles, but
it is likely to remain important as customers run applications with larger
data models.  The application I¹ve been examining has over 1.3 million
LambdaForm$Names and arrays of same which account for about 70MB of memory
usage without checking for retained usage, and there are over 100,000
empty argument lists on LFNs, and another 500,000 with length 1 argument
arrays, so even small improvement would be helpful here.

Would it be possible to discuss these issues and a potential timeline for
fixing them as part of the 8u update releases? Due to our commitments I
doubt we¹ll be able to offer help in the development, but we can offer
help in testing and should be able to provide sample heap dumps for
analysis.

On 22/05/2014 17:51, "MacGregor, Duncan (GE Energy Management)"
<duncan.macgregor at ge.com> wrote:

>I¹ve been doing some work towards reducing the memory footprint of our
>applications under Java 8  and while picking through heap dumps have come
>across a large number of java.lang.Object[] instances that have no
>referrers but are not getting collected. I assume these are something to
>do with static class data or class loader data and that these roots
>aren¹t being represented in the heap dumps? At least one of these arrays
>is of over 150000 elements and is entirely filled with nulls, and there
>are a lot of smaller arrays as well so I would like to find out what is
>generating them and why.
>
>From doing some comparisons with LambdaForm enabled versions of Java 7
>which do not show this behaviour I¹m beginning to think that the
>metaspace garbage collection is not interacting well with LambdaForm, at
>least in our case,
>
>I¹m also finding a large number of DirectMethodHandles for our call site
>bootstrap methods, all of which seem to be identical, so I assume we get
>a new one for every class constant pool? We are loading a significant
>number of classes in total, and using a large number of class loaders
>(partly because of OSGi, and partly because we wanted code that is
>immediately executed as part of the bootstrap process to be executed in
>temporary Cls so that it and its resources could be Gced away), and I¹m
>wondering if that is contributing to this issue.
>
>Any pointers you guys can give would be extremely useful.
>
>Thanks, Duncan.
>_______________________________________________
>mlvm-dev mailing list
>mlvm-dev at openjdk.java.net
>http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev