metaspace proportion of fragmentation

[Thomas Stüfe]

Hi Brian,

On Sat, Oct 22, 2016 at 1:55 AM, Brian Toal <brian.toal at gmail.com> wrote:

> Good evening.  In a application that I'm responsible for, Metaspace is set
> to 1.1GB.  Specifically the following flags are set:
>
> -XX:MetaspaceSize=1152m
> -XX:MaxMetaspaceSize=1152m
> -XX:MinMetaspaceFreeRatio=0
> -XX:MaxMetaspaceFreeRatio=100
>
> However we are getting a OOME when metaspace size hits 80% of 1.1GB.
>

Out of metaspace or out of compressed class space? If the latter, have you
set CompressedClassSpaceSize?


> Doing a bit or research it seems that Metaspace is known to fragement the
> memory when a loader needs to acquire memory from the current chunk, and
> the current chuck can't accomodate the request, the pointer is bumped to
> the next available chunk, meaning any free memory in the previous chunks
> block is gone with the wind.
>

No. The remaining space is put into freelists (both on chunk and on block
level) and used for follow-up requests, should the size fit. In our
experience, we see very little wastage due to "half-eaten blocks/chunks.

There are other possible waste scenarios:

1) you have a lot of class loaders living in parallel. Each one will take a
chunk of memory (its current chunk) and satisfy memory requests from there.
This means that the current chunk always contains a portion of still-unused
memory which may be used by this class loader in the future but already
counts against MaxMetaspaceSize. However, to make this hurt, you really
have to have very many class loaders in parallel, as the maximum possible
overhead for this scenario cannot exceed the size of a medium chunk per
classloader (64k?)

2) The scenario described in this JEP here:
https://bugs.openjdk.java.net/browse/JDK-8166690 .

3) real fragmentation (i.e. a mixture of in-use and free chunks).

In my practice, I keep seeing (2). Hence the JEP, which will hopefully help.

Kind Regards, Thomas



> More than happy if someone corrects my understand here or can point me to
> a good reference that explains this in detail.
>
> My question is, how to do I monitor current usage + fragmentation so the
> proportion of free space can be monitored?
>
> Also is there any tuning that can take place to reduce the proportion of
> fragmentation?
>
> Does compressed cache acquire memory from memory set aside from memory
> allocated via MaxMetaspaceSize?
>
> Thanks in advance.
>
> _______________________________________________
> hotspot-gc-use mailing list
> hotspot-gc-use at openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20161025/5148cd0d/attachment-0001.html>