PING: RFC for new JEP: Reduce metaspace waste by dynamically merging and splitting metaspace chunks.

[Thomas Stüfe]

Hi Mikael,

On Tue, Oct 11, 2016 at 11:26 AM, Mikael Gerdin <mikael.gerdin at oracle.com>
wrote:

> Hi Thomas,
>
> I have some questions about how your suggested change affects the
> allocation policy.
>
> Currently, once a SpaceManager has reached the _small_chunk_limit it will
> only allocate medium chunks even if an allocation request could be
> satisfied with a smaller variant.
>


> If metaspace is sufficiently fragmented by small chunks such that a medium
> chunk cannot be coalesced then we can still fail to satisfy an allocation
> request with large amounts of free memory, is this something that you have
> observed?
>

I did not see this in my tests, but in theory this is still possible even
with my patch. If free and in-use small chunks are tightly interspersed,
this would of course still prevent merging them to medium chunks. I think
to prevent this the chunk would have to be movable, so that they can be
compacted. Or, as a quick fix, the size ratio between small and medium
chunks could be made smaller, so that the chance that one stray in-use
small chunk prevents its free neighbors from being coalesced is smaller.

In the scenarios we observed we saw mostly free small chunks resting in the
freelist, because the class loaders which allocated them were all unloaded.

I try to understand how probable the scenario is you describe: in order to
fill up metaspace with small chunks - so that allocation of new medium
chunks fails - and to intermix free and in-use small chunks such that
coalescation to a medium chunk is not possible but there are still enough
free small "uncoalescable" chunks to matter (what you call "large amounts
of free memory"), one would have to have a quite "unlucky" allocation
pattern.

I think it is certainly possible, but less probable than the scenario this
patch tries to address: that metaspace was allocated in many small chunks
by class loaders which are now all dead and the small chunks are all free.

Kind Regards, Thomas


> Thanks
> /Mikael
>
> On 2016-10-10 10:56, Thomas Stüfe wrote:
>
>> Hi all,
>>
>> May I have please some feedback for this enhancement proposal?
>>
>> https://bugs.openjdk.java.net/browse/JDK-8166690
>> <https://bugs.openjdk.java.net/browse/JDK-8166690>
>>
>>
>> In one very short sentence it proposes a better allocation scheme for
>> Metaspace Chunks in order to reduce fragmentation and metaspace waste.
>>
>> I also added a short presentation which describes the problem and how we
>> solved it in our VM port.
>>
>> https://bugs.openjdk.java.net/secure/attachment/63894/Metasp
>> ace%20Coalescation%20in%20the%20SAP%20JVM.pdf
>>
>> Thank you very much!
>>
>> Kind Regards, Thomas
>>
>>
>> On Tue, Sep 27, 2016 at 10:45 AM, Thomas Stüfe <thomas.stuefe at gmail.com
>> <mailto:thomas.stuefe at gmail.com>> wrote:
>>
>>     Dear all,
>>
>>     please take a look at this Enhancement Proposal for the metaspace
>>     allocator. I hope these are the right groups for this discussion.
>>
>>     https://bugs.openjdk.java.net/browse/JDK-8166690
>>     <https://bugs.openjdk.java.net/browse/JDK-8166690>
>>
>>     Background:
>>
>>     We at SAP see at times at customer installations OOMs in Metaspace
>>     (usually, with compressed class pointers enabled, in Compressed
>>     Class Space). The VM attempts to allocate metaspace and fails,
>>     hitting the CompressedClassSpaceSize limit. Note that we usually set
>>     the limit lower than the default, typically at 256M.
>>
>>     When analyzing, we observed that a large part of the metaspace is
>>     indeed free but "locked in" into metaspace chunks of the wrong size:
>>     often we would find a lot of free small chunks, but the allocation
>>     request was for medium chunks, and failed.
>>
>>     The reason was that if at some point in time a lot of class loaders
>>     were alive, each with only a few small classes loaded. This would
>>     lead to the metaspace being swamped with lots of small chunks. This
>>     is because each SpaceManager first allocates small chunks, only
>>     after a certain amount of allocation requests switches to larger
>> chunks.
>>
>>     These small chunks are free and wait in the freelist, but cannot be
>>     reused for allocation requests which require larger chunks, even if
>>     they are physically adjacent in the virtual space.
>>
>>     We (at SAP) added a patch which allows on-the-fly metaspace chunk
>>     merging - to merge multiple adjacent smaller chunk to form a larger
>>     chunk. This, in combination with the reverse direction - splitting a
>>     large chunk to get smaller chunks - partly negates the
>>     "chunks-are-locked-in-into-their-size" limitation and provides for
>>     better reuse of metaspace chunks. It also provides better
>>     defragmentation as well.
>>
>>     I discussed this fix off-list with Coleen Phillimore and Jon
>>     Masamitsu, and instead of just offering this as a fix, both
>>     recommended to open a JEP for this, because its scope would be
>>     beyond that of a simple fix.
>>
>>     So here is my first JEP :) I hope it follows the right form. Please,
>>     if you have time, take a look and tell us what you think.
>>
>>     Thank you, and Kind Regards,
>>
>>     Thomas Stüfe
>>
>>
>>
>>
>>