RFR 8015391: NPG: With -XX:+UseCompressedKlassPointers OOME due to exhausted metadat, a space could occur when metaspace is almost empty

[Coleen Phillimore]

Hi Mikael,
I don't lurk on hotspot-gc anymore!  I can't keep up with the mails. :)

On 06/20/2013 04:19 PM, Mikael Gerdin wrote:
> Coleen,
> this is not a final review, just some comments on the methodology on 
> how we make this kind of changes.
>
> On 06/20/2013 06:26 PM, Coleen Phillimore wrote:
>> Summary: Allocate medium chunks for class metaspace when class loader
>> has lots of classes
>>
>> I originally made class metaspace keep small chunks and not start
>> allocating from medium chunks, because I thought with only Klass
>> objects, small chunks is enough.  This test has a large set of class
>> loaders who have a lot of classes, so allocated thousands of small
>> chunks each.  Class loaders with that many classes should start
>> allocating from medium chunks, for class metaspace.
>
> Can you describe exactly how making use of medium chunks for class 
> metaspace increases the memory efficiency.
> * Is it because of the overhead of each chunk?
> * Is it because the small size of the chunks lead to wasted memory at 
> the end of each chunk?
> * Any other reason which I didn't mention?

Reasons 1 and 2 above.
>
> In the second case, can you please try with my fix for JDK-8009561?
> The webrev is at http://cr.openjdk.java.net/~mgerdin/8009561/webrev.0/
> and http://cr.openjdk.java.net/~mgerdin/8009561/webrev.0%2b/
>

I missed your fix.   Even if we are reusing small chunks better, I still 
believe class metadata should get a medium chunk like the data metaspace.

> I know you lurk on gc-dev so search for "Request for review: 
> JDK-8009561 NPG: Metaspace fragmentation when retiring a Metachunk"
>
> :)
>
>>
>> I also increased the ClassMediumChunk size to 4k and measured a lot less
>> fragmentation waste than 1K or 2K for this test.  Lastly, the
>> AppClassLoader instance should have as large of an initial metaspace as
>> the bootclass loader.
>
> Fragmentation in the general case or on a specific (possibly 
> degenerate) test case?
> I think we need to be careful here not to optimize for some strange 
> test which just loads a lot of classes and does nothing useful.
> No matter what allocation policy / chunk sizes we come up with there's 
> always a way to design a test which will cause bad behavior.

Yes,  I wish we had a good way to measure this generally.  I did measure 
the degenerate case.

>
>>
>> Tested with vm.quick.testlist and the failing test.
>>
>> open webrev at http://cr.openjdk.java.net/~coleenp/8015391/
>
> The changes to the preallocated exception oops are sane but should 
> probably be done as a separate change...
>

I needed these changes to separate out which metaspace was running out 
of memory.  I need to do this.

Coleen

>
> /Mikael
>
>> bug link at http://bugs.sun.com/view_bug.do?bug_id=8015391
>> local bug link https://jbs.oracle.com/bugs/browse/JDK-8015391
>>
>> Thanks,
>> Coleen
>>
>