RFR 8015391: NPG: With -XX:+UseCompressedKlassPointers OOME due to exhausted metadat, a space could occur when metaspace is almost empty

[Coleen Phillmore]

Hi Mikael,

I tested this with your patch and without my patch, we still get OOM 
[Class] Metadata space.   With both patches it doesn't.   At the end 
this is one of the class loaders that's left.  There were others of 
these class loaders but they get unloaded.

1091) Metachunk: bottom 0x00000007724ed800 top 0x00000007724ede70 end 
0x00000007724ee000 size 256
     used 206 free 50
1092) Metachunk: bottom 0x00000007724ed000 top 0x00000007724ed670 end 
0x00000007724ed800 size 256
     used 206 free 50
total of all chunks 279808 used 225026 free 182 capacity 279808 waste 63344

Even though this is an extreme case, going to 4K word medium chunks 
after using up 4x 256 word small chunks for class metaspace still makes 
sense.  Maybe 4k could be 2k.   I think class loaders either allocate 
few or many classes.   We should tune this more for some in-house 
applications when some of the other problems are resolved.

thanks,
Coleen

On 6/20/2013 4:19 PM, Mikael Gerdin wrote:
> Coleen,
> this is not a final review, just some comments on the methodology on 
> how we make this kind of changes.
>
> On 06/20/2013 06:26 PM, Coleen Phillimore wrote:
>> Summary: Allocate medium chunks for class metaspace when class loader
>> has lots of classes
>>
>> I originally made class metaspace keep small chunks and not start
>> allocating from medium chunks, because I thought with only Klass
>> objects, small chunks is enough.  This test has a large set of class
>> loaders who have a lot of classes, so allocated thousands of small
>> chunks each.  Class loaders with that many classes should start
>> allocating from medium chunks, for class metaspace.
>
> Can you describe exactly how making use of medium chunks for class 
> metaspace increases the memory efficiency.
> * Is it because of the overhead of each chunk?
> * Is it because the small size of the chunks lead to wasted memory at 
> the end of each chunk?
> * Any other reason which I didn't mention?
>
> In the second case, can you please try with my fix for JDK-8009561?
> The webrev is at http://cr.openjdk.java.net/~mgerdin/8009561/webrev.0/
> and http://cr.openjdk.java.net/~mgerdin/8009561/webrev.0%2b/
>
> I know you lurk on gc-dev so search for "Request for review: 
> JDK-8009561 NPG: Metaspace fragmentation when retiring a Metachunk"
>
> :)
>
>>
>> I also increased the ClassMediumChunk size to 4k and measured a lot less
>> fragmentation waste than 1K or 2K for this test.  Lastly, the
>> AppClassLoader instance should have as large of an initial metaspace as
>> the bootclass loader.
>
> Fragmentation in the general case or on a specific (possibly 
> degenerate) test case?
> I think we need to be careful here not to optimize for some strange 
> test which just loads a lot of classes and does nothing useful.
> No matter what allocation policy / chunk sizes we come up with there's 
> always a way to design a test which will cause bad behavior.
>
>>
>> Tested with vm.quick.testlist and the failing test.
>>
>> open webrev at http://cr.openjdk.java.net/~coleenp/8015391/
>
> The changes to the preallocated exception oops are sane but should 
> probably be done as a separate change...
>
>
> /Mikael
>
>> bug link at http://bugs.sun.com/view_bug.do?bug_id=8015391
>> local bug link https://jbs.oracle.com/bugs/browse/JDK-8015391
>>
>> Thanks,
>> Coleen
>>
>