G1GC, Java8u40ea, Metaspace questions

[Yu Zhang]

Hi,

After discussing with the developers, here is the explanation:
 >shows a slow but continuous increase of the "commited" value for 
Metaspace even though "used" after a collect is quite stable and 
"commited" is shrunk after every collect.

This might be due to fragmentation in metadata space.  Even though the 
total used is stable, the memory is fragmented so it has to allocate 
more.  If you have Flight Recorder(a commercial feature), you can see 
this information in JFR.
 >I have found out that I need to set Min and MaxMetaspaceFreeRatio 
really really low (like <=10%) to keep the high-water-mark in check and 
I dont quite see why in the numbers.

This might be related to how the high water mark is calculated. There 
are some ways to improve this but not implemented yet.

Thank you very much for your feedback.

Thanks,
Jenny

On 2/17/2015 11:51 AM, Wolfgang Pedot wrote:
> Hi,
>
> the last log was with a MaxMetaspaceFreeRatio of 10% and that was 
> quite well behaved. The log I have sent before (starting 
> 2015-02-13T20:04:43.512+0100) shows a slow but continuous increase of 
> the "commited" value for Metaspace even though "used" after a collect 
> is quite stable and "commited" is shrunk after every collect. Once a 
> configured MaxMetaspaceSize is reached the FullGCs start. I do have a 
> log showing that but that is with smaller sizes (Metaspace=300MB, 
> MaxMetaspace=305MB, MinMetaspaceFreeRatio=0) so I dont know if it will 
> be a good case. I have found out that I need to set Min and 
> MaxMetaspaceFreeRatio really really low (like <=10%) to keep the 
> high-water-mark in check and I dont quite see why in the numbers.
>
> I can send you the 300/305MB log if it helps, otherwise do you want me 
> to run another test with specific configuration-values?
>
> Wolfgang
>
>
> Am 17.02.2015 20:22, schrieb Yu Zhang:
>> Jon,
>>
>> Here is the version:
>> Java HotSpot(TM) 64-Bit Server VM (25.40-b25) for windows-amd64 JRE 
>> (1.8.0_40-ea-b23), built on Jan 27 2015 19:35:49 by "java_re" with MS 
>> VC++ 10.0 (VS2010)
>>
>> Wolfang,
>> Thanks for the logs.  But the one you sent me seems pretty well 
>> behaved.  There is no full gc.  The interval between initial-marking 
>> for metesapce is not getting longer.  Do you have a log that you saw 
>> the high-warter mark is getting higher, so the concurrent marking is 
>> less frequent, then there is a full gc?
>> Thanks,
>> Jenny
>> On 2/13/2015 11:46 AM, Jon Masamitsu wrote:
>>>
>>> On 2/13/2015 10:25 AM, Wolfgang Pedot wrote:
>>>> Thanks for your reply,
>>>>
>>>> I was under the impression that the MetaspaceSize is only used as a 
>>>> starting value for the high-water mark and 
>>> That's correct.
>>>
>>>> adjusting it will only change the behaviour at startup. Correct me 
>>>> if I am wrong but after the first collect ergonomics 
>>>
>>> If "at startup" includes the  first time the metaspace reaches 
>>> MetaspaceSize, yes that is also correct.
>>>
>>>> would take over and adjust that mark according to 
>>>> Min/MaxMetaspaceFreeRatio, right? My problem is that I have to 
>>> Right.
>>>
>>>> deal with lots of short lived generated classes and it would be a 
>>>> step back to have the garbage-collector set a low mark right before 
>>>> a class-generation spike and then do a Full-GC before raising it 
>>>> again (Full-GC currently takes ~10sec on the real system). Although 
>>>> with MaxMetaSpaceFreeRation being 70% by default that would 
>>>> probably not happen that often.
>>>>
>>>> As for my first question, it is still not clear to me why Metaspace 
>>>> "capacity" never seemed to reach "commited" during my first tests. 
>>>> I have checked the logs for the current instance and with the 
>>>> higher maximum-setting "capacity" and "commited" actually can get 
>>>> quite close now but both are nowhere near the allowed maximum even 
>>>> after I have driven the JVM into multiple FullGCs by creating 
>>>> classes like crazy. Can I assume that RAM that is not 
>>>
>>>
>