G1GC, Java8u40ea, Metaspace questions

[Stefan Karlsson]

Hi Wolfgang,

On 2015-03-05 21:41, Wolfgang Pedot wrote:
> Jenny,
>
> thanks for getting back to me with this info. I think I found a good 
> setting for now and I am letting a smaller system run with that under 
> more normal use (most concurrent cycles triggered by heap with only 
> some Metaspace-spikes).
> Definetly looking forward to use this "for real" after 8u40 is released.

8u40 has now been released.

>
> As for my thoughts below:
> As far as I know otherwise unused Classes are kept alive by their 
> ClassLoaders which are stored in the heap, right?

There are different ways to hold classes alive:
1) You have a live reference to the java.lang.ClassLoader (or subclass) 
object.
2) You have a live reference to any of the java.lang.Class objects 
belonging to the ClassLoader.
3) You have an instance of a class, that is described by any of the 
java.lang.Class objects belonging to the ClassLoader.
4) You have a "dependency" between a class in another ClassLoader, 
refering to a class in the ClassLoader that is kept alive. E.g. from 
class resolution in the constant pool, super classes, interfaces, JSR 
292 specific code.

You have to break all of these chains before your classes and class 
loader will be eligible for class unloading.

> So if Classloaders get promoted to oldGen mixedGCs are required to 
> clean them up before the Classes can be unloaded in the next 
> concurrent cycle. That would explain why it usually takes an 
> additional concurrent cycle (triggered by heap-occupation) after a 
> spike of class generation before Metaspace usage returns to normal. Or 
> maybe stuff that keeps the ClassLoaders alive needs to be collected 
> first...

We only require one marking cycle to clean out metadata. Maybe something 
is holding references to your class loader, classes, instances, but then 
gets cleaned out during the second GC. Things to look out for are, for 
example, SoftReferences and Finalizers.

After the remark phase, after the concurrent marking phase, we have 
enough information to unload the classes. Most of the JVM internal data 
structures are cleaned out during the remark phase, the actual metaspace 
memory is handed back during the cleanup phase.

If the JVM manages to clean out an entire "virtual space area" of 
metadata, the memory will be hand back to the OS and the amount of 
committed memory will be decreased. If not, it puts the committed memory 
onto the free lists so that it can be used by other metaspaces.

StefanK

>
> regards
> Wolfgang
>
>
>
> Am 05.03.2015 20:00, schrieb Yu Zhang:
>> Wolfgang,
>>
>> Thanks for reporting this.  I can reproduce this behavior with a micro.
>> After consulting with Stefan and Jon, it is the current behavior.  
>> For now you can keep MaxMetaspaceFreeRatio low to bring HWM down.  We 
>> might file an enhancement bug on this.
>>
>> You do not need a mixed gc to clean metaspace.
>> Thanks,
>> Jenny
>> On 2/19/2015 3:46 AM, Wolfgang Pedot wrote:
>>> One more, something just came to me:
>>>
>>> Class unloading happens during the concurrent marking-cycle so the 
>>> mixed collects that would free up unused classloaders in oldGen 
>>> happen after that, right?
>>> That would mean the classes can only be cleaned up at the next cycle 
>>> and stay in Metaspace until then. My test causes only 
>>> Metaspace-triggered concurrent cycles so the garbage-collector is 
>>> always behind by one cycle and therefor the amount of classes that 
>>> can be unloaded can be different each time, irregardless of the 
>>> percentage of wasted heap. I guess I have to extend my test-scenario 
>>> in a way that also causes at least some heap-driven concurrent 
>>> cycles and see what happens then.
>>> Still does not explain why I hardly ever see HWM go down but it 
>>> explains some of my more confusing test-results...
>>>
>>> regards
>>> Wolfgang
>>>
>>
>
>