Excessive "GC" memory area size [was: Re: Determining "GC" memory area size]

[Zhengyu Gu]

Hi Glyn,

You can run NMT with detail tracking (-XX:NativeMemoryTracking=detail) 
then use jcmd (jcmd <pid> VM.native_memory detail) to find out all 
allocations for GC.

For example:

[0x00007f9fba000000 - 0x00007f9fbc000000] reserved 32768KB for *GC* from
     [0x00007fa0130260ef] G1CMMarkStack::resize(unsigned long)+0x8f
     [0x00007fa01302cf62] 
G1ConcurrentMark::G1ConcurrentMark(G1CollectedHeap*, 
G1RegionToSpaceMapper*, G1RegionToSpaceMapper*)+0x782
     [0x00007fa01300e94b] G1CollectedHeap::initialize()+0x62b
     [0x00007fa013ca26af] universe_init()+0x6f

         [0x00007f9fba000000 - 0x00007f9fbc000000] committed 32768KB from
             [0x00007fa01302610d] G1CMMarkStack::resize(unsigned long)+0xad
             [0x00007fa01302cf62] 
G1ConcurrentMark::G1ConcurrentMark(G1CollectedHeap*, 
G1RegionToSpaceMapper*, G1RegionToSpaceMapper*)+0x782
             [0x00007fa01300e94b] G1CollectedHeap::initialize()+0x62b
             [0x00007fa013ca26af] universe_init()+0x6f


[0x00007fa012ac5395] ArrayAllocator<unsigned long>::allocate(unsigned 
long, MemoryType)+0x245
[0x00007fa012ac5448] unsigned long* 
BitMap::reallocate<CHeapBitMapAllocator>(CHeapBitMapAllocator const&, 
unsigned long*, unsigned long, unsigned long)+0x38
[0x00007fa012ac509f] CHeapBitMap::initialize(unsigned long)+0x6f
[0x00007fa013093b9c] 
G1PageBasedVirtualSpace::G1PageBasedVirtualSpace(ReservedSpace, unsigned 
long, unsigned long)+0x13c
                              (malloc=129KB *type=GC* #6)

Thanks,

-Zhengyu



On 12/13/2017 04:39 AM, Glyn Normington wrote:
> On the basis of the reply below, we have been thinking of estimating the
> size of the "GC" memory area as a little more than (1/32) * heapSize.
> However, running `java -version` with a 350m heap results in "GC" memory as
> reported by NMT of over 17% of the heap size, so this estimate is way out.
> 
> If the analysis below is accurate, then given the number of GC threads is
> 10 (ParallelGCThreads=8, ConcGCThreads=2), can there be many more
> taskqueues than GC threads?
> 
> Or is there another explanation for the additional "GC" size?
> 
> Thanks,
> Glyn
> 
> [1] https://gist.github.com/glyn/f10b7dea77994ad0f5589a416fbc77fe
> 
> On Fri, Nov 10, 2017 at 7:05 PM, Zhengyu Gu <zgu at redhat.com> wrote:
> 
>> Hi Glyn,
>>
>>
>>> Can anyone here tell me how the GC memory area size is determined? If
>>> there
>>> is documentation, so much the better as we'd prefer not to depend on code
>>> details that might flux arbitrarily.
>>>
>>>
>> GC memory is mainly data structures used by GC runtime. It can be varied
>> by collector used, size of the Java heap, the number of GC threads and etc.
>> and, of course, the application itself.
>>
>> Some are *fixed* costs, which can be estimated. E.g. two marking bitmaps
>> used by G1, each costs 1/64 of heap size (assuming default object
>> alignment).
>>
>> Some are *semi-fixed*, e.g. taskqueue's fixed cost is about 1M for each
>> queue on 64-bits VM, but it can overflow. And the number of task queues is
>> proportional to the number of GC threads.
>>
>> Then there are factors from application itself,  such as object mutation
>> rate, inter-generation/region references, etc.
>>
>>
>> I don't see a single formula for estimating GC memory size. If you are
>> using G1, the biggest overhead comes from 2 bitmaps (1/32 * heap size).
>>
>> Thanks,
>>
>> -Zhengyu
>>
>>
>>
>>
> 
>