Excessive "GC" memory area size [was: Re: Determining "GC" memory area size]
Glyn Normington
gnormington at pivotal.io
Thu Dec 14 11:19:02 UTC 2017
On Wed, Dec 13, 2017 at 5:18 PM, Zhengyu Gu <zgu at redhat.com> wrote:
> Hi Glyn,
>
> On 12/13/2017 11:38 AM, Glyn Normington wrote:
>
>> Hi Zhengyu
>>
>> Thanks again for a great suggestion. JDK 10 gave more insights such as
>> this one showing a task queue-related malloc of 8 MB:
>>
>> 0x000000010d128e58] _ZN20MallocArrayAllocatorI16G1
>> TaskQueueEntryE8allocateEm10MemoryType+0x5c
>> [0x000000010d12372e] _ZN16G1ConcurrentMarkC2EP15G1C
>> ollectedHeapP21G1RegionToSpaceMapperS3_+0x8e4
>> [0x000000010d1198af] _ZN15G1CollectedHeap10initializeEv+0x561
>> [0x000000010d559029] _ZN8Universe15initialize_heapEv+0x4f
>> (malloc=8192KB type=GC #8)
>>
>> The malloc details don't seem to add up to the total, so maybe some
>> allocation sites got missed, but it's definitely better than with JDK 8.
>>
>
> Detail tracking does discard allocation sites with small enough
> allocation, e.g. < 1K when scale = KB. But I will be surprised if the
> disparity is significant.
>
You're right. The malloc details add up to 17524 KB which is almost equal
to the reported total of 17544 KB.
There was another large malloc, which I guess might also be task
queue-related:
[0x000000010d11dbc4]
_ZN20MallocArrayAllocatorI8StarTaskE8allocateEm10MemoryType+0x5c
[0x000000010d119124] _ZN15G1CollectedHeapC2EP17G1CollectorPolicy+0x82c
[0x000000010d11274a]
_ZN11GCArguments23create_heap_with_policyI15G1CollectedHeap17G1CollectorPolicyEEP13CollectedHeapv+0x44
[0x000000010d559019] _ZN8Universe15initialize_heapEv+0x3f
(malloc=8192KB type=GC #8)
>
> Thanks,
>
> -Zhengyu
>
>
>
>
>> Regards,
>> Glyn
>>
>>
>> On Wed, Dec 13, 2017 at 3:16 PM, Zhengyu Gu <zgu at redhat.com <mailto:
>> zgu at redhat.com>> wrote:
>>
>> Hi Glyn,
>>
>> On 12/13/2017 09:35 AM, Glyn Normington wrote:
>>
>> Thanks Zhengyu! That got me quite a bit further.
>>
>> The NMT virtual memory map showed the mmap portion of GC space
>> consisting of:
>> * the two marking bitmaps you mentioned previously, each 1/64 of
>> heap size.
>> * a mark stack of 32 MB which is 8* MarkStackSize (but which
>> could grow to 8* MarkStackSizeMax or 128 MB).
>> * three 700 KB areas, although I'm not sure what these are used
>> for or how to predict their total size.
>>
>> This leaves the malloc portion of GC space "17615KB #2312" which
>> I think was due 2312 calls to malloc. Again, not sure what these
>> are used for or how to predict their total size except that I
>> presume they include the task queues you mentioned previously.
>>
>>
>> Can you try on JDK10? it has enhancement [1] which can show what
>> memory allocation is for, on each allocation site.
>>
>>
>> [1] https://bugs.openjdk.java.net/browse/JDK-8139673
>> <https://bugs.openjdk.java.net/browse/JDK-8139673>
>>
>> Thanks,
>>
>> -Zhengyu
>>
>>
>> Outstanding questions:
>> 1. Is the mmap space which consumed three 700 KB areas above
>> fixed, proportional to heap size, or what?
>> 2. How many task queues are there for a given number of GC
>> threads?
>> 3. Task queues can overflow, so what is the maximum space each
>> one can occupy?
>> 4. Of the malloc space not consumed by task queues, how much is
>> fixed and how much proportional to heap size, number of GC
>> threads, etc.?
>>
>> The aim is to calculate a reasonable estimate of an upper bound
>> for GC space consumption.
>>
>> Raw data for anyone who's interested:
>> https://gist.github.com/glyn/a3bd944eb2943f4f5a98b5ac712efeb6
>> <https://gist.github.com/glyn/a3bd944eb2943f4f5a98b5ac712efeb6>
>>
>> Regards,
>> Glyn
>>
>> On Wed, Dec 13, 2017 at 12:27 PM, Zhengyu Gu <zgu at redhat.com
>> <mailto:zgu at redhat.com> <mailto:zgu at redhat.com
>>
>> <mailto:zgu at redhat.com>>> wrote:
>>
>> Hi Glyn,
>>
>> You can run NMT with detail tracking
>> (-XX:NativeMemoryTracking=detail) then use jcmd (jcmd <pid>
>> VM.native_memory detail) to find out all allocations for GC.
>>
>> For example:
>>
>> [0x00007f9fba000000 - 0x00007f9fbc000000] reserved 32768KB
>> for *GC* from
>> [0x00007fa0130260ef] G1CMMarkStack::resize(unsigned
>> long)+0x8f
>> [0x00007fa01302cf62]
>> G1ConcurrentMark::G1ConcurrentMark(G1CollectedHeap*,
>> G1RegionToSpaceMapper*, G1RegionToSpaceMapper*)+0x782
>> [0x00007fa01300e94b] G1CollectedHeap::initialize()+
>> 0x62b
>> [0x00007fa013ca26af] universe_init()+0x6f
>>
>> [0x00007f9fba000000 - 0x00007f9fbc000000]
>> committed 32768KB
>> from
>> [0x00007fa01302610d]
>> G1CMMarkStack::resize(unsigned
>> long)+0xad
>> [0x00007fa01302cf62]
>> G1ConcurrentMark::G1ConcurrentMark(G1CollectedHeap*,
>> G1RegionToSpaceMapper*, G1RegionToSpaceMapper*)+0x782
>> [0x00007fa01300e94b]
>> G1CollectedHeap::initialize()+0x62b
>> [0x00007fa013ca26af] universe_init()+0x6f
>>
>>
>> [0x00007fa012ac5395] ArrayAllocator<unsigned
>> long>::allocate(unsigned long, MemoryType)+0x245
>> [0x00007fa012ac5448] unsigned long*
>> BitMap::reallocate<CHeapBitMapAllocator>(CHeapBitMapAllocat
>> or
>> const&, unsigned long*, unsigned long, unsigned long)+0x38
>> [0x00007fa012ac509f] CHeapBitMap::initialize(unsigned
>> long)+0x6f
>> [0x00007fa013093b9c]
>> G1PageBasedVirtualSpace::G1PageBasedVirtualSpace(ReservedSp
>> ace,
>> unsigned long, unsigned long)+0x13c
>> (malloc=129KB *type=GC* #6)
>>
>> Thanks,
>>
>> -Zhengyu
>>
>>
>>
>>
>> On 12/13/2017 04:39 AM, Glyn Normington wrote:
>>
>> On the basis of the reply below, we have been thinking of
>> estimating the
>> size of the "GC" memory area as a little more than
>> (1/32) *
>> heapSize.
>> However, running `java -version` with a 350m heap
>> results in
>> "GC" memory as
>> reported by NMT of over 17% of the heap size, so this
>> estimate
>> is way out.
>>
>> If the analysis below is accurate, then given the
>> number of GC
>> threads is
>> 10 (ParallelGCThreads=8, ConcGCThreads=2), can there be
>> many more
>> taskqueues than GC threads?
>>
>> Or is there another explanation for the additional "GC"
>> size?
>>
>> Thanks,
>> Glyn
>>
>> [1]
>> https://gist.github.com/glyn/f10b7dea77994ad0f5589a416fbc77fe
>> <https://gist.github.com/glyn/f10b7dea77994ad0f5589a416fbc77fe>
>> <https://gist.github.com/glyn/
>> f10b7dea77994ad0f5589a416fbc77fe
>> <https://gist.github.com/glyn/f10b7dea77994ad0f5589a416fbc77fe>>
>>
>> On Fri, Nov 10, 2017 at 7:05 PM, Zhengyu Gu
>> <zgu at redhat.com <mailto:zgu at redhat.com>
>> <mailto:zgu at redhat.com <mailto:zgu at redhat.com>>> wrote:
>>
>> Hi Glyn,
>>
>>
>> Can anyone here tell me how the GC memory area
>> size is
>> determined? If
>> there
>> is documentation, so much the better as we'd
>> prefer not
>> to depend on code
>> details that might flux arbitrarily.
>>
>>
>> GC memory is mainly data structures used by GC
>> runtime. It
>> can be varied
>> by collector used, size of the Java heap, the
>> number of GC
>> threads and etc.
>> and, of course, the application itself.
>>
>> Some are *fixed* costs, which can be estimated.
>> E.g. two
>> marking bitmaps
>> used by G1, each costs 1/64 of heap size (assuming
>> default
>> object
>> alignment).
>>
>> Some are *semi-fixed*, e.g. taskqueue's fixed cost
>> is about
>> 1M for each
>> queue on 64-bits VM, but it can overflow. And the
>> number of
>> task queues is
>> proportional to the number of GC threads.
>>
>> Then there are factors from application itself,
>> such as
>> object mutation
>> rate, inter-generation/region references, etc.
>>
>>
>> I don't see a single formula for estimating GC
>> memory size.
>> If you are
>> using G1, the biggest overhead comes from 2 bitmaps
>> (1/32 *
>> heap size).
>>
>> Thanks,
>>
>> -Zhengyu
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> -- Regards,
>> Glyn
>>
>>
>>
>>
>> --
>> Regards,
>> Glyn
>>
>
--
Regards,
Glyn
More information about the hotspot-dev
mailing list