Excessive "GC" memory area size [was: Re: Determining "GC" memory area size]
Zhengyu Gu
zgu at redhat.com
Thu Dec 14 13:39:59 UTC 2017
On 12/14/2017 06:19 AM, Glyn Normington wrote:
> On Wed, Dec 13, 2017 at 5:18 PM, Zhengyu Gu <zgu at redhat.com
> <mailto:zgu at redhat.com>> wrote:
>
> Hi Glyn,
>
> On 12/13/2017 11:38 AM, Glyn Normington wrote:
>
> Hi Zhengyu
>
> Thanks again for a great suggestion. JDK 10 gave more insights
> such as this one showing a task queue-related malloc of 8 MB:
>
> 0x000000010d128e58]
> _ZN20MallocArrayAllocatorI16G1TaskQueueEntryE8allocateEm10MemoryType+0x5c
> [0x000000010d12372e]
> _ZN16G1ConcurrentMarkC2EP15G1CollectedHeapP21G1RegionToSpaceMapperS3_+0x8e4
> [0x000000010d1198af] _ZN15G1CollectedHeap10initializeEv+0x561
> [0x000000010d559029] _ZN8Universe15initialize_heapEv+0x4f
> (malloc=8192KB type=GC #8)
>
> The malloc details don't seem to add up to the total, so maybe
> some allocation sites got missed, but it's definitely better
> than with JDK 8.
>
>
> Detail tracking does discard allocation sites with small enough
> allocation, e.g. < 1K when scale = KB. But I will be surprised if
> the disparity is significant.
>
>
> You're right. The malloc details add up to 17524 KB which is almost
> equal to the reported total of 17544 KB.
>
> There was another large malloc, which I guess might also be task
> queue-related:
>
> [0x000000010d11dbc4]
> _ZN20MallocArrayAllocatorI8StarTaskE8allocateEm10MemoryType+0x5c
> [0x000000010d119124] _ZN15G1CollectedHeapC2EP17G1CollectorPolicy+0x82c
> [0x000000010d11274a]
> _ZN11GCArguments23create_heap_with_policyI15G1CollectedHeap17G1CollectorPolicyEEP13CollectedHeapv+0x44
> [0x000000010d559019] _ZN8Universe15initialize_heapEv+0x3f
> (malloc=8192KB type=GC #8)
>
>
Yes, looks like creation of task queues.
-Zhengyu
> Thanks,
>
> -Zhengyu
>
>
>
>
> Regards,
> Glyn
>
>
> On Wed, Dec 13, 2017 at 3:16 PM, Zhengyu Gu <zgu at redhat.com
> <mailto:zgu at redhat.com> <mailto:zgu at redhat.com
> <mailto:zgu at redhat.com>>> wrote:
>
> Hi Glyn,
>
> On 12/13/2017 09:35 AM, Glyn Normington wrote:
>
> Thanks Zhengyu! That got me quite a bit further.
>
> The NMT virtual memory map showed the mmap portion of
> GC space
> consisting of:
> * the two marking bitmaps you mentioned previously,
> each 1/64 of
> heap size.
> * a mark stack of 32 MB which is 8* MarkStackSize (but
> which
> could grow to 8* MarkStackSizeMax or 128 MB).
> * three 700 KB areas, although I'm not sure what these
> are used
> for or how to predict their total size.
>
> This leaves the malloc portion of GC space "17615KB
> #2312" which
> I think was due 2312 calls to malloc. Again, not sure
> what these
> are used for or how to predict their total size except
> that I
> presume they include the task queues you mentioned
> previously.
>
>
> Can you try on JDK10? it has enhancement [1] which can show
> what
> memory allocation is for, on each allocation site.
>
>
> [1] https://bugs.openjdk.java.net/browse/JDK-8139673
> <https://bugs.openjdk.java.net/browse/JDK-8139673>
> <https://bugs.openjdk.java.net/browse/JDK-8139673
> <https://bugs.openjdk.java.net/browse/JDK-8139673>>
>
> Thanks,
>
> -Zhengyu
>
>
> Outstanding questions:
> 1. Is the mmap space which consumed three 700 KB areas
> above
> fixed, proportional to heap size, or what?
> 2. How many task queues are there for a given number of
> GC threads?
> 3. Task queues can overflow, so what is the maximum
> space each
> one can occupy?
> 4. Of the malloc space not consumed by task queues, how
> much is
> fixed and how much proportional to heap size, number of GC
> threads, etc.?
>
> The aim is to calculate a reasonable estimate of an
> upper bound
> for GC space consumption.
>
> Raw data for anyone who's interested:
> https://gist.github.com/glyn/a3bd944eb2943f4f5a98b5ac712efeb6
> <https://gist.github.com/glyn/a3bd944eb2943f4f5a98b5ac712efeb6>
>
> <https://gist.github.com/glyn/a3bd944eb2943f4f5a98b5ac712efeb6
> <https://gist.github.com/glyn/a3bd944eb2943f4f5a98b5ac712efeb6>>
>
> Regards,
> Glyn
>
> On Wed, Dec 13, 2017 at 12:27 PM, Zhengyu Gu
> <zgu at redhat.com <mailto:zgu at redhat.com>
> <mailto:zgu at redhat.com <mailto:zgu at redhat.com>>
> <mailto:zgu at redhat.com <mailto:zgu at redhat.com>
>
> <mailto:zgu at redhat.com <mailto:zgu at redhat.com>>>> wrote:
>
> Hi Glyn,
>
> You can run NMT with detail tracking
> (-XX:NativeMemoryTracking=detail) then use jcmd
> (jcmd <pid>
> VM.native_memory detail) to find out all
> allocations for GC.
>
> For example:
>
> [0x00007f9fba000000 - 0x00007f9fbc000000] reserved
> 32768KB
> for *GC* from
> [0x00007fa0130260ef]
> G1CMMarkStack::resize(unsigned
> long)+0x8f
> [0x00007fa01302cf62]
> G1ConcurrentMark::G1ConcurrentMark(G1CollectedHeap*,
> G1RegionToSpaceMapper*, G1RegionToSpaceMapper*)+0x782
> [0x00007fa01300e94b]
> G1CollectedHeap::initialize()+0x62b
> [0x00007fa013ca26af] universe_init()+0x6f
>
> [0x00007f9fba000000 - 0x00007f9fbc000000]
> committed 32768KB
> from
> [0x00007fa01302610d]
> G1CMMarkStack::resize(unsigned
> long)+0xad
> [0x00007fa01302cf62]
> G1ConcurrentMark::G1ConcurrentMark(G1CollectedHeap*,
> G1RegionToSpaceMapper*, G1RegionToSpaceMapper*)+0x782
> [0x00007fa01300e94b]
> G1CollectedHeap::initialize()+0x62b
> [0x00007fa013ca26af] universe_init()+0x6f
>
>
> [0x00007fa012ac5395] ArrayAllocator<unsigned
> long>::allocate(unsigned long, MemoryType)+0x245
> [0x00007fa012ac5448] unsigned long*
>
> BitMap::reallocate<CHeapBitMapAllocator>(CHeapBitMapAllocator
> const&, unsigned long*, unsigned long, unsigned
> long)+0x38
> [0x00007fa012ac509f] CHeapBitMap::initialize(unsigned
> long)+0x6f
> [0x00007fa013093b9c]
>
> G1PageBasedVirtualSpace::G1PageBasedVirtualSpace(ReservedSpace,
> unsigned long, unsigned long)+0x13c
> (malloc=129KB
> *type=GC* #6)
>
> Thanks,
>
> -Zhengyu
>
>
>
>
> On 12/13/2017 04:39 AM, Glyn Normington wrote:
>
> On the basis of the reply below, we have been
> thinking of
> estimating the
> size of the "GC" memory area as a little more
> than (1/32) *
> heapSize.
> However, running `java -version` with a 350m heap
> results in
> "GC" memory as
> reported by NMT of over 17% of the heap size,
> so this
> estimate
> is way out.
>
> If the analysis below is accurate, then given the
> number of GC
> threads is
> 10 (ParallelGCThreads=8, ConcGCThreads=2), can
> there be
> many more
> taskqueues than GC threads?
>
> Or is there another explanation for the
> additional "GC"
> size?
>
> Thanks,
> Glyn
>
> [1]
> https://gist.github.com/glyn/f10b7dea77994ad0f5589a416fbc77fe
> <https://gist.github.com/glyn/f10b7dea77994ad0f5589a416fbc77fe>
>
> <https://gist.github.com/glyn/f10b7dea77994ad0f5589a416fbc77fe
> <https://gist.github.com/glyn/f10b7dea77994ad0f5589a416fbc77fe>>
>
> <https://gist.github.com/glyn/f10b7dea77994ad0f5589a416fbc77fe
> <https://gist.github.com/glyn/f10b7dea77994ad0f5589a416fbc77fe>
>
> <https://gist.github.com/glyn/f10b7dea77994ad0f5589a416fbc77fe
> <https://gist.github.com/glyn/f10b7dea77994ad0f5589a416fbc77fe>>>
>
> On Fri, Nov 10, 2017 at 7:05 PM, Zhengyu Gu
> <zgu at redhat.com <mailto:zgu at redhat.com>
> <mailto:zgu at redhat.com <mailto:zgu at redhat.com>>
> <mailto:zgu at redhat.com <mailto:zgu at redhat.com>
> <mailto:zgu at redhat.com <mailto:zgu at redhat.com>>>> wrote:
>
> Hi Glyn,
>
>
> Can anyone here tell me how the GC
> memory area
> size is
> determined? If
> there
> is documentation, so much the better
> as we'd
> prefer not
> to depend on code
> details that might flux arbitrarily.
>
>
> GC memory is mainly data structures used by GC
> runtime. It
> can be varied
> by collector used, size of the Java heap, the
> number of GC
> threads and etc.
> and, of course, the application itself.
>
> Some are *fixed* costs, which can be
> estimated.
> E.g. two
> marking bitmaps
> used by G1, each costs 1/64 of heap size
> (assuming
> default
> object
> alignment).
>
> Some are *semi-fixed*, e.g. taskqueue's
> fixed cost
> is about
> 1M for each
> queue on 64-bits VM, but it can overflow.
> And the
> number of
> task queues is
> proportional to the number of GC threads.
>
> Then there are factors from application
> itself, such as
> object mutation
> rate, inter-generation/region references, etc.
>
>
> I don't see a single formula for estimating GC
> memory size.
> If you are
> using G1, the biggest overhead comes from
> 2 bitmaps
> (1/32 *
> heap size).
>
> Thanks,
>
> -Zhengyu
>
>
>
>
>
>
>
>
>
> -- Regards,
> Glyn
>
>
>
>
> --
> Regards,
> Glyn
>
>
>
>
> --
> Regards,
> Glyn
More information about the hotspot-dev
mailing list