Resident/Shared memory size is showing 3 times the given heap size

[Sundara Mohan M]

Thanks Per and Alex.

This helped a lot on understanding ZGC and process memory consumption
better.

Regards
Sundar

On Fri, Sep 20, 2019 at 9:54 AM Alex Elent <alex at scalyr.com> wrote:

> Hi Sundara,
>
> I had a similar question in January
> http://mail.openjdk.java.net/pipermail/zgc-dev/2019-January/000570.html.
>
> @Erik Österlund <erik.osterlund at oracle.com> recommended Native Memory
> Tracking (NMT) and I've been using it with success.
>
> On Fri, Sep 20, 2019 at 12:44 AM Per Liden <per.liden at oracle.com> wrote:
>
>> Hi,
>>
>> On 9/20/19 3:54 AM, Sundara Mohan M wrote:
>> > Hi,
>> >     We are running our server with ZGC and seeing the resident memory
>> size
>> > is approximately 3 times the given heap size. I am a newbie still
>> trying to
>> > understand basic concepts. Can someone help me understand this better
>> >
>> > 1. Is this expected?
>>
>> Yes
>>
>> > 2. At high level i know ZGC is having multiple view of memory region.
>> Can
>> > you explain how this can happen?
>>
>> Exactly. RSS (Resident set size) basically just says how much memory the
>> process has mapped at the moment. It doesn't take into account that some
>> of these mappings might be backed by the same memory, or might be are
>> shared with other process, etc. A more interesting number, to get a
>> better view of how much memory a process is actually using is PSS
>> (Proportional set size). There are various tools that display PSS (smem,
>> procrank, ps_mem.py, etc), but the raw data is available in
>> /proc/<pid>/smaps_rollup.
>>
>> For example, a JVM running ZGC with a 4G heap looks like this:
>>
>> $ cat /proc/4509/smaps_rollup
>> 00400000-7ffd472e2000 ---p 00000000 00:00 0
>> [rollup]
>> Rss:            12843604 kB
>> Pss:             4451825 kB
>> Shared_Clean:       3232 kB
>> Shared_Dirty:   12582912 kB
>> Private_Clean:     14380 kB
>> Private_Dirty:    243080 kB
>> Referenced:     12843604 kB
>> Anonymous:        243048 kB
>> LazyFree:              0 kB
>> AnonHugePages:         0 kB
>> ShmemPmdMapped:        0 kB
>> Shared_Hugetlb:        0 kB
>> Private_Hugetlb:       0 kB
>> Swap:                  0 kB
>> SwapPss:               0 kB
>> Locked:                0 kB
>>
>> Here the multi-mapping causes the RSS to be inflated by 3x, while PSS
>> shows a more accurate number reflecting the fact that the three
>> different heap mappings are backed by the same memory.
>>
>> > 3. Also i see something like this in /proc/<pid>/maps
>> >
>> > ...
>> > 13fff9a00000-13fff9c00000 rw-s 5f8e00000 00:12 1488924473
>> >   /mnt/tmpfs/java_heap.10087 (deleted)
>> > 13fff9c00000-13fff9e00000 rw-s 5fb000000 00:12 1488924473
>> >   /mnt/tmpfs/java_heap.10087 (deleted)
>> > 13fff9e00000-13fffa000000 rw-s 603200000 00:12 1488924473
>> >   /mnt/tmpfs/java_heap.10087 (deleted)
>> > 13fffa000000-13fffa200000 rw-s 612c00000 00:12 1488924473
>> >   /mnt/tmpfs/java_heap.10087 (deleted)
>> > 13fffa200000-13fffa400000 rw-s 61a800000 00:12 1488924473
>> >   /mnt/tmpfs/java_heap.10087 (deleted)
>> > 13fffa400000-13fffa600000 rw-s 326000000 00:12 1488924473
>> >   /mnt/tmpfs/java_heap.10087 (deleted)
>> > 13fffa600000-13fffa800000 rw-s 5d3600000 00:12 1488924473
>> >   /mnt/tmpfs/java_heap.10087 (deleted)
>> > ...
>> > Why is this still in deleted state?
>>
>> The "(deleted)" state just means that the file has been unlinked. I.e.
>> the file is still in use, but the directory entry
>> /mnt/tmpfs/java_heap.10087 has been deleted (as it should be).
>>
>> > 4. Trying to get heap dump with following command
>> >   jmap -heap <pid> (tried with same user as well root but it is not
>> printing
>> > heap)
>> > Is there something changed regarding dumping heap with ZGC?
>>
>> IIRC, jmap -heap was removed in JDK 9, use jcmd instead:
>>
>> $ jcmd <pid> GC.heap_dump <filename>
>>
>> cheers,
>> Per
>>
>> >
>> > TIA
>> > Sundar
>> >
>>
>