G1 native memory consumption

[Milan Mimica]

Hi


I'm inspecting memory consumption issues of a service running on java-8u102, linux. The service is running for a few days now, and in a few days more it would consume all of 32GB physical memory available, and get killed by OOM Killer.

Questions:

- If the code is not allocating any significant off-heap memory, neither by Unsafe.allocateMemory or by external library, isn't 7GB native memory overhead supposed to be enough for a 25GB heap?

- Why so much memory spent on "Internal" category, apparently from G1 thread?


Find the attached jemalloc heap profile, showing "live" allocations that happened in about 30 hour timespan, and a NMT profile of approximately same period.

Profiling was done after some warm-up time time, and with a manually triggered Full GC in between just to give the JVM a chance to clean up everything.


Relevant JVM options:

-Xms25G
-Xmx25G
-Xss256k
-XX:+UseG1GC
-XX:MaxGCPauseMillis=250

-XX:InitiatingHeapOccupancyPercent=80


$ /opt/java/sun-jdk1.8.0-102_no_setcap/bin/jcmd 60028 VM.native_memory summary.diff scale=MB
60028:

Native Memory Tracking:

Total: reserved=30376MB +599MB, committed=29197MB +601MB

-                 Java Heap (reserved=25600MB, committed=25600MB)
                            (mmap: reserved=25600MB, committed=25600MB)

-                     Class (reserved=1168MB, committed=161MB +1MB)
                            (classes #24813 -590)
                            (malloc=4MB #68786 +965)
                            (mmap: reserved=1164MB, committed=157MB +1MB)

-                    Thread (reserved=246MB -6MB, committed=246MB -6MB)
                            (thread #824 -24)
                            (stack: reserved=241MB -6MB, committed=241MB -6MB)
                            (malloc=3MB #4130 -120)
                            (arena=2MB #1646 -48)

-                      Code (reserved=267MB, committed=145MB +1MB)
                            (malloc=24MB #33869 +368)
                            (mmap: reserved=244MB, committed=122MB +1MB)

-                        GC (reserved=1151MB +26MB, committed=1151MB +26MB)
                            (malloc=169MB +26MB #936843 +282881)
                            (mmap: reserved=982MB, committed=982MB)

-                  Compiler (reserved=2MB, committed=2MB)
                            (malloc=2MB #2930 +31)

-                  Internal (reserved=1817MB +571MB, committed=1817MB +571MB)
                            (malloc=1817MB +571MB #906346 +289268)

-                    Symbol (reserved=28MB, committed=28MB)
                            (malloc=24MB #255887 -135)
                            (arena=4MB #1)

-    Native Memory Tracking (reserved=35MB +9MB, committed=35MB +9MB)
                            (tracking overhead=35MB +9MB)

-                   Tracing (reserved=12MB, committed=12MB)
                            (malloc=12MB #57235 -831)

-                   Unknown (reserved=50MB, committed=0MB)
                            (mmap: reserved=50MB, committed=0MB)


Milan Mimica, Software Engineer / Team Leader

[Visit Infobip website]<http://www.infobip.com/?utm_source=Signature&utm_medium=Email&utm_campaign=Company>

[http://www.infobip.com/images/signatures/mwc17.jpg]  <http://www.infobip.com/en/campaigns/mwc-sweepstakes> It's time for Barcelona! Meet us in hall 8.1, stand F49, 27 February-02 March! <http://www.infobip.com/en/campaigns/mwc-sweepstakes>

Office: Mletacka 12/III, 52100 Pula, Croatia  |  Fax: +38552210979  |  Mobile: +385993061692

Email: Milan.Mimica at infobip.com  |  Skype: mmimicaib

www.infobip.com<http://www.infobip.com>   /   GSMA Associate Member   /   Mobey Forum Member
This message is private and confidential. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Infobip d.o.o. If you have received this message in error, please notify us immediately via email to customer.support at infobip.com<mailto:customer.support at infobip.com> or telephone +442032864235.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20170120/c2786444/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: iz_diff_all.svg
Type: image/svg+xml
Size: 55534 bytes
Desc: iz_diff_all.svg
URL: <http://mail.openjdk.java.net/pipermail/hotspot-gc-use/attachments/20170120/c2786444/iz_diff_all-0001.svg>