November 2019 - zgc-dev

Is ZGC still in experimental?
by Sundara Mohan M 27 Nov '19

27 Nov '19

Hi, Any idea when ZGC will be moved out of experimental flags? Understand it is too early to move it out of experimental but do we have any plan to run it without +UnlockExperimentalVMOptions? Thanks Sundar

3 8

Heap dump is always around 8G on process with 80G heap
by Sundara Mohan M 22 Nov '19

22 Nov '19

Hi, I am trying to take a heap dump of java process with 80G heap with ZGC, this is always giving me around 8G dump file. Same application with ParallelGC running with 48G heap i am getting around 30G dump file. I am using following command on both process and verified both process has same no of request processed and Used memory from gc log is similar jcmd <pid> GC.heap_dump <filename> 1. Why is ZGC heap dump always less compared to process running with ParallelGC? 2. Is there something i am missing? Thanks Sundar

2 2

MMU drops suddenly
by Sundara Mohan M 21 Nov '19

21 Nov '19

Hi, Have noticed following in gc log [2019-11-13T19:24:13.095+0000][69629.984s][info][gc,mmu ] GC(10952) MMU: 2ms/0.0%, 5ms/0.0%, 10ms/23.7%, 20ms/61.9%, 50ms/81.7%, 100ms/90.8% [2019-11-13T20:12:55.339+0000][72552.228s][info][gc,mmu ] GC(11441) MMU: 2ms/0.0%, 5ms/0.0%, 10ms/23.7%, 20ms/61.9%, 50ms/81.7%, *100ms/90.8%* [2019-11-13T21:00:53.415+0000][75430.304s][info][gc,mmu ] GC(11927) MMU: 2ms/0.0%, 5ms/0.0%, 10ms/0.0%, 20ms/0.0%, 50ms/44.2%, *100ms/70.7%* [2019-11-13T21:52:46.244+0000][78543.133s][info][gc,mmu ] GC(12450) MMU: 2ms/0.0%, 5ms/0.0%, 10ms/0.0%, 20ms/0.0%, 50ms/44.2%, 100ms/70.7% [2019-11-13T22:40:35.887+0000][81412.776s][info][gc,mmu ] GC(12946) MMU: 2ms/0.0%, 5ms/0.0%, 10ms/0.0%, 20ms/0.0%, 50ms/44.2%, 100ms/70.7% [2019-11-13T23:27:23.807+0000][84220.696s][info][gc,mmu ] GC(13410) MMU: 2ms/0.0%, 5ms/0.0%, 10ms/0.0%, 20ms/0.0%, 50ms/0.0%, *100ms/43.0%* Was trying to understand what it means and here is my understanding, This says how much minimum CPU available for mutator thread in last Xms 1. Is this correct? 2. Why is this suddenly dropping from (100ms 90% -> 40%) ? Also other time unit it is 0% does that mean my application doesn't get a chance to run? Also i see it never goes back to higher value. 3. Does this measure indicates something good or bad? 3. If this is bad what should i look further to get more insights? Can someone help me to get better understanding on this. TIA, Sundar

2 2

Why does load average on host increases as Allocation Stall happens?
by Sundara Mohan M 19 Nov '19

19 Nov '19

Hi, I have notices Load average on the host increases 5 - 10 times when Allocation Stall happens, trying to understand what causes load average to increase when this happens. Looking at the code zPageAllocator.cpp do { // Start asynchronous GC ZCollectedHeap::heap()->collect(GCCause::_z_allocation_stall); // Wait for allocation to complete or fail page = request.wait(); } while (page == gc_marker); Seems request.wait() is internally doing a get call on ZFuture. 1. Will this use this thread to spin on CPU or it is async (mean this thread will go to sleep and can be woken up when it is ready and other process can occupy this CPU)? 2. Since load average increase matches exactly with allocation stall, is there any other operation (like Flushing page) can cause this behavior? Since i haven't enabled "gc,stats" tag in my logging i missed some information there. Will try to get that information when i can reproduce it. TIA, Sundar

3 3

Upgrade to JDK13 for ZGC?
by Sundara Mohan M 19 Nov '19

19 Nov '19

Hi, I am using ZGC and trying to see if any bug fixes gone in to JDK13 other than "Uncommit memory feature". 1. Was "memory uncommit" is the only feature gone in to JDK13? 2. Is there a way to find all the bug fixed in JDK13 and categorize it by ZGC, so in future i can do it myself. TIA, Sundar

3 4

ZGC Unable to reclaim memory for long time
by Sundara Mohan M 07 Nov '19

07 Nov '19

Hi, I ran into this issue where ZGC is unable to reclaim memory for few hours/days. It just keep printing "Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space" and Allocation Stall happening on that thread. Here is the metrics which shows for some reason even though there is Garbage but it is unable to Reclaim .... [2019-11-04T*08:39:53.986+0000*][1765465.981s][info][gc,heap ] GC(112126) Live: - 6366M (78%) 6366M (78%) 6366M (78%) - - *[2019-11-04T08:39:53.986+0000][1765465.981s][info][gc,heap ] GC(112126) Garbage: - 1735M (21%) 1735M (21%) 1731M (21%)* - - [2019-11-04T08:39:53.986+0000][1765465.981s][info][gc,heap ] GC(112126) Reclaimed: - - 0M (0%) 4M (0%) ... [2019-11-04T16:48:53.742+0000][1794805.738s][info][gc,heap ] GC(135520) Live: - 6367M (78%) 6367M (78%) 6367M (78%) - - *[2019-11-04T16:48:53.742+0000][1794805.738s][info][gc,heap ] GC(135520) Garbage: - 1730M (21%) 1730M (21%) 1724M (21%)* - - [2019-11-04T16:48:53.742+0000][1794805.738s][info][gc,heap ] GC(135520) Reclaimed: - - 0M (0%) 6M (0%) Here it was in this state for ~8hours and it is still happening. It says has a Garbage of 21G but it is not able to Reclaim it everytime it reclaims only 4-6M. Any idea what might be the issue here. TIA Sundar

5 14