November 2018 - hotspot-gc-dev

RFR 8204947: Port ShenandoahTaskTerminator to mainline and make it default
by Zhengyu Gu 08 Dec '18

08 Dec '18

Hi, ShenandoahTaskTerminator is based on Google's work stealing protocol, described in paper [1]. The implementation diverges in a way to emphasize termination latency vs. power efficiency (see comments in code for details). Shenandoah switched to ShenandoahTaskTerminator as default terminator, shortly after it was implemented, and has been very stable since. On performance side, it showed better termination latency on almost all tests/benchmarks I ran, comparing to ParallelTaskTerminator. As the result, we (Shenandoah) purged fallback branches and uses it as the sole task terminator. This patch introduces a new experimental flag "UseOWSTTaskTerminator" (default: true), and switches G1 and Parallel GC to use it as default terminator. You can switch back to ParallelTaskTerminator with JVM option "-XX:+UnlockExperimentalVMOptions -XX:-UseOWSTTaskTerminator" I did not port it to CMS, as it is depreciated. Bug: https://bugs.openjdk.java.net/browse/JDK-8204947 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8204947/webrev.00/ Test: hotspot_gc and vmTestbase_vm_gc on Linux 64. [1] Understanding and improving JVM GC work stealing at the data center scale (http://dl.acm.org/citation.cfm?id=2926706) Thanks, -Zhengyu

8 36

RFR (S/M): 8159440: Move marking of promoted objects during initial mark into the concurrent phas
by Thomas Schatzl 06 Dec '18

06 Dec '18

Hi all, can I have reviews for this cleanup change that unifies how we handle "roots" from survivor and old gen regions? So at the moment, after the concurrent start pause, we have a "scan root regions phase" that walks through the objects in the survivor regions to find roots for the marking. This is needed because the ntams of survivor regions is bottom, i.e. the objects are above so that marking will not otherwise come across these. For promoted objects we have a different mechanism: we mark those objects on the next bitmap, and set ntams to the end of the promoted objects, i.e. the scan of the marking will automatically follow their references. This special handling has several drawbacks: - additional special handling of old gen regions manually adjusting the nTAMS - the promoted objects need to be marked in the pause. This change makes handling of the promoted objects (from ntams to top) equal to survivor regions. For this reason we extend the existing G1RootRegions data structure to remember the regions (both survivor and promoted-into old gen regions). That saves quite a bit of code. CR: https://bugs.openjdk.java.net/browse/JDK-8159440 Webrev: http://cr.openjdk.java.net/~tschatzl/8159440/webrev/ Testing: hs-tier 1-5 Thanks, Thomas

3 7

[JDK 6u21]: does CMS report concurrent mode failure on perm gen expansion ?
by Charlie Hunt 05 Dec '18

05 Dec '18

I'm looking at a GC log and surprised by the "concurrent mode failure" messages. I'm seeing the following GC log entries as the first 8 GCs: 3.398: [Full GC 3.398: [CMS (concurrent mode failure)[YG occupancy: 594957 K (1887488 K)]3.489: [weak refs processing, 0.0000140 secs]: 0K->0K(6291456K), 0.1308660 secs] 594957K->594957K(8178944K), [CMS Perm : 21247K->21247K(21248K)], 0.1310260 secs] [Times: user=0.13 sys=0.03, real=0.14 secs] 3.529: [Full GC 3.529: [CMS (concurrent mode failure)[YG occupancy: 594957 K (1887488 K)]3.617: [weak refs processing, 0.0000110 secs]: 0K->0K(6291456K), 0.1010290 secs] 594957K->594957K(8178944K), [CMS Perm : 21248K->21248K(42496K)], 0.1011530 secs] [Times: user=0.12 sys=0.00, real=0.10 secs] 3.638: [Full GC 3.638: [CMS (concurrent mode failure)[YG occupancy: 662071 K (1887488 K)]3.730: [weak refs processing, 0.0000140 secs]: 0K->0K(6291456K), 0.1068710 secs] 662071K->662071K(8178944K), [CMS Perm : 21319K->21319K(42688K)], 0.1069610 secs] [Times: user=0.15 sys=0.00, real=0.11 secs] 3.772: [Full GC 3.772: [CMS (concurrent mode failure)[YG occupancy: 729184 K (1887488 K)]3.915: [weak refs processing, 0.0000140 secs]: 0K->0K(6291456K), 0.1582170 secs] 729184K->729184K(8178944K), [CMS Perm : 21499K->21499K(42880K)], 0.1583500 secs] [Times: user=0.20 sys=0.00, real=0.16 secs] 3.962: [Full GC 3.962: [CMS (concurrent mode failure)[YG occupancy: 804686 K (1887488 K)]4.107: [weak refs processing, 0.0000130 secs]: 0K->0K(6291456K), 0.1598050 secs] 804686K->804686K(8178944K), [CMS Perm : 21702K->21702K(43072K)], 0.1599420 secs] [Times: user=0.21 sys=0.00, real=0.16 secs] 4.497: [Full GC 4.497: [CMS (concurrent mode failure)[YG occupancy: 939150 K (1887488 K)]4.672: [weak refs processing, 0.0000130 secs]: 0K->0K(6291456K), 0.1895350 secs] 939150K->939150K(8178944K), [CMS Perm : 22202K->22202K(43408K)], 0.1896680 secs] [Times: user=0.23 sys=0.00, real=0.19 secs] 5.054: [Full GC 5.054: [CMS (concurrent mode failure)[YG occupancy: 992483 K (1887488 K)]5.274: [weak refs processing, 0.0000130 secs]: 0K->0K(6291456K), 0.2347170 secs] 992483K->992483K(8178944K), [CMS Perm : 24058K->24058K(44408K)], 0.2348700 secs] [Times: user=0.28 sys=0.00, real=0.23 secs] 6.662: [Full GC 6.662: [CMS (concurrent mode failure)[YG occupancy: 1292895 K (1887488 K)]6.953: [weak refs processing, 0.0000140 secs]: 0K->0K(6291456K), 0.3063690 secs] 1292895K->1292895K(8178944K), [CMS Perm : 30692K->30692K(48120K)], 0.3065860 secs] [Times: user=0.66 sys=0.01, real=0.30 secs] After these 8 GCs, the GC log is much inline with what I'd expect, i.e. ParNew GCs, initial-marks, remarks, etc. In each of the above GCs: - young gen is increasing in occupancy - old gen remains empty - perm gen space size is expanding I can understand the Full GC being reported since perm gen is expanding, but I don't understand the reporting of "concurrent mode failure". Am I missing something? Is it possible HotSpot is reporting concurrent mode failure when perm gen is expanding? Here's the set of command line options in use: -Xmx8G -Xms8G -Xmn1500m -XX:+UseCompressedOops -XX:-UseGCOverheadLimit -XX:+UseConcMarkSweepGC -XX:MaxDirectMemorySize=5000m thanks, charlie ... PS: I didn't go look at / investigate OpenJDK HotSpot source code since this is JDK 6u21 and thought it might be possible something may have changed between 6u21 and what's in OpenJDK.

4 4

Request for CSR review: 8214349: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector
by Roman Kennke 04 Dec '18

04 Dec '18

I need at least one GC expert review the CSR regarding command line options for Shenandoah GC. Also, for some reason, I cannot seem to comment on the CSR, so here's my comments, it might be relevant to Reviewers. ========================================================================== > Was a shorter option prefer than "Shenandoah" considered? Yes. We (shenandoah devs) do use shorthands like ShHeapRegionSize or even ShHRSize when communicating internally, but we know what it means. It seems rather obscure though. Shenandoah prefix seems clearer. > For naming something like "ShenandoahSATBBarrier" please consider > using "ShenandoahSatbBarrier" instead. The JDK API is not consistent > on this point, but camel-casing acronyms in API declarations is now > preferred. Hmm, ok. Will look into changing it. > Do ShenandoahOptimizeInstanceFinals and related options enable > optimizations which violate the spec? At the very least, it taps undefined behavior: when modifying a final field via reflection or JNI or other means, the optimization might lead to see stale values. There is one case where we think it would violate the JMM though: if initializer initializes final field, and the object somehow escapes it, and the field is accessed before initializer is finished, it might see a stale (uninitialized) value, which it must not do. How should we deal with it? Document the potential danger? Remove the flags altogether? > Are there general memory management options which Shenandoah pays > attention to or ignores? We use ParallelGCThreads, ConcGCThreads, ClassUnloading, ClassUnloadingWithConcurrentMark and probably some more. Do you want me to dig them up and document somewhere? (Where?) > Before the request is finalized for the second phase of CSR review, please have at least one GC engineer review the CSR. Ok, I will ping hotspot-gc-dev. Thanks! Roman ==========================================================================

4 7

RFR: 8214377: ZGC: Don't use memset to initialize array of ZForwardingTableEntry
by Per Liden 04 Dec '18

04 Dec '18

Use placement new instead of memset() to avoid compiler warning/error when using GCC 8 with -Werror=class-memaccess (memset to initialize non-trivial object). To avoid any undefined behavior I also added a loop to destruct the objects properly, but this loop will be compiled down to nothing. I ran SPECjbb2015 to see if the execution time for "Prepare Relocation Set" was noticeably affected by this change (assuming memset does a batter job than then loop), but I can't see any real difference. I'll run a few more iterations over night just to make sure. Bug: https://bugs.openjdk.java.net/browse/JDK-8214377 Webrev: http://cr.openjdk.java.net/~pliden/8214377/webrev.0 /Per

4 6

RFR: 8213890: Implementation of JEP 344: Abortable Mixed Collections for G1
by Stefan Johansson 03 Dec '18

03 Dec '18

Hi, Please review this patch for "JEP 344: Abortable Mixed Collections for G1", the JEP is not yet targeted but the goal is to get it into JDK 12. JEP: https://bugs.openjdk.java.net/browse/JDK-8190269 Issue: https://bugs.openjdk.java.net/browse/JDK-8213890 Webrev: http://cr.openjdk.java.net/~sjohanss/8213890/00 Summary ------- When G1 has performed a concurrent cycle old regions are reclaimed in Mixed collections. Before this change the number of old regions in a Mixed collection was chosen in the beginning of the pause and could not be altered later on. With abortable mixed collection we can add a set of optional regions that can be collected if there is time left without breaking the pause target. Having optional regions come with a small overhead and the current implementation is pretty restrictive and aims at changing the default behavior of G1 as little as possible. That means that the some old regions will still be added to the normal collection set and will not be optional, this also ensures that the Mixed collections actually reclaims a number of old regions and that way avoids Full collections. More information can be found in the JEP itself. Testing ------- A lot of local and remote functional testing and performance testing has been done. The performance testing has aimed at ensuring that there is no regression in normal collections and that the impact to Mixed collections when optional regions are not used is as small as possible. There has also been extra testing made where additional code was added to force optional regions to be chosen much more eagerly. This way the new optional region code has been exercised more and been better tested. Thanks, Stefan

3 14

RFR: 8214484: ZGC: Exclude SA tests ClhsdbJhisto and TestHeapDumpFor*
by Per Liden 03 Dec '18

03 Dec '18

There's no support for SA heap walking when using ZGC. Hence the tests for this should not execute when ZGC is enabled. Bug: https://bugs.openjdk.java.net/browse/JDK-8214484 Webrev: http://cr.openjdk.java.net/~pliden/8214484/webrev.0 /Per

4 9

RFR(S) 8214388 CDS dumping fails with java heap fragmentation
by Ioi Lam 29 Nov '18

29 Nov '18

http://cr.openjdk.java.net/~iklam/jdk12/8214388-dumptime-fragmentation.v01/ https://bugs.openjdk.java.net/browse/JDK-8214388 Symptom: ======== "java -Xshare:dump" would intermittently fail with Unable to write archive heap ... due to fragmentation. This usually happens when you try to dump many classes (e.g. 10000) with a relatively small heap (e.g., 1g) with a lot of GC threads (e.g., 24). (Example use case -- Eclipse IDE loads 15,000 classes with 512MB heap.) When GC happens during class loading, some old G1 regions may be placed at the top end of the heap (due to large number of GC threads). Later, when writing the archived heap, G1 tries to allocate contiguous regions from the top end of the heap. This would fail due to the presence of those old regions. Fix: ==== As suggested by Stefan Johansson, we run a full GC with a single GC thread. This guarantees that all old blocks will be moved to the bottom end of the heap. Because there's no API for specifying the number of GC threads dynamically, and CDS dump time doesn't allocates lots of objects, I have statically forced the number of threads to 1 in AdaptiveSizePolicy::calc_active_workers during CDS dump time. (This seems like a more direct way than assigning ParallelGCThreads ...) Notes: ====== 1. Humongous regions cannot move. However, currently we don't do humongous allocations during CDS dump, so we should be fine. I have added diagnostics warnings so if fragmentation does happen in the future, the user can find out why. 2. Fixed a minor bug in HeapShared::check_closed_archive_heap_region_object_class 3. Fixed a bug in MetaspaceShared::read_extra_data, where the symbol/strings would be lost due to GC. 4. Added stress test to successfully archive about 18MB of objects with -Xmx64m. This used to fail even with -Xmx512m on a Solaris box. 5. With default CDS archive generation during JDK build time, -Xmx128m is used. Before this fix, the EDEN region lives at the top of the heap during CDS dump time, and we end up with a 2MB gap between the archive regions and the top of the heap. Because the archive regions cannot move, at run time, using CDS would reduce the max humongous allocation by 2MB. With this fix, the archive regions are now placed at the very top of the heap, so the gap no longer exists. Tests: ====== Running hs-tiers{1-6} for sanity. Thanks - Ioi

1 0

Re: RFR: 8214484: ZGC: Exclude SA tests ClhsdbJhisto and TestHeapDumpFor*
by Per Liden 29 Nov '18

29 Nov '18

Hi Chris, On 2018-11-29 20:04, Chris Plummer wrote: > Hi Per, > > I think there may have been some discussion about this before. Are there > plans to eventually get SA heap walking support for ZGC, and is the list > of disabled tests being maintained somewhere? The latest directive I've received is that we should not spend any more time or resources on adding ZGC support to the SA. I personally agree with that. I think the benefits provided by the SA is dwarfed by the cost to maintain it, so we should phase it out. You should be able to find the excluded tests by grepping for 'vm.gc != "Z"' among the SA tests. cheers, Per > > thanks, > > Chris > > On 11/29/18 7:07 AM, Per Liden wrote: >> There's no support for SA heap walking when using ZGC. Hence the tests >> for this should not execute when ZGC is enabled. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214484 >> Webrev: http://cr.openjdk.java.net/~pliden/8214484/webrev.0 >> >> /Per > >

2 1

Re: RFR: 8214484: ZGC: Exclude SA tests ClhsdbJhisto and TestHeapDumpFor*
by Roman Kennke 29 Nov '18

29 Nov '18

Hi Chris, Per, Shenandoah has the same problem, for the same reasons. And I'm not convinced that G1 is safe either, at least with -XX:+ClassUnloadingWithConcurrentMark. In my opinion, this whole code section needs to be rewritten to use something like CollectedHeap::object_iterate() (which is used by other similar external code like JVMTI for heapdumping/walking). Simply assuming a certain object layout is not very sustainable IMO. Roman > Hi Per, > > I think there may have been some discussion about this before. Are there > plans to eventually get SA heap walking support for ZGC, and is the list > of disabled tests being maintained somewhere? > > thanks, > > Chris > > On 11/29/18 7:07 AM, Per Liden wrote: >> There's no support for SA heap walking when using ZGC. Hence the tests >> for this should not execute when ZGC is enabled. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8214484 >> Webrev: http://cr.openjdk.java.net/~pliden/8214484/webrev.0 >> >> /Per > >

1 0