August 2013 - hotspot-gc-dev

[JDK 6u21]: does CMS report concurrent mode failure on perm gen expansion ?
by Charlie Hunt 05 Dec '18

05 Dec '18

I'm looking at a GC log and surprised by the "concurrent mode failure" messages. I'm seeing the following GC log entries as the first 8 GCs: 3.398: [Full GC 3.398: [CMS (concurrent mode failure)[YG occupancy: 594957 K (1887488 K)]3.489: [weak refs processing, 0.0000140 secs]: 0K->0K(6291456K), 0.1308660 secs] 594957K->594957K(8178944K), [CMS Perm : 21247K->21247K(21248K)], 0.1310260 secs] [Times: user=0.13 sys=0.03, real=0.14 secs] 3.529: [Full GC 3.529: [CMS (concurrent mode failure)[YG occupancy: 594957 K (1887488 K)]3.617: [weak refs processing, 0.0000110 secs]: 0K->0K(6291456K), 0.1010290 secs] 594957K->594957K(8178944K), [CMS Perm : 21248K->21248K(42496K)], 0.1011530 secs] [Times: user=0.12 sys=0.00, real=0.10 secs] 3.638: [Full GC 3.638: [CMS (concurrent mode failure)[YG occupancy: 662071 K (1887488 K)]3.730: [weak refs processing, 0.0000140 secs]: 0K->0K(6291456K), 0.1068710 secs] 662071K->662071K(8178944K), [CMS Perm : 21319K->21319K(42688K)], 0.1069610 secs] [Times: user=0.15 sys=0.00, real=0.11 secs] 3.772: [Full GC 3.772: [CMS (concurrent mode failure)[YG occupancy: 729184 K (1887488 K)]3.915: [weak refs processing, 0.0000140 secs]: 0K->0K(6291456K), 0.1582170 secs] 729184K->729184K(8178944K), [CMS Perm : 21499K->21499K(42880K)], 0.1583500 secs] [Times: user=0.20 sys=0.00, real=0.16 secs] 3.962: [Full GC 3.962: [CMS (concurrent mode failure)[YG occupancy: 804686 K (1887488 K)]4.107: [weak refs processing, 0.0000130 secs]: 0K->0K(6291456K), 0.1598050 secs] 804686K->804686K(8178944K), [CMS Perm : 21702K->21702K(43072K)], 0.1599420 secs] [Times: user=0.21 sys=0.00, real=0.16 secs] 4.497: [Full GC 4.497: [CMS (concurrent mode failure)[YG occupancy: 939150 K (1887488 K)]4.672: [weak refs processing, 0.0000130 secs]: 0K->0K(6291456K), 0.1895350 secs] 939150K->939150K(8178944K), [CMS Perm : 22202K->22202K(43408K)], 0.1896680 secs] [Times: user=0.23 sys=0.00, real=0.19 secs] 5.054: [Full GC 5.054: [CMS (concurrent mode failure)[YG occupancy: 992483 K (1887488 K)]5.274: [weak refs processing, 0.0000130 secs]: 0K->0K(6291456K), 0.2347170 secs] 992483K->992483K(8178944K), [CMS Perm : 24058K->24058K(44408K)], 0.2348700 secs] [Times: user=0.28 sys=0.00, real=0.23 secs] 6.662: [Full GC 6.662: [CMS (concurrent mode failure)[YG occupancy: 1292895 K (1887488 K)]6.953: [weak refs processing, 0.0000140 secs]: 0K->0K(6291456K), 0.3063690 secs] 1292895K->1292895K(8178944K), [CMS Perm : 30692K->30692K(48120K)], 0.3065860 secs] [Times: user=0.66 sys=0.01, real=0.30 secs] After these 8 GCs, the GC log is much inline with what I'd expect, i.e. ParNew GCs, initial-marks, remarks, etc. In each of the above GCs: - young gen is increasing in occupancy - old gen remains empty - perm gen space size is expanding I can understand the Full GC being reported since perm gen is expanding, but I don't understand the reporting of "concurrent mode failure". Am I missing something? Is it possible HotSpot is reporting concurrent mode failure when perm gen is expanding? Here's the set of command line options in use: -Xmx8G -Xms8G -Xmn1500m -XX:+UseCompressedOops -XX:-UseGCOverheadLimit -XX:+UseConcMarkSweepGC -XX:MaxDirectMemorySize=5000m thanks, charlie ... PS: I didn't go look at / investigate OpenJDK HotSpot source code since this is JDK 6u21 and thought it might be possible something may have changed between 6u21 and what's in OpenJDK.

4 4

RFR(S/M): 7132678: G1: verify that the marking bitmaps have no marks for objects over TAMS
by John Cuthbertson 29 Apr '14

29 Apr '14

Hi Everyone, Can I have a couple of volunteers review the changes for this CR? The webrev can be found at: http://cr.openjdk.java.net/~johnc/7132678/webrev.0/ Most of the changes come from a patch that Tony gave me before he left and I had to tweak them slightly to remove a spurious failure. The changes verify that the heap regions don't have any marks between [TAMS, top) at strategic places: start and end of each GC, start and end of remark and cleanup, and when allocating a region. Tony deserves the bulk of the credit so, if possible and there are no objections, I intend to list him as author of the change and include myself as a reviewer. Testing: GC test suite with the both the new flags (separately and together) and a low IHOP value. jprt with the new flags (+IgnoreUnrecognizedVMOptions so that product test runs did not fail). Thanks, JohnC

6 17

Request for review: 6521376: MaxTenuringThreshold and AlwayTenure/NeverTenure consistency
by Tao Mao 15 Nov '13

15 Nov '13

6521376: MaxTenuringThreshold and AlwayTenure/NeverTenure consistency webrev: http://cr.openjdk.java.net/~tamao/6521376/webrev.00/ changeset: For all collectors, (1) Setting -XX:+NeverTenure/-XX:+AlwaysTenure "-XX:+NeverTenure" -> NeverTenure=true; AlwaysTenure=false; MaxTenuringThreshold=markOopDesc::max_age+1 (i.e. 16, use the value instead below); "-XX:+AlwaysTenure" -> NeverTenure=false; AlwaysTenure=true; MaxTenuringThreshold=0; (2) Setting MaxTenuringThreshold (2-a) MaxTenuringThreshold == 0 if (MaxTenuringThreshold == 0): "the cap equals 0" implies AlwaysTenure=true, so set flags accordingly (NeverTenure=false; AlwaysTenure=true). (2-b) MaxTenuringThreshold > 0 But, considering the ergonomics for tenuring threshold (see PSAdaptiveSizePolicy::compute_survivor_space_size_and_threshold(), CMSAdaptiveSizePolicy::compute_survivor_space_size_and_threshold() and ageTable::compute_tenuring_threshold()), we will adjust tenure_threshold up and down but below the cap MaxTenuringThreshold during the application run. Thus, setting MaxTenuringThreshold >= 16 would not necessarily imply that the user wants to apply NeverTenure to the VM for the real tenure_threshold may need to go below MaxTenuringThreshold as the process stats suggest. With that said, if (MaxTenuringThreshold > 0): I simply set NeverTenure/AlwaysTenure to false. (neither NeverTenure nor NeverTenure) and if (MaxTenuringThreshold > 16): set MaxTenuringThreshold to 16 as it is unnecessary to be larger. *(3) -XX:-NeverTenure/-XX:-AlwaysTenure only need to take care of themselves and don't need the flag consistency maintenance here. ******************************************************************************* Take a little convoluted case for example: suppose that we have the following gc options "-XX:+NeverTenure -XX:+MaxTenuringThreshold=18" in this particular order. When we first parse "-XX:+NeverTenure", we would set: NeverTenure=true; AlwaysTenure=false; MaxTenuringThreshold=16. But when we encounter the second option "-XX:+MaxTenuringThreshold=18", we would think the user knows about gc ergonomics for tenuring threshold and wants MaxTenuringThreshold to just be a cap for the ergonomics rather than wants to (implicitly) use NeverTenure. So, we would set NeverTenure=false; AlwaysTenure=false; and finally MaxTenuringThreshold=16 to reflect the cap maximum. *******************************************************************************

4 13

Request for review: 8010506 Typos and errors in gc-related flags in globals.hpp
by Tao Mao 09 Oct '13

09 Oct '13

8010506 Typos and errors in gc-related flags in globals.hpp https://jbs.oracle.com/bugs/browse/JDK-8010506 webrev: http://cr.openjdk.java.net/~tamao/8010506/webrev.00/ Changeset: NewRatio should defined as product(uintx, NewRatio, 2, \ "Ratio of old/new generation sizes") \ instead of product(uintx, NewRatio, 2, \ "Ratio of new/old generation sizes") \ PLUS other typos found by Jesper.

5 17

RFR (M): 7163191 G1: introduce a "heap spanning table" abstraction
by Thomas Schatzl 25 Sep '13

25 Sep '13

Hi all, can I have reviews for the following change? It implements a generic class that allows mapping of subdivision of the Java heap to single elements of an array. It provides a biased pointer for fast access and some convenience functions. Possible applications are mappings from heap addresses to HeapRegion instances (the heap spanning table), the "fast CSet" tables to detect whether a region is in the collection set and others. This change implements the heap spanning table using this new "biased array" instead of manual management/setup of the array. John Cuthbertson already had a stab at it, polishing a change from Tony Printezis. This change is mostly a rewrite, as internal discussion found that moving some functionality of HeapRegionSeq, the watermark, into that generic "biased array" was undesirable. The original mail thread starts here: http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2013-February/005937.… The main change is in the new file G1BiasedArray.hpp, providing the new functionality. Everything else is use of that new data structure (an instance of G1BiasedMappedArray<HeapRegion*>) in the HeapRegionSeq class, and resulting cleanup. bugs.sun.com http://bugs.sun.com/view_bug.do?bug_id=7163191 Webrev: http://cr.openjdk.java.net/~tschatzl/7163191/webrev/ Testing: jprt, dacapo benchmarks Thanks, Thomas

3 8

G1 question: concurrent cleaning of dirty cards
by Doerr, Martin 24 Sep '13

24 Sep '13

Hi all, we have a question about the interaction between G1 post barriers and the refinement thread's concurrent dirty card cleaning. The case in which the G1 post barrier sees a clean card is obviously not problematic, because it will add an entry in a dirty card queue. However, in case in which the Java thread (mutator thread) sees the card already dirtied, it won't enqueue the card again. Which is safe as long as its stored oop (1) is seen and processed (3') by the parallel refinement after having cleaned the card (1'): Java Thread (mutator) Refinement Thread (G1RemSet::concurrentRefineOneCard_impl calls oops_on_card_seq_iterate_careful) (1) store(oop) ( StoreLoad required here ?) (2) load(card==dirty) (1') store(card==clean) (2') StoreLoad barrier (3') load(oop) So the refinement thread seems to rely on getting the oop which was written BEFORE the (2) load(card==dirty) was observed. We wonder how this ordering is guaranteed? There are no StoreLoad barriers in the Java Thread's path. (StoreLoad ordering needs explicit barriers even on TSO platforms.) Kind regards, Martin

5 26

RFR: 7164841: Improvements to the GC log file rotation
by Yumin Qi 16 Sep '13

16 Sep '13

Hi, Can I have your review for this small changes? http://cr.openjdk.java.net/~minqi/7164841/webrev00/ <http://cr.openjdk.java.net/%7Eminqi/7164841/webrev00/> This is for a enhancement to add head/tail message to the logging files to assist reading GC output. 1. modified prompt message if invalid arguments used for log rotating; 2. add time and file name message to log file head/tail. 3. for easily identify which log file is current, use file name like <filename>.n.current, after it reaches maximum size, rename it to <filename>.n On Windows, there is no F_OK (existing test) definition, F_OK is defined as "0" and for _access of VC++, it just describes: modevalue Checks file for 00 Existence only 02 Write-only 04 Read-only 06 Read and write http://msdn.microsoft.com/en-us/library/1w06ktdy.aspx The definition are consistent with unistd.h. Test: JPRT and jtreg. Thanks Yumin

10 43

Request for review: JDK-8009561 NPG: Metaspace fragmentation when retiring a Metachunk
by Mikael Gerdin 10 Sep '13

10 Sep '13

Hi, Can I have some reviews of this small fix to the Metaspace memory allocation path. Problem: When a Metaspace allocation request cannot be satisfied by the current chunk the chunk is retired and a new chunk is requested. This causes whatever is left in the chunk to be effectively leaked. Suggested fix: Put the remaining memory in each chunk on the Metablock freelist so it can be used to satisfy future allocations. Possible addition: When allocating from the block free list, use FreeBlockDictionary<Metablock>::atLeast instead of FreeBlockDictionary<Metablock>::exactly and split the Metablock if it's large enough. One might argue that this increases the fragmentation of the memory on the block free list but I think that we primarily want to use the block free list for small allocations and allocate from chunks for large allocations. Webrev: Only fix: http://cr.openjdk.java.net/~mgerdin/8009561/webrev.0/ Incremental webrev for splitting blocks: http://cr.openjdk.java.net/~mgerdin/8009561/webrev.0%2b/ Bug links: https://jbs.oracle.com/bugs/browse/JDK-8009561 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=8009561 Thanks /Mikael

4 26

Fwd: Request for review (xs) - 8013938: Native OOME on fastdebug VM on Solaris
by Jon Masamitsu 04 Sep '13

04 Sep '13

8013938: Native OOME on fastdebug VM on Solaris This change increases the default value of HeapBaseMinAddress on solaris x64 from 256M to 2G. This changes allows more space on solaris for native memory allocations. 2G is the default value on most other platforms. HeapBaseMinAddress is used in the calculation of the maximum heap size for zero-based compressed oops and the resulting calculated maximum zero-based heap size is reduced by this change. http://cr.openjdk.java.net/~jmasa/8013938/webrev.00/

3 5

RFR: 8020692: TestGCEventMixed.java failed because of timestamp in event after end event
by Erik Helin 04 Sep '13

04 Sep '13

Hi all, this small change ensures that an eventual concurrent mode failure trace event for G1 always is sent before getting a timestamp for the GC trace event. This change can be seen in g1CollectedHeap.cpp. I also updated the code in gcTraceSend.cpp to always explicitly set the endtime with os::elapsed_counter to ensure that the same timer is used for all GC trace events. Webrev: http://cr.openjdk.java.net/~ehelin/8020692/webrev.00/ Bug: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=8020692 Testing: - JPRT - The GC trace tests Thanks, Erik

3 2