From ksakata at openjdk.java.net Thu Apr 1 02:34:23 2021 From: ksakata at openjdk.java.net (Koichi Sakata) Date: Thu, 1 Apr 2021 02:34:23 GMT Subject: RFR: 8176026: SA: Huge heap sizes cause a negative value to be displayed in the jhisto heap total [v2] In-Reply-To: References: <-OxCoBAqyBOqOS5DP3C4NPkq8PFLwwCL6wBXmiqeEzU=.f62e8e0f-418e-4129-a91a-c10e1571466e@github.com> Message-ID: On Fri, 26 Mar 2021 13:57:00 GMT, Koichi Sakata wrote: >> Marked as reviewed by mgkwill at github.com (no known OpenJDK username). > > I just updated the copyright. Could someone sponsor this pull request? It is ready. ------------- PR: https://git.openjdk.java.net/jdk/pull/3087 From ysuenaga at openjdk.java.net Thu Apr 1 04:09:25 2021 From: ysuenaga at openjdk.java.net (Yasumasa Suenaga) Date: Thu, 1 Apr 2021 04:09:25 GMT Subject: RFR: 8176026: SA: Huge heap sizes cause a negative value to be displayed in the jhisto heap total [v3] In-Reply-To: <2nwnR_h9DuOC_2rdT9z7j3ickPchjK9ozWv9obWCP_M=.018c90ea-1fc8-4ee5-b1a6-6b1a65a308e4@github.com> References: <2nwnR_h9DuOC_2rdT9z7j3ickPchjK9ozWv9obWCP_M=.018c90ea-1fc8-4ee5-b1a6-6b1a65a308e4@github.com> Message-ID: On Fri, 26 Mar 2021 13:51:40 GMT, Koichi Sakata wrote: >> When a heap is used more than about 2.1GB, clhsdb jhisto shows a negative number in the total field. >> >> $ java -Xmx20g Sample >> >> $ jhsdb clhsdb --pid 5773 >> Attaching to process 5773, please wait... >> hsdb> jhisto >> ... >> 299: 1 16 jdk.internal.misc.Unsafe >> 300: 3402 10737610256 byte[] >> Total : 15823 -2146661280 >> Heap traversal took 1.793 seconds. >> (Incidentally, the Sample is a program that only allocates many objects.) >> >> #### Details >> This is because in ObjectHistogram class the totalSize variable is int type. >> >> The total size is the total of ObjectHistogramElement#getSize() and getSize() returns long. So I changed int to long in the ObjectHistogram class. >> >> Additionally, I changed the type of the totalCount. This doesn't cause a bug, but ObjectHistogramElement#getCount() also returns long. So it doesn't need to treat it as int, I think. ObjectHistogramElement#compare() also do the same thing, so a class that is a huge size show the bottom of the list. >> >> #### Tests >> The jtreg test was successful. >> $ sudo make run-test TEST=serviceability/sa/ClhsdbJhisto.java >> >> $ cat build/linux-x86_64-server-fastdebug/test-results/jtreg_test_hotspot_jtreg_serviceability_sa_ClhsdbJhisto_java/text/summary.txt >> serviceability/sa/ClhsdbJhisto.java Passed. Execution successful >> >> I confirmed the output with the same program. >> >> $ ./jdk/build/linux-x86_64-server-fastdebug/jdk/bin/java -Xmx20g Sample >> $ ./jdk/build/linux-x86_64-server-fastdebug/jdk/bin/jhsdb clhsdb --pid 10196 >> Attaching to process 10196, please wait... >> hsdb> jhisto >> Object Histogram: >> >> num #instances #bytes Class description >> -------------------------------------------------------------------------- >> 1: 3405 13958838400 byte[] >> 2: 887 109032 java.lang.Class >> ... >> 300: 1 16 jdk.internal.misc.Unsafe >> Total : 15827 13959470288 >> Heap traversal took 1.72 seconds. > > Koichi Sakata has updated the pull request incrementally with one additional commit since the last revision: > > Update copyright Looks good. I will sponsor you. ------------- Marked as reviewed by ysuenaga (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3087 From ksakata at openjdk.java.net Thu Apr 1 04:13:30 2021 From: ksakata at openjdk.java.net (Koichi Sakata) Date: Thu, 1 Apr 2021 04:13:30 GMT Subject: Integrated: 8176026: SA: Huge heap sizes cause a negative value to be displayed in the jhisto heap total In-Reply-To: References: Message-ID: On Fri, 19 Mar 2021 11:49:39 GMT, Koichi Sakata wrote: > When a heap is used more than about 2.1GB, clhsdb jhisto shows a negative number in the total field. > > $ java -Xmx20g Sample > > $ jhsdb clhsdb --pid 5773 > Attaching to process 5773, please wait... > hsdb> jhisto > ... > 299: 1 16 jdk.internal.misc.Unsafe > 300: 3402 10737610256 byte[] > Total : 15823 -2146661280 > Heap traversal took 1.793 seconds. > (Incidentally, the Sample is a program that only allocates many objects.) > > #### Details > This is because in ObjectHistogram class the totalSize variable is int type. > > The total size is the total of ObjectHistogramElement#getSize() and getSize() returns long. So I changed int to long in the ObjectHistogram class. > > Additionally, I changed the type of the totalCount. This doesn't cause a bug, but ObjectHistogramElement#getCount() also returns long. So it doesn't need to treat it as int, I think. ObjectHistogramElement#compare() also do the same thing, so a class that is a huge size show the bottom of the list. > > #### Tests > The jtreg test was successful. > $ sudo make run-test TEST=serviceability/sa/ClhsdbJhisto.java > > $ cat build/linux-x86_64-server-fastdebug/test-results/jtreg_test_hotspot_jtreg_serviceability_sa_ClhsdbJhisto_java/text/summary.txt > serviceability/sa/ClhsdbJhisto.java Passed. Execution successful > > I confirmed the output with the same program. > > $ ./jdk/build/linux-x86_64-server-fastdebug/jdk/bin/java -Xmx20g Sample > $ ./jdk/build/linux-x86_64-server-fastdebug/jdk/bin/jhsdb clhsdb --pid 10196 > Attaching to process 10196, please wait... > hsdb> jhisto > Object Histogram: > > num #instances #bytes Class description > -------------------------------------------------------------------------- > 1: 3405 13958838400 byte[] > 2: 887 109032 java.lang.Class > ... > 300: 1 16 jdk.internal.misc.Unsafe > Total : 15827 13959470288 > Heap traversal took 1.72 seconds. This pull request has now been integrated. Changeset: 39f0b27a Author: Koichi Sakata Committer: Yasumasa Suenaga URL: https://git.openjdk.java.net/jdk/commit/39f0b27a Stats: 5 lines in 2 files changed: 0 ins; 0 del; 5 mod 8176026: SA: Huge heap sizes cause a negative value to be displayed in the jhisto heap total Reviewed-by: cjplummer, kevinw, ysuenaga ------------- PR: https://git.openjdk.java.net/jdk/pull/3087 From shade at openjdk.java.net Thu Apr 1 07:22:25 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 1 Apr 2021 07:22:25 GMT Subject: RFR: 8264484: Replace uses of StringBuffer with StringBuilder in jdk.hotspot.agent [v3] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 23:47:41 GMT, Yasumasa Suenaga wrote: >> Andrey Turbanov has updated the pull request incrementally with one additional commit since the last revision: >> >> 8264484: Replace uses of StringBuffer with StringBuilder in jdk.hotspot.agent >> >> fix indentation > > Looks good! > BTW have you found sponsor? I can sponsor you if you need. @YaSuenag, you can just sponsor without asking, I think. ------------- PR: https://git.openjdk.java.net/jdk/pull/3275 From github.com+741251+turbanoff at openjdk.java.net Thu Apr 1 07:27:08 2021 From: github.com+741251+turbanoff at openjdk.java.net (Andrey Turbanov) Date: Thu, 1 Apr 2021 07:27:08 GMT Subject: Integrated: 8264484: Replace uses of StringBuffer with StringBuilder in jdk.hotspot.agent In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 18:58:24 GMT, Andrey Turbanov wrote: > Found by IntelliJ IDEA inspection `Java | Java language level migration aids | Java 5 | 'StringBuffer' may be 'StringBuilder'` > As suggested in https://github.com/openjdk/jdk/pull/1507#issuecomment-757369003 I've created separate PR for module `jdk.hotspot.agent` > Similar cleanups in the past: https://bugs.openjdk.java.net/browse/JDK-8041679 https://bugs.openjdk.java.net/browse/JDK-8264029 This pull request has now been integrated. Changeset: 6cf10950 Author: Andrey Turbanov Committer: Yasumasa Suenaga URL: https://git.openjdk.java.net/jdk/commit/6cf10950 Stats: 116 lines in 37 files changed: 0 ins; 7 del; 109 mod 8264484: Replace uses of StringBuffer with StringBuilder in jdk.hotspot.agent Reviewed-by: kevinw, amenkov, ysuenaga ------------- PR: https://git.openjdk.java.net/jdk/pull/3275 From pliden at openjdk.java.net Thu Apr 1 07:59:18 2021 From: pliden at openjdk.java.net (Per Liden) Date: Thu, 1 Apr 2021 07:59:18 GMT Subject: RFR: 8260267: vmTestbase/gc/gctests/FinalizeLock/FinalizeLock.java fails with fatal error: Mark stack space exhausted In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 10:58:56 GMT, ?? wrote: > ZGC mark stack will be exhausted when the live object graph is very large, which due to the pseudo-BFS style visiting of the object graph (ZGC gc worker will push children to other stripes which behave like BFS, so the maximum mark stack size is O(num_objects).). But other gc like g1 or Shenandoah use DFS to visiting the object graph, and the mark stack usage merely exceeds 32M, which may due to the maximum mark stack size is related to the depth of the object graph O(depth_of_object_graph). > > The following is the test case used to reproduce the crash, > import java.util.HashMap; > import java.util.HashSet; > > class Test2 { > public static int NODE_COUNT = 25000000; > public static int NODE_COUNT_UB = 1 << 25; > public static int STRING_GEN_INDEX = 1; > public static int WATCHER_COUNT = 100; > public static int MAP_COUNT = 8; > public static int BIN_COUNT = 16; > public static Object[] watchTable = new Object[MAP_COUNT]; > > public static class Watcher { > public static long counter = 0; > public long index; > Watcher() { > index = counter; > counter++; > } > } > > public static int gen_name() { > int cur = STRING_GEN_INDEX++; > int reminder = cur % BIN_COUNT; > cur = cur - reminder + NODE_COUNT_UB * reminder; > return cur; > } > > public static HashMap> get_map(int index) { > return (HashMap>)watchTable[index]; > } > > public static void gen_watcher() { > int name = gen_name(); > HashSet set = new HashSet(); > for (int i = 0; i < WATCHER_COUNT; i++) { > set.add(new Watcher()); > } > for (int i = 0; i < MAP_COUNT; i++) { > get_map(i).put(name, set); > } > } > > public static void setup() { > for (int i = 0; i < MAP_COUNT; i++) { > watchTable[i] = new HashMap>(); > } > for (int i = 0; i < NODE_COUNT; i++) { > gen_watcher(); > } > } > > public static void main(String[] args) { > setup(); > for (int i = 0; i < 10; i++) { > System.gc(); > } > System.out.println("pass"); > } > } > > The command to run the testcase is, > java -XX:+UseZGC -Xmx280g -XX:ZCollectionInterval=10 -Xlog:gc*=debug:file=gc.log:time,level,tags:filesize=5g Test2 As you know, there's an ongoing discussion what a proper fix for this could look like here https://bugs.openjdk.java.net/browse/JDK-8260267, please participate in that discussion. I tested the reproducer provided in this PR, and it seems like https://github.com/openjdk/jdk/compare/master...pliden:reduce_mark_stack_usage_v3 handles it better than the patch proposed here. The patch proposed here seems to roughly doubles the marking time for the provided reproducer. I've updated the patch I proposed, to also take mark stack usage into account. Please see https://bugs.openjdk.java.net/browse/JDK-8260267?focusedCommentId=14410719&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14410719 ------------- PR: https://git.openjdk.java.net/jdk/pull/3262 From mli at openjdk.java.net Thu Apr 1 14:43:42 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Thu, 1 Apr 2021 14:43:42 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v13] In-Reply-To: References: Message-ID: > Summary > ----------- > > Improve G1 Full GC by skip compaction for regions with high survival ratio. > > Backgroud > ----------- > > There are 4 steps in full gc of G1 GC. > - mark live objects > - prepare forwardee > - adjust pointers > - compact > > When full gc occurs, there may be very high percentage of live bytes in some regions. For these regions, it's not efficient to compact them and better to skip them, as there are little space to save but many objects to copy. > > Description > ----------- > > We enhance the full gc implementation for the above situation through following steps: > - accumulate live bytes of every hr in mark phase; (already done by JDK-8263495) > - skip adding regions with high survial ratio, and set the region with high survival ratio as pinned in _region_attr_table during prepare phase; > - nothing special is done in adjust phase, regions with high survial ratio are skipped because of pin setting in the above step; > - nothing special is done in compact phase, regions with high survival ratio are skipped because these regions are skipped when adding regions to compaction set in the prepare phase; > > VM options related > ----------- > > - MarkSweepDeadRatio: we reuse this exising vm option to indicate the high survial ratio threhold (100-MarkSweepDeadRatio) in G1. > - default value of MarkSweepDeadRatio: 5 > > Test > ----------- > > - specjbb2015: no regression > - dacapo: (Attachment is the dacapo h2 full gc pause.) > - 95% of full gc pauses: 10%-19% improvement. > - 5% of full gc pauses: 1.2% improvement. > - 0.1% of full gc pauses: -6.16% improvement. > > $ java -Xmx1g -Xms1g -XX:ParallelGCThreads=4 -Xlog:gc*=info:file=gc.log -jar dacapo-9.12-bach.jar --iterations 5 --size huge --no-pre-iteration-gc h2 Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: minor code improvement. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/2760/files - new: https://git.openjdk.java.net/jdk/pull/2760/files/1f4ead84..63ab0f9c Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2760&range=12 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2760&range=11-12 Stats: 4 lines in 1 file changed: 0 ins; 3 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/2760.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2760/head:pull/2760 PR: https://git.openjdk.java.net/jdk/pull/2760 From hshi at openjdk.java.net Fri Apr 2 02:56:27 2021 From: hshi at openjdk.java.net (Hui Shi) Date: Fri, 2 Apr 2021 02:56:27 GMT Subject: RFR: 8260267: vmTestbase/gc/gctests/FinalizeLock/FinalizeLock.java fails with fatal error: Mark stack space exhausted In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 10:58:56 GMT, ?? wrote: > ZGC mark stack will be exhausted when the live object graph is very large, which due to the pseudo-BFS style visiting of the object graph (ZGC gc worker will push children to other stripes which behave like BFS, so the maximum mark stack size is O(num_objects).). But other gc like g1 or Shenandoah use DFS to visiting the object graph, and the mark stack usage merely exceeds 32M, which may due to the maximum mark stack size is related to the depth of the object graph O(depth_of_object_graph). > > The following is the test case used to reproduce the crash, > import java.util.HashMap; > import java.util.HashSet; > > class Test2 { > public static int NODE_COUNT = 25000000; > public static int NODE_COUNT_UB = 1 << 25; > public static int STRING_GEN_INDEX = 1; > public static int WATCHER_COUNT = 100; > public static int MAP_COUNT = 8; > public static int BIN_COUNT = 16; > public static Object[] watchTable = new Object[MAP_COUNT]; > > public static class Watcher { > public static long counter = 0; > public long index; > Watcher() { > index = counter; > counter++; > } > } > > public static int gen_name() { > int cur = STRING_GEN_INDEX++; > int reminder = cur % BIN_COUNT; > cur = cur - reminder + NODE_COUNT_UB * reminder; > return cur; > } > > public static HashMap> get_map(int index) { > return (HashMap>)watchTable[index]; > } > > public static void gen_watcher() { > int name = gen_name(); > HashSet set = new HashSet(); > for (int i = 0; i < WATCHER_COUNT; i++) { > set.add(new Watcher()); > } > for (int i = 0; i < MAP_COUNT; i++) { > get_map(i).put(name, set); > } > } > > public static void setup() { > for (int i = 0; i < MAP_COUNT; i++) { > watchTable[i] = new HashMap>(); > } > for (int i = 0; i < NODE_COUNT; i++) { > gen_watcher(); > } > } > > public static void main(String[] args) { > setup(); > for (int i = 0; i < 10; i++) { > System.gc(); > } > System.out.println("pass"); > } > } > > The command to run the testcase is, > java -XX:+UseZGC -Xmx280g -XX:ZCollectionInterval=10 -Xlog:gc*=debug:file=gc.log:time,level,tags:filesize=5g Test2 src/hotspot/share/gc/z/zMarkStackAllocator.cpp line 181: > 179: > 180: for (;;) { > 181: if (nmagazine == 0) { It's confusing here _nmagazine can be negative after dec. might add some comment on this method. ------------- PR: https://git.openjdk.java.net/jdk/pull/3262 From github.com+25214855+casparcwang at openjdk.java.net Fri Apr 2 03:02:19 2021 From: github.com+25214855+casparcwang at openjdk.java.net (=?UTF-8?B?546L6LaF?=) Date: Fri, 2 Apr 2021 03:02:19 GMT Subject: Withdrawn: 8260267: vmTestbase/gc/gctests/FinalizeLock/FinalizeLock.java fails with fatal error: Mark stack space exhausted In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 10:58:56 GMT, ?? wrote: > ZGC mark stack will be exhausted when the live object graph is very large, which due to the pseudo-BFS style visiting of the object graph (ZGC gc worker will push children to other stripes which behave like BFS, so the maximum mark stack size is O(num_objects).). But other gc like g1 or Shenandoah use DFS to visiting the object graph, and the mark stack usage merely exceeds 32M, which may due to the maximum mark stack size is related to the depth of the object graph O(depth_of_object_graph). > > The following is the test case used to reproduce the crash, > import java.util.HashMap; > import java.util.HashSet; > > class Test2 { > public static int NODE_COUNT = 25000000; > public static int NODE_COUNT_UB = 1 << 25; > public static int STRING_GEN_INDEX = 1; > public static int WATCHER_COUNT = 100; > public static int MAP_COUNT = 8; > public static int BIN_COUNT = 16; > public static Object[] watchTable = new Object[MAP_COUNT]; > > public static class Watcher { > public static long counter = 0; > public long index; > Watcher() { > index = counter; > counter++; > } > } > > public static int gen_name() { > int cur = STRING_GEN_INDEX++; > int reminder = cur % BIN_COUNT; > cur = cur - reminder + NODE_COUNT_UB * reminder; > return cur; > } > > public static HashMap> get_map(int index) { > return (HashMap>)watchTable[index]; > } > > public static void gen_watcher() { > int name = gen_name(); > HashSet set = new HashSet(); > for (int i = 0; i < WATCHER_COUNT; i++) { > set.add(new Watcher()); > } > for (int i = 0; i < MAP_COUNT; i++) { > get_map(i).put(name, set); > } > } > > public static void setup() { > for (int i = 0; i < MAP_COUNT; i++) { > watchTable[i] = new HashMap>(); > } > for (int i = 0; i < NODE_COUNT; i++) { > gen_watcher(); > } > } > > public static void main(String[] args) { > setup(); > for (int i = 0; i < 10; i++) { > System.gc(); > } > System.out.println("pass"); > } > } > > The command to run the testcase is, > java -XX:+UseZGC -Xmx280g -XX:ZCollectionInterval=10 -Xlog:gc*=debug:file=gc.log:time,level,tags:filesize=5g Test2 This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.java.net/jdk/pull/3262 From akozlov at azul.com Fri Apr 2 09:02:51 2021 From: akozlov at azul.com (Anton Kozlov) Date: Fri, 2 Apr 2021 12:02:51 +0300 Subject: [jdk13u-dev] RFR: 8264640: CMS ParScanClosure misses a barrier In-Reply-To: References: Message-ID: Adding hotspot-gc-dev. It will be great to receive comments from GC experts, even the fix does not make sense for mainline jdk. Thanks, Anton On 4/2/21 11:51 AM, Anton Kozlov wrote: > Hi, please review an original fix for a GC crash. The jdk13u is the latest supported version that still has buggy code, it was deleted in jdk14 as a part of JEP 363: Remove the Concurrent Mark Sweep (CMS) Garbage Collector. So I'm proposing it here. > > The fix is low-risk, on x86-64 it just introduces a compiler barrier to prevent two reads to be reordered as intended by surrounding comments. On CPUs with weaker memory models it introduces CPU barriers as well. > > ------------- > > Commit messages: > - Add missing barriers > > Changes: https://git.openjdk.java.net/jdk13u-dev/pull/165/files > Webrev: https://webrevs.openjdk.java.net/?repo=jdk13u-dev&pr=165&range=00 > Issue: https://bugs.openjdk.java.net/browse/JDK-8264640 > Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod > Patch: https://git.openjdk.java.net/jdk13u-dev/pull/165.diff > Fetch: git fetch https://git.openjdk.java.net/jdk13u-dev pull/165/head:pull/165 > > PR: https://git.openjdk.java.net/jdk13u-dev/pull/165 > From github.com+168222+mgkwill at openjdk.java.net Fri Apr 2 16:38:02 2021 From: github.com+168222+mgkwill at openjdk.java.net (Marcus G K Williams) Date: Fri, 2 Apr 2021 16:38:02 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: > When using LargePageSizeInBytes=1G, os::Linux::reserve_memory_special_huge_tlbfs* cannot select large pages smaller than 1G. Code heap usually uses less than 1G, so currently the code precludes code heap from using > Large pages in this circumstance and when os::Linux::reserve_memory_special_huge_tlbfs* is called page sizes fall back to Linux::page_size() (usually 4k). > > This change allows the above use case by populating all large_page_sizes present in /sys/kernel/mm/hugepages in _page_sizes upon calling os::Linux::setup_large_page_size(). > > In os::Linux::reserve_memory_special_huge_tlbfs* we then select the largest large page size available in _page_sizes that is smaller than bytes being reserved. Marcus G K Williams has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 44 commits: - Merge branch 'master' into update_hlp - Rebase on pull/3073 Signed-off-by: Marcus G K Williams - Merge branch 'pull/3073' into update_hlp - Thomas review. Changed commit_memory_special to return bool to signal if the request succeeded or not. - Self review. Update helper name to better match commit_memory_special(). - Marcus review. Updated comments. - Ivan review Renamed helper to commit_memory_special and updated the comments. - 8262291: Refactor reserve_memory_special_huge_tlbfs - Merge branch 'master' into update_hlp - Addressed kstefanj review suggestions Signed-off-by: Marcus G K Williams - ... and 34 more: https://git.openjdk.java.net/jdk/compare/f60e81bf...f5bfe224 ------------- Changes: https://git.openjdk.java.net/jdk/pull/1153/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1153&range=26 Stats: 282 lines in 4 files changed: 80 ins; 101 del; 101 mod Patch: https://git.openjdk.java.net/jdk/pull/1153.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1153/head:pull/1153 PR: https://git.openjdk.java.net/jdk/pull/1153 From github.com+168222+mgkwill at openjdk.java.net Fri Apr 2 16:55:30 2021 From: github.com+168222+mgkwill at openjdk.java.net (Marcus G K Williams) Date: Fri, 2 Apr 2021 16:55:30 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v23] In-Reply-To: References: <_uIvdHdm5ptjZJ8gEw8AzBJ8PC-16GoEWUuKW5zAXrg=.36c2da1c-685b-4d4d-a4e5-73b3fc48b812@github.com> Message-ID: On Thu, 18 Mar 2021 12:47:40 GMT, Stefan Johansson wrote: > > > This would also give us to to think about a question I haven't made up my mind around. What will `LargePageSizeInBytes` mean after this change? Should the JVM use 1g pages (on a system where 2m i the default) even if `LargePageSizeInBytes` is not set? > > > > > > I see two valid scenarios: > > Me too. > > > a) either use huge pages as best as possible; remove fine-grained control from user hands. So, if we have 1G pages, prefer those over 2M over 4K. Then, we could completely remove LargePageSizeInBytes. There is no need for this switch anymore. > > I agree, preferably we can make it so that the upper layers can use something like `page_size_for_region*` and request a certain page size, but fall back to smaller ones. > > > b) or keep the current behavior. In that case, UseLargePageSize means "use at most default huge page size" even if larger pages are available; and LargePageSizeInBytes can be used to override the large page size to use. > > So it becomes more like a maximium value right? Or at least this is how I've thought about this second scenario. On a system with both 2M (the default size) and 1G pages available you would have to set `LargePageSizeInBytes=1g` to use the 1G pages, but the 2M could still be used for smaller mappings. > > > (a): Its elegant, and efficiently uses system resources when available. But its an incompatible change, and the VM being grabby means we could end up using pages meant for a different process. > > (b): downward compatible. In a sense. With Marcus change, we break downward compatibility already: where "UseLargePageSizeInBytes" before meant "use that or nothing", it now means "use that or whatever smaller page sizes you find". > > I prefer (a), honestly. > > I would also prefer (a). @kstefanj @tstuefe 1. Would (a) [removing fine grained control of large page [Huge Page] sizes with LargePageSizeInBytes and relying on automatic selection of sizes from available] require JEP process? 2. Can we put LargePageSizeInBytes flag in DIAGNOSTIC or EXPERIMENTAL and set it otherwise to the largest available page size? This way we have the default (a) and if users want to change page size they can use diagnostic or experimental and use (b). Also this would lower the amount of change needed. I'm a bit concerned with removing LargePageSizeInBytes altogether. I don't know if some use cases would break or need more fine tuned control of page sizes. I've incorporated https://github.com/openjdk/jdk/pull/3073 into this change, as a good base to build on, this also seemed to be Stefan's optimal path for this change. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From dcubed at openjdk.java.net Fri Apr 2 20:48:33 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 2 Apr 2021 20:48:33 GMT Subject: RFR: 8264662: ProblemList vmTestbase/jit/escape/AdaptiveBlocking/AdaptiveBlocking001/AdaptiveBlocking001.java on win-x64 with ZGC Message-ID: A trivial fix to ProblemList vmTestbase/jit/escape/AdaptiveBlocking/AdaptiveBlocking001/AdaptiveBlocking001.java on win-x64 with ZGC ------------- Commit messages: - 8264662: ProblemList vmTestbase/jit/escape/AdaptiveBlocking/AdaptiveBlocking001/AdaptiveBlocking001.java on win-x64 with ZGC Changes: https://git.openjdk.java.net/jdk/pull/3331/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3331&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264662 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3331.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3331/head:pull/3331 PR: https://git.openjdk.java.net/jdk/pull/3331 From hseigel at openjdk.java.net Fri Apr 2 20:52:17 2021 From: hseigel at openjdk.java.net (Harold Seigel) Date: Fri, 2 Apr 2021 20:52:17 GMT Subject: RFR: 8264662: ProblemList vmTestbase/jit/escape/AdaptiveBlocking/AdaptiveBlocking001/AdaptiveBlocking001.java on win-x64 with ZGC In-Reply-To: References: Message-ID: On Fri, 2 Apr 2021 20:41:07 GMT, Daniel D. Daugherty wrote: > A trivial fix to ProblemList vmTestbase/jit/escape/AdaptiveBlocking/AdaptiveBlocking001/AdaptiveBlocking001.java > on win-x64 with ZGC Looks good and trivial. Thanks, Harold ------------- Marked as reviewed by hseigel (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3331 From dcubed at openjdk.java.net Fri Apr 2 21:35:20 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 2 Apr 2021 21:35:20 GMT Subject: RFR: 8264662: ProblemList vmTestbase/jit/escape/AdaptiveBlocking/AdaptiveBlocking001/AdaptiveBlocking001.java on win-x64 with ZGC In-Reply-To: References: Message-ID: On Fri, 2 Apr 2021 20:49:35 GMT, Harold Seigel wrote: >> A trivial fix to ProblemList vmTestbase/jit/escape/AdaptiveBlocking/AdaptiveBlocking001/AdaptiveBlocking001.java >> on win-x64 with ZGC > > Looks good and trivial. > Thanks, Harold @hseigel - Thanks for the review! ------------- PR: https://git.openjdk.java.net/jdk/pull/3331 From dcubed at openjdk.java.net Fri Apr 2 21:35:21 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Fri, 2 Apr 2021 21:35:21 GMT Subject: Integrated: 8264662: ProblemList vmTestbase/jit/escape/AdaptiveBlocking/AdaptiveBlocking001/AdaptiveBlocking001.java on win-x64 with ZGC In-Reply-To: References: Message-ID: On Fri, 2 Apr 2021 20:41:07 GMT, Daniel D. Daugherty wrote: > A trivial fix to ProblemList vmTestbase/jit/escape/AdaptiveBlocking/AdaptiveBlocking001/AdaptiveBlocking001.java > on win-x64 with ZGC This pull request has now been integrated. Changeset: 9c283da1 Author: Daniel D. Daugherty URL: https://git.openjdk.java.net/jdk/commit/9c283da1 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod 8264662: ProblemList vmTestbase/jit/escape/AdaptiveBlocking/AdaptiveBlocking001/AdaptiveBlocking001.java on win-x64 with ZGC Reviewed-by: hseigel ------------- PR: https://git.openjdk.java.net/jdk/pull/3331 From yyang at openjdk.java.net Mon Apr 5 07:44:04 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Mon, 5 Apr 2021 07:44:04 GMT Subject: RFR: 8264682: MemProfiling does not own Heap_lock when using G1 Message-ID: Trivial fix for JDK-8264682. `Universe::heap()->used()` calls G1Allocator::used_in_alloc_regions() when G1 enabled, it checks whether Heap_lock was owned on this thread's behalf. ------------- Commit messages: - memprofiling_crash Changes: https://git.openjdk.java.net/jdk/pull/3340/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3340&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264682 Stats: 59 lines in 2 files changed: 59 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3340.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3340/head:pull/3340 PR: https://git.openjdk.java.net/jdk/pull/3340 From kbarrett at openjdk.java.net Mon Apr 5 14:31:24 2021 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Mon, 5 Apr 2021 14:31:24 GMT Subject: RFR: 8264682: MemProfiling does not own Heap_lock when using G1 In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 07:34:30 GMT, Yi Yang wrote: > Trivial fix for JDK-8264682. No, not trivial at all. Dropping a GC-specific blob like this into shared code is highly questionable. I'm not sure how this issue should be addressed, but I don't think the proposed change is the right approach. ------------- PR: https://git.openjdk.java.net/jdk/pull/3340 From github.com+71722661+earthling-amzn at openjdk.java.net Mon Apr 5 17:00:44 2021 From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn) Date: Mon, 5 Apr 2021 17:00:44 GMT Subject: RFR: 8264727: Remove extraneous whitespace from phase timings report Message-ID: This extra space doesn't look like it was intended to improve any alignment of text in the report. ------------- Commit messages: - Remove extraneous whitespace from phase timings report Changes: https://git.openjdk.java.net/jdk/pull/3342/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3342&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264727 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3342.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3342/head:pull/3342 PR: https://git.openjdk.java.net/jdk/pull/3342 From shade at openjdk.java.net Mon Apr 5 17:00:45 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 5 Apr 2021 17:00:45 GMT Subject: RFR: 8264727: Remove extraneous whitespace from phase timings report In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 16:45:43 GMT, earthling-amzn wrote: > This extra space doesn't look like it was intended to improve any alignment of text in the report. Marked as reviewed by shade (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3342 From shade at openjdk.java.net Mon Apr 5 17:00:45 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 5 Apr 2021 17:00:45 GMT Subject: RFR: 8264727: Remove extraneous whitespace from phase timings report In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 16:49:47 GMT, Aleksey Shipilev wrote: >> This extra space doesn't look like it was intended to improve any alignment of text in the report. > > Marked as reviewed by shade (Reviewer). Created: https://bugs.openjdk.java.net/browse/JDK-8264727 -- rename this PR to "8264727: Shenandoah: Remove extraneous whitespace from phase timings report" to get it hooked up. ------------- PR: https://git.openjdk.java.net/jdk/pull/3342 From shade at openjdk.java.net Mon Apr 5 17:06:13 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Mon, 5 Apr 2021 17:06:13 GMT Subject: RFR: 8264727: Remove extraneous whitespace from phase timings report In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 16:45:43 GMT, earthling-amzn wrote: > This extra space doesn't look like it was intended to improve any alignment of text in the report. Looks fine and trivial. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3342 From zgu at openjdk.java.net Mon Apr 5 21:50:49 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Mon, 5 Apr 2021 21:50:49 GMT Subject: RFR: 8264718: Shenandoah: enable string deduplication during root scanning Message-ID: Shenandoah used to scan roots at pauses, so it deliberately disables string deduplication during root scanning to avoid extra pause times. Now, Shenandoah scans roots in concurrent phase, it is no longer a concern, we should enable it. Test: - [x] hotspot_gc_shenandoah ------------- Commit messages: - Merge branch 'master' into enable_dedup_roots - init Changes: https://git.openjdk.java.net/jdk/pull/3348/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3348&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264718 Stats: 69 lines in 5 files changed: 40 ins; 25 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/3348.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3348/head:pull/3348 PR: https://git.openjdk.java.net/jdk/pull/3348 From iklam at openjdk.java.net Mon Apr 5 22:02:42 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Mon, 5 Apr 2021 22:02:42 GMT Subject: RFR: 8214455: Relocate CDS archived regions to the top of the G1 heap Message-ID: When the runtime heap and dump time heap have different sizes, it's possible that the archived heap regions are mapped to the middle of the runtime heap. Because the archived heap regions are pinned, this could reduce the size of the largest "humongous" array allocation. In the worst case, the maximum allocatable array length may be half of the optimal value. The fix is to relocate the archived regions to the top of the G1 heap (currently archived regions are supported only by G1 on 64-bit JVM). Note that usually the top of the G1 heap is placed just below the 32GB boundary. As a result, the archived heap regions are at the same location between run time and dump time, so no relocation is necessary. In mach5 testing, we occasionally run into this problem (see https://bugs.openjdk.java.net/browse/JDK-8239089), probably because we fail to reserve the heap below 32GB due to address space layout randomization (ASLR). ------------- Commit messages: - Added -Xlog:cds=debug in case this test fails again - 8214455: Relocate CDS archived regions to the top of the G1 heap Changes: https://git.openjdk.java.net/jdk/pull/3349/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3349&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8214455 Stats: 143 lines in 8 files changed: 139 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/3349.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3349/head:pull/3349 PR: https://git.openjdk.java.net/jdk/pull/3349 From iklam at openjdk.java.net Mon Apr 5 22:48:25 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Mon, 5 Apr 2021 22:48:25 GMT Subject: RFR: 8264682: MemProfiling does not own Heap_lock when using G1 In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 14:29:11 GMT, Kim Barrett wrote: > > Trivial fix for JDK-8264682. > > No, not trivial at all. Dropping a GC-specific blob like this into shared code is highly questionable. I'm not sure how this issue should be addressed, but I don't think the proposed change is the right approach. It looks like the following APIs can be reliably called without holding the Heap_lock. However, I am not sure if they would give the same result. I.e., can you get `Universe::heap()->unused()` by calling `Universe::heap()->capacity() - Universe::heap()->unused()`. JVM_ENTRY_NO_ENV(jlong, JVM_TotalMemory(void)) size_t n = Universe::heap()->capacity(); return convert_size_t_to_jlong(n); JVM_END JVM_ENTRY_NO_ENV(jlong, JVM_FreeMemory(void)) size_t n = Universe::heap()->unused(); return convert_size_t_to_jlong(n); JVM_END ------------- PR: https://git.openjdk.java.net/jdk/pull/3340 From dholmes at openjdk.java.net Tue Apr 6 02:08:36 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Tue, 6 Apr 2021 02:08:36 GMT Subject: RFR: 8264682: MemProfiling does not own Heap_lock when using G1 In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 07:34:30 GMT, Yi Yang wrote: > Trivial fix for JDK-8264682. > > `Universe::heap()->used()` calls G1Allocator::used_in_alloc_regions() when G1 enabled, it checks whether Heap_lock was owned on this thread's behalf. Sorry but I have to agree with Kim that this is the wrong fix for this problem. And the test needs adjusting. Thanks, David src/hotspot/share/runtime/memprofiler.cpp line 120: > 118: // used() calls G1Allocator::used_in_alloc_regions() when G1 enabled, > 119: // it checks whether Heap_lock was owned on this thread's behalf. > 120: G1GC_ONLY(MutexLocker ml(Heap_lock);) This seems really out of place - this code should not need to know anything about the specific locking requirements of different GC's. That should be handled inside the appropriate chunk of GC code. test/hotspot/jtreg/runtime/MemProfiler/MemProfilingWithGC.java line 44: > 42: "UseShenandoahGC", > 43: "UseZGC", > 44: "UseEpsilonGC", You have to allow for the fact that these GC's may not have all been built into the JVM under test. ------------- Changes requested by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3340 From shade at openjdk.java.net Tue Apr 6 06:40:28 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 6 Apr 2021 06:40:28 GMT Subject: RFR: 8264727: Remove extraneous whitespace from phase timings report In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 17:02:55 GMT, Aleksey Shipilev wrote: >> This extra space doesn't look like it was intended to improve any alignment of text in the report. > > Looks fine and trivial. You need to resolve the integration blocker before integration: the PR title and bug synopsis are mismatched. You need "Shenandoah: " in the title. ------------- PR: https://git.openjdk.java.net/jdk/pull/3342 From tschatzl at openjdk.java.net Tue Apr 6 09:44:28 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 6 Apr 2021 09:44:28 GMT Subject: RFR: 8214455: Relocate CDS archived regions to the top of the G1 heap In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 21:48:43 GMT, Ioi Lam wrote: > When the runtime heap and dump time heap have different sizes, it's possible that the archived heap regions are mapped to the middle of the runtime heap. Because the archived heap regions are pinned, this could reduce the size of the largest "humongous" array allocation. In the worst case, the maximum allocatable array length may be half of the optimal value. > > The fix is to relocate the archived regions to the top of the G1 heap (currently archived regions are supported only by G1 on 64-bit JVM). > > Note that usually the top of the G1 heap is placed just below the 32GB boundary. As a result, the archived heap regions are at the same location between run time and dump time, so no relocation is necessary. > > In mach5 testing, we occasionally run into this problem (see https://bugs.openjdk.java.net/browse/JDK-8239089), probably because we fail to reserve the heap below 32GB due to address space layout randomization (ASLR). Lgtm. src/hotspot/share/memory/filemap.cpp line 1846: > 1844: _heap_pointers_need_patching = true; > 1845: } else if (header()->heap_end() != CompressedOops::end()) { > 1846: log_info(cds)("CDS heap data need to be relocated to the end of the runtime heap to reduce fragmentation"); s/need/needs test/hotspot/jtreg/runtime/cds/appcds/cacheObject/HeapFragmentationTest.java line 30: > 28: * @bug 8214455 > 29: * @requires vm.cds.archived.java.heap > 30: * @requires (sun.arch.data.model == "64" & os.maxMemory > 4g) additional space before the & ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3349 From ayang at openjdk.java.net Tue Apr 6 09:54:13 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Tue, 6 Apr 2021 09:54:13 GMT Subject: RFR: 8264424: Support OopStorage bulk allocation In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 11:55:59 GMT, Kim Barrett wrote: > Please review this change to OopStorage to support bulk allocation. A new overload for allocate is provided: > size_t allocate(oop** entries, size_t size) > The approach taken is to claim all of the available entries in the first available block, add as many of those entries as will fit in the buffer, and release any remaining entries back to the block. Only the claim part needs to be done while holding the allocation mutex. This is is optimized for clients that want more than just a couple entries, and minimizes time under the lock. The maximum number of entries (the number of entries in a block) is provided as a constant for sizing requests, to avoid the release of unrequested entries. An application that wants more than that, or a specific number that might not be available from the next block, needs to make multiple bulk allocation calls, but that's still going to be much faster than one at a time allocations. > > Testing: > mach5 tier1 > New gtest for bulk allocation. > I've been using this for a while as part of another in-development change that makes heavy use of this feature. Marked as reviewed by ayang (Committer). src/hotspot/share/gc/shared/oopStorage.cpp line 478: > 476: block = block_for_allocation(); > 477: if (block == NULL) return 0; // Block allocation failed. > 478: // Take exclusive use of this block for allocation. Maybe "Taking all remaining entries, so remove from list."? The "exclusive use" is ensured by the above `MutexLocker`, not `unlink`. src/hotspot/share/gc/shared/oopStorage.hpp line 119: > 117: // Allocates multiple entries, returning them in the ptrs buffer. The > 118: // number of entries that will be allocated is never more than the minimum > 119: // of size and bulk_allocate_limit, but may be less than either. Possibly It took me a few seconds to understand this sentence. Maybe replace it with "postcondition: result <= min(size, bulk_allocate_limit)" below? ------------- PR: https://git.openjdk.java.net/jdk/pull/3264 From tschatzl at openjdk.java.net Tue Apr 6 10:07:26 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 6 Apr 2021 10:07:26 GMT Subject: RFR: 8264027: Refactor "CLEANUP" region printing In-Reply-To: <6NLEJmvE7zRDNprrDS5B3suYSfloxZAdI2zPIPj3yu0=.286cde8e-4429-416c-a62b-2d0f0cf24627@github.com> References: <6NLEJmvE7zRDNprrDS5B3suYSfloxZAdI2zPIPj3yu0=.286cde8e-4429-416c-a62b-2d0f0cf24627@github.com> Message-ID: On Tue, 30 Mar 2021 03:14:41 GMT, Kim Barrett wrote: >> Hi all, >> >> can I have reviews for this small cleanup that refactors "CLEANUP" region status printing? >> >> - where managing a `FreeRegionList` is not necessary, inline the call to the method (which is guarded by a check that if logging is not enabled, we do not print anything anyway) >> - for the single remaining case, create a helper method in `G1HRPrinter` >> >> The first item has been done as per the suggestion from @kstefanj [here](https://github.com/openjdk/jdk/pull/3154#pullrequestreview-619061961) ; the alternative would be to keep the free region lists and call `G1HRPrinter::cleanup` on it later, which I actually initially implemented. I could not find a difference performance-wise. >> >> Testing: checked if there is some perf difference on a heap with 50k regions; tier1-3 > > Looks good. Thanks @kimbarrett @albertnetymk for your reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/3243 From tschatzl at openjdk.java.net Tue Apr 6 10:07:28 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 6 Apr 2021 10:07:28 GMT Subject: Integrated: 8264027: Refactor "CLEANUP" region printing In-Reply-To: References: Message-ID: On Mon, 29 Mar 2021 12:43:12 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small cleanup that refactors "CLEANUP" region status printing? > > - where managing a `FreeRegionList` is not necessary, inline the call to the method (which is guarded by a check that if logging is not enabled, we do not print anything anyway) > - for the single remaining case, create a helper method in `G1HRPrinter` > > The first item has been done as per the suggestion from @kstefanj [here](https://github.com/openjdk/jdk/pull/3154#pullrequestreview-619061961) ; the alternative would be to keep the free region lists and call `G1HRPrinter::cleanup` on it later, which I actually initially implemented. I could not find a difference performance-wise. > > Testing: checked if there is some perf difference on a heap with 50k regions; tier1-3 This pull request has now been integrated. Changeset: bf26a255 Author: Thomas Schatzl URL: https://git.openjdk.java.net/jdk/commit/bf26a255 Stats: 71 lines in 4 files changed: 44 ins; 21 del; 6 mod 8264027: Refactor "CLEANUP" region printing Reviewed-by: kbarrett, ayang ------------- PR: https://git.openjdk.java.net/jdk/pull/3243 From tschatzl at openjdk.java.net Tue Apr 6 10:49:32 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 6 Apr 2021 10:49:32 GMT Subject: RFR: 8264424: Support OopStorage bulk allocation In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 11:55:59 GMT, Kim Barrett wrote: > Please review this change to OopStorage to support bulk allocation. A new overload for allocate is provided: > size_t allocate(oop** entries, size_t size) > The approach taken is to claim all of the available entries in the first available block, add as many of those entries as will fit in the buffer, and release any remaining entries back to the block. Only the claim part needs to be done while holding the allocation mutex. This is is optimized for clients that want more than just a couple entries, and minimizes time under the lock. The maximum number of entries (the number of entries in a block) is provided as a constant for sizing requests, to avoid the release of unrequested entries. An application that wants more than that, or a specific number that might not be available from the next block, needs to make multiple bulk allocation calls, but that's still going to be much faster than one at a time allocations. > > Testing: > mach5 tier1 > New gtest for bulk allocation. > I've been using this for a while as part of another in-development change that makes heavy use of this feature. Lgtm. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3264 From tschatzl at openjdk.java.net Tue Apr 6 14:04:21 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 6 Apr 2021 14:04:21 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v12] In-Reply-To: References: Message-ID: On Wed, 31 Mar 2021 09:59:51 GMT, Hamlin Li wrote: >> [Here](https://github.com/tschatzl/jdk/tree/bot-fixes) is a potential fix for this. This needs some discussion on whether an alternatives would be better or not; more of them could be: >> * just initialize a dummy BOT for not-compacted young region >> * exclude young regions from this dead-wood scheme >> In any case this fix should probably go in separately. > >> [Here](https://github.com/tschatzl/jdk/tree/bot-fixes) is a potential fix for this. > > Thanks for the fix. >> In any case this fix should probably go in separately. > > Do you mean we push this pr, then discuss the fix in another thread? If yes, I will file a bug to track the issue and initialize discussion on this bug. This change should go in after PR#3356 which fixes this issue. I think it is a change that is worth pointing out and discussing separately - as you might see from the long description. ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From coleenp at openjdk.java.net Tue Apr 6 14:34:27 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Tue, 6 Apr 2021 14:34:27 GMT Subject: RFR: 8264682: MemProfiling does not own Heap_lock when using G1 In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 02:04:57 GMT, David Holmes wrote: >> Trivial fix for JDK-8264682. >> >> `Universe::heap()->used()` calls G1Allocator::used_in_alloc_regions() when G1 enabled, it checks whether Heap_lock was owned on this thread's behalf. > > Sorry but I have to agree with Kim that this is the wrong fix for this problem. > > And the test needs adjusting. > > Thanks, > David Unless we can imagine a reason why this code is needed, I think the old memprofiler should be removed. ------------- PR: https://git.openjdk.java.net/jdk/pull/3340 From iklam at openjdk.java.net Tue Apr 6 17:04:19 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Tue, 6 Apr 2021 17:04:19 GMT Subject: RFR: 8214455: Relocate CDS archived regions to the top of the G1 heap [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 09:35:01 GMT, Thomas Schatzl wrote: >> Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: >> >> @tschatzl review -- fix typos and whitespace > > src/hotspot/share/memory/filemap.cpp line 1846: > >> 1844: _heap_pointers_need_patching = true; >> 1845: } else if (header()->heap_end() != CompressedOops::end()) { >> 1846: log_info(cds)("CDS heap data need to be relocated to the end of the runtime heap to reduce fragmentation"); > > s/need/needs Fixed. > test/hotspot/jtreg/runtime/cds/appcds/cacheObject/HeapFragmentationTest.java line 30: > >> 28: * @bug 8214455 >> 29: * @requires vm.cds.archived.java.heap >> 30: * @requires (sun.arch.data.model == "64" & os.maxMemory > 4g) > > additional space before the & Fixed. Thanks for the review. ------------- PR: https://git.openjdk.java.net/jdk/pull/3349 From iklam at openjdk.java.net Tue Apr 6 17:04:18 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Tue, 6 Apr 2021 17:04:18 GMT Subject: RFR: 8214455: Relocate CDS archived regions to the top of the G1 heap [v2] In-Reply-To: References: Message-ID: > When the runtime heap and dump time heap have different sizes, it's possible that the archived heap regions are mapped to the middle of the runtime heap. Because the archived heap regions are pinned, this could reduce the size of the largest "humongous" array allocation. In the worst case, the maximum allocatable array length may be half of the optimal value. > > The fix is to relocate the archived regions to the top of the G1 heap (currently archived regions are supported only by G1 on 64-bit JVM). > > Note that usually the top of the G1 heap is placed just below the 32GB boundary. As a result, the archived heap regions are at the same location between run time and dump time, so no relocation is necessary. > > In mach5 testing, we occasionally run into this problem (see https://bugs.openjdk.java.net/browse/JDK-8239089), probably because we fail to reserve the heap below 32GB due to address space layout randomization (ASLR). Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: @tschatzl review -- fix typos and whitespace ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3349/files - new: https://git.openjdk.java.net/jdk/pull/3349/files/def7db7b..bedd676d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3349&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3349&range=00-01 Stats: 6 lines in 3 files changed: 0 ins; 0 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/3349.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3349/head:pull/3349 PR: https://git.openjdk.java.net/jdk/pull/3349 From mli at openjdk.java.net Wed Apr 7 01:17:22 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Wed, 7 Apr 2021 01:17:22 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v12] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 14:01:15 GMT, Thomas Schatzl wrote: >>> [Here](https://github.com/tschatzl/jdk/tree/bot-fixes) is a potential fix for this. >> >> Thanks for the fix. >>> In any case this fix should probably go in separately. >> >> Do you mean we push this pr, then discuss the fix in another thread? If yes, I will file a bug to track the issue and initialize discussion on this bug. > > This change should go in after PR#3356 which fixes this issue. I think it is a change that is worth pointing out and discussing separately - as you might see from the long description. I see, Thanks Thomas! ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From david.holmes at oracle.com Wed Apr 7 05:21:06 2021 From: david.holmes at oracle.com (David Holmes) Date: Wed, 7 Apr 2021 15:21:06 +1000 Subject: RFR: 8264682: MemProfiling does not own Heap_lock when using G1 In-Reply-To: References: Message-ID: On 7/04/2021 12:34 am, Coleen Phillimore wrote: > On Tue, 6 Apr 2021 02:04:57 GMT, David Holmes wrote: > >>> Trivial fix for JDK-8264682. >>> >>> `Universe::heap()->used()` calls G1Allocator::used_in_alloc_regions() when G1 enabled, it checks whether Heap_lock was owned on this thread's behalf. >> >> Sorry but I have to agree with Kim that this is the wrong fix for this problem. >> >> And the test needs adjusting. >> >> Thanks, >> David > > Unless we can imagine a reason why this code is needed, I think the old memprofiler should be removed. Why was it added? I've no idea who might be using this, or for what. But I think removing it is a separate issue. Cheers, David > ------------- > > PR: https://git.openjdk.java.net/jdk/pull/3340 > From ccheung at openjdk.java.net Wed Apr 7 05:38:25 2021 From: ccheung at openjdk.java.net (Calvin Cheung) Date: Wed, 7 Apr 2021 05:38:25 GMT Subject: RFR: 8214455: Relocate CDS archived regions to the top of the G1 heap [v2] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 17:04:18 GMT, Ioi Lam wrote: >> When the runtime heap and dump time heap have different sizes, it's possible that the archived heap regions are mapped to the middle of the runtime heap. Because the archived heap regions are pinned, this could reduce the size of the largest "humongous" array allocation. In the worst case, the maximum allocatable array length may be half of the optimal value. >> >> The fix is to relocate the archived regions to the top of the G1 heap (currently archived regions are supported only by G1 on 64-bit JVM). >> >> Note that usually the top of the G1 heap is placed just below the 32GB boundary. As a result, the archived heap regions are at the same location between run time and dump time, so no relocation is necessary. >> >> In mach5 testing, we occasionally run into this problem (see https://bugs.openjdk.java.net/browse/JDK-8239089), probably because we fail to reserve the heap below 32GB due to address space layout randomization (ASLR). > > Ioi Lam has updated the pull request incrementally with one additional commit since the last revision: > > @tschatzl review -- fix typos and whitespace Looks good. Just one minor nit in the test case. test/hotspot/jtreg/runtime/cds/appcds/cacheObject/HeapFragmentationTest.java line 84: > 82: // Run with CDS. The archived heap regions should be relocated to avoid fragmentation. > 83: TestCommon.run(TestCommon.concat(execArgs, runTimeHeapSize, mainClass, BUFF_SIZE)) > 84: .assertNormalExit(successOutput); The assertNormalExit line should be indented four spaces relative to the previous line. ------------- Marked as reviewed by ccheung (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3349 From github.com+25214855+casparcwang at openjdk.java.net Wed Apr 7 08:03:27 2021 From: github.com+25214855+casparcwang at openjdk.java.net (=?UTF-8?B?546L6LaF?=) Date: Wed, 7 Apr 2021 08:03:27 GMT Subject: RFR: 8264718: Shenandoah: enable string deduplication during root scanning In-Reply-To: References: Message-ID: <0XUFNDhmkB4OgUxbtnadAcg9daAjdppc0VlVWJMZVPI=.69dcbac1-bbbd-45ba-97ac-2ad8b5c18d52@github.com> On Mon, 5 Apr 2021 21:42:32 GMT, Zhengyu Gu wrote: > Shenandoah used to scan roots at pauses, so it deliberately disables string deduplication during root scanning to avoid extra pause times. > > Now, Shenandoah scans roots in concurrent phase, it is no longer a concern, we should enable it. > > Test: > - [x] hotspot_gc_shenandoah src/hotspot/share/gc/shenandoah/shenandoahSTWMark.cpp line 134: > 132: ShenandoahInitMarkRootsClosure init_mark(task_queues()->queue(worker_id)); > 133: _root_scanner.roots_do(&init_mark, worker_id); > 134: extra empty line ------------- PR: https://git.openjdk.java.net/jdk/pull/3348 From tschatzl at openjdk.java.net Wed Apr 7 08:34:48 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 7 Apr 2021 08:34:48 GMT Subject: RFR: 8264783: G1 BOT verification should not verify beyond allocation threshold Message-ID: The G1 BOT contains an allocation threshold which basically acts as a "last known valid entry" index for the per-region BOT (which are views on the global BOT table). Currently G1 BOT verification actually "verifies" the BOT within a region past that BOT index. This causes issues with young regions; actually there is already code that prevents their BOT verification. This is perfectly fine, allocations in young regions do not update the BOTs. With JDK-8262068/PR #2760 this existing filtering of young regions (such that their BOT is not verified) does not work because there may be young regions that are not compacted (so with a BOT that have that last known valid entry at the start of the region) are still labelled as old. This change proposes to not try to verify the BOT beyond the last known valid index (which is arguably not worth doing), which also covers the existing young filtering. There are alternatives that may work in particular for JDK-8262068: a) always compact young regions (which recreates the BOT) b) create a "dummy" BOT that spans the entire part of the region containing live objects However they were rejected by me because option a) takes time and directly counters that optimization for no reason option b) makes finding the start of an object within these regions slow (they are not refined when there is at least *some* bot) and finally I do not think there is much point in trying to be clever about areas in the BOT that are known to not contain useful values. Testing: tier1-3 ------------- Commit messages: - Bot fixes for 2760 Changes: https://git.openjdk.java.net/jdk/pull/3356/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3356&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264783 Stats: 3 lines in 2 files changed: 1 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3356.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3356/head:pull/3356 PR: https://git.openjdk.java.net/jdk/pull/3356 From mli at openjdk.java.net Wed Apr 7 08:34:49 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Wed, 7 Apr 2021 08:34:49 GMT Subject: RFR: 8264783: G1 BOT verification should not verify beyond allocation threshold In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 13:59:12 GMT, Thomas Schatzl wrote: > The G1 BOT contains an allocation threshold which basically acts as a "last known valid entry" index for the per-region BOT (which are views on the global BOT table). > > Currently G1 BOT verification actually "verifies" the BOT within a region past that BOT index. > > This causes issues with young regions; actually there is already code that prevents their BOT verification. This is perfectly fine, allocations in young regions do not update the BOTs. > > With JDK-8262068/PR #2760 this existing filtering of young regions (such that their BOT is not verified) does not work because there may be young regions that are not compacted (so with a BOT that have that last known valid entry at the start of the region) are still labelled as old. > > This change proposes to not try to verify the BOT beyond the last known valid index (which is arguably not worth doing), which also covers the existing young filtering. > > There are alternatives that may work in particular for JDK-8262068: > a) always compact young regions (which recreates the BOT) > b) create a "dummy" BOT that spans the entire part of the region containing live objects > > However they were rejected by me because > option a) takes time and directly counters that optimization for no reason > option b) makes finding the start of an object within these regions slow (they are not refined when there is at least *some* bot) > > and finally I do not think there is much point in trying to be clever about areas in the BOT that are known to not contain useful values. > > Testing: tier1-3 Marked as reviewed by mli (Reviewer). The change looks good to me. Thanks Thomas! ------------- PR: https://git.openjdk.java.net/jdk/pull/3356 From tschatzl at openjdk.java.net Wed Apr 7 08:38:51 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 7 Apr 2021 08:38:51 GMT Subject: RFR: 8264788: Make SequentialSubTasksDone use-once Message-ID: Hi all, can I have reviews for this change that simplifies code ahead? SequentialSubTasksDone has some machinery to reset itself automatically after all tasks were claimed. This is not used at all and can/should be removed. Test: tier1-3 ------------- Commit messages: - Initial commit Changes: https://git.openjdk.java.net/jdk/pull/3372/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3372&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264788 Stats: 86 lines in 5 files changed: 2 ins; 67 del; 17 mod Patch: https://git.openjdk.java.net/jdk/pull/3372.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3372/head:pull/3372 PR: https://git.openjdk.java.net/jdk/pull/3372 From tschatzl at openjdk.java.net Wed Apr 7 08:54:29 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 7 Apr 2021 08:54:29 GMT Subject: RFR: 8264788: Make SequentialSubTasksDone use-once In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 08:32:11 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that simplifies code ahead? > > SequentialSubTasksDone has some machinery to reset itself automatically after all tasks were claimed. > > This is not used at all and can/should be removed. > > Test: tier1-3 Note that it is possible to remove the CAS loop in `SequentialSubTasksDone` with an atomic add. I wasn't sure to do this, but if somebody thinks it is a good idea (and probably faster in many implementations) similar to the following: bool SequentialSubTasksDone::try_claim_task(uint& t) { if (t < _num_tasks) { t = Atomic::add(&_num_claimed, 1) - 1; } return t < _num_tasks; } (And slightly fixing that assert in the destructor) ------------- PR: https://git.openjdk.java.net/jdk/pull/3372 From yyang at openjdk.java.net Wed Apr 7 09:36:29 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Wed, 7 Apr 2021 09:36:29 GMT Subject: RFR: 8264682: MemProfiling does not own Heap_lock when using G1 In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 14:31:00 GMT, Coleen Phillimore wrote: >> Sorry but I have to agree with Kim that this is the wrong fix for this problem. >> >> And the test needs adjusting. >> >> Thanks, >> David > > Unless we can imagine a reason why this code is needed, I think the old memprofiler should be removed. I'd like to propose a new PR to remove the memprofiler if there are no strong objections. ------------- PR: https://git.openjdk.java.net/jdk/pull/3340 From ayang at openjdk.java.net Wed Apr 7 09:39:37 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Wed, 7 Apr 2021 09:39:37 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v13] In-Reply-To: References: Message-ID: <47rveNWYV7e6_mdxd3EaiTdsxo1SEjtYwmVviLxVo80=.17d8b0ce-4afd-418e-b2ea-e6099a3e7199@github.com> On Thu, 1 Apr 2021 14:43:42 GMT, Hamlin Li wrote: >> Summary >> ----------- >> >> Improve G1 Full GC by skip compaction for regions with high survival ratio. >> >> Backgroud >> ----------- >> >> There are 4 steps in full gc of G1 GC. >> - mark live objects >> - prepare forwardee >> - adjust pointers >> - compact >> >> When full gc occurs, there may be very high percentage of live bytes in some regions. For these regions, it's not efficient to compact them and better to skip them, as there are little space to save but many objects to copy. >> >> Description >> ----------- >> >> We enhance the full gc implementation for the above situation through following steps: >> - accumulate live bytes of every hr in mark phase; (already done by JDK-8263495) >> - skip adding regions with high survial ratio, and set the region with high survival ratio as pinned in _region_attr_table during prepare phase; >> - nothing special is done in adjust phase, regions with high survial ratio are skipped because of pin setting in the above step; >> - nothing special is done in compact phase, regions with high survival ratio are skipped because these regions are skipped when adding regions to compaction set in the prepare phase; >> >> VM options related >> ----------- >> >> - MarkSweepDeadRatio: we reuse this exising vm option to indicate the high survial ratio threhold (100-MarkSweepDeadRatio) in G1. >> - default value of MarkSweepDeadRatio: 5 >> >> Test >> ----------- >> >> - specjbb2015: no regression >> - dacapo: (Attachment is the dacapo h2 full gc pause.) >> - 95% of full gc pauses: 10%-19% improvement. >> - 5% of full gc pauses: 1.2% improvement. >> - 0.1% of full gc pauses: -6.16% improvement. >> >> $ java -Xmx1g -Xms1g -XX:ParallelGCThreads=4 -Xlog:gc*=info:file=gc.log -jar dacapo-9.12-bach.jar --iterations 5 --size huge --no-pre-iteration-gc h2 > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > minor code improvement. Changes requested by ayang (Committer). src/hotspot/share/gc/g1/g1FullCollector.cpp line 238: > 236: if (hr->is_closed_archive()) { > 237: _region_attr_table.set_closed_archive(hr->hrm_index()); > 238: } else if (hr->is_pinned()) { Changing the condition to `hr->is_pinned() || force_pinned` is enough for this method, right? src/hotspot/share/gc/g1/g1FullCollector.hpp line 101: > 99: G1CMBitMap* mark_bitmap(); > 100: ReferenceProcessor* reference_processor(); > 101: size_t live_words(uint region_index) { return _live_stats[region_index]._live_words; } Could you add range check assertion for `region_index`? The extra spaces can be removed; no need to align with previous methods. src/hotspot/share/gc/g1/heapRegion.hpp line 174: > 172: void reset_compacted_after_full_gc(); > 173: // Update pinned heap region (not compacted) to be consistent after Full GC. > 174: void reset_not_compacted_after_full_gc(); Now that this version uses the existing "pinned" mechanism to skip high-live-ratio regions, this method can retain its original name/implementation, right? ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From tschatzl at openjdk.java.net Wed Apr 7 10:32:21 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 7 Apr 2021 10:32:21 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v13] In-Reply-To: <47rveNWYV7e6_mdxd3EaiTdsxo1SEjtYwmVviLxVo80=.17d8b0ce-4afd-418e-b2ea-e6099a3e7199@github.com> References: <47rveNWYV7e6_mdxd3EaiTdsxo1SEjtYwmVviLxVo80=.17d8b0ce-4afd-418e-b2ea-e6099a3e7199@github.com> Message-ID: On Wed, 7 Apr 2021 09:33:48 GMT, Albert Mingkun Yang wrote: >> Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: >> >> minor code improvement. > > src/hotspot/share/gc/g1/heapRegion.hpp line 174: > >> 172: void reset_compacted_after_full_gc(); >> 173: // Update pinned heap region (not compacted) to be consistent after Full GC. >> 174: void reset_not_compacted_after_full_gc(); > > Now that this version uses the existing "pinned" mechanism to skip high-live-ratio regions, this method can retain its original name/implementation, right? That is true, full gc reuses the mechanism originally implemented for pinned regions. However "pinned" is something very specific in G1 context so I think it is better to use a different, more generic name. The regions that are not compacted (and not yet pinned) are not really temporarily pinned. There has been an earlier discussion (not sure if here in this PR) to actually rename the use of "pinned" in G1 full gc to use "not compacted" too, resulting in a CR to rename this (in [JDK-8264423](https://bugs.openjdk.java.net/browse/JDK-8264423)). However I was going to wait for this change to go in before sending out a PR. Also this "not_compacted" name better matches the "compacted" method name above. So I would prefer to keep this as is. ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From tschatzl at openjdk.java.net Wed Apr 7 10:37:19 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 7 Apr 2021 10:37:19 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v13] In-Reply-To: References: Message-ID: On Thu, 1 Apr 2021 14:43:42 GMT, Hamlin Li wrote: >> Summary >> ----------- >> >> Improve G1 Full GC by skip compaction for regions with high survival ratio. >> >> Backgroud >> ----------- >> >> There are 4 steps in full gc of G1 GC. >> - mark live objects >> - prepare forwardee >> - adjust pointers >> - compact >> >> When full gc occurs, there may be very high percentage of live bytes in some regions. For these regions, it's not efficient to compact them and better to skip them, as there are little space to save but many objects to copy. >> >> Description >> ----------- >> >> We enhance the full gc implementation for the above situation through following steps: >> - accumulate live bytes of every hr in mark phase; (already done by JDK-8263495) >> - skip adding regions with high survial ratio, and set the region with high survival ratio as pinned in _region_attr_table during prepare phase; >> - nothing special is done in adjust phase, regions with high survial ratio are skipped because of pin setting in the above step; >> - nothing special is done in compact phase, regions with high survival ratio are skipped because these regions are skipped when adding regions to compaction set in the prepare phase; >> >> VM options related >> ----------- >> >> - MarkSweepDeadRatio: we reuse this exising vm option to indicate the high survial ratio threhold (100-MarkSweepDeadRatio) in G1. >> - default value of MarkSweepDeadRatio: 5 >> >> Test >> ----------- >> >> - specjbb2015: no regression >> - dacapo: (Attachment is the dacapo h2 full gc pause.) >> - 95% of full gc pauses: 10%-19% improvement. >> - 5% of full gc pauses: 1.2% improvement. >> - 0.1% of full gc pauses: -6.16% improvement. >> >> $ java -Xmx1g -Xms1g -XX:ParallelGCThreads=4 -Xlog:gc*=info:file=gc.log -jar dacapo-9.12-bach.jar --iterations 5 --size huge --no-pre-iteration-gc h2 > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > minor code improvement. src/hotspot/share/gc/g1/g1FullCollector.cpp line 229: > 227: > 228: void G1FullCollector::update_attribute_table(HeapRegion* hr) { > 229: if (hr->is_free()) { Another item that has been noted in a recent discussion with @albertnetymk is that with this change "Free" regions are also marked as `normal` in the table. It would be better to keep them as "Invalid". I.e. something like (incorporating @albertnetymk other suggestion): if (hr->is_free()) { return; } else if (hr->is_closed_archive(...) { [...] } else if (hr->is_pinned() || force_pinned) { [...] } else { [...] } There is no real difference as "Free" regions should never be referenced anywhere and the code should assert elsewhere. It's still nice to also have "Free" regions as `Invalid` in that table though. ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From ayang at openjdk.java.net Wed Apr 7 11:49:56 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Wed, 7 Apr 2021 11:49:56 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v13] In-Reply-To: References: <47rveNWYV7e6_mdxd3EaiTdsxo1SEjtYwmVviLxVo80=.17d8b0ce-4afd-418e-b2ea-e6099a3e7199@github.com> Message-ID: On Wed, 7 Apr 2021 10:29:26 GMT, Thomas Schatzl wrote: >> src/hotspot/share/gc/g1/heapRegion.hpp line 174: >> >>> 172: void reset_compacted_after_full_gc(); >>> 173: // Update pinned heap region (not compacted) to be consistent after Full GC. >>> 174: void reset_not_compacted_after_full_gc(); >> >> Now that this version uses the existing "pinned" mechanism to skip high-live-ratio regions, this method can retain its original name/implementation, right? > > That is true, full gc reuses the mechanism originally implemented for pinned regions. However "pinned" is something very specific in G1 context so I think it is better to use a different, more generic name. The regions that are not compacted (and not yet pinned) are not really temporarily pinned. > > There has been an earlier discussion (not sure if here in this PR) to actually rename the use of "pinned" in G1 full gc to use "not compacted" too, resulting in a CR to rename this (in [JDK-8264423](https://bugs.openjdk.java.net/browse/JDK-8264423)). > > However I was going to wait for this change to go in before sending out a PR. > > Also this "not_compacted" name better matches the "compacted" method name above. > > So I would prefer to keep this as is. I see; agree. ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From zgu at openjdk.java.net Wed Apr 7 12:11:18 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Wed, 7 Apr 2021 12:11:18 GMT Subject: RFR: 8264718: Shenandoah: enable string deduplication during root scanning [v2] In-Reply-To: References: Message-ID: > Shenandoah used to scan roots at pauses, so it deliberately disables string deduplication during root scanning to avoid extra pause times. > > Now, Shenandoah scans roots in concurrent phase, it is no longer a concern, we should enable it. > > Test: > - [x] hotspot_gc_shenandoah Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision: Removed excessive empty lines ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3348/files - new: https://git.openjdk.java.net/jdk/pull/3348/files/45344ee4..d3d63b38 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3348&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3348&range=00-01 Stats: 2 lines in 1 file changed: 0 ins; 2 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3348.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3348/head:pull/3348 PR: https://git.openjdk.java.net/jdk/pull/3348 From coleenp at openjdk.java.net Wed Apr 7 13:04:32 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Wed, 7 Apr 2021 13:04:32 GMT Subject: RFR: 8264682: MemProfiling does not own Heap_lock when using G1 In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 09:33:39 GMT, Yi Yang wrote: >> Unless we can imagine a reason why this code is needed, I think the old memprofiler should be removed. > > I'd like to propose a new PR to remove the memprofiler if there are no strong objections. Sure, open a new issue if you want, and link this issue to it. The "feature" was added here: ^As 00018/00000/00000 ^Ad D 1.1 98/04/14 13:18:32 renes 1 0 ^Ac date and time created 98/04/14 13:18:32 by renes ^Ae ------------- PR: https://git.openjdk.java.net/jdk/pull/3340 From shade at openjdk.java.net Wed Apr 7 16:28:36 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 7 Apr 2021 16:28:36 GMT Subject: RFR: 8264718: Shenandoah: enable string deduplication during root scanning [v2] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 12:11:18 GMT, Zhengyu Gu wrote: >> Shenandoah used to scan roots at pauses, so it deliberately disables string deduplication during root scanning to avoid extra pause times. >> >> Now, Shenandoah scans roots in concurrent phase, it is no longer a concern, we should enable it. >> >> Test: >> - [x] hotspot_gc_shenandoah > > Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision: > > Removed excessive empty lines Looks fine, with a minor nit. Fix it before push. src/hotspot/share/gc/shenandoah/shenandoahSTWMark.cpp line 132: > 130: void ShenandoahSTWMark::mark_roots(uint worker_id) { > 131: if (ShenandoahStringDedup::is_enabled()) { > 132: ShenandoahInitMarkRootsClosure init_mark(task_queues()->queue(worker_id)); Excess space here and on the other branch, between `` and `init_mark`. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3348 From zgu at openjdk.java.net Wed Apr 7 17:25:09 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Wed, 7 Apr 2021 17:25:09 GMT Subject: RFR: 8264718: Shenandoah: enable string deduplication during root scanning [v3] In-Reply-To: References: Message-ID: > Shenandoah used to scan roots at pauses, so it deliberately disables string deduplication during root scanning to avoid extra pause times. > > Now, Shenandoah scans roots in concurrent phase, it is no longer a concern, we should enable it. > > Test: > - [x] hotspot_gc_shenandoah Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision: Removed excess spaces ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3348/files - new: https://git.openjdk.java.net/jdk/pull/3348/files/d3d63b38..33862ac2 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3348&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3348&range=01-02 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3348.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3348/head:pull/3348 PR: https://git.openjdk.java.net/jdk/pull/3348 From manc at google.com Wed Apr 7 18:19:27 2021 From: manc at google.com (Man Cao) Date: Wed, 7 Apr 2021 11:19:27 -0700 Subject: Work-in-progress: 8236485: Epoch synchronization protocol for G1 concurrent refinement In-Reply-To: References: <87blr3vc4t.fsf@oldenburg2.str.redhat.com> Message-ID: Has anyone got a chance to take a look? I also implemented the fence removal in interpreter and C1/C2. The patch is quite small, so perhaps it is better to merge them into one pull request: https://github.com/caoman/jdk/tree/8226731fenceRemoval -Man On Tue, Mar 30, 2021 at 7:43 PM Man Cao wrote: > Hi all, > > I finally managed to allocate more time to make progress on this, and > resolved most issues since the last discussion. > I've updated the description in > https://bugs.openjdk.java.net/browse/JDK-8236485, and the current > prototype is the HEAD commit at > https://github.com/caoman/jdk/tree/g1EpochSync. > Notable changes include: > - The protocol uses async handshake from JDK-8238761 > to resolve the > blocking issue from normal handshake. > - In order to support async refinement due to async handshake, added > support for _deferred global queue to G1DirtyCardQueueSet. Buffers rarely > get enqueued to _deferred at run-time. > - The async handshake only executes for a subset of threads. > > I have a couple of questions: > > 1. Code review and patch size. > Should I start a pull request for this change, so it is easier to give > feedback? > > What is the recommended approach to deal with large changes? Currently the > patch is about 1200 lines, without changing the write barrier itself > (JDK-8226731). > Since pushing only this patch will add some overhead to refinement, but > not bring any performance improvement from removing the write barrier's > fence, do you recommend I also implement the write barrier change in the > same patch? > Keeping the epoch sync patch separate from the write barrier patch has > some benefit for testing, in case the epoch patch introduces any bugs. > Currently the new code is mostly guarded by a flag > -XX:+G1TestEpochSyncInConcRefinement, and it will be removed after the > write barrier change. It could be used as an emergency flag to workaround > bugs, instead of backing out of the entire change. We probably cannot have > such a flag if we bundle the changes in one patch (it's too ugly to have a > flag in interpreter and compilers). > > 2. Checking if a remote thread is in _thread_in_Java state. > eosterlund@ pointed out it was incorrect to do JavaThread::thread_state() > == _thread_in_Java. > I looked into thread state transition, and revised it to also compare with > _thread_in_native_trans and _thread_in_vm_trans. > I think it is correct now for the purpose of epoch synchronization. I.e., > it never "misses" a remote thread that is actually in in_Java state. > A detailed comment is here: > https://github.com/caoman/jdk/blob/2047ecefceb074e80d73e0d521d64a220fdc5779/src/hotspot/share/gc/g1/g1EpochSynchronizer.cpp#L67-L90 > . > *Erik, could you take a look and decide if it is correct? If it is still > incorrect, could you advise a proper way to do this?* > > 3. Native write barrier (G1BarrierSet::write_ref_field_post). > The epoch synchronization protocol does not synchronize with threads in > _thread_in_native or _thread_in_vm state. It is much slower if we > synchronize with such threads. > Moreover, there are non-Java threads (e.g. concurrent mark worker) that > could execute the native write barrier. > As a result, it's probably best to keep the StoreLoad fence in the native > write barrier. > The final write post-barrier for JDK-8226731 would be: > Given: > x.a = q > and > p = @x.a > > For Interpreter/C1/C2: > if (p and q in same regions or q == NULL) -> exit > if (card(p) == Dirty) -> exit > card(p) = Dirty; > enqueue(card(p)) > > For the native barrier: > if (p and q in same regions or q == NULL) -> exit > StoreLoad; > if (card(p) == Dirty) -> exit > card(p) = Dirty; > enqueue(card(p)) > > Does the above look good? Do we need to micro-benchmark the potential > overhead added to the native barrier? Or are typical macro-benchmarks > sufficient? > > 4. Performance benchmarking. > I did some preliminary benchmarking with DaCapo and BigRamTester, without > changing the write barrier. This is to measure the overhead added by the > epoch synchronization protocol. > Under the default JVM setting, I didn't see any performance difference. > When I tuned the JVM to refine cards aggressively, there was > still no difference in BigRamTester (probably because it only has 2 > threads). Some DaCapo benchmarks saw 10-15% more CPU usage due to doing > more work in the refinement threads, and 2-5% total throughput regression > for these benchmarks. > The aggressive refinement flags are "-XX:-G1UseAdaptiveConcRefinement > -XX:G1UpdateBufferSize=4 -XX:G1ConcRefinementGreenZone=0 > -XX:G1ConcRefinementYellowZone=1". > > I wonder how important we should treat the aggressive refinement case. > Regression in this case is likely unavoidable, so how much regression is > tolerable? > Also, does anyone know a better benchmark to test refinement with default > JVM settings? Ideally it (1) has many mutator threads; (2) triggers > concurrent refinement frequently; (3) runs with a sizable Xmx (e.g., 2GiB > or above). > > -Man > > > On Thu, Jan 16, 2020 at 4:06 AM Florian Weimer wrote: > >> * Man Cao: >> >> > We had an offline discussion on this. To keep the community in the loop, >> > here is what we discussed. >> > >> > a. Using Linux membarrier syscall or equivalent on other OSes seems a >> > cleaner solution than thread-local handshake (TLH). But we need to have >> a >> > backup mechanism for OSes and older Linuxes that do not have such a >> > syscall. >> >> Can you do with a membarrier call that doesn't require registration? >> >> The usual fallback for membarrier is sending a special signal to all >> threads, and make sure that they have run code in a signal handler >> (possibly using a CPU barrier there). But of course this is rather >> slow. >> >> membarrier has seen some backporting activity, but as far as I can see, >> that hasn't been consistent across architectures. >> >> Thanks, >> Florian >> >> From kbarrett at openjdk.java.net Wed Apr 7 19:31:15 2021 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 7 Apr 2021 19:31:15 GMT Subject: RFR: 8264424: Support OopStorage bulk allocation [v2] In-Reply-To: References: Message-ID: > Please review this change to OopStorage to support bulk allocation. A new overload for allocate is provided: > size_t allocate(oop** entries, size_t size) > The approach taken is to claim all of the available entries in the first available block, add as many of those entries as will fit in the buffer, and release any remaining entries back to the block. Only the claim part needs to be done while holding the allocation mutex. This is is optimized for clients that want more than just a couple entries, and minimizes time under the lock. The maximum number of entries (the number of entries in a block) is provided as a constant for sizing requests, to avoid the release of unrequested entries. An application that wants more than that, or a specific number that might not be available from the next block, needs to make multiple bulk allocation calls, but that's still going to be much faster than one at a time allocations. > > Testing: > mach5 tier1 > New gtest for bulk allocation. > I've been using this for a while as part of another in-development change that makes heavy use of this feature. Kim Barrett has updated the pull request incrementally with one additional commit since the last revision: improve comments per ayang ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3264/files - new: https://git.openjdk.java.net/jdk/pull/3264/files/3ac5fec7..a0c7f91c Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3264&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3264&range=00-01 Stats: 8 lines in 2 files changed: 1 ins; 1 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/3264.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3264/head:pull/3264 PR: https://git.openjdk.java.net/jdk/pull/3264 From kbarrett at openjdk.java.net Wed Apr 7 19:31:16 2021 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 7 Apr 2021 19:31:16 GMT Subject: RFR: 8264424: Support OopStorage bulk allocation [v2] In-Reply-To: References: Message-ID: <5fhtiPtFJqo7Js27D3sAdSl6leSgQIyt2ZfpurN3q_E=.9bbf6a12-69d3-43ab-afdb-ab3ac82c713c@github.com> On Tue, 6 Apr 2021 09:46:52 GMT, Albert Mingkun Yang wrote: >> Kim Barrett has updated the pull request incrementally with one additional commit since the last revision: >> >> improve comments per ayang > > src/hotspot/share/gc/shared/oopStorage.cpp line 478: > >> 476: block = block_for_allocation(); >> 477: if (block == NULL) return 0; // Block allocation failed. >> 478: // Take exclusive use of this block for allocation. > > Maybe "Taking all remaining entries, so remove from list."? The "exclusive use" is ensured by the above `MutexLocker`, not `unlink`. done > src/hotspot/share/gc/shared/oopStorage.hpp line 119: > >> 117: // Allocates multiple entries, returning them in the ptrs buffer. The >> 118: // number of entries that will be allocated is never more than the minimum >> 119: // of size and bulk_allocate_limit, but may be less than either. Possibly > > It took me a few seconds to understand this sentence. Maybe replace it with "postcondition: result <= min(size, bulk_allocate_limit)" below? done ------------- PR: https://git.openjdk.java.net/jdk/pull/3264 From iklam at openjdk.java.net Wed Apr 7 19:46:44 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Wed, 7 Apr 2021 19:46:44 GMT Subject: RFR: 8214455: Relocate CDS archived regions to the top of the G1 heap [v3] In-Reply-To: References: Message-ID: > When the runtime heap and dump time heap have different sizes, it's possible that the archived heap regions are mapped to the middle of the runtime heap. Because the archived heap regions are pinned, this could reduce the size of the largest "humongous" array allocation. In the worst case, the maximum allocatable array length may be half of the optimal value. > > The fix is to relocate the archived regions to the top of the G1 heap (currently archived regions are supported only by G1 on 64-bit JVM). > > Note that usually the top of the G1 heap is placed just below the 32GB boundary. As a result, the archived heap regions are at the same location between run time and dump time, so no relocation is necessary. > > In mach5 testing, we occasionally run into this problem (see https://bugs.openjdk.java.net/browse/JDK-8239089), probably because we fail to reserve the heap below 32GB due to address space layout randomization (ASLR). Ioi Lam has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: - Merge branch 'master' into 8214455-relocated-archived-regions-to-heap-top - fixed indentation (review comment by @calvinccheung) - @tschatzl review -- fix typos and whitespace - Added -Xlog:cds=debug in case this test fails again - 8214455: Relocate CDS archived regions to the top of the G1 heap ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3349/files - new: https://git.openjdk.java.net/jdk/pull/3349/files/bedd676d..43ee9680 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3349&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3349&range=01-02 Stats: 4956 lines in 137 files changed: 2977 ins; 1596 del; 383 mod Patch: https://git.openjdk.java.net/jdk/pull/3349.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3349/head:pull/3349 PR: https://git.openjdk.java.net/jdk/pull/3349 From kbarrett at openjdk.java.net Wed Apr 7 19:47:34 2021 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 7 Apr 2021 19:47:34 GMT Subject: RFR: 8264424: Support OopStorage bulk allocation [v3] In-Reply-To: References: Message-ID: > Please review this change to OopStorage to support bulk allocation. A new overload for allocate is provided: > size_t allocate(oop** entries, size_t size) > The approach taken is to claim all of the available entries in the first available block, add as many of those entries as will fit in the buffer, and release any remaining entries back to the block. Only the claim part needs to be done while holding the allocation mutex. This is is optimized for clients that want more than just a couple entries, and minimizes time under the lock. The maximum number of entries (the number of entries in a block) is provided as a constant for sizing requests, to avoid the release of unrequested entries. An application that wants more than that, or a specific number that might not be available from the next block, needs to make multiple bulk allocation calls, but that's still going to be much faster than one at a time allocations. > > Testing: > mach5 tier1 > New gtest for bulk allocation. > I've been using this for a while as part of another in-development change that makes heavy use of this feature. Kim Barrett has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: - Merge branch 'master' into bulk_alloc - improve comments per ayang - oopstorage bulk allocation ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3264/files - new: https://git.openjdk.java.net/jdk/pull/3264/files/a0c7f91c..6643db84 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3264&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3264&range=01-02 Stats: 26753 lines in 510 files changed: 20294 ins; 3517 del; 2942 mod Patch: https://git.openjdk.java.net/jdk/pull/3264.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3264/head:pull/3264 PR: https://git.openjdk.java.net/jdk/pull/3264 From kbarrett at openjdk.java.net Wed Apr 7 19:47:37 2021 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 7 Apr 2021 19:47:37 GMT Subject: RFR: 8264424: Support OopStorage bulk allocation [v3] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 09:51:47 GMT, Albert Mingkun Yang wrote: >> Kim Barrett has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision: >> >> - Merge branch 'master' into bulk_alloc >> - improve comments per ayang >> - oopstorage bulk allocation > > Marked as reviewed by ayang (Committer). Thanks for reviews @albertnetymk and @tschatzl ------------- PR: https://git.openjdk.java.net/jdk/pull/3264 From kbarrett at openjdk.java.net Wed Apr 7 19:47:38 2021 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Wed, 7 Apr 2021 19:47:38 GMT Subject: Integrated: 8264424: Support OopStorage bulk allocation In-Reply-To: References: Message-ID: On Tue, 30 Mar 2021 11:55:59 GMT, Kim Barrett wrote: > Please review this change to OopStorage to support bulk allocation. A new overload for allocate is provided: > size_t allocate(oop** entries, size_t size) > The approach taken is to claim all of the available entries in the first available block, add as many of those entries as will fit in the buffer, and release any remaining entries back to the block. Only the claim part needs to be done while holding the allocation mutex. This is is optimized for clients that want more than just a couple entries, and minimizes time under the lock. The maximum number of entries (the number of entries in a block) is provided as a constant for sizing requests, to avoid the release of unrequested entries. An application that wants more than that, or a specific number that might not be available from the next block, needs to make multiple bulk allocation calls, but that's still going to be much faster than one at a time allocations. > > Testing: > mach5 tier1 > New gtest for bulk allocation. > I've been using this for a while as part of another in-development change that makes heavy use of this feature. This pull request has now been integrated. Changeset: 22b20f8e Author: Kim Barrett URL: https://git.openjdk.java.net/jdk/commit/22b20f8e Stats: 132 lines in 4 files changed: 118 ins; 1 del; 13 mod 8264424: Support OopStorage bulk allocation Reviewed-by: ayang, tschatzl ------------- PR: https://git.openjdk.java.net/jdk/pull/3264 From iklam at openjdk.java.net Thu Apr 8 00:44:14 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Thu, 8 Apr 2021 00:44:14 GMT Subject: RFR: 8214455: Relocate CDS archived regions to the top of the G1 heap [v3] In-Reply-To: References: Message-ID: <7FUdIE0NHT29sCEmGSXe0FYD_99Xq0HpO6JP5uLoJpA=.68ec144f-4d34-48c0-a6d2-89760d2e7617@github.com> On Tue, 6 Apr 2021 09:41:25 GMT, Thomas Schatzl wrote: >> Ioi Lam has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains five additional commits since the last revision: >> >> - Merge branch 'master' into 8214455-relocated-archived-regions-to-heap-top >> - fixed indentation (review comment by @calvinccheung) >> - @tschatzl review -- fix typos and whitespace >> - Added -Xlog:cds=debug in case this test fails again >> - 8214455: Relocate CDS archived regions to the top of the G1 heap > > Lgtm. Thanks @tschatzl and @calvinccheung for the review. ------------- PR: https://git.openjdk.java.net/jdk/pull/3349 From iklam at openjdk.java.net Thu Apr 8 00:49:20 2021 From: iklam at openjdk.java.net (Ioi Lam) Date: Thu, 8 Apr 2021 00:49:20 GMT Subject: Integrated: 8214455: Relocate CDS archived regions to the top of the G1 heap In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 21:48:43 GMT, Ioi Lam wrote: > When the runtime heap and dump time heap have different sizes, it's possible that the archived heap regions are mapped to the middle of the runtime heap. Because the archived heap regions are pinned, this could reduce the size of the largest "humongous" array allocation. In the worst case, the maximum allocatable array length may be half of the optimal value. > > The fix is to relocate the archived regions to the top of the G1 heap (currently archived regions are supported only by G1 on 64-bit JVM). > > Note that usually the top of the G1 heap is placed just below the 32GB boundary. As a result, the archived heap regions are at the same location between run time and dump time, so no relocation is necessary. > > In mach5 testing, we occasionally run into this problem (see https://bugs.openjdk.java.net/browse/JDK-8239089), probably because we fail to reserve the heap below 32GB due to address space layout randomization (ASLR). This pull request has now been integrated. Changeset: 78d1164c Author: Ioi Lam URL: https://git.openjdk.java.net/jdk/commit/78d1164c Stats: 147 lines in 8 files changed: 139 ins; 0 del; 8 mod 8214455: Relocate CDS archived regions to the top of the G1 heap Reviewed-by: tschatzl, ccheung ------------- PR: https://git.openjdk.java.net/jdk/pull/3349 From ayang at openjdk.java.net Thu Apr 8 09:05:14 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Thu, 8 Apr 2021 09:05:14 GMT Subject: RFR: 8264788: Make SequentialSubTasksDone use-once In-Reply-To: References: Message-ID: <0QDQXiibSb7nKsN3Cjf1dsIkX4tlr6Wc7nhKfpS4DlE=.5e8189c8-3926-4061-a148-aba01bf23bea@github.com> On Wed, 7 Apr 2021 08:32:11 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that simplifies code ahead? > > SequentialSubTasksDone has some machinery to reset itself automatically after all tasks were claimed. > > This is not used at all and can/should be removed. > > Test: tier1-3 Looking at how `SequentialSubTasksDone` is used, I don't really see the benefit of having `assert(_num_claimed == _num_tasks)` in the destructor. If we drop that assertion, `Atomic::add` is more readable. I prefer `Atomic::add` with the assertion removed, but this version is fine as well. ------------- Marked as reviewed by ayang (Committer). PR: https://git.openjdk.java.net/jdk/pull/3372 From mli at openjdk.java.net Thu Apr 8 10:06:54 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Thu, 8 Apr 2021 10:06:54 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v14] In-Reply-To: References: Message-ID: <2wLngv3n2wALz1ZbNgSTNaMKMYKTnh6ISlWEdNzBa9o=.f0996e4c-3bd5-42b3-9cc0-0ec8a600f1dc@github.com> > Summary > ----------- > > Improve G1 Full GC by skip compaction for regions with high survival ratio. > > Backgroud > ----------- > > There are 4 steps in full gc of G1 GC. > - mark live objects > - prepare forwardee > - adjust pointers > - compact > > When full gc occurs, there may be very high percentage of live bytes in some regions. For these regions, it's not efficient to compact them and better to skip them, as there are little space to save but many objects to copy. > > Description > ----------- > > We enhance the full gc implementation for the above situation through following steps: > - accumulate live bytes of every hr in mark phase; (already done by JDK-8263495) > - skip adding regions with high survial ratio, and set the region with high survival ratio as pinned in _region_attr_table during prepare phase; > - nothing special is done in adjust phase, regions with high survial ratio are skipped because of pin setting in the above step; > - nothing special is done in compact phase, regions with high survival ratio are skipped because these regions are skipped when adding regions to compaction set in the prepare phase; > > VM options related > ----------- > > - MarkSweepDeadRatio: we reuse this exising vm option to indicate the high survial ratio threhold (100-MarkSweepDeadRatio) in G1. > - default value of MarkSweepDeadRatio: 5 > > Test > ----------- > > - specjbb2015: no regression > - dacapo: (Attachment is the dacapo h2 full gc pause.) > - 95% of full gc pauses: 10%-19% improvement. > - 5% of full gc pauses: 1.2% improvement. > - 0.1% of full gc pauses: -6.16% improvement. > > $ java -Xmx1g -Xms1g -XX:ParallelGCThreads=4 -Xlog:gc*=info:file=gc.log -jar dacapo-9.12-bach.jar --iterations 5 --size huge --no-pre-iteration-gc h2 Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: add sanity check; refine code in update_attribute_table. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/2760/files - new: https://git.openjdk.java.net/jdk/pull/2760/files/63ab0f9c..5cca26b5 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2760&range=13 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2760&range=12-13 Stats: 12 lines in 3 files changed: 5 ins; 2 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/2760.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2760/head:pull/2760 PR: https://git.openjdk.java.net/jdk/pull/2760 From mli at openjdk.java.net Thu Apr 8 10:06:58 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Thu, 8 Apr 2021 10:06:58 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v14] In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 10:33:32 GMT, Thomas Schatzl wrote: >> Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: >> >> add sanity check; refine code in update_attribute_table. > > src/hotspot/share/gc/g1/g1FullCollector.cpp line 229: > >> 227: >> 228: void G1FullCollector::update_attribute_table(HeapRegion* hr) { >> 229: if (hr->is_free()) { > > Another item that has been noted in a recent discussion with @albertnetymk is that with this change "Free" regions are also marked as `normal` in the table. It would be better to keep them as "Invalid". > > I.e. something like (incorporating @albertnetymk other suggestion): > > if (hr->is_free()) { > return; > } else if (hr->is_closed_archive(...) { > [...] > } else if (hr->is_pinned() || force_pinned) { > [...] > } else { > [...] > } > > There is no real difference as "Free" regions should never be referenced anywhere and the code should assert elsewhere. It's still nice to also have "Free" regions as `Invalid` in that table though. not exactly. please check previous discussion at https://github.com/openjdk/jdk/pull/2760#discussion_r602144952. ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From mli at openjdk.java.net Thu Apr 8 10:07:01 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Thu, 8 Apr 2021 10:07:01 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v13] In-Reply-To: <47rveNWYV7e6_mdxd3EaiTdsxo1SEjtYwmVviLxVo80=.17d8b0ce-4afd-418e-b2ea-e6099a3e7199@github.com> References: <47rveNWYV7e6_mdxd3EaiTdsxo1SEjtYwmVviLxVo80=.17d8b0ce-4afd-418e-b2ea-e6099a3e7199@github.com> Message-ID: On Wed, 7 Apr 2021 09:35:19 GMT, Albert Mingkun Yang wrote: >> Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: >> >> minor code improvement. > > src/hotspot/share/gc/g1/g1FullCollector.cpp line 238: > >> 236: if (hr->is_closed_archive()) { >> 237: _region_attr_table.set_closed_archive(hr->hrm_index()); >> 238: } else if (hr->is_pinned()) { > > Changing the condition to `hr->is_pinned() || force_pinned` is enough for this method, right? not exactly. please check previous discussion at https://github.com/openjdk/jdk/pull/2760#discussion_r602144952. I will merge the 2 conditions as you suggested. ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From tschatzl at openjdk.java.net Thu Apr 8 11:01:25 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 8 Apr 2021 11:01:25 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v14] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 10:02:49 GMT, Hamlin Li wrote: >> src/hotspot/share/gc/g1/g1FullCollector.cpp line 229: >> >>> 227: >>> 228: void G1FullCollector::update_attribute_table(HeapRegion* hr) { >>> 229: if (hr->is_free()) { >> >> Another item that has been noted in a recent discussion with @albertnetymk is that with this change "Free" regions are also marked as `normal` in the table. It would be better to keep them as "Invalid". >> >> I.e. something like (incorporating @albertnetymk other suggestion): >> >> if (hr->is_free()) { >> return; >> } else if (hr->is_closed_archive(...) { >> [...] >> } else if (hr->is_pinned() || force_pinned) { >> [...] >> } else { >> [...] >> } >> >> There is no real difference as "Free" regions should never be referenced anywhere and the code should assert elsewhere. It's still nice to also have "Free" regions as `Invalid` in that table though. > > not exactly. please check previous discussion at https://github.com/openjdk/jdk/pull/2760#discussion_r602144952. Some more investigation and discussion about the BOT handling showed that we need to update the BOT for these Survivor-turned-to-Old regions after all. The reason is that contrary to what I thought, while BOT can handle queries for object start addresses above the "last known valid entry" mentioned in PR#3356, it only does so slowly, and while updating the BOT itself, not updating that "last known valid entry". So every time it queries for an object start in such regions, it starts walking from the bottom of that region. See the call chain `G1BlockOffsetTablePart::block_start` -> `forward_to_block_containing_addr` -> `forward_to_block_containing_addr_slow` where the call to `alloc_block_work` in `g1BlockOffsetTable.cpp:236` only updates local boundary and index (not `_next_offset_threshold` and `_next_offset_index`). This is a problem for young gcs as this behavior will make them slower than expected. Since it is impossible to make the updates to both `_next_offset_threshold` and `_next_offset_index` atomic, and another issue anyway) I would prefer to penalize full gc (or at least keep old behavior) for that. Could you add code that walks such full young regions and does the `cross_threshold` thing? It certainly does not need to actually compact these regions. Thanks, Thomas ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From zgu at openjdk.java.net Thu Apr 8 12:13:25 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Thu, 8 Apr 2021 12:13:25 GMT Subject: Integrated: 8264718: Shenandoah: enable string deduplication during root scanning In-Reply-To: References: Message-ID: <_5JKrTptBZMdek4mggI4ovb80P9BdSBcGmMtZDXozek=.0366e58b-1d22-4bb5-86b0-afb390e4a88a@github.com> On Mon, 5 Apr 2021 21:42:32 GMT, Zhengyu Gu wrote: > Shenandoah used to scan roots at pauses, so it deliberately disables string deduplication during root scanning to avoid extra pause times. > > Now, Shenandoah scans roots in concurrent phase, it is no longer a concern, we should enable it. > > Test: > - [x] hotspot_gc_shenandoah This pull request has now been integrated. Changeset: 3aec2d96 Author: Zhengyu Gu URL: https://git.openjdk.java.net/jdk/commit/3aec2d96 Stats: 69 lines in 5 files changed: 39 ins; 26 del; 4 mod 8264718: Shenandoah: enable string deduplication during root scanning Reviewed-by: shade ------------- PR: https://git.openjdk.java.net/jdk/pull/3348 From mli at openjdk.java.net Thu Apr 8 13:47:29 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Thu, 8 Apr 2021 13:47:29 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v14] In-Reply-To: References: Message-ID: <3qu7jf8dJ_jTlNcs_gzjXE_k-ylkQTnVveXoMBUqhAA=.3e22c73a-98cd-4f46-a663-a7cc5bacd276@github.com> On Thu, 8 Apr 2021 10:58:04 GMT, Thomas Schatzl wrote: >> not exactly. please check previous discussion at https://github.com/openjdk/jdk/pull/2760#discussion_r602144952. > > Some more investigation and discussion about the BOT handling showed that we need to update the BOT for these Survivor-turned-to-Old regions after all. > > The reason is that contrary to what I thought, while BOT can handle queries for object start addresses above the "last known valid entry" (materialized in `_next_offset_threshold` and `_next_offset_index` ) mentioned in PR#3356, it only does so slowly, and while updating the BOT itself, not updating that "last known valid entry". > So every time it queries for an object start in such regions, it starts walking from the bottom of that region. > > See the call chain `G1BlockOffsetTablePart::block_start` -> `forward_to_block_containing_addr` -> `forward_to_block_containing_addr_slow` where the call to `alloc_block_work` in `g1BlockOffsetTable.cpp:236` only updates local boundary and index (not `_next_offset_threshold` and `_next_offset_index`). > > This is a problem for young gcs as this behavior will make them slower than expected. Since it is impossible to make the updates to both `_next_offset_threshold` and `_next_offset_index` atomic, and another issue anyway; feel free to file an issue) I would prefer to penalize full gc (that is, keep old behavior) for that. > > The alternative would be to just `alloc_block` the whole region (so that `next_offset_index` and friend are at the top), but I think given the rarity of this case and full gc in general it is better to do the extra work in the full gc. > > Could you add code that walks such full young regions and does the `cross_threshold` thing? This additional code certainly does not need to actually compact these regions. > > Thanks, > Thomas Thanks for discussion! I had the same concern (BOTs of these Survivor-turned-to-Old regions contains no useful info, so might make it very slow when finding block start in these regions), but I was not sure if there is some "lazy" mechanism to fill valid BOTs info for these regions when they are accessed subsequently. I just not have time to do further investigation, now I got the answer from you. Sure I will add code to fill valid BOTs info for these young regions by the end of full gc. I'm not sure if it's OK for me to do this BOTs filling action in a separate issue? As we already had such a long discussion, I think it's might be better for us to initialize another discussion for this specific follow-up issue. Please kindly let me know your thoughts. ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From tschatzl at openjdk.java.net Thu Apr 8 15:05:42 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 8 Apr 2021 15:05:42 GMT Subject: RFR: 8264788: Make SequentialSubTasksDone use-once [v2] In-Reply-To: References: Message-ID: > Hi all, > > can I have reviews for this change that simplifies code ahead? > > SequentialSubTasksDone has some machinery to reset itself automatically after all tasks were claimed. > > This is not used at all and can/should be removed. > > Test: tier1-3 Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: ayang review ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3372/files - new: https://git.openjdk.java.net/jdk/pull/3372/files/53930970..919ff6e7 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3372&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3372&range=00-01 Stats: 11 lines in 2 files changed: 1 ins; 4 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/3372.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3372/head:pull/3372 PR: https://git.openjdk.java.net/jdk/pull/3372 From tschatzl at openjdk.java.net Thu Apr 8 15:05:42 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 8 Apr 2021 15:05:42 GMT Subject: RFR: 8264788: Make SequentialSubTasksDone use-once [v2] In-Reply-To: <0QDQXiibSb7nKsN3Cjf1dsIkX4tlr6Wc7nhKfpS4DlE=.5e8189c8-3926-4061-a148-aba01bf23bea@github.com> References: <0QDQXiibSb7nKsN3Cjf1dsIkX4tlr6Wc7nhKfpS4DlE=.5e8189c8-3926-4061-a148-aba01bf23bea@github.com> Message-ID: On Thu, 8 Apr 2021 09:02:20 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> ayang review > > Looking at how `SequentialSubTasksDone` is used, I don't really see the benefit of having `assert(_num_claimed == _num_tasks)` in the destructor. If we drop that assertion, `Atomic::add` is more readable. > > I prefer `Atomic::add` with the assertion removed, but this version is fine as well. I changed to the `Atomic::add` but kept the assert (modified). ------------- PR: https://git.openjdk.java.net/jdk/pull/3372 From yyang at openjdk.java.net Fri Apr 9 07:18:24 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Fri, 9 Apr 2021 07:18:24 GMT Subject: Withdrawn: 8264682: MemProfiling does not own Heap_lock when using G1 In-Reply-To: References: Message-ID: <22s7OOLgMBFF1tkEbH8vNgr7fuOFcPjNJNjn-U7t5CI=.ab0ee64e-1e2e-4d9a-b734-451d61f71f6a@github.com> On Mon, 5 Apr 2021 07:34:30 GMT, Yi Yang wrote: > Trivial fix for JDK-8264682. > > `Universe::heap()->used()` calls G1Allocator::used_in_alloc_regions() when G1 enabled, it checks whether Heap_lock was owned on this thread's behalf. This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.java.net/jdk/pull/3340 From tschatzl at openjdk.java.net Fri Apr 9 10:53:20 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 9 Apr 2021 10:53:20 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v14] In-Reply-To: <3qu7jf8dJ_jTlNcs_gzjXE_k-ylkQTnVveXoMBUqhAA=.3e22c73a-98cd-4f46-a663-a7cc5bacd276@github.com> References: <3qu7jf8dJ_jTlNcs_gzjXE_k-ylkQTnVveXoMBUqhAA=.3e22c73a-98cd-4f46-a663-a7cc5bacd276@github.com> Message-ID: On Thu, 8 Apr 2021 13:44:35 GMT, Hamlin Li wrote: >> Some more investigation and discussion about the BOT handling showed that we need to update the BOT for these Survivor-turned-to-Old regions after all. >> >> The reason is that contrary to what I thought, while BOT can handle queries for object start addresses above the "last known valid entry" (materialized in `_next_offset_threshold` and `_next_offset_index` ) mentioned in PR#3356, it only does so slowly, and while updating the BOT itself, not updating that "last known valid entry". >> So every time it queries for an object start in such regions, it starts walking from the bottom of that region. >> >> See the call chain `G1BlockOffsetTablePart::block_start` -> `forward_to_block_containing_addr` -> `forward_to_block_containing_addr_slow` where the call to `alloc_block_work` in `g1BlockOffsetTable.cpp:236` only updates local boundary and index (not `_next_offset_threshold` and `_next_offset_index`). >> >> This is a problem for young gcs as this behavior will make them slower than expected. Since it is impossible to make the updates to both `_next_offset_threshold` and `_next_offset_index` atomic, and another issue anyway; feel free to file an issue) I would prefer to penalize full gc (that is, keep old behavior) for that. >> >> The alternative would be to just `alloc_block` the whole region (so that `next_offset_index` and friend are at the top), but I think given the rarity of this case and full gc in general it is better to do the extra work in the full gc. >> >> Could you add code that walks such full young regions and does the `cross_threshold` thing? This additional code certainly does not need to actually compact these regions. >> >> Thanks, >> Thomas > > Thanks for discussion! I had the same concern (BOTs of these Survivor-turned-to-Old regions contains no useful info, so might make it very slow when finding block start in these regions), but I was not sure if there is some "lazy" mechanism to fill valid BOTs info for these regions when they are accessed subsequently. I just not have time to do further investigation, now I got the answer from you. > > Sure I will add code to fill valid BOTs info for these young regions by the end of full gc. > I'm not sure if it's OK for me to do this BOTs filling action in a separate issue? As we already had such a long discussion, I think it's might be better for us to initialize another discussion for this specific follow-up issue. Please kindly let me know your thoughts. It is fine with me to fix this separately. There is some lazy mechanism to fill this BOT, but it does not work in this case (which is the gist of what I said above). ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From akozlov at azul.com Fri Apr 9 11:35:34 2021 From: akozlov at azul.com (Anton Kozlov) Date: Fri, 9 Apr 2021 14:35:34 +0300 Subject: [jdk13u-dev] RFR: 8264640: CMS ParScanClosure misses a barrier In-Reply-To: <1512B9D3-78B4-4AB1-8B4C-D2BAA6344EFC@azul.com> References: <1512B9D3-78B4-4AB1-8B4C-D2BAA6344EFC@azul.com> Message-ID: John, thank you for review and the comment! Thanks, Anton On 4/8/21 9:33 PM, John Cuthbertson wrote: > Hi Anton, > > This looks good to me. I think I?m still a reviewer for the jdk-updates project. > > For the benefit of everyone else... > > We were seeing this as a crash when obtaining the size of an object to be copied. The klass was observed to be transiently NULL. We found that the object, reached through another reference path, had already been copied and the from-space oop placed on the task queue for subsequent reference field scanning. The task queue, however, had overflowed and the from-space oop was placed on the shared overflow queue where objects are chained together through their klass field. If the reads are ordered as they are in the code then everything is OK as per the comment at line 105 (in ParScanClosure::do_oop_work) but we found that gcc had reordered the reads in the non-compressed oops case. So the mark word is read and the object is observed to not forwarded (yet). Then, via another reference path, the object is copied, forwarded, and placed on the overflow task queue ? over writing the from-space object?s klass. Then in the original path the klass is read and observed to be NULL or the next overflow entry ? leading to the crash. When the from-space oop is dequeued, its klass is restored ? which is what was observed in the core file. > > Using worker thread local queues, -XX:+ParGCUseLocalOverflow, seems to workaround the problem. > > Thanks, > > John Cuthbertson > >> On Apr 2, 2021, at 2:02 AM, Anton Kozlov wrote: >> >> Adding hotspot-gc-dev. It will be great to receive comments from GC experts, even the fix does not make sense for mainline jdk. >> >> Thanks, >> Anton >> >> On 4/2/21 11:51 AM, Anton Kozlov wrote: >>> Hi, please review an original fix for a GC crash. The jdk13u is the latest supported version that still has buggy code, it was deleted in jdk14 as a part of JEP 363: Remove the Concurrent Mark Sweep (CMS) Garbage Collector. So I'm proposing it here. >>> The fix is low-risk, on x86-64 it just introduces a compiler barrier to prevent two reads to be reordered as intended by surrounding comments. On CPUs with weaker memory models it introduces CPU barriers as well. >>> ------------- >>> Commit messages: >>> - Add missing barriers >>> Changes: https://git.openjdk.java.net/jdk13u-dev/pull/165/files >>> Webrev: https://webrevs.openjdk.java.net/?repo=jdk13u-dev&pr=165&range=00 >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8264640 >>> Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod >>> Patch: https://git.openjdk.java.net/jdk13u-dev/pull/165.diff >>> Fetch: git fetch https://git.openjdk.java.net/jdk13u-dev pull/165/head:pull/165 >>> PR: https://git.openjdk.java.net/jdk13u-dev/pull/165 > From shade at openjdk.java.net Fri Apr 9 12:01:24 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 9 Apr 2021 12:01:24 GMT Subject: RFR: 8264727: Remove extraneous whitespace from phase timings report In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 16:45:43 GMT, earthling-amzn wrote: > This extra space doesn't look like it was intended to improve any alignment of text in the report. @earthling-amzn Still want to integrate this? You need to title this PR correctly for bots to catch it up. ------------- PR: https://git.openjdk.java.net/jdk/pull/3342 From tschatzl at openjdk.java.net Fri Apr 9 12:29:28 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 9 Apr 2021 12:29:28 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v14] In-Reply-To: References: <3qu7jf8dJ_jTlNcs_gzjXE_k-ylkQTnVveXoMBUqhAA=.3e22c73a-98cd-4f46-a663-a7cc5bacd276@github.com> Message-ID: On Fri, 9 Apr 2021 10:49:11 GMT, Thomas Schatzl wrote: >> Thanks for discussion! I had the same concern (BOTs of these Survivor-turned-to-Old regions contains no useful info, so might make it very slow when finding block start in these regions), but I was not sure if there is some "lazy" mechanism to fill valid BOTs info for these regions when they are accessed subsequently. I just not have time to do further investigation, now I got the answer from you. >> >> Sure I will add code to fill valid BOTs info for these young regions by the end of full gc. >> I'm not sure if it's OK for me to do this BOTs filling action in a separate issue? As we already had such a long discussion, I think it's might be better for us to initialize another discussion for this specific follow-up issue. Please kindly let me know your thoughts. > > It is fine with me to fix this separately. > > There is some lazy mechanism to fill this BOT, but it does not work in this case (which is the gist of what I said above). Note that if that were fixed, I think PR #3356 would not be needed - because then young region BOTs have the expected contents :) Your call. ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From mli at openjdk.java.net Fri Apr 9 12:48:20 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Fri, 9 Apr 2021 12:48:20 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v14] In-Reply-To: References: <3qu7jf8dJ_jTlNcs_gzjXE_k-ylkQTnVveXoMBUqhAA=.3e22c73a-98cd-4f46-a663-a7cc5bacd276@github.com> Message-ID: On Fri, 9 Apr 2021 12:26:35 GMT, Thomas Schatzl wrote: >> It is fine with me to fix this separately. >> >> There is some lazy mechanism to fill this BOT, but it does not work in this case (which is the gist of what I said above). > > Note that if that were fixed, I think PR #3356 would not be needed - because then young region BOTs have the expected contents :) Your call. Thanks Thomas! Although I'm not sure if pr #3356 is still necessary. But sure, I will add code to fill BOTs for "young" regions as a follow-up fix/enhanacement of this issue, I have just created an bug to track it https://bugs.openjdk.java.net/browse/JDK-8264987. So after this pr is approved and integrated I will initialize the pr for JDK-8264987. ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From github.com+71722661+earthling-amzn at openjdk.java.net Fri Apr 9 17:10:27 2021 From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn) Date: Fri, 9 Apr 2021 17:10:27 GMT Subject: Integrated: 8264727: Shenandoah: Remove extraneous whitespace from phase timings report In-Reply-To: References: Message-ID: On Mon, 5 Apr 2021 16:45:43 GMT, earthling-amzn wrote: > This extra space doesn't look like it was intended to improve any alignment of text in the report. This pull request has now been integrated. Changeset: ec31b3a1 Author: William Kemper Committer: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/ec31b3a1 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod 8264727: Shenandoah: Remove extraneous whitespace from phase timings report Reviewed-by: shade ------------- PR: https://git.openjdk.java.net/jdk/pull/3342 From zgu at openjdk.java.net Fri Apr 9 21:20:45 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Fri, 9 Apr 2021 21:20:45 GMT Subject: RFR: 8265012: Shenandoah: Backout JDK-8264718 Message-ID: <2dkdGG9JW1wdsKKluHOS33VAIne3WdE7xxUgIlfSQ2c=.65c62947-47cf-466c-b773-046afcff5fa6@github.com> It turns out that enquening string deduplication candidates during concurrent root scanning may result lock rank inversion between stack watermark lock and string dedup queue lock, if the scanning is triggered by stack watermark and dedup buffer happens to be full. Backout JDK-8264718 for now, will retry after Kim's string deduplication refactoring. Test: - [x] hotspot_gc_shenandoah ------------- Commit messages: - JDK-8265012 Changes: https://git.openjdk.java.net/jdk/pull/3423/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3423&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265012 Stats: 69 lines in 5 files changed: 26 ins; 39 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/3423.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3423/head:pull/3423 PR: https://git.openjdk.java.net/jdk/pull/3423 From shade at openjdk.java.net Sat Apr 10 05:54:21 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Sat, 10 Apr 2021 05:54:21 GMT Subject: RFR: 8265012: Shenandoah: Backout JDK-8264718 In-Reply-To: <2dkdGG9JW1wdsKKluHOS33VAIne3WdE7xxUgIlfSQ2c=.65c62947-47cf-466c-b773-046afcff5fa6@github.com> References: <2dkdGG9JW1wdsKKluHOS33VAIne3WdE7xxUgIlfSQ2c=.65c62947-47cf-466c-b773-046afcff5fa6@github.com> Message-ID: On Fri, 9 Apr 2021 21:14:22 GMT, Zhengyu Gu wrote: > It turns out that enquening string deduplication candidates during concurrent root scanning may result lock rank inversion between stack watermark lock and string dedup queue lock, if the scanning is triggered by stack watermark and dedup buffer happens to be full. > > Backout JDK-8264718 for now, will retry after Kim's string deduplication refactoring. > > Test: > - [x] hotspot_gc_shenandoah Looks fine. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3423 From ayang at openjdk.java.net Mon Apr 12 09:01:56 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Mon, 12 Apr 2021 09:01:56 GMT Subject: RFR: 8264783: G1 BOT verification should not verify beyond allocation threshold In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 13:59:12 GMT, Thomas Schatzl wrote: > The G1 BOT contains an allocation threshold which basically acts as a "last known valid entry" index for the per-region BOT (which are views on the global BOT table). > > Currently G1 BOT verification actually "verifies" the BOT within a region past that BOT index. > > This causes issues with young regions; actually there is already code that prevents their BOT verification. This is perfectly fine, allocations in young regions do not update the BOTs. > > With JDK-8262068/PR #2760 this existing filtering of young regions (such that their BOT is not verified) does not work because there may be young regions that are not compacted (so with a BOT that have that last known valid entry at the start of the region) are still labelled as old. > > This change proposes to not try to verify the BOT beyond the last known valid index (which is arguably not worth doing), which also covers the existing young filtering. > > There are alternatives that may work in particular for JDK-8262068: > a) always compact young regions (which recreates the BOT) > b) create a "dummy" BOT that spans the entire part of the region containing live objects > > However they were rejected by me because > option a) takes time and directly counters that optimization for no reason > option b) makes finding the start of an object within these regions slow (they are not refined when there is at least *some* bot) > > and finally I do not think there is much point in trying to be clever about areas in the BOT that are known to not contain useful values. > > Testing: tier1-3 Marked as reviewed by ayang (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3356 From ayang at openjdk.java.net Mon Apr 12 09:07:00 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Mon, 12 Apr 2021 09:07:00 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v14] In-Reply-To: <2wLngv3n2wALz1ZbNgSTNaMKMYKTnh6ISlWEdNzBa9o=.f0996e4c-3bd5-42b3-9cc0-0ec8a600f1dc@github.com> References: <2wLngv3n2wALz1ZbNgSTNaMKMYKTnh6ISlWEdNzBa9o=.f0996e4c-3bd5-42b3-9cc0-0ec8a600f1dc@github.com> Message-ID: On Thu, 8 Apr 2021 10:06:54 GMT, Hamlin Li wrote: >> Summary >> ----------- >> >> Improve G1 Full GC by skip compaction for regions with high survival ratio. >> >> Backgroud >> ----------- >> >> There are 4 steps in full gc of G1 GC. >> - mark live objects >> - prepare forwardee >> - adjust pointers >> - compact >> >> When full gc occurs, there may be very high percentage of live bytes in some regions. For these regions, it's not efficient to compact them and better to skip them, as there are little space to save but many objects to copy. >> >> Description >> ----------- >> >> We enhance the full gc implementation for the above situation through following steps: >> - accumulate live bytes of every hr in mark phase; (already done by JDK-8263495) >> - skip adding regions with high survial ratio, and set the region with high survival ratio as pinned in _region_attr_table during prepare phase; >> - nothing special is done in adjust phase, regions with high survial ratio are skipped because of pin setting in the above step; >> - nothing special is done in compact phase, regions with high survival ratio are skipped because these regions are skipped when adding regions to compaction set in the prepare phase; >> >> VM options related >> ----------- >> >> - MarkSweepDeadRatio: we reuse this exising vm option to indicate the high survial ratio threhold (100-MarkSweepDeadRatio) in G1. >> - default value of MarkSweepDeadRatio: 5 >> >> Test >> ----------- >> >> - specjbb2015: no regression >> - dacapo: (Attachment is the dacapo h2 full gc pause.) >> - 95% of full gc pauses: 10%-19% improvement. >> - 5% of full gc pauses: 1.2% improvement. >> - 0.1% of full gc pauses: -6.16% improvement. >> >> $ java -Xmx1g -Xms1g -XX:ParallelGCThreads=4 -Xlog:gc*=info:file=gc.log -jar dacapo-9.12-bach.jar --iterations 5 --size huge --no-pre-iteration-gc h2 > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > add sanity check; refine code in update_attribute_table. Marked as reviewed by ayang (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From tschatzl at openjdk.java.net Mon Apr 12 09:17:46 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 12 Apr 2021 09:17:46 GMT Subject: RFR: 8264783: G1 BOT verification should not verify beyond allocation threshold In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 01:16:05 GMT, Hamlin Li wrote: >> The G1 BOT contains an allocation threshold which basically acts as a "last known valid entry" index for the per-region BOT (which are views on the global BOT table). >> >> Currently G1 BOT verification actually "verifies" the BOT within a region past that BOT index. >> >> This causes issues with young regions; actually there is already code that prevents their BOT verification. This is perfectly fine, allocations in young regions do not update the BOTs. >> >> With JDK-8262068/PR #2760 this existing filtering of young regions (such that their BOT is not verified) does not work because there may be young regions that are not compacted (so with a BOT that have that last known valid entry at the start of the region) are still labelled as old. >> >> This change proposes to not try to verify the BOT beyond the last known valid index (which is arguably not worth doing), which also covers the existing young filtering. >> >> There are alternatives that may work in particular for JDK-8262068: >> a) always compact young regions (which recreates the BOT) >> b) create a "dummy" BOT that spans the entire part of the region containing live objects >> >> However they were rejected by me because >> option a) takes time and directly counters that optimization for no reason >> option b) makes finding the start of an object within these regions slow (they are not refined when there is at least *some* bot) >> >> and finally I do not think there is much point in trying to be clever about areas in the BOT that are known to not contain useful values. >> >> Testing: tier1-3 > > The change looks good to me. > Thanks Thomas! Thanks @Hamlin-Li @albertnetymk for your revies. ------------- PR: https://git.openjdk.java.net/jdk/pull/3356 From tschatzl at openjdk.java.net Mon Apr 12 09:17:46 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 12 Apr 2021 09:17:46 GMT Subject: Integrated: 8264783: G1 BOT verification should not verify beyond allocation threshold In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 13:59:12 GMT, Thomas Schatzl wrote: > The G1 BOT contains an allocation threshold which basically acts as a "last known valid entry" index for the per-region BOT (which are views on the global BOT table). > > Currently G1 BOT verification actually "verifies" the BOT within a region past that BOT index. > > This causes issues with young regions; actually there is already code that prevents their BOT verification. This is perfectly fine, allocations in young regions do not update the BOTs. > > With JDK-8262068/PR #2760 this existing filtering of young regions (such that their BOT is not verified) does not work because there may be young regions that are not compacted (so with a BOT that have that last known valid entry at the start of the region) are still labelled as old. > > This change proposes to not try to verify the BOT beyond the last known valid index (which is arguably not worth doing), which also covers the existing young filtering. > > There are alternatives that may work in particular for JDK-8262068: > a) always compact young regions (which recreates the BOT) > b) create a "dummy" BOT that spans the entire part of the region containing live objects > > However they were rejected by me because > option a) takes time and directly counters that optimization for no reason > option b) makes finding the start of an object within these regions slow (they are not refined when there is at least *some* bot) > > and finally I do not think there is much point in trying to be clever about areas in the BOT that are known to not contain useful values. > > Testing: tier1-3 This pull request has now been integrated. Changeset: e604320b Author: Thomas Schatzl URL: https://git.openjdk.java.net/jdk/commit/e604320b Stats: 3 lines in 2 files changed: 1 ins; 0 del; 2 mod 8264783: G1 BOT verification should not verify beyond allocation threshold Reviewed-by: mli, ayang ------------- PR: https://git.openjdk.java.net/jdk/pull/3356 From sjohanss at openjdk.java.net Mon Apr 12 10:18:40 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Mon, 12 Apr 2021 10:18:40 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v23] In-Reply-To: References: <_uIvdHdm5ptjZJ8gEw8AzBJ8PC-16GoEWUuKW5zAXrg=.36c2da1c-685b-4d4d-a4e5-73b3fc48b812@github.com> Message-ID: On Thu, 18 Mar 2021 12:47:40 GMT, Stefan Johansson wrote: >>> This would also give us to to think about a question I haven't made up my mind around. What will `LargePageSizeInBytes` mean after this change? Should the JVM use 1g pages (on a system where 2m i the default) even if `LargePageSizeInBytes` is not set? >> >> I see two valid scenarios: >> a) either use huge pages as best as possible; remove fine-grained control from user hands. So, if we have 1G pages, prefer those over 2M over 4K. Then, we could completely remove LargePageSizeInBytes. There is no need for this switch anymore. >> b) or keep the current behavior. In that case, UseLargePageSize means "use at most default huge page size" even if larger pages are available; and LargePageSizeInBytes can be used to override the large page size to use. >> >> (a): Its elegant, and efficiently uses system resources when available. But its an incompatible change, and the VM being grabby means we could end up using pages meant for a different process. >> (b): downward compatible. In a sense. With Marcus change, we break downward compatibility already: where "UseLargePageSizeInBytes" before meant "use that or nothing", it now means "use that or whatever smaller page sizes you find". >> >> I prefer (a), honestly. >> >> ..Thomas > >> > This would also give us to to think about a question I haven't made up my mind around. What will `LargePageSizeInBytes` mean after this change? Should the JVM use 1g pages (on a system where 2m i the default) even if `LargePageSizeInBytes` is not set? >> >> I see two valid scenarios: > > Me too. > >> a) either use huge pages as best as possible; remove fine-grained control from user hands. So, if we have 1G pages, prefer those over 2M over 4K. Then, we could completely remove LargePageSizeInBytes. There is no need for this switch anymore. > > I agree, preferably we can make it so that the upper layers can use something like `page_size_for_region*` and request a certain page size, but fall back to smaller ones. > >> b) or keep the current behavior. In that case, UseLargePageSize means "use at most default huge page size" even if larger pages are available; and LargePageSizeInBytes can be used to override the large page size to use. >> > > So it becomes more like a maximium value right? Or at least this is how I've thought about this second scenario. On a system with both 2M (the default size) and 1G pages available you would have to set `LargePageSizeInBytes=1g` to use the 1G pages, but the 2M could still be used for smaller mappings. > >> (a): Its elegant, and efficiently uses system resources when available. But its an incompatible change, and the VM being grabby means we could end up using pages meant for a different process. >> (b): downward compatible. In a sense. With Marcus change, we break downward compatibility already: where "UseLargePageSizeInBytes" before meant "use that or nothing", it now means "use that or whatever smaller page sizes you find". >> >> I prefer (a), honestly. >> > I would also prefer (a). > @kstefanj @tstuefe > > 1. Would (a) [removing fine grained control of large page [Huge Page] sizes with LargePageSizeInBytes and relying on automatic selection of sizes from available] require JEP process? > Not a JEP, but likely a CSR. If we change the meaning of `LargePageSizeInBytes` (or deprecate it) a CSR will be needed. > 2. Can we put LargePageSizeInBytes flag in DIAGNOSTIC or EXPERIMENTAL and set it otherwise to the largest available page size? This way we have the default (a) and if users want to change page size they can use diagnostic or experimental and use (b). Also this would lower the amount of change needed. I'm a bit concerned with removing LargePageSizeInBytes altogether. I don't know if some use cases would break or need more fine tuned control of page sizes. > Changing the flag to diagnostic or experimental would also require a CSR and I don't see any real benefit of doing that. > > It appears the comment in the flag supports a solution like [2] above, but we could change change to `max page size or 0 to let VM choose`: "Large page size (0 to let VM choose the page size)" > > Current LargePageSizeInBytes flag code: > > ``` > product(size_t, LargePageSizeInBytes, 0, \ > "Large page size (0 to let VM choose the page size)") \ > range(0, max_uintx) \ > ``` > Reading this I wonder if the easiest way forward would just be to keep it this way. If the flag is set, only that page size is used, if not all configured page sizes can be used. So if someone sets `LargePagesSizeInBytes` only that size will end up in the available page sizes map. I might need to think a bit more about this, but if there is no problems with that approach it would be quite nice. > I've incorporated #3073 into this change, as a good base to build on, this also seemed to be Stefan's optimal path for this change. Hopefully I can push #3073 soon, and then you can just merge master to get this code correctly. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From mli at openjdk.java.net Mon Apr 12 11:04:51 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Mon, 12 Apr 2021 11:04:51 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v13] In-Reply-To: References: Message-ID: On Tue, 6 Apr 2021 14:01:15 GMT, Thomas Schatzl wrote: >> Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: >> >> minor code improvement. > > This change should go in after PR#3356 which fixes this issue. I think it is a change that is worth pointing out and discussing separately - as you might see from the long description. @tschatzl Hi Thomas, I'm not sure if I'm ready to integrate this patch, would you mind to help confirm, or give some further comments? Thanks ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From iwalulya at openjdk.java.net Mon Apr 12 11:24:07 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Mon, 12 Apr 2021 11:24:07 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates Message-ID: Hi all, Please review this change that skips the rebuild remset phase and the associated updates after rebuild, If during Remark, G1 finds that there are no remembered sets to rebuild. Test: tier1-5 ------------- Commit messages: - remove trailing white space - Merge branch 'master' into JDK-8199407 - variable name refactor - variable name refactor - skip iterations that depend on total_selected_for_rebuild - skip the rebuild remembered set phase if remark did not select any regions for rebuilding Changes: https://git.openjdk.java.net/jdk/pull/3434/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3434&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8199407 Stats: 28 lines in 4 files changed: 19 ins; 1 del; 8 mod Patch: https://git.openjdk.java.net/jdk/pull/3434.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3434/head:pull/3434 PR: https://git.openjdk.java.net/jdk/pull/3434 From tschatzl at openjdk.java.net Mon Apr 12 12:30:38 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 12 Apr 2021 12:30:38 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v14] In-Reply-To: <2wLngv3n2wALz1ZbNgSTNaMKMYKTnh6ISlWEdNzBa9o=.f0996e4c-3bd5-42b3-9cc0-0ec8a600f1dc@github.com> References: <2wLngv3n2wALz1ZbNgSTNaMKMYKTnh6ISlWEdNzBa9o=.f0996e4c-3bd5-42b3-9cc0-0ec8a600f1dc@github.com> Message-ID: On Thu, 8 Apr 2021 10:06:54 GMT, Hamlin Li wrote: >> Summary >> ----------- >> >> Improve G1 Full GC by skip compaction for regions with high survival ratio. >> >> Backgroud >> ----------- >> >> There are 4 steps in full gc of G1 GC. >> - mark live objects >> - prepare forwardee >> - adjust pointers >> - compact >> >> When full gc occurs, there may be very high percentage of live bytes in some regions. For these regions, it's not efficient to compact them and better to skip them, as there are little space to save but many objects to copy. >> >> Description >> ----------- >> >> We enhance the full gc implementation for the above situation through following steps: >> - accumulate live bytes of every hr in mark phase; (already done by JDK-8263495) >> - skip adding regions with high survial ratio, and set the region with high survival ratio as pinned in _region_attr_table during prepare phase; >> - nothing special is done in adjust phase, regions with high survial ratio are skipped because of pin setting in the above step; >> - nothing special is done in compact phase, regions with high survival ratio are skipped because these regions are skipped when adding regions to compaction set in the prepare phase; >> >> VM options related >> ----------- >> >> - MarkSweepDeadRatio: we reuse this exising vm option to indicate the high survial ratio threhold (100-MarkSweepDeadRatio) in G1. >> - default value of MarkSweepDeadRatio: 5 >> >> Test >> ----------- >> >> - specjbb2015: no regression >> - dacapo: (Attachment is the dacapo h2 full gc pause.) >> - 95% of full gc pauses: 10%-19% improvement. >> - 5% of full gc pauses: 1.2% improvement. >> - 0.1% of full gc pauses: -6.16% improvement. >> >> $ java -Xmx1g -Xms1g -XX:ParallelGCThreads=4 -Xlog:gc*=info:file=gc.log -jar dacapo-9.12-bach.jar --iterations 5 --size huge --no-pre-iteration-gc h2 > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > add sanity check; refine code in update_attribute_table. Ready to integrate. ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From mli at openjdk.java.net Mon Apr 12 12:41:54 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Mon, 12 Apr 2021 12:41:54 GMT Subject: RFR: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio [v14] In-Reply-To: References: <2wLngv3n2wALz1ZbNgSTNaMKMYKTnh6ISlWEdNzBa9o=.f0996e4c-3bd5-42b3-9cc0-0ec8a600f1dc@github.com> Message-ID: On Mon, 12 Apr 2021 12:27:50 GMT, Thomas Schatzl wrote: >> Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: >> >> add sanity check; refine code in update_attribute_table. > > Ready to integrate. Thanks @tschatzl @kstefanj @albertnetymk for reviewing! ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From mli at openjdk.java.net Mon Apr 12 12:45:00 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Mon, 12 Apr 2021 12:45:00 GMT Subject: Integrated: JDK-8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio In-Reply-To: References: Message-ID: On Sat, 27 Feb 2021 06:30:55 GMT, Hamlin Li wrote: > Summary > ----------- > > Improve G1 Full GC by skip compaction for regions with high survival ratio. > > Backgroud > ----------- > > There are 4 steps in full gc of G1 GC. > - mark live objects > - prepare forwardee > - adjust pointers > - compact > > When full gc occurs, there may be very high percentage of live bytes in some regions. For these regions, it's not efficient to compact them and better to skip them, as there are little space to save but many objects to copy. > > Description > ----------- > > We enhance the full gc implementation for the above situation through following steps: > - accumulate live bytes of every hr in mark phase; (already done by JDK-8263495) > - skip adding regions with high survial ratio, and set the region with high survival ratio as pinned in _region_attr_table during prepare phase; > - nothing special is done in adjust phase, regions with high survial ratio are skipped because of pin setting in the above step; > - nothing special is done in compact phase, regions with high survival ratio are skipped because these regions are skipped when adding regions to compaction set in the prepare phase; > > VM options related > ----------- > > - MarkSweepDeadRatio: we reuse this exising vm option to indicate the high survial ratio threhold (100-MarkSweepDeadRatio) in G1. > - default value of MarkSweepDeadRatio: 5 > > Test > ----------- > > - specjbb2015: no regression > - dacapo: (Attachment is the dacapo h2 full gc pause.) > - 95% of full gc pauses: 10%-19% improvement. > - 5% of full gc pauses: 1.2% improvement. > - 0.1% of full gc pauses: -6.16% improvement. > > $ java -Xmx1g -Xms1g -XX:ParallelGCThreads=4 -Xlog:gc*=info:file=gc.log -jar dacapo-9.12-bach.jar --iterations 5 --size huge --no-pre-iteration-gc h2 This pull request has now been integrated. Changeset: be0d46c1 Author: Hamlin Li URL: https://git.openjdk.java.net/jdk/commit/be0d46c1 Stats: 214 lines in 17 files changed: 174 ins; 6 del; 34 mod 8262068: Improve G1 Full GC by skipping compaction for regions with high survival ratio Co-authored-by: Shoubing Ma Reviewed-by: sjohanss, ayang ------------- PR: https://git.openjdk.java.net/jdk/pull/2760 From zgu at openjdk.java.net Mon Apr 12 12:50:51 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Mon, 12 Apr 2021 12:50:51 GMT Subject: Integrated: 8265012: Shenandoah: Backout JDK-8264718 In-Reply-To: <2dkdGG9JW1wdsKKluHOS33VAIne3WdE7xxUgIlfSQ2c=.65c62947-47cf-466c-b773-046afcff5fa6@github.com> References: <2dkdGG9JW1wdsKKluHOS33VAIne3WdE7xxUgIlfSQ2c=.65c62947-47cf-466c-b773-046afcff5fa6@github.com> Message-ID: On Fri, 9 Apr 2021 21:14:22 GMT, Zhengyu Gu wrote: > It turns out that enquening string deduplication candidates during concurrent root scanning may result lock rank inversion between stack watermark lock and string dedup queue lock, if the scanning is triggered by stack watermark and dedup buffer happens to be full. > > Backout JDK-8264718 for now, will retry after Kim's string deduplication refactoring. > > Test: > - [x] hotspot_gc_shenandoah This pull request has now been integrated. Changeset: 125184e9 Author: Zhengyu Gu URL: https://git.openjdk.java.net/jdk/commit/125184e9 Stats: 69 lines in 5 files changed: 26 ins; 39 del; 4 mod 8265012: Shenandoah: Backout JDK-8264718 Reviewed-by: shade ------------- PR: https://git.openjdk.java.net/jdk/pull/3423 From tschatzl at openjdk.java.net Mon Apr 12 13:03:48 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 12 Apr 2021 13:03:48 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 11:14:20 GMT, Ivan Walulya wrote: > Hi all, > > Please review this change that skips the rebuild remset phase and the associated updates after rebuild, If during Remark, G1 finds that there are no remembered sets to rebuild. > > Test: tier1-5 Would it be possible to add a test that checks for this case? I.e. allocate some objects of the approximate amount of a region, keeping them live, do a full gc to move them to old gen, then start a concurrent cycle that should print that new message? src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 1324: > 1322: > 1323: verify_during_pause(G1HeapVerifier::G1VerifyCleanup, VerifyOption_G1UsePrevMarking, "Cleanup before"); > 1324: bool has_rebuilt_remembered_sets = (total_selected_for_rebuild() > 0); Please add a newline before. src/hotspot/share/gc/g1/g1ConcurrentMark.hpp line 460: > 458: // set phase at all. > 459: HeapWord* volatile* _top_at_rebuild_starts; > 460: uint _total_selected_for_rebuild; Something like `// Number of regions selected for rebuild in the Remark pause` should be added here. It seems unnecessary to actually store that number, but sufficient that there is any need to rebuild regions. That would simplify the conditions in two places. ------------- PR: https://git.openjdk.java.net/jdk/pull/3434 From tschatzl at openjdk.java.net Mon Apr 12 13:11:59 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 12 Apr 2021 13:11:59 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 11:14:20 GMT, Ivan Walulya wrote: > Hi all, > > Please review this change that skips the rebuild remset phase and the associated updates after rebuild, If during Remark, G1 finds that there are no remembered sets to rebuild. > > Test: tier1-5 See other comments. ------------- Changes requested by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3434 From iwalulya at openjdk.java.net Mon Apr 12 13:12:01 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Mon, 12 Apr 2021 13:12:01 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 12:55:14 GMT, Thomas Schatzl wrote: >> Hi all, >> >> Please review this change that skips the rebuild remset phase and the associated updates after rebuild, If during Remark, G1 finds that there are no remembered sets to rebuild. >> >> Test: tier1-5 > > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 1324: > >> 1322: >> 1323: verify_during_pause(G1HeapVerifier::G1VerifyCleanup, VerifyOption_G1UsePrevMarking, "Cleanup before"); >> 1324: bool has_rebuilt_remembered_sets = (total_selected_for_rebuild() > 0); > > Please add a newline before. noted, thanks > src/hotspot/share/gc/g1/g1ConcurrentMark.hpp line 460: > >> 458: // set phase at all. >> 459: HeapWord* volatile* _top_at_rebuild_starts; >> 460: uint _total_selected_for_rebuild; > > Something like `// Number of regions selected for rebuild in the Remark pause` should be added here. > It seems unnecessary to actually store that number, but sufficient that there is any need to rebuild regions. That would simplify the conditions in two places. Indeed, a bool has_rebuilt_remembered_sets should suffice. ------------- PR: https://git.openjdk.java.net/jdk/pull/3434 From tschatzl at openjdk.java.net Mon Apr 12 13:33:41 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 12 Apr 2021 13:33:41 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 11:14:20 GMT, Ivan Walulya wrote: > Hi all, > > Please review this change that skips the rebuild remset phase and the associated updates after rebuild, If during Remark, G1 finds that there are no remembered sets to rebuild. > > Test: tier1-5 Changes requested by tschatzl (Reviewer). src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 1330: > 1328: _g1h->heap_region_iterate(&cl); > 1329: } else { > 1330: log_debug(gc, remset, tracking)("No Remembered Sets To Update After Rebuild"); Maybe make this `gc+phases=debug` too like the one above so that either of those shows up with the same log options. Also there does not seem to be a reason for the upper casing here. (The argument to `GCTraceTime` is a phase name, but this seems like a regular statement) src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 1962: > 1960: // skip the rebuild remembered set phase > 1961: if (total_selected_for_rebuild() == 0) { > 1962: log_debug(gc, remset, tracking)("Skipping Remembered Set Rebuild. No Regions Selected For Rebuild."); Maybe better is `log_debug(gc, marking)` here, somewhat fitting with the other messages when running with `gc+marking=debug`. Also the upper casing the in the second sentence seems weird, `No regions selected for rebuild` seems better (`Remembered Set Rebuild` is the name of that phase, and we do the upper casing for them already. But the second statement seems to be a regular statement) ------------- PR: https://git.openjdk.java.net/jdk/pull/3434 From iwalulya at openjdk.java.net Mon Apr 12 13:46:33 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Mon, 12 Apr 2021 13:46:33 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates In-Reply-To: References: Message-ID: <1qoeFNu6xecMrJtTbvAIOM43vqoozO2XX9E2tkpXjD4=.686e9da9-dfc1-4673-a254-a97b42659803@github.com> On Mon, 12 Apr 2021 13:27:10 GMT, Thomas Schatzl wrote: >> Hi all, >> >> Please review this change that skips the rebuild remset phase and the associated updates after rebuild, If during Remark, G1 finds that there are no remembered sets to rebuild. >> >> Test: tier1-5 > > src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 1962: > >> 1960: // skip the rebuild remembered set phase >> 1961: if (total_selected_for_rebuild() == 0) { >> 1962: log_debug(gc, remset, tracking)("Skipping Remembered Set Rebuild. No Regions Selected For Rebuild."); > > Maybe better is `log_debug(gc, marking)` here, somewhat fitting with the other messages when running with `gc+marking=debug`. Also the upper casing the in the second sentence seems weird, `No regions selected for rebuild` seems better (`Remembered Set Rebuild` is the name of that phase, and we do the upper casing for them already. But the second statement seems to be a regular statement) Thanks, will make both changes ------------- PR: https://git.openjdk.java.net/jdk/pull/3434 From mikael at openjdk.java.net Mon Apr 12 16:11:01 2021 From: mikael at openjdk.java.net (Mikael Vidstedt) Date: Mon, 12 Apr 2021 16:11:01 GMT Subject: RFR: 8265082: test/hotspot/jtreg/gc/g1/TestG1SkipCompaction.java fails validate-source In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 16:03:24 GMT, Daniel D. Daugherty wrote: > A trivial fix to fix the "validate-source" failure in Tier1 due to the bad copyright > header in test/hotspot/jtreg/gc/g1/TestG1SkipCompaction.java. Marked as reviewed by mikael (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3438 From dcubed at openjdk.java.net Mon Apr 12 16:11:01 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 12 Apr 2021 16:11:01 GMT Subject: RFR: 8265082: test/hotspot/jtreg/gc/g1/TestG1SkipCompaction.java fails validate-source Message-ID: A trivial fix to fix the "validate-source" failure in Tier1 due to the bad copyright header in test/hotspot/jtreg/gc/g1/TestG1SkipCompaction.java. ------------- Commit messages: - 8265082: test/hotspot/jtreg/gc/g1/TestG1SkipCompaction.java fails validate-source Changes: https://git.openjdk.java.net/jdk/pull/3438/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3438&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265082 Stats: 8 lines in 1 file changed: 4 ins; 2 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3438.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3438/head:pull/3438 PR: https://git.openjdk.java.net/jdk/pull/3438 From erikj at openjdk.java.net Mon Apr 12 16:11:01 2021 From: erikj at openjdk.java.net (Erik Joelsson) Date: Mon, 12 Apr 2021 16:11:01 GMT Subject: RFR: 8265082: test/hotspot/jtreg/gc/g1/TestG1SkipCompaction.java fails validate-source In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 16:03:24 GMT, Daniel D. Daugherty wrote: > A trivial fix to fix the "validate-source" failure in Tier1 due to the bad copyright > header in test/hotspot/jtreg/gc/g1/TestG1SkipCompaction.java. Marked as reviewed by erikj (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3438 From dcubed at openjdk.java.net Mon Apr 12 16:17:47 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 12 Apr 2021 16:17:47 GMT Subject: RFR: 8265082: test/hotspot/jtreg/gc/g1/TestG1SkipCompaction.java fails validate-source In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 16:06:00 GMT, Mikael Vidstedt wrote: >> A trivial fix to fix the "validate-source" failure in Tier1 due to the bad copyright >> header in test/hotspot/jtreg/gc/g1/TestG1SkipCompaction.java. > > Marked as reviewed by mikael (Reviewer). @vidmik - Thanks for the fast review! ------------- PR: https://git.openjdk.java.net/jdk/pull/3438 From dcubed at openjdk.java.net Mon Apr 12 16:17:47 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 12 Apr 2021 16:17:47 GMT Subject: Integrated: 8265082: test/hotspot/jtreg/gc/g1/TestG1SkipCompaction.java fails validate-source In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 16:03:24 GMT, Daniel D. Daugherty wrote: > A trivial fix to fix the "validate-source" failure in Tier1 due to the bad copyright > header in test/hotspot/jtreg/gc/g1/TestG1SkipCompaction.java. This pull request has now been integrated. Changeset: f4794378 Author: Daniel D. Daugherty URL: https://git.openjdk.java.net/jdk/commit/f4794378 Stats: 8 lines in 1 file changed: 4 ins; 2 del; 2 mod 8265082: test/hotspot/jtreg/gc/g1/TestG1SkipCompaction.java fails validate-source Reviewed-by: mikael, erikj ------------- PR: https://git.openjdk.java.net/jdk/pull/3438 From dcubed at openjdk.java.net Mon Apr 12 16:25:34 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Mon, 12 Apr 2021 16:25:34 GMT Subject: RFR: 8265082: test/hotspot/jtreg/gc/g1/TestG1SkipCompaction.java fails validate-source In-Reply-To: References: Message-ID: On Mon, 12 Apr 2021 16:06:34 GMT, Erik Joelsson wrote: >> A trivial fix to fix the "validate-source" failure in Tier1 due to the bad copyright >> header in test/hotspot/jtreg/gc/g1/TestG1SkipCompaction.java. > > Marked as reviewed by erikj (Reviewer). @erikj79 - Thanks for your review also. ------------- PR: https://git.openjdk.java.net/jdk/pull/3438 From pliden at openjdk.java.net Tue Apr 13 07:11:24 2021 From: pliden at openjdk.java.net (Per Liden) Date: Tue, 13 Apr 2021 07:11:24 GMT Subject: RFR: 8261759: ZGC: ZWorker Threads Continue Marking After System.exit() called Message-ID: During JVM termination, ZCollectedHeap::stop() currently only returns once any ongoing GC cycle has completed. However, waiting for a GC cycle to complete can take awhile, so the JVM might not terminate as promptly as one might expect. This patch adds abortion checks to workers doing marking and relocation to allow ZWorker threads to terminate promptly. In addition, before and after concurrent phases, the ZDriver thread checks if `should_terminate()` is true and prematurely terminates the GC cycle if that's the case. Testing: * Tier1-7 on Linux. * SPECjbb2015, comparing score and marking times against baseline (5 iterations each), no difference observed. * Ad-hoc programs calling System.exit() during marking/relocation. * Manual Ctrl-C testing. ------------- Commit messages: - 8261759: ZGC: ZWorker Threads Continue Marking After System.exit() called Changes: https://git.openjdk.java.net/jdk/pull/3453/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3453&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8261759 Stats: 150 lines in 7 files changed: 141 ins; 0 del; 9 mod Patch: https://git.openjdk.java.net/jdk/pull/3453.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3453/head:pull/3453 PR: https://git.openjdk.java.net/jdk/pull/3453 From pliden at openjdk.java.net Tue Apr 13 08:25:06 2021 From: pliden at openjdk.java.net (Per Liden) Date: Tue, 13 Apr 2021 08:25:06 GMT Subject: RFR: 8265116: ZGC: Steal local stacks instead of flushing them Message-ID: As part of addressing the issue of excessive mark stack usage, it was observed that sometimes mark stack utilization can be very low. For example, casparcwang at tencent.com reported that when running Apache Zookeeper, mark stack utilization was ~10%. The problem is that GC workers are a bit too eager to flush mark stacks. Currently, whenever a GC worker has drained its home stripe it will always flush all remaining mark stacks. This causes non-full mark stacks to be flushed. This in turn means that the GC worker who receives that non-full stack (small amount of work) tends to also produce non-full stacks (small amount of work) for other GC workers. This can lead to a vicious cycle, which results in excessive mark stack memory usage. The solution proposed here is to not let GC workers flush non-full stacks. After draining the home stripe, a GC worker will instead try to steal any non-full local stack belonging to a different stripe, and if no such stacks exist it will (as it does today) try to steal a global stack belonging to a different stripe. (Note: This PR only addresses one part of the mark stack usage problem. Other improvements, such as the discussed "mark before push"-patch will be a separate PR). Testing: * Tier1-7. * SPECjbb2015, score and mark times unaffected. * casparcwang at tencent.com reports that this patch (in combination with the "mark before push"-patch) solves the problem for Apache Zookeeper. ------------- Commit messages: - 8265116: ZGC: Steal local stacks instead of flushing them Changes: https://git.openjdk.java.net/jdk/pull/3455/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3455&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265116 Stats: 41 lines in 4 files changed: 25 ins; 2 del; 14 mod Patch: https://git.openjdk.java.net/jdk/pull/3455.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3455/head:pull/3455 PR: https://git.openjdk.java.net/jdk/pull/3455 From mli at openjdk.java.net Tue Apr 13 10:14:08 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Tue, 13 Apr 2021 10:14:08 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc Message-ID: in JDK-8262068, we have an enhancement for full gc which will skip compacting some regions with high survivor ratio. There might be some young regions among these skipped regions. These young regions are not filled with BOTs info as they're young. But after full GC these young regions will become old regions, and the BOTs info is better to be filled for performance consideration. ------------- Commit messages: - JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc Changes: https://git.openjdk.java.net/jdk/pull/3459/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3459&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264987 Stats: 33 lines in 2 files changed: 33 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3459.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3459/head:pull/3459 PR: https://git.openjdk.java.net/jdk/pull/3459 From pliden at openjdk.java.net Tue Apr 13 11:03:10 2021 From: pliden at openjdk.java.net (Per Liden) Date: Tue, 13 Apr 2021 11:03:10 GMT Subject: RFR: 8260267: ZGC: Reduce mark stack usage Message-ID: <1Qn76cgJaltPI2LtFQFfdqCG8SzOJwg3ORVQc8pK3MY=.918a159f-4f3d-408c-8cca-5b2bf727a0b2@github.com> Under some conditions, striped marking can cause excessive mark stack memory usage. For example, when the object graph has one or more N:1 relations ships, where N is a large number. To avoid pushing mark stack entries to already marked objects, and thus reduce mark stack memory usage, this patch adds does two things: * GC threads will under some conditions mark the object before pushing, else (if the condition to mark failed) * GC threads will check if the object is already marked before pushing. The conditions for marking before pushing is: * We are using a single stripe, in which cases striped marking is essentially disabled anyway. * We see excessive mark stack usage. In other words, the mark stack allocator indicates "high usage". To make this easier to review, this PR contains two commits: * `ZGC: Reduce mark stack usage` - This patch adds the infrastructure needed to mark before push, but only does so if we are using a single stripe. * `ZGC: Mark before push if mark stack usage is high` - This patch adds the condition to mark before push if the mark stack allocator indicates "high usage". As a bonus, this patch will also uncommit memory used for mark stacks after the marking has completed. Testing: * Tier1-7 * SPECjbb2015, score and marking times unaffected * casparcwang at tencent.com reports that this patch, in combination with PR #3455, solves the problem for Apache Zookeeper. ------------- Commit messages: - ZGC: Mark before push if mark stack usage is high - ZGC: Reduce mark stack usage Changes: https://git.openjdk.java.net/jdk/pull/3460/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3460&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8260267 Stats: 285 lines in 19 files changed: 192 ins; 41 del; 52 mod Patch: https://git.openjdk.java.net/jdk/pull/3460.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3460/head:pull/3460 PR: https://git.openjdk.java.net/jdk/pull/3460 From pliden at openjdk.java.net Tue Apr 13 11:25:05 2021 From: pliden at openjdk.java.net (Per Liden) Date: Tue, 13 Apr 2021 11:25:05 GMT Subject: RFR: 8265127: ZGC: Fix incorrect reporting of reclaimed memory Message-ID: When Java threads relocate objects, ZGC will over-report number of reclaimed bytes and under-report amount of garbage left after GC. This happens because the logic calculating the total number of reclaimed bytes assumes all objects where relocated by GC workers. The result is that the information printed in the GC log can be skewed. This patch solves this by keeping track of the number of bytes relocated by Java threads, and using that number in the logic calculating the total number of reclaimed bytes. ------------- Commit messages: - 8265127: ZGC: Fix incorrect reporting of reclaimed memory Changes: https://git.openjdk.java.net/jdk/pull/3462/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3462&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265127 Stats: 80 lines in 9 files changed: 54 ins; 1 del; 25 mod Patch: https://git.openjdk.java.net/jdk/pull/3462.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3462/head:pull/3462 PR: https://git.openjdk.java.net/jdk/pull/3462 From sjohanss at openjdk.java.net Tue Apr 13 12:31:16 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Tue, 13 Apr 2021 12:31:16 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc In-Reply-To: References: Message-ID: <6n-c6tEyftC3R47cgjooQTn7zM3izEMaZJhEvH_WzPo=.a2a42d0d-12ac-4593-ac4d-a94a20860652@github.com> On Tue, 13 Apr 2021 10:07:17 GMT, Hamlin Li wrote: > in JDK-8262068, we have an enhancement for full gc which will skip compacting some regions with high survivor ratio. There might be some young regions among these skipped regions. These young regions are not filled with BOTs info as they're young. > But after full GC these young regions will become old regions, and the BOTs info is better to be filled for performance consideration. src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp line 190: > 188: } > 189: assert(next_addr == limit, "Should stop the scan at the limit."); > 190: } I think this could be implemented using `apply_to_marked_objects()`. Let the closure keep track of the threshold for the region and the `apply()` function could look something like this: size_t apply(oop object) { size_t size = object->size(); HeapWord* addr = cast_from_oop(object); HeapWord* next_addr = addr + size; if (next_addr > _threshold) { _threshold = hr->cross_threshold(addr, next_addr); } return size; } ------------- PR: https://git.openjdk.java.net/jdk/pull/3459 From sjohanss at openjdk.java.net Tue Apr 13 12:36:32 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Tue, 13 Apr 2021 12:36:32 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc In-Reply-To: <6n-c6tEyftC3R47cgjooQTn7zM3izEMaZJhEvH_WzPo=.a2a42d0d-12ac-4593-ac4d-a94a20860652@github.com> References: <6n-c6tEyftC3R47cgjooQTn7zM3izEMaZJhEvH_WzPo=.a2a42d0d-12ac-4593-ac4d-a94a20860652@github.com> Message-ID: On Tue, 13 Apr 2021 12:28:27 GMT, Stefan Johansson wrote: >> in JDK-8262068, we have an enhancement for full gc which will skip compacting some regions with high survivor ratio. There might be some young regions among these skipped regions. These young regions are not filled with BOTs info as they're young. >> But after full GC these young regions will become old regions, and the BOTs info is better to be filled for performance consideration. > > src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp line 190: > >> 188: } >> 189: assert(next_addr == limit, "Should stop the scan at the limit."); >> 190: } > > I think this could be implemented using `apply_to_marked_objects()`. Let the closure keep track of the threshold for the region and the `apply()` function could look something like this: > > size_t apply(oop object) { > size_t size = object->size(); > HeapWord* addr = cast_from_oop(object); > HeapWord* next_addr = addr + size; > if (next_addr > _threshold) { > _threshold = hr->cross_threshold(addr, next_addr); > } > return size; > } I realize that we need to keep the previous address in the closure as well, for the case when we step over the threshold because of an unmarked object. ------------- PR: https://git.openjdk.java.net/jdk/pull/3459 From mli at openjdk.java.net Wed Apr 14 04:22:58 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Wed, 14 Apr 2021 04:22:58 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc In-Reply-To: References: <6n-c6tEyftC3R47cgjooQTn7zM3izEMaZJhEvH_WzPo=.a2a42d0d-12ac-4593-ac4d-a94a20860652@github.com> Message-ID: On Tue, 13 Apr 2021 12:32:51 GMT, Stefan Johansson wrote: >> src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp line 190: >> >>> 188: } >>> 189: assert(next_addr == limit, "Should stop the scan at the limit."); >>> 190: } >> >> I think this could be implemented using `apply_to_marked_objects()`. Let the closure keep track of the threshold for the region and the `apply()` function could look something like this: >> >> size_t apply(oop object) { >> size_t size = object->size(); >> HeapWord* addr = cast_from_oop(object); >> HeapWord* next_addr = addr + size; >> if (next_addr > _threshold) { >> _threshold = hr->cross_threshold(addr, next_addr); >> } >> return size; >> } > > I realize that we need to keep the previous address in the closure as well, for the case when we step over the threshold because of an unmarked object. Hi Stefan, Thanks for the suggestion. At first thought, I think it's a good way to reuse the traversal ability of HeapRegion::apply_to_marked_objects. But when I try to implement it actually, seems to me it might make the code a little complicated. For the first part, in the new G1UpdateBotClosure, we need to indroduce an _pre_addr as you suggested. this part is implemented as below, it's not that complicated, please check the following code snippet: void G1FullGCPrepareTask::G1CalculatePointersClosure::update_bot(HeapRegion* hr) { G1UpdateBotClosure updateBot(hr); hr->apply_to_marked_objects(_bitmap, &updateBot); } G1FullGCPrepareTask::G1UpdateBotClosure::G1UpdateBotClosure(HeapRegion* hr) : _hr(hr), _pre_addr(hr->bottom()), _threshold(hr->initialize_threshold()) { } size_t G1FullGCPrepareTask::G1UpdateBotClosure::apply(oop object) { HeapWord* addr = cast_from_oop(object); size_t size = object->size(); HeapWord* next_addr = addr + size; if(addr > _threshold) { _threshold = _hr->cross_threshold(_pre_addr, addr); } if (next_addr > _threshold) { _threshold = _hr->cross_threshold(addr, next_addr); } _pre_addr = next_addr;+ return size; } But, the above code does not consider the situation: one or several dead objects are at the end of the heap region, at this situation, we needs to update the bot outside of the G1UpdateBotClosure after hr->apply_to_marked_objects(...). I did not implement this part yet, but seems to me it makes the code a little complicated and not that readable, so would like to discuss with you first. In summary, seems it does not make the code more readable, and will increase the complexity. How do you think about it? ------------- PR: https://git.openjdk.java.net/jdk/pull/3459 From mli at openjdk.java.net Wed Apr 14 04:25:55 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Wed, 14 Apr 2021 04:25:55 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc In-Reply-To: References: <6n-c6tEyftC3R47cgjooQTn7zM3izEMaZJhEvH_WzPo=.a2a42d0d-12ac-4593-ac4d-a94a20860652@github.com> Message-ID: On Wed, 14 Apr 2021 04:18:54 GMT, Hamlin Li wrote: >> I realize that we need to keep the previous address in the closure as well, for the case when we step over the threshold because of an unmarked object. > > Hi Stefan, Thanks for the suggestion. > At first thought, I think it's a good way to reuse the traversal ability of HeapRegion::apply_to_marked_objects. > > But when I try to implement it actually, seems to me it might make the code a little complicated. > > For the first part, in the new G1UpdateBotClosure, we need to indroduce an _pre_addr as you suggested. this part is implemented as below, it's not that complicated, please check the following code snippet: > > void G1FullGCPrepareTask::G1CalculatePointersClosure::update_bot(HeapRegion* hr) { > G1UpdateBotClosure updateBot(hr); > hr->apply_to_marked_objects(_bitmap, &updateBot); > } > > G1FullGCPrepareTask::G1UpdateBotClosure::G1UpdateBotClosure(HeapRegion* hr) : > _hr(hr), > _pre_addr(hr->bottom()), > _threshold(hr->initialize_threshold()) { } > > size_t G1FullGCPrepareTask::G1UpdateBotClosure::apply(oop object) { > HeapWord* addr = cast_from_oop(object); > size_t size = object->size(); > HeapWord* next_addr = addr + size; > > if(addr > _threshold) { > _threshold = _hr->cross_threshold(_pre_addr, addr); > } > if (next_addr > _threshold) { > _threshold = _hr->cross_threshold(addr, next_addr); > } > _pre_addr = next_addr;+ return size; > } > > > > But, the above code does not consider the situation: one or several dead objects are at the end of the heap region, at this situation, we needs to update the bot outside of the G1UpdateBotClosure after hr->apply_to_marked_objects(...). I did not implement this part yet, but seems to me it makes the code a little complicated and not that readable, so would like to discuss with you first. > > In summary, seems it does not make the code more readable, and will increase the complexity. > How do you think about it? BTW, may I ask what's the format do you use the quote a snippet of code, seems my snippet of code lost all format info(e.g. indent). ------------- PR: https://git.openjdk.java.net/jdk/pull/3459 From sjohanss at openjdk.java.net Wed Apr 14 07:19:57 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Wed, 14 Apr 2021 07:19:57 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc In-Reply-To: References: <6n-c6tEyftC3R47cgjooQTn7zM3izEMaZJhEvH_WzPo=.a2a42d0d-12ac-4593-ac4d-a94a20860652@github.com> Message-ID: On Wed, 14 Apr 2021 04:23:20 GMT, Hamlin Li wrote: >> Hi Stefan, Thanks for the suggestion. >> At first thought, I think it's a good way to reuse the traversal ability of HeapRegion::apply_to_marked_objects. >> >> But when I try to implement it actually, seems to me it might make the code a little complicated. >> >> For the first part, in the new G1UpdateBotClosure, we need to indroduce an _pre_addr as you suggested. this part is implemented as below, it's not that complicated, please check the following code snippet: >> >> void G1FullGCPrepareTask::G1CalculatePointersClosure::update_bot(HeapRegion* hr) { >> G1UpdateBotClosure updateBot(hr); >> hr->apply_to_marked_objects(_bitmap, &updateBot); >> } >> >> G1FullGCPrepareTask::G1UpdateBotClosure::G1UpdateBotClosure(HeapRegion* hr) : >> _hr(hr), >> _pre_addr(hr->bottom()), >> _threshold(hr->initialize_threshold()) { } >> >> size_t G1FullGCPrepareTask::G1UpdateBotClosure::apply(oop object) { >> HeapWord* addr = cast_from_oop(object); >> size_t size = object->size(); >> HeapWord* next_addr = addr + size; >> >> if(addr > _threshold) { >> _threshold = _hr->cross_threshold(_pre_addr, addr); >> } >> if (next_addr > _threshold) { >> _threshold = _hr->cross_threshold(addr, next_addr); >> } >> _pre_addr = next_addr;+ return size; >> } >> >> >> >> But, the above code does not consider the situation: one or several dead objects are at the end of the heap region, at this situation, we needs to update the bot outside of the G1UpdateBotClosure after hr->apply_to_marked_objects(...). I did not implement this part yet, but seems to me it makes the code a little complicated and not that readable, so would like to discuss with you first. >> >> In summary, seems it does not make the code more readable, and will increase the complexity. >> How do you think about it? > > BTW, may I ask what's the format do you use the quote a snippet of code, seems my snippet of code lost all format info(e.g. indent). Easiest is to mark the code and push the "Insert code" button (<>). You can also manually write ``` to start a code block. I'll get back to you about the code above later. ------------- PR: https://git.openjdk.java.net/jdk/pull/3459 From pliden at openjdk.java.net Wed Apr 14 07:21:19 2021 From: pliden at openjdk.java.net (Per Liden) Date: Wed, 14 Apr 2021 07:21:19 GMT Subject: RFR: 8265136: ZGC: Expose GarbageCollectorMXBeans for both pauses and cycles Message-ID: JDK-8240679 introduced a change where the information exposed via the GarbageCollectorMXBean went from being related to the GC pauses to instead be related to the GC cycles. This helped provide more accurate heap usage information. However, some users have noticed that that you no longer get timing information related to the GC pauses, only related to the GC cycle. Shenandoah has opted for having two distinct memory managers to provide timing information about both pauses and cycles. To align how concurent GCs report this information, ZGC should probably do the same. Testing: * Tier1-7 with ZGC * Manual testing with relevant jtreg tests ------------- Commit messages: - 8265136: ZGC: Expose GarbageCollectorMXBeans for both pauses and cycles Changes: https://git.openjdk.java.net/jdk/pull/3483/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3483&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265136 Stats: 179 lines in 9 files changed: 107 ins; 25 del; 47 mod Patch: https://git.openjdk.java.net/jdk/pull/3483.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3483/head:pull/3483 PR: https://git.openjdk.java.net/jdk/pull/3483 From pliden at openjdk.java.net Wed Apr 14 07:40:25 2021 From: pliden at openjdk.java.net (Per Liden) Date: Wed, 14 Apr 2021 07:40:25 GMT Subject: RFR: 8261759: ZGC: ZWorker Threads Continue Marking After System.exit() called [v2] In-Reply-To: References: Message-ID: <8y7MS1mYK0NHKafx7FFElNLyCsFk9iZz1HUet-DDZZs=.d4fa0f25-c796-493c-ad22-60b4e8b7f84e@github.com> > During JVM termination, ZCollectedHeap::stop() currently only returns once any ongoing GC cycle has completed. However, waiting for a GC cycle to complete can take awhile, so the JVM might not terminate as promptly as one might expect. > > This patch adds abortion checks to workers doing marking and relocation to allow ZWorker threads to terminate promptly. In addition, before and after concurrent phases, the ZDriver thread checks if `should_terminate()` is true and prematurely terminates the GC cycle if that's the case. > > Testing: > * Tier1-7 on Linux. > * SPECjbb2015, comparing score and marking times against baseline (5 iterations each), no difference observed. > * Ad-hoc programs calling System.exit() during marking/relocation. > * Manual Ctrl-C testing. Per Liden has updated the pull request incrementally with one additional commit since the last revision: Add missing include of precompiled.hpp ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3453/files - new: https://git.openjdk.java.net/jdk/pull/3453/files/cddd18e8..73603d6d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3453&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3453&range=00-01 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3453.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3453/head:pull/3453 PR: https://git.openjdk.java.net/jdk/pull/3453 From mli at openjdk.java.net Wed Apr 14 07:44:58 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Wed, 14 Apr 2021 07:44:58 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc In-Reply-To: References: <6n-c6tEyftC3R47cgjooQTn7zM3izEMaZJhEvH_WzPo=.a2a42d0d-12ac-4593-ac4d-a94a20860652@github.com> Message-ID: <8hdFxwH8CQIR1hbL1bjuxU9en2HpCgtVeOTb07mMWb0=.b27436f6-6442-4f98-b652-0719d76851f6@github.com> On Wed, 14 Apr 2021 07:16:51 GMT, Stefan Johansson wrote: >> BTW, may I ask what's the format do you use the quote a snippet of code, seems my snippet of code lost all format info(e.g. indent). > > Easiest is to mark the code and push the "Insert code" button (<>). You can also manually write ``` to start a code block. > > I'll get back to you about the code above later. Thanks Stefan, it works. ------------- PR: https://git.openjdk.java.net/jdk/pull/3459 From sjohanss at openjdk.java.net Wed Apr 14 08:27:58 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Wed, 14 Apr 2021 08:27:58 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc In-Reply-To: <8hdFxwH8CQIR1hbL1bjuxU9en2HpCgtVeOTb07mMWb0=.b27436f6-6442-4f98-b652-0719d76851f6@github.com> References: <6n-c6tEyftC3R47cgjooQTn7zM3izEMaZJhEvH_WzPo=.a2a42d0d-12ac-4593-ac4d-a94a20860652@github.com> <8hdFxwH8CQIR1hbL1bjuxU9en2HpCgtVeOTb07mMWb0=.b27436f6-6442-4f98-b652-0719d76851f6@github.com> Message-ID: On Wed, 14 Apr 2021 07:42:28 GMT, Hamlin Li wrote: >> Easiest is to mark the code and push the "Insert code" button (<>). You can also manually write ``` to start a code block. >> >> I'll get back to you about the code above later. > > Thanks Stefan, it works. Looked a bit more at the code now and I agree that it didn't become as neat as I had hoped. Solving the problem you describe with dead objects at the end should be fairly easy in the closure-destructor I guess. Something like: ~G1UpdateBotClosure() { if (_hr->top() > _threshold) { _hr->cross_threshold(_pre_addr, _hr->top()); } } And a few comments in the apply method will help readability: size_t apply(oop object) { size_t size = object->size(); HeapWord* addr = cast_from_oop(object); HeapWord* next_addr = addr + size; // Threshold was crossed by dead object. if(addr > _threshold) { _threshold = _hr->cross_threshold(_pre_addr, addr); } // Threshold is crossed by this object. if (next_addr > _threshold) { _threshold = _hr->cross_threshold(addr, next_addr); } _pre_addr = next_addr; return size; } But still it might be better to instead streamline the current approach: Suggestion: HeapWord* limit = hr->top(); HeapWord* next_addr = hr->bottom(); HeapWord* threshold = hr->initialize_threshold(); HeapWord* pre_addr; while (next_addr < limit) { pre_addr = next_addr; next_addr = _bitmap->get_next_marked_addr(next_addr + 1, limit); if (next_addr > threshold) { threshold = hr->cross_threshold(pre_addr, next_addr); } } assert(next_addr == limit, "Should stop the scan at the limit."); } Here I removed the `Prefetch` because we do not need to touch the heap. For this to be true I also removed the small optimization to use `obj->size()`, which is probably not an optimization if we don't need to do anything with the "object". After doing this I agree that reusing `apply_to_marked_objects()` is probably not helping. ------------- PR: https://git.openjdk.java.net/jdk/pull/3459 From tschatzl at openjdk.java.net Wed Apr 14 08:39:08 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 14 Apr 2021 08:39:08 GMT Subject: RFR: 8264818: G1: Improve liveness check for empty pinned regions after full gc marking Message-ID: Hi all, can I have reviews for this small change improving the check for empty Open Archive regions using information that has recently been added to the full gc in [JDK-8263495](https://bugs.openjdk.java.net/browse/JDK-8263495) instead of scanning the region? Testing: tier1-3, manual check that the freeing still works using some hack to create a dummy Open Archive region which contents are completely unreferenced and then running a "Hello World" app doing a system.gc. The patch for the VM to create that archive is [here](https://github.com/tschatzl/jdk/tree/a8264818-create-dummy-open-archive). ------------- Commit messages: - Initial commit Changes: https://git.openjdk.java.net/jdk/pull/3485/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3485&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264818 Stats: 4 lines in 1 file changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/3485.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3485/head:pull/3485 PR: https://git.openjdk.java.net/jdk/pull/3485 From sjohanss at openjdk.java.net Wed Apr 14 08:40:00 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Wed, 14 Apr 2021 08:40:00 GMT Subject: RFR: 8264788: Make SequentialSubTasksDone use-once [v2] In-Reply-To: References: Message-ID: On Thu, 8 Apr 2021 15:05:42 GMT, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that simplifies code ahead? >> >> SequentialSubTasksDone has some machinery to reset itself automatically after all tasks were claimed. >> >> This is not used at all and can/should be removed. >> >> Test: tier1-3 > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > ayang review Looks good. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3372 From tschatzl at openjdk.java.net Wed Apr 14 08:43:12 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 14 Apr 2021 08:43:12 GMT Subject: RFR: 8264423: G1: Rename full gc attribute table states Message-ID: Hi all, please review this cleanup change suggested in PR#2760 that renames some code in the g1 full gc attribute table. This is the description from the CR: > Since JDK-8253600 g1 full gc has its own (temporary) attribute table storing information that it needs for evacuation. > > Currently the naming corresponds to the attributes in HeapRegion. This is somewhat confusing, as they do not completely match. > > While discussing this in the review for JDK-8262068 we thought of changing this to more reflect the purpose: > > I.e. > closed -> not_marked_through (or "always live" or "always_marked" something similar to indicate that we do not need to mark through them) > pinned -> not_compacted > normal -> compacted (or just keep "normal" as it is some internal state) Some additional renaming has been performed. Testing: tier1-5 ------------- Commit messages: - Some more renamings - Initial version for testing Changes: https://git.openjdk.java.net/jdk/pull/3486/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3486&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8264423 Stats: 66 lines in 8 files changed: 4 ins; 16 del; 46 mod Patch: https://git.openjdk.java.net/jdk/pull/3486.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3486/head:pull/3486 PR: https://git.openjdk.java.net/jdk/pull/3486 From sjohanss at openjdk.java.net Wed Apr 14 08:43:58 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Wed, 14 Apr 2021 08:43:58 GMT Subject: RFR: 8264818: G1: Improve liveness check for empty pinned regions after full gc marking In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 08:32:33 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small change improving the check for empty Open Archive regions using information that has recently been added to the full gc in [JDK-8263495](https://bugs.openjdk.java.net/browse/JDK-8263495) instead of scanning the region? > > Testing: tier1-3, manual check that the freeing still works using some hack to create a dummy Open Archive region which contents are completely unreferenced and then running a "Hello World" app doing a system.gc. The patch for the VM to create that archive is [here](https://github.com/tschatzl/jdk/tree/a8264818-create-dummy-open-archive). Looks good. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3485 From mli at openjdk.java.net Wed Apr 14 09:09:22 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Wed, 14 Apr 2021 09:09:22 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc [v2] In-Reply-To: References: Message-ID: > in JDK-8262068, we have an enhancement for full gc which will skip compacting some regions with high survivor ratio. There might be some young regions among these skipped regions. These young regions are not filled with BOTs info as they're young. > But after full GC these young regions will become old regions, and the BOTs info is better to be filled for performance consideration. Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: refine the code. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3459/files - new: https://git.openjdk.java.net/jdk/pull/3459/files/591e14b6..962e6579 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3459&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3459&range=00-01 Stats: 8 lines in 1 file changed: 0 ins; 7 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3459.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3459/head:pull/3459 PR: https://git.openjdk.java.net/jdk/pull/3459 From ayang at openjdk.java.net Wed Apr 14 09:17:58 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Wed, 14 Apr 2021 09:17:58 GMT Subject: RFR: 8264818: G1: Improve liveness check for empty pinned regions after full gc marking In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 08:32:33 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small change improving the check for empty Open Archive regions using information that has recently been added to the full gc in [JDK-8263495](https://bugs.openjdk.java.net/browse/JDK-8263495) instead of scanning the region? > > Testing: tier1-3, manual check that the freeing still works using some hack to create a dummy Open Archive region which contents are completely unreferenced and then running a "Hello World" app doing a system.gc. The patch for the VM to create that archive is [here](https://github.com/openjdk/jdk/commit/47d740d244b09b104df635c07ee49f89f480ebc6). Marked as reviewed by ayang (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3485 From tschatzl at openjdk.java.net Wed Apr 14 10:55:10 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 14 Apr 2021 10:55:10 GMT Subject: RFR: 8264423: G1: Rename full gc attribute table states [v2] In-Reply-To: References: Message-ID: > Hi all, > > please review this cleanup change suggested in PR#2760 that renames some code in the g1 full gc attribute table. > > This is the description from the CR: > >> Since JDK-8253600 g1 full gc has its own (temporary) attribute table storing information that it needs for evacuation. >> >> Currently the naming corresponds to the attributes in HeapRegion. This is somewhat confusing, as they do not completely match. >> >> While discussing this in the review for JDK-8262068 we thought of changing this to more reflect the purpose: >> >> I.e. >> closed -> not_marked_through (or "always live" or "always_marked" something similar to indicate that we do not need to mark through them) >> pinned -> not_compacted >> normal -> compacted (or just keep "normal" as it is some internal state) > > Some additional renaming has been performed. > > Testing: tier1-5 Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: sjohanss review - remove double negation ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3486/files - new: https://git.openjdk.java.net/jdk/pull/3486/files/f0faf8a9..2047bc11 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3486&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3486&range=00-01 Stats: 18 lines in 4 files changed: 5 ins; 8 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/3486.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3486/head:pull/3486 PR: https://git.openjdk.java.net/jdk/pull/3486 From sjohanss at openjdk.java.net Wed Apr 14 11:00:05 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Wed, 14 Apr 2021 11:00:05 GMT Subject: RFR: 8264423: G1: Rename full gc attribute table states [v2] In-Reply-To: References: Message-ID: <9XlJIYda_WZrhd2PZM30ZDcJx1E_GrBNBWc4A4kOpJ0=.d3eccb94-5ee3-4064-923e-03745da3ba02@github.com> On Wed, 14 Apr 2021 10:55:10 GMT, Thomas Schatzl wrote: >> Hi all, >> >> please review this cleanup change suggested in PR#2760 that renames some code in the g1 full gc attribute table. >> >> This is the description from the CR: >> >>> Since JDK-8253600 g1 full gc has its own (temporary) attribute table storing information that it needs for evacuation. >>> >>> Currently the naming corresponds to the attributes in HeapRegion. This is somewhat confusing, as they do not completely match. >>> >>> While discussing this in the review for JDK-8262068 we thought of changing this to more reflect the purpose: >>> >>> I.e. >>> closed -> not_marked_through (or "always live" or "always_marked" something similar to indicate that we do not need to mark through them) >>> pinned -> not_compacted >>> normal -> compacted (or just keep "normal" as it is some internal state) >> >> Some additional renaming has been performed. >> >> Testing: tier1-5 > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > sjohanss review - remove double negation Marked as reviewed by sjohanss (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3486 From tschatzl at openjdk.java.net Wed Apr 14 12:11:13 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 14 Apr 2021 12:11:13 GMT Subject: RFR: 8264423: G1: Rename full gc attribute table states [v3] In-Reply-To: References: Message-ID: > Hi all, > > please review this cleanup change suggested in PR#2760 that renames some code in the g1 full gc attribute table. > > This is the description from the CR: > >> Since JDK-8253600 g1 full gc has its own (temporary) attribute table storing information that it needs for evacuation. >> >> Currently the naming corresponds to the attributes in HeapRegion. This is somewhat confusing, as they do not completely match. >> >> While discussing this in the review for JDK-8262068 we thought of changing this to more reflect the purpose: >> >> I.e. >> closed -> not_marked_through (or "always live" or "always_marked" something similar to indicate that we do not need to mark through them) >> pinned -> not_compacted >> normal -> compacted (or just keep "normal" as it is some internal state) > > Some additional renaming has been performed. > > Testing: tier1-5 Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: - Skip marking - Try always live ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3486/files - new: https://git.openjdk.java.net/jdk/pull/3486/files/2047bc11..2cce9dc2 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3486&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3486&range=01-02 Stats: 21 lines in 7 files changed: 0 ins; 1 del; 20 mod Patch: https://git.openjdk.java.net/jdk/pull/3486.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3486/head:pull/3486 PR: https://git.openjdk.java.net/jdk/pull/3486 From mli at openjdk.java.net Wed Apr 14 12:11:14 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Wed, 14 Apr 2021 12:11:14 GMT Subject: RFR: 8264423: G1: Rename full gc attribute table states [v2] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 10:55:10 GMT, Thomas Schatzl wrote: >> Hi all, >> >> please review this cleanup change suggested in PR#2760 that renames some code in the g1 full gc attribute table. >> >> This is the description from the CR: >> >>> Since JDK-8253600 g1 full gc has its own (temporary) attribute table storing information that it needs for evacuation. >>> >>> Currently the naming corresponds to the attributes in HeapRegion. This is somewhat confusing, as they do not completely match. >>> >>> While discussing this in the review for JDK-8262068 we thought of changing this to more reflect the purpose: >>> >>> I.e. >>> closed -> not_marked_through (or "always live" or "always_marked" something similar to indicate that we do not need to mark through them) >>> pinned -> not_compacted >>> normal -> compacted (or just keep "normal" as it is some internal state) >> >> Some additional renaming has been performed. >> >> Testing: tier1-5 > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > sjohanss review - remove double negation src/hotspot/share/gc/g1/g1FullGCHeapRegionAttr.hpp line 71: > 69: } > 70: > 71: bool is_compacted_or_marked_through(uint idx) const { Should this be rename to is_compacted_or_not_marked_through? As the condition is indeed == Compacted || == NotMarkedThrough. ------------- PR: https://git.openjdk.java.net/jdk/pull/3486 From sjohanss at openjdk.java.net Wed Apr 14 12:13:59 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Wed, 14 Apr 2021 12:13:59 GMT Subject: RFR: 8264423: G1: Rename full gc attribute table states [v2] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 10:55:10 GMT, Thomas Schatzl wrote: >> Hi all, >> >> please review this cleanup change suggested in PR#2760 that renames some code in the g1 full gc attribute table. >> >> This is the description from the CR: >> >>> Since JDK-8253600 g1 full gc has its own (temporary) attribute table storing information that it needs for evacuation. >>> >>> Currently the naming corresponds to the attributes in HeapRegion. This is somewhat confusing, as they do not completely match. >>> >>> While discussing this in the review for JDK-8262068 we thought of changing this to more reflect the purpose: >>> >>> I.e. >>> closed -> not_marked_through (or "always live" or "always_marked" something similar to indicate that we do not need to mark through them) >>> pinned -> not_compacted >>> normal -> compacted (or just keep "normal" as it is some internal state) >> >> Some additional renaming has been performed. >> >> Testing: tier1-5 > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > sjohanss review - remove double negation I think `SkipMarking` is better than the other alternatives so far. ------------- PR: https://git.openjdk.java.net/jdk/pull/3486 From mli at openjdk.java.net Wed Apr 14 12:18:02 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Wed, 14 Apr 2021 12:18:02 GMT Subject: RFR: 8264818: G1: Improve liveness check for empty pinned regions after full gc marking In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 08:32:33 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small change improving the check for empty Open Archive regions using information that has recently been added to the full gc in [JDK-8263495](https://bugs.openjdk.java.net/browse/JDK-8263495) instead of scanning the region? > > Testing: tier1-3, manual check that the freeing still works using some hack to create a dummy Open Archive region which contents are completely unreferenced and then running a "Hello World" app doing a system.gc. The patch for the VM to create that archive is [here](https://github.com/openjdk/jdk/commit/47d740d244b09b104df635c07ee49f89f480ebc6). looks good. ------------- Marked as reviewed by mli (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3485 From mli at openjdk.java.net Wed Apr 14 12:19:02 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Wed, 14 Apr 2021 12:19:02 GMT Subject: RFR: 8264423: G1: Rename full gc attribute table states [v2] In-Reply-To: References: Message-ID: <1cQ6wpxvGajA2o9lvqcmswPWsAQXvuMfqsvhJ5XIYuc=.80187f95-f22a-4a35-91c0-baebceade6a8@github.com> On Wed, 14 Apr 2021 12:11:09 GMT, Stefan Johansson wrote: > I think `SkipMarking` is better than the other alternatives so far. agree! ------------- PR: https://git.openjdk.java.net/jdk/pull/3486 From tschatzl at openjdk.java.net Wed Apr 14 12:19:06 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 14 Apr 2021 12:19:06 GMT Subject: RFR: 8264423: G1: Rename full gc attribute table states [v2] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 12:05:48 GMT, Hamlin Li wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> sjohanss review - remove double negation > > src/hotspot/share/gc/g1/g1FullGCHeapRegionAttr.hpp line 71: > >> 69: } >> 70: >> 71: bool is_compacted_or_marked_through(uint idx) const { > > Should this be rename to is_compacted_or_not_marked_through? > As the condition is indeed == Compacted || == NotMarkedThrough. Yes, that's true. It should actually be called `is_compacted_or_not_marked_through`. ------------- PR: https://git.openjdk.java.net/jdk/pull/3486 From tschatzl at openjdk.java.net Wed Apr 14 12:19:07 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 14 Apr 2021 12:19:07 GMT Subject: RFR: 8264423: G1: Rename full gc attribute table states [v2] In-Reply-To: References: Message-ID: <02mgVTeyLNIx0imUwRmJSFOXii_SarlZktQm3_pYBkE=.41196e96-a6fe-40d3-a50e-bd180512e2bb@github.com> On Wed, 14 Apr 2021 12:16:02 GMT, Thomas Schatzl wrote: >> src/hotspot/share/gc/g1/g1FullGCHeapRegionAttr.hpp line 71: >> >>> 69: } >>> 70: >>> 71: bool is_compacted_or_marked_through(uint idx) const { >> >> Should this be rename to is_compacted_or_not_marked_through? >> As the condition is indeed == Compacted || == NotMarkedThrough. > > Yes, that's true. It should actually be called `is_compacted_or_not_marked_through`. See the other comment from me about naming variants provided by now. ------------- PR: https://git.openjdk.java.net/jdk/pull/3486 From tschatzl at openjdk.java.net Wed Apr 14 12:21:59 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 14 Apr 2021 12:21:59 GMT Subject: RFR: 8264423: G1: Rename full gc attribute table states [v3] In-Reply-To: References: Message-ID: <9LQJrzIEwMJbT1ZzyZiMSJU7vhGWrASBeMI8lj4tqcs=.f2046949-231b-4575-86c9-3060650058b2@github.com> On Wed, 14 Apr 2021 12:11:13 GMT, Thomas Schatzl wrote: >> Hi all, >> >> please review this cleanup change suggested in PR#2760 that renames some code in the g1 full gc attribute table. >> >> This is the description from the CR: >> >>> Since JDK-8253600 g1 full gc has its own (temporary) attribute table storing information that it needs for evacuation. >>> >>> Currently the naming corresponds to the attributes in HeapRegion. This is somewhat confusing, as they do not completely match. >>> >>> While discussing this in the review for JDK-8262068 we thought of changing this to more reflect the purpose: >>> >>> I.e. >>> closed -> not_marked_through (or "always live" or "always_marked" something similar to indicate that we do not need to mark through them) >>> pinned -> not_compacted >>> normal -> compacted (or just keep "normal" as it is some internal state) >> >> Some additional renaming has been performed. >> >> Testing: tier1-5 > > Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: > > - Skip marking > - Try always live Given the apparent consensus, I'll keep `SkipMarking`. Doing some testing before any integration. ------------- PR: https://git.openjdk.java.net/jdk/pull/3486 From mli at openjdk.java.net Wed Apr 14 12:35:02 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Wed, 14 Apr 2021 12:35:02 GMT Subject: RFR: 8264423: G1: Rename full gc attribute table states [v3] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 12:11:13 GMT, Thomas Schatzl wrote: >> Hi all, >> >> please review this cleanup change suggested in PR#2760 that renames some code in the g1 full gc attribute table. >> >> This is the description from the CR: >> >>> Since JDK-8253600 g1 full gc has its own (temporary) attribute table storing information that it needs for evacuation. >>> >>> Currently the naming corresponds to the attributes in HeapRegion. This is somewhat confusing, as they do not completely match. >>> >>> While discussing this in the review for JDK-8262068 we thought of changing this to more reflect the purpose: >>> >>> I.e. >>> closed -> not_marked_through (or "always live" or "always_marked" something similar to indicate that we do not need to mark through them) >>> pinned -> not_compacted >>> normal -> compacted (or just keep "normal" as it is some internal state) >> >> Some additional renaming has been performed. >> >> Testing: tier1-5 > > Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: > > - Skip marking > - Try always live looks good. ------------- Marked as reviewed by mli (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3486 From mli at openjdk.java.net Wed Apr 14 12:43:06 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Wed, 14 Apr 2021 12:43:06 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc [v2] In-Reply-To: References: <6n-c6tEyftC3R47cgjooQTn7zM3izEMaZJhEvH_WzPo=.a2a42d0d-12ac-4593-ac4d-a94a20860652@github.com> <8hdFxwH8CQIR1hbL1bjuxU9en2HpCgtVeOTb07mMWb0=.b27436f6-6442-4f98-b652-0719d76851f6@github.com> Message-ID: On Wed, 14 Apr 2021 08:25:28 GMT, Stefan Johansson wrote: >> Thanks Stefan, it works. > > Looked a bit more at the code now and I agree that it didn't become as neat as I had hoped. Solving the problem you describe with dead objects at the end should be fairly easy in the closure-destructor I guess. Something like: > > ~G1UpdateBotClosure() { > if (_hr->top() > _threshold) { > _hr->cross_threshold(_pre_addr, _hr->top()); > } > } > > > And a few comments in the apply method will help readability: > > size_t apply(oop object) { > size_t size = object->size(); > HeapWord* addr = cast_from_oop(object); > HeapWord* next_addr = addr + size; > > // Threshold was crossed by dead object. > if(addr > _threshold) { > _threshold = _hr->cross_threshold(_pre_addr, addr); > } > > // Threshold is crossed by this object. > if (next_addr > _threshold) { > _threshold = _hr->cross_threshold(addr, next_addr); > } > > _pre_addr = next_addr; > return size; > } > > > But still it might be better to instead streamline the current approach: > Suggestion: > > HeapWord* limit = hr->top(); > HeapWord* next_addr = hr->bottom(); > HeapWord* threshold = hr->initialize_threshold(); > HeapWord* pre_addr; > > while (next_addr < limit) { > pre_addr = next_addr; > next_addr = _bitmap->get_next_marked_addr(next_addr + 1, limit); > > if (next_addr > threshold) { > threshold = hr->cross_threshold(pre_addr, next_addr); > } > } > assert(next_addr == limit, "Should stop the scan at the limit."); > } > > Here I removed the `Prefetch` because we do not need to touch the heap. For this to be true I also removed the small optimization to use `obj->size()`, which is probably not an optimization if we don't need to do anything with the "object". > > After doing this I agree that reusing `apply_to_marked_objects()` is probably not helping. Hi Stefan, I think the refactoring you suggested is great, I just modified as you suggested and the pre-submit tests run successfully. ------------- PR: https://git.openjdk.java.net/jdk/pull/3459 From sjohanss at openjdk.java.net Wed Apr 14 14:46:55 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Wed, 14 Apr 2021 14:46:55 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: On Fri, 2 Apr 2021 16:38:02 GMT, Marcus G K Williams wrote: >> When using LargePageSizeInBytes=1G, os::Linux::reserve_memory_special_huge_tlbfs* cannot select large pages smaller than 1G. Code heap usually uses less than 1G, so currently the code precludes code heap from using >> Large pages in this circumstance and when os::Linux::reserve_memory_special_huge_tlbfs* is called page sizes fall back to Linux::page_size() (usually 4k). >> >> This change allows the above use case by populating all large_page_sizes present in /sys/kernel/mm/hugepages in _page_sizes upon calling os::Linux::setup_large_page_size(). >> >> In os::Linux::reserve_memory_special_huge_tlbfs* we then select the largest large page size available in _page_sizes that is smaller than bytes being reserved. > > Marcus G K Williams has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 44 commits: > > - Merge branch 'master' into update_hlp > - Rebase on pull/3073 > > Signed-off-by: Marcus G K Williams > - Merge branch 'pull/3073' into update_hlp > - Thomas review. > > Changed commit_memory_special to return bool to signal if the request succeeded or not. > - Self review. > > Update helper name to better match commit_memory_special(). > - Marcus review. > > Updated comments. > - Ivan review > > Renamed helper to commit_memory_special and updated the comments. > - 8262291: Refactor reserve_memory_special_huge_tlbfs > - Merge branch 'master' into update_hlp > - Addressed kstefanj review suggestions > > Signed-off-by: Marcus G K Williams > - ... and 34 more: https://git.openjdk.java.net/jdk/compare/f60e81bf...f5bfe224 So now #3073 has been integrated, so you could merge with master to get this change. I also discussed the meaning of `LargePageSizeInBytes` with some folks internally and @stefank pointed me to: https://docs.oracle.com/en/java/javase/16/docs/specs/man/java.html There the flag is defined as: -XX:LargePageSizeInBytes=size Sets the maximum size (in bytes) for large pages used for the Java heap. The size argument must be a power of 2 (2, 4, 8, 16, and so on). Append the letter k or K to indicate kilobytes, m or M to indicate megabytes, or g or G to indicate gigabytes. By default, the size is set to 0, meaning that the JVM chooses the size for large pages automatically. ... So with this and what's in the code, the default behavior for the VM should be to try to use the best suitable large page size available for a mapping. If `LargePageSizeInBytes` is set this will limit the page sizes used to only include the ones less than or equal to `LargePageSizeInBytes`. Does that make sense to everyone? This will change the behavior compared to the what we have today: * Default (`LargePageSizeInBytes=0`) - old behavior - use the systems default large page size - new behavior - use all configure large page sizes * Value set (`LargePageSizeInBytes=X`) - old behavior - use only large pages of size X - new behavior - use large pages of size X or smaller So the new behavior better fits what's in the docs, but I still suspect we might need a CSR. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From github.com+168222+mgkwill at openjdk.java.net Wed Apr 14 19:04:10 2021 From: github.com+168222+mgkwill at openjdk.java.net (Marcus G K Williams) Date: Wed, 14 Apr 2021 19:04:10 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 14:43:34 GMT, Stefan Johansson wrote: > So now #3073 has been integrated, so you could merge with master to get this change. > I will merge master and pickup change form #3073 today. > I also discussed the meaning of `LargePageSizeInBytes` with some folks internally and @stefank pointed me to: > https://docs.oracle.com/en/java/javase/16/docs/specs/man/java.html > > There the flag is defined as: > > ``` > -XX:LargePageSizeInBytes=size > Sets the maximum size (in bytes) for large pages used for the Java heap. The size argument must be > a power of 2 (2, 4, 8, 16, and so on). Append the letter k or K to indicate kilobytes, m or M to > indicate megabytes, or g or G to indicate gigabytes. By default, the size is set to 0, meaning that > the JVM chooses the size for large pages automatically. ... > ``` > > So with this and what's in the code, the default behavior for the VM should be to try to use the best suitable large page size available for a mapping. If `LargePageSizeInBytes` is set this will limit the page sizes used to only include the ones less than or equal to `LargePageSizeInBytes`. > > Does that make sense to everyone? Thanks for the clarification @kstefanj. This makes sense and fits within my original goals for the change. What are your thoughts @tstuefe ? > This will change the behavior compared to the what we have today: > > * Default (`LargePageSizeInBytes=0`) > > * old behavior - use the systems default large page size > * new behavior - use all configure large page sizes > * Value set (`LargePageSizeInBytes=X`) > > * old behavior - use only large pages of size X > * new behavior - use large pages of size X or smaller > > So the new behavior better fits what's in the docs, but I still suspect we might need a CSR. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From zgu at openjdk.java.net Thu Apr 15 00:12:51 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Thu, 15 Apr 2021 00:12:51 GMT Subject: RFR: 8265239: Shenandoah: Shenandoah heap region count could be 1 short Message-ID: <-8G7NK-sSvff9jC-zGGQBkkzO9rsoCv_B_tSDhoVlhA=.cbb907e0-f891-46db-af1a-0a2cf520eed4@github.com> henandoah uses MaxHeapSize to calculate region size, then aligns up heap size to region size alignment, but when it calculates region count, it does not take alignment into account, that may result 1 region short than actual count. MaxHeapSize eventually aligns to HeapAlignment (which is region size in Shenandoah case) in GCArguments::initialize_heap_flags_and_sizes(), but it is too late. Test: - [x] hotspot_gc_shenandoah ------------- Commit messages: - JDK-8265239 Changes: https://git.openjdk.java.net/jdk/pull/3504/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3504&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265239 Stats: 2 lines in 2 files changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3504.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3504/head:pull/3504 PR: https://git.openjdk.java.net/jdk/pull/3504 From sspitsyn at openjdk.java.net Thu Apr 15 04:23:41 2021 From: sspitsyn at openjdk.java.net (Serguei Spitsyn) Date: Thu, 15 Apr 2021 04:23:41 GMT Subject: RFR: 8265136: ZGC: Expose GarbageCollectorMXBeans for both pauses and cycles In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 07:13:50 GMT, Per Liden wrote: > JDK-8240679 introduced a change where the information exposed via the GarbageCollectorMXBean went from being related to the GC pauses to instead be related to the GC cycles. This helped provide more accurate heap usage information. However, some users have noticed that that you no longer get timing information related to the GC pauses, only related to the GC cycle. > > Shenandoah has opted for having two distinct memory managers to provide timing information about both pauses and cycles. To align how concurent GCs report this information, ZGC should probably do the same. > > Testing: > * Tier1-7 with ZGC > * Manual testing with relevant jtreg tests Hi Per, It looks good to me. Thanks, Serguei ------------- Marked as reviewed by sspitsyn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3483 From stuefe at openjdk.java.net Thu Apr 15 05:55:39 2021 From: stuefe at openjdk.java.net (Thomas Stuefe) Date: Thu, 15 Apr 2021 05:55:39 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 14:43:34 GMT, Stefan Johansson wrote: >> Marcus G K Williams has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 44 commits: >> >> - Merge branch 'master' into update_hlp >> - Rebase on pull/3073 >> >> Signed-off-by: Marcus G K Williams >> - Merge branch 'pull/3073' into update_hlp >> - Thomas review. >> >> Changed commit_memory_special to return bool to signal if the request succeeded or not. >> - Self review. >> >> Update helper name to better match commit_memory_special(). >> - Marcus review. >> >> Updated comments. >> - Ivan review >> >> Renamed helper to commit_memory_special and updated the comments. >> - 8262291: Refactor reserve_memory_special_huge_tlbfs >> - Merge branch 'master' into update_hlp >> - Addressed kstefanj review suggestions >> >> Signed-off-by: Marcus G K Williams >> - ... and 34 more: https://git.openjdk.java.net/jdk/compare/f60e81bf...f5bfe224 > > So now #3073 has been integrated, so you could merge with master to get this change. > > I also discussed the meaning of `LargePageSizeInBytes` with some folks internally and @stefank pointed me to: > https://docs.oracle.com/en/java/javase/16/docs/specs/man/java.html > > There the flag is defined as: > > -XX:LargePageSizeInBytes=size > Sets the maximum size (in bytes) for large pages used for the Java heap. The size argument must be > a power of 2 (2, 4, 8, 16, and so on). Append the letter k or K to indicate kilobytes, m or M to > indicate megabytes, or g or G to indicate gigabytes. By default, the size is set to 0, meaning that > the JVM chooses the size for large pages automatically. ... > > > So with this and what's in the code, the default behavior for the VM should be to try to use the best suitable large page size available for a mapping. If `LargePageSizeInBytes` is set this will limit the page sizes used to only include the ones less than or equal to `LargePageSizeInBytes`. > > Does that make sense to everyone? > > This will change the behavior compared to the what we have today: > * Default (`LargePageSizeInBytes=0`) > - old behavior - use the systems default large page size > - new behavior - use all configure large page sizes > * Value set (`LargePageSizeInBytes=X`) > - old behavior - use only large pages of size X > - new behavior - use large pages of size X or smaller > > So the new behavior better fits what's in the docs, but I still suspect we might need a CSR. The behavior proposed by @kstefanj in https://github.com/openjdk/jdk/pull/1153#issuecomment-819573437 sounds good and elegant. A small concern, with `LargePageSizeInBytes=0` we would now use any configured page size, with preference given to the largest possible one, right? Which may take away those pages from some other application on machines where the admin has a carefully groomed large page setup. Say, a database in addition to the java VM. Should this worry us, we could slightly change the meaning of LargePageSizeInBytes=0 to mean "default system page size". This would result in: * Default (`LargePageSizeInBytes=0`) * old behavior - use the systems default large page size * new behavior - use all configured page sizes up to and including the systems default large page size But I am fine with the original proposal too; then the admin in this example would have to manually specify LargePageSizeInBytes=default system page size to prevent the VM from grabbing other page sizes. In any case it would require a release note. Also a CSR. I am very busy atm but May looks better and I could assist with the CSR if needed. Cheers, Thomas ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From tschatzl at openjdk.java.net Thu Apr 15 07:46:37 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 15 Apr 2021 07:46:37 GMT Subject: RFR: 8264818: G1: Improve liveness check for empty pinned regions after full gc marking In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 08:40:56 GMT, Stefan Johansson wrote: >> Hi all, >> >> can I have reviews for this small change improving the check for empty Open Archive regions using information that has recently been added to the full gc in [JDK-8263495](https://bugs.openjdk.java.net/browse/JDK-8263495) instead of scanning the region? >> >> Testing: tier1-3, manual check that the freeing still works using some hack to create a dummy Open Archive region which contents are completely unreferenced and then running a "Hello World" app doing a system.gc. The patch for the VM to create that archive is [here](https://github.com/openjdk/jdk/commit/47d740d244b09b104df635c07ee49f89f480ebc6). > > Looks good. Thanks @kstefanj @Hamlin-Li @albertnetymk for your reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/3485 From tschatzl at openjdk.java.net Thu Apr 15 07:46:38 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 15 Apr 2021 07:46:38 GMT Subject: RFR: 8264423: G1: Rename full gc attribute table states [v2] In-Reply-To: References: Message-ID: <-LhFOtZY-OvDxysnRweto4uwXYiteQNVW5FM7xsT_fs=.cf13cede-23d3-460d-b0c9-93dc367c7667@github.com> On Wed, 14 Apr 2021 12:11:09 GMT, Stefan Johansson wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> sjohanss review - remove double negation > > I think `SkipMarking` is better than the other alternatives so far. Retesting looks good. Thanks @kstefanj @Hamlin-Li for your reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/3486 From tschatzl at openjdk.java.net Thu Apr 15 08:11:38 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 15 Apr 2021 08:11:38 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc [v2] In-Reply-To: References: Message-ID: <3ZmqjzfowYDp6ZLabZRr6O-jGhc0j3uzCFBTKSI9dvE=.f8e12aba-300f-4f25-a1e0-710c5c274bd8@github.com> On Wed, 14 Apr 2021 09:09:22 GMT, Hamlin Li wrote: >> in JDK-8262068, we have an enhancement for full gc which will skip compacting some regions with high survivor ratio. There might be some young regions among these skipped regions. These young regions are not filled with BOTs info as they're young. >> But after full GC these young regions will become old regions, and the BOTs info is better to be filled for performance consideration. > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > refine the code. Minor nits. src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp line 71: > 69: force_pinned = true; > 70: if (hr->is_young()) { > 71: // Old regions have BOTs info for performance consideration, but young regions I'd replace this comment with something like this: // G1 updates the BOT for old region contents incrementally, but young regions lack BOT // information for performance reasons. // Recreate BOT information of high live ratio young regions here to keep expected // performance during scanning their card tables in the collection pauses later. Or something like this... src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp line 170: > 168: > 169: void G1FullGCPrepareTask::G1CalculatePointersClosure::update_bot(HeapRegion* hr) { > 170: HeapWord* limit = hr->top(); Maybe make this `const`. src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp line 173: > 171: HeapWord* next_addr = hr->bottom(); > 172: HeapWord* threshold = hr->initialize_threshold(); > 173: HeapWord* pre_addr; Minor nit: typically the "friend" of `next` is `prev`, not `pre` in traversals. ------------- Changes requested by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3459 From tschatzl at openjdk.java.net Thu Apr 15 08:13:38 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 15 Apr 2021 08:13:38 GMT Subject: RFR: 8264788: Make SequentialSubTasksDone use-once [v2] In-Reply-To: <0QDQXiibSb7nKsN3Cjf1dsIkX4tlr6Wc7nhKfpS4DlE=.5e8189c8-3926-4061-a148-aba01bf23bea@github.com> References: <0QDQXiibSb7nKsN3Cjf1dsIkX4tlr6Wc7nhKfpS4DlE=.5e8189c8-3926-4061-a148-aba01bf23bea@github.com> Message-ID: On Thu, 8 Apr 2021 09:02:20 GMT, Albert Mingkun Yang wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> ayang review > > Looking at how `SequentialSubTasksDone` is used, I don't really see the benefit of having `assert(_num_claimed == _num_tasks)` in the destructor. If we drop that assertion, `Atomic::add` is more readable. > > I prefer `Atomic::add` with the assertion removed, but this version is fine as well. Thanks @albertnetymk @kstefanj for your reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/3372 From tschatzl at openjdk.java.net Thu Apr 15 08:13:39 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 15 Apr 2021 08:13:39 GMT Subject: Integrated: 8264788: Make SequentialSubTasksDone use-once In-Reply-To: References: Message-ID: On Wed, 7 Apr 2021 08:32:11 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that simplifies code ahead? > > SequentialSubTasksDone has some machinery to reset itself automatically after all tasks were claimed. > > This is not used at all and can/should be removed. > > Test: tier1-3 This pull request has now been integrated. Changeset: 125a8479 Author: Thomas Schatzl URL: https://git.openjdk.java.net/jdk/commit/125a8479 Stats: 94 lines in 5 files changed: 3 ins; 71 del; 20 mod 8264788: Make SequentialSubTasksDone use-once Reviewed-by: ayang, sjohanss ------------- PR: https://git.openjdk.java.net/jdk/pull/3372 From tschatzl at openjdk.java.net Thu Apr 15 08:15:36 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 15 Apr 2021 08:15:36 GMT Subject: Integrated: 8264423: G1: Rename full gc attribute table states In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 08:37:06 GMT, Thomas Schatzl wrote: > Hi all, > > please review this cleanup change suggested in PR#2760 that renames some code in the g1 full gc attribute table. > > This is the description from the CR: > >> Since JDK-8253600 g1 full gc has its own (temporary) attribute table storing information that it needs for evacuation. >> >> Currently the naming corresponds to the attributes in HeapRegion. This is somewhat confusing, as they do not completely match. >> >> While discussing this in the review for JDK-8262068 we thought of changing this to more reflect the purpose: >> >> I.e. >> closed -> not_marked_through (or "always live" or "always_marked" something similar to indicate that we do not need to mark through them) >> pinned -> not_compacted >> normal -> compacted (or just keep "normal" as it is some internal state) > > Some additional renaming has been performed. > > Testing: tier1-5 This pull request has now been integrated. Changeset: 75da1e9a Author: Thomas Schatzl URL: https://git.openjdk.java.net/jdk/commit/75da1e9a Stats: 71 lines in 8 files changed: 6 ins; 22 del; 43 mod 8264423: G1: Rename full gc attribute table states Reviewed-by: sjohanss, mli ------------- PR: https://git.openjdk.java.net/jdk/pull/3486 From tschatzl at openjdk.java.net Thu Apr 15 08:16:46 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 15 Apr 2021 08:16:46 GMT Subject: RFR: 8257774: G1: Trigger collect when free region count drops below threshold to prevent evacuation failures [v3] In-Reply-To: References: <5Ro0P4NPaUuG-LPpgqHY5S_kh_EVpwRgxqc5jTCano4=.afc3f2a4-dc9e-46c1-ac84-298cf143fa41@github.com> Message-ID: On Thu, 25 Mar 2021 02:04:11 GMT, Aditya Mandaleeka wrote: >> _This PR picks up from [this previous PR](https://github.com/openjdk/jdk/pull/1650) by @charliegracie._ >> >> I won't repeat the full description from the prior PR, but the general idea is to add the notion of a "proactive GC" to G1 which gets triggered in the slow allocation path if the number of free regions drops below the amount that would be required to complete a GC if it happened at that moment. The threshold is based on the survival rates from eden and survivor spaces along with the space required for tenured space evacuations. >> >> There are a couple of outstanding issues/questions known: >> - _Interaction with GCLocker_: In the case where we determine that a proactive GC is required and GC locker is active, we don't allow the young gen to expand (instead threads will stall). @tschatzl raised this and suggested that it should be discussed as part of the review. >> - _Disable proactive GC heuristic during initialization_: I ran into an issue in testing where code in the universe initialization codepath was tripping the proactive GC heuristic, leading to a GC being triggered before the VM has finished initialization. I need to find a good way to prevent this from happening. There may already be a mechanism for this, but I couldn't find one so I added a temporary placeholder (`zzinit_complete`) just to unblock testing. > > Aditya Mandaleeka has updated the pull request incrementally with one additional commit since the last revision: > > Fix format specifiers. About the second question: I think `Universe::is_fully_initialized()` would work for this. ------------- PR: https://git.openjdk.java.net/jdk/pull/3143 From tschatzl at openjdk.java.net Thu Apr 15 08:38:47 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 15 Apr 2021 08:38:47 GMT Subject: RFR: 8264818: G1: Improve liveness check for empty pinned regions after full gc marking [v2] In-Reply-To: References: Message-ID: > Hi all, > > can I have reviews for this small change improving the check for empty Open Archive regions using information that has recently been added to the full gc in [JDK-8263495](https://bugs.openjdk.java.net/browse/JDK-8263495) instead of scanning the region? > > Testing: tier1-3, manual check that the freeing still works using some hack to create a dummy Open Archive region which contents are completely unreferenced and then running a "Hello World" app doing a system.gc. The patch for the VM to create that archive is [here](https://github.com/openjdk/jdk/commit/47d740d244b09b104df635c07ee49f89f480ebc6). Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: - Merge branch 'master' into 8264818-improve-liveness-check - Initial commit ------------- Changes: https://git.openjdk.java.net/jdk/pull/3485/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3485&range=01 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3485.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3485/head:pull/3485 PR: https://git.openjdk.java.net/jdk/pull/3485 From tschatzl at openjdk.java.net Thu Apr 15 08:49:03 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 15 Apr 2021 08:49:03 GMT Subject: RFR: 8264818: G1: Improve liveness check for empty pinned regions after full gc marking [v3] In-Reply-To: References: Message-ID: > Hi all, > > can I have reviews for this small change improving the check for empty Open Archive regions using information that has recently been added to the full gc in [JDK-8263495](https://bugs.openjdk.java.net/browse/JDK-8263495) instead of scanning the region? > > Testing: tier1-3, manual check that the freeing still works using some hack to create a dummy Open Archive region which contents are completely unreferenced and then running a "Hello World" app doing a system.gc. The patch for the VM to create that archive is [here](https://github.com/openjdk/jdk/commit/47d740d244b09b104df635c07ee49f89f480ebc6). Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: Fix merge error in comments ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3485/files - new: https://git.openjdk.java.net/jdk/pull/3485/files/edd03164..f7a44ddc Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3485&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3485&range=01-02 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3485.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3485/head:pull/3485 PR: https://git.openjdk.java.net/jdk/pull/3485 From tschatzl at openjdk.java.net Thu Apr 15 08:49:03 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 15 Apr 2021 08:49:03 GMT Subject: Integrated: 8264818: G1: Improve liveness check for empty pinned regions after full gc marking In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 08:32:33 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this small change improving the check for empty Open Archive regions using information that has recently been added to the full gc in [JDK-8263495](https://bugs.openjdk.java.net/browse/JDK-8263495) instead of scanning the region? > > Testing: tier1-3, manual check that the freeing still works using some hack to create a dummy Open Archive region which contents are completely unreferenced and then running a "Hello World" app doing a system.gc. The patch for the VM to create that archive is [here](https://github.com/openjdk/jdk/commit/47d740d244b09b104df635c07ee49f89f480ebc6). This pull request has now been integrated. Changeset: bba16f61 Author: Thomas Schatzl URL: https://git.openjdk.java.net/jdk/commit/bba16f61 Stats: 5 lines in 1 file changed: 0 ins; 0 del; 5 mod 8264818: G1: Improve liveness check for empty pinned regions after full gc marking Reviewed-by: sjohanss, ayang, mli ------------- PR: https://git.openjdk.java.net/jdk/pull/3485 From sjohanss at openjdk.java.net Thu Apr 15 08:49:43 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Thu, 15 Apr 2021 08:49:43 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: On Fri, 2 Apr 2021 16:38:02 GMT, Marcus G K Williams wrote: >> When using LargePageSizeInBytes=1G, os::Linux::reserve_memory_special_huge_tlbfs* cannot select large pages smaller than 1G. Code heap usually uses less than 1G, so currently the code precludes code heap from using >> Large pages in this circumstance and when os::Linux::reserve_memory_special_huge_tlbfs* is called page sizes fall back to Linux::page_size() (usually 4k). >> >> This change allows the above use case by populating all large_page_sizes present in /sys/kernel/mm/hugepages in _page_sizes upon calling os::Linux::setup_large_page_size(). >> >> In os::Linux::reserve_memory_special_huge_tlbfs* we then select the largest large page size available in _page_sizes that is smaller than bytes being reserved. > > Marcus G K Williams has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 44 commits: > > - Merge branch 'master' into update_hlp > - Rebase on pull/3073 > > Signed-off-by: Marcus G K Williams > - Merge branch 'pull/3073' into update_hlp > - Thomas review. > > Changed commit_memory_special to return bool to signal if the request succeeded or not. > - Self review. > > Update helper name to better match commit_memory_special(). > - Marcus review. > > Updated comments. > - Ivan review > > Renamed helper to commit_memory_special and updated the comments. > - 8262291: Refactor reserve_memory_special_huge_tlbfs > - Merge branch 'master' into update_hlp > - Addressed kstefanj review suggestions > > Signed-off-by: Marcus G K Williams > - ... and 34 more: https://git.openjdk.java.net/jdk/compare/f60e81bf...f5bfe224 I would be good with that alteration as well and it will be an even smaller default change in many cases which might be nice. I think we should kick off the CSR work as soon as possible, so I'll look into that. And adding a release note for this change is also a good idea. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From iwalulya at openjdk.java.net Thu Apr 15 09:34:45 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Thu, 15 Apr 2021 09:34:45 GMT Subject: RFR: 8265119: G1: update_remset_before_rebuild mixes liveness in words with liveness in bytes Message-ID: Hi all, Please review this change to rename ConcurrentMark::liveness() to live_words() making it clear that this represents the number of live words between bottom and nTAMS. Adding ConcurrentMark::live_bytes() to return the same value in bytes. Additionally, I add a test for G1MixedGCLiveThresholdPercent flag. Testing: tier 1-5. ------------- Commit messages: - Rename ConcurrentMark::liveness() to live_words(), add live_bytes and test for G1MixedGCLiveThresholdPercent Changes: https://git.openjdk.java.net/jdk/pull/3511/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3511&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265119 Stats: 143 lines in 5 files changed: 135 ins; 0 del; 8 mod Patch: https://git.openjdk.java.net/jdk/pull/3511.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3511/head:pull/3511 PR: https://git.openjdk.java.net/jdk/pull/3511 From tschatzl at openjdk.java.net Thu Apr 15 09:58:37 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 15 Apr 2021 09:58:37 GMT Subject: RFR: 8257774: G1: Trigger collect when free region count drops below threshold to prevent evacuation failures [v3] In-Reply-To: References: <5Ro0P4NPaUuG-LPpgqHY5S_kh_EVpwRgxqc5jTCano4=.afc3f2a4-dc9e-46c1-ac84-298cf143fa41@github.com> Message-ID: <7fhhdOeBBekHCIpWMOH42FZXk7Ko2NuPhN7oAweqH5w=.d8ed25e2-7396-4ad3-aaa9-ada0d9fee6b3@github.com> On Thu, 25 Mar 2021 02:04:11 GMT, Aditya Mandaleeka wrote: >> _This PR picks up from [this previous PR](https://github.com/openjdk/jdk/pull/1650) by @charliegracie._ >> >> I won't repeat the full description from the prior PR, but the general idea is to add the notion of a "proactive GC" to G1 which gets triggered in the slow allocation path if the number of free regions drops below the amount that would be required to complete a GC if it happened at that moment. The threshold is based on the survival rates from eden and survivor spaces along with the space required for tenured space evacuations. >> >> There are a couple of outstanding issues/questions known: >> - _Interaction with GCLocker_: In the case where we determine that a proactive GC is required and GC locker is active, we don't allow the young gen to expand (instead threads will stall). @tschatzl raised this and suggested that it should be discussed as part of the review. >> - _Disable proactive GC heuristic during initialization_: I ran into an issue in testing where code in the universe initialization codepath was tripping the proactive GC heuristic, leading to a GC being triggered before the VM has finished initialization. I need to find a good way to prevent this from happening. There may already be a mechanism for this, but I couldn't find one so I added a temporary placeholder (`zzinit_complete`) just to unblock testing. > > Aditya Mandaleeka has updated the pull request incrementally with one additional commit since the last revision: > > Fix format specifiers. The current code makes the thread stall if there is a proactive gc request and a gc locker enabled. This behavior is reasonable to me: otherwise since there is a high likelihood will likely be an evacuation failure, that gclocker induced gc will take a long time - given that we want to avoid this exact situation, but others might disagree. ------------- PR: https://git.openjdk.java.net/jdk/pull/3143 From tschatzl at openjdk.java.net Thu Apr 15 10:03:39 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 15 Apr 2021 10:03:39 GMT Subject: RFR: 8257774: G1: Trigger collect when free region count drops below threshold to prevent evacuation failures [v3] In-Reply-To: References: <5Ro0P4NPaUuG-LPpgqHY5S_kh_EVpwRgxqc5jTCano4=.afc3f2a4-dc9e-46c1-ac84-298cf143fa41@github.com> Message-ID: On Thu, 25 Mar 2021 02:04:11 GMT, Aditya Mandaleeka wrote: >> _This PR picks up from [this previous PR](https://github.com/openjdk/jdk/pull/1650) by @charliegracie._ >> >> I won't repeat the full description from the prior PR, but the general idea is to add the notion of a "proactive GC" to G1 which gets triggered in the slow allocation path if the number of free regions drops below the amount that would be required to complete a GC if it happened at that moment. The threshold is based on the survival rates from eden and survivor spaces along with the space required for tenured space evacuations. >> >> There are a couple of outstanding issues/questions known: >> - _Interaction with GCLocker_: In the case where we determine that a proactive GC is required and GC locker is active, we don't allow the young gen to expand (instead threads will stall). @tschatzl raised this and suggested that it should be discussed as part of the review. >> - _Disable proactive GC heuristic during initialization_: I ran into an issue in testing where code in the universe initialization codepath was tripping the proactive GC heuristic, leading to a GC being triggered before the VM has finished initialization. I need to find a good way to prevent this from happening. There may already be a mechanism for this, but I couldn't find one so I added a temporary placeholder (`zzinit_complete`) just to unblock testing. > > Aditya Mandaleeka has updated the pull request incrementally with one additional commit since the last revision: > > Fix format specifiers. During some internal discussion with the people that had looked at the change a bit, we agreed that the additional `attempt_lock_free_first` parameter is very ugly and complicates the code a lot. [Here](https://github.com/tschatzl/jdk/commit/6c10cb3e6d71fcbb487385e23ea77d3c29f31b4d) is an attempt to remove this parameter by an explicit method that allocates using a new region. It does look nicer to me. No testing has been performed, just to see how it would look like. The change also removes that `use_retained_region_if_available` parameter form the lock-free `attempt_allocation` call. With your recent changes, all paths should try it (there has been a case where this was not true, but I think that one was errorneous), so I removed that parameter too. Please also merge with latest once more, the change is a bit out of date. ------------- PR: https://git.openjdk.java.net/jdk/pull/3143 From tschatzl at openjdk.java.net Thu Apr 15 10:17:38 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 15 Apr 2021 10:17:38 GMT Subject: RFR: 8265119: G1: update_remset_before_rebuild mixes liveness in words with liveness in bytes In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 09:29:07 GMT, Ivan Walulya wrote: > Hi all, > > Please review this change to rename ConcurrentMark::liveness() to live_words() making it clear that this represents the number of live words between bottom and nTAMS. Adding ConcurrentMark::live_bytes() to return the same value in bytes. Additionally, I add a test for G1MixedGCLiveThresholdPercent flag. > > Testing: tier 1-5. Lgtm apart from the few nits. src/hotspot/share/gc/g1/g1ConcurrentMark.hpp line 465: > 463: // live words between bottom and nTAMS. > 464: size_t live_words(uint region) const { return _region_mark_stats[region]._live_words; } > 465: Something like "same as above but returns bytes" would be nice. test/hotspot/jtreg/gc/g1/TestMixedGCLiveThreshold.java line 68: > 66: Asserts.assertEquals(regionsSelected, expectedRebuild, > 67: (expectedRebuild ? > 68: "No Regions selected for rebuild. G1MixedGCLiveThresholdPercent="+ liveThresholdPercent + Spaces before the operator at the end. There is another one below too. test/hotspot/jtreg/gc/g1/TestMixedGCLiveThreshold.java line 88: > 86: "-Xmx10M"}); > 87: > 88: basicOpts.add("-XX:G1MixedGCLiveThresholdPercent="+percent); Whitespace before and after the `+`. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3511 From iwalulya at openjdk.java.net Thu Apr 15 10:40:54 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Thu, 15 Apr 2021 10:40:54 GMT Subject: RFR: 8265119: G1: update_remset_before_rebuild mixes liveness in words with liveness in bytes [v2] In-Reply-To: References: Message-ID: > Hi all, > > Please review this change to rename ConcurrentMark::liveness() to live_words() making it clear that this represents the number of live words between bottom and nTAMS. Adding ConcurrentMark::live_bytes() to return the same value in bytes. Additionally, I add a test for G1MixedGCLiveThresholdPercent flag. > > Testing: tier 1-5. Ivan Walulya has updated the pull request incrementally with one additional commit since the last revision: changes proposed by Thomas review ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3511/files - new: https://git.openjdk.java.net/jdk/pull/3511/files/924f9ffe..1803f998 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3511&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3511&range=00-01 Stats: 4 lines in 2 files changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/3511.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3511/head:pull/3511 PR: https://git.openjdk.java.net/jdk/pull/3511 From iwalulya at openjdk.java.net Thu Apr 15 10:40:55 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Thu, 15 Apr 2021 10:40:55 GMT Subject: RFR: 8265119: G1: update_remset_before_rebuild mixes liveness in words with liveness in bytes [v2] In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 10:14:35 GMT, Thomas Schatzl wrote: >> Ivan Walulya has updated the pull request incrementally with one additional commit since the last revision: >> >> changes proposed by Thomas review > > Lgtm apart from the few nits. @tschatzl thanks for the review, I fixed the mentioned nits. ------------- PR: https://git.openjdk.java.net/jdk/pull/3511 From mli at openjdk.java.net Thu Apr 15 11:47:06 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Thu, 15 Apr 2021 11:47:06 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc [v3] In-Reply-To: References: Message-ID: > in JDK-8262068, we have an enhancement for full gc which will skip compacting some regions with high survivor ratio. There might be some young regions among these skipped regions. These young regions are not filled with BOTs info as they're young. > But after full GC these young regions will become old regions, and the BOTs info is better to be filled for performance consideration. Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: refine comments, names. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3459/files - new: https://git.openjdk.java.net/jdk/pull/3459/files/962e6579..f3524bac Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3459&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3459&range=01-02 Stats: 10 lines in 1 file changed: 0 ins; 2 del; 8 mod Patch: https://git.openjdk.java.net/jdk/pull/3459.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3459/head:pull/3459 PR: https://git.openjdk.java.net/jdk/pull/3459 From mli at openjdk.java.net Thu Apr 15 12:15:02 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Thu, 15 Apr 2021 12:15:02 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc [v4] In-Reply-To: References: Message-ID: <81xO8TaRnqmfY3Eey4s-ojYwfZBZZyhcm5ZJ4r0hX3E=.bc4d48eb-e07a-49eb-a26b-cd20bddc2e51@github.com> > in JDK-8262068, we have an enhancement for full gc which will skip compacting some regions with high survivor ratio. There might be some young regions among these skipped regions. These young regions are not filled with BOTs info as they're young. > But after full GC these young regions will become old regions, and the BOTs info is better to be filled for performance consideration. Hamlin Li has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: - Merge branch 'master' into fill-bots-in-full-gc - refine comments, names. - refine the code. - JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc ------------- Changes: https://git.openjdk.java.net/jdk/pull/3459/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3459&range=03 Stats: 25 lines in 2 files changed: 24 ins; 1 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3459.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3459/head:pull/3459 PR: https://git.openjdk.java.net/jdk/pull/3459 From mli at openjdk.java.net Thu Apr 15 12:15:53 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Thu, 15 Apr 2021 12:15:53 GMT Subject: RFR: JDK-8265259: G1: Fix HeapRegion::block_is_obj for unloading class in full gc Message-ID: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> In JDK-8262068, we have introduced an enhancement to skip compacting some heap regions. But the objs in these regions might have been dead, and their classes might have been unloaded, at this situation, we need following change to make sure we don't get into trouble when calls HeapRegion::block_is_obj(const HeapWord* p) and subsequent calls e.g. to get obj size. ------------- Commit messages: - JDK-8265259: G1: Fix HeapRegion::block_is_obj for unloading class in full gc Changes: https://git.openjdk.java.net/jdk/pull/3514/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3514&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265259 Stats: 6 lines in 1 file changed: 5 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3514.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3514/head:pull/3514 PR: https://git.openjdk.java.net/jdk/pull/3514 From mli at openjdk.java.net Thu Apr 15 12:29:37 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Thu, 15 Apr 2021 12:29:37 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc [v2] In-Reply-To: <3ZmqjzfowYDp6ZLabZRr6O-jGhc0j3uzCFBTKSI9dvE=.f8e12aba-300f-4f25-a1e0-710c5c274bd8@github.com> References: <3ZmqjzfowYDp6ZLabZRr6O-jGhc0j3uzCFBTKSI9dvE=.f8e12aba-300f-4f25-a1e0-710c5c274bd8@github.com> Message-ID: On Thu, 15 Apr 2021 08:07:13 GMT, Thomas Schatzl wrote: >> Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: >> >> refine the code. > > src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp line 71: > >> 69: force_pinned = true; >> 70: if (hr->is_young()) { >> 71: // Old regions have BOTs info for performance consideration, but young regions > > I'd replace this comment with something like this: > > // G1 updates the BOT for old region contents incrementally, but young regions lack BOT > // information for performance reasons. > // Recreate BOT information of high live ratio young regions here to keep expected > // performance during scanning their card tables in the collection pauses later. > > Or something like this... Thanks Thomas for refining the comments. ------------- PR: https://git.openjdk.java.net/jdk/pull/3459 From sjohanss at openjdk.java.net Thu Apr 15 15:35:51 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Thu, 15 Apr 2021 15:35:51 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc [v4] In-Reply-To: <81xO8TaRnqmfY3Eey4s-ojYwfZBZZyhcm5ZJ4r0hX3E=.bc4d48eb-e07a-49eb-a26b-cd20bddc2e51@github.com> References: <81xO8TaRnqmfY3Eey4s-ojYwfZBZZyhcm5ZJ4r0hX3E=.bc4d48eb-e07a-49eb-a26b-cd20bddc2e51@github.com> Message-ID: <-fvpCX-fvQW9qPu03cretUdHnWrNb30bNssnO9-mc5k=.56470439-483e-4855-b5f8-54fe6917ee77@github.com> On Thu, 15 Apr 2021 12:15:02 GMT, Hamlin Li wrote: >> in JDK-8262068, we have an enhancement for full gc which will skip compacting some regions with high survivor ratio. There might be some young regions among these skipped regions. These young regions are not filled with BOTs info as they're young. >> But after full GC these young regions will become old regions, and the BOTs info is better to be filled for performance consideration. > > Hamlin Li has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: > > - Merge branch 'master' into fill-bots-in-full-gc > - refine comments, names. > - refine the code. > - JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc Looks good. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3459 From sjohanss at openjdk.java.net Thu Apr 15 17:15:36 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Thu, 15 Apr 2021 17:15:36 GMT Subject: RFR: JDK-8265259: G1: Fix HeapRegion::block_is_obj for unloading class in full gc In-Reply-To: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> References: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> Message-ID: On Thu, 15 Apr 2021 12:07:47 GMT, Hamlin Li wrote: > In JDK-8262068, we have introduced an enhancement to skip compacting some heap regions. > But the objs in these regions might have been dead, and their classes might have been unloaded, at this situation, we need following change to make sure we don't get into trouble when calls HeapRegion::block_is_obj(const HeapWord* p) and subsequent calls e.g. to get obj size. Change looks fine, but I would like the comment to be altered a bit. src/hotspot/share/gc/g1/heapRegion.inline.hpp line 148: > 146: // they're not marked in the full gc. > 147: // So, only when ClassUnloading is false, it's safe to tell an obj is indeed an obj when > 148: // it's under the top of the region, otherwise we have to go to the slow path below. This now only talks about the Full GC case and previously we only needed this when unloading during a concurrent cycle. I would like this case to be mentioned as well. Maybe something like: Suggestion: // When class unloading is enabled it is not safe to only consider top() to conclude if the // given pointer is a valid object. The situation can occur both for class unloading in a // Full GC and during a concurrent cycle. // During a Full GC regions can be excluded from compaction due to high live ration, and // because of this there can be stale objects for unloaded classes left in these regions. // During a concurrent cycle class unloading is done after marking is complete and objects // for the unloaded classes will be stale until the regions are collected. Feel free to change the wording. ------------- Changes requested by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3514 From sjohanss at openjdk.java.net Thu Apr 15 17:27:33 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Thu, 15 Apr 2021 17:27:33 GMT Subject: RFR: JDK-8265259: G1: Fix HeapRegion::block_is_obj for unloading class in full gc In-Reply-To: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> References: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> Message-ID: On Thu, 15 Apr 2021 12:07:47 GMT, Hamlin Li wrote: > In JDK-8262068, we have introduced an enhancement to skip compacting some heap regions. > But the objs in these regions might have been dead, and their classes might have been unloaded, at this situation, we need following change to make sure we don't get into trouble when calls HeapRegion::block_is_obj(const HeapWord* p) and subsequent calls e.g. to get obj size. Browsed the code a bit more, don't we need to change the condition here as well? https://github.com/openjdk/jdk/blob/838c11fc0ea5f1118bd5a20d1849e67b4941ecff/src/hotspot/share/gc/g1/heapRegion.inline.hpp#L127 ------------- PR: https://git.openjdk.java.net/jdk/pull/3514 From sjohanss at openjdk.java.net Thu Apr 15 17:37:35 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Thu, 15 Apr 2021 17:37:35 GMT Subject: RFR: 8265119: G1: update_remset_before_rebuild mixes liveness in words with liveness in bytes [v2] In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 10:40:54 GMT, Ivan Walulya wrote: >> Hi all, >> >> Please review this change to rename ConcurrentMark::liveness() to live_words() making it clear that this represents the number of live words between bottom and nTAMS. Adding ConcurrentMark::live_bytes() to return the same value in bytes. Additionally, I add a test for G1MixedGCLiveThresholdPercent flag. >> >> Testing: tier 1-5. > > Ivan Walulya has updated the pull request incrementally with one additional commit since the last revision: > > changes proposed by Thomas review Looks good. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3511 From iwalulya at openjdk.java.net Thu Apr 15 18:15:36 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Thu, 15 Apr 2021 18:15:36 GMT Subject: RFR: 8265119: G1: update_remset_before_rebuild mixes liveness in words with liveness in bytes [v2] In-Reply-To: References: Message-ID: On Thu, 15 Apr 2021 17:34:56 GMT, Stefan Johansson wrote: >> Ivan Walulya has updated the pull request incrementally with one additional commit since the last revision: >> >> changes proposed by Thomas review > > Looks good. Thanks @kstefanj ------------- PR: https://git.openjdk.java.net/jdk/pull/3511 From mli at openjdk.java.net Fri Apr 16 03:59:40 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Fri, 16 Apr 2021 03:59:40 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc [v4] In-Reply-To: <-fvpCX-fvQW9qPu03cretUdHnWrNb30bNssnO9-mc5k=.56470439-483e-4855-b5f8-54fe6917ee77@github.com> References: <81xO8TaRnqmfY3Eey4s-ojYwfZBZZyhcm5ZJ4r0hX3E=.bc4d48eb-e07a-49eb-a26b-cd20bddc2e51@github.com> <-fvpCX-fvQW9qPu03cretUdHnWrNb30bNssnO9-mc5k=.56470439-483e-4855-b5f8-54fe6917ee77@github.com> Message-ID: On Thu, 15 Apr 2021 15:32:20 GMT, Stefan Johansson wrote: >> Hamlin Li has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: >> >> - Merge branch 'master' into fill-bots-in-full-gc >> - refine comments, names. >> - refine the code. >> - JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc > > Looks good. Thanks @kstefanj @tschatzl for review, I think I'm good to integrate this patch now, would you mind to help double confirm? ------------- PR: https://git.openjdk.java.net/jdk/pull/3459 From mli at openjdk.java.net Fri Apr 16 04:00:33 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Fri, 16 Apr 2021 04:00:33 GMT Subject: RFR: JDK-8265259: G1: Fix HeapRegion::block_is_obj for unloading class in full gc In-Reply-To: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> References: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> Message-ID: On Thu, 15 Apr 2021 12:07:47 GMT, Hamlin Li wrote: > In JDK-8262068, we have introduced an enhancement to skip compacting some heap regions. > But the objs in these regions might have been dead, and their classes might have been unloaded, at this situation, we need following change to make sure we don't get into trouble when calls HeapRegion::block_is_obj(const HeapWord* p) and subsequent calls e.g. to get obj size. Nice catch! I will update the patch accordingly. ------------- PR: https://git.openjdk.java.net/jdk/pull/3514 From mli at openjdk.java.net Fri Apr 16 04:00:34 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Fri, 16 Apr 2021 04:00:34 GMT Subject: RFR: JDK-8265259: G1: Fix HeapRegion::block_is_obj for unloading class in full gc In-Reply-To: References: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> Message-ID: On Thu, 15 Apr 2021 17:11:32 GMT, Stefan Johansson wrote: >> In JDK-8262068, we have introduced an enhancement to skip compacting some heap regions. >> But the objs in these regions might have been dead, and their classes might have been unloaded, at this situation, we need following change to make sure we don't get into trouble when calls HeapRegion::block_is_obj(const HeapWord* p) and subsequent calls e.g. to get obj size. > > src/hotspot/share/gc/g1/heapRegion.inline.hpp line 148: > >> 146: // they're not marked in the full gc. >> 147: // So, only when ClassUnloading is false, it's safe to tell an obj is indeed an obj when >> 148: // it's under the top of the region, otherwise we have to go to the slow path below. > > This now only talks about the Full GC case and previously we only needed this when unloading during a concurrent cycle. I would like this case to be mentioned as well. Maybe something like: > Suggestion: > > // When class unloading is enabled it is not safe to only consider top() to conclude if the > // given pointer is a valid object. The situation can occur both for class unloading in a > // Full GC and during a concurrent cycle. > // During a Full GC regions can be excluded from compaction due to high live ration, and > // because of this there can be stale objects for unloaded classes left in these regions. > // During a concurrent cycle class unloading is done after marking is complete and objects > // for the unloaded classes will be stale until the regions are collected. > > > Feel free to change the wording. Thanks for refining the comments. ------------- PR: https://git.openjdk.java.net/jdk/pull/3514 From mli at openjdk.java.net Fri Apr 16 04:13:59 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Fri, 16 Apr 2021 04:13:59 GMT Subject: RFR: JDK-8265259: G1: Fix HeapRegion::block_is_obj for unloading class in full gc [v2] In-Reply-To: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> References: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> Message-ID: > In JDK-8262068, we have introduced an enhancement to skip compacting some heap regions. > But the objs in these regions might have been dead, and their classes might have been unloaded, at this situation, we need following change to make sure we don't get into trouble when calls HeapRegion::block_is_obj(const HeapWord* p) and subsequent calls e.g. to get obj size. Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: fix one more code path, assert; refine comments. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3514/files - new: https://git.openjdk.java.net/jdk/pull/3514/files/7c8562ca..54c04977 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3514&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3514&range=00-01 Stats: 9 lines in 1 file changed: 2 ins; 0 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/3514.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3514/head:pull/3514 PR: https://git.openjdk.java.net/jdk/pull/3514 From tschatzl at openjdk.java.net Fri Apr 16 07:30:43 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 16 Apr 2021 07:30:43 GMT Subject: RFR: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc [v4] In-Reply-To: <81xO8TaRnqmfY3Eey4s-ojYwfZBZZyhcm5ZJ4r0hX3E=.bc4d48eb-e07a-49eb-a26b-cd20bddc2e51@github.com> References: <81xO8TaRnqmfY3Eey4s-ojYwfZBZZyhcm5ZJ4r0hX3E=.bc4d48eb-e07a-49eb-a26b-cd20bddc2e51@github.com> Message-ID: On Thu, 15 Apr 2021 12:15:02 GMT, Hamlin Li wrote: >> in JDK-8262068, we have an enhancement for full gc which will skip compacting some regions with high survivor ratio. There might be some young regions among these skipped regions. These young regions are not filled with BOTs info as they're young. >> But after full GC these young regions will become old regions, and the BOTs info is better to be filled for performance consideration. > > Hamlin Li has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains four commits: > > - Merge branch 'master' into fill-bots-in-full-gc > - refine comments, names. > - refine the code. > - JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc Ship it. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3459 From iwalulya at openjdk.java.net Fri Apr 16 07:32:39 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Fri, 16 Apr 2021 07:32:39 GMT Subject: Integrated: 8265119: G1: update_remset_before_rebuild mixes liveness in words with liveness in bytes In-Reply-To: References: Message-ID: <3qx4KOLPDQshFBVBXWT3nW5tfrh9_uPp0LKePUwzl8c=.b21228a2-b355-43c4-8ca3-86cf88dedede@github.com> On Thu, 15 Apr 2021 09:29:07 GMT, Ivan Walulya wrote: > Hi all, > > Please review this change to rename ConcurrentMark::liveness() to live_words() making it clear that this represents the number of live words between bottom and nTAMS. Adding ConcurrentMark::live_bytes() to return the same value in bytes. Additionally, I add a test for G1MixedGCLiveThresholdPercent flag. > > Testing: tier 1-5. This pull request has now been integrated. Changeset: fc89fe64 Author: Ivan Walulya URL: https://git.openjdk.java.net/jdk/commit/fc89fe64 Stats: 143 lines in 5 files changed: 135 ins; 0 del; 8 mod 8265119: G1: update_remset_before_rebuild mixes liveness in words with liveness in bytes Reviewed-by: tschatzl, sjohanss ------------- PR: https://git.openjdk.java.net/jdk/pull/3511 From tschatzl at openjdk.java.net Fri Apr 16 07:32:37 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 16 Apr 2021 07:32:37 GMT Subject: RFR: 8265119: G1: update_remset_before_rebuild mixes liveness in words with liveness in bytes [v2] In-Reply-To: References: Message-ID: <-Wr063H2QxIJM221SXE4uEurr77SspGnzBKDkl81l-Y=.7becbd1b-2e5a-402c-8c67-1a209ed98618@github.com> On Thu, 15 Apr 2021 10:40:54 GMT, Ivan Walulya wrote: >> Hi all, >> >> Please review this change to rename ConcurrentMark::liveness() to live_words() making it clear that this represents the number of live words between bottom and nTAMS. Adding ConcurrentMark::live_bytes() to return the same value in bytes. Additionally, I add a test for G1MixedGCLiveThresholdPercent flag. >> >> Testing: tier 1-5. > > Ivan Walulya has updated the pull request incrementally with one additional commit since the last revision: > > changes proposed by Thomas review Still good. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3511 From shade at openjdk.java.net Fri Apr 16 07:45:36 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 16 Apr 2021 07:45:36 GMT Subject: RFR: 8265239: Shenandoah: Shenandoah heap region count could be off by 1 In-Reply-To: <-8G7NK-sSvff9jC-zGGQBkkzO9rsoCv_B_tSDhoVlhA=.cbb907e0-f891-46db-af1a-0a2cf520eed4@github.com> References: <-8G7NK-sSvff9jC-zGGQBkkzO9rsoCv_B_tSDhoVlhA=.cbb907e0-f891-46db-af1a-0a2cf520eed4@github.com> Message-ID: <11iX8w9PWODsE9sTprWV7G7wDh1JGjmtY0UppsQJb50=.bfb8a717-b01f-425e-842e-be0858dc5e53@github.com> On Thu, 15 Apr 2021 00:06:05 GMT, Zhengyu Gu wrote: > henandoah uses MaxHeapSize to calculate region size, then aligns up heap size to region size alignment, but when it calculates region count, it does not take alignment into account, that may result 1 region short than actual count. > > MaxHeapSize eventually aligns to HeapAlignment (which is region size in Shenandoah case) in GCArguments::initialize_heap_flags_and_sizes(), but it is too late. > > Test: > - [x] hotspot_gc_shenandoah Looks fine. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3504 From tschatzl at openjdk.java.net Fri Apr 16 07:58:42 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 16 Apr 2021 07:58:42 GMT Subject: RFR: 8265330: G1: Fix comment in G1FullGCPrepareTask::G1CalculatePointersClosure Message-ID: Hi all, can I have reviews for this trivial(?) comment update? Thanks, Thomas Testing: local compilation ------------- Commit messages: - Fix comment Changes: https://git.openjdk.java.net/jdk/pull/3538/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3538&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265330 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3538.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3538/head:pull/3538 PR: https://git.openjdk.java.net/jdk/pull/3538 From shade at openjdk.java.net Fri Apr 16 08:10:34 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 16 Apr 2021 08:10:34 GMT Subject: RFR: 8265330: G1: Fix comment in G1FullGCPrepareTask::G1CalculatePointersClosure In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 07:53:07 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this trivial(?) comment update? > > Thanks, > Thomas > > Testing: local compilation Yes, looks trivial. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3538 From mli at openjdk.java.net Fri Apr 16 08:10:35 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Fri, 16 Apr 2021 08:10:35 GMT Subject: Integrated: JDK-8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc In-Reply-To: References: Message-ID: <3kRA3u1w6rKNZ7BuW-htAI4aEs_2oICqLm13ERzeGzU=.60dc3c0d-c576-47d6-b49b-9c540be0789d@github.com> On Tue, 13 Apr 2021 10:07:17 GMT, Hamlin Li wrote: > in JDK-8262068, we have an enhancement for full gc which will skip compacting some regions with high survivor ratio. There might be some young regions among these skipped regions. These young regions are not filled with BOTs info as they're young. > But after full GC these young regions will become old regions, and the BOTs info is better to be filled for performance consideration. This pull request has now been integrated. Changeset: b4ba74e3 Author: Hamlin Li URL: https://git.openjdk.java.net/jdk/commit/b4ba74e3 Stats: 25 lines in 2 files changed: 24 ins; 1 del; 0 mod 8264987: G1: Fill BOTs for Survivor-turned-to-Old regions in full gc Co-authored-by: Shoubing Ma Reviewed-by: tschatzl, sjohanss ------------- PR: https://git.openjdk.java.net/jdk/pull/3459 From mli at openjdk.java.net Fri Apr 16 08:10:34 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Fri, 16 Apr 2021 08:10:34 GMT Subject: RFR: 8265330: G1: Fix comment in G1FullGCPrepareTask::G1CalculatePointersClosure In-Reply-To: References: Message-ID: <5OHZsrnPj3g4iZo7su6qYs1QMiGDdehqnMS40IzkVWs=.3704d8a0-8f49-4006-a3ea-8004f06198ed@github.com> On Fri, 16 Apr 2021 07:53:07 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this trivial(?) comment update? > > Thanks, > Thomas > > Testing: local compilation LGTM, thanks ------------- Marked as reviewed by mli (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3538 From tschatzl at openjdk.java.net Fri Apr 16 09:13:03 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 16 Apr 2021 09:13:03 GMT Subject: RFR: 8265330: G1: Fix comment in G1FullGCPrepareTask::G1CalculatePointersClosure [v2] In-Reply-To: References: Message-ID: > Hi all, > > can I have reviews for this trivial(?) comment update? > > Thanks, > Thomas > > Testing: local compilation Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: - Merge branch 'master' into 8265330-fix-comment - Fix comment ------------- Changes: https://git.openjdk.java.net/jdk/pull/3538/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3538&range=01 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3538.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3538/head:pull/3538 PR: https://git.openjdk.java.net/jdk/pull/3538 From tschatzl at openjdk.java.net Fri Apr 16 09:30:37 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 16 Apr 2021 09:30:37 GMT Subject: RFR: 8265330: G1: Fix comment in G1FullGCPrepareTask::G1CalculatePointersClosure [v2] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 08:05:39 GMT, Aleksey Shipilev wrote: >> Thomas Schatzl has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: >> >> - Merge branch 'master' into 8265330-fix-comment >> - Fix comment > > Yes, looks trivial. Thanks @shipilev @Hamlin-Li for your reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/3538 From tschatzl at openjdk.java.net Fri Apr 16 09:30:37 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 16 Apr 2021 09:30:37 GMT Subject: Integrated: 8265330: G1: Fix comment in G1FullGCPrepareTask::G1CalculatePointersClosure In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 07:53:07 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this trivial(?) comment update? > > Thanks, > Thomas > > Testing: local compilation This pull request has now been integrated. Changeset: 1d66a155 Author: Thomas Schatzl URL: https://git.openjdk.java.net/jdk/commit/1d66a155 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod 8265330: G1: Fix comment in G1FullGCPrepareTask::G1CalculatePointersClosure Reviewed-by: shade, mli ------------- PR: https://git.openjdk.java.net/jdk/pull/3538 From erik.osterlund at oracle.com Fri Apr 16 09:48:49 2021 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 16 Apr 2021 11:48:49 +0200 Subject: [External] : Re: Work-in-progress: 8236485: Epoch synchronization protocol for G1 concurrent refinement In-Reply-To: References: <87blr3vc4t.fsf@oldenburg2.str.redhat.com> Message-ID: <77227c55-93bb-3fef-7fed-1503f54cf523@oracle.com> Hi Man, On 2021-03-31 04:43, Man Cao wrote: > Hi all, > > I finally managed to allocate more time to make progress on this, and > resolved most issues since the last discussion. > I've updated the description in > https://bugs.openjdk.java.net/browse/JDK-8236485 > , and the current > prototype is the HEAD commit at > https://github.com/caoman/jdk/tree/g1EpochSync > . > Notable changes include: > - The protocol uses async handshake?from JDK-8238761 > ?to resolve the > blocking issue from normal handshake. > - In order to support async refinement?due to async handshake, added > support for?_deferred global queue to G1DirtyCardQueueSet. Buffers > rarely get enqueued to _deferred at run-time. > - The async handshake only executes for a subset of threads. > > I have a couple of questions: > > 1. Code review and patch size. > Should I start a pull request for this change, so it is easier to give > feedback? > > What is the recommended approach to deal with large changes? Currently > the patch is about 1200 lines, without changing the write barrier itself > (JDK-8226731). > Since pushing only this patch will add some overhead to refinement, but > not bring any performance improvement from removing the write barrier's > fence, do you recommend I also implement the write barrier change in the > same patch? > Keeping the epoch sync patch separate from the write barrier patch has > some benefit for testing, in case the?epoch patch introduces any bugs. > Currently the new code is mostly guarded by a flag > -XX:+G1TestEpochSyncInConcRefinement, and it will be removed after the > write barrier change. It could be used as an emergency flag to > workaround bugs, instead of backing out of the entire change. We > probably cannot have such a flag if we bundle the changes in one patch > (it's too ugly to have a flag in interpreter and compilers). Since the second part is so small, I'd definitely contribute this as a single patch where the complexity vs gain is more possible to evaluate. > 2. Checking if a remote thread is in?_thread_in_Java?state. > eosterlund@ pointed out it was incorrect to?do > JavaThread::thread_state() ==?_thread_in_Java. > I looked into thread state transition, and revised it to also compare > with _thread_in_native_trans and?_thread_in_vm_trans. > I think it is correct now for the purpose of epoch synchronization. > I.e., it never "misses" a remote thread that is actually in in_Java state. > A detailed comment is here: > https://github.com/caoman/jdk/blob/2047ecefceb074e80d73e0d521d64a220fdc5779/src/hotspot/share/gc/g1/g1EpochSynchronizer.cpp#L67-L90 > . > *Erik, could you take a look and decide if it is correct? If it is still > incorrect, could you advise a proper way to do this?* This is still incorrect. Let's start with the why and then what to do about it. A thread that has not armed the poll value of remote JavaThreads, may not read said thread's state to draw any conclusions from it whatsoever. Even if you had, the way you can reason about remote thread states is typically only to prove threads are in blocked or native, not to prove they are in Java (excluding the vm state). And the way that the different transitions are fenced reflect that, so that only that can be determined. The reasoning is that the states are there mostly for safepoint/handshake synchronization. And they can wait for responsive threads, but not threads that won't respond (blocked and native). I don't get why you want this filtering though. Seems like a premature optimization to in practice not poke threads that are blocked or in native. But those threads are handled efficiently by the handshaking mechanism anyway, as the handshaking mechanism already filters out such threads (but in a correct way), and when they come back they will quickly disarm themselves and continue. So I think the solution is to just handshake all threads and be done with it, letting the handshaking mechanism perform the proper filtering in a context where it is valid. I don't think it will make any perf difference. > 3. Native write barrier (G1BarrierSet::write_ref_field_post). > The epoch synchronization protocol does not synchronize with threads in > _thread_in_native or?_thread_in_vm state. It is much slower if we > synchronize with such threads. Is it really though? Threads are in VM for a short time by design (not to impact time to safepoint), and threads in native will have the handshake operation performed by the handshaking thread (a no-op), and then continue. The handshakee will just see it needs to flip a bit when coming back from native. I can't understand why it would make a difference. Do you have numbers showing this? > Moreover, there are non-Java threads (e.g.?concurrent mark worker) that > could execute the native write barrier. > As a result, it's probably best to keep the StoreLoad fence in the > native write barrier. Yes, runtime barriers are stone cold, so shouldn't matter if it performs a fence. > The final write post-barrier for JDK-8226731 would be: > Given: > x.a = q > and > p = @x.a > > For Interpreter/C1/C2: > if (p and q in same regions or q == NULL) -> exit > if (card(p) == Dirty) -> exit > card(p) = Dirty; > enqueue(card(p)) > > For the native barrier: > if (p and q in same regions or q == NULL) -> exit > StoreLoad; > if (card(p) == Dirty) -> exit > card(p) = Dirty; > enqueue(card(p)) > > Does the above look?good? Do we need to micro-benchmark the potential > overhead added to the native barrier? Or are typical macro-benchmarks > sufficient? > > 4. Performance benchmarking. > I did some preliminary benchmarking with DaCapo and BigRamTester, > without changing the write barrier. This is to measure the overhead > added by the epoch synchronization protocol. > Under the default JVM setting, I didn't see any performance difference. > When I tuned the JVM to refine cards aggressively, there was > still?no?difference in BigRamTester (probably because it only has 2 > threads). Some DaCapo benchmarks saw 10-15% more CPU usage due to doing > more work in the refinement threads, and 2-5% total throughput > regression for these benchmarks. > The aggressive refinement flags are "-XX:-G1UseAdaptiveConcRefinement > -XX:G1UpdateBufferSize=4 -XX:G1ConcRefinementGreenZone=0 > -XX:G1ConcRefinementYellowZone=1". > > I wonder how important we should treat the aggressive refinement case. > Regression in this case is likely unavoidable, so how much regression is > tolerable? > Also, does?anyone know a better benchmark to test refinement with > default JVM settings? Ideally it (1) has many mutator threads; (2) > triggers concurrent refinement frequently; (3) runs with a sizable Xmx > (e.g., 2GiB or above). I'll probably leave the rest of the discussion to other G1 folks. /Erik > -Man > > > On Thu, Jan 16, 2020 at 4:06 AM Florian Weimer > wrote: > > * Man Cao: > > > We had an offline discussion on this. To keep the community in > the loop, > > here is what we discussed. > > > > a. Using Linux membarrier syscall or equivalent on other OSes seems a > > cleaner solution than thread-local handshake (TLH). But we need > to have a > > backup mechanism for OSes and older Linuxes that do not have such a > > syscall. > > Can you do with a membarrier call that doesn't require registration? > > The usual fallback for membarrier is sending a special signal to all > threads, and make sure that they have run code in a signal handler > (possibly using a CPU barrier there).? But of course this is rather > slow. > > membarrier has seen some backporting activity, but as far as I can see, > that hasn't been consistent across architectures. > > Thanks, > Florian > From iwalulya at openjdk.java.net Fri Apr 16 10:01:33 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Fri, 16 Apr 2021 10:01:33 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates [v2] In-Reply-To: References: Message-ID: > Hi all, > > Please review this change that skips the rebuild remset phase and the associated updates after rebuild, If during Remark, G1 finds that there are no remembered sets to rebuild. > > Test: tier1-5 Ivan Walulya has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: - Fix issues raised in Thomas review, and add test - Merge remote-tracking branch 'upstream/master' into JDK-8199407 - remove trailing white space - Merge branch 'master' into JDK-8199407 - variable name refactor - variable name refactor - skip iterations that depend on total_selected_for_rebuild - skip the rebuild remembered set phase if remark did not select any regions for rebuilding ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3434/files - new: https://git.openjdk.java.net/jdk/pull/3434/files/9022646e..b9233696 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3434&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3434&range=00-01 Stats: 39977 lines in 1660 files changed: 8245 ins; 27492 del; 4240 mod Patch: https://git.openjdk.java.net/jdk/pull/3434.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3434/head:pull/3434 PR: https://git.openjdk.java.net/jdk/pull/3434 From shade at openjdk.java.net Fri Apr 16 10:23:55 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 16 Apr 2021 10:23:55 GMT Subject: RFR: 8265335: Epsilon: Minor typo in EpsilonElasticTLABDecay description Message-ID: Trivial: should be "shrink", not "shrik". ------------- Commit messages: - 8265335: Epsilon: Minor typo in EpsilonElasticTLABDecay description Changes: https://git.openjdk.java.net/jdk/pull/3543/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3543&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265335 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3543.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3543/head:pull/3543 PR: https://git.openjdk.java.net/jdk/pull/3543 From lkorinth at openjdk.java.net Fri Apr 16 10:59:38 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Fri, 16 Apr 2021 10:59:38 GMT Subject: RFR: 8231672: Simplify the reference processing parallelization framework [v4] In-Reply-To: References: <-rOOA973lhaMao-B9DGErn5L7_lKP3m9sakfvKfxKB0=.ec9ecf61-0f7e-49a4-8370-b3755a7ef059@github.com> Message-ID: On Thu, 25 Mar 2021 08:00:58 GMT, Kim Barrett wrote: >> The `ResourceMark`s was used in _some_ of the previous nested closures. See removal of `G1STWRefProcTaskProxy` and `G1CMRefProcTaskProxy`. > > Yes, and I'm not sure why any of them exist, other than some paranoia that there might be some not properly scoped resource allocations somewhere inside that we want to squash on the way out of the thread. That is, these seem like just hiding potential bugs. But you probably don't want to chase down any actual bugs of that sort as part of this change, so okay, leave them. Hi, and sorry for the long silence from me. I got feedback from Albert on improving the structure of this change and to use proxy classes instead of ClosureContexts. This will solve the problem with heap allocated closures and also improve readability IMO. The Change is big, so I will not push it to this review thread before you agree with that it is the right way to go, but you can see the change here: https://github.com/lkorinth/jdk/commit/5cdddacd037c7ea4baf648262b9637540c5ec3e4 If you agree with me, I will push that change to this review. ------------- PR: https://git.openjdk.java.net/jdk/pull/2782 From tschatzl at openjdk.java.net Fri Apr 16 11:59:36 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 16 Apr 2021 11:59:36 GMT Subject: RFR: JDK-8265259: G1: Fix HeapRegion::block_is_obj for unloading class in full gc [v2] In-Reply-To: References: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> Message-ID: <8gD25_Yebk85NcCZvj-_v9uTml2SppyDofYEAI5TqhA=.ffdf0771-6ae3-4cab-a673-d33a890599b2@github.com> On Fri, 16 Apr 2021 04:13:59 GMT, Hamlin Li wrote: >> In JDK-8262068, we have introduced an enhancement to skip compacting some heap regions. >> But the objs in these regions might have been dead, and their classes might have been unloaded, at this situation, we need following change to make sure we don't get into trouble when calls HeapRegion::block_is_obj(const HeapWord* p) and subsequent calls e.g. to get obj size. > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > fix one more code path, assert; refine comments. Lgtm. Thanks for following up on this. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3514 From sjohanss at openjdk.java.net Fri Apr 16 12:03:41 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Fri, 16 Apr 2021 12:03:41 GMT Subject: RFR: JDK-8265259: G1: Fix HeapRegion::block_is_obj for unloading class in full gc [v2] In-Reply-To: References: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> Message-ID: On Fri, 16 Apr 2021 04:13:59 GMT, Hamlin Li wrote: >> In JDK-8262068, we have introduced an enhancement to skip compacting some heap regions. >> But the objs in these regions might have been dead, and their classes might have been unloaded, at this situation, we need following change to make sure we don't get into trouble when calls HeapRegion::block_is_obj(const HeapWord* p) and subsequent calls e.g. to get obj size. > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > fix one more code path, assert; refine comments. This looks good. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3514 From tschatzl at openjdk.java.net Fri Apr 16 12:06:38 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 16 Apr 2021 12:06:38 GMT Subject: RFR: 8265335: Epsilon: Minor typo in EpsilonElasticTLABDecay description In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 10:16:50 GMT, Aleksey Shipilev wrote: > Trivial: should be "shrink", not "shrik". Lgtm and trivial. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3543 From iwalulya at openjdk.java.net Fri Apr 16 12:07:50 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Fri, 16 Apr 2021 12:07:50 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates [v2] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 12:00:52 GMT, Thomas Schatzl wrote: >> Ivan Walulya has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: >> >> - Fix issues raised in Thomas review, and add test >> - Merge remote-tracking branch 'upstream/master' into JDK-8199407 >> - remove trailing white space >> - Merge branch 'master' into JDK-8199407 >> - variable name refactor >> - variable name refactor >> - skip iterations that depend on total_selected_for_rebuild >> - skip the rebuild remembered set phase if remark did not select any regions for rebuilding > > test/hotspot/jtreg/gc/g1/TestSkipRebuildRemsetPhase.java line 85: > >> 83: >> 84: private static Object alloc() { >> 85: // G1MixedGCLiveThresholdPercent=20 > > Maybe this comment could be elaborated a bit further - i.e. "Since G1Mixed...ent is 20%, make sure to allocate something that is larger than that so that it will not be collected and the expected message printed." > Or just remove it. I think it is better to elaborate than remove. ------------- PR: https://git.openjdk.java.net/jdk/pull/3434 From tschatzl at openjdk.java.net Fri Apr 16 12:07:45 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 16 Apr 2021 12:07:45 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates [v2] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 10:01:33 GMT, Ivan Walulya wrote: >> Hi all, >> >> Please review this change that skips the rebuild remset phase and the associated updates after rebuild, If during Remark, G1 finds that there are no remembered sets to rebuild. >> >> Test: tier1-5 > > Ivan Walulya has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: > > - Fix issues raised in Thomas review, and add test > - Merge remote-tracking branch 'upstream/master' into JDK-8199407 > - remove trailing white space > - Merge branch 'master' into JDK-8199407 > - variable name refactor > - variable name refactor > - skip iterations that depend on total_selected_for_rebuild > - skip the rebuild remembered set phase if remark did not select any regions for rebuilding Lgtm, maybe that comment could be improved. Forgot last check for whether the messages are at an appropriate level, so reverting back to "Request changes". Sorry. test/hotspot/jtreg/gc/g1/TestSkipRebuildRemsetPhase.java line 85: > 83: > 84: private static Object alloc() { > 85: // G1MixedGCLiveThresholdPercent=20 Maybe this comment could be elaborated a bit further - i.e. "Since G1Mixed...ent is 20%, make sure to allocate something that is larger than that so that it will not be collected and the expected message printed." Or just remove it. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3434Changes requested by tschatzl (Reviewer). From shade at openjdk.java.net Fri Apr 16 12:13:35 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 16 Apr 2021 12:13:35 GMT Subject: RFR: 8265335: Epsilon: Minor typo in EpsilonElasticTLABDecay description In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 10:16:50 GMT, Aleksey Shipilev wrote: > Trivial: should be "shrink", not "shrik". Thanks! ------------- PR: https://git.openjdk.java.net/jdk/pull/3543 From shade at openjdk.java.net Fri Apr 16 12:13:36 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 16 Apr 2021 12:13:36 GMT Subject: Integrated: 8265335: Epsilon: Minor typo in EpsilonElasticTLABDecay description In-Reply-To: References: Message-ID: <0kXJmBI5ic4ElpSBewbV7szRmI9LSge-gIgkXIC4Me4=.f7871d88-24d4-4171-b1d9-8db192f8a112@github.com> On Fri, 16 Apr 2021 10:16:50 GMT, Aleksey Shipilev wrote: > Trivial: should be "shrink", not "shrik". This pull request has now been integrated. Changeset: 17b6592d Author: Aleksey Shipilev URL: https://git.openjdk.java.net/jdk/commit/17b6592d Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8265335: Epsilon: Minor typo in EpsilonElasticTLABDecay description Reviewed-by: tschatzl ------------- PR: https://git.openjdk.java.net/jdk/pull/3543 From zgu at openjdk.java.net Fri Apr 16 12:30:36 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Fri, 16 Apr 2021 12:30:36 GMT Subject: Integrated: 8265239: Shenandoah: Shenandoah heap region count could be off by 1 In-Reply-To: <-8G7NK-sSvff9jC-zGGQBkkzO9rsoCv_B_tSDhoVlhA=.cbb907e0-f891-46db-af1a-0a2cf520eed4@github.com> References: <-8G7NK-sSvff9jC-zGGQBkkzO9rsoCv_B_tSDhoVlhA=.cbb907e0-f891-46db-af1a-0a2cf520eed4@github.com> Message-ID: On Thu, 15 Apr 2021 00:06:05 GMT, Zhengyu Gu wrote: > henandoah uses MaxHeapSize to calculate region size, then aligns up heap size to region size alignment, but when it calculates region count, it does not take alignment into account, that may result 1 region short than actual count. > > MaxHeapSize eventually aligns to HeapAlignment (which is region size in Shenandoah case) in GCArguments::initialize_heap_flags_and_sizes(), but it is too late. > > Test: > - [x] hotspot_gc_shenandoah This pull request has now been integrated. Changeset: ff5bb8cf Author: Zhengyu Gu URL: https://git.openjdk.java.net/jdk/commit/ff5bb8cf Stats: 2 lines in 2 files changed: 1 ins; 0 del; 1 mod 8265239: Shenandoah: Shenandoah heap region count could be off by 1 Reviewed-by: shade ------------- PR: https://git.openjdk.java.net/jdk/pull/3504 From mli at openjdk.java.net Fri Apr 16 12:44:36 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Fri, 16 Apr 2021 12:44:36 GMT Subject: Integrated: JDK-8265259: G1: Fix HeapRegion::block_is_obj for unloading class in full gc In-Reply-To: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> References: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> Message-ID: On Thu, 15 Apr 2021 12:07:47 GMT, Hamlin Li wrote: > In JDK-8262068, we have introduced an enhancement to skip compacting some heap regions. > But the objs in these regions might have been dead, and their classes might have been unloaded, at this situation, we need following change to make sure we don't get into trouble when calls HeapRegion::block_is_obj(const HeapWord* p) and subsequent calls e.g. to get obj size. This pull request has now been integrated. Changeset: 714298a5 Author: Hamlin Li URL: https://git.openjdk.java.net/jdk/commit/714298a5 Stats: 10 lines in 1 file changed: 7 ins; 0 del; 3 mod 8265259: G1: Fix HeapRegion::block_is_obj for unloading class in full gc Reviewed-by: sjohanss, tschatzl ------------- PR: https://git.openjdk.java.net/jdk/pull/3514 From mli at openjdk.java.net Fri Apr 16 12:44:35 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Fri, 16 Apr 2021 12:44:35 GMT Subject: RFR: JDK-8265259: G1: Fix HeapRegion::block_is_obj for unloading class in full gc [v2] In-Reply-To: References: <7AC0Hx-mxOrJnuktlB_sdMipDVpm5UdZcQLaB37An7w=.e83b08f1-19a0-4bd9-b168-35d8954b4bc9@github.com> Message-ID: On Fri, 16 Apr 2021 04:13:59 GMT, Hamlin Li wrote: >> In JDK-8262068, we have introduced an enhancement to skip compacting some heap regions. >> But the objs in these regions might have been dead, and their classes might have been unloaded, at this situation, we need following change to make sure we don't get into trouble when calls HeapRegion::block_is_obj(const HeapWord* p) and subsequent calls e.g. to get obj size. > > Hamlin Li has updated the pull request incrementally with one additional commit since the last revision: > > fix one more code path, assert; refine comments. ------------- PR: https://git.openjdk.java.net/jdk/pull/3514 From tschatzl at openjdk.java.net Fri Apr 16 14:00:54 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 16 Apr 2021 14:00:54 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates [v2] In-Reply-To: References: Message-ID: <11NaZghkYpDG8rFCukrrzp_Mh7oUcsM02ikcJNzLISU=.33978659-aeae-4ae9-b363-1fc486900938@github.com> On Fri, 16 Apr 2021 10:01:33 GMT, Ivan Walulya wrote: >> Hi all, >> >> Please review this change that skips the rebuild remset phase and the associated updates after rebuild, If during Remark, G1 finds that there are no remembered sets to rebuild. >> >> Test: tier1-5 > > Ivan Walulya has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: > > - Fix issues raised in Thomas review, and add test > - Merge remote-tracking branch 'upstream/master' into JDK-8199407 > - remove trailing white space > - Merge branch 'master' into JDK-8199407 > - variable name refactor > - variable name refactor > - skip iterations that depend on total_selected_for_rebuild > - skip the rebuild remembered set phase if remark did not select any regions for rebuilding Lgtm after looking at some logs and playing with some log options. Sorry for the confusion. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3434 From iwalulya at openjdk.java.net Fri Apr 16 14:24:06 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Fri, 16 Apr 2021 14:24:06 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates [v3] In-Reply-To: References: Message-ID: > Hi all, > > Please review this change that skips the rebuild remset phase and the associated updates after rebuild, If during Remark, G1 finds that there are no remembered sets to rebuild. > > Test: tier1-5 Ivan Walulya has updated the pull request incrementally with one additional commit since the last revision: change comment ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3434/files - new: https://git.openjdk.java.net/jdk/pull/3434/files/b9233696..41744103 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3434&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3434&range=01-02 Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3434.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3434/head:pull/3434 PR: https://git.openjdk.java.net/jdk/pull/3434 From iwalulya at openjdk.java.net Fri Apr 16 14:24:10 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Fri, 16 Apr 2021 14:24:10 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates [v2] In-Reply-To: <11NaZghkYpDG8rFCukrrzp_Mh7oUcsM02ikcJNzLISU=.33978659-aeae-4ae9-b363-1fc486900938@github.com> References: <11NaZghkYpDG8rFCukrrzp_Mh7oUcsM02ikcJNzLISU=.33978659-aeae-4ae9-b363-1fc486900938@github.com> Message-ID: On Fri, 16 Apr 2021 13:57:30 GMT, Thomas Schatzl wrote: >> Ivan Walulya has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision: >> >> - Fix issues raised in Thomas review, and add test >> - Merge remote-tracking branch 'upstream/master' into JDK-8199407 >> - remove trailing white space >> - Merge branch 'master' into JDK-8199407 >> - variable name refactor >> - variable name refactor >> - skip iterations that depend on total_selected_for_rebuild >> - skip the rebuild remembered set phase if remark did not select any regions for rebuilding > > Lgtm after looking at some logs and playing with some log options. Sorry for the confusion. Thanks @tschatzl for the review ------------- PR: https://git.openjdk.java.net/jdk/pull/3434 From zgu at openjdk.java.net Fri Apr 16 14:34:58 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Fri, 16 Apr 2021 14:34:58 GMT Subject: RFR: 8265348: Shenandoah: Use timed wait to sleep control thread Message-ID: I would like to bring the patch with the same synopsis from Shenandoah repo [1] to mainline, it should help what I am working on. Test: - [x] hotspot_gc_shenandoah [1] https://github.com/openjdk/shenandoah/pull/28 ------------- Commit messages: - Merge branch 'master' into JDK-8265348-control-timed-wait - init Changes: https://git.openjdk.java.net/jdk/pull/3547/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3547&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265348 Stats: 16 lines in 2 files changed: 15 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3547.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3547/head:pull/3547 PR: https://git.openjdk.java.net/jdk/pull/3547 From zgu at openjdk.java.net Fri Apr 16 14:34:58 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Fri, 16 Apr 2021 14:34:58 GMT Subject: RFR: 8265348: Shenandoah: Use timed wait to sleep control thread In-Reply-To: References: Message-ID: <_TC5a8CZBDq0pyz6GGxnIV9GPRZ285V1c-X-NL8KYyQ=.c37015fc-8851-498c-810d-3e387341105c@github.com> On Fri, 16 Apr 2021 14:27:29 GMT, Zhengyu Gu wrote: > I would like to bring the patch with the same synopsis from Shenandoah repo [1] to mainline, it should help what I am working on. > > Test: > - [x] hotspot_gc_shenandoah > > [1] https://github.com/openjdk/shenandoah/pull/28 add author @earthling-amzn ------------- PR: https://git.openjdk.java.net/jdk/pull/3547 From lkorinth at openjdk.java.net Fri Apr 16 14:37:39 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Fri, 16 Apr 2021 14:37:39 GMT Subject: RFR: 8231672: Simplify the reference processing parallelization framework [v4] In-Reply-To: <-rOOA973lhaMao-B9DGErn5L7_lKP3m9sakfvKfxKB0=.ec9ecf61-0f7e-49a4-8370-b3755a7ef059@github.com> References: <-rOOA973lhaMao-B9DGErn5L7_lKP3m9sakfvKfxKB0=.ec9ecf61-0f7e-49a4-8370-b3755a7ef059@github.com> Message-ID: On Thu, 11 Mar 2021 10:18:27 GMT, Leo Korinth wrote: >> With the change of parallel gc to use the same workgang implementation that other gcs uses, it is possible to remove the abstraction used to hide different worker thread management implementations (TaskExecutor). >> >> At the same time as removing the TaskExecutor, also remove part of the special casing (mostly code duplication), for single threaded execution. >> >> I consider the new method `prepare_run_task` that modifies the state of the context a step backwards from what was. However, I think removing the Executor with its proxy-tasks and removing separate code paths for serial execution makes the change worth this step back. We also removes ~270 lines of code. >> >> Some comments: >> 1) I use references in some places where they can replace pointers. I could go much further here, but I did not want to bloat the pull request, maybe later change all closures to be references as well? Should I create enhancements for this? >> >> 2) I added an enum class ThreadModel instead of using a boolean, this could also be used in many more places. I dislike sending lots of bools with a comment like `true /* _is_mt */`. It also adds type safety if a method takes many bools. However, with this limited change, and not many hard-coded bools, it feels a bit overkill and I am willing to remove the enum, what do you think? >> >> Testing: >> hotspot_gc and tier 1-3 has passed earlier versions before minor cleanups. I intend to re-run tests after review changes. > > Leo Korinth has updated the pull request incrementally with one additional commit since the last revision: > > remove maybe prefix, change in one place to nullable Hi, and sorry for the long silence from me. I got feedback from Albert on improving the structure of this change and to use proxy classes instead of ClosureContexts. This will solve the problem with heap allocated closures and also improve readability IMO. The change is big, so I will not push it to this review thread before you agree with that it is the right way to go, but you can see the change here: https://github.com/lkorinth/jdk/commit/5cdddacd037c7ea4baf648262b9637540c5ec3e4 If you agree with me, I will push that change to this review. ------------- PR: https://git.openjdk.java.net/jdk/pull/2782 From shade at openjdk.java.net Fri Apr 16 14:58:39 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Fri, 16 Apr 2021 14:58:39 GMT Subject: RFR: 8265348: Shenandoah: Use timed wait to sleep control thread In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 14:27:29 GMT, Zhengyu Gu wrote: > I would like to bring the patch with the same synopsis from Shenandoah repo [1] to mainline, it should help what I am working on. > > Test: > - [x] hotspot_gc_shenandoah > > [1] https://github.com/openjdk/shenandoah/pull/28 Wait, there are questions about lock ranking. I wanted to address it in https://bugs.openjdk.java.net/browse/JDK-8264851. ------------- PR: https://git.openjdk.java.net/jdk/pull/3547 From zgu at openjdk.java.net Fri Apr 16 15:14:34 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Fri, 16 Apr 2021 15:14:34 GMT Subject: Withdrawn: 8265348: Shenandoah: Use timed wait to sleep control thread In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 14:27:29 GMT, Zhengyu Gu wrote: > I would like to bring the patch with the same synopsis from Shenandoah repo [1] to mainline, it should help what I am working on. > > Test: > - [x] hotspot_gc_shenandoah > > [1] https://github.com/openjdk/shenandoah/pull/28 This pull request has been closed without being integrated. ------------- PR: https://git.openjdk.java.net/jdk/pull/3547 From zgu at openjdk.java.net Fri Apr 16 15:14:34 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Fri, 16 Apr 2021 15:14:34 GMT Subject: RFR: 8265348: Shenandoah: Use timed wait to sleep control thread In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 14:55:51 GMT, Aleksey Shipilev wrote: > Wait, there are questions about lock ranking. I wanted to address it in https://bugs.openjdk.java.net/browse/JDK-8264851. I did not realize JDK-8264851. I am closing this as duplicate. ------------- PR: https://git.openjdk.java.net/jdk/pull/3547 From kvn at openjdk.java.net Fri Apr 16 17:31:37 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Fri, 16 Apr 2021 17:31:37 GMT Subject: RFR: 8265322: C2: Simplify control inputs for BarrierSetC2::obj_allocate In-Reply-To: <_NicB3VpqFwTiu2-bKLNhfnQAxJmbtND66g3Md5ck5g=.f9b4edcd-fa2b-4917-9092-bccc38b202df@github.com> References: <_NicB3VpqFwTiu2-bKLNhfnQAxJmbtND66g3Md5ck5g=.f9b4edcd-fa2b-4917-9092-bccc38b202df@github.com> Message-ID: On Fri, 16 Apr 2021 03:06:19 GMT, Yi Yang wrote: > This PR simplifies control inputs for BarrierSetC2::obj_allocate. In most cases, it doesn't change anything since `toobig_false` is equivalent to `ctrl`. In rare case, `toobig_false` is created for Unsafe.allocateInstance while instance size is not statically known, `ctrl` would become control input of IfNode whose projects are `toobig_false` and `toobig_true`, old eden_end and old_eden_top can simply accept `toobig_false` as their control input rather than `ctrl`. This should be reviewed by GC group who owns this code. ------------- PR: https://git.openjdk.java.net/jdk/pull/3529 From kvn at openjdk.java.net Fri Apr 16 17:39:40 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Fri, 16 Apr 2021 17:39:40 GMT Subject: RFR: 8265322: C2: Simplify control inputs for BarrierSetC2::obj_allocate In-Reply-To: <_NicB3VpqFwTiu2-bKLNhfnQAxJmbtND66g3Md5ck5g=.f9b4edcd-fa2b-4917-9092-bccc38b202df@github.com> References: <_NicB3VpqFwTiu2-bKLNhfnQAxJmbtND66g3Md5ck5g=.f9b4edcd-fa2b-4917-9092-bccc38b202df@github.com> Message-ID: On Fri, 16 Apr 2021 03:06:19 GMT, Yi Yang wrote: > This PR simplifies control inputs for BarrierSetC2::obj_allocate. In most cases, it doesn't change anything since `toobig_false` is equivalent to `ctrl`. In rare case, `toobig_false` is created for Unsafe.allocateInstance while instance size is not statically known, `ctrl` would become control input of IfNode whose projects are `toobig_false` and `toobig_true`, old eden_end and old_eden_top can simply accept `toobig_false` as their control input rather than `ctrl`. >From compiler code POV the fix is reasonable and correct. Note, the path when `initial_slow_test != NULL` is not rare. It is frequent for arrays allocation when `length` is not constant. ------------- Marked as reviewed by kvn (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3529 From adityam at openjdk.java.net Sat Apr 17 01:15:33 2021 From: adityam at openjdk.java.net (Aditya Mandaleeka) Date: Sat, 17 Apr 2021 01:15:33 GMT Subject: RFR: 8257774: G1: Trigger collect when free region count drops below threshold to prevent evacuation failures [v3] In-Reply-To: References: <5Ro0P4NPaUuG-LPpgqHY5S_kh_EVpwRgxqc5jTCano4=.afc3f2a4-dc9e-46c1-ac84-298cf143fa41@github.com> Message-ID: On Thu, 15 Apr 2021 10:00:11 GMT, Thomas Schatzl wrote: >> Aditya Mandaleeka has updated the pull request incrementally with one additional commit since the last revision: >> >> Fix format specifiers. > > During some internal discussion with the people that had looked at the change a bit, we agreed that the additional `attempt_lock_free_first` parameter is very ugly and complicates the code a lot. > > [Here](https://github.com/tschatzl/jdk/commit/6c10cb3e6d71fcbb487385e23ea77d3c29f31b4d) is an attempt to remove this parameter by an explicit method that allocates using a new region. It does look nicer to me. No testing has been performed, just to see how it would look like. > > The change also removes that `use_retained_region_if_available` parameter form the lock-free `attempt_allocation` call. With your recent changes, all paths should try it (there has been a case where this was not true, but I think that one was errorneous), so I removed that parameter too. > > Please also merge with latest once more, the change is a bit out of date. Thanks @tschatzl, will review the feedback, re-test, and update the PR (likely sometime next week). ------------- PR: https://git.openjdk.java.net/jdk/pull/3143 From manc at google.com Sat Apr 17 04:07:14 2021 From: manc at google.com (Man Cao) Date: Fri, 16 Apr 2021 21:07:14 -0700 Subject: Work-in-progress: 8236485: Epoch synchronization protocol for G1 concurrent refinement In-Reply-To: <77227c55-93bb-3fef-7fed-1503f54cf523@oracle.com> References: <87blr3vc4t.fsf@oldenburg2.str.redhat.com> <77227c55-93bb-3fef-7fed-1503f54cf523@oracle.com> Message-ID: Hi Erik, Thank you for the feedback! A thread that has not armed the poll value of remote JavaThreads, may > not read said thread's state to draw any conclusions from it whatsoever. > Even if you had, the way you can reason about remote thread states is > typically only to prove threads are in blocked or native, not to prove > they are in Java (excluding the vm state). And the way that the > different transitions are fenced reflect that, so that only that can be > determined. The reasoning is that the states are there mostly for > safepoint/handshake synchronization. And they can wait for responsive > threads, but not threads that won't respond (blocked and native). Thanks for clarifying this design, and I understand it better now. I found a solid example: the transition from in_java state to vm state has no fence, so if the mutator store to Java heap is reordered after the store to the thread state, it will cause problems. I don't get why you want this filtering though. Seems like a premature > optimization to in practice not poke threads that are blocked or in > native. But those threads are handled efficiently by the handshaking > mechanism anyway, as the handshaking mechanism already filters out such > threads (but in a correct way), and when they come back they will > quickly disarm themselves and continue. So I think the solution is to > just handshake all threads and be done with it, letting the handshaking > mechanism perform the proper filtering in a context where it is valid. I > don't think it will make any perf difference. This filtering is inherited from the old prototype. Specifically "jthread->is_online_vm" from here . I found filtering by thread's state is effective to reduce both: (1) the number of epoch synchronizations that must do handshake; (2) the number of remote threads to handshake with. This was important when the requesting thread needed to block and wait for the handshake to finish, which was quite expensive. Now the handshakes are fully asynchronous, so filtering by thread state hopefully is not that important. I will try the handshaking with all threads approach, but need to resolve an issue with AsyncHandshakeClosure below. Is it really though? Threads are in VM for a short time by design (not > to impact time to safepoint), and threads in native will have the > handshake operation performed by the handshaking thread (a no-op), and > then continue. The handshakee will just see it needs to flip a bit when > coming back from native. I can't understand why it would make a > difference. Do you have numbers showing this? Similar to above, this was partly due to the experience that it was expensive to synchronously wait for handshakes to finish. I haven't collected performance numbers from the asynchronous implementation yet, and will collect it for _thread_in_vm first. For handshaking with a thread in native and the "handshaking with all threads" approach, I need to resolve this issue: The epoch synchronization uses a no-op AsyncHandshakeClosure, but Handshake::execute() doesn't execute such closure for a thread that is in native or blocked. I agree with David's comment in JDK-8238761 that the term "async handshake" is conflated, and probably need to implement something new in handshake.cpp. -Man From thomas.stuefe at gmail.com Sat Apr 17 04:11:25 2021 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 17 Apr 2021 06:11:25 +0200 Subject: Unconditional messages on large page reservation errors Message-ID: Hi all, In os::reserve_memory_special, we print unconditional warnings to stderr in case large page reservation fails. Unconditional printouts like these can interfere with parsers parsing VM output, and can accumulate and cause high memory footprint (see e.g. https://bugs.openjdk.java.net/browse/JDK-8265332 ). Large page reservations may fail at any time os::reserve_memory_special() function is called, e.g. because the large page pool is temporarily exhausted. And os::reserve_memory_special() is a general purpose function, not only used for the heap. Running out of large pages is not fatal, since the caller can just fall back to normal page allocation. Which is what we do when reserving the java heap. I think unconditional printouts should only happen in case of fatal errors, when the VM is about to die. The unconditional warning probably made sense in the context of reserving the java heap, if the user explicitly specified UseLargePages. I propose to change this to either - if large page allocation for the heap fails, trace with info level and fall back to small pages. Leave it up to the user to increase UL and monitor log output to find out about this. This is what we usually do when system APIs fail. - continue printing the message with error level, but exit the VM. If it's serious enough to unconditionally notify the user, it's serious enough to stop the VM. I prefer the former. What do you think? Thanks and best Regards, Thomas From ayang at openjdk.java.net Sun Apr 18 09:00:41 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Sun, 18 Apr 2021 09:00:41 GMT Subject: RFR: 8265136: ZGC: Expose GarbageCollectorMXBeans for both pauses and cycles In-Reply-To: References: Message-ID: On Wed, 14 Apr 2021 07:13:50 GMT, Per Liden wrote: > JDK-8240679 introduced a change where the information exposed via the GarbageCollectorMXBean went from being related to the GC pauses to instead be related to the GC cycles. This helped provide more accurate heap usage information. However, some users have noticed that that you no longer get timing information related to the GC pauses, only related to the GC cycle. > > Shenandoah has opted for having two distinct memory managers to provide timing information about both pauses and cycles. To align how concurent GCs report this information, ZGC should probably do the same. > > Testing: > * Tier1-7 with ZGC > * Manual testing with relevant jtreg tests Marked as reviewed by ayang (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3483 From yyang at openjdk.java.net Mon Apr 19 02:22:34 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Mon, 19 Apr 2021 02:22:34 GMT Subject: RFR: 8265322: C2: Simplify control inputs for BarrierSetC2::obj_allocate In-Reply-To: References: <_NicB3VpqFwTiu2-bKLNhfnQAxJmbtND66g3Md5ck5g=.f9b4edcd-fa2b-4917-9092-bccc38b202df@github.com> Message-ID: On Fri, 16 Apr 2021 17:37:05 GMT, Vladimir Kozlov wrote: > From compiler code POV the fix is reasonable and correct. Thank you @vnkozlov! Do you think it's reasonable to move PhaseMacroExpand::set_eden_pointers to BarrierSetC2? It seems that that's GC knowledge area about how to set eden_top/eden_end w or w/o turning UseTLAB. > Note, the path when `initial_slow_test != NULL` is not rare. It is frequent for arrays allocation when `length` is not constant. Yes, I missed the AllocateArrayNode when skimming the code. Thanks for pointing out this. Let's wait for other hotspot gc/compiler folks for more reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/3529 From ayang at openjdk.java.net Mon Apr 19 07:24:04 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Mon, 19 Apr 2021 07:24:04 GMT Subject: RFR: 8265414: Variable assigned but not used in G1FreeHumongousRegionClosure Message-ID: Trivial change of removing an unused field. ------------- Commit messages: - proxy-set Changes: https://git.openjdk.java.net/jdk/pull/3565/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3565&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265414 Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3565.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3565/head:pull/3565 PR: https://git.openjdk.java.net/jdk/pull/3565 From tschatzl at openjdk.java.net Mon Apr 19 09:47:36 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 19 Apr 2021 09:47:36 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates [v3] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 14:24:06 GMT, Ivan Walulya wrote: >> Hi all, >> >> Please review this change that skips the rebuild remset phase and the associated updates after rebuild, If during Remark, G1 finds that there are no remembered sets to rebuild. >> >> Test: tier1-5 > > Ivan Walulya has updated the pull request incrementally with one additional commit since the last revision: > > change comment Still good. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3434 From tschatzl at openjdk.java.net Mon Apr 19 09:47:36 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 19 Apr 2021 09:47:36 GMT Subject: RFR: 8265414: Variable assigned but not used in G1FreeHumongousRegionClosure In-Reply-To: References: Message-ID: <30ip8odZEi7FWjBrQmU2-XgEFsgVFeevFx2Ui2CDmzQ=.1c887991-8c30-48f7-a33a-f6d488944fb1@github.com> On Mon, 19 Apr 2021 07:17:26 GMT, Albert Mingkun Yang wrote: > Trivial change of removing an unused field. Lgtm, and trivial. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3565 From iwalulya at openjdk.java.net Mon Apr 19 09:53:34 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Mon, 19 Apr 2021 09:53:34 GMT Subject: RFR: 8265414: Variable assigned but not used in G1FreeHumongousRegionClosure In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 07:17:26 GMT, Albert Mingkun Yang wrote: > Trivial change of removing an unused field. Marked as reviewed by iwalulya (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3565 From ayang at openjdk.java.net Mon Apr 19 10:01:37 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Mon, 19 Apr 2021 10:01:37 GMT Subject: RFR: 8265414: Variable assigned but not used in G1FreeHumongousRegionClosure In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 07:17:26 GMT, Albert Mingkun Yang wrote: > Trivial change of removing an unused field. Thanks for the review. ------------- PR: https://git.openjdk.java.net/jdk/pull/3565 From ayang at openjdk.java.net Mon Apr 19 10:01:37 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Mon, 19 Apr 2021 10:01:37 GMT Subject: Integrated: 8265414: Variable assigned but not used in G1FreeHumongousRegionClosure In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 07:17:26 GMT, Albert Mingkun Yang wrote: > Trivial change of removing an unused field. This pull request has now been integrated. Changeset: 7d01c988 Author: Albert Mingkun Yang URL: https://git.openjdk.java.net/jdk/commit/7d01c988 Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod 8265414: Variable assigned but not used in G1FreeHumongousRegionClosure Reviewed-by: tschatzl, iwalulya ------------- PR: https://git.openjdk.java.net/jdk/pull/3565 From sjohanss at openjdk.java.net Mon Apr 19 11:33:36 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Mon, 19 Apr 2021 11:33:36 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates [v3] In-Reply-To: References: Message-ID: On Fri, 16 Apr 2021 14:24:06 GMT, Ivan Walulya wrote: >> Hi all, >> >> Please review this change that skips the rebuild remset phase and the associated updates after rebuild, If during Remark, G1 finds that there are no remembered sets to rebuild. >> >> Test: tier1-5 > > Ivan Walulya has updated the pull request incrementally with one additional commit since the last revision: > > change comment Looks good, only a minor nit regarding a comment. src/hotspot/share/gc/g1/g1ConcurrentMark.hpp line 460: > 458: // set phase at all. > 459: HeapWord* volatile* _top_at_rebuild_starts; > 460: // If true, then Remark pause selected regions for rebuilding. Suggestion: // True when Remark pause selected regions for rebuilding. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3434 From iwalulya at openjdk.java.net Mon Apr 19 11:54:17 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Mon, 19 Apr 2021 11:54:17 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates [v4] In-Reply-To: References: Message-ID: > Hi all, > > Please review this change that skips the rebuild remset phase and the associated updates after rebuild, If during Remark, G1 finds that there are no remembered sets to rebuild. > > Test: tier1-5 Ivan Walulya has updated the pull request incrementally with one additional commit since the last revision: Update src/hotspot/share/gc/g1/g1ConcurrentMark.hpp Co-authored-by: Stefan Johansson <54407259+kstefanj at users.noreply.github.com> ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3434/files - new: https://git.openjdk.java.net/jdk/pull/3434/files/41744103..cbd3c47c Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3434&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3434&range=02-03 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3434.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3434/head:pull/3434 PR: https://git.openjdk.java.net/jdk/pull/3434 From iwalulya at openjdk.java.net Mon Apr 19 11:54:20 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Mon, 19 Apr 2021 11:54:20 GMT Subject: RFR: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates [v3] In-Reply-To: References: Message-ID: <0_Ry-8b6T5aS32c36Zt4FUhNPnXhJ43mdwfbkSgq5nA=.727b5f7d-783b-4213-a565-379bec1c7edc@github.com> On Mon, 19 Apr 2021 11:30:41 GMT, Stefan Johansson wrote: >> Ivan Walulya has updated the pull request incrementally with one additional commit since the last revision: >> >> change comment > > Looks good, only a minor nit regarding a comment. Thanks @kstefanj ------------- PR: https://git.openjdk.java.net/jdk/pull/3434 From ayang at openjdk.java.net Mon Apr 19 12:25:58 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Mon, 19 Apr 2021 12:25:58 GMT Subject: RFR: 8265435: Remove dummy lists in G1CalculatePointersClosure Message-ID: Remove the dummy list and some general cleanup. ------------- Commit messages: - freelist Changes: https://git.openjdk.java.net/jdk/pull/3568/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3568&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265435 Stats: 49 lines in 2 files changed: 15 ins; 31 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3568.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3568/head:pull/3568 PR: https://git.openjdk.java.net/jdk/pull/3568 From tschatzl at openjdk.java.net Mon Apr 19 12:36:59 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 19 Apr 2021 12:36:59 GMT Subject: RFR: 8265436: G1: Improve gc+phases log output during full gc Message-ID: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> Hi all, can I have reviews for this fix to remove recently introduced log messages from `debug` to `trace` level? If you select `gc+phases=debug` logging with [JDK-8262068](https://bugs.openjdk.java.net/browse/JDK-8262068), you get: [0.695s][debug][gc,phases] GC(51) Phase 1: Weak Processing 0.309ms [0.695s][debug][gc,phases] GC(51) ClassLoaderData 0.003ms [0.695s][debug][gc,phases] GC(51) Trigger cleanups 0.002ms [0.699s][debug][gc,phases] GC(51) Phase 1: Class Unloading and Cleanup 4.056ms [0.699s][info ][gc,phases] GC(51) Phase 1: Mark live objects 7.722ms [0.699s][debug][gc,phases] GC(51) Phase 2: skip compaction region index: 0, live words: 125610 [repeated for every region skipped] the last per-region message is quite annyoing and just fills up logs as the code prints one line per affected region (which may be a lot), so I suggest to move it to `trace` level. I did suggest to add some nice phase 2 region summary in [JDK-8265437](https://bugs.openjdk.java.net/browse/JDK-8265437), but that is out of scope for this change. Testing: local testing Thanks, Thomas ------------- Commit messages: - Initial change Changes: https://git.openjdk.java.net/jdk/pull/3569/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3569&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265436 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3569.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3569/head:pull/3569 PR: https://git.openjdk.java.net/jdk/pull/3569 From ayang at openjdk.java.net Mon Apr 19 12:42:35 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Mon, 19 Apr 2021 12:42:35 GMT Subject: RFR: 8265436: G1: Improve gc+phases log output during full gc In-Reply-To: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> References: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> Message-ID: On Mon, 19 Apr 2021 12:28:58 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this fix to remove recently introduced log messages from `debug` to `trace` level? > > If you select `gc+phases=debug` logging with [JDK-8262068](https://bugs.openjdk.java.net/browse/JDK-8262068), you get: > > [0.695s][debug][gc,phases] GC(51) Phase 1: Weak Processing 0.309ms > [0.695s][debug][gc,phases] GC(51) ClassLoaderData 0.003ms > [0.695s][debug][gc,phases] GC(51) Trigger cleanups 0.002ms > [0.699s][debug][gc,phases] GC(51) Phase 1: Class Unloading and Cleanup 4.056ms > [0.699s][info ][gc,phases] GC(51) Phase 1: Mark live objects 7.722ms > [0.699s][debug][gc,phases] GC(51) Phase 2: skip compaction region index: 0, live words: 125610 > [repeated for every region skipped] > > the last per-region message is quite annyoing and just fills up logs as the code prints one line per affected region (which may be a lot), so I suggest to move it to `trace` level. I did suggest to add some nice phase 2 region summary in [JDK-8265437](https://bugs.openjdk.java.net/browse/JDK-8265437), but that is out of scope for this change. > > Testing: local testing > > Thanks, > Thomas Marked as reviewed by ayang (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3569 From thomas.schatzl at oracle.com Mon Apr 19 13:14:15 2021 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 19 Apr 2021 15:14:15 +0200 Subject: Work-in-progress: 8236485: Epoch synchronization protocol for G1 concurrent refinement In-Reply-To: References: <87blr3vc4t.fsf@oldenburg2.str.redhat.com> Message-ID: <1e8e9fbe-6897-34fd-d536-dc6b03eec765@oracle.com> Hi Man, On 31.03.21 04:43, Man Cao wrote: > Hi all, > > I finally managed to allocate more time to make progress on this, and > resolved most issues since the last discussion. > I've updated the description in > https://bugs.openjdk.java.net/browse/JDK-8236485 > , and the current > prototype is the HEAD commit at > https://github.com/caoman/jdk/tree/g1EpochSync > Notable changes include: > - The protocol uses async handshake?from JDK-8238761 > ?to resolve the > blocking issue from normal handshake. > - In order to support async refinement?due to async handshake, added > support for?_deferred global queue to G1DirtyCardQueueSet. Buffers > rarely get enqueued to _deferred at run-time. > - The async handshake only executes for a subset of threads. > > I have a couple of questions: > > 1. Code review and patch size. > Should I start a pull request for this change, so it is easier to give > feedback? I think email is easy enough. > > What is the recommended approach to deal with large changes? Currently > the patch is about 1200 lines, without changing the write barrier itself > (JDK-8226731). > Since pushing only this patch will add some overhead to refinement, but > not bring any performance improvement from removing the write barrier's > fence, do you recommend I also implement the write barrier change in the > same patch? > Keeping the epoch sync patch separate from the write barrier patch has > some benefit for testing, in case the?epoch patch introduces any bugs. > Currently the new code is mostly guarded by a flag > -XX:+G1TestEpochSyncInConcRefinement, and it will be removed after the > write barrier change. It could be used as an emergency flag to > workaround bugs, instead of backing out of the entire change. We > probably cannot have such a flag if we bundle the changes in one patch > (it's too ugly to have a flag in interpreter and compilers). Erik's suggestion is fine with me. > > 3. Native write barrier (G1BarrierSet::write_ref_field_post). > The epoch synchronization protocol does not synchronize with threads in > _thread_in_native or?_thread_in_vm state. It is much slower if we > synchronize with such threads. > Moreover, there are non-Java threads (e.g.?concurrent mark worker) that > could execute the native write barrier. > As a result, it's probably best to keep the StoreLoad fence in the > native write barrier. I agree. > The final write post-barrier for JDK-8226731 would be: > Given: > x.a = q > and > p = @x.a > > For Interpreter/C1/C2: > if (p and q in same regions or q == NULL) -> exit > if (card(p) == Dirty) -> exit > card(p) = Dirty; > enqueue(card(p)) > > For the native barrier: > if (p and q in same regions or q == NULL) -> exit > StoreLoad; > if (card(p) == Dirty) -> exit > card(p) = Dirty; > enqueue(card(p)) > > Does the above look?good? Do we need to micro-benchmark the potential > overhead added to the native barrier? Or are typical macro-benchmarks > sufficient? This is a bit worse than before (it could be optimized to filter out from-young regions using the heap region table or querying HeapRegion directly to get the same effect wrt to number of cards attempted to enqueue) for the native barrier, buut since these are "stone cold" we do not care too much. > 4. Performance benchmarking. > I did some preliminary benchmarking with DaCapo and BigRamTester, > without changing the write barrier. This is to measure the overhead > added by the epoch synchronization protocol. > Under the default JVM setting, I didn't see any performance difference. > When I tuned the JVM to refine cards aggressively, there was > still?no?difference in BigRamTester (probably because it only has 2 > threads). Some DaCapo benchmarks saw 10-15% more CPU usage due to doing > more work in the refinement threads, and 2-5% total throughput > regression for these benchmarks. > The aggressive refinement flags are "-XX:-G1UseAdaptiveConcRefinement > -XX:G1UpdateBufferSize=4 -XX:G1ConcRefinementGreenZone=0 > -XX:G1ConcRefinementYellowZone=1". > > I wonder how important we should treat the aggressive refinement case. > Regression in this case is likely unavoidable, so how much regression is > tolerable? This case is interesting to know about but not important. > Also, does?anyone know a better benchmark to test refinement with > default JVM settings? Ideally it (1) has many mutator threads; (2) > triggers concurrent refinement frequently; (3) runs with a sizable Xmx > (e.g., 2GiB or above). Let me see if I can find something. Thomas From pliden at openjdk.java.net Mon Apr 19 13:14:58 2021 From: pliden at openjdk.java.net (Per Liden) Date: Mon, 19 Apr 2021 13:14:58 GMT Subject: RFR: 8261759: ZGC: ZWorker Threads Continue Marking After System.exit() called [v3] In-Reply-To: References: Message-ID: > During JVM termination, ZCollectedHeap::stop() currently only returns once any ongoing GC cycle has completed. However, waiting for a GC cycle to complete can take awhile, so the JVM might not terminate as promptly as one might expect. > > This patch adds abortion checks to workers doing marking and relocation to allow ZWorker threads to terminate promptly. In addition, before and after concurrent phases, the ZDriver thread checks if `should_terminate()` is true and prematurely terminates the GC cycle if that's the case. > > Testing: > * Tier1-7 on Linux. > * SPECjbb2015, comparing score and marking times against baseline (5 iterations each), no difference observed. > * Ad-hoc programs calling System.exit() during marking/relocation. > * Manual Ctrl-C testing. Per Liden has updated the pull request incrementally with one additional commit since the last revision: Forward in-place if GC aborted ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3453/files - new: https://git.openjdk.java.net/jdk/pull/3453/files/73603d6d..d0e6a56a Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3453&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3453&range=01-02 Stats: 51 lines in 5 files changed: 36 ins; 9 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/3453.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3453/head:pull/3453 PR: https://git.openjdk.java.net/jdk/pull/3453 From pliden at openjdk.java.net Mon Apr 19 13:22:38 2021 From: pliden at openjdk.java.net (Per Liden) Date: Mon, 19 Apr 2021 13:22:38 GMT Subject: RFR: 8261759: ZGC: ZWorker Threads Continue Marking After System.exit() called [v3] In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 13:14:58 GMT, Per Liden wrote: >> During JVM termination, ZCollectedHeap::stop() currently only returns once any ongoing GC cycle has completed. However, waiting for a GC cycle to complete can take awhile, so the JVM might not terminate as promptly as one might expect. >> >> This patch adds abortion checks to workers doing marking and relocation to allow ZWorker threads to terminate promptly. In addition, before and after concurrent phases, the ZDriver thread checks if `should_terminate()` is true and prematurely terminates the GC cycle if that's the case. >> >> Testing: >> * Tier1-7 on Linux. >> * SPECjbb2015, comparing score and marking times against baseline (5 iterations each), no difference observed. >> * Ad-hoc programs calling System.exit() during marking/relocation. >> * Manual Ctrl-C testing. > > Per Liden has updated the pull request incrementally with one additional commit since the last revision: > > Forward in-place if GC aborted PR updated with "Forward in-place if GC aborted" commit. This resolves a possible dead-lock during JVM shutdown where the GC has been ordered to stop/abort but there are Java threads that failed to relocate an object and are now waiting for the GC to do it for them. In this cases, we signal to all such waiters that the GC har aborted and that they should just go ahead and forward the object in-place. ------------- PR: https://git.openjdk.java.net/jdk/pull/3453 From ayang at openjdk.java.net Mon Apr 19 13:40:35 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Mon, 19 Apr 2021 13:40:35 GMT Subject: RFR: 8265127: ZGC: Fix incorrect reporting of reclaimed memory In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 11:18:10 GMT, Per Liden wrote: > When Java threads relocate objects, ZGC will over-report number of reclaimed bytes and under-report amount of garbage left after GC. This happens because the logic calculating the total number of reclaimed bytes assumes all objects where relocated by GC workers. The result is that the information printed in the GC log can be skewed. > > This patch solves this by keeping track of the number of bytes relocated by Java threads, and using that number in the logic calculating the total number of reclaimed bytes. Marked as reviewed by ayang (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3462 From iwalulya at openjdk.java.net Mon Apr 19 14:41:38 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Mon, 19 Apr 2021 14:41:38 GMT Subject: Integrated: 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates In-Reply-To: References: Message-ID: <7IR1hX9TYXfR9KL_LVADcXOqJBE86mPACZuHkQ72BLA=.b4f5b303-83e5-44d3-ab6d-9055544b8c28@github.com> On Mon, 12 Apr 2021 11:14:20 GMT, Ivan Walulya wrote: > Hi all, > > Please review this change that skips the rebuild remset phase and the associated updates after rebuild, If during Remark, G1 finds that there are no remembered sets to rebuild. > > Test: tier1-5 This pull request has now been integrated. Changeset: d9e40dd5 Author: Ivan Walulya URL: https://git.openjdk.java.net/jdk/commit/d9e40dd5 Stats: 125 lines in 5 files changed: 115 ins; 2 del; 8 mod 8199407: Skip Rebuild Remset Phase if there are no rebuild candidates Reviewed-by: tschatzl, sjohanss ------------- PR: https://git.openjdk.java.net/jdk/pull/3434 From kvn at openjdk.java.net Mon Apr 19 17:29:37 2021 From: kvn at openjdk.java.net (Vladimir Kozlov) Date: Mon, 19 Apr 2021 17:29:37 GMT Subject: RFR: 8265322: C2: Simplify control inputs for BarrierSetC2::obj_allocate In-Reply-To: <_NicB3VpqFwTiu2-bKLNhfnQAxJmbtND66g3Md5ck5g=.f9b4edcd-fa2b-4917-9092-bccc38b202df@github.com> References: <_NicB3VpqFwTiu2-bKLNhfnQAxJmbtND66g3Md5ck5g=.f9b4edcd-fa2b-4917-9092-bccc38b202df@github.com> Message-ID: On Fri, 16 Apr 2021 03:06:19 GMT, Yi Yang wrote: > While just using BarrierSetC2::obj_allocate in https://github.com/openjdk/valhalla/pull/385, I noticed the control input `ctrl` for obj_allocate is not so much necessary. This PR simplifies control inputs for BarrierSetC2::obj_allocate. In most cases, it doesn't change anything since `toobig_false` is equivalent to `ctrl`. In other cases, `toobig_false` is created for Unsafe.allocateInstance while instance size is not statically known, `ctrl` would become control input of IfNode whose projects are `toobig_false` and `toobig_true`, old eden_end and old_eden_top can simply accept `toobig_false` as their control input rather than `ctrl`. > > From compiler code POV the fix is reasonable and correct. > Do you think it's reasonable to move PhaseMacroExpand::set_eden_pointers to BarrierSetC2? It seems that that's GC knowledge area about how to set eden_top/eden_end w or w/o turning UseTLAB. It is up to GC group but it is common code for all GCs. As I understand BarrierSetC2 is used for GC variations code. ------------- PR: https://git.openjdk.java.net/jdk/pull/3529 From thomas.schatzl at oracle.com Mon Apr 19 20:30:22 2021 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 19 Apr 2021 22:30:22 +0200 Subject: FYI: G1 pauses may be extremely long with upcoming EA build JDK-17+18 Message-ID: <156793ce-b4ef-cf28-17e6-05047df7b9ec@oracle.com> Hi all, during perf testing we noticed that due to some recent change (JDK-8262068) GC pauses after a G1 full GC may be extremely slow. The problem has been fixed with JDK-8264987 that has already been integrated. This change will be available with the following EA build JDK-17+19. More technical information: After a full gc, young generation regions that are mostly (>95%) full at the time of GC do not have a data structure, the so-called Block Offset Table (BOT), fully set up. This data structure is used to efficiently find the start of the next Java object given an arbitrary location in the heap. This is an operation in some cases very frequently executed during GC. When G1 needs to perform that lookup within these regions, typically when trying to find cross-generational references during garbage collection, that BOT can not be used and G1 needs to use a very slow fallback. This situation will not self-heal until the affected regions are evacuated or compacted again. I would expect an application that will suffer from this problem would need to do the following: a) loads or generates almost exclusively long-lived data (e.g. "database") with the intent to put it into the old generation b) does a full gc (e.g. via system.gc()) at the end of this operation to compact that data in the old generation before doing actual work. c) this "database structure" references the young generation during that time a lot because it mutates a lot d) performs young collection(s) Most applications and benchmarks will not be affected because all the conditions need to be true. In some stress test with 20gb heap we are using young collection pause times went up from 200ms to 30s. ;) So if you notice a regression with that build, please update your sources and recompile or wait for JDK-17+19. Thanks, Thomas From mli at openjdk.java.net Tue Apr 20 01:59:11 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Tue, 20 Apr 2021 01:59:11 GMT Subject: RFR: 8265436: G1: Improve gc+phases log output during full gc In-Reply-To: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> References: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> Message-ID: On Mon, 19 Apr 2021 12:28:58 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this fix to remove recently introduced log messages from `debug` to `trace` level? > > If you select `gc+phases=debug` logging with [JDK-8262068](https://bugs.openjdk.java.net/browse/JDK-8262068), you get: > > [0.695s][debug][gc,phases] GC(51) Phase 1: Weak Processing 0.309ms > [0.695s][debug][gc,phases] GC(51) ClassLoaderData 0.003ms > [0.695s][debug][gc,phases] GC(51) Trigger cleanups 0.002ms > [0.699s][debug][gc,phases] GC(51) Phase 1: Class Unloading and Cleanup 4.056ms > [0.699s][info ][gc,phases] GC(51) Phase 1: Mark live objects 7.722ms > [0.699s][debug][gc,phases] GC(51) Phase 2: skip compaction region index: 0, live words: 125610 > [repeated for every region skipped] > > the last per-region message is quite annyoing and just fills up logs as the code prints one line per affected region (which may be a lot), so I suggest to move it to `trace` level. I did suggest to add some nice phase 2 region summary in [JDK-8265437](https://bugs.openjdk.java.net/browse/JDK-8265437), but that is out of scope for this change. > > Testing: local testing > > Thanks, > Thomas looks good. ------------- Marked as reviewed by mli (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3569 From mli at openjdk.java.net Tue Apr 20 02:17:02 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Tue, 20 Apr 2021 02:17:02 GMT Subject: RFR: 8265436: G1: Improve gc+phases log output during full gc In-Reply-To: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> References: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> Message-ID: On Mon, 19 Apr 2021 12:28:58 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this fix to remove recently introduced log messages from `debug` to `trace` level? > > If you select `gc+phases=debug` logging with [JDK-8262068](https://bugs.openjdk.java.net/browse/JDK-8262068), you get: > > [0.695s][debug][gc,phases] GC(51) Phase 1: Weak Processing 0.309ms > [0.695s][debug][gc,phases] GC(51) ClassLoaderData 0.003ms > [0.695s][debug][gc,phases] GC(51) Trigger cleanups 0.002ms > [0.699s][debug][gc,phases] GC(51) Phase 1: Class Unloading and Cleanup 4.056ms > [0.699s][info ][gc,phases] GC(51) Phase 1: Mark live objects 7.722ms > [0.699s][debug][gc,phases] GC(51) Phase 2: skip compaction region index: 0, live words: 125610 > [repeated for every region skipped] > > the last per-region message is quite annyoing and just fills up logs as the code prints one line per affected region (which may be a lot), so I suggest to move it to `trace` level. I did suggest to add some nice phase 2 region summary in [JDK-8265437](https://bugs.openjdk.java.net/browse/JDK-8265437), but that is out of scope for this change. > > Testing: local testing > > Thanks, > Thomas just realized, the test code needs to be adjusted accordingly. ''' public class TestG1SkipCompaction { public static void runTest() throws Exception { final String[] arguments = { "-XX:+UseG1GC", "-XX:MarkSweepDeadRatio=3", "-Xmx8m", "-Xms8M", "-Xlog:gc+phases=debug", // here, from debug to trace "-XX:G1HeapRegionSize=1m", GCTest.class.getName() }; ''' ------------- PR: https://git.openjdk.java.net/jdk/pull/3569 From pliden at openjdk.java.net Tue Apr 20 07:57:07 2021 From: pliden at openjdk.java.net (Per Liden) Date: Tue, 20 Apr 2021 07:57:07 GMT Subject: Integrated: 8265136: ZGC: Expose GarbageCollectorMXBeans for both pauses and cycles In-Reply-To: References: Message-ID: <_xjkRtU8YzJs9Medo7heL5IBS4816UdsaKXdfbuqPn4=.c180d053-ef84-454c-99a0-641067f491db@github.com> On Wed, 14 Apr 2021 07:13:50 GMT, Per Liden wrote: > JDK-8240679 introduced a change where the information exposed via the GarbageCollectorMXBean went from being related to the GC pauses to instead be related to the GC cycles. This helped provide more accurate heap usage information. However, some users have noticed that that you no longer get timing information related to the GC pauses, only related to the GC cycle. > > Shenandoah has opted for having two distinct memory managers to provide timing information about both pauses and cycles. To align how concurent GCs report this information, ZGC should probably do the same. > > Testing: > * Tier1-7 with ZGC > * Manual testing with relevant jtreg tests This pull request has now been integrated. Changeset: 79798c65 Author: Per Liden URL: https://git.openjdk.java.net/jdk/commit/79798c65 Stats: 179 lines in 9 files changed: 107 ins; 25 del; 47 mod 8265136: ZGC: Expose GarbageCollectorMXBeans for both pauses and cycles Reviewed-by: sspitsyn, ayang ------------- PR: https://git.openjdk.java.net/jdk/pull/3483 From pliden at openjdk.java.net Tue Apr 20 07:57:06 2021 From: pliden at openjdk.java.net (Per Liden) Date: Tue, 20 Apr 2021 07:57:06 GMT Subject: RFR: 8265136: ZGC: Expose GarbageCollectorMXBeans for both pauses and cycles In-Reply-To: References: Message-ID: <4DvtADYdaqAdqNMUObnhpI5j7O_7EfVoerLh7Aih_s0=.10899634-2901-4334-94e4-07c56d627383@github.com> On Thu, 15 Apr 2021 04:20:28 GMT, Serguei Spitsyn wrote: >> JDK-8240679 introduced a change where the information exposed via the GarbageCollectorMXBean went from being related to the GC pauses to instead be related to the GC cycles. This helped provide more accurate heap usage information. However, some users have noticed that that you no longer get timing information related to the GC pauses, only related to the GC cycle. >> >> Shenandoah has opted for having two distinct memory managers to provide timing information about both pauses and cycles. To align how concurent GCs report this information, ZGC should probably do the same. >> >> Testing: >> * Tier1-7 with ZGC >> * Manual testing with relevant jtreg tests > > Hi Per, > It looks good to me. > Thanks, > Serguei Thanks for reviewing @sspitsyn and @albertnetymk! ------------- PR: https://git.openjdk.java.net/jdk/pull/3483 From tschatzl at openjdk.java.net Tue Apr 20 08:27:37 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 20 Apr 2021 08:27:37 GMT Subject: RFR: 8265436: G1: Improve gc+phases log output during full gc [v2] In-Reply-To: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> References: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> Message-ID: <-qc0cQ6M0lEAycQ8_fsVopJe85Aoym0otMaIW84aHug=.83e86c4e-506d-481a-94d0-9860679336b7@github.com> > Hi all, > > can I have reviews for this fix to remove recently introduced log messages from `debug` to `trace` level? > > If you select `gc+phases=debug` logging with [JDK-8262068](https://bugs.openjdk.java.net/browse/JDK-8262068), you get: > > [0.695s][debug][gc,phases] GC(51) Phase 1: Weak Processing 0.309ms > [0.695s][debug][gc,phases] GC(51) ClassLoaderData 0.003ms > [0.695s][debug][gc,phases] GC(51) Trigger cleanups 0.002ms > [0.699s][debug][gc,phases] GC(51) Phase 1: Class Unloading and Cleanup 4.056ms > [0.699s][info ][gc,phases] GC(51) Phase 1: Mark live objects 7.722ms > [0.699s][debug][gc,phases] GC(51) Phase 2: skip compaction region index: 0, live words: 125610 > [repeated for every region skipped] > > the last per-region message is quite annyoing and just fills up logs as the code prints one line per affected region (which may be a lot), so I suggest to move it to `trace` level. I did suggest to add some nice phase 2 region summary in [JDK-8265437](https://bugs.openjdk.java.net/browse/JDK-8265437), but that is out of scope for this change. > > Testing: local testing > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: mli comment ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3569/files - new: https://git.openjdk.java.net/jdk/pull/3569/files/340167e3..19f5650d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3569&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3569&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3569.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3569/head:pull/3569 PR: https://git.openjdk.java.net/jdk/pull/3569 From sjohanss at openjdk.java.net Tue Apr 20 08:37:05 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Tue, 20 Apr 2021 08:37:05 GMT Subject: RFR: 8265435: Remove dummy lists in G1CalculatePointersClosure In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 12:20:21 GMT, Albert Mingkun Yang wrote: > Remove the dummy list and some general cleanup. Nice cleanup. Just a small naming suggestion :) src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp line 43: > 41: > 42: template > 43: void G1FullGCPrepareTask::G1CalculatePointersClosure::free_region(HeapRegion* hr) { Since this is only for "pinned regions", maybe call this `free_pinned_region()`. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3568 From tschatzl at openjdk.java.net Tue Apr 20 08:46:23 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 20 Apr 2021 08:46:23 GMT Subject: RFR: 8265450: Merge PreservedMarksSet::restore code paths Message-ID: Hi all, can I have reviews for this refactoring that merges parallel and serial preserved marks restoration during evacuation failure into a single code path, simply by having the serial path call the parallel path in the VM thread. This just reduces code and avoids the potential bugs when updating one path but not the other (which already happened). The execution difference should not really matter for Serial GC where the serial path is executed: - we are talking about the evacuation failure path - there are likely a lot of preserved marks (random writes to the heap) that offset these two additional calls and some tiny superfluous member initialization. Testing: tier1 Thanks, Thomas ------------- Commit messages: - Changes - Fix compilation - Initial change - Initial version Changes: https://git.openjdk.java.net/jdk/pull/3584/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3584&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265450 Stats: 49 lines in 1 file changed: 11 ins; 14 del; 24 mod Patch: https://git.openjdk.java.net/jdk/pull/3584.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3584/head:pull/3584 PR: https://git.openjdk.java.net/jdk/pull/3584 From ayang at openjdk.java.net Tue Apr 20 08:59:37 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Tue, 20 Apr 2021 08:59:37 GMT Subject: RFR: 8265435: Remove dummy lists in G1CalculatePointersClosure [v2] In-Reply-To: References: Message-ID: > Remove the dummy list and some general cleanup. Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: review ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3568/files - new: https://git.openjdk.java.net/jdk/pull/3568/files/6f3b9f83..a3fda8b6 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3568&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3568&range=00-01 Stats: 4 lines in 2 files changed: 0 ins; 0 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/3568.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3568/head:pull/3568 PR: https://git.openjdk.java.net/jdk/pull/3568 From tschatzl at openjdk.java.net Tue Apr 20 08:59:38 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 20 Apr 2021 08:59:38 GMT Subject: RFR: 8265435: Remove dummy lists in G1CalculatePointersClosure [v2] In-Reply-To: References: Message-ID: <9nKXo0bKDk9oEAaU6xrcrvSsIbLdOEog_Jfmup5QAg8=.e6a53c3e-b4c5-4155-b2c7-624fb034b74e@github.com> On Tue, 20 Apr 2021 08:56:31 GMT, Albert Mingkun Yang wrote: >> Remove the dummy list and some general cleanup. > > Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: > > review Lgtm. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3568 From ayang at openjdk.java.net Tue Apr 20 08:59:39 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Tue, 20 Apr 2021 08:59:39 GMT Subject: RFR: 8265435: Remove dummy lists in G1CalculatePointersClosure [v2] In-Reply-To: References: Message-ID: <0r95hFQes2frli9R4u1eCpc9fpB0uuPHqbom3S2sijA=.69af89f0-18d6-4445-a5fd-6d178d7b4e0d@github.com> On Tue, 20 Apr 2021 08:33:22 GMT, Stefan Johansson wrote: >> Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: >> >> review > > src/hotspot/share/gc/g1/g1FullGCPrepareTask.cpp line 43: > >> 41: >> 42: template >> 43: void G1FullGCPrepareTask::G1CalculatePointersClosure::free_region(HeapRegion* hr) { > > Since this is only for "pinned regions", maybe call this `free_pinned_region()`. Makes sense; thanks for the suggestion. Fixed. ------------- PR: https://git.openjdk.java.net/jdk/pull/3568 From tschatzl at openjdk.java.net Tue Apr 20 09:39:07 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 20 Apr 2021 09:39:07 GMT Subject: RFR: 8265436: G1: Improve gc+phases log output during full gc In-Reply-To: References: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> Message-ID: On Tue, 20 Apr 2021 02:14:39 GMT, Hamlin Li wrote: >> Hi all, >> >> can I have reviews for this fix to remove recently introduced log messages from `debug` to `trace` level? >> >> If you select `gc+phases=debug` logging with [JDK-8262068](https://bugs.openjdk.java.net/browse/JDK-8262068), you get: >> >> [0.695s][debug][gc,phases] GC(51) Phase 1: Weak Processing 0.309ms >> [0.695s][debug][gc,phases] GC(51) ClassLoaderData 0.003ms >> [0.695s][debug][gc,phases] GC(51) Trigger cleanups 0.002ms >> [0.699s][debug][gc,phases] GC(51) Phase 1: Class Unloading and Cleanup 4.056ms >> [0.699s][info ][gc,phases] GC(51) Phase 1: Mark live objects 7.722ms >> [0.699s][debug][gc,phases] GC(51) Phase 2: skip compaction region index: 0, live words: 125610 >> [repeated for every region skipped] >> >> the last per-region message is quite annyoing and just fills up logs as the code prints one line per affected region (which may be a lot), so I suggest to move it to `trace` level. I did suggest to add some nice phase 2 region summary in [JDK-8265437](https://bugs.openjdk.java.net/browse/JDK-8265437), but that is out of scope for this change. >> >> Testing: local testing >> >> Thanks, >> Thomas > > just realized, the test code needs to be adjusted accordingly. > ''' > public class TestG1SkipCompaction { > public static void runTest() throws Exception { > final String[] arguments = { > "-XX:+UseG1GC", > "-XX:MarkSweepDeadRatio=3", > "-Xmx8m", > "-Xms8M", > "-Xlog:gc+phases=debug", // here, from debug to trace > "-XX:G1HeapRegionSize=1m", > GCTest.class.getName() > }; > ''' I will integrate after tier1-2 testing that runs that test passed. Thanks @Hamlin-Li @albertnetymk for your reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/3569 From tschatzl at openjdk.java.net Tue Apr 20 09:55:38 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 20 Apr 2021 09:55:38 GMT Subject: RFR: 8265436: G1: Improve gc+phases log output during full gc [v3] In-Reply-To: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> References: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> Message-ID: > Hi all, > > can I have reviews for this fix to remove recently introduced log messages from `debug` to `trace` level? > > If you select `gc+phases=debug` logging with [JDK-8262068](https://bugs.openjdk.java.net/browse/JDK-8262068), you get: > > [0.695s][debug][gc,phases] GC(51) Phase 1: Weak Processing 0.309ms > [0.695s][debug][gc,phases] GC(51) ClassLoaderData 0.003ms > [0.695s][debug][gc,phases] GC(51) Trigger cleanups 0.002ms > [0.699s][debug][gc,phases] GC(51) Phase 1: Class Unloading and Cleanup 4.056ms > [0.699s][info ][gc,phases] GC(51) Phase 1: Mark live objects 7.722ms > [0.699s][debug][gc,phases] GC(51) Phase 2: skip compaction region index: 0, live words: 125610 > [repeated for every region skipped] > > the last per-region message is quite annyoing and just fills up logs as the code prints one line per affected region (which may be a lot), so I suggest to move it to `trace` level. I did suggest to add some nice phase 2 region summary in [JDK-8265437](https://bugs.openjdk.java.net/browse/JDK-8265437), but that is out of scope for this change. > > Testing: local testing > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: Fix error message in test ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3569/files - new: https://git.openjdk.java.net/jdk/pull/3569/files/19f5650d..f4137317 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3569&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3569&range=01-02 Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3569.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3569/head:pull/3569 PR: https://git.openjdk.java.net/jdk/pull/3569 From ayang at openjdk.java.net Tue Apr 20 10:36:05 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Tue, 20 Apr 2021 10:36:05 GMT Subject: RFR: 8265116: ZGC: Steal local stacks instead of flushing them In-Reply-To: References: Message-ID: <7TvL-lvD4DQSiRzGWIGifJhXeEcuaHWDmbW3XvDFNw0=.605b493c-8f29-4af5-99bc-4f8f5a825852@github.com> On Tue, 13 Apr 2021 08:18:29 GMT, Per Liden wrote: > As part of addressing the issue of excessive mark stack usage, it was observed that sometimes mark stack utilization can be very low. For example, casparcwang at tencent.com reported that when running Apache Zookeeper, mark stack utilization was ~10%. > > The problem is that GC workers are a bit too eager to flush mark stacks. Currently, whenever a GC worker has drained its home stripe it will always flush all remaining mark stacks. This causes non-full mark stacks to be flushed. This in turn means that the GC worker who receives that non-full stack (small amount of work) tends to also produce non-full stacks (small amount of work) for other GC workers. This can lead to a vicious cycle, which results in excessive mark stack memory usage. > > The solution proposed here is to not let GC workers flush non-full stacks. After draining the home stripe, a GC worker will instead try to steal any non-full local stack belonging to a different stripe, and if no such stacks exist it will (as it does today) try to steal a global stack belonging to a different stripe. > > (Note: This PR only addresses one part of the mark stack usage problem. Other improvements, such as the discussed "mark before push"-patch will be a separate PR). > > Testing: > * Tier1-7. > * SPECjbb2015, score and mark times unaffected. > * casparcwang at tencent.com reports that this patch (in combination with the "mark before push"-patch) solves the problem for Apache Zookeeper. Marked as reviewed by ayang (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3455 From tschatzl at openjdk.java.net Tue Apr 20 10:39:05 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 20 Apr 2021 10:39:05 GMT Subject: RFR: 8265436: G1: Improve gc+phases log output during full gc [v3] In-Reply-To: References: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> Message-ID: On Tue, 20 Apr 2021 09:55:38 GMT, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this fix to remove recently introduced log messages from `debug` to `trace` level? >> >> If you select `gc+phases=debug` logging with [JDK-8262068](https://bugs.openjdk.java.net/browse/JDK-8262068), you get: >> >> [0.695s][debug][gc,phases] GC(51) Phase 1: Weak Processing 0.309ms >> [0.695s][debug][gc,phases] GC(51) ClassLoaderData 0.003ms >> [0.695s][debug][gc,phases] GC(51) Trigger cleanups 0.002ms >> [0.699s][debug][gc,phases] GC(51) Phase 1: Class Unloading and Cleanup 4.056ms >> [0.699s][info ][gc,phases] GC(51) Phase 1: Mark live objects 7.722ms >> [0.699s][debug][gc,phases] GC(51) Phase 2: skip compaction region index: 0, live words: 125610 >> [repeated for every region skipped] >> >> the last per-region message is quite annyoing and just fills up logs as the code prints one line per affected region (which may be a lot), so I suggest to move it to `trace` level. I did suggest to add some nice phase 2 region summary in [JDK-8265437](https://bugs.openjdk.java.net/browse/JDK-8265437), but that is out of scope for this change. >> >> Testing: local testing >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > Fix error message in test Manual tier1,2 testing has been successful. ------------- PR: https://git.openjdk.java.net/jdk/pull/3569 From tschatzl at openjdk.java.net Tue Apr 20 10:39:06 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 20 Apr 2021 10:39:06 GMT Subject: Integrated: 8265436: G1: Improve gc+phases log output during full gc In-Reply-To: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> References: <7r_5txx5zU9tvkQhR7_7k9IPDsGn15OYle0a4xlo2gs=.2b305670-b546-43f1-8e3b-1f94ac603d12@github.com> Message-ID: On Mon, 19 Apr 2021 12:28:58 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this fix to remove recently introduced log messages from `debug` to `trace` level? > > If you select `gc+phases=debug` logging with [JDK-8262068](https://bugs.openjdk.java.net/browse/JDK-8262068), you get: > > [0.695s][debug][gc,phases] GC(51) Phase 1: Weak Processing 0.309ms > [0.695s][debug][gc,phases] GC(51) ClassLoaderData 0.003ms > [0.695s][debug][gc,phases] GC(51) Trigger cleanups 0.002ms > [0.699s][debug][gc,phases] GC(51) Phase 1: Class Unloading and Cleanup 4.056ms > [0.699s][info ][gc,phases] GC(51) Phase 1: Mark live objects 7.722ms > [0.699s][debug][gc,phases] GC(51) Phase 2: skip compaction region index: 0, live words: 125610 > [repeated for every region skipped] > > the last per-region message is quite annyoing and just fills up logs as the code prints one line per affected region (which may be a lot), so I suggest to move it to `trace` level. I did suggest to add some nice phase 2 region summary in [JDK-8265437](https://bugs.openjdk.java.net/browse/JDK-8265437), but that is out of scope for this change. > > Testing: local testing > > Thanks, > Thomas This pull request has now been integrated. Changeset: 072ae7ec Author: Thomas Schatzl URL: https://git.openjdk.java.net/jdk/commit/072ae7ec Stats: 4 lines in 2 files changed: 0 ins; 1 del; 3 mod 8265436: G1: Improve gc+phases log output during full gc Reviewed-by: ayang, mli ------------- PR: https://git.openjdk.java.net/jdk/pull/3569 From sjohanss at openjdk.java.net Tue Apr 20 10:39:11 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Tue, 20 Apr 2021 10:39:11 GMT Subject: RFR: 8265450: Merge PreservedMarksSet::restore code paths In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 08:39:56 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this refactoring that merges parallel and serial preserved marks restoration during evacuation failure into a single code path, simply by having the serial path call the parallel path in the VM thread. > > This just reduces code and avoids the potential bugs when updating one path but not the other (which already happened). > > The execution difference should not really matter for Serial GC where the serial path is executed: > - we are talking about the evacuation failure path > - there are likely a lot of preserved marks (random writes to the heap) that offset these two additional calls and some tiny superfluous member initialization. > > Testing: tier1 > > Thanks, > Thomas Looks good. Just a small comment. src/hotspot/share/gc/shared/preservedMarks.cpp line 100: > 98: volatile size_t _total_size; > 99: #ifdef ASSERT > 100: size_t _total_size_before; I know `_total_size` is what was used before, but what do you think about changing it to `_total_count` now when we are touching it. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3584 From tschatzl at openjdk.java.net Tue Apr 20 10:52:27 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 20 Apr 2021 10:52:27 GMT Subject: RFR: 8265461: G1: Forwarding pointer removal thread sizing Message-ID: Hi all, can I have reviews for this change that improves forwarding pointer removal handling by adding thread sizing to the work gang? Since work is distributed per region at the moment, it does not make sense to start more threads than regions. This can particularly help performance with object/region pinning. Testing: tier1-3, using self-verification code Thanks, Thomas ------------- Commit messages: - Add verification - Initial commit Changes: https://git.openjdk.java.net/jdk/pull/3585/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3585&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265461 Stats: 79 lines in 9 files changed: 50 ins; 7 del; 22 mod Patch: https://git.openjdk.java.net/jdk/pull/3585.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3585/head:pull/3585 PR: https://git.openjdk.java.net/jdk/pull/3585 From tschatzl at openjdk.java.net Tue Apr 20 11:30:09 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 20 Apr 2021 11:30:09 GMT Subject: RFR: 8265450: Merge PreservedMarksSet::restore code paths In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 10:33:51 GMT, Stefan Johansson wrote: >> Hi all, >> >> can I have reviews for this refactoring that merges parallel and serial preserved marks restoration during evacuation failure into a single code path, simply by having the serial path call the parallel path in the VM thread. >> >> This just reduces code and avoids the potential bugs when updating one path but not the other (which already happened). >> >> The execution difference should not really matter for Serial GC where the serial path is executed: >> - we are talking about the evacuation failure path >> - there are likely a lot of preserved marks (random writes to the heap) that offset these two additional calls and some tiny superfluous member initialization. >> >> Testing: tier1 >> >> Thanks, >> Thomas > > src/hotspot/share/gc/shared/preservedMarks.cpp line 100: > >> 98: volatile size_t _total_size; >> 99: #ifdef ASSERT >> 100: size_t _total_size_before; > > I know `_total_size` is what was used before, but what do you think about changing it to `_total_count` now when we are touching it. Should we rename `size` to `count` all the way through? Also `PreservedMarks` uses `size`, and that is where this comes from. There are likely other places where we use `size` instead of `count` (or `num`, depending on when the code has been written the first time). Not sure this is the appropriate CR to do that, but not against it either. ------------- PR: https://git.openjdk.java.net/jdk/pull/3584 From fmatte at openjdk.java.net Tue Apr 20 12:09:24 2021 From: fmatte at openjdk.java.net (Fairoz Matte) Date: Tue, 20 Apr 2021 12:09:24 GMT Subject: RFR: 8265326: Strange Characters in G1GC GC Log Message-ID: There was an character formating issue on mac-os due to usage of " `FormatBuffer<128>`" Used explicit variable to hold the FormatBuffer ------------- Commit messages: - 8265326: Strange Characters in G1GC GC Log Changes: https://git.openjdk.java.net/jdk/pull/3589/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3589&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265326 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3589.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3589/head:pull/3589 PR: https://git.openjdk.java.net/jdk/pull/3589 From tschatzl at openjdk.java.net Tue Apr 20 12:24:04 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 20 Apr 2021 12:24:04 GMT Subject: RFR: 8265326: Strange Characters in G1GC GC Log In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 12:02:58 GMT, Fairoz Matte wrote: > There was an character formating issue on mac-os due to usage of " > `FormatBuffer<128>`" > Used explicit variable to hold the FormatBuffer Marked as reviewed by tschatzl (Reviewer). src/hotspot/share/gc/g1/g1ConcurrentMarkThread.cpp line 143: > 141: GCIdMark gc_id_mark; > 142: FormatBuffer<128> _format("Concurrent %s Cycle", _state == FullMark ? "Mark" : "Undo"); > 143: GCTraceConcTime(Info, gc) tt(_format); Suggestion: FormatBuffer<128> title("Concurrent %s Cycle", _state == FullMark ? "Mark" : "Undo"); GCTraceConcTime(Info, gc) tt(title); Leading underscores are reserved for members of structs and classes in Hotspot. I suggested `title` here, but `format` would be as good. Apart from that lgtm. Thanks for investigating this. ------------- PR: https://git.openjdk.java.net/jdk/pull/3589 From sjohanss at openjdk.java.net Tue Apr 20 12:24:05 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Tue, 20 Apr 2021 12:24:05 GMT Subject: RFR: 8265326: Strange Characters in G1GC GC Log In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 12:19:08 GMT, Thomas Schatzl wrote: >> There was an character formating issue on mac-os due to usage of " >> `FormatBuffer<128>`" >> Used explicit variable to hold the FormatBuffer > > src/hotspot/share/gc/g1/g1ConcurrentMarkThread.cpp line 143: > >> 141: GCIdMark gc_id_mark; >> 142: FormatBuffer<128> _format("Concurrent %s Cycle", _state == FullMark ? "Mark" : "Undo"); >> 143: GCTraceConcTime(Info, gc) tt(_format); > > Suggestion: > > FormatBuffer<128> title("Concurrent %s Cycle", _state == FullMark ? "Mark" : "Undo"); > GCTraceConcTime(Info, gc) tt(title); > > Leading underscores are reserved for members of structs and classes in Hotspot. I suggested `title` here, but `format` would be as good. > > Apart from that lgtm. Thanks for investigating this. Was just about to make the same comment :) ------------- PR: https://git.openjdk.java.net/jdk/pull/3589 From fmatte at openjdk.java.net Tue Apr 20 12:28:09 2021 From: fmatte at openjdk.java.net (Fairoz Matte) Date: Tue, 20 Apr 2021 12:28:09 GMT Subject: RFR: 8265326: Strange Characters in G1GC GC Log In-Reply-To: References: Message-ID: <0lx70EEoMFLsM-ocCXwjvtc3FXvZmAzvT7SM8D-0Xis=.102c776e-903a-4839-bb8f-ee51acf39ffc@github.com> On Tue, 20 Apr 2021 12:21:40 GMT, Stefan Johansson wrote: >> src/hotspot/share/gc/g1/g1ConcurrentMarkThread.cpp line 143: >> >>> 141: GCIdMark gc_id_mark; >>> 142: FormatBuffer<128> _format("Concurrent %s Cycle", _state == FullMark ? "Mark" : "Undo"); >>> 143: GCTraceConcTime(Info, gc) tt(_format); >> >> Suggestion: >> >> FormatBuffer<128> title("Concurrent %s Cycle", _state == FullMark ? "Mark" : "Undo"); >> GCTraceConcTime(Info, gc) tt(title); >> >> Leading underscores are reserved for members of structs and classes in Hotspot. I suggested `title` here, but `format` would be as good. >> >> Apart from that lgtm. Thanks for investigating this. > > Was just about to make the same comment :) Thanks Tomas for the review, will update with title ------------- PR: https://git.openjdk.java.net/jdk/pull/3589 From fmatte at openjdk.java.net Tue Apr 20 12:38:44 2021 From: fmatte at openjdk.java.net (Fairoz Matte) Date: Tue, 20 Apr 2021 12:38:44 GMT Subject: RFR: 8265326: Strange Characters in G1GC GC Log [v2] In-Reply-To: References: Message-ID: > There was an character formating issue on mac-os due to usage of " > `FormatBuffer<128>`" > Used explicit variable to hold the FormatBuffer Fairoz Matte has updated the pull request incrementally with one additional commit since the last revision: Update g1ConcurrentMarkThread.cpp ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3589/files - new: https://git.openjdk.java.net/jdk/pull/3589/files/e82c9087..630a944b Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3589&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3589&range=00-01 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3589.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3589/head:pull/3589 PR: https://git.openjdk.java.net/jdk/pull/3589 From fmatte at openjdk.java.net Tue Apr 20 12:38:47 2021 From: fmatte at openjdk.java.net (Fairoz Matte) Date: Tue, 20 Apr 2021 12:38:47 GMT Subject: RFR: 8265326: Strange Characters in G1GC GC Log [v2] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 12:35:02 GMT, Fairoz Matte wrote: >> There was an character formating issue on mac-os due to usage of " >> `FormatBuffer<128>`" >> Used explicit variable to hold the FormatBuffer > > Fairoz Matte has updated the pull request incrementally with one additional commit since the last revision: > > Update g1ConcurrentMarkThread.cpp Updated, as per the Thomas suggestions ------------- PR: https://git.openjdk.java.net/jdk/pull/3589 From fmatte at openjdk.java.net Tue Apr 20 12:38:48 2021 From: fmatte at openjdk.java.net (Fairoz Matte) Date: Tue, 20 Apr 2021 12:38:48 GMT Subject: RFR: 8265326: Strange Characters in G1GC GC Log [v2] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 12:20:52 GMT, Thomas Schatzl wrote: >> Fairoz Matte has updated the pull request incrementally with one additional commit since the last revision: >> >> Update g1ConcurrentMarkThread.cpp > > Marked as reviewed by tschatzl (Reviewer). @tschatzl Please sponsor this change (I am not a committer), Thanks, ------------- PR: https://git.openjdk.java.net/jdk/pull/3589 From ayang at openjdk.java.net Tue Apr 20 12:52:08 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Tue, 20 Apr 2021 12:52:08 GMT Subject: RFR: 8265435: Remove dummy lists in G1CalculatePointersClosure [v2] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 08:59:37 GMT, Albert Mingkun Yang wrote: >> Remove the dummy list and some general cleanup. > > Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: > > review Thanks for the review. ------------- PR: https://git.openjdk.java.net/jdk/pull/3568 From ayang at openjdk.java.net Tue Apr 20 12:52:09 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Tue, 20 Apr 2021 12:52:09 GMT Subject: Integrated: 8265435: Remove dummy lists in G1CalculatePointersClosure In-Reply-To: References: Message-ID: On Mon, 19 Apr 2021 12:20:21 GMT, Albert Mingkun Yang wrote: > Remove the dummy list and some general cleanup. This pull request has now been integrated. Changeset: c52e5f2b Author: Albert Mingkun Yang URL: https://git.openjdk.java.net/jdk/commit/c52e5f2b Stats: 49 lines in 2 files changed: 15 ins; 31 del; 3 mod 8265435: Remove dummy lists in G1CalculatePointersClosure Reviewed-by: sjohanss, tschatzl ------------- PR: https://git.openjdk.java.net/jdk/pull/3568 From tschatzl at openjdk.java.net Tue Apr 20 13:11:06 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 20 Apr 2021 13:11:06 GMT Subject: RFR: 8265326: Strange Characters in G1GC GC Log [v2] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 12:38:44 GMT, Fairoz Matte wrote: >> There was an character formating issue on mac-os due to usage of " >> `FormatBuffer<128>`" >> Used explicit variable to hold the FormatBuffer > > Fairoz Matte has updated the pull request incrementally with one additional commit since the last revision: > > Update g1ConcurrentMarkThread.cpp Will do after the 24h wait period expires tomorrow. ------------- PR: https://git.openjdk.java.net/jdk/pull/3589 From sjohanss at openjdk.java.net Tue Apr 20 14:11:28 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Tue, 20 Apr 2021 14:11:28 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: On Fri, 2 Apr 2021 16:38:02 GMT, Marcus G K Williams wrote: >> When using LargePageSizeInBytes=1G, os::Linux::reserve_memory_special_huge_tlbfs* cannot select large pages smaller than 1G. Code heap usually uses less than 1G, so currently the code precludes code heap from using >> Large pages in this circumstance and when os::Linux::reserve_memory_special_huge_tlbfs* is called page sizes fall back to Linux::page_size() (usually 4k). >> >> This change allows the above use case by populating all large_page_sizes present in /sys/kernel/mm/hugepages in _page_sizes upon calling os::Linux::setup_large_page_size(). >> >> In os::Linux::reserve_memory_special_huge_tlbfs* we then select the largest large page size available in _page_sizes that is smaller than bytes being reserved. > > Marcus G K Williams has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 44 commits: > > - Merge branch 'master' into update_hlp > - Rebase on pull/3073 > > Signed-off-by: Marcus G K Williams > - Merge branch 'pull/3073' into update_hlp > - Thomas review. > > Changed commit_memory_special to return bool to signal if the request succeeded or not. > - Self review. > > Update helper name to better match commit_memory_special(). > - Marcus review. > > Updated comments. > - Ivan review > > Renamed helper to commit_memory_special and updated the comments. > - 8262291: Refactor reserve_memory_special_huge_tlbfs > - Merge branch 'master' into update_hlp > - Addressed kstefanj review suggestions > > Signed-off-by: Marcus G K Williams > - ... and 34 more: https://git.openjdk.java.net/jdk/compare/f60e81bf...f5bfe224 Just created a CSR for this: [JDK-8265517](https://bugs.openjdk.java.net/browse/JDK-8265517) Please take a look at it and provide feedback if you have any. @mgkwill, please update the patch with the text from the CSR for `LargePageSizeInBytes`. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From sjohanss at openjdk.java.net Tue Apr 20 14:47:05 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Tue, 20 Apr 2021 14:47:05 GMT Subject: RFR: 8265326: Strange Characters in G1GC GC Log [v2] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 12:38:44 GMT, Fairoz Matte wrote: >> There was an character formating issue on mac-os due to usage of " >> `FormatBuffer<128>`" >> Used explicit variable to hold the FormatBuffer > > Fairoz Matte has updated the pull request incrementally with one additional commit since the last revision: > > Update g1ConcurrentMarkThread.cpp Looks good. Did you do any further analysis why the explicit variable was needed? I see there is at least one more place where a `FormatBuffer` is used in a constructor. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3589 From github.com+168222+mgkwill at openjdk.java.net Tue Apr 20 15:06:41 2021 From: github.com+168222+mgkwill at openjdk.java.net (Marcus G K Williams) Date: Tue, 20 Apr 2021 15:06:41 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v28] In-Reply-To: References: Message-ID: > Change the meaning of LargePageSizeInBytes to be the maximum large page size the JVM may use (not the only one). A default value of zero will mean to allow the JVM use large page sizes up to the system's default large page size. Marcus G K Williams has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 46 commits: - Fix merge Signed-off-by: Marcus G K Williams - Merge branch 'master' into update_hlp - Merge branch 'master' into update_hlp - Rebase on pull/3073 Signed-off-by: Marcus G K Williams - Merge branch 'pull/3073' into update_hlp - Thomas review. Changed commit_memory_special to return bool to signal if the request succeeded or not. - Self review. Update helper name to better match commit_memory_special(). - Marcus review. Updated comments. - Ivan review Renamed helper to commit_memory_special and updated the comments. - 8262291: Refactor reserve_memory_special_huge_tlbfs - ... and 36 more: https://git.openjdk.java.net/jdk/compare/713483c7...f99fd5dd ------------- Changes: https://git.openjdk.java.net/jdk/pull/1153/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1153&range=27 Stats: 134 lines in 3 files changed: 64 ins; 47 del; 23 mod Patch: https://git.openjdk.java.net/jdk/pull/1153.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1153/head:pull/1153 PR: https://git.openjdk.java.net/jdk/pull/1153 From github.com+168222+mgkwill at openjdk.java.net Tue Apr 20 15:14:12 2021 From: github.com+168222+mgkwill at openjdk.java.net (Marcus G K Williams) Date: Tue, 20 Apr 2021 15:14:12 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: <_ltdF3SwIhzoZ5GhSiK2r3mUHLDApXaCVY78sk1gr9M=.c16cf61f-fe99-4610-abb3-9798d6d19053@github.com> On Tue, 20 Apr 2021 14:07:48 GMT, Stefan Johansson wrote: > Just created a CSR for this: [JDK-8265517](https://bugs.openjdk.java.net/browse/JDK-8265517) > Please take a look at it and provide feedback if you have any. > > @mgkwill, please update the patch with the text from the CSR for `LargePageSizeInBytes`. Thanks for creating the CSR @stefank! I've read through it and it looks reasonable to me. I've changed the text of the summary of this patch to match the summary of the CSR. Please let me know if more context from the CSR is needed. A modification of the code is needed to accomplish what @tstuefe suggested: > Default (`LargePageSizeInBytes=0`) > * old behavior - use the systems default large page size > * new behavior - use all configured page sizes up to and including the systems default large page size I'll update with a suggested route in the next couple of days. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From stuefe at openjdk.java.net Tue Apr 20 15:38:12 2021 From: stuefe at openjdk.java.net (Thomas Stuefe) Date: Tue, 20 Apr 2021 15:38:12 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 14:07:48 GMT, Stefan Johansson wrote: >> Marcus G K Williams has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 44 commits: >> >> - Merge branch 'master' into update_hlp >> - Rebase on pull/3073 >> >> Signed-off-by: Marcus G K Williams >> - Merge branch 'pull/3073' into update_hlp >> - Thomas review. >> >> Changed commit_memory_special to return bool to signal if the request succeeded or not. >> - Self review. >> >> Update helper name to better match commit_memory_special(). >> - Marcus review. >> >> Updated comments. >> - Ivan review >> >> Renamed helper to commit_memory_special and updated the comments. >> - 8262291: Refactor reserve_memory_special_huge_tlbfs >> - Merge branch 'master' into update_hlp >> - Addressed kstefanj review suggestions >> >> Signed-off-by: Marcus G K Williams >> - ... and 34 more: https://git.openjdk.java.net/jdk/compare/f60e81bf...f5bfe224 > > Just created a CSR for this: [JDK-8265517](https://bugs.openjdk.java.net/browse/JDK-8265517) > Please take a look at it and provide feedback if you have any. > > @mgkwill, please update the patch with the text from the CSR for `LargePageSizeInBytes`. Hi @kstefanj, (@mgkwill : there are multiple Stefans, it can get confusing) CSR looks good overall. Thanks for doing this. I comment here to give Marcus a way to participate (not sure if he has JBS access). Some points: - does it have to have the same title as the issue it is associated with? A title like "Change definition of LargePageSizeInBytes" would be easier to understand and search for. - Can the CSR please state that it only affects Linux? Side note, I had a look at the Windows implementation. It processes LargePageSizeInBytes, but I am not sure it has any effect. On Windows, one obtains the "minimum large page size" and uses that value to align the size requested from VirtualAlloc. Thats it, there is no way to specify an explicit large page size when reserving. LargePageSizeInBytes overwrites this "minimum large page size", but all this does is it somewhat affects the allocated size. So it may be that we could just remove this switch from Windows altogether. - "The Java man pages already defines " -> "The Java man page already defines " - Do we allow explicit -XX:LargePageSizeInBytes=0 ? Seems somewhat pointless. Do we handle it today? Cheers, Thomas ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From github.com+168222+mgkwill at openjdk.java.net Tue Apr 20 15:54:11 2021 From: github.com+168222+mgkwill at openjdk.java.net (Marcus G K Williams) Date: Tue, 20 Apr 2021 15:54:11 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 14:07:48 GMT, Stefan Johansson wrote: >> Marcus G K Williams has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 44 commits: >> >> - Merge branch 'master' into update_hlp >> - Rebase on pull/3073 >> >> Signed-off-by: Marcus G K Williams >> - Merge branch 'pull/3073' into update_hlp >> - Thomas review. >> >> Changed commit_memory_special to return bool to signal if the request succeeded or not. >> - Self review. >> >> Update helper name to better match commit_memory_special(). >> - Marcus review. >> >> Updated comments. >> - Ivan review >> >> Renamed helper to commit_memory_special and updated the comments. >> - 8262291: Refactor reserve_memory_special_huge_tlbfs >> - Merge branch 'master' into update_hlp >> - Addressed kstefanj review suggestions >> >> Signed-off-by: Marcus G K Williams >> - ... and 34 more: https://git.openjdk.java.net/jdk/compare/f60e81bf...f5bfe224 > > Just created a CSR for this: [JDK-8265517](https://bugs.openjdk.java.net/browse/JDK-8265517) > Please take a look at it and provide feedback if you have any. > > @mgkwill, please update the patch with the text from the CSR for `LargePageSizeInBytes`. > Hi @kstefanj, > > (@mgkwill : there are multiple Stefans, it can get confusing) Thanks for including me @tstuefe and noting the highlighting of the wrong Stefan. Whoops :P . Updated. > > CSR looks good overall. Thanks for doing this. I comment here to give Marcus a way to participate (not sure if he has JBS access). Some points: > I don't have access yet. Still working towards my authorship. I have a few patches in but not sure of the threshold/process. > * does it have to have the same title as the issue it is associated with? A title like "Change definition of LargePageSizeInBytes" would be easier to understand and search for. +1 > * Can the CSR please state that it only affects Linux? > +1 > Side note, I had a look at the Windows implementation. It processes LargePageSizeInBytes, but I am not sure it has any effect. On Windows, one obtains the "minimum large page size" and uses that value to align the size requested from VirtualAlloc. Thats it, there is no way to specify an explicit large page size when reserving. LargePageSizeInBytes overwrites this "minimum large page size", but all this does is it somewhat affects the allocated size. So it may be that we could just remove this switch from Windows altogether. > Can this be handled in the same CSR? > * "The Java man pages already defines " -> "The Java man page already defines " +1 > * Do we allow explicit -XX:LargePageSizeInBytes=0 ? Seems somewhat pointless. Do we handle it today? Default is `LargePageSizeInBytes=0` - so it seems we would treat `-XX:LargePageSizeInBytes=0` as the same as default. We don't have a specific case for 0 except default handling as far as I can see. ` product(size_t, LargePageSizeInBytes, 0, \ "Large page size (0 to let VM choose the page size)") \ range(0, max_uintx) \ ` > > Cheers, Thomas ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From kbarrett at openjdk.java.net Tue Apr 20 19:58:08 2021 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Tue, 20 Apr 2021 19:58:08 GMT Subject: RFR: 8265326: Strange Characters in G1GC GC Log [v2] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 12:38:44 GMT, Fairoz Matte wrote: >> There was an character formating issue on mac-os due to usage of " >> `FormatBuffer<128>`" >> Used explicit variable to hold the FormatBuffer > > Fairoz Matte has updated the pull request incrementally with one additional commit since the last revision: > > Update g1ConcurrentMarkThread.cpp Marked as reviewed by kbarrett (Reviewer). Looks good. The lifetime of the FormatBuffer temporary ends when the declaration statement ends, not when the declared variable's lifetime ends. ------------- PR: https://git.openjdk.java.net/jdk/pull/3589 From fmatte at openjdk.java.net Wed Apr 21 03:58:09 2021 From: fmatte at openjdk.java.net (Fairoz Matte) Date: Wed, 21 Apr 2021 03:58:09 GMT Subject: RFR: 8265326: Strange Characters in G1GC GC Log [v2] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 12:38:44 GMT, Fairoz Matte wrote: >> There was an character formating issue on mac-os due to usage of " >> `FormatBuffer<128>`" >> Used explicit variable to hold the FormatBuffer > > Fairoz Matte has updated the pull request incrementally with one additional commit since the last revision: > > Update g1ConcurrentMarkThread.cpp Thanks Stefan and Kim for the reviews, ------------- PR: https://git.openjdk.java.net/jdk/pull/3589 From yyang at openjdk.java.net Wed Apr 21 10:07:38 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Wed, 21 Apr 2021 10:07:38 GMT Subject: RFR: 8265322: C2: Simplify control inputs for BarrierSetC2::obj_allocate In-Reply-To: <_NicB3VpqFwTiu2-bKLNhfnQAxJmbtND66g3Md5ck5g=.f9b4edcd-fa2b-4917-9092-bccc38b202df@github.com> References: <_NicB3VpqFwTiu2-bKLNhfnQAxJmbtND66g3Md5ck5g=.f9b4edcd-fa2b-4917-9092-bccc38b202df@github.com> Message-ID: On Fri, 16 Apr 2021 03:06:19 GMT, Yi Yang wrote: > While just using BarrierSetC2::obj_allocate in https://github.com/openjdk/valhalla/pull/385, I noticed the control input `ctrl` for obj_allocate is not so much necessary. This PR simplifies control inputs for BarrierSetC2::obj_allocate. In most cases, it doesn't change anything since `toobig_false` is equivalent to `ctrl`. In other cases, `toobig_false` is created for Unsafe.allocateInstance while instance size is not statically known, `ctrl` would become control input of IfNode whose projects are `toobig_false` and `toobig_true`, old eden_end and old_eden_top can simply accept `toobig_false` as their control input rather than `ctrl`. Can hotspot-gc folks help review this PR? Thanks!?? Yang ------------- PR: https://git.openjdk.java.net/jdk/pull/3529 From sjohanss at openjdk.java.net Wed Apr 21 10:16:41 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Wed, 21 Apr 2021 10:16:41 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 15:51:29 GMT, Marcus G K Williams wrote: >> Just created a CSR for this: [JDK-8265517](https://bugs.openjdk.java.net/browse/JDK-8265517) >> Please take a look at it and provide feedback if you have any. >> >> @mgkwill, please update the patch with the text from the CSR for `LargePageSizeInBytes`. > >> Hi @kstefanj, >> >> (@mgkwill : there are multiple Stefans, it can get confusing) > > Thanks for including me @tstuefe and noting the highlighting of the wrong Stefan. Whoops :P . Updated. >> >> CSR looks good overall. Thanks for doing this. I comment here to give Marcus a way to participate (not sure if he has JBS access). Some points: >> > > I don't have access yet. Still working towards my authorship. I have a few patches in but not sure of the threshold/process. > >> * does it have to have the same title as the issue it is associated with? A title like "Change definition of LargePageSizeInBytes" would be easier to understand and search for. > > +1 > >> * Can the CSR please state that it only affects Linux? >> > > +1 > >> Side note, I had a look at the Windows implementation. It processes LargePageSizeInBytes, but I am not sure it has any effect. On Windows, one obtains the "minimum large page size" and uses that value to align the size requested from VirtualAlloc. Thats it, there is no way to specify an explicit large page size when reserving. LargePageSizeInBytes overwrites this "minimum large page size", but all this does is it somewhat affects the allocated size. So it may be that we could just remove this switch from Windows altogether. >> > > Can this be handled in the same CSR? > >> * "The Java man pages already defines " -> "The Java man page already defines " > > +1 > >> * Do we allow explicit -XX:LargePageSizeInBytes=0 ? Seems somewhat pointless. Do we handle it today? > > Default is `LargePageSizeInBytes=0` - so it seems we would treat `-XX:LargePageSizeInBytes=0` as the same as default. We don't have a specific case for 0 except default handling as far as I can see. > > ` product(size_t, LargePageSizeInBytes, 0, \ > "Large page size (0 to let VM choose the page size)") \ > range(0, max_uintx) \ > ` > >> >> Cheers, Thomas Hi @mgkwill and @tstuefe! > A modification of the code is needed to accomplish what @tstuefe suggested: > > > Default (`LargePageSizeInBytes=0`) > > > > * old behavior - use the systems default large page size > > * new behavior - use all configured page sizes up to and including the systems default large page size > > I'll update with a suggested route in the next couple of days. Nice, an easy approach should be to just remove all large page sizes larger than the decided maximum from _page_sizes. > CSR looks good overall. Thanks for doing this. I comment here to give Marcus a way to participate (not sure if he has JBS access). Some points: > > * does it have to have the same title as the issue it is associated with? A title like "Change definition of LargePageSizeInBytes" would be easier to understand and search for. > I found some other CSRs that had different titels compared to the issues, so change it. Maybe we should also change the name of this issue as well. To something like: Allow multiple large page sizes to be used on Linux > * Can the CSR please state that it only affects Linux? > Not sure we should say only affects Linux, but maybe that only Linux implements support for the new definition (allowing multiple page sizes to be used). But the definition for the flag is the same for all OSes. On Windows we won't make use of the somewhat relaxed definition and it will continue to mean maximum large page size (and only). Is this ok with you? > > Side note, I had a look at the Windows implementation. It processes LargePageSizeInBytes, but I am not sure it has any effect. On Windows, one obtains the "minimum large page size" and uses that value to align the size requested from VirtualAlloc. Thats it, there is no way to specify an explicit large page size when reserving. LargePageSizeInBytes overwrites this "minimum large page size", but all this does is it somewhat affects the allocated size. So it may be that we could just remove this switch from Windows altogether. > Interesting, well it doesn't really go against what we say in the CSR right? > > * "The Java man pages already defines " -> "The Java man page already defines " > Fixed. > * Do we allow explicit -XX:LargePageSizeInBytes=0 ? Seems somewhat pointless. Do we handle it today? Actually we don't, but we probably should handle it. Would be kind of strange preventing someone from setting the default. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From tschatzl at openjdk.java.net Wed Apr 21 10:49:42 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 21 Apr 2021 10:49:42 GMT Subject: RFR: 8265326: Strange Characters in G1GC GC Log [v2] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 12:38:44 GMT, Fairoz Matte wrote: >> There was an character formating issue on mac-os due to usage of " >> `FormatBuffer<128>`" >> Used explicit variable to hold the FormatBuffer > > Fairoz Matte has updated the pull request incrementally with one additional commit since the last revision: > > Update g1ConcurrentMarkThread.cpp Still good. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3589 From lkorinth at openjdk.java.net Wed Apr 21 11:42:04 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Wed, 21 Apr 2021 11:42:04 GMT Subject: RFR: 8231672: Simplify the reference processing parallelization framework [v5] In-Reply-To: References: Message-ID: > With the change of parallel gc to use the same workgang implementation that other gcs uses, it is possible to remove the abstraction used to hide different worker thread management implementations (TaskExecutor). > > At the same time as removing the TaskExecutor, also remove part of the special casing (mostly code duplication), for single threaded execution. > > I consider the new method `prepare_run_task` that modifies the state of the context a step backwards from what was. However, I think removing the Executor with its proxy-tasks and removing separate code paths for serial execution makes the change worth this step back. We also removes ~270 lines of code. > > Some comments: > 1) I use references in some places where they can replace pointers. I could go much further here, but I did not want to bloat the pull request, maybe later change all closures to be references as well? Should I create enhancements for this? > > 2) I added an enum class ThreadModel instead of using a boolean, this could also be used in many more places. I dislike sending lots of bools with a comment like `true /* _is_mt */`. It also adds type safety if a method takes many bools. However, with this limited change, and not many hard-coded bools, it feels a bit overkill and I am willing to remove the enum, what do you think? > > Testing: > hotspot_gc and tier 1-3 has passed earlier versions before minor cleanups. I intend to re-run tests after review changes. Leo Korinth has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits: - make RefProcTask a StackObj - Merge branch 'master' into _8231672 - using proxies instead of ClosureContexts - remove maybe prefix, change in one place to nullable - Merge branch 'master' into _8231672 - Changes after first review - 8231672: Simplify the reference processing parallelization framework ------------- Changes: https://git.openjdk.java.net/jdk/pull/2782/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2782&range=04 Stats: 890 lines in 18 files changed: 162 ins; 519 del; 209 mod Patch: https://git.openjdk.java.net/jdk/pull/2782.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2782/head:pull/2782 PR: https://git.openjdk.java.net/jdk/pull/2782 From lkorinth at openjdk.java.net Wed Apr 21 11:47:44 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Wed, 21 Apr 2021 11:47:44 GMT Subject: RFR: 8231672: Simplify the reference processing parallelization framework [v5] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 11:42:04 GMT, Leo Korinth wrote: >> With the change of parallel gc to use the same workgang implementation that other gcs uses, it is possible to remove the abstraction used to hide different worker thread management implementations (TaskExecutor). >> >> At the same time as removing the TaskExecutor, also remove part of the special casing (mostly code duplication), for single threaded execution. >> >> I consider the new method `prepare_run_task` that modifies the state of the context a step backwards from what was. However, I think removing the Executor with its proxy-tasks and removing separate code paths for serial execution makes the change worth this step back. We also removes ~270 lines of code. >> >> Some comments: >> 1) I use references in some places where they can replace pointers. I could go much further here, but I did not want to bloat the pull request, maybe later change all closures to be references as well? Should I create enhancements for this? >> >> 2) I added an enum class ThreadModel instead of using a boolean, this could also be used in many more places. I dislike sending lots of bools with a comment like `true /* _is_mt */`. It also adds type safety if a method takes many bools. However, with this limited change, and not many hard-coded bools, it feels a bit overkill and I am willing to remove the enum, what do you think? >> >> Testing: >> hotspot_gc and tier 1-3 has passed earlier versions before minor cleanups. I intend to re-run tests after review changes. > > Leo Korinth has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits: > > - make RefProcTask a StackObj > - Merge branch 'master' into _8231672 > - using proxies instead of ClosureContexts > - remove maybe prefix, change in one place to nullable > - Merge branch 'master' into _8231672 > - Changes after first review > - 8231672: Simplify the reference processing parallelization framework So in addition to using proxy classes instead of ClosureContexts, I have also fixed an earlier mistake of mine when getting the ParCompactionManager and PSPromotionManager in the VM thread I will no longer use the index of worker_id and instead use the dedicated manager for the VM thread. ------------- PR: https://git.openjdk.java.net/jdk/pull/2782 From fmatte at openjdk.java.net Wed Apr 21 11:51:36 2021 From: fmatte at openjdk.java.net (Fairoz Matte) Date: Wed, 21 Apr 2021 11:51:36 GMT Subject: Integrated: 8265326: Strange Characters in G1GC GC Log In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 12:02:58 GMT, Fairoz Matte wrote: > There was an character formating issue on mac-os due to usage of " > `FormatBuffer<128>`" > Used explicit variable to hold the FormatBuffer This pull request has now been integrated. Changeset: da860290 Author: Fairoz Matte Committer: Thomas Schatzl URL: https://git.openjdk.java.net/jdk/commit/da860290 Stats: 3 lines in 1 file changed: 0 ins; 0 del; 3 mod 8265326: Strange Characters in G1GC GC Log Reviewed-by: tschatzl, sjohanss, kbarrett ------------- PR: https://git.openjdk.java.net/jdk/pull/3589 From shade at openjdk.java.net Wed Apr 21 12:37:40 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Wed, 21 Apr 2021 12:37:40 GMT Subject: RFR: 8261495: Shenandoah: reconsider update references memory ordering [v4] In-Reply-To: References: <4RLKvcdaWu0Cu6owC3yGoVY1KVEsYjBZEFJhfdwnhWg=.65fbeae1-58f6-48d3-a2ed-981858ef7da9@github.com> Message-ID: On Tue, 16 Feb 2021 13:21:00 GMT, Aleksey Shipilev wrote: >> Shenandoah update heap references code uses default Atomic::cmpxchg to avoid races with mutator updates. Hotspot's default for atomic operations is memory_order_conservative, which emits two-way memory fences around the CASes at least on AArch64 and PPC64. >> >> This seems to be excessive for Shenandoah update references code, and "release" is enough. We do not seem to piggyback on update-references memory effects anywhere (in fact, if not for mutator, we would not even need a CAS). But, there is an interplay with concurrent evacuation and updates from self-healing. >> >> Average time goes down, the number of GC cycles go up, since the cycles are shorter. >> >> Additional testing: >> - [x] Linux x86_64 hotspot_gc_shenandoah >> - [x] Linux AArch64 hotspot_gc_shenandoah >> - [x] Linux AArch64 tier1 with Shenandoah > > Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 10 additional commits since the last revision: > > - Comment touchup > - Specialize out witness-checking methods, drop acquire again > - Even more explanation > - Move the comment > - Also handle clearing the oops > - Minor touchups to the comment > - Merge branch 'master' into JDK-8261495-shenandoah-updaterefs-memord > - Use release only > - Do acq_rel instead > - 8261495: Shenandoah: reconsider update references memory ordering Not yet, bot. Waiting for JDK-8261579 to remeasure perf differences. ------------- PR: https://git.openjdk.java.net/jdk/pull/2498 From stuefe at openjdk.java.net Wed Apr 21 12:49:39 2021 From: stuefe at openjdk.java.net (Thomas Stuefe) Date: Wed, 21 Apr 2021 12:49:39 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 15:51:29 GMT, Marcus G K Williams wrote: >> Just created a CSR for this: [JDK-8265517](https://bugs.openjdk.java.net/browse/JDK-8265517) >> Please take a look at it and provide feedback if you have any. >> >> @mgkwill, please update the patch with the text from the CSR for `LargePageSizeInBytes`. > >> Hi @kstefanj, >> >> (@mgkwill : there are multiple Stefans, it can get confusing) > > Thanks for including me @tstuefe and noting the highlighting of the wrong Stefan. Whoops :P . Updated. >> >> CSR looks good overall. Thanks for doing this. I comment here to give Marcus a way to participate (not sure if he has JBS access). Some points: >> > > I don't have access yet. Still working towards my authorship. I have a few patches in but not sure of the threshold/process. > >> * does it have to have the same title as the issue it is associated with? A title like "Change definition of LargePageSizeInBytes" would be easier to understand and search for. > > +1 > >> * Can the CSR please state that it only affects Linux? >> > > +1 > >> Side note, I had a look at the Windows implementation. It processes LargePageSizeInBytes, but I am not sure it has any effect. On Windows, one obtains the "minimum large page size" and uses that value to align the size requested from VirtualAlloc. Thats it, there is no way to specify an explicit large page size when reserving. LargePageSizeInBytes overwrites this "minimum large page size", but all this does is it somewhat affects the allocated size. So it may be that we could just remove this switch from Windows altogether. >> > > Can this be handled in the same CSR? > >> * "The Java man pages already defines " -> "The Java man page already defines " > > +1 > >> * Do we allow explicit -XX:LargePageSizeInBytes=0 ? Seems somewhat pointless. Do we handle it today? > > Default is `LargePageSizeInBytes=0` - so it seems we would treat `-XX:LargePageSizeInBytes=0` as the same as default. We don't have a specific case for 0 except default handling as far as I can see. > > ` product(size_t, LargePageSizeInBytes, 0, \ > "Large page size (0 to let VM choose the page size)") \ > range(0, max_uintx) \ > ` > >> >> Cheers, Thomas > Hi @mgkwill and @tstuefe! > > > A modification of the code is needed to accomplish what @tstuefe suggested: > > > Default (`LargePageSizeInBytes=0`) > > > > > > * old behavior - use the systems default large page size > > > * new behavior - use all configured page sizes up to and including the systems default large page size > > > > > > I'll update with a suggested route in the next couple of days. > > Nice, an easy approach should be to just remove all large page sizes larger than the decided maximum from _page_sizes. > > > CSR looks good overall. Thanks for doing this. I comment here to give Marcus a way to participate (not sure if he has JBS access). Some points: > > > > * does it have to have the same title as the issue it is associated with? A title like "Change definition of LargePageSizeInBytes" would be easier to understand and search for. > > I found some other CSRs that had different titels compared to the issues, so change it. Maybe we should also change the name of this issue as well. To something like: > Allow multiple large page sizes to be used on Linux > > > * Can the CSR please state that it only affects Linux? > > Not sure we should say only affects Linux, but maybe that only Linux implements support for the new definition (allowing multiple page sizes to be used). But the definition for the flag is the same for all OSes. On Windows we won't make use of the somewhat relaxed definition and it will continue to mean maximum large page size (and only). Is this ok with you? > > > Side note, I had a look at the Windows implementation. It processes LargePageSizeInBytes, but I am not sure it has any effect. On Windows, one obtains the "minimum large page size" and uses that value to align the size requested from VirtualAlloc. Thats it, there is no way to specify an explicit large page size when reserving. LargePageSizeInBytes overwrites this "minimum large page size", but all this does is it somewhat affects the allocated size. So it may be that we could just remove this switch from Windows altogether. > > Interesting, well it doesn't really go against what we say in the CSR right? I am fine with leaving Windows implicitly covered. > > > * "The Java man pages already defines " -> "The Java man page already defines " > > Fixed. > > > * Do we allow explicit -XX:LargePageSizeInBytes=0 ? Seems somewhat pointless. Do we handle it today? > > Actually we don't, but we probably should handle it. Would be kind of strange preventing someone from setting the default. I marked the CSR as reviewed. Thanks & Cheers, Thomas ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From lkorinth at openjdk.java.net Wed Apr 21 13:05:41 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Wed, 21 Apr 2021 13:05:41 GMT Subject: RFR: 8265461: G1: Forwarding pointer removal thread sizing In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 09:34:45 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that improves forwarding pointer removal handling by adding thread sizing to the work gang? > > Since work is distributed per region at the moment, it does not make sense to start more threads than regions. > > This can particularly help performance with object/region pinning. > > Testing: tier1-3, using self-verification code > > Thanks, > Thomas src/hotspot/share/gc/g1/g1CollectedHeap.cpp line 3665: > 3663: > 3664: _expand_heap_after_alloc_failure = true; > 3665: _num_regions_failed_evacuation = false; _num_regions_failed_evacuation = 0; ? src/hotspot/share/gc/g1/g1EvacFailure.cpp line 211: > 209: > 210: public: > 211: RemoveSelfForwardPtrHRClosure(G1RedirtyCardsQueueSet* rdcqs, uint worker_id, uint volatile* num_failed_regions) : Maybe increment failed regions through G1CollectedHeap instead of a pointer to the value? src/hotspot/share/gc/g1/g1EvacFailure.cpp line 285: > 283: uint G1ParRemoveSelfForwardPtrsTask::num_failed_regions() const { > 284: return Atomic::load(&_num_failed_regions); > 285: } Maybe add newline, C++11 allows no newline, but github complains. ------------- PR: https://git.openjdk.java.net/jdk/pull/3585 From sjohanss at openjdk.java.net Wed Apr 21 13:10:39 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Wed, 21 Apr 2021 13:10:39 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 12:46:21 GMT, Thomas Stuefe wrote: >>> Hi @kstefanj, >>> >>> (@mgkwill : there are multiple Stefans, it can get confusing) >> >> Thanks for including me @tstuefe and noting the highlighting of the wrong Stefan. Whoops :P . Updated. >>> >>> CSR looks good overall. Thanks for doing this. I comment here to give Marcus a way to participate (not sure if he has JBS access). Some points: >>> >> >> I don't have access yet. Still working towards my authorship. I have a few patches in but not sure of the threshold/process. >> >>> * does it have to have the same title as the issue it is associated with? A title like "Change definition of LargePageSizeInBytes" would be easier to understand and search for. >> >> +1 >> >>> * Can the CSR please state that it only affects Linux? >>> >> >> +1 >> >>> Side note, I had a look at the Windows implementation. It processes LargePageSizeInBytes, but I am not sure it has any effect. On Windows, one obtains the "minimum large page size" and uses that value to align the size requested from VirtualAlloc. Thats it, there is no way to specify an explicit large page size when reserving. LargePageSizeInBytes overwrites this "minimum large page size", but all this does is it somewhat affects the allocated size. So it may be that we could just remove this switch from Windows altogether. >>> >> >> Can this be handled in the same CSR? >> >>> * "The Java man pages already defines " -> "The Java man page already defines " >> >> +1 >> >>> * Do we allow explicit -XX:LargePageSizeInBytes=0 ? Seems somewhat pointless. Do we handle it today? >> >> Default is `LargePageSizeInBytes=0` - so it seems we would treat `-XX:LargePageSizeInBytes=0` as the same as default. We don't have a specific case for 0 except default handling as far as I can see. >> >> ` product(size_t, LargePageSizeInBytes, 0, \ >> "Large page size (0 to let VM choose the page size)") \ >> range(0, max_uintx) \ >> ` >> >>> >>> Cheers, Thomas > >> Hi @mgkwill and @tstuefe! >> >> > A modification of the code is needed to accomplish what @tstuefe suggested: >> > > Default (`LargePageSizeInBytes=0`) >> > > >> > > * old behavior - use the systems default large page size >> > > * new behavior - use all configured page sizes up to and including the systems default large page size >> > >> > >> > I'll update with a suggested route in the next couple of days. >> >> Nice, an easy approach should be to just remove all large page sizes larger than the decided maximum from _page_sizes. >> >> > CSR looks good overall. Thanks for doing this. I comment here to give Marcus a way to participate (not sure if he has JBS access). Some points: >> > >> > * does it have to have the same title as the issue it is associated with? A title like "Change definition of LargePageSizeInBytes" would be easier to understand and search for. >> >> I found some other CSRs that had different titels compared to the issues, so change it. Maybe we should also change the name of this issue as well. To something like: >> Allow multiple large page sizes to be used on Linux >> >> > * Can the CSR please state that it only affects Linux? >> >> Not sure we should say only affects Linux, but maybe that only Linux implements support for the new definition (allowing multiple page sizes to be used). But the definition for the flag is the same for all OSes. On Windows we won't make use of the somewhat relaxed definition and it will continue to mean maximum large page size (and only). Is this ok with you? >> >> > Side note, I had a look at the Windows implementation. It processes LargePageSizeInBytes, but I am not sure it has any effect. On Windows, one obtains the "minimum large page size" and uses that value to align the size requested from VirtualAlloc. Thats it, there is no way to specify an explicit large page size when reserving. LargePageSizeInBytes overwrites this "minimum large page size", but all this does is it somewhat affects the allocated size. So it may be that we could just remove this switch from Windows altogether. >> >> Interesting, well it doesn't really go against what we say in the CSR right? > > I am fine with leaving Windows implicitly covered. > >> >> > * "The Java man pages already defines " -> "The Java man page already defines " >> >> Fixed. >> >> > * Do we allow explicit -XX:LargePageSizeInBytes=0 ? Seems somewhat pointless. Do we handle it today? >> >> Actually we don't, but we probably should handle it. Would be kind of strange preventing someone from setting the default. > > I marked the CSR as reviewed. > > Thanks & Cheers, Thomas Thanks for reviewing the the CSR @tstuefe. Internal discussions around the text in the CSR pointed out the "system" might not be the best wording when it comes to default size. I will update the CSR using "for the environment" instead, for example: > ?By default, the size is set to 0, meaning that the JVM will use the default large page size for the environment as the maximum size for large pages.? Hope this is still good. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From iwalulya at openjdk.java.net Wed Apr 21 14:21:35 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Wed, 21 Apr 2021 14:21:35 GMT Subject: RFR: 8265461: G1: Forwarding pointer removal thread sizing In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 09:34:45 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that improves forwarding pointer removal handling by adding thread sizing to the work gang? > > Since work is distributed per region at the moment, it does not make sense to start more threads than regions. > > This can particularly help performance with object/region pinning. > > Testing: tier1-3, using self-verification code > > Thanks, > Thomas src/hotspot/share/gc/g1/g1CollectedHeap.inline.hpp line 200: > 198: } > 199: > 200: void G1CollectedHeap::region_failed_evacuation() { "region_failed_evacuation" name does not give any indication on what the method actually does. ------------- PR: https://git.openjdk.java.net/jdk/pull/3585 From iwalulya at openjdk.java.net Wed Apr 21 14:28:34 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Wed, 21 Apr 2021 14:28:34 GMT Subject: RFR: 8265450: Merge PreservedMarksSet::restore code paths In-Reply-To: References: Message-ID: <5s9HudarnDsw5G3jqHOVTnNkqVdMQAHbZ63cxfKhLQM=.b6fd32bc-caa8-4abc-a218-ab6d2dd3bb1c@github.com> On Tue, 20 Apr 2021 08:39:56 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this refactoring that merges parallel and serial preserved marks restoration during evacuation failure into a single code path, simply by having the serial path call the parallel path in the VM thread. > > This just reduces code and avoids the potential bugs when updating one path but not the other (which already happened). > > The execution difference should not really matter for Serial GC where the serial path is executed: > - we are talking about the evacuation failure path > - there are likely a lot of preserved marks (random writes to the heap) that offset these two additional calls and some tiny superfluous member initialization. > > Testing: tier1 > > Thanks, > Thomas lgtm! ------------- Marked as reviewed by iwalulya (Committer). PR: https://git.openjdk.java.net/jdk/pull/3584 From stuefe at openjdk.java.net Wed Apr 21 15:31:41 2021 From: stuefe at openjdk.java.net (Thomas Stuefe) Date: Wed, 21 Apr 2021 15:31:41 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 12:46:21 GMT, Thomas Stuefe wrote: >>> Hi @kstefanj, >>> >>> (@mgkwill : there are multiple Stefans, it can get confusing) >> >> Thanks for including me @tstuefe and noting the highlighting of the wrong Stefan. Whoops :P . Updated. >>> >>> CSR looks good overall. Thanks for doing this. I comment here to give Marcus a way to participate (not sure if he has JBS access). Some points: >>> >> >> I don't have access yet. Still working towards my authorship. I have a few patches in but not sure of the threshold/process. >> >>> * does it have to have the same title as the issue it is associated with? A title like "Change definition of LargePageSizeInBytes" would be easier to understand and search for. >> >> +1 >> >>> * Can the CSR please state that it only affects Linux? >>> >> >> +1 >> >>> Side note, I had a look at the Windows implementation. It processes LargePageSizeInBytes, but I am not sure it has any effect. On Windows, one obtains the "minimum large page size" and uses that value to align the size requested from VirtualAlloc. Thats it, there is no way to specify an explicit large page size when reserving. LargePageSizeInBytes overwrites this "minimum large page size", but all this does is it somewhat affects the allocated size. So it may be that we could just remove this switch from Windows altogether. >>> >> >> Can this be handled in the same CSR? >> >>> * "The Java man pages already defines " -> "The Java man page already defines " >> >> +1 >> >>> * Do we allow explicit -XX:LargePageSizeInBytes=0 ? Seems somewhat pointless. Do we handle it today? >> >> Default is `LargePageSizeInBytes=0` - so it seems we would treat `-XX:LargePageSizeInBytes=0` as the same as default. We don't have a specific case for 0 except default handling as far as I can see. >> >> ` product(size_t, LargePageSizeInBytes, 0, \ >> "Large page size (0 to let VM choose the page size)") \ >> range(0, max_uintx) \ >> ` >> >>> >>> Cheers, Thomas > >> Hi @mgkwill and @tstuefe! >> >> > A modification of the code is needed to accomplish what @tstuefe suggested: >> > > Default (`LargePageSizeInBytes=0`) >> > > >> > > * old behavior - use the systems default large page size >> > > * new behavior - use all configured page sizes up to and including the systems default large page size >> > >> > >> > I'll update with a suggested route in the next couple of days. >> >> Nice, an easy approach should be to just remove all large page sizes larger than the decided maximum from _page_sizes. >> >> > CSR looks good overall. Thanks for doing this. I comment here to give Marcus a way to participate (not sure if he has JBS access). Some points: >> > >> > * does it have to have the same title as the issue it is associated with? A title like "Change definition of LargePageSizeInBytes" would be easier to understand and search for. >> >> I found some other CSRs that had different titels compared to the issues, so change it. Maybe we should also change the name of this issue as well. To something like: >> Allow multiple large page sizes to be used on Linux >> >> > * Can the CSR please state that it only affects Linux? >> >> Not sure we should say only affects Linux, but maybe that only Linux implements support for the new definition (allowing multiple page sizes to be used). But the definition for the flag is the same for all OSes. On Windows we won't make use of the somewhat relaxed definition and it will continue to mean maximum large page size (and only). Is this ok with you? >> >> > Side note, I had a look at the Windows implementation. It processes LargePageSizeInBytes, but I am not sure it has any effect. On Windows, one obtains the "minimum large page size" and uses that value to align the size requested from VirtualAlloc. Thats it, there is no way to specify an explicit large page size when reserving. LargePageSizeInBytes overwrites this "minimum large page size", but all this does is it somewhat affects the allocated size. So it may be that we could just remove this switch from Windows altogether. >> >> Interesting, well it doesn't really go against what we say in the CSR right? > > I am fine with leaving Windows implicitly covered. > >> >> > * "The Java man pages already defines " -> "The Java man page already defines " >> >> Fixed. >> >> > * Do we allow explicit -XX:LargePageSizeInBytes=0 ? Seems somewhat pointless. Do we handle it today? >> >> Actually we don't, but we probably should handle it. Would be kind of strange preventing someone from setting the default. > > I marked the CSR as reviewed. > > Thanks & Cheers, Thomas > Thanks for reviewing the the CSR @tstuefe. Internal discussions around the text in the CSR pointed out the "system" might not be the best wording when it comes to default size. I will update the CSR using "for the environment" instead, for example: > > > ?By default, the size is set to 0, meaning that the JVM will use the default large page size for the environment as the maximum size for large pages.? > > Hope this is still good. Still good. I was vaguely concerned about the "default" in "default system page size" because that is a Linux concept; on other Unices which have multiple page sizes there is no concept of a default (eg AIX, HPUX). But this is just idle nitpicking, the CSR is clear the way you wrote it. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From tschatzl at openjdk.java.net Wed Apr 21 15:48:57 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 21 Apr 2021 15:48:57 GMT Subject: RFR: 8265461: G1: Forwarding pointer removal thread sizing [v2] In-Reply-To: References: Message-ID: > Hi all, > > can I have reviews for this change that improves forwarding pointer removal handling by adding thread sizing to the work gang? > > Since work is distributed per region at the moment, it does not make sense to start more threads than regions. > > This can particularly help performance with object/region pinning. > > Testing: tier1-3, using self-verification code > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: - Added comment - iwalulya lkorinth reviews ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3585/files - new: https://git.openjdk.java.net/jdk/pull/3585/files/17ba2812..baf8d0d6 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3585&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3585&range=00-01 Stats: 7 lines in 5 files changed: 2 ins; 0 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/3585.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3585/head:pull/3585 PR: https://git.openjdk.java.net/jdk/pull/3585 From tschatzl at openjdk.java.net Wed Apr 21 15:48:58 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 21 Apr 2021 15:48:58 GMT Subject: RFR: 8265461: G1: Forwarding pointer removal thread sizing [v2] In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 13:59:09 GMT, Leo Korinth wrote: >> Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: >> >> - Added comment >> - iwalulya lkorinth reviews > > src/hotspot/share/gc/g1/g1CollectedHeap.cpp line 3665: > >> 3663: >> 3664: _expand_heap_after_alloc_failure = true; >> 3665: _num_regions_failed_evacuation = false; > > _num_regions_failed_evacuation = 0; ? Fixed. > src/hotspot/share/gc/g1/g1EvacFailure.cpp line 211: > >> 209: >> 210: public: >> 211: RemoveSelfForwardPtrHRClosure(G1RedirtyCardsQueueSet* rdcqs, uint worker_id, uint volatile* num_failed_regions) : > > Maybe increment failed regions through G1CollectedHeap instead of a pointer to the value? You probably mean `G1ParRemoveSelfForwardPtrsTask`. I would not want to add that counter to `G1CollectedHeap` if possible. The suggestion seems to make the code just more verbose, not "cleaner", but that is subjective of course. ------------- PR: https://git.openjdk.java.net/jdk/pull/3585 From tschatzl at openjdk.java.net Wed Apr 21 15:48:59 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 21 Apr 2021 15:48:59 GMT Subject: RFR: 8265461: G1: Forwarding pointer removal thread sizing [v2] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 13:54:49 GMT, Ivan Walulya wrote: >> Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: >> >> - Added comment >> - iwalulya lkorinth reviews > > src/hotspot/share/gc/g1/g1CollectedHeap.inline.hpp line 200: > >> 198: } >> 199: >> 200: void G1CollectedHeap::region_failed_evacuation() { > > "region_failed_evacuation" name does not give any indication on what the method actually does. Changed to `notify_region_failed_evacuation()` and added comment. ------------- PR: https://git.openjdk.java.net/jdk/pull/3585 From tschatzl at openjdk.java.net Wed Apr 21 15:52:34 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 21 Apr 2021 15:52:34 GMT Subject: RFR: 8265450: Merge PreservedMarksSet::restore code paths In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 10:36:21 GMT, Stefan Johansson wrote: >> Hi all, >> >> can I have reviews for this refactoring that merges parallel and serial preserved marks restoration during evacuation failure into a single code path, simply by having the serial path call the parallel path in the VM thread. >> >> This just reduces code and avoids the potential bugs when updating one path but not the other (which already happened). >> >> The execution difference should not really matter for Serial GC where the serial path is executed: >> - we are talking about the evacuation failure path >> - there are likely a lot of preserved marks (random writes to the heap) that offset these two additional calls and some tiny superfluous member initialization. >> >> Testing: tier1 >> >> Thanks, >> Thomas > > Looks good. Just a small comment. Thanks @kstefanj @walulyai for your reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/3584 From tschatzl at openjdk.java.net Wed Apr 21 15:52:34 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 21 Apr 2021 15:52:34 GMT Subject: RFR: 8265450: Merge PreservedMarksSet::restore code paths In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 11:26:58 GMT, Thomas Schatzl wrote: >> src/hotspot/share/gc/shared/preservedMarks.cpp line 100: >> >>> 98: volatile size_t _total_size; >>> 99: #ifdef ASSERT >>> 100: size_t _total_size_before; >> >> I know `_total_size` is what was used before, but what do you think about changing it to `_total_count` now when we are touching it. > > Should we rename `size` to `count` all the way through? Also `PreservedMarks` uses `size`, and that is where this comes from. There are likely other places where we use `size` instead of `count` (or `num`, depending on when the code has been written the first time). > Not sure this is the appropriate CR to do that, but not against it either. I decided to keep this kind of renaming out of this CR. We can discuss this offline and file an issue that I can look at. ------------- PR: https://git.openjdk.java.net/jdk/pull/3584 From tschatzl at openjdk.java.net Wed Apr 21 15:52:35 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 21 Apr 2021 15:52:35 GMT Subject: Integrated: 8265450: Merge PreservedMarksSet::restore code paths In-Reply-To: References: Message-ID: On Tue, 20 Apr 2021 08:39:56 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this refactoring that merges parallel and serial preserved marks restoration during evacuation failure into a single code path, simply by having the serial path call the parallel path in the VM thread. > > This just reduces code and avoids the potential bugs when updating one path but not the other (which already happened). > > The execution difference should not really matter for Serial GC where the serial path is executed: > - we are talking about the evacuation failure path > - there are likely a lot of preserved marks (random writes to the heap) that offset these two additional calls and some tiny superfluous member initialization. > > Testing: tier1 > > Thanks, > Thomas This pull request has now been integrated. Changeset: 325edbc3 Author: Thomas Schatzl URL: https://git.openjdk.java.net/jdk/commit/325edbc3 Stats: 49 lines in 1 file changed: 11 ins; 14 del; 24 mod 8265450: Merge PreservedMarksSet::restore code paths Reviewed-by: sjohanss, iwalulya ------------- PR: https://git.openjdk.java.net/jdk/pull/3584 From github.com+168222+mgkwill at openjdk.java.net Wed Apr 21 18:03:39 2021 From: github.com+168222+mgkwill at openjdk.java.net (Marcus G K Williams) Date: Wed, 21 Apr 2021 18:03:39 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 12:46:21 GMT, Thomas Stuefe wrote: > Nice, an easy approach should be to just remove all large page sizes larger than the decided maximum from _page_sizes. Sounds like a good approach, I'll do that. >> * Do we allow explicit -XX:LargePageSizeInBytes=0 ? Seems somewhat pointless. Do we handle it today? > Actually we don't, but we probably should handle it. Would be kind of strange preventing someone from setting the default. How should we handle this differently? ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From github.com+168222+mgkwill at openjdk.java.net Wed Apr 21 23:08:23 2021 From: github.com+168222+mgkwill at openjdk.java.net (Marcus G K Williams) Date: Wed, 21 Apr 2021 23:08:23 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 12:46:21 GMT, Thomas Stuefe wrote: > I found some other CSRs that had different titels compared to the issues, so change it. Maybe we should also change the name of this issue as well. To something like: > Allow multiple large page sizes to be used on Linux I'd be in favor of this. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From dholmes at openjdk.java.net Thu Apr 22 04:37:20 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Thu, 22 Apr 2021 04:37:20 GMT Subject: RFR: 8265702: ZGC on macOS/aarch64 In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 21:10:02 GMT, Per Liden wrote: > This patch enables ZGC on macOS/aarch64. It does three things: > 1) Enables building of ZGC on this platform. > 2) Adds `os::processor_id()`, which for now always returns 0. > 3) Fixes a WX issue in `OptoRuntime::handle_exception_C()`, where the stackwater mark might unnecessarily process a frame when the thread is in WXExec mode. In this case, the we're not touching any oops, so we don't need to process any frames. > > Testing: Passes the same tests as macOS/x86_64 (with the exception of pre-existing issues unrelated to ZGC). Hi Per, Looks good. One minor nit. Thanks, David src/hotspot/os/bsd/os_bsd.cpp line 2149: > 2147: > 2148: return (uint)processor_id; > 2149: #else Please add a comment indicating what platform this else is referring to. ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3609 From stefank at openjdk.java.net Thu Apr 22 06:01:26 2021 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Thu, 22 Apr 2021 06:01:26 GMT Subject: RFR: 8265702: ZGC on macOS/aarch64 In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 21:10:02 GMT, Per Liden wrote: > This patch enables ZGC on macOS/aarch64. It does three things: > 1) Enables building of ZGC on this platform. > 2) Adds `os::processor_id()`, which for now always returns 0. > 3) Fixes a WX issue in `OptoRuntime::handle_exception_C()`, where the stackwater mark might unnecessarily process a frame when the thread is in WXExec mode. In this case, the we're not touching any oops, so we don't need to process any frames. > > Testing: Passes the same tests as macOS/x86_64 (with the exception of pre-existing issues unrelated to ZGC). Marked as reviewed by stefank (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3609 From pliden at openjdk.java.net Thu Apr 22 06:43:46 2021 From: pliden at openjdk.java.net (Per Liden) Date: Thu, 22 Apr 2021 06:43:46 GMT Subject: RFR: 8265702: ZGC on macOS/aarch64 [v2] In-Reply-To: References: Message-ID: <2yB8guCDtonp_f251AssKN1-8TuSYqH2zjqjrEmO7ks=.704854fb-c601-4417-a34c-46e7242f2743@github.com> On Thu, 22 Apr 2021 04:26:11 GMT, David Holmes wrote: >> Per Liden has updated the pull request incrementally with one additional commit since the last revision: >> >> Add comment to #else > > src/hotspot/os/bsd/os_bsd.cpp line 2149: > >> 2147: >> 2148: return (uint)processor_id; >> 2149: #else > > Please add a comment indicating what platform this else is referring to. Will fix! ------------- PR: https://git.openjdk.java.net/jdk/pull/3609 From pliden at openjdk.java.net Thu Apr 22 06:43:45 2021 From: pliden at openjdk.java.net (Per Liden) Date: Thu, 22 Apr 2021 06:43:45 GMT Subject: RFR: 8265702: ZGC on macOS/aarch64 [v2] In-Reply-To: References: Message-ID: <7qkG6INxywBIAyzPqO812c7Pn7IUp5gZzfcjl27lE2E=.73fa7b17-7f51-4cdc-b621-e44d3b619740@github.com> > This patch enables ZGC on macOS/aarch64. It does three things: > 1) Enables building of ZGC on this platform. > 2) Adds `os::processor_id()`, which for now always returns 0. > 3) Fixes a WX issue in `OptoRuntime::handle_exception_C()`, where the stackwater mark might unnecessarily process a frame when the thread is in WXExec mode. In this case, the we're not touching any oops, so we don't need to process any frames. > > Testing: Passes the same tests as macOS/x86_64 (with the exception of pre-existing issues unrelated to ZGC). Per Liden has updated the pull request incrementally with one additional commit since the last revision: Add comment to #else ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3609/files - new: https://git.openjdk.java.net/jdk/pull/3609/files/c8913936..dd8c42ff Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3609&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3609&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3609.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3609/head:pull/3609 PR: https://git.openjdk.java.net/jdk/pull/3609 From sjohanss at openjdk.java.net Thu Apr 22 06:57:22 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Thu, 22 Apr 2021 06:57:22 GMT Subject: RFR: 8265450: Merge PreservedMarksSet::restore code paths In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 15:47:57 GMT, Thomas Schatzl wrote: >> Should we rename `size` to `count` all the way through? Also `PreservedMarks` uses `size`, and that is where this comes from. There are likely other places where we use `size` instead of `count` (or `num`, depending on when the code has been written the first time). >> Not sure this is the appropriate CR to do that, but not against it either. > > I decided to keep this kind of renaming out of this CR. We can discuss this offline and file an issue that I can look at. Sorry for missing your reply. Notification got lost, I'm fine with doing this as a separate thing if we feel the need. ------------- PR: https://git.openjdk.java.net/jdk/pull/3584 From sjohanss at openjdk.java.net Thu Apr 22 07:05:28 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Thu, 22 Apr 2021 07:05:28 GMT Subject: RFR: 8256155: os::Linux Populate all large_page_sizes, select smallest page size in reserve_memory_special_huge_tlbfs* [v27] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 18:00:52 GMT, Marcus G K Williams wrote: > > > * Do we allow explicit -XX:LargePageSizeInBytes=0 ? Seems somewhat pointless. Do we handle it today? > > > Actually we don't, but we probably should handle it. Would be kind of strange preventing someone from setting the default. > > How should we handle this differently? I just mean that if `LargePageSizeInBytes=0` on the command line that should give the same result as not setting anything. Right now you get a warning, and the default behavior. The warning should go away since it is documented to mean use the default. > $ java -XX:+UseLargePages -XX:LargePageSizeInBytes=0 -version > Java HotSpot(TM) 64-Bit Server VM warning: Setting LargePageSizeInBytes=0 has no effect on this OS. Using the default large page size 2048K. I also updated the title for the issue in JBS and I suspect that you will get a notification to change it here as well. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From lkorinth at openjdk.java.net Thu Apr 22 10:10:22 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Thu, 22 Apr 2021 10:10:22 GMT Subject: RFR: 8265461: G1: Forwarding pointer removal thread sizing [v2] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 15:43:14 GMT, Thomas Schatzl wrote: >> src/hotspot/share/gc/g1/g1EvacFailure.cpp line 211: >> >>> 209: >>> 210: public: >>> 211: RemoveSelfForwardPtrHRClosure(G1RedirtyCardsQueueSet* rdcqs, uint worker_id, uint volatile* num_failed_regions) : >> >> Maybe increment failed regions through G1CollectedHeap instead of a pointer to the value? > > You probably mean `G1ParRemoveSelfForwardPtrsTask`. I would not want to add that counter to `G1CollectedHeap` if possible. The suggestion seems to make the code just more verbose, not "cleaner", but that is subjective of course. Okay. ------------- PR: https://git.openjdk.java.net/jdk/pull/3585 From iwalulya at openjdk.java.net Thu Apr 22 10:56:21 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Thu, 22 Apr 2021 10:56:21 GMT Subject: RFR: 8265461: G1: Forwarding pointer removal thread sizing [v2] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 15:48:57 GMT, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that improves forwarding pointer removal handling by adding thread sizing to the work gang? >> >> Since work is distributed per region at the moment, it does not make sense to start more threads than regions. >> >> This can particularly help performance with object/region pinning. >> >> Testing: tier1-3, using self-verification code >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: > > - Added comment > - iwalulya lkorinth reviews lgtm! ------------- Marked as reviewed by iwalulya (Committer). PR: https://git.openjdk.java.net/jdk/pull/3585 From ayang at openjdk.java.net Thu Apr 22 11:42:43 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Thu, 22 Apr 2021 11:42:43 GMT Subject: RFR: 8265681: G1: general cleanup for G1FullGCHeapRegionAttr Message-ID: <21DRpur7yybPfLDRtSXiTZhLLG_j0EuDxEWJq3zn-pw=.7a0a7a32-b1ea-4bb1-8aba-5b2ed121b51f@github.com> General cleanups of `G1FullGCHeapRegionAttr`, its APIs and usage. Regarding the naming, `SkipCompaction` is another option, but I went with the current version for the consistent `ing` suffix across all valid states.. ------------- Commit messages: - api Changes: https://git.openjdk.java.net/jdk/pull/3621/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3621&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265681 Stats: 68 lines in 10 files changed: 20 ins; 2 del; 46 mod Patch: https://git.openjdk.java.net/jdk/pull/3621.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3621/head:pull/3621 PR: https://git.openjdk.java.net/jdk/pull/3621 From lkorinth at openjdk.java.net Thu Apr 22 12:47:23 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Thu, 22 Apr 2021 12:47:23 GMT Subject: RFR: 8265461: G1: Forwarding pointer removal thread sizing [v2] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 15:48:57 GMT, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that improves forwarding pointer removal handling by adding thread sizing to the work gang? >> >> Since work is distributed per region at the moment, it does not make sense to start more threads than regions. >> >> This can particularly help performance with object/region pinning. >> >> Testing: tier1-3, using self-verification code >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: > > - Added comment > - iwalulya lkorinth reviews Marked as reviewed by lkorinth (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3585 From tschatzl at openjdk.java.net Thu Apr 22 13:09:24 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 22 Apr 2021 13:09:24 GMT Subject: RFR: 8265461: G1: Forwarding pointer removal thread sizing [v2] In-Reply-To: References: Message-ID: On Thu, 22 Apr 2021 12:44:52 GMT, Leo Korinth wrote: >> Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: >> >> - Added comment >> - iwalulya lkorinth reviews > > Marked as reviewed by lkorinth (Committer). Thanks @lkorinth @walulyai for your reviews. ------------- PR: https://git.openjdk.java.net/jdk/pull/3585 From sjohanss at openjdk.java.net Thu Apr 22 15:17:25 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Thu, 22 Apr 2021 15:17:25 GMT Subject: RFR: 8265461: G1: Forwarding pointer removal thread sizing [v2] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 15:48:57 GMT, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that improves forwarding pointer removal handling by adding thread sizing to the work gang? >> >> Since work is distributed per region at the moment, it does not make sense to start more threads than regions. >> >> This can particularly help performance with object/region pinning. >> >> Testing: tier1-3, using self-verification code >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: > > - Added comment > - iwalulya lkorinth reviews src/hotspot/share/gc/g1/heapRegion.cpp line 142: > 140: if (clear_space) clear(SpaceDecorator::Mangle); > 141: > 142: Atomic::store(&_evacuation_failed, false); Can more than one thread attempt this for a given region? I wonder if there really is a need for using `Atomic::store`, but anyhow I think it would fit better to use `reset_evacuation_failed()`. ------------- PR: https://git.openjdk.java.net/jdk/pull/3585 From sjohanss at openjdk.java.net Thu Apr 22 15:24:23 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Thu, 22 Apr 2021 15:24:23 GMT Subject: RFR: 8265461: G1: Forwarding pointer removal thread sizing [v2] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 15:48:57 GMT, Thomas Schatzl wrote: >> Hi all, >> >> can I have reviews for this change that improves forwarding pointer removal handling by adding thread sizing to the work gang? >> >> Since work is distributed per region at the moment, it does not make sense to start more threads than regions. >> >> This can particularly help performance with object/region pinning. >> >> Testing: tier1-3, using self-verification code >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: > > - Added comment > - iwalulya lkorinth reviews Looks good, just a couple of small comment you can deal with if you like. src/hotspot/share/gc/g1/g1CollectedHeap.cpp line 3665: > 3663: > 3664: _expand_heap_after_alloc_failure = true; > 3665: Atomic::store(&_num_regions_failed_evacuation, 0u); This could just be `_num_regions_failed_evacuation = 0;` right? Or am I missing something? It sticks out a bit and making one wonder if there is someone else change this value as well. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3585 From rkennke at openjdk.java.net Thu Apr 22 15:24:34 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 22 Apr 2021 15:24:34 GMT Subject: RFR: 8265759: Shenandoah: Avoid race for referent in assert Message-ID: There is an assert is ShenandoahReferenceProcessor::drop() which is racy: when referent is concurrently cleaned it may enter mark-bit-map with NULL even though we just checked that the referent is not NULL. Testing: - [x] hotspot_gc_shenandoah ------------- Commit messages: - 8265759: Shenandoah: Avoid race for referent in assert Changes: https://git.openjdk.java.net/jdk/pull/3632/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3632&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265759 Stats: 5 lines in 1 file changed: 3 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3632.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3632/head:pull/3632 PR: https://git.openjdk.java.net/jdk/pull/3632 From gziemski at openjdk.java.net Thu Apr 22 16:00:33 2021 From: gziemski at openjdk.java.net (Gerard Ziemski) Date: Thu, 22 Apr 2021 16:00:33 GMT Subject: RFR: 8265702: ZGC on macOS/aarch64 [v2] In-Reply-To: <7qkG6INxywBIAyzPqO812c7Pn7IUp5gZzfcjl27lE2E=.73fa7b17-7f51-4cdc-b621-e44d3b619740@github.com> References: <7qkG6INxywBIAyzPqO812c7Pn7IUp5gZzfcjl27lE2E=.73fa7b17-7f51-4cdc-b621-e44d3b619740@github.com> Message-ID: On Thu, 22 Apr 2021 06:43:45 GMT, Per Liden wrote: >> This patch enables ZGC on macOS/aarch64. It does three things: >> 1) Enables building of ZGC on this platform. >> 2) Adds `os::processor_id()`, which for now always returns 0. >> 3) Fixes a WX issue in `OptoRuntime::handle_exception_C()`, where the stackwater mark might unnecessarily process a frame when the thread is in WXExec mode. In this case, the we're not touching any oops, so we don't need to process any frames. >> >> Testing: Passes the same tests as macOS/x86_64 (with the exception of pre-existing issues unrelated to ZGC). > > Per Liden has updated the pull request incrementally with one additional commit since the last revision: > > Add comment to #else Marked as reviewed by gziemski (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3609 From shade at openjdk.java.net Thu Apr 22 16:39:23 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Thu, 22 Apr 2021 16:39:23 GMT Subject: RFR: 8265759: Shenandoah: Avoid race for referent in assert In-Reply-To: References: Message-ID: On Thu, 22 Apr 2021 15:17:25 GMT, Roman Kennke wrote: > There is an assert is ShenandoahReferenceProcessor::drop() which is racy: when referent is concurrently cleaned it may enter mark-bit-map with NULL even though we just checked that the referent is not NULL. > > Testing: > - [x] hotspot_gc_shenandoah Looks good! Minor nit: src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 392: > 390: oop referent = reference_referent(reference); > 391: assert(referent == NULL || > 392: ShenandoahHeap::heap()->marking_context()->is_marked(referent), "only drop references with alive referents"); I would prefer it to be line-broken like this: assert(referent == NULL || ShenandoahHeap::heap()->marking_context()->is_marked(referent), "only drop references with alive referents"); ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3632 From tschatzl at openjdk.java.net Thu Apr 22 18:22:28 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 22 Apr 2021 18:22:28 GMT Subject: RFR: 8265461: G1: Forwarding pointer removal thread sizing [v2] In-Reply-To: References: Message-ID: On Thu, 22 Apr 2021 15:21:17 GMT, Stefan Johansson wrote: >> Thomas Schatzl has updated the pull request incrementally with two additional commits since the last revision: >> >> - Added comment >> - iwalulya lkorinth reviews > > Looks good, just a couple of small comment you can deal with if you like. thanks for your review, @kstefanj ------------- PR: https://git.openjdk.java.net/jdk/pull/3585 From tschatzl at openjdk.java.net Thu Apr 22 18:22:29 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 22 Apr 2021 18:22:29 GMT Subject: Integrated: 8265461: G1: Forwarding pointer removal thread sizing In-Reply-To: References: Message-ID: <2CiWLgNeTmQu9Rfh7VokBbkmgyTEjCe72v8F2GfFA2o=.87745653-ec67-4c54-a1bf-9aaa91ed8af1@github.com> On Tue, 20 Apr 2021 09:34:45 GMT, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that improves forwarding pointer removal handling by adding thread sizing to the work gang? > > Since work is distributed per region at the moment, it does not make sense to start more threads than regions. > > This can particularly help performance with object/region pinning. > > Testing: tier1-3, using self-verification code > > Thanks, > Thomas This pull request has now been integrated. Changeset: b930bb1a Author: Thomas Schatzl URL: https://git.openjdk.java.net/jdk/commit/b930bb1a Stats: 81 lines in 9 files changed: 52 ins; 7 del; 22 mod 8265461: G1: Forwarding pointer removal thread sizing Reviewed-by: iwalulya, lkorinth, sjohanss ------------- PR: https://git.openjdk.java.net/jdk/pull/3585 From zgu at openjdk.java.net Thu Apr 22 18:36:25 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Thu, 22 Apr 2021 18:36:25 GMT Subject: RFR: 8265759: Shenandoah: Avoid race for referent in assert In-Reply-To: References: Message-ID: <0KmTMHHuk4LL2JNO_JqNJBeurroj1Ha-4BinnPxgRkQ=.82e6fee7-273b-42c1-8880-73848f8c43af@github.com> On Thu, 22 Apr 2021 15:17:25 GMT, Roman Kennke wrote: > There is an assert is ShenandoahReferenceProcessor::drop() which is racy: when referent is concurrently cleaned it may enter mark-bit-map with NULL even though we just checked that the referent is not NULL. > > Testing: > - [x] hotspot_gc_shenandoah Marked as reviewed by zgu (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3632 From rkennke at openjdk.java.net Thu Apr 22 19:19:48 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Thu, 22 Apr 2021 19:19:48 GMT Subject: RFR: 8265759: Shenandoah: Avoid race for referent in assert [v2] In-Reply-To: References: Message-ID: > There is an assert is ShenandoahReferenceProcessor::drop() which is racy: when referent is concurrently cleaned it may enter mark-bit-map with NULL even though we just checked that the referent is not NULL. > > Testing: > - [x] hotspot_gc_shenandoah Roman Kennke has updated the pull request incrementally with one additional commit since the last revision: Fix formatting ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3632/files - new: https://git.openjdk.java.net/jdk/pull/3632/files/73a36ca4..18b8a6f5 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3632&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3632&range=00-01 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3632.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3632/head:pull/3632 PR: https://git.openjdk.java.net/jdk/pull/3632 From dcubed at openjdk.java.net Thu Apr 22 20:41:42 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 22 Apr 2021 20:41:42 GMT Subject: RFR: 8265786: ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC Message-ID: <93s66lybQInA-m-ux9P5Q0fFa2z3V1zczdxramwwGYI=.c072d88d-d236-44a5-9eff-04f8251aaac0@github.com> A trivial fix to ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC. ------------- Commit messages: - 8265786: ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC Changes: https://git.openjdk.java.net/jdk/pull/3641/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3641&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265786 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3641.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3641/head:pull/3641 PR: https://git.openjdk.java.net/jdk/pull/3641 From darcy at openjdk.java.net Thu Apr 22 20:41:42 2021 From: darcy at openjdk.java.net (Joe Darcy) Date: Thu, 22 Apr 2021 20:41:42 GMT Subject: RFR: 8265786: ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC In-Reply-To: <93s66lybQInA-m-ux9P5Q0fFa2z3V1zczdxramwwGYI=.c072d88d-d236-44a5-9eff-04f8251aaac0@github.com> References: <93s66lybQInA-m-ux9P5Q0fFa2z3V1zczdxramwwGYI=.c072d88d-d236-44a5-9eff-04f8251aaac0@github.com> Message-ID: <5xqsUIn2UVr-3jNeRsyY9-ccGqVTc1q_HutE4--55NM=.338b58dd-4725-46b0-bb0c-862910e660be@github.com> On Thu, 22 Apr 2021 20:32:33 GMT, Daniel D. Daugherty wrote: > A trivial fix to ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC. Marked as reviewed by darcy (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3641 From dcubed at openjdk.java.net Thu Apr 22 20:48:25 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 22 Apr 2021 20:48:25 GMT Subject: Integrated: 8265786: ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC In-Reply-To: <93s66lybQInA-m-ux9P5Q0fFa2z3V1zczdxramwwGYI=.c072d88d-d236-44a5-9eff-04f8251aaac0@github.com> References: <93s66lybQInA-m-ux9P5Q0fFa2z3V1zczdxramwwGYI=.c072d88d-d236-44a5-9eff-04f8251aaac0@github.com> Message-ID: <6M9SoNqyIVD--eY8uEVNetiDuW2-UzQr7kH9hUIyPIM=.b9be38d6-7302-4423-a6a6-ee3d7be15687@github.com> On Thu, 22 Apr 2021 20:32:33 GMT, Daniel D. Daugherty wrote: > A trivial fix to ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC. This pull request has now been integrated. Changeset: e81baead Author: Daniel D. Daugherty URL: https://git.openjdk.java.net/jdk/commit/e81baead Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod 8265786: ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC Reviewed-by: darcy ------------- PR: https://git.openjdk.java.net/jdk/pull/3641 From dcubed at openjdk.java.net Thu Apr 22 20:48:24 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 22 Apr 2021 20:48:24 GMT Subject: RFR: 8265786: ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC In-Reply-To: <5xqsUIn2UVr-3jNeRsyY9-ccGqVTc1q_HutE4--55NM=.338b58dd-4725-46b0-bb0c-862910e660be@github.com> References: <93s66lybQInA-m-ux9P5Q0fFa2z3V1zczdxramwwGYI=.c072d88d-d236-44a5-9eff-04f8251aaac0@github.com> <5xqsUIn2UVr-3jNeRsyY9-ccGqVTc1q_HutE4--55NM=.338b58dd-4725-46b0-bb0c-862910e660be@github.com> Message-ID: On Thu, 22 Apr 2021 20:34:48 GMT, Joe Darcy wrote: >> A trivial fix to ProblemList serviceability/sa/sadebugd/DisableRegistryTest.java on ZGC. > > Marked as reviewed by darcy (Reviewer). @jddarcy - Thanks for the fast review! ------------- PR: https://git.openjdk.java.net/jdk/pull/3641 From adityam at openjdk.java.net Thu Apr 22 21:40:39 2021 From: adityam at openjdk.java.net (Aditya Mandaleeka) Date: Thu, 22 Apr 2021 21:40:39 GMT Subject: RFR: 8257774: G1: Trigger collect when free region count drops below threshold to prevent evacuation failures [v4] In-Reply-To: <5Ro0P4NPaUuG-LPpgqHY5S_kh_EVpwRgxqc5jTCano4=.afc3f2a4-dc9e-46c1-ac84-298cf143fa41@github.com> References: <5Ro0P4NPaUuG-LPpgqHY5S_kh_EVpwRgxqc5jTCano4=.afc3f2a4-dc9e-46c1-ac84-298cf143fa41@github.com> Message-ID: > _This PR picks up from [this previous PR](https://github.com/openjdk/jdk/pull/1650) by @charliegracie._ > > I won't repeat the full description from the prior PR, but the general idea is to add the notion of a "proactive GC" to G1 which gets triggered in the slow allocation path if the number of free regions drops below the amount that would be required to complete a GC if it happened at that moment. The threshold is based on the survival rates from eden and survivor spaces along with the space required for tenured space evacuations. > > There are a couple of outstanding issues/questions known: > - _Interaction with GCLocker_: In the case where we determine that a proactive GC is required and GC locker is active, we don't allow the young gen to expand (instead threads will stall). @tschatzl raised this and suggested that it should be discussed as part of the review. > - _Disable proactive GC heuristic during initialization_: I ran into an issue in testing where code in the universe initialization codepath was tripping the proactive GC heuristic, leading to a GC being triggered before the VM has finished initialization. I need to find a good way to prevent this from happening. There may already be a mechanism for this, but I couldn't find one so I added a temporary placeholder (`zzinit_complete`) just to unblock testing. Aditya Mandaleeka has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: - Remove temp init variable, update code after rebase. - Refactor allocation funcs, removing attempt_lock_free_first. - Fix format specifiers. - Fix a few warnings as well as a Windows build error. - Fix whitespace and comment. - Further refactoring/changes. - Address feedback and make other changes. - Improve G1GC accounting for humongous objects If there is a burst of humongous allocations they can consume all of the free regions left in the heap. If there are no free regions for to-space a collect will fail to evacuate all objects. This makes the GC very slow. Since, humongous regions can be collected in G1GC young collects force a young collect before the number of free regions becomes less than the number the GC is likely to require. Fix TestGCLogMessages.java so it still causes Evacuation failures. Signed-off-by: Charlie Gracie ------------- Changes: https://git.openjdk.java.net/jdk/pull/3143/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3143&range=03 Stats: 207 lines in 11 files changed: 167 ins; 13 del; 27 mod Patch: https://git.openjdk.java.net/jdk/pull/3143.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3143/head:pull/3143 PR: https://git.openjdk.java.net/jdk/pull/3143 From adityam at openjdk.java.net Thu Apr 22 21:59:33 2021 From: adityam at openjdk.java.net (Aditya Mandaleeka) Date: Thu, 22 Apr 2021 21:59:33 GMT Subject: RFR: 8257774: G1: Trigger collect when free region count drops below threshold to prevent evacuation failures [v4] In-Reply-To: References: <5Ro0P4NPaUuG-LPpgqHY5S_kh_EVpwRgxqc5jTCano4=.afc3f2a4-dc9e-46c1-ac84-298cf143fa41@github.com> Message-ID: <_Z9CQ3H2RDvDgBjsDEtx_ykIxUCBOFWSfdO87JZ_cqE=.bf46b8dd-fc03-409b-8d1c-b865c6cc3a2e@github.com> On Thu, 22 Apr 2021 21:40:39 GMT, Aditya Mandaleeka wrote: >> _This PR picks up from [this previous PR](https://github.com/openjdk/jdk/pull/1650) by @charliegracie._ >> >> I won't repeat the full description from the prior PR, but the general idea is to add the notion of a "proactive GC" to G1 which gets triggered in the slow allocation path if the number of free regions drops below the amount that would be required to complete a GC if it happened at that moment. The threshold is based on the survival rates from eden and survivor spaces along with the space required for tenured space evacuations. >> >> There are a couple of outstanding issues/questions known: >> - _Interaction with GCLocker_: In the case where we determine that a proactive GC is required and GC locker is active, we don't allow the young gen to expand (instead threads will stall). @tschatzl raised this and suggested that it should be discussed as part of the review. >> - _Disable proactive GC heuristic during initialization_: I ran into an issue in testing where code in the universe initialization codepath was tripping the proactive GC heuristic, leading to a GC being triggered before the VM has finished initialization. I need to find a good way to prevent this from happening. There may already be a mechanism for this, but I couldn't find one so I added a temporary placeholder (`zzinit_complete`) just to unblock testing. > > Aditya Mandaleeka has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: > > - Remove temp init variable, update code after rebase. > - Refactor allocation funcs, removing attempt_lock_free_first. > - Fix format specifiers. > - Fix a few warnings as well as a Windows build error. > - Fix whitespace and comment. > - Further refactoring/changes. > - Address feedback and make other changes. > - Improve G1GC accounting for humongous objects > > If there is a burst of humongous allocations they can > consume all of the free regions left in the heap. If > there are no free regions for to-space a collect will > fail to evacuate all objects. This makes the GC very > slow. Since, humongous regions can be collected in > G1GC young collects force a young collect before the > number of free regions becomes less than the number > the GC is likely to require. > > Fix TestGCLogMessages.java so it still causes Evacuation > failures. > > Signed-off-by: Charlie Gracie Thanks again for the review. I agree that the proposed refactor of the alloc functions looks good, and I've applied it. Functional testing of the latest code is ongoing (doing hs_gc and vmTestbase_vm_gc in release and fastdebug on x64 Linux) but so far everything that's run has passed. What are your thoughts on further steps involved before getting this change merged? For instance, is there value/interest in putting the proactive GCs behind an -XX: option? Every such flag adds complexity and surface area for testing, so I'm somewhat hesitant but I'm interested to hear others' opinions on this... src/hotspot/share/gc/g1/g1Policy.cpp line 786: > 784: update_rs_length_prediction(); > 785: > 786: // Is this the right place? Should it be in the below? Can a reviewer confirm this so that I can remove the comment? ------------- PR: https://git.openjdk.java.net/jdk/pull/3143 From ayang at openjdk.java.net Fri Apr 23 07:52:33 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Fri, 23 Apr 2021 07:52:33 GMT Subject: RFR: 8252089: psParallelCompact internal counters are global Message-ID: <7FydXMy7gkW3WynlKi3a83NeGWiYUZWq4NscWblAjy0=.f564b51f-84de-4cab-9dcc-9d403b26cfe6@github.com> Trivial change of removing debug-only counters. ------------- Commit messages: - counter Changes: https://git.openjdk.java.net/jdk/pull/3648/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3648&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8252089 Stats: 33 lines in 2 files changed: 0 ins; 33 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3648.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3648/head:pull/3648 PR: https://git.openjdk.java.net/jdk/pull/3648 From rkennke at openjdk.java.net Fri Apr 23 08:04:26 2021 From: rkennke at openjdk.java.net (Roman Kennke) Date: Fri, 23 Apr 2021 08:04:26 GMT Subject: Integrated: 8265759: Shenandoah: Avoid race for referent in assert In-Reply-To: References: Message-ID: <4d05qb-uXi2pvxRf6fawk5ReEwXIQevn6F-zt2tH8Zw=.c1c630c7-6b33-4525-8c08-3c834217e11c@github.com> On Thu, 22 Apr 2021 15:17:25 GMT, Roman Kennke wrote: > There is an assert is ShenandoahReferenceProcessor::drop() which is racy: when referent is concurrently cleaned it may enter mark-bit-map with NULL even though we just checked that the referent is not NULL. > > Testing: > - [x] hotspot_gc_shenandoah This pull request has now been integrated. Changeset: 191f1fc4 Author: Roman Kennke URL: https://git.openjdk.java.net/jdk/commit/191f1fc4 Stats: 5 lines in 1 file changed: 3 ins; 0 del; 2 mod 8265759: Shenandoah: Avoid race for referent in assert Reviewed-by: shade, zgu ------------- PR: https://git.openjdk.java.net/jdk/pull/3632 From sjohanss at openjdk.java.net Fri Apr 23 09:28:25 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Fri, 23 Apr 2021 09:28:25 GMT Subject: RFR: 8252089: psParallelCompact internal counters are global In-Reply-To: <7FydXMy7gkW3WynlKi3a83NeGWiYUZWq4NscWblAjy0=.f564b51f-84de-4cab-9dcc-9d403b26cfe6@github.com> References: <7FydXMy7gkW3WynlKi3a83NeGWiYUZWq4NscWblAjy0=.f564b51f-84de-4cab-9dcc-9d403b26cfe6@github.com> Message-ID: On Fri, 23 Apr 2021 07:45:41 GMT, Albert Mingkun Yang wrote: > Trivial change of removing debug-only counters. Looks good. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3648 From tschatzl at openjdk.java.net Fri Apr 23 10:01:24 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 23 Apr 2021 10:01:24 GMT Subject: RFR: 8252089: psParallelCompact internal counters are global In-Reply-To: <7FydXMy7gkW3WynlKi3a83NeGWiYUZWq4NscWblAjy0=.f564b51f-84de-4cab-9dcc-9d403b26cfe6@github.com> References: <7FydXMy7gkW3WynlKi3a83NeGWiYUZWq4NscWblAjy0=.f564b51f-84de-4cab-9dcc-9d403b26cfe6@github.com> Message-ID: On Fri, 23 Apr 2021 07:45:41 GMT, Albert Mingkun Yang wrote: > Trivial change of removing debug-only counters. Lgtm. Please change the title of the CR to reflect what is actually done (e.g. "Remove psParallelCompact internal debug counters") before integration. When I first looked at the title I thought I thought this was about making some counters local (assuming that the problem is that these counters are global as the title indicates to me), not removing them. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3648 From tschatzl at openjdk.java.net Fri Apr 23 10:09:25 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 23 Apr 2021 10:09:25 GMT Subject: RFR: 8265681: G1: general cleanup for G1FullGCHeapRegionAttr In-Reply-To: <21DRpur7yybPfLDRtSXiTZhLLG_j0EuDxEWJq3zn-pw=.7a0a7a32-b1ea-4bb1-8aba-5b2ed121b51f@github.com> References: <21DRpur7yybPfLDRtSXiTZhLLG_j0EuDxEWJq3zn-pw=.7a0a7a32-b1ea-4bb1-8aba-5b2ed121b51f@github.com> Message-ID: <8uTgCIUIJaVtGuI4UXkGPgQOWfj4K-IcMxWMOzJeEEY=.4850eff0-4c00-4f73-a94d-210da687de28@github.com> On Thu, 22 Apr 2021 11:33:20 GMT, Albert Mingkun Yang wrote: > General cleanups of `G1FullGCHeapRegionAttr`, its APIs and usage. > > Regarding the naming, `SkipCompaction` is another option, but I went with the current version for the consistent `ing` suffix across all valid states.. Ship it :) ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3621 From tschatzl at openjdk.java.net Fri Apr 23 12:01:50 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 23 Apr 2021 12:01:50 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask Message-ID: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> JDK-8214237 is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality just to split up JDK-8214237 a bit. Here is the full change that uses this API to implement JDK-8214237 (which I'll split up a bit before) to see how it will be used (in G1YoungGCPostEvacuateTasks.hpp. Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. Thanks, Thomas ------------- Commit messages: - Whitespace fixes - Initial commit Changes: https://git.openjdk.java.net/jdk/pull/3653/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265842 Stats: 386 lines in 3 files changed: 386 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3653.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3653/head:pull/3653 PR: https://git.openjdk.java.net/jdk/pull/3653 From sjohanss at openjdk.java.net Fri Apr 23 13:33:28 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Fri, 23 Apr 2021 13:33:28 GMT Subject: RFR: 8265681: G1: general cleanup for G1FullGCHeapRegionAttr In-Reply-To: <21DRpur7yybPfLDRtSXiTZhLLG_j0EuDxEWJq3zn-pw=.7a0a7a32-b1ea-4bb1-8aba-5b2ed121b51f@github.com> References: <21DRpur7yybPfLDRtSXiTZhLLG_j0EuDxEWJq3zn-pw=.7a0a7a32-b1ea-4bb1-8aba-5b2ed121b51f@github.com> Message-ID: On Thu, 22 Apr 2021 11:33:20 GMT, Albert Mingkun Yang wrote: > General cleanups of `G1FullGCHeapRegionAttr`, its APIs and usage. > > Regarding the naming, `SkipCompaction` is another option, but I went with the current version for the consistent `ing` suffix across all valid states.. Very nice cleanup. Looks great! ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3621 From tschatzl at openjdk.java.net Fri Apr 23 13:36:23 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 23 Apr 2021 13:36:23 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask In-Reply-To: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: On Fri, 23 Apr 2021 11:54:49 GMT, Thomas Schatzl wrote: > [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). > > To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. > > This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. > > [Here](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). > > Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. > > Thanks, > Thomas There is one question I would like to have discussed: to make the code more compact the API uses constructor and destructor for setup/teardown-like methods. This makes use a bit more clumsy that expected, see the `G1CollectedHeap::post_evacuate_cleanup_1/2` methods: void G1CollectedHeap::post_evacuate_cleanup_1(G1ParScanThreadStateSet* per_thread_states, G1RedirtyCardsQueueSet* rdcqs) { Ticks start = Ticks::now(); { G1PostEvacuateCollectionSetCleanupTask1 cl(per_thread_states, rdcqs); uint num_threads = MAX2(1u, MIN2(cl.num_busy_workers(), workers()->active_workers())); cl.set_max_workers(num_threads); workers()->run_task(&cl, num_threads); } [...] } i.e. since the constructor is used for setup, there is some small code duplication necessary to time execution of such a `G1BatchedGangTask`. @walulyai suggested (similar to other hotspot code) to introduce a `setup` method, but no `teardown`. There is no reason for me to not to, any concerns or other ideas? ------------- PR: https://git.openjdk.java.net/jdk/pull/3653 From iwalulya at openjdk.java.net Fri Apr 23 15:54:29 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Fri, 23 Apr 2021 15:54:29 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask In-Reply-To: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: On Fri, 23 Apr 2021 11:54:49 GMT, Thomas Schatzl wrote: > [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). > > To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. > > This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. > > [Here](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). > > Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. > > Thanks, > Thomas src/hotspot/share/gc/g1/g1BatchedGangTask.cpp line 46: > 44: task = Atomic::load(&_num_serial_tasks_done); > 45: if (task < _serial_tasks.length()) { > 46: task = Atomic::add(&_num_serial_tasks_done, 1) - 1; I think `Atomic::fetch_and_add` is more appropriate in this case Suggestion: task = Atomic::fetch_and_add(&_num_serial_tasks_done, 1); test/hotspot/gtest/gc/g1/test_g1BatchedGangTask.cpp line 147: > 145: G1TestBatchedGangTask() : G1BatchedGangTask("Batched Gang Test Task", NULL) { > 146: _serial_tasks.push(new G1SerialTestSubTask()); > 147: _parallel_tasks.push(new G1ParallelTestSubTask()); I suggest we add methods for adding serial and parallel tasks to the `G1BatchedGangTask`interface. ------------- PR: https://git.openjdk.java.net/jdk/pull/3653 From ayang at openjdk.java.net Fri Apr 23 20:11:35 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Fri, 23 Apr 2021 20:11:35 GMT Subject: RFR: 8234446: Post-CMS workgroup hierarchy cleanup Message-ID: Collapse workgroup hierarchy and some general cleanup. Tested; hotspot_gc ------------- Commit messages: - rename GangWorker - AbstractGangWorker - WorkGang - cleanup Changes: https://git.openjdk.java.net/jdk/pull/3663/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3663&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8234446 Stats: 291 lines in 5 files changed: 86 ins; 165 del; 40 mod Patch: https://git.openjdk.java.net/jdk/pull/3663.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3663/head:pull/3663 PR: https://git.openjdk.java.net/jdk/pull/3663 From github.com+168222+mgkwill at openjdk.java.net Fri Apr 23 22:24:38 2021 From: github.com+168222+mgkwill at openjdk.java.net (Marcus G K Williams) Date: Fri, 23 Apr 2021 22:24:38 GMT Subject: RFR: 8256155: Allow multiple large page sizes to be used on Linux [v29] In-Reply-To: References: Message-ID: > Change the meaning of LargePageSizeInBytes to be the maximum large page size the JVM may use (not the only one). A default value of zero will mean to allow the JVM use large page sizes up to the system's default large page size. Marcus G K Williams has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 48 commits: - Set LargePageSizeInBytes to largepage upper limit Signed-off-by: Marcus G K Williams - Merge branch 'master' into update_hlp - Fix merge Signed-off-by: Marcus G K Williams - Merge branch 'master' into update_hlp - Merge branch 'master' into update_hlp - Rebase on pull/3073 Signed-off-by: Marcus G K Williams - Merge branch 'pull/3073' into update_hlp - Thomas review. Changed commit_memory_special to return bool to signal if the request succeeded or not. - Self review. Update helper name to better match commit_memory_special(). - Marcus review. Updated comments. - ... and 38 more: https://git.openjdk.java.net/jdk/compare/5aab1609...6f063309 ------------- Changes: https://git.openjdk.java.net/jdk/pull/1153/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=1153&range=28 Stats: 138 lines in 3 files changed: 68 ins; 47 del; 23 mod Patch: https://git.openjdk.java.net/jdk/pull/1153.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1153/head:pull/1153 PR: https://git.openjdk.java.net/jdk/pull/1153 From github.com+168222+mgkwill at openjdk.java.net Fri Apr 23 22:39:27 2021 From: github.com+168222+mgkwill at openjdk.java.net (Marcus G K Williams) Date: Fri, 23 Apr 2021 22:39:27 GMT Subject: RFR: 8256155: Allow multiple large page sizes to be used on Linux [v27] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 18:00:52 GMT, Marcus G K Williams wrote: > an easy approach should be to just remove all large page sizes larger than the decided maximum from _page_sizes. It seemed easier to forgo adding large page sizes if they were less than `LargePageSizeInBytes` and `LargePageSizeInBytes != 0` or default. Please let me know if the solution seems appropriate. > The warning should go away since it is documented to mean use the default. Done. Feedback there would also be helpful. I'm seeing one failure with the TestTracePageSizes test that I'm yet to figure out. On a run with `run main/othervm -XX:+AlwaysPreTouch -Xlog:pagesize:ps-%p.log -XX:+UseSerialGC -XX:+UseTransparentHugePages TestTracePageSizes` ============================== Test summary ============================== TEST TOTAL PASS FAIL ERROR jtreg:test/hotspot/jtreg/runtime/os/TestTracePageSizes.java >> 6 5 1 0 << ============================== TEST FAILURE ... --- >From logfile: [0.004s][info][pagesize] Heap: min=8M max=15452M base=0x000000043a400000 page_size=2M size=15452M From smaps: [43a400000, 440000000) pageSize=4KB isTHP=false isHUGETLB=false Failure: 4 != 2048 STDERR: java.lang.AssertionError: Page sizes mismatch: 4 != 2048 at TestTracePageSizes.main(TestTracePageSizes.java:202) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) at java.base/java.lang.Thread.run(Thread.java:831) JavaTest Message: Test threw exception: java.lang.AssertionError: Page sizes mismatch: 4 != 2048 JavaTest Message: shutting down test ... reason: User specified action: run main/othervm -XX:+AlwaysPreTouch -Xlog:pagesize:ps-%p.log -XX:+UseSerialGC -XX:+UseTransparentHugePages TestTracePageSizes Mode: othervm [/othervm specified] elapsed time (seconds): 0.803 configuration: STDOUT: Added range: [43a400000, 440000000) pageSize=4KB isTHP=false isHUGETLB=false Added range: [440000000, 480000000) pageSize=4KB isTHP=true isHUGETLB=false Added range: [480000000, 57c2a0000) pageSize=4KB isTHP=false isHUGETLB=false Two guesses so far: It seems like a possible issue with `ReservedSpace::actual_reserved_page_size` returning the wrong page size. Or could this base page address be 4k padding for THP largepage? ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From ayang at openjdk.java.net Sat Apr 24 09:49:34 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Sat, 24 Apr 2021 09:49:34 GMT Subject: RFR: 8231672: Simplify the reference processing parallelization framework [v5] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 11:42:04 GMT, Leo Korinth wrote: >> With the change of parallel gc to use the same workgang implementation that other gcs uses, it is possible to remove the abstraction used to hide different worker thread management implementations (TaskExecutor). >> >> At the same time as removing the TaskExecutor, also remove part of the special casing (mostly code duplication), for single threaded execution. >> >> I consider the new method `prepare_run_task` that modifies the state of the context a step backwards from what was. However, I think removing the Executor with its proxy-tasks and removing separate code paths for serial execution makes the change worth this step back. We also removes ~270 lines of code. >> >> Some comments: >> 1) I use references in some places where they can replace pointers. I could go much further here, but I did not want to bloat the pull request, maybe later change all closures to be references as well? Should I create enhancements for this? >> >> 2) I added an enum class ThreadModel instead of using a boolean, this could also be used in many more places. I dislike sending lots of bools with a comment like `true /* _is_mt */`. It also adds type safety if a method takes many bools. However, with this limited change, and not many hard-coded bools, it feels a bit overkill and I am willing to remove the enum, what do you think? >> >> Testing: >> hotspot_gc and tier 1-3 has passed earlier versions before minor cleanups. I intend to re-run tests after review changes. > > Leo Korinth has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits: > > - make RefProcTask a StackObj > - Merge branch 'master' into _8231672 > - using proxies instead of ClosureContexts > - remove maybe prefix, change in one place to nullable > - Merge branch 'master' into _8231672 > - Changes after first review > - 8231672: Simplify the reference processing parallelization framework Changes requested by ayang (Committer). src/hotspot/share/gc/parallel/psCompactionManager.hpp line 42: > 40: > 41: class ParCompactionManager : public CHeapObj { > 42: friend class CompactionWithStealingTask; `CompactionWithStealingTask` and `UpdateAndFillClosure` are dead code. src/hotspot/share/gc/shared/referenceProcessor.cpp line 224: > 222: > 223: { > 224: RefProcTotalPhaseTimesTracker tt(RefPhase1, &phase_times, this); `RefProcTotalPhaseTimesTracker` doesn't really use `this`, right? src/hotspot/share/gc/shared/referenceProcessor.cpp line 786: > 784: > 785: proxy_task.prepare_run_task(task, num_queues(), processing_is_mt() ? RefProcThreadModel::Multi : RefProcThreadModel::Single, marks_oops_alive); > 786: if (gang != NULL && processing_is_mt()) { Since `gang` is only used in thin branch, moving it inside `if` might be better. Besides, the assertion on L779 could be replaced by the code below, IMO. if (processing_is_mt()) { WorkGang* gang = Universe::heap()->safepoint_workers(); assert(gang != nullptr, "..."); ... } src/hotspot/share/gc/shared/referenceProcessor.hpp line 591: > 589: enum class RefProcThreadModel { Multi, Single }; > 590: > 591: class RefProcTask : StackObj { Maybe put some comments here, explaining what `RefProcTask` and `RefProcProxyTask` are, their interaction/relation. src/hotspot/share/gc/shared/referenceProcessor.hpp line 631: > 629: uint index(uint id) const { > 630: return (_tm == RefProcThreadModel::Single) ? 0 : id; > 631: } This method is only used by G1; I wonder if it's possible to not include it in the API of `RefProcProxyTask`. ------------- PR: https://git.openjdk.java.net/jdk/pull/2782 From stefan.johansson at oracle.com Sat Apr 24 12:31:32 2021 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Sat, 24 Apr 2021 14:31:32 +0200 Subject: Unconditional messages on large page reservation errors In-Reply-To: References: Message-ID: <77a081c6-f736-0f0c-d700-99ee94fb69ee@oracle.com> Hi Thomas, Sorry for the late reply. On 2021-04-17 06:11, Thomas St?fe wrote: > Hi all, > > In os::reserve_memory_special, we print unconditional warnings to stderr in > case large page reservation fails. Unconditional printouts like these can > interfere with parsers parsing VM output, and can accumulate and cause high > memory footprint (see e.g. https://bugs.openjdk.java.net/browse/JDK-8265332 > ). I see unconditional warnings only for shm-case, in the hugetlbfs-case we use warn_on_commit_special_failure() which only print the warning if large pages was explicitly requested. Still we can only end up here if large pages are enabled and that needs to be done explicitly so the warnings are kind of unconditional. > > Large page reservations may fail at any time os::reserve_memory_special() > function is called, e.g. because the large page pool is temporarily > exhausted. And os::reserve_memory_special() is a general purpose function, > not only used for the heap. Running out of large pages is not fatal, since > the caller can just fall back to normal page allocation. Which is what we > do when reserving the java heap. I think unconditional printouts should > only happen in case of fatal errors, when the VM is about to die. > I don't really agree here since the performance implications of not using large pages are quite big. I think it is fair to issue the warning since in most cases it signals that there is an environment configuration problem. For testing we have the possibility to use -XX:-PrintWarnings and I saw you used that to fix the issue mentioned above. Also, what would the use-case for warning() be if not for printing information in non-fatal but problematic situations (which I think this is). > The unconditional warning probably made sense in the context of reserving > the java heap, if the user explicitly specified UseLargePages. I propose to > change this to either > - if large page allocation for the heap fails, trace with info level > and fall back to small pages. Leave it up to the user to increase UL and > monitor log output to find out about this. This is what we usually do when > system APIs fail. > - continue printing the message with error level, but exit the VM. If > it's serious enough to unconditionally notify the user, it's serious enough > to stop the VM. > > I prefer the former. What do you think? As I said above, I see a value in warning if you don't get what you are requesting. But I know others that think exiting is a better strategy. If I'm not mistaken ZGC won't start if it can't reserve enough large pages. One thing that I would like to change in this are is for these warnings to be converted to use UL. That way we could turn of warnings on a much finer granularity and we wouldn't have to use jio_snprintf to compose messages. Thanks for bringing this up for discussion, StefanJ > > Thanks and best Regards, > > Thomas > From manavjeet18295 at iiitd.ac.in Sat Apr 24 12:59:12 2021 From: manavjeet18295 at iiitd.ac.in (Manavjeet Singh) Date: Sat, 24 Apr 2021 18:29:12 +0530 Subject: Error while using self compiled openJDK14 in eclipse Message-ID: Hi! I compiled OpenJDK-14 on my machine and it is working fine when used from the command line. I want to build a program using it but when trying to add it to Eclipse(v4.18) through Window->Preferences->Installed JREs->Add->StandardVM and giving the path jdk14/build/linux-x86_64-server-release/jdk/. But it is showing the error: "Target is not a JDK root. System library not found". Have I made some mistake while compiling the jdk? Any help would be appreciated. Thanks From lgxbslgx at gmail.com Sat Apr 24 13:10:24 2021 From: lgxbslgx at gmail.com (Guoxiong Li) Date: Sat, 24 Apr 2021 21:10:24 +0800 Subject: Error while using self compiled openJDK14 in eclipse In-Reply-To: References: Message-ID: Hi Manavjeet, I haven't used eclipse for a long time so my idea may be wrong. According to your description, I think you should use the path `jdk14/build/linux-x86_64-server-release*/images/jdk*` instead. --- Best Regards, Guoxiong. On Sat, Apr 24, 2021 at 8:59 PM Manavjeet Singh wrote: > Hi! > I compiled OpenJDK-14 on my machine and it is working fine when used from > the command line. I want to build a program using it but when trying to add > it to Eclipse(v4.18) through Window->Preferences->Installed > JREs->Add->StandardVM and giving the path > jdk14/build/linux-x86_64-server-release/jdk/. > But it is showing the error: "Target is not a JDK root. System library not > found". > Have I made some mistake while compiling the jdk? Any help would be > appreciated. > Thanks > From manavjeet18295 at iiitd.ac.in Sat Apr 24 13:28:39 2021 From: manavjeet18295 at iiitd.ac.in (Manavjeet Singh) Date: Sat, 24 Apr 2021 18:58:39 +0530 Subject: Error while using self compiled openJDK14 in eclipse In-Reply-To: References: Message-ID: Hi Guoxiong, There isn't any images folder under 'linux-x86_64-server-release'. I have only the following files(not bold ones are the folders). bootcycle-spec.gmk build.log.old configure.log hotspot Makefile support buildjdk-spec.gmk buildtools configure.log.old jdk make-support build.log compare.sh configure-support jdk.code-workspace spec.gmk And the following filed under 'linux-x86_64-server-release/jdk/' bin include man _optimize_image_exec.cmdline _optimize_image_exec.marker conf lib modules _optimize_image_exec.log release On Sat, Apr 24, 2021 at 6:40 PM Guoxiong Li wrote: > Hi Manavjeet, > > I haven't used eclipse for a long time so my idea may be wrong. > According to your description, I think you should use > the path `jdk14/build/linux-x86_64-server-release*/images/jdk*` instead. > > --- > Best Regards, > Guoxiong. > > On Sat, Apr 24, 2021 at 8:59 PM Manavjeet Singh < > manavjeet18295 at iiitd.ac.in> wrote: > >> Hi! >> I compiled OpenJDK-14 on my machine and it is working fine when used from >> the command line. I want to build a program using it but when trying to >> add >> it to Eclipse(v4.18) through Window->Preferences->Installed >> JREs->Add->StandardVM and giving the path >> jdk14/build/linux-x86_64-server-release/jdk/. >> But it is showing the error: "Target is not a JDK root. System library not >> found". >> Have I made some mistake while compiling the jdk? Any help would be >> appreciated. >> Thanks >> > From lgxbslgx at gmail.com Sat Apr 24 14:14:31 2021 From: lgxbslgx at gmail.com (Guoxiong Li) Date: Sat, 24 Apr 2021 22:14:31 +0800 Subject: Error while using self compiled openJDK14 in eclipse In-Reply-To: References: Message-ID: Hi Manavjeet, It seems like an issue about building. Could you move this thread to build-dev at openjdk.java.net ? We can continue the discussion there. ---- Best Regards, Guoxiong. On Sat, Apr 24, 2021 at 9:28 PM Manavjeet Singh wrote: > Hi Guoxiong, > There isn't any images folder under 'linux-x86_64-server-release'. I have > only the following files(not bold ones are the folders). > bootcycle-spec.gmk build.log.old configure.log hotspot > Makefile support > buildjdk-spec.gmk buildtools configure.log.old jdk > make-support > build.log compare.sh configure-support jdk.code-workspace > spec.gmk > > And the following filed under 'linux-x86_64-server-release/jdk/' > bin include man _optimize_image_exec.cmdline > _optimize_image_exec.marker > conf lib modules _optimize_image_exec.log release > > > > On Sat, Apr 24, 2021 at 6:40 PM Guoxiong Li wrote: > >> Hi Manavjeet, >> >> I haven't used eclipse for a long time so my idea may be wrong. >> According to your description, I think you should use >> the path `jdk14/build/linux-x86_64-server-release*/images/jdk*` instead. >> >> --- >> Best Regards, >> Guoxiong. >> >> On Sat, Apr 24, 2021 at 8:59 PM Manavjeet Singh < >> manavjeet18295 at iiitd.ac.in> wrote: >> >>> Hi! >>> I compiled OpenJDK-14 on my machine and it is working fine when used from >>> the command line. I want to build a program using it but when trying to >>> add >>> it to Eclipse(v4.18) through Window->Preferences->Installed >>> JREs->Add->StandardVM and giving the path >>> jdk14/build/linux-x86_64-server-release/jdk/. >>> But it is showing the error: "Target is not a JDK root. System library >>> not >>> found". >>> Have I made some mistake while compiling the jdk? Any help would be >>> appreciated. >>> Thanks >>> >> From ayang at openjdk.java.net Sat Apr 24 16:39:28 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Sat, 24 Apr 2021 16:39:28 GMT Subject: RFR: 8252089: Remove psParallelCompact internal debug counters In-Reply-To: <7FydXMy7gkW3WynlKi3a83NeGWiYUZWq4NscWblAjy0=.f564b51f-84de-4cab-9dcc-9d403b26cfe6@github.com> References: <7FydXMy7gkW3WynlKi3a83NeGWiYUZWq4NscWblAjy0=.f564b51f-84de-4cab-9dcc-9d403b26cfe6@github.com> Message-ID: On Fri, 23 Apr 2021 07:45:41 GMT, Albert Mingkun Yang wrote: > Trivial change of removing debug-only counters. Thanks for the review. ------------- PR: https://git.openjdk.java.net/jdk/pull/3648 From ayang at openjdk.java.net Sat Apr 24 16:39:28 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Sat, 24 Apr 2021 16:39:28 GMT Subject: Integrated: 8252089: Remove psParallelCompact internal debug counters In-Reply-To: <7FydXMy7gkW3WynlKi3a83NeGWiYUZWq4NscWblAjy0=.f564b51f-84de-4cab-9dcc-9d403b26cfe6@github.com> References: <7FydXMy7gkW3WynlKi3a83NeGWiYUZWq4NscWblAjy0=.f564b51f-84de-4cab-9dcc-9d403b26cfe6@github.com> Message-ID: On Fri, 23 Apr 2021 07:45:41 GMT, Albert Mingkun Yang wrote: > Trivial change of removing debug-only counters. This pull request has now been integrated. Changeset: 0257ac14 Author: Albert Mingkun Yang URL: https://git.openjdk.java.net/jdk/commit/0257ac14 Stats: 33 lines in 2 files changed: 0 ins; 33 del; 0 mod 8252089: Remove psParallelCompact internal debug counters Reviewed-by: sjohanss, tschatzl ------------- PR: https://git.openjdk.java.net/jdk/pull/3648 From ayang at openjdk.java.net Sun Apr 25 08:33:55 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Sun, 25 Apr 2021 08:33:55 GMT Subject: RFR: 8184134: HeapRegion::LogOfHRGrainWords is unused Message-ID: Trivial change of removing unused code. ------------- Commit messages: - word Changes: https://git.openjdk.java.net/jdk/pull/3671/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3671&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8184134 Stats: 8 lines in 2 files changed: 0 ins; 6 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3671.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3671/head:pull/3671 PR: https://git.openjdk.java.net/jdk/pull/3671 From mli at openjdk.java.net Mon Apr 26 02:42:47 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Mon, 26 Apr 2021 02:42:47 GMT Subject: RFR: JDK-8265928: update copyright in several files Message-ID: I missed update the copyrights in several files, and per suggested in https://github.com/openjdk/jdk/pull/2379#pullrequestreview-584943730, it's better to update them in a separate CR, this is the one to fix them in a batch. ------------- Commit messages: - copyrights update. Changes: https://git.openjdk.java.net/jdk/pull/3682/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3682&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265928 Stats: 18 lines in 18 files changed: 0 ins; 0 del; 18 mod Patch: https://git.openjdk.java.net/jdk/pull/3682.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3682/head:pull/3682 PR: https://git.openjdk.java.net/jdk/pull/3682 From mli at openjdk.java.net Mon Apr 26 02:45:45 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Mon, 26 Apr 2021 02:45:45 GMT Subject: RFR: JDK-8265394: G1: Improve assert in HeapRegion::reset_not_compacted_after_full_gc In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 01:15:13 GMT, Hamlin Li wrote: > This is a trivial assert message improvement/fix. seems that the Build failure on mac is not related to this change. ------------- PR: https://git.openjdk.java.net/jdk/pull/3680 From mli at openjdk.java.net Mon Apr 26 02:45:45 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Mon, 26 Apr 2021 02:45:45 GMT Subject: RFR: JDK-8265394: G1: Improve assert in HeapRegion::reset_not_compacted_after_full_gc Message-ID: This is a trivial assert message improvement/fix. ------------- Commit messages: - JDK-8265394: G1: Improve assert in HeapRegion::reset_not_compacted_after_full_gc Changes: https://git.openjdk.java.net/jdk/pull/3680/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3680&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265394 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3680.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3680/head:pull/3680 PR: https://git.openjdk.java.net/jdk/pull/3680 From kbarrett at openjdk.java.net Mon Apr 26 04:24:37 2021 From: kbarrett at openjdk.java.net (Kim Barrett) Date: Mon, 26 Apr 2021 04:24:37 GMT Subject: RFR: 8184134: HeapRegion::LogOfHRGrainWords is unused In-Reply-To: References: Message-ID: <91mXfvnKTfsB8MQyCMbU1kEYvBeWsxfq8x_hpBC3h8w=.1572a298-f41c-4a64-9931-591c971a69d5@github.com> On Sun, 25 Apr 2021 08:26:07 GMT, Albert Mingkun Yang wrote: > Trivial change of removing unused code. Looks good. ------------- Marked as reviewed by kbarrett (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3671 From ayang at openjdk.java.net Mon Apr 26 06:36:25 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Mon, 26 Apr 2021 06:36:25 GMT Subject: Integrated: 8265681: G1: general cleanup for G1FullGCHeapRegionAttr In-Reply-To: <21DRpur7yybPfLDRtSXiTZhLLG_j0EuDxEWJq3zn-pw=.7a0a7a32-b1ea-4bb1-8aba-5b2ed121b51f@github.com> References: <21DRpur7yybPfLDRtSXiTZhLLG_j0EuDxEWJq3zn-pw=.7a0a7a32-b1ea-4bb1-8aba-5b2ed121b51f@github.com> Message-ID: On Thu, 22 Apr 2021 11:33:20 GMT, Albert Mingkun Yang wrote: > General cleanups of `G1FullGCHeapRegionAttr`, its APIs and usage. > > Regarding the naming, `SkipCompaction` is another option, but I went with the current version for the consistent `ing` suffix across all valid states.. This pull request has now been integrated. Changeset: 56fbef0f Author: Albert Mingkun Yang URL: https://git.openjdk.java.net/jdk/commit/56fbef0f Stats: 68 lines in 10 files changed: 20 ins; 2 del; 46 mod 8265681: G1: general cleanup for G1FullGCHeapRegionAttr Reviewed-by: tschatzl, sjohanss ------------- PR: https://git.openjdk.java.net/jdk/pull/3621 From ayang at openjdk.java.net Mon Apr 26 06:36:25 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Mon, 26 Apr 2021 06:36:25 GMT Subject: RFR: 8265681: G1: general cleanup for G1FullGCHeapRegionAttr In-Reply-To: <21DRpur7yybPfLDRtSXiTZhLLG_j0EuDxEWJq3zn-pw=.7a0a7a32-b1ea-4bb1-8aba-5b2ed121b51f@github.com> References: <21DRpur7yybPfLDRtSXiTZhLLG_j0EuDxEWJq3zn-pw=.7a0a7a32-b1ea-4bb1-8aba-5b2ed121b51f@github.com> Message-ID: On Thu, 22 Apr 2021 11:33:20 GMT, Albert Mingkun Yang wrote: > General cleanups of `G1FullGCHeapRegionAttr`, its APIs and usage. > > Regarding the naming, `SkipCompaction` is another option, but I went with the current version for the consistent `ing` suffix across all valid states.. Thanks for the review. ------------- PR: https://git.openjdk.java.net/jdk/pull/3621 From sjohanss at openjdk.java.net Mon Apr 26 08:01:32 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Mon, 26 Apr 2021 08:01:32 GMT Subject: RFR: 8184134: HeapRegion::LogOfHRGrainWords is unused In-Reply-To: References: Message-ID: <0irw_SA8MYxN3jq4t8ANyClnjXt4bCbb0uVd-TPYbKs=.3975e672-4deb-45e7-976c-5d6628d83c44@github.com> On Sun, 25 Apr 2021 08:26:07 GMT, Albert Mingkun Yang wrote: > Trivial change of removing unused code. Looks good, just a minor comment. src/hotspot/share/gc/g1/heapRegion.hpp line 321: > 319: // well as other related fields that are based on the heap region > 320: // size (LogOfHRGrainBytes / CardsPerRegion). > 321: // All those fields are considered constant Please reformat the comment to use similar length of lines. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3671 From sjohanss at openjdk.java.net Mon Apr 26 08:21:31 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Mon, 26 Apr 2021 08:21:31 GMT Subject: RFR: 8256155: Allow multiple large page sizes to be used on Linux [v27] In-Reply-To: References: Message-ID: On Fri, 23 Apr 2021 22:34:16 GMT, Marcus G K Williams wrote: > > an easy approach should be to just remove all large page sizes larger than the decided maximum from _page_sizes. > > It seemed easier to forgo adding large page sizes if they were less than `LargePageSizeInBytes` and `LargePageSizeInBytes != 0` or default. Please let me know if the solution seems appropriate. > Sure, this is fine as well. I have some comments bug will post them in the code. > > The warning should go away since it is documented to mean use the default. > > Done. Feedback there would also be helpful. > > I'm seeing one failure with the TestTracePageSizes test that I'm yet to figure out. On a run with `run main/othervm -XX:+AlwaysPreTouch -Xlog:pagesize:ps-%p.log -XX:+UseSerialGC -XX:+UseTransparentHugePages TestTracePageSizes` > reason: User specified action: run main/othervm -XX:+AlwaysPreTouch -Xlog:pagesize:ps-%p.log -XX:+UseSerialGC -XX:+UseTransparentHugePages TestTracePageSizes > ... > Added range: [43a400000, 440000000) pageSize=4KB isTHP=false isHUGETLB=false > Added range: [440000000, 480000000) pageSize=4KB isTHP=true isHUGETLB=false > Added range: [480000000, 57c2a0000) pageSize=4KB isTHP=false isHUGETLB=false > > Two guesses so far: It seems like a possible issue with `ReservedSpace::actual_reserved_page_size` returning the wrong page size. Or could this base page address be 4k padding for THP largepage? This seems to be a problem with the test, it assumes that the whole range for the heap should be using the same page size but for THP this might not be true since it is up to the system to decide. Been able to reproduce it locally as well and it seems more likely to happen when the heap large, might be that the whole range can't be satisfied with large pages and for some reason the system decides to not use them for the first range `[43a400000, 440000000)`. This range is still 2M aligned, so it should be able to. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From pliden at openjdk.java.net Mon Apr 26 08:49:30 2021 From: pliden at openjdk.java.net (Per Liden) Date: Mon, 26 Apr 2021 08:49:30 GMT Subject: RFR: 8265702: ZGC on macOS/aarch64 [v2] In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 23:04:43 GMT, Erik Joelsson wrote: >> Per Liden has updated the pull request incrementally with one additional commit since the last revision: >> >> Add comment to #else > > Build change looks good. Thanks @erikj79, @dholmes-ora, @stefank and @gerard-ziemski for reviewing! ------------- PR: https://git.openjdk.java.net/jdk/pull/3609 From pliden at openjdk.java.net Mon Apr 26 08:49:31 2021 From: pliden at openjdk.java.net (Per Liden) Date: Mon, 26 Apr 2021 08:49:31 GMT Subject: Integrated: 8265702: ZGC on macOS/aarch64 In-Reply-To: References: Message-ID: On Wed, 21 Apr 2021 21:10:02 GMT, Per Liden wrote: > This patch enables ZGC on macOS/aarch64. It does three things: > 1) Enables building of ZGC on this platform. > 2) Adds `os::processor_id()`, which for now always returns 0. > 3) Fixes a WX issue in `OptoRuntime::handle_exception_C()`, where the stackwater mark might unnecessarily process a frame when the thread is in WXExec mode. In this case, the we're not touching any oops, so we don't need to process any frames. > > Testing: Passes the same tests as macOS/x86_64 (with the exception of pre-existing issues unrelated to ZGC). This pull request has now been integrated. Changeset: 0d08d735 Author: Per Liden URL: https://git.openjdk.java.net/jdk/commit/0d08d735 Stats: 13 lines in 3 files changed: 7 ins; 1 del; 5 mod 8265702: ZGC on macOS/aarch64 Reviewed-by: erikj, dholmes, stefank, gziemski ------------- PR: https://git.openjdk.java.net/jdk/pull/3609 From sjohanss at openjdk.java.net Mon Apr 26 08:53:45 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Mon, 26 Apr 2021 08:53:45 GMT Subject: RFR: 8256155: Allow multiple large page sizes to be used on Linux [v29] In-Reply-To: References: Message-ID: On Fri, 23 Apr 2021 22:24:38 GMT, Marcus G K Williams wrote: >> Change the meaning of LargePageSizeInBytes to be the maximum large page size the JVM may use (not the only one). A default value of zero will mean to allow the JVM use large page sizes up to the system's default large page size. > > Marcus G K Williams has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 48 commits: > > - Set LargePageSizeInBytes to largepage upper limit > > Signed-off-by: Marcus G K Williams > - Merge branch 'master' into update_hlp > - Fix merge > > Signed-off-by: Marcus G K Williams > - Merge branch 'master' into update_hlp > - Merge branch 'master' into update_hlp > - Rebase on pull/3073 > > Signed-off-by: Marcus G K Williams > - Merge branch 'pull/3073' into update_hlp > - Thomas review. > > Changed commit_memory_special to return bool to signal if the request succeeded or not. > - Self review. > > Update helper name to better match commit_memory_special(). > - Marcus review. > > Updated comments. > - ... and 38 more: https://git.openjdk.java.net/jdk/compare/5aab1609...6f063309 Changes requested by sjohanss (Reviewer). src/hotspot/os/linux/os_linux.cpp line 3737: > 3735: // Populate large page sizes to _page_sizes. Add pages that > 3736: // are less than or equal to LargePageSizeInBytes, except when LargePageSizeInBytes=0 > 3737: // or FLAG_IS_DEFAULT(LargePageSizeInBytes), add all sizes What we currently have in the CSR is to use the default large page size as the max if `LargePageSizeInBytes`is not set. If we first check if `LargePageSizeInBytes` have been set and update `_large_page_size` accordingly above (see comment below). I think this comment should say something like: Suggestion: // Populate _page_sizes with large page sizes less than or equal to // _large_page_size. src/hotspot/os/linux/os_linux.cpp line 3767: > 3765: } else { > 3766: _large_page_size = default_large_page_size; > 3767: } Move this part up to be done before you populate `_page_sizes` with the page sizes from `all_page_sizes`. Then you can use `_large_page_size` as the maximum page size that you add to `_page_sizes` and you don't have to care about `LargePageSizeInBytes` in that for-loop. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From sjohanss at openjdk.java.net Mon Apr 26 08:53:46 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Mon, 26 Apr 2021 08:53:46 GMT Subject: RFR: 8256155: Allow multiple large page sizes to be used on Linux [v29] In-Reply-To: <2UqkjBQ_wVQcz_qx2wjchZanODAw8-QB-qROH9kdLM0=.e1980df2-de0c-412f-b219-5ac2eb38fafb@github.com> References: <2UqkjBQ_wVQcz_qx2wjchZanODAw8-QB-qROH9kdLM0=.e1980df2-de0c-412f-b219-5ac2eb38fafb@github.com> Message-ID: On Mon, 26 Apr 2021 08:34:26 GMT, Stefan Johansson wrote: >> src/hotspot/os/linux/os_linux.cpp line 3767: >> >>> 3765: } else { >>> 3766: _large_page_size = default_large_page_size; >>> 3767: } >> >> Move this part up to be done before you populate `_page_sizes` with the page sizes from `all_page_sizes`. Then you can use `_large_page_size` as the maximum page size that you add to `_page_sizes` and you don't have to care about `LargePageSizeInBytes` in that for-loop. > > I also think that the log-tag should be `pagesize`. The comment here could also be updated: // Check LargePageSizeInBytes and setup _large_page_size to hold // the maximum allowed large page size. If not set the default large // page size is used as the maximum. ``` ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From sjohanss at openjdk.java.net Mon Apr 26 08:53:46 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Mon, 26 Apr 2021 08:53:46 GMT Subject: RFR: 8256155: Allow multiple large page sizes to be used on Linux [v29] In-Reply-To: References: Message-ID: <2UqkjBQ_wVQcz_qx2wjchZanODAw8-QB-qROH9kdLM0=.e1980df2-de0c-412f-b219-5ac2eb38fafb@github.com> On Mon, 26 Apr 2021 08:33:44 GMT, Stefan Johansson wrote: >> Marcus G K Williams has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 48 commits: >> >> - Set LargePageSizeInBytes to largepage upper limit >> >> Signed-off-by: Marcus G K Williams >> - Merge branch 'master' into update_hlp >> - Fix merge >> >> Signed-off-by: Marcus G K Williams >> - Merge branch 'master' into update_hlp >> - Merge branch 'master' into update_hlp >> - Rebase on pull/3073 >> >> Signed-off-by: Marcus G K Williams >> - Merge branch 'pull/3073' into update_hlp >> - Thomas review. >> >> Changed commit_memory_special to return bool to signal if the request succeeded or not. >> - Self review. >> >> Update helper name to better match commit_memory_special(). >> - Marcus review. >> >> Updated comments. >> - ... and 38 more: https://git.openjdk.java.net/jdk/compare/5aab1609...6f063309 > > src/hotspot/os/linux/os_linux.cpp line 3767: > >> 3765: } else { >> 3766: _large_page_size = default_large_page_size; >> 3767: } > > Move this part up to be done before you populate `_page_sizes` with the page sizes from `all_page_sizes`. Then you can use `_large_page_size` as the maximum page size that you add to `_page_sizes` and you don't have to care about `LargePageSizeInBytes` in that for-loop. I also think that the log-tag should be `pagesize`. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From ayang at openjdk.java.net Mon Apr 26 08:56:53 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Mon, 26 Apr 2021 08:56:53 GMT Subject: RFR: 8184134: HeapRegion::LogOfHRGrainWords is unused [v2] In-Reply-To: References: Message-ID: > Trivial change of removing unused code. Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: review ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3671/files - new: https://git.openjdk.java.net/jdk/pull/3671/files/192a7a47..4b1933f7 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3671&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3671&range=00-01 Stats: 5 lines in 1 file changed: 0 ins; 1 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/3671.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3671/head:pull/3671 PR: https://git.openjdk.java.net/jdk/pull/3671 From pliden at openjdk.java.net Mon Apr 26 11:00:08 2021 From: pliden at openjdk.java.net (Per Liden) Date: Mon, 26 Apr 2021 11:00:08 GMT Subject: RFR: 8261759: ZGC: ZWorker Threads Continue Marking After System.exit() called [v4] In-Reply-To: References: Message-ID: > During JVM termination, ZCollectedHeap::stop() currently only returns once any ongoing GC cycle has completed. However, waiting for a GC cycle to complete can take awhile, so the JVM might not terminate as promptly as one might expect. > > This patch adds abortion checks to workers doing marking and relocation to allow ZWorker threads to terminate promptly. In addition, before and after concurrent phases, the ZDriver thread checks if `should_terminate()` is true and prematurely terminates the GC cycle if that's the case. > > Testing: > * Tier1-7 on Linux. > * SPECjbb2015, comparing score and marking times against baseline (5 iterations each), no difference observed. > * Ad-hoc programs calling System.exit() during marking/relocation. > * Manual Ctrl-C testing. Per Liden has updated the pull request incrementally with one additional commit since the last revision: Add comment to macro ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3453/files - new: https://git.openjdk.java.net/jdk/pull/3453/files/d0e6a56a..53cf20a8 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3453&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3453&range=02-03 Stats: 5 lines in 1 file changed: 5 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3453.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3453/head:pull/3453 PR: https://git.openjdk.java.net/jdk/pull/3453 From tschatzl at openjdk.java.net Mon Apr 26 11:30:55 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 26 Apr 2021 11:30:55 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask [v2] In-Reply-To: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: > [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). > > To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. > > This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. > > [Here](https://github.com/openjdk/jdk/compare/master...tschatzl:parallel-phases-join-everything) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). > > Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: iwalulya review ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3653/files - new: https://git.openjdk.java.net/jdk/pull/3653/files/166e72ef..f00f1021 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=00-01 Stats: 21 lines in 3 files changed: 14 ins; 1 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/3653.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3653/head:pull/3653 PR: https://git.openjdk.java.net/jdk/pull/3653 From sjohanss at openjdk.java.net Mon Apr 26 11:38:28 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Mon, 26 Apr 2021 11:38:28 GMT Subject: RFR: 8265127: ZGC: Fix incorrect reporting of reclaimed memory In-Reply-To: References: Message-ID: <1t37ZAQcWzYh9gBCRuS1RlFpWldn3PC_H8TiMpIgu8A=.9c046d8b-7638-4094-b16e-979314501a45@github.com> On Tue, 13 Apr 2021 11:18:10 GMT, Per Liden wrote: > When Java threads relocate objects, ZGC will over-report number of reclaimed bytes and under-report amount of garbage left after GC. This happens because the logic calculating the total number of reclaimed bytes assumes all objects where relocated by GC workers. The result is that the information printed in the GC log can be skewed. > > This patch solves this by keeping track of the number of bytes relocated by Java threads, and using that number in the logic calculating the total number of reclaimed bytes. Looks good. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3462 From sjohanss at openjdk.java.net Mon Apr 26 11:38:29 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Mon, 26 Apr 2021 11:38:29 GMT Subject: RFR: 8265116: ZGC: Steal local stacks instead of flushing them In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 08:18:29 GMT, Per Liden wrote: > As part of addressing the issue of excessive mark stack usage, it was observed that sometimes mark stack utilization can be very low. For example, casparcwang at tencent.com reported that when running Apache Zookeeper, mark stack utilization was ~10%. > > The problem is that GC workers are a bit too eager to flush mark stacks. Currently, whenever a GC worker has drained its home stripe it will always flush all remaining mark stacks. This causes non-full mark stacks to be flushed. This in turn means that the GC worker who receives that non-full stack (small amount of work) tends to also produce non-full stacks (small amount of work) for other GC workers. This can lead to a vicious cycle, which results in excessive mark stack memory usage. > > The solution proposed here is to not let GC workers flush non-full stacks. After draining the home stripe, a GC worker will instead try to steal any non-full local stack belonging to a different stripe, and if no such stacks exist it will (as it does today) try to steal a global stack belonging to a different stripe. > > (Note: This PR only addresses one part of the mark stack usage problem. Other improvements, such as the discussed "mark before push"-patch will be a separate PR). > > Testing: > * Tier1-7. > * SPECjbb2015, score and mark times unaffected. > * casparcwang at tencent.com reports that this patch (in combination with the "mark before push"-patch) solves the problem for Apache Zookeeper. Looks good. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3455 From sjohanss at openjdk.java.net Mon Apr 26 11:40:29 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Mon, 26 Apr 2021 11:40:29 GMT Subject: RFR: 8261759: ZGC: ZWorker Threads Continue Marking After System.exit() called [v4] In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 11:00:08 GMT, Per Liden wrote: >> During JVM termination, ZCollectedHeap::stop() currently only returns once any ongoing GC cycle has completed. However, waiting for a GC cycle to complete can take awhile, so the JVM might not terminate as promptly as one might expect. >> >> This patch adds abortion checks to workers doing marking and relocation to allow ZWorker threads to terminate promptly. In addition, before and after concurrent phases, the ZDriver thread checks if `should_terminate()` is true and prematurely terminates the GC cycle if that's the case. >> >> Testing: >> * Tier1-7 on Linux. >> * SPECjbb2015, comparing score and marking times against baseline (5 iterations each), no difference observed. >> * Ad-hoc programs calling System.exit() during marking/relocation. >> * Manual Ctrl-C testing. > > Per Liden has updated the pull request incrementally with one additional commit since the last revision: > > Add comment to macro Looks good, thanks for adding the comment for the macro. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3453 From tschatzl at openjdk.java.net Mon Apr 26 11:43:56 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 26 Apr 2021 11:43:56 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask [v3] In-Reply-To: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: <5URVIwO7WbF0Q9BmaNHcjCkRtwkmEvmY5gEKKvfxE3Q=.e9e94c80-4177-48a8-9a9d-edf2260f01f8@github.com> > [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). > > To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. > > This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. > > [Here](https://github.com/openjdk/jdk/compare/master...tschatzl:parallel-phases-join-everything) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). > > Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: iwalulya: change to fetch_and_add() ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3653/files - new: https://git.openjdk.java.net/jdk/pull/3653/files/f00f1021..712b0c6f Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=01-02 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3653.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3653/head:pull/3653 PR: https://git.openjdk.java.net/jdk/pull/3653 From tschatzl at openjdk.java.net Mon Apr 26 11:47:31 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 26 Apr 2021 11:47:31 GMT Subject: RFR: JDK-8265928: update copyright in several files In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 02:36:13 GMT, Hamlin Li wrote: > I missed update the copyrights in several files, and per suggested in https://github.com/openjdk/jdk/pull/2379#pullrequestreview-584943730, it's better to update them in a separate CR, this is the one to fix them in a batch. Lgtm. Please upper case the first word in the description (here and in the CR). ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3682 From ayang at openjdk.java.net Mon Apr 26 11:53:27 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Mon, 26 Apr 2021 11:53:27 GMT Subject: RFR: 8261759: ZGC: ZWorker Threads Continue Marking After System.exit() called [v4] In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 11:00:08 GMT, Per Liden wrote: >> During JVM termination, ZCollectedHeap::stop() currently only returns once any ongoing GC cycle has completed. However, waiting for a GC cycle to complete can take awhile, so the JVM might not terminate as promptly as one might expect. >> >> This patch adds abortion checks to workers doing marking and relocation to allow ZWorker threads to terminate promptly. In addition, before and after concurrent phases, the ZDriver thread checks if `should_terminate()` is true and prematurely terminates the GC cycle if that's the case. >> >> Testing: >> * Tier1-7 on Linux. >> * SPECjbb2015, comparing score and marking times against baseline (5 iterations each), no difference observed. >> * Ad-hoc programs calling System.exit() during marking/relocation. >> * Manual Ctrl-C testing. > > Per Liden has updated the pull request incrementally with one additional commit since the last revision: > > Add comment to macro Marked as reviewed by ayang (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3453 From tschatzl at openjdk.java.net Mon Apr 26 11:53:30 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 26 Apr 2021 11:53:30 GMT Subject: RFR: 8257774: G1: Trigger collect when free region count drops below threshold to prevent evacuation failures [v4] In-Reply-To: <_Z9CQ3H2RDvDgBjsDEtx_ykIxUCBOFWSfdO87JZ_cqE=.bf46b8dd-fc03-409b-8d1c-b865c6cc3a2e@github.com> References: <5Ro0P4NPaUuG-LPpgqHY5S_kh_EVpwRgxqc5jTCano4=.afc3f2a4-dc9e-46c1-ac84-298cf143fa41@github.com> <_Z9CQ3H2RDvDgBjsDEtx_ykIxUCBOFWSfdO87JZ_cqE=.bf46b8dd-fc03-409b-8d1c-b865c6cc3a2e@github.com> Message-ID: On Thu, 22 Apr 2021 21:55:02 GMT, Aditya Mandaleeka wrote: >> Aditya Mandaleeka has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits: >> >> - Remove temp init variable, update code after rebase. >> - Refactor allocation funcs, removing attempt_lock_free_first. >> - Fix format specifiers. >> - Fix a few warnings as well as a Windows build error. >> - Fix whitespace and comment. >> - Further refactoring/changes. >> - Address feedback and make other changes. >> - Improve G1GC accounting for humongous objects >> >> If there is a burst of humongous allocations they can >> consume all of the free regions left in the heap. If >> there are no free regions for to-space a collect will >> fail to evacuate all objects. This makes the GC very >> slow. Since, humongous regions can be collected in >> G1GC young collects force a young collect before the >> number of free regions becomes less than the number >> the GC is likely to require. >> >> Fix TestGCLogMessages.java so it still causes Evacuation >> failures. >> >> Signed-off-by: Charlie Gracie > > src/hotspot/share/gc/g1/g1Policy.cpp line 786: > >> 784: update_rs_length_prediction(); >> 785: >> 786: // Is this the right place? Should it be in the below? > > Can a reviewer confirm this so that I can remove the comment? The comment can be removed imho. ------------- PR: https://git.openjdk.java.net/jdk/pull/3143 From pliden at openjdk.java.net Mon Apr 26 11:56:28 2021 From: pliden at openjdk.java.net (Per Liden) Date: Mon, 26 Apr 2021 11:56:28 GMT Subject: RFR: 8265127: ZGC: Fix incorrect reporting of reclaimed memory In-Reply-To: References: Message-ID: <6Thlxi0jkyNLRSkZ8LWklMO8n4O5HmMphIUXXYAmifI=.51e34efe-fdf4-4913-97d5-1041a1cadc98@github.com> On Mon, 19 Apr 2021 13:37:51 GMT, Albert Mingkun Yang wrote: >> When Java threads relocate objects, ZGC will over-report number of reclaimed bytes and under-report amount of garbage left after GC. This happens because the logic calculating the total number of reclaimed bytes assumes all objects where relocated by GC workers. The result is that the information printed in the GC log can be skewed. >> >> This patch solves this by keeping track of the number of bytes relocated by Java threads, and using that number in the logic calculating the total number of reclaimed bytes. > > Marked as reviewed by ayang (Committer). Thanks @albertnetymk and @kstefanj for reviewing! ------------- PR: https://git.openjdk.java.net/jdk/pull/3462 From pliden at openjdk.java.net Mon Apr 26 11:56:29 2021 From: pliden at openjdk.java.net (Per Liden) Date: Mon, 26 Apr 2021 11:56:29 GMT Subject: Integrated: 8265127: ZGC: Fix incorrect reporting of reclaimed memory In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 11:18:10 GMT, Per Liden wrote: > When Java threads relocate objects, ZGC will over-report number of reclaimed bytes and under-report amount of garbage left after GC. This happens because the logic calculating the total number of reclaimed bytes assumes all objects where relocated by GC workers. The result is that the information printed in the GC log can be skewed. > > This patch solves this by keeping track of the number of bytes relocated by Java threads, and using that number in the logic calculating the total number of reclaimed bytes. This pull request has now been integrated. Changeset: a9367dbd Author: Per Liden URL: https://git.openjdk.java.net/jdk/commit/a9367dbd Stats: 79 lines in 9 files changed: 54 ins; 1 del; 24 mod 8265127: ZGC: Fix incorrect reporting of reclaimed memory Reviewed-by: ayang, sjohanss ------------- PR: https://git.openjdk.java.net/jdk/pull/3462 From pliden at openjdk.java.net Mon Apr 26 12:01:27 2021 From: pliden at openjdk.java.net (Per Liden) Date: Mon, 26 Apr 2021 12:01:27 GMT Subject: RFR: 8265116: ZGC: Steal local stacks instead of flushing them In-Reply-To: <7TvL-lvD4DQSiRzGWIGifJhXeEcuaHWDmbW3XvDFNw0=.605b493c-8f29-4af5-99bc-4f8f5a825852@github.com> References: <7TvL-lvD4DQSiRzGWIGifJhXeEcuaHWDmbW3XvDFNw0=.605b493c-8f29-4af5-99bc-4f8f5a825852@github.com> Message-ID: On Tue, 20 Apr 2021 10:33:07 GMT, Albert Mingkun Yang wrote: >> As part of addressing the issue of excessive mark stack usage, it was observed that sometimes mark stack utilization can be very low. For example, casparcwang at tencent.com reported that when running Apache Zookeeper, mark stack utilization was ~10%. >> >> The problem is that GC workers are a bit too eager to flush mark stacks. Currently, whenever a GC worker has drained its home stripe it will always flush all remaining mark stacks. This causes non-full mark stacks to be flushed. This in turn means that the GC worker who receives that non-full stack (small amount of work) tends to also produce non-full stacks (small amount of work) for other GC workers. This can lead to a vicious cycle, which results in excessive mark stack memory usage. >> >> The solution proposed here is to not let GC workers flush non-full stacks. After draining the home stripe, a GC worker will instead try to steal any non-full local stack belonging to a different stripe, and if no such stacks exist it will (as it does today) try to steal a global stack belonging to a different stripe. >> >> (Note: This PR only addresses one part of the mark stack usage problem. Other improvements, such as the discussed "mark before push"-patch will be a separate PR). >> >> Testing: >> * Tier1-7. >> * SPECjbb2015, score and mark times unaffected. >> * casparcwang at tencent.com reports that this patch (in combination with the "mark before push"-patch) solves the problem for Apache Zookeeper. > > Marked as reviewed by ayang (Committer). Thanks @albertnetymk and @kstefanj for reviewing! ------------- PR: https://git.openjdk.java.net/jdk/pull/3455 From pliden at openjdk.java.net Mon Apr 26 12:01:28 2021 From: pliden at openjdk.java.net (Per Liden) Date: Mon, 26 Apr 2021 12:01:28 GMT Subject: Integrated: 8265116: ZGC: Steal local stacks instead of flushing them In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 08:18:29 GMT, Per Liden wrote: > As part of addressing the issue of excessive mark stack usage, it was observed that sometimes mark stack utilization can be very low. For example, casparcwang at tencent.com reported that when running Apache Zookeeper, mark stack utilization was ~10%. > > The problem is that GC workers are a bit too eager to flush mark stacks. Currently, whenever a GC worker has drained its home stripe it will always flush all remaining mark stacks. This causes non-full mark stacks to be flushed. This in turn means that the GC worker who receives that non-full stack (small amount of work) tends to also produce non-full stacks (small amount of work) for other GC workers. This can lead to a vicious cycle, which results in excessive mark stack memory usage. > > The solution proposed here is to not let GC workers flush non-full stacks. After draining the home stripe, a GC worker will instead try to steal any non-full local stack belonging to a different stripe, and if no such stacks exist it will (as it does today) try to steal a global stack belonging to a different stripe. > > (Note: This PR only addresses one part of the mark stack usage problem. Other improvements, such as the discussed "mark before push"-patch will be a separate PR). > > Testing: > * Tier1-7. > * SPECjbb2015, score and mark times unaffected. > * casparcwang at tencent.com reports that this patch (in combination with the "mark before push"-patch) solves the problem for Apache Zookeeper. This pull request has now been integrated. Changeset: 31abe68f Author: Per Liden URL: https://git.openjdk.java.net/jdk/commit/31abe68f Stats: 41 lines in 4 files changed: 25 ins; 2 del; 14 mod 8265116: ZGC: Steal local stacks instead of flushing them Co-authored-by: Wang Chao Reviewed-by: ayang, sjohanss ------------- PR: https://git.openjdk.java.net/jdk/pull/3455 From pliden at openjdk.java.net Mon Apr 26 12:13:53 2021 From: pliden at openjdk.java.net (Per Liden) Date: Mon, 26 Apr 2021 12:13:53 GMT Subject: RFR: 8261759: ZGC: ZWorker Threads Continue Marking After System.exit() called [v5] In-Reply-To: References: Message-ID: > During JVM termination, ZCollectedHeap::stop() currently only returns once any ongoing GC cycle has completed. However, waiting for a GC cycle to complete can take awhile, so the JVM might not terminate as promptly as one might expect. > > This patch adds abortion checks to workers doing marking and relocation to allow ZWorker threads to terminate promptly. In addition, before and after concurrent phases, the ZDriver thread checks if `should_terminate()` is true and prematurely terminates the GC cycle if that's the case. > > Testing: > * Tier1-7 on Linux. > * SPECjbb2015, comparing score and marking times against baseline (5 iterations each), no difference observed. > * Ad-hoc programs calling System.exit() during marking/relocation. > * Manual Ctrl-C testing. Per Liden has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains five commits: - Merge master - Add comment to macro - Forward in-place if GC aborted - Add missing include of precompiled.hpp - 8261759: ZGC: ZWorker Threads Continue Marking After System.exit() called ------------- Changes: https://git.openjdk.java.net/jdk/pull/3453/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3453&range=04 Stats: 193 lines in 10 files changed: 176 ins; 2 del; 15 mod Patch: https://git.openjdk.java.net/jdk/pull/3453.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3453/head:pull/3453 PR: https://git.openjdk.java.net/jdk/pull/3453 From pliden at openjdk.java.net Mon Apr 26 12:18:28 2021 From: pliden at openjdk.java.net (Per Liden) Date: Mon, 26 Apr 2021 12:18:28 GMT Subject: Integrated: 8261759: ZGC: ZWorker Threads Continue Marking After System.exit() called In-Reply-To: References: Message-ID: On Tue, 13 Apr 2021 07:03:10 GMT, Per Liden wrote: > During JVM termination, ZCollectedHeap::stop() currently only returns once any ongoing GC cycle has completed. However, waiting for a GC cycle to complete can take awhile, so the JVM might not terminate as promptly as one might expect. > > This patch adds abortion checks to workers doing marking and relocation to allow ZWorker threads to terminate promptly. In addition, before and after concurrent phases, the ZDriver thread checks if `should_terminate()` is true and prematurely terminates the GC cycle if that's the case. > > Testing: > * Tier1-7 on Linux. > * SPECjbb2015, comparing score and marking times against baseline (5 iterations each), no difference observed. > * Ad-hoc programs calling System.exit() during marking/relocation. > * Manual Ctrl-C testing. This pull request has now been integrated. Changeset: c3ac6900 Author: Per Liden URL: https://git.openjdk.java.net/jdk/commit/c3ac6900 Stats: 193 lines in 10 files changed: 176 ins; 2 del; 15 mod 8261759: ZGC: ZWorker Threads Continue Marking After System.exit() called Reviewed-by: sjohanss, ayang ------------- PR: https://git.openjdk.java.net/jdk/pull/3453 From pliden at openjdk.java.net Mon Apr 26 12:18:27 2021 From: pliden at openjdk.java.net (Per Liden) Date: Mon, 26 Apr 2021 12:18:27 GMT Subject: RFR: 8261759: ZGC: ZWorker Threads Continue Marking After System.exit() called [v4] In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 11:50:48 GMT, Albert Mingkun Yang wrote: >> Per Liden has updated the pull request incrementally with one additional commit since the last revision: >> >> Add comment to macro > > Marked as reviewed by ayang (Committer). Thanks @albertnetymk and @kstefanj for reviewing! ------------- PR: https://git.openjdk.java.net/jdk/pull/3453 From tschatzl at openjdk.java.net Mon Apr 26 12:28:31 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Mon, 26 Apr 2021 12:28:31 GMT Subject: RFR: JDK-8265394: G1: Improve assert in HeapRegion::reset_not_compacted_after_full_gc In-Reply-To: References: Message-ID: <_CCuFLnoP8c-ZlUFKSqMaIVTpiYp94E6ae5U1YyisAY=.be6992b6-4261-493b-b3a5-2ecb8fd7d17e@github.com> On Mon, 26 Apr 2021 01:15:13 GMT, Hamlin Li wrote: > This is a trivial assert message improvement/fix. Lgtm and trivial. ------------- Marked as reviewed by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3680 From pliden at openjdk.java.net Mon Apr 26 13:25:45 2021 From: pliden at openjdk.java.net (Per Liden) Date: Mon, 26 Apr 2021 13:25:45 GMT Subject: RFR: 8260267: ZGC: Reduce mark stack usage [v2] In-Reply-To: <1Qn76cgJaltPI2LtFQFfdqCG8SzOJwg3ORVQc8pK3MY=.918a159f-4f3d-408c-8cca-5b2bf727a0b2@github.com> References: <1Qn76cgJaltPI2LtFQFfdqCG8SzOJwg3ORVQc8pK3MY=.918a159f-4f3d-408c-8cca-5b2bf727a0b2@github.com> Message-ID: > Under some conditions, striped marking can cause excessive mark stack memory usage. For example, when the object graph has one or more N:1 relations ships, where N is a large number. > > To avoid pushing mark stack entries to already marked objects, and thus reduce mark stack memory usage, this patch adds does two things: > * GC threads will under some conditions mark the object before pushing, else (if the condition to mark failed) > * GC threads will check if the object is already marked before pushing. > > The conditions for marking before pushing is: > * We are using a single stripe, in which cases striped marking is essentially disabled anyway. > * We see excessive mark stack usage. In other words, the mark stack allocator indicates "high usage". > > To make this easier to review, this PR contains two commits: > * `ZGC: Reduce mark stack usage` - This patch adds the infrastructure needed to mark before push, but only does so if we are using a single stripe. > * `ZGC: Mark before push if mark stack usage is high` - This patch adds the condition to mark before push if the mark stack allocator indicates "high usage". As a bonus, this patch will also uncommit memory used for mark stacks after the marking has completed. > > Testing: > * Tier1-7 > * SPECjbb2015, score and marking times unaffected > * casparcwang at tencent.com reports that this patch, in combination with PR #3455, solves the problem for Apache Zookeeper. Per Liden has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits: - Merge master - ZGC: Mark before push if mark stack usage is high - ZGC: Reduce mark stack usage ------------- Changes: https://git.openjdk.java.net/jdk/pull/3460/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3460&range=01 Stats: 283 lines in 19 files changed: 192 ins; 41 del; 50 mod Patch: https://git.openjdk.java.net/jdk/pull/3460.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3460/head:pull/3460 PR: https://git.openjdk.java.net/jdk/pull/3460 From mli at openjdk.java.net Mon Apr 26 14:30:27 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Mon, 26 Apr 2021 14:30:27 GMT Subject: RFR: JDK-8265928: G1: Update copyright in several files In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 02:36:13 GMT, Hamlin Li wrote: > I missed update the copyrights in several files, and per suggested in https://github.com/openjdk/jdk/pull/2379#pullrequestreview-584943730, it's better to update them in a separate CR, this is the one to fix them in a batch. Thanks Thomas for your review. ------------- PR: https://git.openjdk.java.net/jdk/pull/3682 From mli at openjdk.java.net Mon Apr 26 14:30:28 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Mon, 26 Apr 2021 14:30:28 GMT Subject: Integrated: JDK-8265928: G1: Update copyright in several files In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 02:36:13 GMT, Hamlin Li wrote: > I missed update the copyrights in several files, and per suggested in https://github.com/openjdk/jdk/pull/2379#pullrequestreview-584943730, it's better to update them in a separate CR, this is the one to fix them in a batch. This pull request has now been integrated. Changeset: 68011c64 Author: Hamlin Li URL: https://git.openjdk.java.net/jdk/commit/68011c64 Stats: 17 lines in 17 files changed: 0 ins; 0 del; 17 mod 8265928: G1: Update copyright in several files Reviewed-by: tschatzl ------------- PR: https://git.openjdk.java.net/jdk/pull/3682 From mli at openjdk.java.net Mon Apr 26 14:40:35 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Mon, 26 Apr 2021 14:40:35 GMT Subject: RFR: JDK-8265394: G1: Improve assert in HeapRegion::reset_not_compacted_after_full_gc In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 01:15:13 GMT, Hamlin Li wrote: > This is a trivial assert message improvement/fix. Thanks Thomas for your review. ------------- PR: https://git.openjdk.java.net/jdk/pull/3680 From mli at openjdk.java.net Mon Apr 26 14:46:46 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Mon, 26 Apr 2021 14:46:46 GMT Subject: RFR: JDK-8265394: G1: Improve assert in HeapRegion::reset_not_compacted_after_full_gc [v2] In-Reply-To: References: Message-ID: > This is a trivial assert message improvement/fix. Hamlin Li has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains two commits: - Merge branch 'master' into improve-assert - JDK-8265394: G1: Improve assert in HeapRegion::reset_not_compacted_after_full_gc ------------- Changes: https://git.openjdk.java.net/jdk/pull/3680/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3680&range=01 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3680.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3680/head:pull/3680 PR: https://git.openjdk.java.net/jdk/pull/3680 From mli at openjdk.java.net Mon Apr 26 14:46:46 2021 From: mli at openjdk.java.net (Hamlin Li) Date: Mon, 26 Apr 2021 14:46:46 GMT Subject: Integrated: JDK-8265394: G1: Improve assert in HeapRegion::reset_not_compacted_after_full_gc In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 01:15:13 GMT, Hamlin Li wrote: > This is a trivial assert message improvement/fix. This pull request has now been integrated. Changeset: 8559a530 Author: Hamlin Li URL: https://git.openjdk.java.net/jdk/commit/8559a530 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8265394: G1: Improve assert in HeapRegion::reset_not_compacted_after_full_gc Reviewed-by: tschatzl ------------- PR: https://git.openjdk.java.net/jdk/pull/3680 From zgu at openjdk.java.net Mon Apr 26 19:18:50 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Mon, 26 Apr 2021 19:18:50 GMT Subject: RFR: 8265995: Shenandoah: Move ShenandoahInitMarkRootsClosure close to its use Message-ID: ShenandoahInitMarkRootsClosure is only used by STW mark, let's move it there. Test: - [x] hotspot_gc_shenandoah ------------- Commit messages: - JDK-8265995 Changes: https://git.openjdk.java.net/jdk/pull/3701/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3701&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265995 Stats: 49 lines in 4 files changed: 24 ins; 25 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3701.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3701/head:pull/3701 PR: https://git.openjdk.java.net/jdk/pull/3701 From ayang at openjdk.java.net Mon Apr 26 21:23:36 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Mon, 26 Apr 2021 21:23:36 GMT Subject: RFR: 8184134: HeapRegion::LogOfHRGrainWords is unused [v2] In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 08:56:53 GMT, Albert Mingkun Yang wrote: >> Trivial change of removing unused code. > > Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: > > review Thanks for the review. ------------- PR: https://git.openjdk.java.net/jdk/pull/3671 From ayang at openjdk.java.net Mon Apr 26 21:23:36 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Mon, 26 Apr 2021 21:23:36 GMT Subject: Integrated: 8184134: HeapRegion::LogOfHRGrainWords is unused In-Reply-To: References: Message-ID: <8pHBBtiI3iWI9ytCSvj2u2Nt5BPs0J_40sVb6vGS0OI=.0ce592a4-c1d6-46c7-a083-3b08627ccbb6@github.com> On Sun, 25 Apr 2021 08:26:07 GMT, Albert Mingkun Yang wrote: > Trivial change of removing unused code. This pull request has now been integrated. Changeset: cb29b19f Author: Albert Mingkun Yang URL: https://git.openjdk.java.net/jdk/commit/cb29b19f Stats: 11 lines in 2 files changed: 0 ins; 7 del; 4 mod 8184134: HeapRegion::LogOfHRGrainWords is unused Reviewed-by: kbarrett, sjohanss ------------- PR: https://git.openjdk.java.net/jdk/pull/3671 From jiefu at openjdk.java.net Mon Apr 26 23:25:54 2021 From: jiefu at openjdk.java.net (Jie Fu) Date: Mon, 26 Apr 2021 23:25:54 GMT Subject: RFR: 8241354: ZGC still crashes in containers with NUMA due to get_mempolicy is disabled by default Message-ID: Hi all, ZGC still crashes in containers hosting on NUMA machines due to an fatal error [1], which really limits the usage of zgc. The reason is that get_mempolicy is blocked by default in dockers [2]. With the default docker configuration, ZGC will always crash with NUMA. Other collectors are also somehow affected by this restriction. But unlike zgc, they won't crash and can still work in dockers. So we'd better fix this bug as soon as possible since more and more people would like to use zgc. The patch was prepared under the guidance of StefanK [3] and many thanks to him. And we'd like to provide a user friendly -Xlog:gc+init log in a separate RFE after this issue. Testing: - tier1 ~ tier3 on Linux/x64 Thanks. Best regards, Jie [1] https://github.com/openjdk/jdk/blob/master/src/hotspot/os/linux/gc/z/zNUMA_linux.cpp#L64 [2] https://docs.docker.com/engine/security/seccomp/ [3] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028927.html ------------- Commit messages: - 8241354: ZGC still crashes in containers with NUMA due to get_mempolicy is disabled by default Changes: https://git.openjdk.java.net/jdk/pull/3704/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3704&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8241354 Stats: 17 lines in 1 file changed: 15 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3704.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3704/head:pull/3704 PR: https://git.openjdk.java.net/jdk/pull/3704 From qingfeng.yy at alibaba-inc.com Tue Apr 27 02:26:12 2021 From: qingfeng.yy at alibaba-inc.com (Yi Yang) Date: Tue, 27 Apr 2021 10:26:12 +0800 Subject: =?UTF-8?B?UmU6IFJGUjogODI2NTMyMjogQzI6IFNpbXBsaWZ5IGNvbnRyb2wgaW5wdXRzIGZvciBCYXJy?= =?UTF-8?B?aWVyU2V0QzI6Om9ial9hbGxvY2F0ZQ==?= In-Reply-To: <_NicB3VpqFwTiu2-bKLNhfnQAxJmbtND66g3Md5ck5g=.f9b4edcd-fa2b-4917-9092-bccc38b202df@github.com> References: <_NicB3VpqFwTiu2-bKLNhfnQAxJmbtND66g3Md5ck5g=.f9b4edcd-fa2b-4917-9092-bccc38b202df@github.com> Message-ID: <1320ae9b-f7c1-480b-bdda-5ca223fc84bf.qingfeng.yy@alibaba-inc.com> Gentle Ping :- Can I have a second review of this patch? Thanks,Yang ------------------------------------------------------------------ From:Yi Yang Send Time:2021 Apr. 16 (Fri.) 11:16 To:hotspot-compiler-dev Subject:RFR: 8265322: C2: Simplify control inputs for BarrierSetC2::obj_allocate This PR simplifies control inputs for BarrierSetC2::obj_allocate. In most cases, it doesn't change anything since `toobig_false` is equivalent to `ctrl`. In rare case, `toobig_false` is created for Unsafe.allocateInstance while instance size is not statically known, `ctrl` would become control input of IfNode whose projects are `toobig_false` and `toobig_true`, old eden_end and old_eden_top can simply accept `toobig_false` as their control input rather than `ctrl`. ------------- Commit messages: - simplify Changes: https://git.openjdk.java.net/jdk/pull/3529/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3529&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265322 Stats: 5 lines in 3 files changed: 0 ins; 0 del; 5 mod Patch: https://git.openjdk.java.net/jdk/pull/3529.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3529/head:pull/3529 PR: https://git.openjdk.java.net/jdk/pull/3529 From shade at openjdk.java.net Tue Apr 27 06:15:36 2021 From: shade at openjdk.java.net (Aleksey Shipilev) Date: Tue, 27 Apr 2021 06:15:36 GMT Subject: RFR: 8265995: Shenandoah: Move ShenandoahInitMarkRootsClosure close to its use In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 17:55:41 GMT, Zhengyu Gu wrote: > ShenandoahInitMarkRootsClosure is only used by STW mark, let's move it there. > > Test: > - [x] hotspot_gc_shenandoah Looks fine. ------------- Marked as reviewed by shade (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3701 From tschatzl at openjdk.java.net Tue Apr 27 08:57:04 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 27 Apr 2021 08:57:04 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask [v4] In-Reply-To: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: > [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). > > To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. > > This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. > > [Here](https://github.com/openjdk/jdk/compare/master...tschatzl:parallel-phases-join-everything) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). > > Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: Improve documentation ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3653/files - new: https://git.openjdk.java.net/jdk/pull/3653/files/712b0c6f..7b616382 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=02-03 Stats: 19 lines in 1 file changed: 11 ins; 0 del; 8 mod Patch: https://git.openjdk.java.net/jdk/pull/3653.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3653/head:pull/3653 PR: https://git.openjdk.java.net/jdk/pull/3653 From sjohanss at openjdk.java.net Tue Apr 27 09:26:36 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Tue, 27 Apr 2021 09:26:36 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask In-Reply-To: References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: On Fri, 23 Apr 2021 13:33:51 GMT, Thomas Schatzl wrote: > This makes use a bit more clumsy that expected, see the `G1CollectedHeap::post_evacuate_cleanup_1/2` methods: > > ``` > void G1CollectedHeap::post_evacuate_cleanup_1(G1ParScanThreadStateSet* per_thread_states, > G1RedirtyCardsQueueSet* rdcqs) { > Ticks start = Ticks::now(); > { > G1PostEvacuateCollectionSetCleanupTask1 cl(per_thread_states, rdcqs); > uint num_threads = MAX2(1u, MIN2(cl.num_busy_workers(), workers()->active_workers())); > cl.set_max_workers(num_threads); > workers()->run_task(&cl, num_threads); > } > [...] > } > ``` > > i.e. since the constructor is used for setup, there is some small code duplication necessary to time execution of such a `G1BatchedGangTask`. @walulyai suggested (similar to other hotspot code) to introduce a `setup` method, but no `teardown`. > > There is no reason for me to not to, any concerns or other ideas? I'm not sure I exactly understand how `setup()` would be used, but a `run_batched_task(...)` might help some of the duplication: void G1CollectedHeap::run_batched_task(G1BatchedGangTask* task) { uint num_threads = MAX2(1u, MIN2(cl.num_busy_workers(), workers()->active_workers())); task.set_max_workers(num_threads); workers()->run_task(task); } I assume the work in the constructor and destructor should be measured, since it is in the example, otherwise the timing could be added to the helper as well. Basically the only other question/concern I have after looking at the patch is if we can some how get rid or change `num_busy_workers()`. I guess we might need some way to estimate the needed number of workers but does it have to return a `double`. To me it is strange that something can require a fraction of a worker. I understand that certain task are very small and for those we could consider returning 0. Or would that cause other problems? ------------- PR: https://git.openjdk.java.net/jdk/pull/3653 From tschatzl at openjdk.java.net Tue Apr 27 09:39:08 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 27 Apr 2021 09:39:08 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask [v5] In-Reply-To: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: > [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). > > To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. > > This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. > > [Here](https://github.com/openjdk/jdk/compare/master...tschatzl:parallel-phases-join-everything) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). > > Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: Fix whitespace once more ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3653/files - new: https://git.openjdk.java.net/jdk/pull/3653/files/7b616382..fb6a96b3 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=03-04 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3653.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3653/head:pull/3653 PR: https://git.openjdk.java.net/jdk/pull/3653 From tschatzl at openjdk.java.net Tue Apr 27 11:27:06 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 27 Apr 2021 11:27:06 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask [v6] In-Reply-To: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: > [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). > > To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. > > This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. > > [Here](https://github.com/openjdk/jdk/compare/master...tschatzl:parallel-phases-join-everything) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). > > Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: More documentation about usage, added one memory_order_relaxed ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3653/files - new: https://git.openjdk.java.net/jdk/pull/3653/files/fb6a96b3..b08fa575 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=04-05 Stats: 12 lines in 2 files changed: 9 ins; 0 del; 3 mod Patch: https://git.openjdk.java.net/jdk/pull/3653.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3653/head:pull/3653 PR: https://git.openjdk.java.net/jdk/pull/3653 From lkorinth at openjdk.java.net Tue Apr 27 11:58:44 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Tue, 27 Apr 2021 11:58:44 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask [v5] In-Reply-To: References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: On Tue, 27 Apr 2021 09:39:08 GMT, Thomas Schatzl wrote: >> [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). >> >> To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. >> >> This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. >> >> [Here](https://github.com/openjdk/jdk/compare/master...tschatzl:parallel-phases-join-everything) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). >> >> Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > Fix whitespace once more Looks good to me. src/hotspot/share/gc/g1/g1BatchedGangTask.hpp line 35: > 33: class GrowableArrayCHeap; > 34: > 35: // G1AbstractSubTask represents a task to be performed either within an within an -> within a src/hotspot/share/gc/g1/g1BatchedGangTask.hpp line 66: > 64: // as long as to amortize worker startup costs. > 65: // Called by G1BatchedGangTask to determine total number of workers. > 66: virtual double num_busy_workers() const { return 1.0; } maybe make pure virtual? src/hotspot/share/gc/g1/g1BatchedGangTask.hpp line 86: > 84: // of "serial" or the list of "parallel" tasks. > 85: // During execution in the work gang, this class will make sure that the "serial" > 86: // tasks are executed by a single worker only exactly once, but different "serial" worker only exactly once -> worker exactly once src/hotspot/share/gc/g1/g1BatchedGangTask.hpp line 89: > 87: // tasks may be executed in parallel using different workers. "Parallel" tasks' > 88: // do_work() method may be called by different workers passing a different > 89: // worker_id at the same time, but at most once per given worker id. worker id -> worker_id (I hope to one day rename worker_id to work_id, work_unit, or something else, but if we are not referring to the variable, we should not describe it as a worker id, as it does not describe a worker but a unit of work. ------------- Marked as reviewed by lkorinth (Committer). PR: https://git.openjdk.java.net/jdk/pull/3653 From lkorinth at openjdk.java.net Tue Apr 27 12:28:45 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Tue, 27 Apr 2021 12:28:45 GMT Subject: RFR: 8231672: Simplify the reference processing parallelization framework [v5] In-Reply-To: References: Message-ID: On Sat, 24 Apr 2021 09:30:17 GMT, Albert Mingkun Yang wrote: >> Leo Korinth has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits: >> >> - make RefProcTask a StackObj >> - Merge branch 'master' into _8231672 >> - using proxies instead of ClosureContexts >> - remove maybe prefix, change in one place to nullable >> - Merge branch 'master' into _8231672 >> - Changes after first review >> - 8231672: Simplify the reference processing parallelization framework > > src/hotspot/share/gc/shared/referenceProcessor.cpp line 786: > >> 784: >> 785: proxy_task.prepare_run_task(task, num_queues(), processing_is_mt() ? RefProcThreadModel::Multi : RefProcThreadModel::Single, marks_oops_alive); >> 786: if (gang != NULL && processing_is_mt()) { > > Since `gang` is only used in thin branch, moving it inside `if` might be better. Besides, the assertion on L779 could be replaced by the code below, IMO. > > if (processing_is_mt()) { > WorkGang* gang = Universe::heap()->safepoint_workers(); > assert(gang != nullptr, "..."); > ... > } Yes! ------------- PR: https://git.openjdk.java.net/jdk/pull/2782 From lkorinth at openjdk.java.net Tue Apr 27 12:36:42 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Tue, 27 Apr 2021 12:36:42 GMT Subject: RFR: 8231672: Simplify the reference processing parallelization framework [v5] In-Reply-To: References: Message-ID: On Sat, 24 Apr 2021 09:32:44 GMT, Albert Mingkun Yang wrote: >> Leo Korinth has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits: >> >> - make RefProcTask a StackObj >> - Merge branch 'master' into _8231672 >> - using proxies instead of ClosureContexts >> - remove maybe prefix, change in one place to nullable >> - Merge branch 'master' into _8231672 >> - Changes after first review >> - 8231672: Simplify the reference processing parallelization framework > > src/hotspot/share/gc/parallel/psCompactionManager.hpp line 42: > >> 40: >> 41: class ParCompactionManager : public CHeapObj { >> 42: friend class CompactionWithStealingTask; > > `CompactionWithStealingTask` and `UpdateAndFillClosure` are dead code. I will remove them. > src/hotspot/share/gc/shared/referenceProcessor.hpp line 591: > >> 589: enum class RefProcThreadModel { Multi, Single }; >> 590: >> 591: class RefProcTask : StackObj { > > Maybe put some comments here, explaining what `RefProcTask` and `RefProcProxyTask` are, their interaction/relation. I will try to do that. ------------- PR: https://git.openjdk.java.net/jdk/pull/2782 From zgu at openjdk.java.net Tue Apr 27 12:43:36 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Tue, 27 Apr 2021 12:43:36 GMT Subject: Integrated: 8265995: Shenandoah: Move ShenandoahInitMarkRootsClosure close to its use In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 17:55:41 GMT, Zhengyu Gu wrote: > ShenandoahInitMarkRootsClosure is only used by STW mark, let's move it there. > > Test: > - [x] hotspot_gc_shenandoah This pull request has now been integrated. Changeset: 9adbf156 Author: Zhengyu Gu URL: https://git.openjdk.java.net/jdk/commit/9adbf156 Stats: 49 lines in 4 files changed: 24 ins; 25 del; 0 mod 8265995: Shenandoah: Move ShenandoahInitMarkRootsClosure close to its use Reviewed-by: shade ------------- PR: https://git.openjdk.java.net/jdk/pull/3701 From lkorinth at openjdk.java.net Tue Apr 27 12:54:47 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Tue, 27 Apr 2021 12:54:47 GMT Subject: RFR: 8231672: Simplify the reference processing parallelization framework [v5] In-Reply-To: References: Message-ID: On Sat, 24 Apr 2021 09:45:28 GMT, Albert Mingkun Yang wrote: >> Leo Korinth has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits: >> >> - make RefProcTask a StackObj >> - Merge branch 'master' into _8231672 >> - using proxies instead of ClosureContexts >> - remove maybe prefix, change in one place to nullable >> - Merge branch 'master' into _8231672 >> - Changes after first review >> - 8231672: Simplify the reference processing parallelization framework > > src/hotspot/share/gc/shared/referenceProcessor.cpp line 224: > >> 222: >> 223: { >> 224: RefProcTotalPhaseTimesTracker tt(RefPhase1, &phase_times, this); > > `RefProcTotalPhaseTimesTracker` doesn't really use `this`, right? Nice catch! Pre-existing, but I will clean it up. ------------- PR: https://git.openjdk.java.net/jdk/pull/2782 From lkorinth at openjdk.java.net Tue Apr 27 13:14:42 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Tue, 27 Apr 2021 13:14:42 GMT Subject: RFR: 8231672: Simplify the reference processing parallelization framework [v5] In-Reply-To: References: Message-ID: On Sat, 24 Apr 2021 09:43:11 GMT, Albert Mingkun Yang wrote: >> Leo Korinth has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains seven commits: >> >> - make RefProcTask a StackObj >> - Merge branch 'master' into _8231672 >> - using proxies instead of ClosureContexts >> - remove maybe prefix, change in one place to nullable >> - Merge branch 'master' into _8231672 >> - Changes after first review >> - 8231672: Simplify the reference processing parallelization framework > > src/hotspot/share/gc/shared/referenceProcessor.hpp line 631: > >> 629: uint index(uint id) const { >> 630: return (_tm == RefProcThreadModel::Single) ? 0 : id; >> 631: } > > This method is only used by G1; I wonder if it's possible to not include it in the API of `RefProcProxyTask`. I could --- and maybe should --- but then I need to supply `_tm` as an argument, further I need to find a logical place to put the function and the declaration, and if I add it to `referenceProcessor.?pp` I have not made it less generic. If I add it to a G1 file, it seems to me arbitrary which to use. If you find a nice place to put it, I will move it, otherwise I will keep it where it is. I will not create a new file for the function and would in that case prefer to inline expand the code. ------------- PR: https://git.openjdk.java.net/jdk/pull/2782 From eosterlund at openjdk.java.net Tue Apr 27 13:48:45 2021 From: eosterlund at openjdk.java.net (Erik =?UTF-8?B?w5ZzdGVybHVuZA==?=) Date: Tue, 27 Apr 2021 13:48:45 GMT Subject: RFR: 8266055: ZGC: ZHeap::print_extended_on() doesn't disable deferred delete In-Reply-To: References: Message-ID: On Tue, 27 Apr 2021 13:42:02 GMT, Per Liden wrote: > In `ZHeap::print_extended_on()`, the last call to `_page_allocator.enable_deferred_delete()` should be `_page_allocator.disable_deferred_delete()`. Looks fantastic. ------------- Marked as reviewed by eosterlund (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3717 From pliden at openjdk.java.net Tue Apr 27 13:48:45 2021 From: pliden at openjdk.java.net (Per Liden) Date: Tue, 27 Apr 2021 13:48:45 GMT Subject: RFR: 8266055: ZGC: ZHeap::print_extended_on() doesn't disable deferred delete Message-ID: In `ZHeap::print_extended_on()`, the last call to `_page_allocator.enable_deferred_delete()` should be `_page_allocator.disable_deferred_delete()`. ------------- Commit messages: - 8266055: ZGC: ZHeap::print_extended_on() doesn't disable deferred delete Changes: https://git.openjdk.java.net/jdk/pull/3717/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3717&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8266055 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3717.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3717/head:pull/3717 PR: https://git.openjdk.java.net/jdk/pull/3717 From lkorinth at openjdk.java.net Tue Apr 27 13:54:15 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Tue, 27 Apr 2021 13:54:15 GMT Subject: RFR: 8231672: Simplify the reference processing parallelization framework [v6] In-Reply-To: References: Message-ID: > With the change of parallel gc to use the same workgang implementation that other gcs uses, it is possible to remove the abstraction used to hide different worker thread management implementations (TaskExecutor). > > At the same time as removing the TaskExecutor, also remove part of the special casing (mostly code duplication), for single threaded execution. > > I consider the new method `prepare_run_task` that modifies the state of the context a step backwards from what was. However, I think removing the Executor with its proxy-tasks and removing separate code paths for serial execution makes the change worth this step back. We also removes ~270 lines of code. > > Some comments: > 1) I use references in some places where they can replace pointers. I could go much further here, but I did not want to bloat the pull request, maybe later change all closures to be references as well? Should I create enhancements for this? > > 2) I added an enum class ThreadModel instead of using a boolean, this could also be used in many more places. I dislike sending lots of bools with a comment like `true /* _is_mt */`. It also adds type safety if a method takes many bools. However, with this limited change, and not many hard-coded bools, it feels a bit overkill and I am willing to remove the enum, what do you think? > > Testing: > hotspot_gc and tier 1-3 has passed earlier versions before minor cleanups. I intend to re-run tests after review changes. Leo Korinth has updated the pull request incrementally with one additional commit since the last revision: Albert ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/2782/files - new: https://git.openjdk.java.net/jdk/pull/2782/files/fbdbe8c6..8ad0304c Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2782&range=05 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2782&range=04-05 Stats: 29 lines in 5 files changed: 14 ins; 7 del; 8 mod Patch: https://git.openjdk.java.net/jdk/pull/2782.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2782/head:pull/2782 PR: https://git.openjdk.java.net/jdk/pull/2782 From tschatzl at openjdk.java.net Tue Apr 27 14:11:36 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 27 Apr 2021 14:11:36 GMT Subject: RFR: 8234446: Post-CMS workgroup hierarchy cleanup In-Reply-To: References: Message-ID: On Fri, 23 Apr 2021 20:06:03 GMT, Albert Mingkun Yang wrote: > Collapse workgroup hierarchy and some general cleanup. > > Tested; hotspot_gc Mostly further cleanup suggestions. src/hotspot/share/gc/shared/workerManager.hpp line 53: > 51: // If initializing = false, there are already some number of worker > 52: // threads and a failure would not be optimal but should not be fatal. > 53: static uint add_workers (WorkGang* holder, s/holder/workers src/hotspot/share/gc/shared/workerManager.hpp line 61: > 59: > 60: // Log (at trace level) a change in the number of created workers. > 61: static void log_worker_creation(WorkGang* holder, s/holder/workers src/hotspot/share/gc/shared/workgroup.cpp line 133: > 131: > 132: WorkGang::WorkGang(const char* name, uint workers, bool are_GC_task_threads, bool are_ConcurrentGC_threads) : > 133: _workers(NULL), Pre-existing: the indentation is completely arbitrary here. Please use something like 2 spaces. src/hotspot/share/gc/shared/workgroup.hpp line 84: > 82: // The work gang is the collection of workers to execute tasks. > 83: // The number of workers run for a task is "_active_workers" > 84: // while "_total_workers" is the number of available of workers. Pre-existing: The second "of" is one too many. src/hotspot/share/gc/shared/workgroup.hpp line 86: > 84: // while "_total_workers" is the number of available of workers. > 85: class WorkGang : public CHeapObj { > 86: private: The `private` can be removed. ------------- Changes requested by tschatzl (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3663 From ayang at openjdk.java.net Tue Apr 27 14:25:56 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Tue, 27 Apr 2021 14:25:56 GMT Subject: RFR: 8234446: Post-CMS workgroup hierarchy cleanup [v2] In-Reply-To: References: Message-ID: > Collapse workgroup hierarchy and some general cleanup. > > Tested; hotspot_gc Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: review ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3663/files - new: https://git.openjdk.java.net/jdk/pull/3663/files/ea674346..a194f5ff Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3663&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3663&range=00-01 Stats: 17 lines in 3 files changed: 0 ins; 1 del; 16 mod Patch: https://git.openjdk.java.net/jdk/pull/3663.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3663/head:pull/3663 PR: https://git.openjdk.java.net/jdk/pull/3663 From ayang at openjdk.java.net Tue Apr 27 14:31:38 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Tue, 27 Apr 2021 14:31:38 GMT Subject: RFR: 8266055: ZGC: ZHeap::print_extended_on() doesn't disable deferred delete In-Reply-To: References: Message-ID: On Tue, 27 Apr 2021 13:42:02 GMT, Per Liden wrote: > In `ZHeap::print_extended_on()`, the last call to `_page_allocator.enable_deferred_delete()` should be `_page_allocator.disable_deferred_delete()`. Marked as reviewed by ayang (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3717 From tschatzl at openjdk.java.net Tue Apr 27 14:32:10 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Tue, 27 Apr 2021 14:32:10 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask [v7] In-Reply-To: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: > [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). > > To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. > > This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. > > [Here](https://github.com/openjdk/jdk/compare/master...tschatzl:parallel-phases-join-everything) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). > > Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: lkorinth review ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3653/files - new: https://git.openjdk.java.net/jdk/pull/3653/files/b08fa575..3c8d04a5 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=06 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=05-06 Stats: 6 lines in 1 file changed: 0 ins; 2 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/3653.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3653/head:pull/3653 PR: https://git.openjdk.java.net/jdk/pull/3653 From ayang at openjdk.java.net Tue Apr 27 15:41:42 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Tue, 27 Apr 2021 15:41:42 GMT Subject: RFR: 8231672: Simplify the reference processing parallelization framework [v5] In-Reply-To: References: Message-ID: <9uzbwDoXClDVWwnxXi-aKFad8-vRgtmOsCDTDKfyHuw=.b2853d0b-c3b4-449c-ad08-1e49745ea7d4@github.com> On Tue, 27 Apr 2021 13:11:38 GMT, Leo Korinth wrote: >> src/hotspot/share/gc/shared/referenceProcessor.hpp line 631: >> >>> 629: uint index(uint id) const { >>> 630: return (_tm == RefProcThreadModel::Single) ? 0 : id; >>> 631: } >> >> This method is only used by G1; I wonder if it's possible to not include it in the API of `RefProcProxyTask`. > > I could --- and maybe should --- but then I need to supply `_tm` as an argument, further I need to find a logical place to put the function and the declaration, and if I add it to `referenceProcessor.?pp` I have not made it less generic. If I add it to a G1 file, it seems to me arbitrary which to use. If you find a nice place to put it, I will move it, otherwise I will keep it where it is. I will not create a new file for the function and would in that case prefer to inline expand the code. Since the body of `uint index(uint id) const` is just one line, I was thinking the caller can just inline it. Taking `G1FullGCRefProcProxyTask` as an example. before G1FullKeepAliveClosure keep_alive(_collector.marker(index(worker_id))); G1FollowStackClosure* complete_gc = _collector.marker(index(worker_id))->stack_closure(); after auto index = _tm == RefProcThreadModel::Single ? 0 : worker_id; auto marker = _collector.marker(index); G1FullKeepAliveClosure keep_alive(marker); G1FollowStackClosure* complete_gc = marker->stack_closure(); My concern with including `uint index(uint id) const` in the API of `RefProcProxyTask` is that its name/implementation is too generic. In the context of `RefProcProxyTask`, it's not obvious what `id` is, what `index` should return, and why `(_tm == RefProcThreadModel::Single) ? 0 : id;` is correct. ------------- PR: https://git.openjdk.java.net/jdk/pull/2782 From zgu at openjdk.java.net Tue Apr 27 18:42:45 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Tue, 27 Apr 2021 18:42:45 GMT Subject: RFR: 8266083: Shenandoah: Consolidate dedup/no dedup oop closures Message-ID: We can use template to consolidate dedup/no dedup oop closures. ------------- Commit messages: - update - init Changes: https://git.openjdk.java.net/jdk/pull/3724/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3724&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8266083 Stats: 90 lines in 3 files changed: 13 ins; 53 del; 24 mod Patch: https://git.openjdk.java.net/jdk/pull/3724.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3724/head:pull/3724 PR: https://git.openjdk.java.net/jdk/pull/3724 From lkorinth at openjdk.java.net Tue Apr 27 22:06:29 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Tue, 27 Apr 2021 22:06:29 GMT Subject: RFR: 8231672: Simplify the reference processing parallelization framework [v7] In-Reply-To: References: Message-ID: > With the change of parallel gc to use the same workgang implementation that other gcs uses, it is possible to remove the abstraction used to hide different worker thread management implementations (TaskExecutor). > > At the same time as removing the TaskExecutor, also remove part of the special casing (mostly code duplication), for single threaded execution. > > I consider the new method `prepare_run_task` that modifies the state of the context a step backwards from what was. However, I think removing the Executor with its proxy-tasks and removing separate code paths for serial execution makes the change worth this step back. We also removes ~270 lines of code. > > Some comments: > 1) I use references in some places where they can replace pointers. I could go much further here, but I did not want to bloat the pull request, maybe later change all closures to be references as well? Should I create enhancements for this? > > 2) I added an enum class ThreadModel instead of using a boolean, this could also be used in many more places. I dislike sending lots of bools with a comment like `true /* _is_mt */`. It also adds type safety if a method takes many bools. However, with this limited change, and not many hard-coded bools, it feels a bit overkill and I am willing to remove the enum, what do you think? > > Testing: > hotspot_gc and tier 1-3 has passed earlier versions before minor cleanups. I intend to re-run tests after review changes. Leo Korinth has updated the pull request incrementally with one additional commit since the last revision: inline index ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/2782/files - new: https://git.openjdk.java.net/jdk/pull/2782/files/8ad0304c..4d41ed82 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2782&range=06 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2782&range=05-06 Stats: 14 lines in 4 files changed: 3 ins; 4 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/2782.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2782/head:pull/2782 PR: https://git.openjdk.java.net/jdk/pull/2782 From lkorinth at openjdk.java.net Tue Apr 27 22:10:13 2021 From: lkorinth at openjdk.java.net (Leo Korinth) Date: Tue, 27 Apr 2021 22:10:13 GMT Subject: RFR: 8231672: Simplify the reference processing parallelization framework [v5] In-Reply-To: <9uzbwDoXClDVWwnxXi-aKFad8-vRgtmOsCDTDKfyHuw=.b2853d0b-c3b4-449c-ad08-1e49745ea7d4@github.com> References: <9uzbwDoXClDVWwnxXi-aKFad8-vRgtmOsCDTDKfyHuw=.b2853d0b-c3b4-449c-ad08-1e49745ea7d4@github.com> Message-ID: On Tue, 27 Apr 2021 15:39:02 GMT, Albert Mingkun Yang wrote: >> I could --- and maybe should --- but then I need to supply `_tm` as an argument, further I need to find a logical place to put the function and the declaration, and if I add it to `referenceProcessor.?pp` I have not made it less generic. If I add it to a G1 file, it seems to me arbitrary which to use. If you find a nice place to put it, I will move it, otherwise I will keep it where it is. I will not create a new file for the function and would in that case prefer to inline expand the code. > > Since the body of `uint index(uint id) const` is just one line, I was thinking the caller can just inline it. Taking `G1FullGCRefProcProxyTask` as an example. > > before > > G1FullKeepAliveClosure keep_alive(_collector.marker(index(worker_id))); > G1FollowStackClosure* complete_gc = _collector.marker(index(worker_id))->stack_closure(); > > > after > > auto index = _tm == RefProcThreadModel::Single ? 0 : worker_id; > auto marker = _collector.marker(index); > G1FullKeepAliveClosure keep_alive(marker); > G1FollowStackClosure* complete_gc = marker->stack_closure(); > > > My concern with including `uint index(uint id) const` in the API of `RefProcProxyTask` is that its name/implementation is too generic. In the context of `RefProcProxyTask`, it's not obvious what `id` is, what `index` should return, and why `(_tm == RefProcThreadModel::Single) ? 0 : id;` is correct. I inlined it. ------------- PR: https://git.openjdk.java.net/jdk/pull/2782 From pliden at openjdk.java.net Wed Apr 28 05:52:55 2021 From: pliden at openjdk.java.net (Per Liden) Date: Wed, 28 Apr 2021 05:52:55 GMT Subject: Integrated: 8266055: ZGC: ZHeap::print_extended_on() doesn't disable deferred delete In-Reply-To: References: Message-ID: On Tue, 27 Apr 2021 13:42:02 GMT, Per Liden wrote: > In `ZHeap::print_extended_on()`, the last call to `_page_allocator.enable_deferred_delete()` should be `_page_allocator.disable_deferred_delete()`. This pull request has now been integrated. Changeset: 2201e11c Author: Per Liden URL: https://git.openjdk.java.net/jdk/commit/2201e11cbfc506d986cd4adedc78b83ee497cbb8 Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod 8266055: ZGC: ZHeap::print_extended_on() doesn't disable deferred delete Reviewed-by: eosterlund, ayang ------------- PR: https://git.openjdk.java.net/jdk/pull/3717 From pliden at openjdk.java.net Wed Apr 28 05:52:55 2021 From: pliden at openjdk.java.net (Per Liden) Date: Wed, 28 Apr 2021 05:52:55 GMT Subject: RFR: 8266055: ZGC: ZHeap::print_extended_on() doesn't disable deferred delete In-Reply-To: References: Message-ID: On Tue, 27 Apr 2021 13:43:09 GMT, Erik ?sterlund wrote: >> In `ZHeap::print_extended_on()`, the last call to `_page_allocator.enable_deferred_delete()` should be `_page_allocator.disable_deferred_delete()`. > > Looks fantastic. Thanks @fisk and @albertnetymk for reviewing! ------------- PR: https://git.openjdk.java.net/jdk/pull/3717 From ayang at openjdk.java.net Wed Apr 28 06:36:57 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Wed, 28 Apr 2021 06:36:57 GMT Subject: RFR: 8231672: Simplify the reference processing parallelization framework [v7] In-Reply-To: References: Message-ID: On Tue, 27 Apr 2021 22:06:29 GMT, Leo Korinth wrote: >> With the change of parallel gc to use the same workgang implementation that other gcs uses, it is possible to remove the abstraction used to hide different worker thread management implementations (TaskExecutor). >> >> At the same time as removing the TaskExecutor, also remove part of the special casing (mostly code duplication), for single threaded execution. >> >> I consider the new method `prepare_run_task` that modifies the state of the context a step backwards from what was. However, I think removing the Executor with its proxy-tasks and removing separate code paths for serial execution makes the change worth this step back. We also removes ~270 lines of code. >> >> Some comments: >> 1) I use references in some places where they can replace pointers. I could go much further here, but I did not want to bloat the pull request, maybe later change all closures to be references as well? Should I create enhancements for this? >> >> 2) I added an enum class ThreadModel instead of using a boolean, this could also be used in many more places. I dislike sending lots of bools with a comment like `true /* _is_mt */`. It also adds type safety if a method takes many bools. However, with this limited change, and not many hard-coded bools, it feels a bit overkill and I am willing to remove the enum, what do you think? >> >> Testing: >> hotspot_gc and tier 1-3 has passed earlier versions before minor cleanups. I intend to re-run tests after review changes. > > Leo Korinth has updated the pull request incrementally with one additional commit since the last revision: > > inline index Thank you for the revision. ------------- Marked as reviewed by ayang (Committer). PR: https://git.openjdk.java.net/jdk/pull/2782 From pliden at openjdk.java.net Wed Apr 28 07:24:51 2021 From: pliden at openjdk.java.net (Per Liden) Date: Wed, 28 Apr 2021 07:24:51 GMT Subject: RFR: 8241354: ZGC still crashes in containers with NUMA due to get_mempolicy is disabled by default In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 23:19:46 GMT, Jie Fu wrote: > Hi all, > > ZGC still crashes in containers hosting on NUMA machines due to an fatal error [1], which really limits the usage of zgc. > > The reason is that get_mempolicy is blocked by default in dockers [2]. > With the default docker configuration, ZGC will always crash with NUMA. > > Other collectors are also somehow affected by this restriction. > But unlike zgc, they won't crash and can still work in dockers. > So we'd better fix this bug as soon as possible since more and more people would like to use zgc. > > The patch was prepared under the guidance of StefanK [3] and many thanks to him. > And we'd like to provide a user friendly -Xlog:gc+init log in a separate RFE after this issue. > > Testing: > - tier1 ~ tier3 on Linux/x64 > > Thanks. > Best regards, > Jie > > [1] https://github.com/openjdk/jdk/blob/master/src/hotspot/os/linux/gc/z/zNUMA_linux.cpp#L64 > [2] https://docs.docker.com/engine/security/seccomp/ > [3] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028927.html The approach looks good. However, I'd like to suggest the following adjustments https://github.com/pliden/jdk/commit/9cff734de0f03b4990cab55c7052e3e7f3b6a8f1 on top of this PR, which breaks out the syscall into a separate function and renames a few things. ------------- Changes requested by pliden (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3704 From tschatzl at openjdk.java.net Wed Apr 28 08:06:51 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Wed, 28 Apr 2021 08:06:51 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask In-Reply-To: References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: <4fkP4AR8oGWMjRvp8bL3cCPdZY1iMJ1EC2kCIq9YWLE=.c5f9f93a-a6e6-49c9-8120-11507752cbe0@github.com> On Tue, 27 Apr 2021 09:24:08 GMT, Stefan Johansson wrote: > [...] > I'm not sure I exactly understand how `setup()` would be used, but a `run_batched_task(...)` might help some of the duplication: > [...] Exactly that has been introduced in the follow-up change. >[...] > > Basically the only other question/concern I have after looking at the patch is if we can some how get rid or change `num_busy_workers()`. I guess we might need some way to estimate the needed number of workers but does it have to return a `double`. To me it is strange that something can require a fraction of a worker. I understand that certain task are very small and for those we could consider returning 0. Or would that cause other problems? Generally there are two interesting cases when running a gang task: using one (or maybe two) threads, and using all of them as indicated by `WorkGang::active_workers()`. Particularly with more sub tasks in a batch the latter will probably more common than now, so the "in-between" ones will be exceedingly rare. The case where we use just one worker is particularly interesting though: we could avoid spinning up a worker thread in this case completely, using the VM thread to execute the batch (all gang tasks at least in G1 support that at this time, we just do not do it). With an integer `num_busy_workers()` value it is very hard to achieve that (i.e. `sum(num_busy_workers()) <= 1.0` given such a batch task always contains at least few serial tasks. And if everyone returns 1 at minimum, there is no way this will work reasonably. Note that currently (and in that follow-up CR) particularly for these serial tasks the `num_busy_workers()` return value isn't particularly well set (and I just recently "fixed" one for a parallel task...) so that there is a good chance it will not work out of the box, but we haven't enabled using the VM thread anyway yet. And there is the question whether we can somehow either completely drop or move to concurrent some of that work; however in many cases pre/post evacuation work is very small compared to actual evacuation. Work for another day... ------------- PR: https://git.openjdk.java.net/jdk/pull/3653 From jiefu at openjdk.java.net Wed Apr 28 08:20:17 2021 From: jiefu at openjdk.java.net (Jie Fu) Date: Wed, 28 Apr 2021 08:20:17 GMT Subject: RFR: 8241354: ZGC still crashes in containers with NUMA due to get_mempolicy is disabled by default [v2] In-Reply-To: References: Message-ID: > Hi all, > > ZGC still crashes in containers hosting on NUMA machines due to an fatal error [1], which really limits the usage of zgc. > > The reason is that get_mempolicy is blocked by default in dockers [2]. > With the default docker configuration, ZGC will always crash with NUMA. > > Other collectors are also somehow affected by this restriction. > But unlike zgc, they won't crash and can still work in dockers. > So we'd better fix this bug as soon as possible since more and more people would like to use zgc. > > The patch was prepared under the guidance of StefanK [3] and many thanks to him. > And we'd like to provide a user friendly -Xlog:gc+init log in a separate RFE after this issue. > > Testing: > - tier1 ~ tier3 on Linux/x64 > > Thanks. > Best regards, > Jie > > [1] https://github.com/openjdk/jdk/blob/master/src/hotspot/os/linux/gc/z/zNUMA_linux.cpp#L64 > [2] https://docs.docker.com/engine/security/seccomp/ > [3] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028927.html Jie Fu has updated the pull request incrementally with one additional commit since the last revision: Add pliden's refinement ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3704/files - new: https://git.openjdk.java.net/jdk/pull/3704/files/eb3ac39b..c6f431a3 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3704&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3704&range=00-01 Stats: 14 lines in 1 file changed: 2 ins; 0 del; 12 mod Patch: https://git.openjdk.java.net/jdk/pull/3704.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3704/head:pull/3704 PR: https://git.openjdk.java.net/jdk/pull/3704 From jiefu at openjdk.java.net Wed Apr 28 08:20:17 2021 From: jiefu at openjdk.java.net (Jie Fu) Date: Wed, 28 Apr 2021 08:20:17 GMT Subject: RFR: 8241354: ZGC still crashes in containers with NUMA due to get_mempolicy is disabled by default In-Reply-To: References: Message-ID: On Mon, 26 Apr 2021 23:19:46 GMT, Jie Fu wrote: > Hi all, > > ZGC still crashes in containers hosting on NUMA machines due to an fatal error [1], which really limits the usage of zgc. > > The reason is that get_mempolicy is blocked by default in dockers [2]. > With the default docker configuration, ZGC will always crash with NUMA. > > Other collectors are also somehow affected by this restriction. > But unlike zgc, they won't crash and can still work in dockers. > So we'd better fix this bug as soon as possible since more and more people would like to use zgc. > > The patch was prepared under the guidance of StefanK [3] and many thanks to him. > And we'd like to provide a user friendly -Xlog:gc+init log in a separate RFE after this issue. > > Testing: > - tier1 ~ tier3 on Linux/x64 > > Thanks. > Best regards, > Jie > > [1] https://github.com/openjdk/jdk/blob/master/src/hotspot/os/linux/gc/z/zNUMA_linux.cpp#L64 > [2] https://docs.docker.com/engine/security/seccomp/ > [3] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028927.html > The approach looks good. However, I'd like to suggest the following adjustments [pliden at 9cff734](https://github.com/pliden/jdk/commit/9cff734de0f03b4990cab55c7052e3e7f3b6a8f1) on top of this PR, which breaks out the syscall into a separate function and renames a few things. Thanks @pliden for your review and help. Updated. ------------- PR: https://git.openjdk.java.net/jdk/pull/3704 From sjohanss at openjdk.java.net Wed Apr 28 10:34:53 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Wed, 28 Apr 2021 10:34:53 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask In-Reply-To: <4fkP4AR8oGWMjRvp8bL3cCPdZY1iMJ1EC2kCIq9YWLE=.c5f9f93a-a6e6-49c9-8120-11507752cbe0@github.com> References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> <4fkP4AR8oGWMjRvp8bL3cCPdZY1iMJ1EC2kCIq9YWLE=.c5f9f93a-a6e6-49c9-8120-11507752cbe0@github.com> Message-ID: On Wed, 28 Apr 2021 08:04:15 GMT, Thomas Schatzl wrote: > > [...] > > I'm not sure I exactly understand how `setup()` would be used, but a `run_batched_task(...)` might help some of the duplication: > > [...] > > Exactly that has been introduced in the follow-up change. > Nice! > > [...] > > Basically the only other question/concern I have after looking at the patch is if we can some how get rid or change `num_busy_workers()`. I guess we might need some way to estimate the needed number of workers but does it have to return a `double`. To me it is strange that something can require a fraction of a worker. I understand that certain task are very small and for those we could consider returning 0. Or would that cause other problems? > > Generally there are two interesting cases when running a gang task: using one (or maybe two) threads, and using all of them as indicated by `WorkGang::active_workers()`. Particularly with more sub tasks in a batch the latter will probably more common than now, so the "in-between" ones will be exceedingly rare. > > The case where we use just one worker is particularly interesting though: we could avoid spinning up a worker thread in this case completely, using the VM thread to execute the batch (all gang tasks at least in G1 support that at this time, we just do not do it). With an integer `num_busy_workers()` value it is very hard to achieve that (i.e. `sum(num_busy_workers()) <= 1.0` given such a batch task always contains at least few serial tasks. And if everyone returns 1 at minimum, there is no way this will work reasonably. > > Note that currently (and in that follow-up CR) particularly for these serial tasks the `num_busy_workers()` return value isn't particularly well set (and I just recently "fixed" one for a parallel task...) so that there is a good chance it will not work out of the box, but we haven't enabled using the VM thread anyway yet. > And there is the question whether we can somehow either completely drop or move to concurrent some of that work; however in many cases pre/post evacuation work is very small compared to actual evacuation. Work for another day... I see your points and agree there is a value in being able to have a way to define the cost of a sub-task. What I have the biggest problem with is that `num_busy_workers()` isn't a uint (which `num_workers` usually is). So maybe we could call it `double G1AbstractSubTask::cost()/worker_cost()` instead. It would be the same thing, the estimated amount of workers needed. To me it feels more natural that the cost can be a fraction. Maybe also rename `uint G1BatchedGangTask::num_busy_workers()` to `num_workers()` or `estimated_workers()`. Would this be ok with you? ------------- PR: https://git.openjdk.java.net/jdk/pull/3653 From neliasso at openjdk.java.net Wed Apr 28 12:58:55 2021 From: neliasso at openjdk.java.net (Nils Eliasson) Date: Wed, 28 Apr 2021 12:58:55 GMT Subject: RFR: 8265322: C2: Simplify control inputs for BarrierSetC2::obj_allocate In-Reply-To: <_NicB3VpqFwTiu2-bKLNhfnQAxJmbtND66g3Md5ck5g=.f9b4edcd-fa2b-4917-9092-bccc38b202df@github.com> References: <_NicB3VpqFwTiu2-bKLNhfnQAxJmbtND66g3Md5ck5g=.f9b4edcd-fa2b-4917-9092-bccc38b202df@github.com> Message-ID: On Fri, 16 Apr 2021 03:06:19 GMT, Yi Yang wrote: > While just using BarrierSetC2::obj_allocate in https://github.com/openjdk/valhalla/pull/385, I noticed the control input `ctrl` for obj_allocate is not so much necessary. This PR simplifies control inputs for BarrierSetC2::obj_allocate. In most cases, it doesn't change anything since `toobig_false` is equivalent to `ctrl`. In other cases, `toobig_false` is created for Unsafe.allocateInstance while instance size is not statically known, `ctrl` would become control input of IfNode whose projects are `toobig_false` and `toobig_true`, old eden_end and old_eden_top can simply accept `toobig_false` as their control input rather than `ctrl`. Looks good. ------------- Marked as reviewed by neliasso (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3529 From pliden at openjdk.java.net Wed Apr 28 13:22:51 2021 From: pliden at openjdk.java.net (Per Liden) Date: Wed, 28 Apr 2021 13:22:51 GMT Subject: RFR: 8241354: ZGC still crashes in containers with NUMA due to get_mempolicy is disabled by default [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 08:20:17 GMT, Jie Fu wrote: >> Hi all, >> >> ZGC still crashes in containers hosting on NUMA machines due to an fatal error [1], which really limits the usage of zgc. >> >> The reason is that get_mempolicy is blocked by default in dockers [2]. >> With the default docker configuration, ZGC will always crash with NUMA. >> >> Other collectors are also somehow affected by this restriction. >> But unlike zgc, they won't crash and can still work in dockers. >> So we'd better fix this bug as soon as possible since more and more people would like to use zgc. >> >> The patch was prepared under the guidance of StefanK [3] and many thanks to him. >> And we'd like to provide a user friendly -Xlog:gc+init log in a separate RFE after this issue. >> >> Testing: >> - tier1 ~ tier3 on Linux/x64 >> >> Thanks. >> Best regards, >> Jie >> >> [1] https://github.com/openjdk/jdk/blob/master/src/hotspot/os/linux/gc/z/zNUMA_linux.cpp#L64 >> [2] https://docs.docker.com/engine/security/seccomp/ >> [3] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028927.html > > Jie Fu has updated the pull request incrementally with one additional commit since the last revision: > > Add pliden's refinement Thanks for updating. Looks good. ------------- Marked as reviewed by pliden (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3704 From jiefu at openjdk.java.net Wed Apr 28 14:13:50 2021 From: jiefu at openjdk.java.net (Jie Fu) Date: Wed, 28 Apr 2021 14:13:50 GMT Subject: RFR: 8241354: ZGC still crashes in containers with NUMA due to get_mempolicy is disabled by default [v2] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 13:20:06 GMT, Per Liden wrote: > Thanks for updating. Looks good. Thanks @pliden . Will push it tomorrow if there is no objection. ------------- PR: https://git.openjdk.java.net/jdk/pull/3704 From ayang at openjdk.java.net Wed Apr 28 18:25:55 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Wed, 28 Apr 2021 18:25:55 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask [v7] In-Reply-To: References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: On Tue, 27 Apr 2021 14:32:10 GMT, Thomas Schatzl wrote: >> [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). >> >> To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. >> >> This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. >> >> [Here](https://github.com/openjdk/jdk/compare/master...tschatzl:parallel-phases-join-everything) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). >> >> Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > lkorinth review Marked as reviewed by ayang (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3653 From github.com+168222+mgkwill at openjdk.java.net Wed Apr 28 18:38:07 2021 From: github.com+168222+mgkwill at openjdk.java.net (Marcus G K Williams) Date: Wed, 28 Apr 2021 18:38:07 GMT Subject: RFR: 8256155: Allow multiple large page sizes to be used on Linux [v29] In-Reply-To: References: Message-ID: <7DAPl6-yypQPTFb6cc6hBTRxMGKFMeVI5-RTuFacdsY=.e14b980d-c7f6-491b-a52a-d037e7c96559@github.com> On Mon, 26 Apr 2021 08:31:29 GMT, Stefan Johansson wrote: >> Marcus G K Williams has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 48 commits: >> >> - Set LargePageSizeInBytes to largepage upper limit >> >> Signed-off-by: Marcus G K Williams >> - Merge branch 'master' into update_hlp >> - Fix merge >> >> Signed-off-by: Marcus G K Williams >> - Merge branch 'master' into update_hlp >> - Merge branch 'master' into update_hlp >> - Rebase on pull/3073 >> >> Signed-off-by: Marcus G K Williams >> - Merge branch 'pull/3073' into update_hlp >> - Thomas review. >> >> Changed commit_memory_special to return bool to signal if the request succeeded or not. >> - Self review. >> >> Update helper name to better match commit_memory_special(). >> - Marcus review. >> >> Updated comments. >> - ... and 38 more: https://git.openjdk.java.net/jdk/compare/5aab1609...6f063309 > > src/hotspot/os/linux/os_linux.cpp line 3737: > >> 3735: // Populate large page sizes to _page_sizes. Add pages that >> 3736: // are less than or equal to LargePageSizeInBytes, except when LargePageSizeInBytes=0 >> 3737: // or FLAG_IS_DEFAULT(LargePageSizeInBytes), add all sizes > > What we currently have in the CSR is to use the default large page size as the max if `LargePageSizeInBytes`is not set. If we first check if `LargePageSizeInBytes` have been set and update `_large_page_size` accordingly above (see comment below). I think this comment should say something like: > > Suggestion: > > // Populate _page_sizes with large page sizes less than or equal to > // _large_page_size. +1 ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From github.com+168222+mgkwill at openjdk.java.net Wed Apr 28 18:36:53 2021 From: github.com+168222+mgkwill at openjdk.java.net (Marcus G K Williams) Date: Wed, 28 Apr 2021 18:36:53 GMT Subject: RFR: 8256155: Allow multiple large page sizes to be used on Linux [v30] In-Reply-To: References: Message-ID: > Change the meaning of LargePageSizeInBytes to be the maximum large page size the JVM may use (not the only one). A default value of zero will mean to allow the JVM use large page sizes up to the system's default large page size. Marcus G K Williams has updated the pull request incrementally with one additional commit since the last revision: kstefanj review Signed-off-by: Marcus G K Williams ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/1153/files - new: https://git.openjdk.java.net/jdk/pull/1153/files/6f063309..6cd6be15 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=1153&range=29 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=1153&range=28-29 Stats: 34 lines in 1 file changed: 11 ins; 12 del; 11 mod Patch: https://git.openjdk.java.net/jdk/pull/1153.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/1153/head:pull/1153 PR: https://git.openjdk.java.net/jdk/pull/1153 From github.com+168222+mgkwill at openjdk.java.net Wed Apr 28 18:38:31 2021 From: github.com+168222+mgkwill at openjdk.java.net (Marcus G K Williams) Date: Wed, 28 Apr 2021 18:38:31 GMT Subject: RFR: 8256155: Allow multiple large page sizes to be used on Linux [v29] In-Reply-To: References: <2UqkjBQ_wVQcz_qx2wjchZanODAw8-QB-qROH9kdLM0=.e1980df2-de0c-412f-b219-5ac2eb38fafb@github.com> Message-ID: On Mon, 26 Apr 2021 08:47:07 GMT, Stefan Johansson wrote: >> I also think that the log-tag should be `pagesize`. > > The comment here could also be updated: > > // Check LargePageSizeInBytes and setup _large_page_size to hold > // the maximum allowed large page size. If not set the default large > // page size is used as the maximum. ``` +1 ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From jiefu at openjdk.java.net Wed Apr 28 22:40:52 2021 From: jiefu at openjdk.java.net (Jie Fu) Date: Wed, 28 Apr 2021 22:40:52 GMT Subject: Integrated: 8241354: ZGC still crashes in containers with NUMA due to get_mempolicy is disabled by default In-Reply-To: References: Message-ID: <9Ik8s3HxBtI6f9v4sUxd-_V_y-Yve6ItazqwY6G9a68=.d82c5e73-81f1-47c6-a102-34a1a24e8e84@github.com> On Mon, 26 Apr 2021 23:19:46 GMT, Jie Fu wrote: > Hi all, > > ZGC still crashes in containers hosting on NUMA machines due to an fatal error [1], which really limits the usage of zgc. > > The reason is that get_mempolicy is blocked by default in dockers [2]. > With the default docker configuration, ZGC will always crash with NUMA. > > Other collectors are also somehow affected by this restriction. > But unlike zgc, they won't crash and can still work in dockers. > So we'd better fix this bug as soon as possible since more and more people would like to use zgc. > > The patch was prepared under the guidance of StefanK [3] and many thanks to him. > And we'd like to provide a user friendly -Xlog:gc+init log in a separate RFE after this issue. > > Testing: > - tier1 ~ tier3 on Linux/x64 > > Thanks. > Best regards, > Jie > > [1] https://github.com/openjdk/jdk/blob/master/src/hotspot/os/linux/gc/z/zNUMA_linux.cpp#L64 > [2] https://docs.docker.com/engine/security/seccomp/ > [3] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028927.html This pull request has now been integrated. Changeset: 794cefe8 Author: Jie Fu URL: https://git.openjdk.java.net/jdk/commit/794cefe8f92ab0c9d5c7c5265232518e8ffb166a Stats: 20 lines in 1 file changed: 17 ins; 0 del; 3 mod 8241354: ZGC still crashes in containers with NUMA due to get_mempolicy is disabled by default Co-authored-by: Per Liden Reviewed-by: pliden ------------- PR: https://git.openjdk.java.net/jdk/pull/3704 From jiefu at openjdk.java.net Thu Apr 29 01:34:11 2021 From: jiefu at openjdk.java.net (Jie Fu) Date: Thu, 29 Apr 2021 01:34:11 GMT Subject: RFR: 8266217: ZGC: Improve the -Xlog:gc+init output for NUMA Message-ID: Hi all, This patch improves the -Xlog:gc+init output for NUMA, which is suggested by StefanK [1]. The implementation just follows how UseLargePages are setup and printed. Before (in docker, not support get_mempolicy) [0.007s][info][gc,init] Initializing The Z Garbage Collector [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) [0.007s][info][gc,init] NUMA Support: Disabled ... After (in docker, not support get_mempolicy) [0.007s][info][gc,init] Initializing The Z Garbage Collector [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) [0.007s][info][gc,init] NUMA Support: Unsupported ... Testing: - tier1~3 on Linux/x64, no regression Thanks. Best regards, Jie [1] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028927.html ------------- Commit messages: - 8266217: ZGC: Improve the -Xlog:gc+init output for NUMA Changes: https://git.openjdk.java.net/jdk/pull/3778/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3778&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8266217 Stats: 37 lines in 6 files changed: 21 ins; 0 del; 16 mod Patch: https://git.openjdk.java.net/jdk/pull/3778.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3778/head:pull/3778 PR: https://git.openjdk.java.net/jdk/pull/3778 From stefank at openjdk.java.net Thu Apr 29 06:01:52 2021 From: stefank at openjdk.java.net (Stefan Karlsson) Date: Thu, 29 Apr 2021 06:01:52 GMT Subject: RFR: 8266217: ZGC: Improve the -Xlog:gc+init output for NUMA In-Reply-To: References: Message-ID: <_o9APcx8JdFDgpkgAD6fhsagfXA6UATg3f82051c0DE=.78f8b2ef-28f2-4c1b-93a4-8ec7d2c3e642@github.com> On Wed, 28 Apr 2021 23:02:46 GMT, Jie Fu wrote: > Hi all, > > This patch improves the -Xlog:gc+init output for NUMA, which is suggested by StefanK [1]. > The implementation just follows how UseLargePages are setup and printed. > > Before (in docker, not support get_mempolicy) > > [0.007s][info][gc,init] Initializing The Z Garbage Collector > [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) > [0.007s][info][gc,init] NUMA Support: Disabled > ... > > > After (in docker, not support get_mempolicy) > > [0.007s][info][gc,init] Initializing The Z Garbage Collector > [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) > [0.007s][info][gc,init] NUMA Support: Unsupported > ... > > > Testing: > - tier1~3 on Linux/x64, no regression > > Thanks. > Best regards, > Jie > > > [1] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028927.html Marked as reviewed by stefank (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3778 From jiefu at openjdk.java.net Thu Apr 29 06:07:50 2021 From: jiefu at openjdk.java.net (Jie Fu) Date: Thu, 29 Apr 2021 06:07:50 GMT Subject: RFR: 8266217: ZGC: Improve the -Xlog:gc+init output for NUMA In-Reply-To: <_o9APcx8JdFDgpkgAD6fhsagfXA6UATg3f82051c0DE=.78f8b2ef-28f2-4c1b-93a4-8ec7d2c3e642@github.com> References: <_o9APcx8JdFDgpkgAD6fhsagfXA6UATg3f82051c0DE=.78f8b2ef-28f2-4c1b-93a4-8ec7d2c3e642@github.com> Message-ID: <1DHnztFBf3Z9PtlnqzdMa4QbUQn-Wz__-e55r-gBduI=.80767e90-24f5-4ff7-9491-1d14f57b8332@github.com> On Thu, 29 Apr 2021 05:59:08 GMT, Stefan Karlsson wrote: >> Hi all, >> >> This patch improves the -Xlog:gc+init output for NUMA, which is suggested by StefanK [1]. >> The implementation just follows how UseLargePages are setup and printed. >> >> Before (in docker, not support get_mempolicy) >> >> [0.007s][info][gc,init] Initializing The Z Garbage Collector >> [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) >> [0.007s][info][gc,init] NUMA Support: Disabled >> ... >> >> >> After (in docker, not support get_mempolicy) >> >> [0.007s][info][gc,init] Initializing The Z Garbage Collector >> [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) >> [0.007s][info][gc,init] NUMA Support: Unsupported >> ... >> >> >> Testing: >> - tier1~3 on Linux/x64, no regression >> >> Thanks. >> Best regards, >> Jie >> >> >> [1] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028927.html > > Marked as reviewed by stefank (Reviewer). Thanks @stefank for your review and help. ------------- PR: https://git.openjdk.java.net/jdk/pull/3778 From tschatzl at openjdk.java.net Thu Apr 29 07:18:11 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 29 Apr 2021 07:18:11 GMT Subject: RFR: 8266217: ZGC: Improve the -Xlog:gc+init output for NUMA In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 23:02:46 GMT, Jie Fu wrote: > Hi all, > > This patch improves the -Xlog:gc+init output for NUMA, which is suggested by StefanK [1]. > The implementation just follows how UseLargePages are setup and printed. > > Before (in docker, not support get_mempolicy) > > [0.007s][info][gc,init] Initializing The Z Garbage Collector > [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) > [0.007s][info][gc,init] NUMA Support: Disabled > ... > > > After (in docker, not support get_mempolicy) > > [0.007s][info][gc,init] Initializing The Z Garbage Collector > [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) > [0.007s][info][gc,init] NUMA Support: Unsupported > ... > > > Testing: > - tier1~3 on Linux/x64, no regression > > Thanks. > Best regards, > Jie > > > [1] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028927.html src/hotspot/share/gc/z/zNUMA.cpp line 41: > 39: > 40: const char* ZNUMA::to_string() { > 41: switch (_state) { Maybe consider to have a `static const char* strings[] = {"Enabled", "Unsupported", "Disabled" };` (in the correct order) constant here and index by the array (and maybe an assert to catch out of bounds access if/when adds an entry and forgets to update this list). This is typically more readable then a long switch case like this, but idk other people's opinions. ------------- PR: https://git.openjdk.java.net/jdk/pull/3778 From tschatzl at openjdk.java.net Thu Apr 29 07:26:52 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 29 Apr 2021 07:26:52 GMT Subject: RFR: 8266217: ZGC: Improve the -Xlog:gc+init output for NUMA In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 23:02:46 GMT, Jie Fu wrote: > Hi all, > > This patch improves the -Xlog:gc+init output for NUMA, which is suggested by StefanK [1]. > The implementation just follows how UseLargePages are setup and printed. > > Before (in docker, not support get_mempolicy) > > [0.007s][info][gc,init] Initializing The Z Garbage Collector > [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) > [0.007s][info][gc,init] NUMA Support: Disabled > ... > > > After (in docker, not support get_mempolicy) > > [0.007s][info][gc,init] Initializing The Z Garbage Collector > [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) > [0.007s][info][gc,init] NUMA Support: Unsupported > ... > > > Testing: > - tier1~3 on Linux/x64, no regression > > Thanks. > Best regards, > Jie > > > [1] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028927.html Marked as reviewed by tschatzl (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3778 From tschatzl at openjdk.java.net Thu Apr 29 07:26:53 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 29 Apr 2021 07:26:53 GMT Subject: RFR: 8266217: ZGC: Improve the -Xlog:gc+init output for NUMA In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 07:14:35 GMT, Thomas Schatzl wrote: >> Hi all, >> >> This patch improves the -Xlog:gc+init output for NUMA, which is suggested by StefanK [1]. >> The implementation just follows how UseLargePages are setup and printed. >> >> Before (in docker, not support get_mempolicy) >> >> [0.007s][info][gc,init] Initializing The Z Garbage Collector >> [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) >> [0.007s][info][gc,init] NUMA Support: Disabled >> ... >> >> >> After (in docker, not support get_mempolicy) >> >> [0.007s][info][gc,init] Initializing The Z Garbage Collector >> [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) >> [0.007s][info][gc,init] NUMA Support: Unsupported >> ... >> >> >> Testing: >> - tier1~3 on Linux/x64, no regression >> >> Thanks. >> Best regards, >> Jie >> >> >> [1] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028927.html > > src/hotspot/share/gc/z/zNUMA.cpp line 41: > >> 39: >> 40: const char* ZNUMA::to_string() { >> 41: switch (_state) { > > Maybe consider to have a `static const char* strings[] = {"Enabled", "Unsupported", "Disabled" };` (in the correct order) constant here and index by the array (and maybe an assert to catch out of bounds access if/when adds an entry and forgets to update this list). > This is typically more readable then a long switch case like this, but idk other people's opinions. And of course it's not a particular important suggestion, feel free to ignore. ------------- PR: https://git.openjdk.java.net/jdk/pull/3778 From jiefu at openjdk.java.net Thu Apr 29 07:43:53 2021 From: jiefu at openjdk.java.net (Jie Fu) Date: Thu, 29 Apr 2021 07:43:53 GMT Subject: RFR: 8266217: ZGC: Improve the -Xlog:gc+init output for NUMA In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 07:24:12 GMT, Thomas Schatzl wrote: >> src/hotspot/share/gc/z/zNUMA.cpp line 41: >> >>> 39: >>> 40: const char* ZNUMA::to_string() { >>> 41: switch (_state) { >> >> Maybe consider to have a `static const char* strings[] = {"Enabled", "Unsupported", "Disabled" };` (in the correct order) constant here and index by the array (and maybe an assert to catch out of bounds access if/when adds an entry and forgets to update this list). >> This is typically more readable then a long switch case like this, but idk other people's opinions. > > And of course it's not a particular important suggestion, feel free to ignore. > Maybe consider to have a `static const char* strings[] = {"Enabled", "Unsupported", "Disabled" };` (in the correct order) constant here and index by the array (and maybe an assert to catch out of bounds access if/when adds an entry and forgets to update this list). > This is typically more readable then a long switch case like this, but idk other people's opinions. Thanks @tschatzl for your review. I didn't think of this skill before. Thanks for your teaching. Let's just keep it this time since there are only 3 cases for `_state`. And I'd like to use your style of code next time. Thanks. Best regards, Jie ------------- PR: https://git.openjdk.java.net/jdk/pull/3778 From pliden at openjdk.java.net Thu Apr 29 07:49:49 2021 From: pliden at openjdk.java.net (Per Liden) Date: Thu, 29 Apr 2021 07:49:49 GMT Subject: RFR: 8266217: ZGC: Improve the -Xlog:gc+init output for NUMA In-Reply-To: References: Message-ID: <-InWs3qj2mpzahmSV00Gnw3YvAHnFAvjYrIxPK2naHo=.bf6fc4d2-6bf1-40bd-b8b1-7b45c3d10abf@github.com> On Wed, 28 Apr 2021 23:02:46 GMT, Jie Fu wrote: > Hi all, > > This patch improves the -Xlog:gc+init output for NUMA, which is suggested by StefanK [1]. > The implementation just follows how UseLargePages are setup and printed. > > Before (in docker, not support get_mempolicy) > > [0.007s][info][gc,init] Initializing The Z Garbage Collector > [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) > [0.007s][info][gc,init] NUMA Support: Disabled > ... > > > After (in docker, not support get_mempolicy) > > [0.007s][info][gc,init] Initializing The Z Garbage Collector > [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) > [0.007s][info][gc,init] NUMA Support: Unsupported > ... > > > Testing: > - tier1~3 on Linux/x64, no regression > > Thanks. > Best regards, > Jie > > > [1] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028927.html Looks good. ------------- Marked as reviewed by pliden (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3778 From pliden at openjdk.java.net Thu Apr 29 07:49:50 2021 From: pliden at openjdk.java.net (Per Liden) Date: Thu, 29 Apr 2021 07:49:50 GMT Subject: RFR: 8266217: ZGC: Improve the -Xlog:gc+init output for NUMA In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 07:41:18 GMT, Jie Fu wrote: >> And of course it's not a particular important suggestion, feel free to ignore. > >> Maybe consider to have a `static const char* strings[] = {"Enabled", "Unsupported", "Disabled" };` (in the correct order) constant here and index by the array (and maybe an assert to catch out of bounds access if/when adds an entry and forgets to update this list). >> This is typically more readable then a long switch case like this, but idk other people's opinions. > > Thanks @tschatzl for your review. > > I didn't think of this skill before. > Thanks for your teaching. > > Let's just keep it this time since there are only 3 cases for `_state`. > And I'd like to use your style of code next time. > > Thanks. > Best regards, > Jie I'd like to keep the current proposal, since it's more inline with what we do in other places in ZGC, like `ZLargePages::to_string()`. ------------- PR: https://git.openjdk.java.net/jdk/pull/3778 From tschatzl at openjdk.java.net Thu Apr 29 08:05:26 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 29 Apr 2021 08:05:26 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask [v8] In-Reply-To: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: > [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). > > To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. > > This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. > > [Here](https://github.com/openjdk/jdk/compare/master...tschatzl:parallel-phases-join-everything) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). > > Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: sjohanss review ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3653/files - new: https://git.openjdk.java.net/jdk/pull/3653/files/3c8d04a5..6ca0b29f Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=07 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=06-07 Stats: 13 lines in 3 files changed: 1 ins; 0 del; 12 mod Patch: https://git.openjdk.java.net/jdk/pull/3653.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3653/head:pull/3653 PR: https://git.openjdk.java.net/jdk/pull/3653 From jiefu at openjdk.java.net Thu Apr 29 08:14:55 2021 From: jiefu at openjdk.java.net (Jie Fu) Date: Thu, 29 Apr 2021 08:14:55 GMT Subject: RFR: 8266217: ZGC: Improve the -Xlog:gc+init output for NUMA In-Reply-To: <-InWs3qj2mpzahmSV00Gnw3YvAHnFAvjYrIxPK2naHo=.bf6fc4d2-6bf1-40bd-b8b1-7b45c3d10abf@github.com> References: <-InWs3qj2mpzahmSV00Gnw3YvAHnFAvjYrIxPK2naHo=.bf6fc4d2-6bf1-40bd-b8b1-7b45c3d10abf@github.com> Message-ID: On Thu, 29 Apr 2021 07:46:33 GMT, Per Liden wrote: > Looks good. Thanks @pliden . ------------- PR: https://git.openjdk.java.net/jdk/pull/3778 From sjohanss at openjdk.java.net Thu Apr 29 08:17:51 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Thu, 29 Apr 2021 08:17:51 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask [v8] In-Reply-To: References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: On Thu, 29 Apr 2021 08:05:26 GMT, Thomas Schatzl wrote: >> [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). >> >> To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. >> >> This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. >> >> [Here](https://github.com/openjdk/jdk/compare/master...tschatzl:parallel-phases-join-everything) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). >> >> Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > sjohanss review Thanks Thomas for updating the names. This looks good. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3653 From ayang at openjdk.java.net Thu Apr 29 09:07:18 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Thu, 29 Apr 2021 09:07:18 GMT Subject: RFR: 8234446: Post-CMS workgroup hierarchy cleanup [v3] In-Reply-To: References: Message-ID: > Collapse workgroup hierarchy and some general cleanup. > > Tested; hotspot_gc Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: undo indentation change ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3663/files - new: https://git.openjdk.java.net/jdk/pull/3663/files/a194f5ff..c435e98d Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3663&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3663&range=01-02 Stats: 9 lines in 1 file changed: 0 ins; 0 del; 9 mod Patch: https://git.openjdk.java.net/jdk/pull/3663.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3663/head:pull/3663 PR: https://git.openjdk.java.net/jdk/pull/3663 From ayang at openjdk.java.net Thu Apr 29 09:13:53 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Thu, 29 Apr 2021 09:13:53 GMT Subject: RFR: 8234446: Post-CMS workgroup hierarchy cleanup [v3] In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 09:07:18 GMT, Albert Mingkun Yang wrote: >> Collapse workgroup hierarchy and some general cleanup. >> >> Tested; hotspot_gc > > Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: > > undo indentation change There is some preexisting inconsistency regarding the indentation of fields initializers in constructors, e.g. `SubTasksDone` uses 2-space, while `WorkGang` uses 4-space. In order not to make the review process unnecessarily harder, I have kept the original indentation, despite the inconsistency. ------------- PR: https://git.openjdk.java.net/jdk/pull/3663 From tschatzl at openjdk.java.net Thu Apr 29 09:48:15 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 29 Apr 2021 09:48:15 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask [v9] In-Reply-To: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: <1xQTdVkoD_Fu0BXTKk8KEKxT_YO-nLfzkhdiP06VcVw=.0e83b2f4-9f6d-4d85-a6d7-3cd2435c05ea@github.com> > [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). > > To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. > > This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. > > [Here](https://github.com/openjdk/jdk/compare/master...tschatzl:parallel-phases-join-everything) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). > > Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: iwalulya review, removing try_claim_serial_task optimization ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3653/files - new: https://git.openjdk.java.net/jdk/pull/3653/files/6ca0b29f..bb84ac67 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=08 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3653&range=07-08 Stats: 4 lines in 1 file changed: 0 ins; 3 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3653.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3653/head:pull/3653 PR: https://git.openjdk.java.net/jdk/pull/3653 From iwalulya at openjdk.java.net Thu Apr 29 09:48:16 2021 From: iwalulya at openjdk.java.net (Ivan Walulya) Date: Thu, 29 Apr 2021 09:48:16 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask [v8] In-Reply-To: References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: On Thu, 29 Apr 2021 08:05:26 GMT, Thomas Schatzl wrote: >> [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). >> >> To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. >> >> This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. >> >> [Here](https://github.com/openjdk/jdk/compare/master...tschatzl:parallel-phases-join-everything) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). >> >> Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. >> >> Thanks, >> Thomas > > Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: > > sjohanss review lgtm! ------------- Marked as reviewed by iwalulya (Committer). PR: https://git.openjdk.java.net/jdk/pull/3653 From tschatzl at openjdk.java.net Thu Apr 29 09:48:16 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 29 Apr 2021 09:48:16 GMT Subject: RFR: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask [v8] In-Reply-To: References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: On Thu, 29 Apr 2021 09:42:33 GMT, Ivan Walulya wrote: >> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: >> >> sjohanss review > > lgtm! Thanks @walulyai @kstefanj @albertnetymk @lkorinth for your reviews! ------------- PR: https://git.openjdk.java.net/jdk/pull/3653 From tschatzl at openjdk.java.net Thu Apr 29 10:44:51 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 29 Apr 2021 10:44:51 GMT Subject: RFR: 8234446: Post-CMS workgroup hierarchy cleanup [v3] In-Reply-To: References: Message-ID: <1AcuG6nFsJraQVqow5JpD3HVspIwAsOrGshicEAUM4o=.49ed01ab-d2bd-4f6d-85ec-34fc9aea264e@github.com> On Thu, 29 Apr 2021 09:07:18 GMT, Albert Mingkun Yang wrote: >> Collapse workgroup hierarchy and some general cleanup. >> >> Tested; hotspot_gc > > Albert Mingkun Yang has updated the pull request incrementally with one additional commit since the last revision: > > undo indentation change Marked as reviewed by tschatzl (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3663 From yyang at openjdk.java.net Thu Apr 29 11:48:56 2021 From: yyang at openjdk.java.net (Yi Yang) Date: Thu, 29 Apr 2021 11:48:56 GMT Subject: RFR: 8265322: C2: Simplify control inputs for BarrierSetC2::obj_allocate In-Reply-To: References: <_NicB3VpqFwTiu2-bKLNhfnQAxJmbtND66g3Md5ck5g=.f9b4edcd-fa2b-4917-9092-bccc38b202df@github.com> Message-ID: On Mon, 19 Apr 2021 17:26:14 GMT, Vladimir Kozlov wrote: >> While just using BarrierSetC2::obj_allocate in https://github.com/openjdk/valhalla/pull/385, I noticed the control input `ctrl` for obj_allocate is not so much necessary. This PR simplifies control inputs for BarrierSetC2::obj_allocate. In most cases, it doesn't change anything since `toobig_false` is equivalent to `ctrl`. In other cases, `toobig_false` is created for Unsafe.allocateInstance while instance size is not statically known, `ctrl` would become control input of IfNode whose projects are `toobig_false` and `toobig_true`, old eden_end and old_eden_top can simply accept `toobig_false` as their control input rather than `ctrl`. > >> > From compiler code POV the fix is reasonable and correct. >> Do you think it's reasonable to move PhaseMacroExpand::set_eden_pointers to BarrierSetC2? It seems that that's GC knowledge area about how to set eden_top/eden_end w or w/o turning UseTLAB. > > It is up to GC group but it is common code for all GCs. As I understand BarrierSetC2 is used for GC variations code. Thanks @vnkozlov and @neliasso for reviews! ------------- PR: https://git.openjdk.java.net/jdk/pull/3529 From ayang at openjdk.java.net Thu Apr 29 12:58:08 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Thu, 29 Apr 2021 12:58:08 GMT Subject: RFR: 8266295: Remove unused _concurrent_iteration_safe_limit Message-ID: Small change of removing an unused field; possible leftover of CMS. ------------- Commit messages: - limit Changes: https://git.openjdk.java.net/jdk/pull/3795/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3795&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8266295 Stats: 22 lines in 4 files changed: 0 ins; 21 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/3795.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3795/head:pull/3795 PR: https://git.openjdk.java.net/jdk/pull/3795 From pliden at openjdk.java.net Thu Apr 29 13:48:11 2021 From: pliden at openjdk.java.net (Per Liden) Date: Thu, 29 Apr 2021 13:48:11 GMT Subject: RFR: 8260267: ZGC: Reduce mark stack usage [v3] In-Reply-To: <1Qn76cgJaltPI2LtFQFfdqCG8SzOJwg3ORVQc8pK3MY=.918a159f-4f3d-408c-8cca-5b2bf727a0b2@github.com> References: <1Qn76cgJaltPI2LtFQFfdqCG8SzOJwg3ORVQc8pK3MY=.918a159f-4f3d-408c-8cca-5b2bf727a0b2@github.com> Message-ID: > Under some conditions, striped marking can cause excessive mark stack memory usage. For example, when the object graph has one or more N:1 relations ships, where N is a large number. > > To avoid pushing mark stack entries to already marked objects, and thus reduce mark stack memory usage, this patch adds does two things: > * GC threads will under some conditions mark the object before pushing, else (if the condition to mark failed) > * GC threads will check if the object is already marked before pushing. > > The conditions for marking before pushing is: > * We are using a single stripe, in which cases striped marking is essentially disabled anyway. > * We see excessive mark stack usage. In other words, the mark stack allocator indicates "high usage". > > To make this easier to review, this PR contains two commits: > * `ZGC: Reduce mark stack usage` - This patch adds the infrastructure needed to mark before push, but only does so if we are using a single stripe. > * `ZGC: Mark before push if mark stack usage is high` - This patch adds the condition to mark before push if the mark stack allocator indicates "high usage". As a bonus, this patch will also uncommit memory used for mark stacks after the marking has completed. > > Testing: > * Tier1-7 > * SPECjbb2015, score and marking times unaffected > * casparcwang at tencent.com reports that this patch, in combination with PR #3455, solves the problem for Apache Zookeeper. Per Liden has updated the pull request incrementally with one additional commit since the last revision: Review comments from StefanJ and Albert ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3460/files - new: https://git.openjdk.java.net/jdk/pull/3460/files/96071230..77a65449 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3460&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3460&range=01-02 Stats: 93 lines in 2 files changed: 67 ins; 20 del; 6 mod Patch: https://git.openjdk.java.net/jdk/pull/3460.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3460/head:pull/3460 PR: https://git.openjdk.java.net/jdk/pull/3460 From tschatzl at openjdk.java.net Thu Apr 29 14:41:53 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 29 Apr 2021 14:41:53 GMT Subject: RFR: 8266295: Remove unused _concurrent_iteration_safe_limit In-Reply-To: References: Message-ID: <6CF3QF_Mgbqtz7bQeu4-KjD73R6XFH8FRQdolROBook=.a291c0e5-849b-4f37-9c04-27ea0b01cc91@github.com> On Thu, 29 Apr 2021 12:51:53 GMT, Albert Mingkun Yang wrote: > Small change of removing an unused field; possible leftover of CMS. Marked as reviewed by tschatzl (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3795 From tschatzl at openjdk.java.net Thu Apr 29 14:46:52 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Thu, 29 Apr 2021 14:46:52 GMT Subject: Integrated: 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask In-Reply-To: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> References: <0cl-xHZbIwImCT773chfP2TNjeHOg2iDn6nnyFdnmy8=.e0642889-a256-4d97-a474-58f240eb1f6f@github.com> Message-ID: On Fri, 23 Apr 2021 11:54:49 GMT, Thomas Schatzl wrote: > [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) is going to join different somewhat independent (sub-)tasks of the post evacuation phase into basically two gangtasks to improve performance (reduce startup and shutdown of work gangs). > > To facilitate doing that, introduce an API that manages subtasks of a given gangtask and provide some helpers to do common work. > > This CR is only about adding the API and testing to somewhat simplify the actual move of the post evacuate tasks into parallel tasks. There is no new functionality here, only to split up [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) a bit. > > [Here](https://github.com/openjdk/jdk/compare/master...tschatzl:parallel-phases-join-everything) is the full change that uses this API to implement JDK-8214237 (which I'll try to split up a bit before) to see how it will be used (in [G1YoungGCPostEvacuateTasks.hpp](https://github.com/openjdk/jdk/commit/1d8667143e7d106b96c25c0b876c37f1250be132#diff-64bc43a24e11978920bb9523cb1507a7275ace22d9eead4c34177339a3258e42) ). > > Testing: new gtest, tier1-5 with mentioned patch for JDK-8214237. > > Thanks, > Thomas This pull request has now been integrated. Changeset: c76ce287 Author: Thomas Schatzl URL: https://git.openjdk.java.net/jdk/commit/c76ce28754658135a544dbfc6b8f767a4185842e Stats: 415 lines in 3 files changed: 415 ins; 0 del; 0 mod 8265842: G1: Introduce API to run multiple separate tasks in a single gangtask Reviewed-by: lkorinth, ayang, sjohanss, iwalulya ------------- PR: https://git.openjdk.java.net/jdk/pull/3653 From bpb at openjdk.java.net Thu Apr 29 16:15:13 2021 From: bpb at openjdk.java.net (Brian Burkhalter) Date: Thu, 29 Apr 2021 16:15:13 GMT Subject: RFR: 8266299: ProblemList runtime/stringtable/StringTableCleaningTest.java on linux-aarch64 with ZGC In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 16:04:17 GMT, Daniel D. Daugherty wrote: > A trivial fix to ProblemList runtime/stringtable/StringTableCleaningTest.java on linux-aarch64 with ZGC. Marked as reviewed by bpb (Reviewer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3797 From dcubed at openjdk.java.net Thu Apr 29 16:15:13 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 29 Apr 2021 16:15:13 GMT Subject: RFR: 8266299: ProblemList runtime/stringtable/StringTableCleaningTest.java on linux-aarch64 with ZGC Message-ID: A trivial fix to ProblemList runtime/stringtable/StringTableCleaningTest.java on linux-aarch64 with ZGC. ------------- Commit messages: - 8266299: ProblemListruntime/stringtable/StringTableCleaningTest.java on linux-aarch64 with ZGC Changes: https://git.openjdk.java.net/jdk/pull/3797/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3797&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8266299 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3797.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3797/head:pull/3797 PR: https://git.openjdk.java.net/jdk/pull/3797 From dcubed at openjdk.java.net Thu Apr 29 16:23:51 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 29 Apr 2021 16:23:51 GMT Subject: RFR: 8266299: ProblemList runtime/stringtable/StringTableCleaningTest.java on linux-aarch64 with ZGC In-Reply-To: References: Message-ID: On Thu, 29 Apr 2021 16:09:54 GMT, Brian Burkhalter wrote: >> A trivial fix to ProblemList runtime/stringtable/StringTableCleaningTest.java on linux-aarch64 with ZGC. > > Marked as reviewed by bpb (Reviewer). @bplb - Thanks for the fast review! ------------- PR: https://git.openjdk.java.net/jdk/pull/3797 From dcubed at openjdk.java.net Thu Apr 29 17:28:52 2021 From: dcubed at openjdk.java.net (Daniel D.Daugherty) Date: Thu, 29 Apr 2021 17:28:52 GMT Subject: Integrated: 8266299: ProblemList runtime/stringtable/StringTableCleaningTest.java on linux-aarch64 with ZGC In-Reply-To: References: Message-ID: <_9ag0yqm71qrdldOactP10c7YDea5kMjdOx4Hl_1Qaw=.4b245b32-bee6-4b19-b0b5-18f2c91dbce0@github.com> On Thu, 29 Apr 2021 16:04:17 GMT, Daniel D. Daugherty wrote: > A trivial fix to ProblemList runtime/stringtable/StringTableCleaningTest.java on linux-aarch64 with ZGC. This pull request has now been integrated. Changeset: 51b2fb5c Author: Daniel D. Daugherty URL: https://git.openjdk.java.net/jdk/commit/51b2fb5c848171046d5c2fb822f09d6535a23c4c Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod 8266299: ProblemList runtime/stringtable/StringTableCleaningTest.java on linux-aarch64 with ZGC Reviewed-by: bpb ------------- PR: https://git.openjdk.java.net/jdk/pull/3797 From adityam at openjdk.java.net Thu Apr 29 18:41:24 2021 From: adityam at openjdk.java.net (Aditya Mandaleeka) Date: Thu, 29 Apr 2021 18:41:24 GMT Subject: RFR: 8257774: G1: Trigger collect when free region count drops below threshold to prevent evacuation failures [v5] In-Reply-To: <5Ro0P4NPaUuG-LPpgqHY5S_kh_EVpwRgxqc5jTCano4=.afc3f2a4-dc9e-46c1-ac84-298cf143fa41@github.com> References: <5Ro0P4NPaUuG-LPpgqHY5S_kh_EVpwRgxqc5jTCano4=.afc3f2a4-dc9e-46c1-ac84-298cf143fa41@github.com> Message-ID: > _This PR picks up from [this previous PR](https://github.com/openjdk/jdk/pull/1650) by @charliegracie._ > > I won't repeat the full description from the prior PR, but the general idea is to add the notion of a "proactive GC" to G1 which gets triggered in the slow allocation path if the number of free regions drops below the amount that would be required to complete a GC if it happened at that moment. The threshold is based on the survival rates from eden and survivor spaces along with the space required for tenured space evacuations. > > There are a couple of outstanding issues/questions known: > - _Interaction with GCLocker_: In the case where we determine that a proactive GC is required and GC locker is active, we don't allow the young gen to expand (instead threads will stall). @tschatzl raised this and suggested that it should be discussed as part of the review. > - _Disable proactive GC heuristic during initialization_: I ran into an issue in testing where code in the universe initialization codepath was tripping the proactive GC heuristic, leading to a GC being triggered before the VM has finished initialization. I need to find a good way to prevent this from happening. There may already be a mechanism for this, but I couldn't find one so I added a temporary placeholder (`zzinit_complete`) just to unblock testing. Aditya Mandaleeka has updated the pull request incrementally with one additional commit since the last revision: Remove temp comment. ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3143/files - new: https://git.openjdk.java.net/jdk/pull/3143/files/66fa0977..80df203e Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3143&range=04 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3143&range=03-04 Stats: 2 lines in 1 file changed: 0 ins; 2 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3143.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3143/head:pull/3143 PR: https://git.openjdk.java.net/jdk/pull/3143 From sjohanss at openjdk.java.net Thu Apr 29 19:38:00 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Thu, 29 Apr 2021 19:38:00 GMT Subject: RFR: 8260267: ZGC: Reduce mark stack usage [v3] In-Reply-To: References: <1Qn76cgJaltPI2LtFQFfdqCG8SzOJwg3ORVQc8pK3MY=.918a159f-4f3d-408c-8cca-5b2bf727a0b2@github.com> Message-ID: On Thu, 29 Apr 2021 13:48:11 GMT, Per Liden wrote: >> Under some conditions, striped marking can cause excessive mark stack memory usage. For example, when the object graph has one or more N:1 relations ships, where N is a large number. >> >> To avoid pushing mark stack entries to already marked objects, and thus reduce mark stack memory usage, this patch adds does two things: >> * GC threads will under some conditions mark the object before pushing, else (if the condition to mark failed) >> * GC threads will check if the object is already marked before pushing. >> >> The conditions for marking before pushing is: >> * We are using a single stripe, in which cases striped marking is essentially disabled anyway. >> * We see excessive mark stack usage. In other words, the mark stack allocator indicates "high usage". >> >> To make this easier to review, this PR contains two commits: >> * `ZGC: Reduce mark stack usage` - This patch adds the infrastructure needed to mark before push, but only does so if we are using a single stripe. >> * `ZGC: Mark before push if mark stack usage is high` - This patch adds the condition to mark before push if the mark stack allocator indicates "high usage". As a bonus, this patch will also uncommit memory used for mark stacks after the marking has completed. >> >> Testing: >> * Tier1-7 >> * SPECjbb2015, score and marking times unaffected >> * casparcwang at tencent.com reports that this patch, in combination with PR #3455, solves the problem for Apache Zookeeper. > > Per Liden has updated the pull request incrementally with one additional commit since the last revision: > > Review comments from StefanJ and Albert Looks good, just two minor comments. src/hotspot/share/gc/z/zMarkStackAllocator.cpp line 72: > 70: > 71: static bool is_high_usage(size_t size) { > 72: // Consider usage to be high if we've used more than a 8th of the available space. "a 8th" -> "an 8th" or "one 8th" src/hotspot/share/gc/z/zMarkStackAllocator.cpp line 101: > 99: > 100: log_debug(gc, marking)("Expanding mark stack space: " SIZE_FORMAT "M->" SIZE_FORMAT "M (%s Usage)", > 101: old_size / M, new_size / M, _high_usage ? "High" : "Low"); A bit strange to only have "Low" and "High" usage, maybe "Normal" and "High" would be more accurate. ------------- Marked as reviewed by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3460 From sjohanss at openjdk.java.net Thu Apr 29 19:52:52 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Thu, 29 Apr 2021 19:52:52 GMT Subject: RFR: 8256155: Allow multiple large page sizes to be used on Linux [v15] In-Reply-To: References: Message-ID: On Sat, 13 Mar 2021 07:25:12 GMT, Thomas Stuefe wrote: >> src/hotspot/os/linux/os_linux.cpp line 4046: >> >>> 4044: // Select large_page_size from _page_sizes >>> 4045: // that is smaller than size_t bytes >>> 4046: size_t large_page_size = os::page_size_for_region_aligned(bytes, 1); >> >> This one also needs to use `os::page_size_for_region_unaligned(...)` since we know we have a size that needs both small and large pages. > > +1 to what stefan wrote My comment here is a bit outdated. Now this function is used for both the "mixed" and the "only" case. As I see it we have to possibilities here. Either we stall this change a bit longer and let me finish the `ReservedSpace` work I've been doing lately. I have a patch coming out tonight or tomorrow that will add a page size member to `ReservedSpace`. I could then make a change to the `reserve_memory_special` API to pass a page size along with the other parameters and then this change will only be about enabling multiple page sizes. Everything else should just work. Alternatively we have to come up with some helper that decide the page size given the current alignment and size. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From sjohanss at openjdk.java.net Thu Apr 29 19:55:54 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Thu, 29 Apr 2021 19:55:54 GMT Subject: RFR: 8256155: Allow multiple large page sizes to be used on Linux [v30] In-Reply-To: References: Message-ID: <2Jj6MiFwxq9a3sNZl5fQPuzk6rgYf_fXJ6ztmXbY61g=.432eb729-e52d-4c7a-9e47-7f9a44fcaedb@github.com> On Wed, 28 Apr 2021 18:36:53 GMT, Marcus G K Williams wrote: >> Change the meaning of LargePageSizeInBytes to be the maximum large page size the JVM may use (not the only one). A default value of zero will mean to allow the JVM use large page sizes up to the system's default large page size. > > Marcus G K Williams has updated the pull request incrementally with one additional commit since the last revision: > > kstefanj review > > Signed-off-by: Marcus G K Williams src/hotspot/share/runtime/os.cpp line 1483: > 1481: for (size_t page_size = page_sizes().largest(); page_size != 0; > 1482: page_size = page_sizes().next_smaller(page_size)) { > 1483: if (page_size <= max_page_size && page_size > (size_t)vm_page_size()) { Is this really needed? ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From ayang at openjdk.java.net Thu Apr 29 22:11:00 2021 From: ayang at openjdk.java.net (Albert Mingkun Yang) Date: Thu, 29 Apr 2021 22:11:00 GMT Subject: RFR: 8260267: ZGC: Reduce mark stack usage [v3] In-Reply-To: References: <1Qn76cgJaltPI2LtFQFfdqCG8SzOJwg3ORVQc8pK3MY=.918a159f-4f3d-408c-8cca-5b2bf727a0b2@github.com> Message-ID: On Thu, 29 Apr 2021 13:48:11 GMT, Per Liden wrote: >> Under some conditions, striped marking can cause excessive mark stack memory usage. For example, when the object graph has one or more N:1 relations ships, where N is a large number. >> >> To avoid pushing mark stack entries to already marked objects, and thus reduce mark stack memory usage, this patch adds does two things: >> * GC threads will under some conditions mark the object before pushing, else (if the condition to mark failed) >> * GC threads will check if the object is already marked before pushing. >> >> The conditions for marking before pushing is: >> * We are using a single stripe, in which cases striped marking is essentially disabled anyway. >> * We see excessive mark stack usage. In other words, the mark stack allocator indicates "high usage". >> >> To make this easier to review, this PR contains two commits: >> * `ZGC: Reduce mark stack usage` - This patch adds the infrastructure needed to mark before push, but only does so if we are using a single stripe. >> * `ZGC: Mark before push if mark stack usage is high` - This patch adds the condition to mark before push if the mark stack allocator indicates "high usage". As a bonus, this patch will also uncommit memory used for mark stacks after the marking has completed. >> >> Testing: >> * Tier1-7 >> * SPECjbb2015, score and marking times unaffected >> * casparcwang at tencent.com reports that this patch, in combination with PR #3455, solves the problem for Apache Zookeeper. > > Per Liden has updated the pull request incrementally with one additional commit since the last revision: > > Review comments from StefanJ and Albert Marked as reviewed by ayang (Committer). ------------- PR: https://git.openjdk.java.net/jdk/pull/3460 From jiefu at openjdk.java.net Thu Apr 29 23:09:52 2021 From: jiefu at openjdk.java.net (Jie Fu) Date: Thu, 29 Apr 2021 23:09:52 GMT Subject: Integrated: 8266217: ZGC: Improve the -Xlog:gc+init output for NUMA In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 23:02:46 GMT, Jie Fu wrote: > Hi all, > > This patch improves the -Xlog:gc+init output for NUMA, which is suggested by StefanK [1]. > The implementation just follows how UseLargePages are setup and printed. > > Before (in docker, not support get_mempolicy) > > [0.007s][info][gc,init] Initializing The Z Garbage Collector > [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) > [0.007s][info][gc,init] NUMA Support: Disabled > ... > > > After (in docker, not support get_mempolicy) > > [0.007s][info][gc,init] Initializing The Z Garbage Collector > [0.007s][info][gc,init] Version: 17-internal+0-adhoc..jdk (fastdebug) > [0.007s][info][gc,init] NUMA Support: Unsupported > ... > > > Testing: > - tier1~3 on Linux/x64, no regression > > Thanks. > Best regards, > Jie > > > [1] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028927.html This pull request has now been integrated. Changeset: 5ecef01c Author: Jie Fu URL: https://git.openjdk.java.net/jdk/commit/5ecef01c4a9c7994a0fd20f926d0571c0c719f6e Stats: 37 lines in 6 files changed: 21 ins; 0 del; 16 mod 8266217: ZGC: Improve the -Xlog:gc+init output for NUMA Reviewed-by: stefank, tschatzl, pliden ------------- PR: https://git.openjdk.java.net/jdk/pull/3778 From sjohanss at openjdk.java.net Fri Apr 30 07:50:56 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Fri, 30 Apr 2021 07:50:56 GMT Subject: RFR: 8256155: Allow multiple large page sizes to be used on Linux [v30] In-Reply-To: References: Message-ID: On Wed, 28 Apr 2021 18:36:53 GMT, Marcus G K Williams wrote: >> Change the meaning of LargePageSizeInBytes to be the maximum large page size the JVM may use (not the only one). A default value of zero will mean to allow the JVM use large page sizes up to the system's default large page size. > > Marcus G K Williams has updated the pull request incrementally with one additional commit since the last revision: > > kstefanj review > > Signed-off-by: Marcus G K Williams Filed [JDK-8266349](https://bugs.openjdk.java.net/browse/JDK-8266349) for passing page size down to `reserve_memory_special`. As I mentioned [here](https://github.com/openjdk/jdk/pull/1153/files#r623353767), if we do this change before this one. We should not have to care about anything but adding the available and configure page sizes to `os::_page_sizes`. ------------- PR: https://git.openjdk.java.net/jdk/pull/1153 From tschatzl at openjdk.java.net Fri Apr 30 09:43:29 2021 From: tschatzl at openjdk.java.net (Thomas Schatzl) Date: Fri, 30 Apr 2021 09:43:29 GMT Subject: RFR: 8214237: Join parallel phases post evacuation [v2] In-Reply-To: References: Message-ID: > Hi all, > > can I get reviews for this change that joins all non-object moving post evacuation tasks into two parallel batched phases? > > This has the advantage of avoiding spinup and teardown of the separate tasks and also makes a few tasks that are done serially at the moment run in parallel to each other. > > For that, extensive analysis of dependencies has been performed, and I found that we need two `G1BatchedGangTasks` (introduced just for this purpose): > > The first one, `G1PostEvacuateCollectionSetCleanupTask1` contains the following subtasks: > * Merge PSS (serial) > * Recalculate Used (serial) > * Remove Self Forwards (parallel, on evacuation failure) > * Clear Card Table (parallel) > > The second one, `G1PostEvacuateCollectionSetCleanupTask2` contains the following subtasks: > * Eagerly Reclaim Humongous Objects (serial) > * Purge Code Roots (serial) > * Reset Hot Card Cache (serial) > * Update Derived Pointers (serial) > * Redirty Logged Cards (parallel) > * Restore Preserved Marks (parallel, on evacuation failure) > * Free Collection Set (parallel) > > Feel free to propose better names for the batch tasks :) The distribution has been somewhat arbitrary with the following limitations: > > **Redirty Logged Cards** depends on **Clear Card Table** - we need to clear card table scanning information from the card table before redirtying (Clear Card Table could actually split into the part that clears the "Young Gen" marks and the parts that actually contain card scan information, but it does not seem to provide an advantage). > **Redirty Cards** depends on ***Merge PSS** - that phase merge per thread DCQs into global dcq; redirty cards could of course take from the per-thread dcq if needed > **Free Collection Set** depends on **Merge PSS** to merge per thread surviving young words to single one and **Recalculate Used** as it updates the heap usage counters. > > More dependencies between other phases not merged are noted on the [JDK-8214237](https://bugs.openjdk.java.net/browse/JDK-8214237) page. > > This change seems huge (~1k LOC changes), but actually it isn't: lots of code has been moved 1:1 from `g1CollectedHeap.cpp` to the new `g1YoungGCPostEvacuateTasks.*` files. `g1CollectedHeap.cpp` just got way too huge with this change so it was really useful to start splitting the young gc related things into new files like has been done for the full GC. I'll continue extending this move, but there just has to be a start somewhere. > > I tried to do this change in a few steps, but that unfortunately caused too many weird intermediate workarounds steps in the timing code (`G1GCPhaseTimes`). So let me try to cut down the change into steps for you. > > So the change can be split up into > - move functionality to `g1YoungGCPostEvacuateTasks.cpp`: this affected dismantling existing AbstractGangTasks into main code (constructor, destructor, do_work method) and move that code to the corresponding `G1AbstractSubTask`. > - The helper code typically inlined into that `AbstractGangTask` were moved as well close to that `G1AbstractSubTask`. > - wiring up stuff to work with `G1BatchedGangTask` > > And the review tasks should roughly be: > - verify that the copy&paste has been correct (there is a list below) > - verify that dependencies are correct (see above) > - minor wiring changes > > Here's a summary for the copy&paste changes: > * `RedirtyLoggedCardTableEntryClosure` moved verbatim to `g1YoungGCPostEvacuateTasks.cpp` > * `G1RedirtyLoggedCardsTask` were transformed into a `G1AbstractSubTask` > * `G1CollectedHeap::remove_self_forwarding_pointers()` and `G1CollectedHeap::restore_after_evac_failure` were transformed into a `G1AbstractSubTask` > * recalculate used memory activity has been extracted from the code and put into `G1PostEvacuateCollectionSetCleanupTask1::RecalculateUsedTask` > * serial tasks that were about calling a single method were wrapped into a `G1AbstractSubTask` that just calls the same method > * `G1FreeCollectionSetTask` moved to `G1PostEvacuateCollectionSetCleanupTask2::FreeCollectionSetTask` > * `G1FreeHumongousRegionClosure` moved verbatim to `g1YoungGCPostEvacuateTasks.cpp`, and wrapped into a `G1AbstractSubTask` > > Minor other things: > * the condition for executing and printing logs for when doing eager reclaim has been unified: sometimes the logs have been printed without any eager reclaim because they did not correspond > > Feel free to ask questions. > > Testing: tier1-5 multiple times > > Thanks, > Thomas Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision: Remove test debug code ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3811/files - new: https://git.openjdk.java.net/jdk/pull/3811/files/1be3f1c9..5526184c Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3811&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3811&range=00-01 Stats: 7 lines in 1 file changed: 0 ins; 0 del; 7 mod Patch: https://git.openjdk.java.net/jdk/pull/3811.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3811/head:pull/3811 PR: https://git.openjdk.java.net/jdk/pull/3811 From pliden at openjdk.java.net Fri Apr 30 12:19:20 2021 From: pliden at openjdk.java.net (Per Liden) Date: Fri, 30 Apr 2021 12:19:20 GMT Subject: RFR: 8266056: runtime/stringtable/StringTableCleaningTest.java failed with "RuntimeException: Missing Callback in [10, 11]" Message-ID: This patch addresses two issues with the StringTableCleaningTest test. 1) The test doesn't fully take into account that a concurrent GC might not complete (it can abort if the VM wants to terminate). findCallback() should return -1 if no callbacks are found, instead of failing the test. 2) The regexp patterns used for ZGC needs to be more exact. The end pattern now also matches aborted GCs, which it shouldn't. This leads to "Garbage Collection (*) Aborted" being incorrectly matched as a GC end. This patch also makes sure the output from the test VM ends up in the test log, otherwise it's impossible to tell what went wrong if this test fails in the future. This patch also removes the test from the ProblemList. Testing: * Passed Tier1-3. * Manual testing of StringTableCleaningTest with various GCs. ------------- Commit messages: - 8266056: runtime/stringtable/StringTableCleaningTest.java failed with "RuntimeException: Missing Callback in [10, 11]" Changes: https://git.openjdk.java.net/jdk/pull/3815/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3815&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8266056 Stats: 8 lines in 2 files changed: 3 ins; 1 del; 4 mod Patch: https://git.openjdk.java.net/jdk/pull/3815.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3815/head:pull/3815 PR: https://git.openjdk.java.net/jdk/pull/3815 From coleenp at openjdk.java.net Fri Apr 30 12:23:53 2021 From: coleenp at openjdk.java.net (Coleen Phillimore) Date: Fri, 30 Apr 2021 12:23:53 GMT Subject: RFR: 8266056: runtime/stringtable/StringTableCleaningTest.java failed with "RuntimeException: Missing Callback in [10, 11]" In-Reply-To: References: Message-ID: On Fri, 30 Apr 2021 12:08:00 GMT, Per Liden wrote: > This patch addresses two issues with the StringTableCleaningTest test. > > 1) The test doesn't fully take into account that a concurrent GC might not complete (it can abort if the VM wants to terminate). findCallback() should return -1 if no callbacks are found, instead of failing the test. > > 2) The regexp patterns used for ZGC needs to be more exact. The end pattern now also matches aborted GCs, which it shouldn't. This leads to "Garbage Collection (*) Aborted" being incorrectly matched as a GC end. > > This patch also makes sure the output from the test VM ends up in the test log, otherwise it's impossible to tell what went wrong if this test fails in the future. > > This patch also removes the test from the ProblemList. > > Testing: > * Passed Tier1-3. > * Manual testing of StringTableCleaningTest with various GCs. Looks good and trivial. ------------- Marked as reviewed by coleenp (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3815 From pliden at openjdk.java.net Fri Apr 30 12:30:19 2021 From: pliden at openjdk.java.net (Per Liden) Date: Fri, 30 Apr 2021 12:30:19 GMT Subject: RFR: 8260267: ZGC: Reduce mark stack usage [v4] In-Reply-To: <1Qn76cgJaltPI2LtFQFfdqCG8SzOJwg3ORVQc8pK3MY=.918a159f-4f3d-408c-8cca-5b2bf727a0b2@github.com> References: <1Qn76cgJaltPI2LtFQFfdqCG8SzOJwg3ORVQc8pK3MY=.918a159f-4f3d-408c-8cca-5b2bf727a0b2@github.com> Message-ID: > Under some conditions, striped marking can cause excessive mark stack memory usage. For example, when the object graph has one or more N:1 relations ships, where N is a large number. > > To avoid pushing mark stack entries to already marked objects, and thus reduce mark stack memory usage, this patch adds does two things: > * GC threads will under some conditions mark the object before pushing, else (if the condition to mark failed) > * GC threads will check if the object is already marked before pushing. > > The conditions for marking before pushing is: > * We are using a single stripe, in which cases striped marking is essentially disabled anyway. > * We see excessive mark stack usage. In other words, the mark stack allocator indicates "high usage". > > To make this easier to review, this PR contains two commits: > * `ZGC: Reduce mark stack usage` - This patch adds the infrastructure needed to mark before push, but only does so if we are using a single stripe. > * `ZGC: Mark before push if mark stack usage is high` - This patch adds the condition to mark before push if the mark stack allocator indicates "high usage". As a bonus, this patch will also uncommit memory used for mark stacks after the marking has completed. > > Testing: > * Tier1-7 > * SPECjbb2015, score and marking times unaffected > * casparcwang at tencent.com reports that this patch, in combination with PR #3455, solves the problem for Apache Zookeeper. Per Liden has updated the pull request incrementally with one additional commit since the last revision: Review comments from StefanJ ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3460/files - new: https://git.openjdk.java.net/jdk/pull/3460/files/77a65449..f1a98aba Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3460&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3460&range=02-03 Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3460.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3460/head:pull/3460 PR: https://git.openjdk.java.net/jdk/pull/3460 From pliden at openjdk.java.net Fri Apr 30 12:30:20 2021 From: pliden at openjdk.java.net (Per Liden) Date: Fri, 30 Apr 2021 12:30:20 GMT Subject: RFR: 8260267: ZGC: Reduce mark stack usage [v3] In-Reply-To: References: <1Qn76cgJaltPI2LtFQFfdqCG8SzOJwg3ORVQc8pK3MY=.918a159f-4f3d-408c-8cca-5b2bf727a0b2@github.com> Message-ID: On Thu, 29 Apr 2021 19:27:28 GMT, Stefan Johansson wrote: >> Per Liden has updated the pull request incrementally with one additional commit since the last revision: >> >> Review comments from StefanJ and Albert > > src/hotspot/share/gc/z/zMarkStackAllocator.cpp line 72: > >> 70: >> 71: static bool is_high_usage(size_t size) { >> 72: // Consider usage to be high if we've used more than a 8th of the available space. > > "a 8th" -> "an 8th" or "one 8th" Fixed! > src/hotspot/share/gc/z/zMarkStackAllocator.cpp line 101: > >> 99: >> 100: log_debug(gc, marking)("Expanding mark stack space: " SIZE_FORMAT "M->" SIZE_FORMAT "M (%s Usage)", >> 101: old_size / M, new_size / M, _high_usage ? "High" : "Low"); > > A bit strange to only have "Low" and "High" usage, maybe "Normal" and "High" would be more accurate. Fixed! ------------- PR: https://git.openjdk.java.net/jdk/pull/3460 From dholmes at openjdk.java.net Fri Apr 30 12:42:11 2021 From: dholmes at openjdk.java.net (David Holmes) Date: Fri, 30 Apr 2021 12:42:11 GMT Subject: RFR: 8266056: runtime/stringtable/StringTableCleaningTest.java failed with "RuntimeException: Missing Callback in [10, 11]" In-Reply-To: References: Message-ID: On Fri, 30 Apr 2021 12:08:00 GMT, Per Liden wrote: > This patch addresses two issues with the StringTableCleaningTest test. > > 1) The test doesn't fully take into account that a concurrent GC might not complete (it can abort if the VM wants to terminate). findCallback() should return -1 if no callbacks are found, instead of failing the test. > > 2) The regexp patterns used for ZGC needs to be more exact. The end pattern now also matches aborted GCs, which it shouldn't. This leads to "Garbage Collection (*) Aborted" being incorrectly matched as a GC end. > > This patch also makes sure the output from the test VM ends up in the test log, otherwise it's impossible to tell what went wrong if this test fails in the future. > > This patch also removes the test from the ProblemList. > > Testing: > * Passed Tier1-3. > * Manual testing of StringTableCleaningTest with various GCs. Functional fix seems fine but a suggested change to the reporting. Thanks, David test/hotspot/jtreg/runtime/stringtable/StringTableCleaningTest.java line 63: > 61: OutputAnalyzer output = ProcessTools.executeTestJvm(subargs); > 62: output.outputTo(System.out); > 63: output.errorTo(System.out); The normal way to do this is to call output.reportDiagnosticSummary(). But you only do that after the actual checks, when you know you have success, as in the error cases it will be called anyway and you don't want the output twice. ------------- Marked as reviewed by dholmes (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3815 From pliden at openjdk.java.net Fri Apr 30 13:08:15 2021 From: pliden at openjdk.java.net (Per Liden) Date: Fri, 30 Apr 2021 13:08:15 GMT Subject: RFR: 8266056: runtime/stringtable/StringTableCleaningTest.java failed with "RuntimeException: Missing Callback in [10, 11]" [v2] In-Reply-To: References: Message-ID: <48rwfXGoo5MF4daRoiJ_6-mPKs6S49GOyp6wk45i60o=.1b9ebaa6-e85f-4abe-9222-8aa01903d9f5@github.com> > This patch addresses two issues with the StringTableCleaningTest test. > > 1) The test doesn't fully take into account that a concurrent GC might not complete (it can abort if the VM wants to terminate). findCallback() should return -1 if no callbacks are found, instead of failing the test. > > 2) The regexp patterns used for ZGC needs to be more exact. The end pattern now also matches aborted GCs, which it shouldn't. This leads to "Garbage Collection (*) Aborted" being incorrectly matched as a GC end. > > This patch also makes sure the output from the test VM ends up in the test log, otherwise it's impossible to tell what went wrong if this test fails in the future. > > This patch also removes the test from the ProblemList. > > Testing: > * Passed Tier1-3. > * Manual testing of StringTableCleaningTest with various GCs. Per Liden has updated the pull request incrementally with one additional commit since the last revision: Review comments from David ------------- Changes: - all: https://git.openjdk.java.net/jdk/pull/3815/files - new: https://git.openjdk.java.net/jdk/pull/3815/files/f98123ce..d991bbc3 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3815&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3815&range=00-01 Stats: 2 lines in 1 file changed: 0 ins; 2 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/3815.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3815/head:pull/3815 PR: https://git.openjdk.java.net/jdk/pull/3815 From pliden at openjdk.java.net Fri Apr 30 13:08:16 2021 From: pliden at openjdk.java.net (Per Liden) Date: Fri, 30 Apr 2021 13:08:16 GMT Subject: RFR: 8266056: runtime/stringtable/StringTableCleaningTest.java failed with "RuntimeException: Missing Callback in [10, 11]" [v2] In-Reply-To: References: Message-ID: On Fri, 30 Apr 2021 12:36:13 GMT, David Holmes wrote: >> Per Liden has updated the pull request incrementally with one additional commit since the last revision: >> >> Review comments from David > > test/hotspot/jtreg/runtime/stringtable/StringTableCleaningTest.java line 63: > >> 61: OutputAnalyzer output = ProcessTools.executeTestJvm(subargs); >> 62: output.outputTo(System.out); >> 63: output.errorTo(System.out); > > The normal way to do this is to call output.reportDiagnosticSummary(). But you only do that after the actual checks, when you know you have success, as in the error cases it will be called anyway and you don't want the output twice. Hmm, I see now that adding lines those is unnecessary, so I'll just remove them. Both shouldHaveExitValue() and checkOutput() seem to do the right thing. Not sure why I thought they didn't. ------------- PR: https://git.openjdk.java.net/jdk/pull/3815 From sjohanss at openjdk.java.net Fri Apr 30 13:09:59 2021 From: sjohanss at openjdk.java.net (Stefan Johansson) Date: Fri, 30 Apr 2021 13:09:59 GMT Subject: RFR: 8257774: G1: Trigger collect when free region count drops below threshold to prevent evacuation failures [v5] In-Reply-To: References: <5Ro0P4NPaUuG-LPpgqHY5S_kh_EVpwRgxqc5jTCano4=.afc3f2a4-dc9e-46c1-ac84-298cf143fa41@github.com> Message-ID: On Thu, 29 Apr 2021 18:41:24 GMT, Aditya Mandaleeka wrote: >> _This PR picks up from [this previous PR](https://github.com/openjdk/jdk/pull/1650) by @charliegracie._ >> >> I won't repeat the full description from the prior PR, but the general idea is to add the notion of a "proactive GC" to G1 which gets triggered in the slow allocation path if the number of free regions drops below the amount that would be required to complete a GC if it happened at that moment. The threshold is based on the survival rates from eden and survivor spaces along with the space required for tenured space evacuations. >> >> There are a couple of outstanding issues/questions known: >> - _Interaction with GCLocker_: In the case where we determine that a proactive GC is required and GC locker is active, we don't allow the young gen to expand (instead threads will stall). @tschatzl raised this and suggested that it should be discussed as part of the review. >> - _Disable proactive GC heuristic during initialization_: I ran into an issue in testing where code in the universe initialization codepath was tripping the proactive GC heuristic, leading to a GC being triggered before the VM has finished initialization. I need to find a good way to prevent this from happening. There may already be a mechanism for this, but I couldn't find one so I added a temporary placeholder (`zzinit_complete`) just to unblock testing. > > Aditya Mandaleeka has updated the pull request incrementally with one additional commit since the last revision: > > Remove temp comment. I think this is starting to look pretty good. No major comments at this point, but I will have to take one more look in detail at the calculations in the policy. src/hotspot/share/gc/g1/g1AllocRegion.hpp line 173: > 171: // to the locking protocol. > 172: // Tries to allocate at least min_word_size words, and at most desired_word_size. > 173: // Returns the actual size of the block in actual_word_size. Please format these lines to fit the rest of the comment: Suggestion: // to the locking protocol. The min and desired word size allow // specifying a minimum and maximum size of the allocation. The // actual size of allocation is returned in actual_word_size. `use_retained_region_if_available` is no longer used remove from comment. src/hotspot/share/gc/g1/g1Allocator.inline.hpp line 56: > 54: uint node_index = current_node_index(); > 55: > 56: Remove at least one of those two blank lines. src/hotspot/share/gc/g1/g1Policy.cpp line 1422: > 1420: } > 1421: > 1422: if (_g1h->young_regions_count() == 0 && _collection_set->candidates() != NULL && _collection_set->candidates()->is_empty()) { Add a comment that we have no regions to collect. Adding a `has_candidates()` to `G1CollectionSet` seems to make sense now when we check for candidates a couple of times and will make this condition read a bit better as well: Suggestion: if (_g1h->young_regions_count() == 0 && !_collection_set->has_candidates()) { // Don't attempt a proactive GC when there are no regions to collect. src/hotspot/share/gc/g1/g1Policy.cpp line 1468: > 1466: // Old regions > 1467: G1CollectionSetCandidates *candidates = _collection_set->candidates(); > 1468: if (candidates == NULL || candidates->is_empty()) { Could use `!_collection_set->has_candidates()` if added. src/hotspot/share/gc/g1/g1VMOperations.cpp line 125: > 123: G1CollectedHeap* g1h = G1CollectedHeap::heap(); > 124: > 125: if (_word_size > 0 && _gc_cause != GCCause::_g1_proactive_collection) { What do you think about adding a helper to ?VM_G1CollectForAllocation`: bool try_allocate_before_gc() { if (_gc_cause == GCCause::_g1_proactive_collection)?{ // Never allocate before proactive GCs. return false; } return true; } The condition would then read a bit better: Suggestion: if (try_allocate_before_gc() && _word_size > 0) { ------------- Changes requested by sjohanss (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/3143 From pliden at openjdk.java.net Fri Apr 30 13:18:20 2021 From: pliden at openjdk.java.net (Per Liden) Date: Fri, 30 Apr 2021 13:18:20 GMT Subject: RFR: 8265984: Concurrent GC: Some tests fail "assert(is_frame_safe(f)) failed: Frame must be safe" Message-ID: JDK-8265702 introduced a change in `OptoRuntime::handle_exception_C()` where it no longer processed frames when asking for the caller. This change turned out to cause intermittent test failures. We shouldn't have to process frames here, since we're not touching any oops, but it seems that processing frames here avoids some other issue. To avoid noise in out testing I suggest we revert to processing frames here for now (and add the required ThreadWXEnable for macos/aarch64), while we continue to investigate why this is needed. Testing: Passed tier1-7 on linux/x86 and macos/aarch64 ------------- Commit messages: - 8265984: Concurrent GC: Some tests fail "assert(is_frame_safe(f)) failed: Frame must be safe" Changes: https://git.openjdk.java.net/jdk/pull/3814/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3814&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8265984 Stats: 5 lines in 2 files changed: 3 ins; 0 del; 2 mod Patch: https://git.openjdk.java.net/jdk/pull/3814.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/3814/head:pull/3814 PR: https://git.openjdk.java.net/jdk/pull/3814 From zgu at openjdk.java.net Fri Apr 30 14:06:03 2021 From: zgu at openjdk.java.net (Zhengyu Gu) Date: Fri, 30 Apr 2021 14:06:03 GMT Subject: RFR: 8265984: Concurrent GC: Some tests fail "assert(is_frame_safe(f)) failed: Frame must be safe" In-Reply-To: References: Message-ID: On Fri, 30 Apr 2021 11:13:10 GMT, Per Liden wrote: > JDK-8265702 introduced a change in `OptoRuntime::handle_exception_C()` where it no longer processed frames when asking for the caller. This change turned out to cause intermittent test failures. We shouldn't have to process frames here, since we're not touching any oops, but it seems that processing frames here avoids some other issue. To avoid noise in out testing I suggest we revert to processing frames here for now (and add the required ThreadWXEnable for macos/aarch64), while we continue to investigate why this is needed. > > Testing: Passed tier1-7 on linux/x86 and macos/aarch64 src/hotspot/share/opto/runtime.cpp line 1380: > 1378: > 1379: if (nm != NULL) { > 1380: RegisterMap map(current, false); Should you add `MACOS_AARCH64_ONLY(ThreadWXEnable wx(WXWrite, current));` here, as you suggested in CR? Seems the code change is inconsistent with the comments above. ------------- PR: https://git.openjdk.java.net/jdk/pull/3814