From mikael.vidstedt at oracle.com Tue Jul 10 19:59:55 2018 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 10 Jul 2018 12:59:55 -0700 Subject: RFR(XS): 8207007: Add missing license header to zHash.inline.hpp Message-ID: <1DAD0F32-68FF-4F09-BF23-56CBFDC1F992@oracle.com> Please review this small change which adds the missing license header to one of the ZGC files: Bug: https://bugs.openjdk.java.net/browse/JDK-8207007 Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8207007/webrev.00/open/webrev/ Cheers, Mikael From thomas.schatzl at oracle.com Tue Jul 10 20:26:19 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 10 Jul 2018 22:26:19 +0200 Subject: RFR(XS): 8207007: Add missing license header to zHash.inline.hpp In-Reply-To: <1DAD0F32-68FF-4F09-BF23-56CBFDC1F992@oracle.com> References: <1DAD0F32-68FF-4F09-BF23-56CBFDC1F992@oracle.com> Message-ID: <58fb563e491cf8959023c518b115085c4e3ad4e0.camel@oracle.com> Hi, On Tue, 2018-07-10 at 12:59 -0700, Mikael Vidstedt wrote: > Please review this small change which adds the missing license header > to one of the ZGC files: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8207007 > Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8207007/webrev.00/ > open/webrev/ ev.00/open/webrev/> > looks good to me. Thomas From per.liden at oracle.com Tue Jul 10 21:00:32 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 10 Jul 2018 23:00:32 +0200 Subject: RFR(XS): 8207007: Add missing license header to zHash.inline.hpp In-Reply-To: <1DAD0F32-68FF-4F09-BF23-56CBFDC1F992@oracle.com> References: <1DAD0F32-68FF-4F09-BF23-56CBFDC1F992@oracle.com> Message-ID: <17ad62d5-9998-755a-fab1-eafd0eea4c97@oracle.com> Looks good /Per On 2018-07-10 21:59, Mikael Vidstedt wrote: > > Please review this small change which adds the missing license header to one of the ZGC files: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8207007 > Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8207007/webrev.00/open/webrev/ > > Cheers, > Mikael > From simone.bordet at gmail.com Fri Jul 13 17:35:12 2018 From: simone.bordet at gmail.com (Simone Bordet) Date: Fri, 13 Jul 2018 19:35:12 +0200 Subject: Understanding ZGC details Message-ID: Hi, first of all congrats and thanks for ZGC. I've been running a benchmark of mine (https://docs.cometd.org/current/reference/#_benchmarking), a Servlet-based chat application on top of Jetty, to compare G1, Shenandoah and ZGC. The hope is to produce some conference material to spread the word about ZGC (for which personally I have high expectations). Meanwhile I'm trying to understand a bit more about ZGC and I have few questions (I have read the GC handbook from Jones et al., but I'm new to Linux memory tricks so please pardon my ignorance on the matter; I have also watched Per's presentation at JFokus 2018). I don't quite get the mechanism of colored pointers and how it exactly works. I see that ZGC maps the heap space into 3 views (in ZPhysicalMemoryBacking::map()). What is the purpose of mapping into 3 views (I get the number 3, it's for the number of phases; but why cannot they be handled with just 1 view)? Skimming around in the code - not deeply, I admit - seems to be that pointers to those views are always de-colored before being dereferenced, so I am a bit puzzled about those 3 views. Assuming I'm wrong, is the mechanism similar to C4, where a view is mprotect()-ed on a phase flip? If so, I am also assuming that this would generate a SEGV that will run some code that will fix the pointer. Can you please detail where is the SEGV handling code that does that? I'm stopping here feeling that I'm offroad already :) I'm sure if you put me back on the road I'll have more questions. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From gil at azul.com Fri Jul 13 17:52:45 2018 From: gil at azul.com (Gil Tene) Date: Fri, 13 Jul 2018 17:52:45 +0000 Subject: Understanding ZGC details In-Reply-To: References: Message-ID: > On Jul 13, 2018, at 10:35 AM, Simone Bordet wrote: > > Hi, > > first of all congrats and thanks for ZGC. > > I've been running a benchmark of mine > (https://docs.cometd.org/current/reference/#_benchmarking), a > Servlet-based chat application on top of Jetty, to compare G1, > Shenandoah and ZGC. > The hope is to produce some conference material to spread the word > about ZGC (for which personally I have high expectations). > > Meanwhile I'm trying to understand a bit more about ZGC and I have few > questions (I have read the GC handbook from Jones et al., but I'm new > to Linux memory tricks so please pardon my ignorance on the matter; I > have also watched Per's presentation at JFokus 2018). > > I don't quite get the mechanism of colored pointers and how it exactly works. > I see that ZGC maps the heap space into 3 views (in > ZPhysicalMemoryBacking::map()). > What is the purpose of mapping into 3 views (I get the number 3, it's > for the number of phases; but why cannot they be handled with just 1 > view)? > > Skimming around in the code - not deeply, I admit - seems to be that > pointers to those views are always de-colored before being > dereferenced, so I am a bit puzzled about those 3 views. > Assuming I'm wrong, is the mechanism similar to C4, where a view is > mprotect()-ed on a phase flip? > If so, I am also assuming that this would generate a SEGV that will > run some code that will fix the pointer. Can you please detail where > is the SEGV handling code that does that? Speaking about the C4 mechanism: while C4 does perform various mapping manipulations for various reasons, LVB detection does not use SEGVs and does not rely on memory protection. Instead, The C4 LVB fast path uses a test & jmp (a single u-op on x86) to verify that the loaded reference is not in a "currently invalid" phase, with a slow path that fixes the reference to be in a valid phase (doing whatever is needed to achieve that: ensure marking, fixup to point to relocated object location, or perform actual object relocation if needed). My understanding is the the ZGC barrier mechanism is doing pretty much the same thing including the use of a test & jmp for the fast path. It is basically a C4 LVB. BTW, adding Zing to the set of JVM configs you comparatively test above would be a good idea ;-) > > I'm stopping here feeling that I'm offroad already :) > > I'm sure if you put me back on the road I'll have more questions. > > Thanks! > > -- > Simone Bordet > --- > Finally, no matter how good the architecture and design are, > to deliver bug-free software with optimal performance and reliability, > the implementation technique must be flawless. Victoria Livschitz From per.liden at oracle.com Sat Jul 14 14:08:29 2018 From: per.liden at oracle.com (Per Liden) Date: Sat, 14 Jul 2018 16:08:29 +0200 Subject: Understanding ZGC details In-Reply-To: References: Message-ID: <2e31a5dc-2598-d671-13b2-415aa2a5ed1f@oracle.com> Hi Simone, On 2018-07-13 19:35, Simone Bordet wrote: > Hi, > > first of all congrats and thanks for ZGC. Thanks! > > I've been running a benchmark of mine > (https://docs.cometd.org/current/reference/#_benchmarking), a > Servlet-based chat application on top of Jetty, to compare G1, > Shenandoah and ZGC. > The hope is to produce some conference material to spread the word > about ZGC (for which personally I have high expectations). Cool! > > Meanwhile I'm trying to understand a bit more about ZGC and I have few > questions (I have read the GC handbook from Jones et al., but I'm new > to Linux memory tricks so please pardon my ignorance on the matter; I > have also watched Per's presentation at JFokus 2018). > > I don't quite get the mechanism of colored pointers and how it exactly works. > I see that ZGC maps the heap space into 3 views (in > ZPhysicalMemoryBacking::map()). > What is the purpose of mapping into 3 views (I get the number 3, it's > for the number of phases; but why cannot they be handled with just 1 > view)? > > Skimming around in the code - not deeply, I admit - seems to be that > pointers to those views are always de-colored before being > dereferenced, so I am a bit puzzled about those 3 views. Actually, pointers are not de-colored, which is the reason why we need the three heap views on x86 (where we also don't have HW support for VA-masking/de-coloring). Slide 33 in the JFokus slide pack tries to illustrate this. (Internally in ZGC we sometimes remove the colors and the heap base, and we call the result of that an "offset". These "offsets" are however a pure internal thing and these are never exposed to or used by Java threads). > Assuming I'm wrong, is the mechanism similar to C4, where a view is > mprotect()-ed on a phase flip? > If so, I am also assuming that this would generate a SEGV that will > run some code that will fix the pointer. Can you please detail where > is the SEGV handling code that does that? I think Gil might have answered this already, but just to reiterate, ZGC does not use SEGV to catch bad pointers. The load barrier (the test-and-jump instruction sequence) will catch all pointers with "bad" colors before they are used. cheers, Per (I'm on vacation at the moment, so my e-mail responsiveness might not be super good during the next two weeks) > > I'm stopping here feeling that I'm offroad already :) > > I'm sure if you put me back on the road I'll have more questions. > > Thanks! > From simone.bordet at gmail.com Mon Jul 16 14:08:06 2018 From: simone.bordet at gmail.com (Simone Bordet) Date: Mon, 16 Jul 2018 16:08:06 +0200 Subject: Understanding ZGC details In-Reply-To: References: Message-ID: Hi Gil, keeping this between us since it's not really relevant for ZGC :) On Fri, Jul 13, 2018 at 7:52 PM Gil Tene wrote: > Speaking about the C4 mechanism: while C4 does perform various mapping manipulations > for various reasons, LVB detection does not use SEGVs and does not rely on memory > protection. Instead, The C4 LVB fast path uses a test & jmp (a single u-op on x86) to verify > that the loaded reference is not in a "currently invalid" phase, with a slow path that fixes the > reference to be in a valid phase (doing whatever is needed to achieve that: ensure marking, > fixup to point to relocated object location, or perform actual object relocation if needed). Ok. So I'm reading the GC Handbook from Jones et al. 2nd edition, and they say that Pauless and C4 protects from-pages. If I read correctly, from-pages are protected so that if a mutator dereferences a pointer, it will trap, and the trap will "fix" the pointer, either by copying it to to-space (so the mutator does the job), or by looking at side metadata for the new address. I understand that if there is a JIT-injected load barrier always present, this work of "fixing" the pointer can be done by the slow path of the load barrier, but then (for what pertains pointers) protecting pages is not necessary; but then, why the GC handbook mentions them? Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From gil at azul.com Mon Jul 16 16:07:44 2018 From: gil at azul.com (Gil Tene) Date: Mon, 16 Jul 2018 16:07:44 +0000 Subject: Understanding ZGC details In-Reply-To: References: , Message-ID: <591A92E9-812C-44FC-9B19-E677CB857E2F@azul.com> Sent from my iPad On Jul 16, 2018, at 7:08 AM, Simone Bordet > wrote: Hi Gil, keeping this between us since it's not really relevant for ZGC :) On Fri, Jul 13, 2018 at 7:52 PM Gil Tene > wrote: Speaking about the C4 mechanism: while C4 does perform various mapping manipulations for various reasons, LVB detection does not use SEGVs and does not rely on memory protection. Instead, The C4 LVB fast path uses a test & jmp (a single u-op on x86) to verify that the loaded reference is not in a "currently invalid" phase, with a slow path that fixes the reference to be in a valid phase (doing whatever is needed to achieve that: ensure marking, fixup to point to relocated object location, or perform actual object relocation if needed). Ok. So I'm reading the GC Handbook from Jones et al. 2nd edition, and they say that Pauless and C4 protects from-pages. If I read correctly, from-pages are protected so that if a mutator dereferences a pointer, it will trap, and the trap will "fix" the pointer, either by copying it to to-space (so the mutator does the job), or by looking at side metadata for the new address. Look at the C4 ISMM paper (https://www.azul.com/files/c4_paper_acm2.pdf) for the actual terminology and use. The paper defines (in section 2) what an LVB is, the invariants it maintains, and explains the various ways to ensure barriered references match an expected state. It also discusses implementations variants (section 4), where it explains how LVB fast path test implementations could vary dramatically (e.g. between interpreted, JIT?ed, runtime C++ codes, or on different hardware architectures, and obviously by specific implementation choices) without changing semantics, and provides pseudo code (appendix A) for the logical coverage. The TLDR summary in the context of your above question would be: ?Protected? means ?logically protected (by the algorithm) from mutator access?. ?Trap? or ?trigger? means ?do something other than the fast path?., and ?Page? means ?a power-of-2 aligned range of addresses?. Virtual memory protection and privilege-changing trapping is one possible (but not very efficient or practical for the common case) way to implement an LVB, but the vast majority of LVB tests are implemented via user-mode testing and conditional branching. Adhering to actual virtual memory page boundaries, and actually matching collector protection with virtual memory protection has some bonus logical benefits, including allowing multiple implementations of the same LVB semantics enforcement in the same code base, as well as enhanced error detection that goes beyond the LVB semantics requirements (but helps a lot in stabilizing actual collector implementation, which like all software, may include bugs at some point). The fast path of an LVB test can be optimized to a test-and-jump, and that?s how it?s been implemented for roughly a decade for reasonable (currently up to 2TB) heap sizes. The size of the mask being tested and the amount of pre-shifting involved (if any) determines the possible granularity of protected page set selection for a relocation phase flip. E.g. selecting the entire (up to 2TB on x86) heap or an entire generation at the same time allows for no shifting at all (mask in place), test-and-jump. Selecting sub-phases (e.g. relocating 1/8th or 1/16th of the heap at a time) can fit with 64 bit register masks using an extra shift, higher granularity can be achieved with vector registers, and arbitrary granularity (down to a single page if wished for) could be achieved by testing against a memory (rather than a register) bitmask. I understand that if there is a JIT-injected load barrier always present, this work of "fixing" the pointer can be done by the slow path of the load barrier, but then (for what pertains pointers) protecting pages is not necessary; but then, why the GC handbook mentions them? I believe that the current edition of the handbook was finalized before they had access to the published C4 ISM! paper... The fact that some C4 algorithm implementations (like the ones we actually use in our products) *also* use virtual page mapping and ritual page protection for unrelated (as in not algorithmically necessary but otherwise useful) purposes, like error detection, additional optimization opportunities, and code stabilization, may have confused the C4 algorithm description with some implementation concerns. Thanks! -- Simone Bordet --- Finally, no matter how good the architecture and design are, to deliver bug-free software with optimal performance and reliability, the implementation technique must be flawless. Victoria Livschitz From brett.wooldridge at gmail.com Tue Jul 17 19:02:56 2018 From: brett.wooldridge at gmail.com (Brett Wooldridge) Date: Wed, 18 Jul 2018 04:02:56 +0900 Subject: Class Unloading Message-ID: I have two questions: 1/ I understand from the slides that class unloading is "not yet". But does this mean that class unloading is simply not present, or that it *is present* but not parallel? 2/ I seem recall reading somewhere that ZGC does not play nice with GraalVM. Is that true and/or still true? Thanks for the awesome work on ZGC! -Brett p.s. a case of your choice of beer to whoever gets ZGC into the official JDK builds for MacOS X. From per.liden at oracle.com Tue Jul 17 19:44:22 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 17 Jul 2018 21:44:22 +0200 Subject: Class Unloading In-Reply-To: References: Message-ID: <1d29b27d-d40d-58a2-c491-4a683fb8dd28@oracle.com> Hi Brett, On 2018-07-17 21:02, Brett Wooldridge wrote: > I have two questions: > > 1/ I understand from the slides that class unloading is "not yet". But > does this mean that class unloading is simply not present, or that it *is > present* but not parallel? It means it's not present at all. But work in ongoing and it will most likely be there in JDK 12. > > 2/ I seem recall reading somewhere that ZGC does not play nice with > GraalVM. Is that true and/or still true? Yes, the Graal JIT does not yet know how to generate load barriers for ZGC. But it's on our road map. > > Thanks for the awesome work on ZGC! Thanks! /Per > > -Brett > > p.s. a case of your choice of beer to whoever gets ZGC into the official > JDK builds for MacOS X. > From coleen.phillimore at oracle.com Tue Jul 17 19:57:24 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 17 Jul 2018 15:57:24 -0400 Subject: Class Unloading In-Reply-To: References: Message-ID: <0fd2cbad-c3fa-ff39-b624-4bfd24c4ebe4@oracle.com> On 7/17/18 3:02 PM, Brett Wooldridge wrote: > I have two questions: > > 1/ I understand from the slides that class unloading is "not yet". But > does this mean that class unloading is simply not present, or that it *is > present* but not parallel? It's not presently done.? We're currently working on stop the world class unloading first and then concurrent class unloading next. I'll leave it to the others to answer the second question (I believe that the load barriers are missing in graal so ZGC won't work with graal), and to claim the beer. thanks, Coleen > > 2/ I seem recall reading somewhere that ZGC does not play nice with > GraalVM. Is that true and/or still true? > > Thanks for the awesome work on ZGC! > > -Brett > > p.s. a case of your choice of beer to whoever gets ZGC into the official > JDK builds for MacOS X. From yasuenag at gmail.com Wed Jul 18 02:59:35 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 18 Jul 2018 11:59:35 +0900 Subject: RFR: 8207756: jstat should show CGC STW phases of ZGC Message-ID: Hi all, Please review this change: JBS: https://bugs.openjdk.java.net/browse/JDK-8207756 webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/ jstat shows CGC STW phases as CGC and CGCT fields since JDK-8153333 . However, ZGC does not adapt to it yet as following: ``` $ jstat -gc 1234 1 1 S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT CGC CGCT GCT - - - - - - 514048.0 12288.0 4352.0 3731.2 0.0 0.0 - - - - - - 0.000 ``` I uploaded webrev which is for jdk/jdk repo. Please tell me if it should be for zgc repo. I'm not an author of ZGC. So I need a sponsor. (I'm a Reviewer of jdk. If this change should be pushed to jdk/jdk, please tell me. I will push it after reviewing.) Thanks, Yasumasa From yasuenag at gmail.com Thu Jul 19 14:03:24 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 19 Jul 2018 23:03:24 +0900 Subject: RFR: 8207843: HSDB cannot show Object Histogram when ZGC is working Message-ID: <0e8ce2e6-e043-2320-e660-a2f1f4df820e@gmail.com> Hi all, Please review this webrev. JBS: https://bugs.openjdk.java.net/browse/JDK-8207843 webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207843/webrev.00/ I encountered AssertionFailure when I attached HSDB to the process which is working with ZGC as below: sun.jvm.hotspot.utilities.AssertionFailure: Unexpected CollectedHeap type: sun.jvm.hotspot.gc.z.ZCollectedHeap at jdk.hotspot.agent/sun.jvm.hotspot.utilities.Assert.that(Assert.java:32) at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.collectLiveRegions(ObjectHeap.java:448) at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.iterate(ObjectHeap.java:173) at jdk.hotspot.agent/sun.jvm.hotspot.HSDB$VisitHeap.run(HSDB.java:1741) at jdk.hotspot.agent/sun.jvm.hotspot.utilities.WorkerThread$MainLoop.run(WorkerThread.java:70) at java.base/java.lang.Thread.run(Thread.java:832) ObjectHeap#collectLiveRegions() branches by instance type of CollectedHeap. However it does not support ZCollectedHeap. So I add ZCollectedHeap to it and add some methods to iterate ZPageTable. Thanks, Yasumasa From yasuenag at gmail.com Thu Jul 26 12:30:15 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 26 Jul 2018 21:30:15 +0900 Subject: PING: RFR: 8207756: jstat should show CGC STW phases of ZGC In-Reply-To: References: Message-ID: <11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com> PING: Could you review it? > webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/ Yasumasa On 2018/07/18 11:59, Yasumasa Suenaga wrote: > Hi all, > > Please review this change: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8207756 > webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/ > > jstat shows CGC STW phases as CGC and CGCT fields since JDK-8153333 . > However, ZGC does not adapt to it yet as following: > > ``` > $ jstat -gc 1234 1 1 > S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT CGC CGCT GCT > - - - - - - 514048.0 12288.0 4352.0 3731.2 0.0 0.0 - - - - - - 0.000 > ``` > > I uploaded webrev which is for jdk/jdk repo. Please tell me if it > should be for zgc repo. > I'm not an author of ZGC. So I need a sponsor. > (I'm a Reviewer of jdk. If this change should be pushed to jdk/jdk, > please tell me. I will push it after reviewing.) > > > Thanks, > > Yasumasa > From yasuenag at gmail.com Thu Jul 26 12:30:44 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 26 Jul 2018 21:30:44 +0900 Subject: PING: RFR: 8207843: HSDB cannot show Object Histogram when ZGC is working In-Reply-To: <0e8ce2e6-e043-2320-e660-a2f1f4df820e@gmail.com> References: <0e8ce2e6-e043-2320-e660-a2f1f4df820e@gmail.com> Message-ID: <3d4f8faf-e26a-0e6b-6df1-73b6600ee5a0@gmail.com> PING: Could you review it? > webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207843/webrev.00/ Yasumasa On 2018/07/19 23:03, Yasumasa Suenaga wrote: > Hi all, > > Please review this webrev. > > ???? JBS: https://bugs.openjdk.java.net/browse/JDK-8207843 > ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207843/webrev.00/ > > I encountered AssertionFailure when I attached HSDB to the process which is working with ZGC as below: > > sun.jvm.hotspot.utilities.AssertionFailure: Unexpected CollectedHeap type: sun.jvm.hotspot.gc.z.ZCollectedHeap > ??? at jdk.hotspot.agent/sun.jvm.hotspot.utilities.Assert.that(Assert.java:32) > ??? at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.collectLiveRegions(ObjectHeap.java:448) > ??? at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.iterate(ObjectHeap.java:173) > ??? at jdk.hotspot.agent/sun.jvm.hotspot.HSDB$VisitHeap.run(HSDB.java:1741) > ??? at jdk.hotspot.agent/sun.jvm.hotspot.utilities.WorkerThread$MainLoop.run(WorkerThread.java:70) > ??? at java.base/java.lang.Thread.run(Thread.java:832) > > ObjectHeap#collectLiveRegions() branches by instance type of CollectedHeap. However it does not support ZCollectedHeap. > So I add ZCollectedHeap to it and add some methods to iterate ZPageTable. > > > Thanks, > > Yasumasa From erik.osterlund at oracle.com Thu Jul 26 12:54:01 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 26 Jul 2018 14:54:01 +0200 Subject: PING: RFR: 8207756: jstat should show CGC STW phases of ZGC In-Reply-To: <11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com> References: <11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com> Message-ID: <5B59C469.4060909@oracle.com> Hi Yasumasa, After the integration to jdk-jdk, we want smaller things like this to go in upstream instead. So I think this should go into the hotspot-gc list and be pushed upstream. You can put a ZGC: prefix to the email. Thanks, /Erik On 2018-07-26 14:30, Yasumasa Suenaga wrote: > PING: Could you review it? > >> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/ > > > Yasumasa > > > On 2018/07/18 11:59, Yasumasa Suenaga wrote: >> Hi all, >> >> Please review this change: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8207756 >> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/ >> >> jstat shows CGC STW phases as CGC and CGCT fields since JDK-8153333 . >> However, ZGC does not adapt to it yet as following: >> >> ``` >> $ jstat -gc 1234 1 1 >> S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT CGC >> CGCT GCT >> - - - - - - 514048.0 12288.0 4352.0 3731.2 0.0 0.0 - - - - - - 0.000 >> ``` >> >> I uploaded webrev which is for jdk/jdk repo. Please tell me if it >> should be for zgc repo. >> I'm not an author of ZGC. So I need a sponsor. >> (I'm a Reviewer of jdk. If this change should be pushed to jdk/jdk, >> please tell me. I will push it after reviewing.) >> >> >> Thanks, >> >> Yasumasa >> From per.liden at oracle.com Thu Jul 26 13:30:14 2018 From: per.liden at oracle.com (=?utf-8?Q?Per_Lid=C3=A9n?=) Date: Thu, 26 Jul 2018 15:30:14 +0200 Subject: PING: RFR: 8207756: jstat should show CGC STW phases of ZGC In-Reply-To: <11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com> References: <11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com> Message-ID: <107AD8D6-1E1F-4469-9352-8CCD3C84AA67@oracle.com> Hi, Please have a look at the zServiceability class, which is where code like this is intended to live. /Per > On 26 Jul 2018, at 14:30, Yasumasa Suenaga wrote: > > PING: Could you review it? > >> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/ > > > Yasumasa > > >> On 2018/07/18 11:59, Yasumasa Suenaga wrote: >> Hi all, >> Please review this change: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8207756 >> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/ >> jstat shows CGC STW phases as CGC and CGCT fields since JDK-8153333 . >> However, ZGC does not adapt to it yet as following: >> ``` >> $ jstat -gc 1234 1 1 >> S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT CGC CGCT GCT >> - - - - - - 514048.0 12288.0 4352.0 3731.2 0.0 0.0 - - - - - - 0.000 >> ``` >> I uploaded webrev which is for jdk/jdk repo. Please tell me if it >> should be for zgc repo. >> I'm not an author of ZGC. So I need a sponsor. >> (I'm a Reviewer of jdk. If this change should be pushed to jdk/jdk, >> please tell me. I will push it after reviewing.) >> Thanks, >> Yasumasa From yasuenag at gmail.com Thu Jul 26 13:44:30 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 26 Jul 2018 22:44:30 +0900 Subject: PING: RFR: 8207756: jstat should show CGC STW phases of ZGC In-Reply-To: <107AD8D6-1E1F-4469-9352-8CCD3C84AA67@oracle.com> References: <11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com> <107AD8D6-1E1F-4469-9352-8CCD3C84AA67@oracle.com> Message-ID: <51cc06a1-b027-4208-f2d9-327cbd009564@gmail.com> CC'ing to hotspot-gc-dev Hi Per, I've looked at ZServiceabilityCounters in zServiceability.cpp . IMHO it is unsuitable because CGC counter means STW in concurrent GC. So I think it is suitable to implement in VMOperation for ZGC. Thanks, Yasumasa On 2018/07/26 22:30, Per Lid?n wrote: > Hi, > > Please have a look at the zServiceability class, which is where code like this is intended to live. > > /Per > >> On 26 Jul 2018, at 14:30, Yasumasa Suenaga wrote: >> >> PING: Could you review it? >> >>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/ >> >> >> Yasumasa >> >> >>> On 2018/07/18 11:59, Yasumasa Suenaga wrote: >>> Hi all, >>> Please review this change: >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8207756 >>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/ >>> jstat shows CGC STW phases as CGC and CGCT fields since JDK-8153333 . >>> However, ZGC does not adapt to it yet as following: >>> ``` >>> $ jstat -gc 1234 1 1 >>> S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT CGC CGCT GCT >>> - - - - - - 514048.0 12288.0 4352.0 3731.2 0.0 0.0 - - - - - - 0.000 >>> ``` >>> I uploaded webrev which is for jdk/jdk repo. Please tell me if it >>> should be for zgc repo. >>> I'm not an author of ZGC. So I need a sponsor. >>> (I'm a Reviewer of jdk. If this change should be pushed to jdk/jdk, >>> please tell me. I will push it after reviewing.) >>> Thanks, >>> Yasumasa > From yasuenag at gmail.com Thu Jul 26 13:52:10 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Thu, 26 Jul 2018 22:52:10 +0900 Subject: ZGC: RFR: 8207843: HSDB cannot show Object Histogram when ZGC is working In-Reply-To: <3d4f8faf-e26a-0e6b-6df1-73b6600ee5a0@gmail.com> References: <0e8ce2e6-e043-2320-e660-a2f1f4df820e@gmail.com> <3d4f8faf-e26a-0e6b-6df1-73b6600ee5a0@gmail.com> Message-ID: <06ceb864-bca5-d89c-c54e-fbfce3585066@gmail.com> CC'ing to hotspot-gc-dev On 2018/07/26 21:30, Yasumasa Suenaga wrote: > PING: Could you review it? > >> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207843/webrev.00/ > > > Yasumasa > > > On 2018/07/19 23:03, Yasumasa Suenaga wrote: >> Hi all, >> >> Please review this webrev. >> >> ????? JBS: https://bugs.openjdk.java.net/browse/JDK-8207843 >> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207843/webrev.00/ >> >> I encountered AssertionFailure when I attached HSDB to the process which is working with ZGC as below: >> >> sun.jvm.hotspot.utilities.AssertionFailure: Unexpected CollectedHeap type: sun.jvm.hotspot.gc.z.ZCollectedHeap >> ???? at jdk.hotspot.agent/sun.jvm.hotspot.utilities.Assert.that(Assert.java:32) >> ???? at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.collectLiveRegions(ObjectHeap.java:448) >> ???? at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.iterate(ObjectHeap.java:173) >> ???? at jdk.hotspot.agent/sun.jvm.hotspot.HSDB$VisitHeap.run(HSDB.java:1741) >> ???? at jdk.hotspot.agent/sun.jvm.hotspot.utilities.WorkerThread$MainLoop.run(WorkerThread.java:70) >> ???? at java.base/java.lang.Thread.run(Thread.java:832) >> >> ObjectHeap#collectLiveRegions() branches by instance type of CollectedHeap. However it does not support ZCollectedHeap. >> So I add ZCollectedHeap to it and add some methods to iterate ZPageTable. >> >> >> Thanks, >> >> Yasumasa From erik.osterlund at oracle.com Thu Jul 26 14:19:58 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 26 Jul 2018 16:19:58 +0200 Subject: PING: RFR: 8207756: jstat should show CGC STW phases of ZGC In-Reply-To: <51cc06a1-b027-4208-f2d9-327cbd009564@gmail.com> References: <11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com> <107AD8D6-1E1F-4469-9352-8CCD3C84AA67@oracle.com> <51cc06a1-b027-4208-f2d9-327cbd009564@gmail.com> Message-ID: <5B59D88E.3040708@oracle.com> Hi Yasumasa, There is only one global instance of CollectorCounters. Do we really have to pass it around into constructors and dump it in ZDriver? I don't see why it would be in any way problematic to just put this global instance of counters in ZServiceabilityCounters. And put the TraceCollectorStats tcs(_cgc_counters); stuff in ZServiceabilityCountersTracer or something like that. Also I saw this: _cgc_counters = new CollectorCounters("Z stop-the-world phases", 2); Does this assume there are two pauses for a ZGC collection cycle? There are at least 3, but possibly more: 1) Mark start 2) Mark end (this could recur multiple times due to unlucky races with mutators resurrecting weak references right before hitting mark end, causing a new concurrent phase if a large object graph was reachable from this weak ref). So we really can't know for sure how many pauses there will be. 3) Relocate start (done after reference processing and concurrent selection of relocaiton set) Thanks, /Erik On 2018-07-26 15:44, Yasumasa Suenaga wrote: > CC'ing to hotspot-gc-dev > > Hi Per, > > I've looked at ZServiceabilityCounters in zServiceability.cpp . > IMHO it is unsuitable because CGC counter means STW in concurrent GC. > So I think it is suitable to implement in VMOperation for ZGC. > > Thanks, > > Yasumasa > > > On 2018/07/26 22:30, Per Lid?n wrote: >> Hi, >> >> Please have a look at the zServiceability class, which is where code >> like this is intended to live. >> >> /Per >> >>> On 26 Jul 2018, at 14:30, Yasumasa Suenaga wrote: >>> >>> PING: Could you review it? >>> >>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/ >>> >>> >>> Yasumasa >>> >>> >>>> On 2018/07/18 11:59, Yasumasa Suenaga wrote: >>>> Hi all, >>>> Please review this change: >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8207756 >>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/ >>>> jstat shows CGC STW phases as CGC and CGCT fields since JDK-8153333 . >>>> However, ZGC does not adapt to it yet as following: >>>> ``` >>>> $ jstat -gc 1234 1 1 >>>> S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT CGC >>>> CGCT GCT >>>> - - - - - - 514048.0 12288.0 4352.0 3731.2 0.0 0.0 - - - - - - >>>> 0.000 >>>> ``` >>>> I uploaded webrev which is for jdk/jdk repo. Please tell me if it >>>> should be for zgc repo. >>>> I'm not an author of ZGC. So I need a sponsor. >>>> (I'm a Reviewer of jdk. If this change should be pushed to jdk/jdk, >>>> please tell me. I will push it after reviewing.) >>>> Thanks, >>>> Yasumasa >> From yasuenag at gmail.com Fri Jul 27 14:31:03 2018 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Fri, 27 Jul 2018 23:31:03 +0900 Subject: ZGC: RFR: 8207756: jstat should show CGC STW phases of ZGC In-Reply-To: <5B59D88E.3040708@oracle.com> References: <11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com> <107AD8D6-1E1F-4469-9352-8CCD3C84AA67@oracle.com> <51cc06a1-b027-4208-f2d9-327cbd009564@gmail.com> <5B59D88E.3040708@oracle.com> Message-ID: <5858d3d5-7eed-18ae-a34a-5fbefd0a90ed@gmail.com> Hi Erik, Per, Thank you for your comment. I uploaded new webrev. It has CGC counter in ZServiceability. http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.01/ I know ZGC has three STW phases. They are called by VM_ZOperation. So I had added CGC counter in it at first. However I agree with you if ZServiceability should have it. Yasumasa On 2018/07/26 23:19, Erik ?sterlund wrote: > Hi Yasumasa, > > There is only one global instance of CollectorCounters. Do we really have to pass it around into constructors and dump it in ZDriver? > I don't see why it would be in any way problematic to just put this global instance of counters in ZServiceabilityCounters. And put the TraceCollectorStats tcs(_cgc_counters); stuff in ZServiceabilityCountersTracer or something like that. > > Also I saw this: _cgc_counters = new CollectorCounters("Z stop-the-world phases", 2); > > Does this assume there are two pauses for a ZGC collection cycle? There are at least 3, but possibly more: > > 1) Mark start > 2) Mark end (this could recur multiple times due to unlucky races with mutators resurrecting weak references right before hitting mark end, causing a new concurrent phase if a large object graph was reachable from this weak ref). So we really can't know for sure how many pauses there will be. > 3) Relocate start (done after reference processing and concurrent selection of relocaiton set) > > Thanks, > /Erik > > On 2018-07-26 15:44, Yasumasa Suenaga wrote: >> CC'ing to hotspot-gc-dev >> >> Hi Per, >> >> I've looked at ZServiceabilityCounters in zServiceability.cpp . >> IMHO it is unsuitable because CGC counter means STW in concurrent GC. So I think it is suitable to implement in VMOperation for ZGC. >> >> Thanks, >> >> Yasumasa >> >> >> On 2018/07/26 22:30, Per Lid?n wrote: >>> Hi, >>> >>> Please have a look at the zServiceability class, which is where code like this is intended to live. >>> >>> /Per >>> >>>> On 26 Jul 2018, at 14:30, Yasumasa Suenaga wrote: >>>> >>>> PING: Could you review it? >>>> >>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/ >>>> >>>> >>>> Yasumasa >>>> >>>> >>>>> On 2018/07/18 11:59, Yasumasa Suenaga wrote: >>>>> Hi all, >>>>> Please review this change: >>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8207756 >>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/ >>>>> jstat shows CGC STW phases as CGC and CGCT fields since JDK-8153333 . >>>>> However, ZGC does not adapt to it yet as following: >>>>> ``` >>>>> $ jstat -gc 1234 1 1 >>>>> ? S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT CGC CGCT GCT >>>>> ?? - - - - - - 514048.0 12288.0 4352.0 3731.2 0.0 0.0 - - - - - - 0.000 >>>>> ``` >>>>> I uploaded webrev which is for jdk/jdk repo. Please tell me if it >>>>> should be for zgc repo. >>>>> I'm not an author of ZGC. So I need a sponsor. >>>>> (I'm a Reviewer of jdk. If this change should be pushed to jdk/jdk, >>>>> please tell me. I will push it after reviewing.) >>>>> Thanks, >>>>> Yasumasa >>> >