From mikael.vidstedt at oracle.com Tue Jul 10 19:59:55 2018
From: mikael.vidstedt at oracle.com (Mikael Vidstedt)
Date: Tue, 10 Jul 2018 12:59:55 -0700
Subject: RFR(XS): 8207007: Add missing license header to zHash.inline.hpp
Message-ID: <1DAD0F32-68FF-4F09-BF23-56CBFDC1F992@oracle.com>
Please review this small change which adds the missing license header to one of the ZGC files:
Bug: https://bugs.openjdk.java.net/browse/JDK-8207007
Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8207007/webrev.00/open/webrev/
Cheers,
Mikael
From thomas.schatzl at oracle.com Tue Jul 10 20:26:19 2018
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 10 Jul 2018 22:26:19 +0200
Subject: RFR(XS): 8207007: Add missing license header to zHash.inline.hpp
In-Reply-To: <1DAD0F32-68FF-4F09-BF23-56CBFDC1F992@oracle.com>
References: <1DAD0F32-68FF-4F09-BF23-56CBFDC1F992@oracle.com>
Message-ID: <58fb563e491cf8959023c518b115085c4e3ad4e0.camel@oracle.com>
Hi,
On Tue, 2018-07-10 at 12:59 -0700, Mikael Vidstedt wrote:
> Please review this small change which adds the missing license header
> to one of the ZGC files:
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8207007
> Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8207007/webrev.00/
> open/webrev/ ev.00/open/webrev/>
>
looks good to me.
Thomas
From per.liden at oracle.com Tue Jul 10 21:00:32 2018
From: per.liden at oracle.com (Per Liden)
Date: Tue, 10 Jul 2018 23:00:32 +0200
Subject: RFR(XS): 8207007: Add missing license header to zHash.inline.hpp
In-Reply-To: <1DAD0F32-68FF-4F09-BF23-56CBFDC1F992@oracle.com>
References: <1DAD0F32-68FF-4F09-BF23-56CBFDC1F992@oracle.com>
Message-ID: <17ad62d5-9998-755a-fab1-eafd0eea4c97@oracle.com>
Looks good
/Per
On 2018-07-10 21:59, Mikael Vidstedt wrote:
>
> Please review this small change which adds the missing license header to one of the ZGC files:
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8207007
> Webrev: http://cr.openjdk.java.net/~mikael/webrevs/8207007/webrev.00/open/webrev/
>
> Cheers,
> Mikael
>
From simone.bordet at gmail.com Fri Jul 13 17:35:12 2018
From: simone.bordet at gmail.com (Simone Bordet)
Date: Fri, 13 Jul 2018 19:35:12 +0200
Subject: Understanding ZGC details
Message-ID:
Hi,
first of all congrats and thanks for ZGC.
I've been running a benchmark of mine
(https://docs.cometd.org/current/reference/#_benchmarking), a
Servlet-based chat application on top of Jetty, to compare G1,
Shenandoah and ZGC.
The hope is to produce some conference material to spread the word
about ZGC (for which personally I have high expectations).
Meanwhile I'm trying to understand a bit more about ZGC and I have few
questions (I have read the GC handbook from Jones et al., but I'm new
to Linux memory tricks so please pardon my ignorance on the matter; I
have also watched Per's presentation at JFokus 2018).
I don't quite get the mechanism of colored pointers and how it exactly works.
I see that ZGC maps the heap space into 3 views (in
ZPhysicalMemoryBacking::map()).
What is the purpose of mapping into 3 views (I get the number 3, it's
for the number of phases; but why cannot they be handled with just 1
view)?
Skimming around in the code - not deeply, I admit - seems to be that
pointers to those views are always de-colored before being
dereferenced, so I am a bit puzzled about those 3 views.
Assuming I'm wrong, is the mechanism similar to C4, where a view is
mprotect()-ed on a phase flip?
If so, I am also assuming that this would generate a SEGV that will
run some code that will fix the pointer. Can you please detail where
is the SEGV handling code that does that?
I'm stopping here feeling that I'm offroad already :)
I'm sure if you put me back on the road I'll have more questions.
Thanks!
--
Simone Bordet
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
From gil at azul.com Fri Jul 13 17:52:45 2018
From: gil at azul.com (Gil Tene)
Date: Fri, 13 Jul 2018 17:52:45 +0000
Subject: Understanding ZGC details
In-Reply-To:
References:
Message-ID:
> On Jul 13, 2018, at 10:35 AM, Simone Bordet wrote:
>
> Hi,
>
> first of all congrats and thanks for ZGC.
>
> I've been running a benchmark of mine
> (https://docs.cometd.org/current/reference/#_benchmarking), a
> Servlet-based chat application on top of Jetty, to compare G1,
> Shenandoah and ZGC.
> The hope is to produce some conference material to spread the word
> about ZGC (for which personally I have high expectations).
>
> Meanwhile I'm trying to understand a bit more about ZGC and I have few
> questions (I have read the GC handbook from Jones et al., but I'm new
> to Linux memory tricks so please pardon my ignorance on the matter; I
> have also watched Per's presentation at JFokus 2018).
>
> I don't quite get the mechanism of colored pointers and how it exactly works.
> I see that ZGC maps the heap space into 3 views (in
> ZPhysicalMemoryBacking::map()).
> What is the purpose of mapping into 3 views (I get the number 3, it's
> for the number of phases; but why cannot they be handled with just 1
> view)?
>
> Skimming around in the code - not deeply, I admit - seems to be that
> pointers to those views are always de-colored before being
> dereferenced, so I am a bit puzzled about those 3 views.
> Assuming I'm wrong, is the mechanism similar to C4, where a view is
> mprotect()-ed on a phase flip?
> If so, I am also assuming that this would generate a SEGV that will
> run some code that will fix the pointer. Can you please detail where
> is the SEGV handling code that does that?
Speaking about the C4 mechanism: while C4 does perform various mapping manipulations
for various reasons, LVB detection does not use SEGVs and does not rely on memory
protection. Instead, The C4 LVB fast path uses a test & jmp (a single u-op on x86) to verify
that the loaded reference is not in a "currently invalid" phase, with a slow path that fixes the
reference to be in a valid phase (doing whatever is needed to achieve that: ensure marking,
fixup to point to relocated object location, or perform actual object relocation if needed).
My understanding is the the ZGC barrier mechanism is doing pretty much the same thing
including the use of a test & jmp for the fast path. It is basically a C4 LVB.
BTW, adding Zing to the set of JVM configs you comparatively test above would be a good idea ;-)
>
> I'm stopping here feeling that I'm offroad already :)
>
> I'm sure if you put me back on the road I'll have more questions.
>
> Thanks!
>
> --
> Simone Bordet
> ---
> Finally, no matter how good the architecture and design are,
> to deliver bug-free software with optimal performance and reliability,
> the implementation technique must be flawless. Victoria Livschitz
From per.liden at oracle.com Sat Jul 14 14:08:29 2018
From: per.liden at oracle.com (Per Liden)
Date: Sat, 14 Jul 2018 16:08:29 +0200
Subject: Understanding ZGC details
In-Reply-To:
References:
Message-ID: <2e31a5dc-2598-d671-13b2-415aa2a5ed1f@oracle.com>
Hi Simone,
On 2018-07-13 19:35, Simone Bordet wrote:
> Hi,
>
> first of all congrats and thanks for ZGC.
Thanks!
>
> I've been running a benchmark of mine
> (https://docs.cometd.org/current/reference/#_benchmarking), a
> Servlet-based chat application on top of Jetty, to compare G1,
> Shenandoah and ZGC.
> The hope is to produce some conference material to spread the word
> about ZGC (for which personally I have high expectations).
Cool!
>
> Meanwhile I'm trying to understand a bit more about ZGC and I have few
> questions (I have read the GC handbook from Jones et al., but I'm new
> to Linux memory tricks so please pardon my ignorance on the matter; I
> have also watched Per's presentation at JFokus 2018).
>
> I don't quite get the mechanism of colored pointers and how it exactly works.
> I see that ZGC maps the heap space into 3 views (in
> ZPhysicalMemoryBacking::map()).
> What is the purpose of mapping into 3 views (I get the number 3, it's
> for the number of phases; but why cannot they be handled with just 1
> view)?
>
> Skimming around in the code - not deeply, I admit - seems to be that
> pointers to those views are always de-colored before being
> dereferenced, so I am a bit puzzled about those 3 views.
Actually, pointers are not de-colored, which is the reason why we need
the three heap views on x86 (where we also don't have HW support for
VA-masking/de-coloring). Slide 33 in the JFokus slide pack tries to
illustrate this.
(Internally in ZGC we sometimes remove the colors and the heap base, and
we call the result of that an "offset". These "offsets" are however a
pure internal thing and these are never exposed to or used by Java threads).
> Assuming I'm wrong, is the mechanism similar to C4, where a view is
> mprotect()-ed on a phase flip?
> If so, I am also assuming that this would generate a SEGV that will
> run some code that will fix the pointer. Can you please detail where
> is the SEGV handling code that does that?
I think Gil might have answered this already, but just to reiterate, ZGC
does not use SEGV to catch bad pointers. The load barrier (the
test-and-jump instruction sequence) will catch all pointers with "bad"
colors before they are used.
cheers,
Per
(I'm on vacation at the moment, so my e-mail responsiveness might not be
super good during the next two weeks)
>
> I'm stopping here feeling that I'm offroad already :)
>
> I'm sure if you put me back on the road I'll have more questions.
>
> Thanks!
>
From simone.bordet at gmail.com Mon Jul 16 14:08:06 2018
From: simone.bordet at gmail.com (Simone Bordet)
Date: Mon, 16 Jul 2018 16:08:06 +0200
Subject: Understanding ZGC details
In-Reply-To:
References:
Message-ID:
Hi Gil,
keeping this between us since it's not really relevant for ZGC :)
On Fri, Jul 13, 2018 at 7:52 PM Gil Tene wrote:
> Speaking about the C4 mechanism: while C4 does perform various mapping manipulations
> for various reasons, LVB detection does not use SEGVs and does not rely on memory
> protection. Instead, The C4 LVB fast path uses a test & jmp (a single u-op on x86) to verify
> that the loaded reference is not in a "currently invalid" phase, with a slow path that fixes the
> reference to be in a valid phase (doing whatever is needed to achieve that: ensure marking,
> fixup to point to relocated object location, or perform actual object relocation if needed).
Ok. So I'm reading the GC Handbook from Jones et al. 2nd edition, and
they say that Pauless and C4 protects from-pages.
If I read correctly, from-pages are protected so that if a mutator
dereferences a pointer, it will trap, and the trap will "fix" the
pointer, either by copying it to to-space (so the mutator does the
job), or by looking at side metadata for the new address.
I understand that if there is a JIT-injected load barrier always
present, this work of "fixing" the pointer can be done by the slow
path of the load barrier, but then (for what pertains pointers)
protecting pages is not necessary; but then, why the GC handbook
mentions them?
Thanks!
--
Simone Bordet
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
From gil at azul.com Mon Jul 16 16:07:44 2018
From: gil at azul.com (Gil Tene)
Date: Mon, 16 Jul 2018 16:07:44 +0000
Subject: Understanding ZGC details
In-Reply-To:
References:
,
Message-ID: <591A92E9-812C-44FC-9B19-E677CB857E2F@azul.com>
Sent from my iPad
On Jul 16, 2018, at 7:08 AM, Simone Bordet > wrote:
Hi Gil,
keeping this between us since it's not really relevant for ZGC :)
On Fri, Jul 13, 2018 at 7:52 PM Gil Tene > wrote:
Speaking about the C4 mechanism: while C4 does perform various mapping manipulations
for various reasons, LVB detection does not use SEGVs and does not rely on memory
protection. Instead, The C4 LVB fast path uses a test & jmp (a single u-op on x86) to verify
that the loaded reference is not in a "currently invalid" phase, with a slow path that fixes the
reference to be in a valid phase (doing whatever is needed to achieve that: ensure marking,
fixup to point to relocated object location, or perform actual object relocation if needed).
Ok. So I'm reading the GC Handbook from Jones et al. 2nd edition, and
they say that Pauless and C4 protects from-pages.
If I read correctly, from-pages are protected so that if a mutator
dereferences a pointer, it will trap, and the trap will "fix" the
pointer, either by copying it to to-space (so the mutator does the
job), or by looking at side metadata for the new address.
Look at the C4 ISMM paper (https://www.azul.com/files/c4_paper_acm2.pdf) for the actual terminology and use. The paper defines (in section 2) what an LVB is, the invariants it maintains, and explains the various ways to ensure barriered references match an expected state. It also discusses implementations variants (section 4), where it explains how LVB fast path test implementations could vary dramatically (e.g. between interpreted, JIT?ed, runtime C++ codes, or on different hardware architectures, and obviously by specific implementation choices) without changing semantics, and provides pseudo code (appendix A) for the logical coverage.
The TLDR summary in the context of your above question would be: ?Protected? means ?logically protected (by the algorithm) from mutator access?. ?Trap? or ?trigger? means ?do something other than the fast path?., and ?Page? means ?a power-of-2 aligned range of addresses?. Virtual memory protection and privilege-changing trapping is one possible (but not very efficient or practical for the common case) way to implement an LVB, but the vast majority of LVB tests are implemented via user-mode testing and conditional branching.
Adhering to actual virtual memory page boundaries, and actually matching collector protection with virtual memory protection has some bonus logical benefits, including allowing multiple implementations of the same LVB semantics enforcement in the same code base, as well as enhanced error detection that goes beyond the LVB semantics requirements (but helps a lot in stabilizing actual collector implementation, which like all software, may include bugs at some point).
The fast path of an LVB test can be optimized to a test-and-jump, and that?s how it?s been implemented for roughly a decade for reasonable (currently up to 2TB) heap sizes. The size of the mask being tested and the amount of pre-shifting involved (if any) determines the possible granularity of protected page set selection for a relocation phase flip. E.g. selecting the entire (up to 2TB on x86) heap or an entire generation at the same time allows for no shifting at all (mask in place), test-and-jump. Selecting sub-phases (e.g. relocating 1/8th or 1/16th of the heap at a time) can fit with 64 bit register masks using an extra shift, higher granularity can be achieved with vector registers, and arbitrary granularity (down to a single page if wished for) could be achieved by testing against a memory (rather than a register) bitmask.
I understand that if there is a JIT-injected load barrier always
present, this work of "fixing" the pointer can be done by the slow
path of the load barrier, but then (for what pertains pointers)
protecting pages is not necessary; but then, why the GC handbook
mentions them?
I believe that the current edition of the handbook was finalized before they had access to the published C4 ISM! paper... The fact that some C4 algorithm implementations (like the ones we actually use in our products) *also* use virtual page mapping and ritual page protection for unrelated (as in not algorithmically necessary but otherwise useful) purposes, like error detection, additional optimization opportunities, and code stabilization, may have confused the C4 algorithm description with some implementation concerns.
Thanks!
--
Simone Bordet
---
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless. Victoria Livschitz
From brett.wooldridge at gmail.com Tue Jul 17 19:02:56 2018
From: brett.wooldridge at gmail.com (Brett Wooldridge)
Date: Wed, 18 Jul 2018 04:02:56 +0900
Subject: Class Unloading
Message-ID:
I have two questions:
1/ I understand from the slides that class unloading is "not yet". But
does this mean that class unloading is simply not present, or that it *is
present* but not parallel?
2/ I seem recall reading somewhere that ZGC does not play nice with
GraalVM. Is that true and/or still true?
Thanks for the awesome work on ZGC!
-Brett
p.s. a case of your choice of beer to whoever gets ZGC into the official
JDK builds for MacOS X.
From per.liden at oracle.com Tue Jul 17 19:44:22 2018
From: per.liden at oracle.com (Per Liden)
Date: Tue, 17 Jul 2018 21:44:22 +0200
Subject: Class Unloading
In-Reply-To:
References:
Message-ID: <1d29b27d-d40d-58a2-c491-4a683fb8dd28@oracle.com>
Hi Brett,
On 2018-07-17 21:02, Brett Wooldridge wrote:
> I have two questions:
>
> 1/ I understand from the slides that class unloading is "not yet". But
> does this mean that class unloading is simply not present, or that it *is
> present* but not parallel?
It means it's not present at all. But work in ongoing and it will most
likely be there in JDK 12.
>
> 2/ I seem recall reading somewhere that ZGC does not play nice with
> GraalVM. Is that true and/or still true?
Yes, the Graal JIT does not yet know how to generate load barriers for
ZGC. But it's on our road map.
>
> Thanks for the awesome work on ZGC!
Thanks!
/Per
>
> -Brett
>
> p.s. a case of your choice of beer to whoever gets ZGC into the official
> JDK builds for MacOS X.
>
From coleen.phillimore at oracle.com Tue Jul 17 19:57:24 2018
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 17 Jul 2018 15:57:24 -0400
Subject: Class Unloading
In-Reply-To:
References:
Message-ID: <0fd2cbad-c3fa-ff39-b624-4bfd24c4ebe4@oracle.com>
On 7/17/18 3:02 PM, Brett Wooldridge wrote:
> I have two questions:
>
> 1/ I understand from the slides that class unloading is "not yet". But
> does this mean that class unloading is simply not present, or that it *is
> present* but not parallel?
It's not presently done.? We're currently working on stop the world
class unloading first and then concurrent class unloading next.
I'll leave it to the others to answer the second question (I believe
that the load barriers are missing in graal so ZGC won't work with
graal), and to claim the beer.
thanks,
Coleen
>
> 2/ I seem recall reading somewhere that ZGC does not play nice with
> GraalVM. Is that true and/or still true?
>
> Thanks for the awesome work on ZGC!
>
> -Brett
>
> p.s. a case of your choice of beer to whoever gets ZGC into the official
> JDK builds for MacOS X.
From yasuenag at gmail.com Wed Jul 18 02:59:35 2018
From: yasuenag at gmail.com (Yasumasa Suenaga)
Date: Wed, 18 Jul 2018 11:59:35 +0900
Subject: RFR: 8207756: jstat should show CGC STW phases of ZGC
Message-ID:
Hi all,
Please review this change:
JBS: https://bugs.openjdk.java.net/browse/JDK-8207756
webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/
jstat shows CGC STW phases as CGC and CGCT fields since JDK-8153333 .
However, ZGC does not adapt to it yet as following:
```
$ jstat -gc 1234 1 1
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT CGC CGCT GCT
- - - - - - 514048.0 12288.0 4352.0 3731.2 0.0 0.0 - - - - - - 0.000
```
I uploaded webrev which is for jdk/jdk repo. Please tell me if it
should be for zgc repo.
I'm not an author of ZGC. So I need a sponsor.
(I'm a Reviewer of jdk. If this change should be pushed to jdk/jdk,
please tell me. I will push it after reviewing.)
Thanks,
Yasumasa
From yasuenag at gmail.com Thu Jul 19 14:03:24 2018
From: yasuenag at gmail.com (Yasumasa Suenaga)
Date: Thu, 19 Jul 2018 23:03:24 +0900
Subject: RFR: 8207843: HSDB cannot show Object Histogram when ZGC is working
Message-ID: <0e8ce2e6-e043-2320-e660-a2f1f4df820e@gmail.com>
Hi all,
Please review this webrev.
JBS: https://bugs.openjdk.java.net/browse/JDK-8207843
webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207843/webrev.00/
I encountered AssertionFailure when I attached HSDB to the process which is working with ZGC as below:
sun.jvm.hotspot.utilities.AssertionFailure: Unexpected CollectedHeap type: sun.jvm.hotspot.gc.z.ZCollectedHeap
at jdk.hotspot.agent/sun.jvm.hotspot.utilities.Assert.that(Assert.java:32)
at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.collectLiveRegions(ObjectHeap.java:448)
at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.iterate(ObjectHeap.java:173)
at jdk.hotspot.agent/sun.jvm.hotspot.HSDB$VisitHeap.run(HSDB.java:1741)
at jdk.hotspot.agent/sun.jvm.hotspot.utilities.WorkerThread$MainLoop.run(WorkerThread.java:70)
at java.base/java.lang.Thread.run(Thread.java:832)
ObjectHeap#collectLiveRegions() branches by instance type of CollectedHeap. However it does not support ZCollectedHeap.
So I add ZCollectedHeap to it and add some methods to iterate ZPageTable.
Thanks,
Yasumasa
From yasuenag at gmail.com Thu Jul 26 12:30:15 2018
From: yasuenag at gmail.com (Yasumasa Suenaga)
Date: Thu, 26 Jul 2018 21:30:15 +0900
Subject: PING: RFR: 8207756: jstat should show CGC STW phases of ZGC
In-Reply-To:
References:
Message-ID: <11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com>
PING: Could you review it?
> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/
Yasumasa
On 2018/07/18 11:59, Yasumasa Suenaga wrote:
> Hi all,
>
> Please review this change:
>
> JBS: https://bugs.openjdk.java.net/browse/JDK-8207756
> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/
>
> jstat shows CGC STW phases as CGC and CGCT fields since JDK-8153333 .
> However, ZGC does not adapt to it yet as following:
>
> ```
> $ jstat -gc 1234 1 1
> S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT CGC CGCT GCT
> - - - - - - 514048.0 12288.0 4352.0 3731.2 0.0 0.0 - - - - - - 0.000
> ```
>
> I uploaded webrev which is for jdk/jdk repo. Please tell me if it
> should be for zgc repo.
> I'm not an author of ZGC. So I need a sponsor.
> (I'm a Reviewer of jdk. If this change should be pushed to jdk/jdk,
> please tell me. I will push it after reviewing.)
>
>
> Thanks,
>
> Yasumasa
>
From yasuenag at gmail.com Thu Jul 26 12:30:44 2018
From: yasuenag at gmail.com (Yasumasa Suenaga)
Date: Thu, 26 Jul 2018 21:30:44 +0900
Subject: PING: RFR: 8207843: HSDB cannot show Object Histogram when ZGC is
working
In-Reply-To: <0e8ce2e6-e043-2320-e660-a2f1f4df820e@gmail.com>
References: <0e8ce2e6-e043-2320-e660-a2f1f4df820e@gmail.com>
Message-ID: <3d4f8faf-e26a-0e6b-6df1-73b6600ee5a0@gmail.com>
PING: Could you review it?
> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207843/webrev.00/
Yasumasa
On 2018/07/19 23:03, Yasumasa Suenaga wrote:
> Hi all,
>
> Please review this webrev.
>
> ???? JBS: https://bugs.openjdk.java.net/browse/JDK-8207843
> ? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207843/webrev.00/
>
> I encountered AssertionFailure when I attached HSDB to the process which is working with ZGC as below:
>
> sun.jvm.hotspot.utilities.AssertionFailure: Unexpected CollectedHeap type: sun.jvm.hotspot.gc.z.ZCollectedHeap
> ??? at jdk.hotspot.agent/sun.jvm.hotspot.utilities.Assert.that(Assert.java:32)
> ??? at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.collectLiveRegions(ObjectHeap.java:448)
> ??? at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.iterate(ObjectHeap.java:173)
> ??? at jdk.hotspot.agent/sun.jvm.hotspot.HSDB$VisitHeap.run(HSDB.java:1741)
> ??? at jdk.hotspot.agent/sun.jvm.hotspot.utilities.WorkerThread$MainLoop.run(WorkerThread.java:70)
> ??? at java.base/java.lang.Thread.run(Thread.java:832)
>
> ObjectHeap#collectLiveRegions() branches by instance type of CollectedHeap. However it does not support ZCollectedHeap.
> So I add ZCollectedHeap to it and add some methods to iterate ZPageTable.
>
>
> Thanks,
>
> Yasumasa
From erik.osterlund at oracle.com Thu Jul 26 12:54:01 2018
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Thu, 26 Jul 2018 14:54:01 +0200
Subject: PING: RFR: 8207756: jstat should show CGC STW phases of ZGC
In-Reply-To: <11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com>
References:
<11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com>
Message-ID: <5B59C469.4060909@oracle.com>
Hi Yasumasa,
After the integration to jdk-jdk, we want smaller things like this to go
in upstream instead. So I think this should go into the hotspot-gc list
and be pushed upstream. You can put a ZGC: prefix to the email.
Thanks,
/Erik
On 2018-07-26 14:30, Yasumasa Suenaga wrote:
> PING: Could you review it?
>
>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/
>
>
> Yasumasa
>
>
> On 2018/07/18 11:59, Yasumasa Suenaga wrote:
>> Hi all,
>>
>> Please review this change:
>>
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8207756
>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/
>>
>> jstat shows CGC STW phases as CGC and CGCT fields since JDK-8153333 .
>> However, ZGC does not adapt to it yet as following:
>>
>> ```
>> $ jstat -gc 1234 1 1
>> S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT CGC
>> CGCT GCT
>> - - - - - - 514048.0 12288.0 4352.0 3731.2 0.0 0.0 - - - - - - 0.000
>> ```
>>
>> I uploaded webrev which is for jdk/jdk repo. Please tell me if it
>> should be for zgc repo.
>> I'm not an author of ZGC. So I need a sponsor.
>> (I'm a Reviewer of jdk. If this change should be pushed to jdk/jdk,
>> please tell me. I will push it after reviewing.)
>>
>>
>> Thanks,
>>
>> Yasumasa
>>
From per.liden at oracle.com Thu Jul 26 13:30:14 2018
From: per.liden at oracle.com (=?utf-8?Q?Per_Lid=C3=A9n?=)
Date: Thu, 26 Jul 2018 15:30:14 +0200
Subject: PING: RFR: 8207756: jstat should show CGC STW phases of ZGC
In-Reply-To: <11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com>
References:
<11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com>
Message-ID: <107AD8D6-1E1F-4469-9352-8CCD3C84AA67@oracle.com>
Hi,
Please have a look at the zServiceability class, which is where code like this is intended to live.
/Per
> On 26 Jul 2018, at 14:30, Yasumasa Suenaga wrote:
>
> PING: Could you review it?
>
>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/
>
>
> Yasumasa
>
>
>> On 2018/07/18 11:59, Yasumasa Suenaga wrote:
>> Hi all,
>> Please review this change:
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8207756
>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/
>> jstat shows CGC STW phases as CGC and CGCT fields since JDK-8153333 .
>> However, ZGC does not adapt to it yet as following:
>> ```
>> $ jstat -gc 1234 1 1
>> S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT CGC CGCT GCT
>> - - - - - - 514048.0 12288.0 4352.0 3731.2 0.0 0.0 - - - - - - 0.000
>> ```
>> I uploaded webrev which is for jdk/jdk repo. Please tell me if it
>> should be for zgc repo.
>> I'm not an author of ZGC. So I need a sponsor.
>> (I'm a Reviewer of jdk. If this change should be pushed to jdk/jdk,
>> please tell me. I will push it after reviewing.)
>> Thanks,
>> Yasumasa
From yasuenag at gmail.com Thu Jul 26 13:44:30 2018
From: yasuenag at gmail.com (Yasumasa Suenaga)
Date: Thu, 26 Jul 2018 22:44:30 +0900
Subject: PING: RFR: 8207756: jstat should show CGC STW phases of ZGC
In-Reply-To: <107AD8D6-1E1F-4469-9352-8CCD3C84AA67@oracle.com>
References:
<11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com>
<107AD8D6-1E1F-4469-9352-8CCD3C84AA67@oracle.com>
Message-ID: <51cc06a1-b027-4208-f2d9-327cbd009564@gmail.com>
CC'ing to hotspot-gc-dev
Hi Per,
I've looked at ZServiceabilityCounters in zServiceability.cpp .
IMHO it is unsuitable because CGC counter means STW in concurrent GC. So I think it is suitable to implement in VMOperation for ZGC.
Thanks,
Yasumasa
On 2018/07/26 22:30, Per Lid?n wrote:
> Hi,
>
> Please have a look at the zServiceability class, which is where code like this is intended to live.
>
> /Per
>
>> On 26 Jul 2018, at 14:30, Yasumasa Suenaga wrote:
>>
>> PING: Could you review it?
>>
>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/
>>
>>
>> Yasumasa
>>
>>
>>> On 2018/07/18 11:59, Yasumasa Suenaga wrote:
>>> Hi all,
>>> Please review this change:
>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8207756
>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/
>>> jstat shows CGC STW phases as CGC and CGCT fields since JDK-8153333 .
>>> However, ZGC does not adapt to it yet as following:
>>> ```
>>> $ jstat -gc 1234 1 1
>>> S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT CGC CGCT GCT
>>> - - - - - - 514048.0 12288.0 4352.0 3731.2 0.0 0.0 - - - - - - 0.000
>>> ```
>>> I uploaded webrev which is for jdk/jdk repo. Please tell me if it
>>> should be for zgc repo.
>>> I'm not an author of ZGC. So I need a sponsor.
>>> (I'm a Reviewer of jdk. If this change should be pushed to jdk/jdk,
>>> please tell me. I will push it after reviewing.)
>>> Thanks,
>>> Yasumasa
>
From yasuenag at gmail.com Thu Jul 26 13:52:10 2018
From: yasuenag at gmail.com (Yasumasa Suenaga)
Date: Thu, 26 Jul 2018 22:52:10 +0900
Subject: ZGC: RFR: 8207843: HSDB cannot show Object Histogram when ZGC is
working
In-Reply-To: <3d4f8faf-e26a-0e6b-6df1-73b6600ee5a0@gmail.com>
References: <0e8ce2e6-e043-2320-e660-a2f1f4df820e@gmail.com>
<3d4f8faf-e26a-0e6b-6df1-73b6600ee5a0@gmail.com>
Message-ID: <06ceb864-bca5-d89c-c54e-fbfce3585066@gmail.com>
CC'ing to hotspot-gc-dev
On 2018/07/26 21:30, Yasumasa Suenaga wrote:
> PING: Could you review it?
>
>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207843/webrev.00/
>
>
> Yasumasa
>
>
> On 2018/07/19 23:03, Yasumasa Suenaga wrote:
>> Hi all,
>>
>> Please review this webrev.
>>
>> ????? JBS: https://bugs.openjdk.java.net/browse/JDK-8207843
>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207843/webrev.00/
>>
>> I encountered AssertionFailure when I attached HSDB to the process which is working with ZGC as below:
>>
>> sun.jvm.hotspot.utilities.AssertionFailure: Unexpected CollectedHeap type: sun.jvm.hotspot.gc.z.ZCollectedHeap
>> ???? at jdk.hotspot.agent/sun.jvm.hotspot.utilities.Assert.that(Assert.java:32)
>> ???? at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.collectLiveRegions(ObjectHeap.java:448)
>> ???? at jdk.hotspot.agent/sun.jvm.hotspot.oops.ObjectHeap.iterate(ObjectHeap.java:173)
>> ???? at jdk.hotspot.agent/sun.jvm.hotspot.HSDB$VisitHeap.run(HSDB.java:1741)
>> ???? at jdk.hotspot.agent/sun.jvm.hotspot.utilities.WorkerThread$MainLoop.run(WorkerThread.java:70)
>> ???? at java.base/java.lang.Thread.run(Thread.java:832)
>>
>> ObjectHeap#collectLiveRegions() branches by instance type of CollectedHeap. However it does not support ZCollectedHeap.
>> So I add ZCollectedHeap to it and add some methods to iterate ZPageTable.
>>
>>
>> Thanks,
>>
>> Yasumasa
From erik.osterlund at oracle.com Thu Jul 26 14:19:58 2018
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Thu, 26 Jul 2018 16:19:58 +0200
Subject: PING: RFR: 8207756: jstat should show CGC STW phases of ZGC
In-Reply-To: <51cc06a1-b027-4208-f2d9-327cbd009564@gmail.com>
References:
<11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com>
<107AD8D6-1E1F-4469-9352-8CCD3C84AA67@oracle.com>
<51cc06a1-b027-4208-f2d9-327cbd009564@gmail.com>
Message-ID: <5B59D88E.3040708@oracle.com>
Hi Yasumasa,
There is only one global instance of CollectorCounters. Do we really
have to pass it around into constructors and dump it in ZDriver?
I don't see why it would be in any way problematic to just put this
global instance of counters in ZServiceabilityCounters. And put the
TraceCollectorStats tcs(_cgc_counters); stuff in
ZServiceabilityCountersTracer or something like that.
Also I saw this: _cgc_counters = new CollectorCounters("Z stop-the-world
phases", 2);
Does this assume there are two pauses for a ZGC collection cycle? There
are at least 3, but possibly more:
1) Mark start
2) Mark end (this could recur multiple times due to unlucky races with
mutators resurrecting weak references right before hitting mark end,
causing a new concurrent phase if a large object graph was reachable
from this weak ref). So we really can't know for sure how many pauses
there will be.
3) Relocate start (done after reference processing and concurrent
selection of relocaiton set)
Thanks,
/Erik
On 2018-07-26 15:44, Yasumasa Suenaga wrote:
> CC'ing to hotspot-gc-dev
>
> Hi Per,
>
> I've looked at ZServiceabilityCounters in zServiceability.cpp .
> IMHO it is unsuitable because CGC counter means STW in concurrent GC.
> So I think it is suitable to implement in VMOperation for ZGC.
>
> Thanks,
>
> Yasumasa
>
>
> On 2018/07/26 22:30, Per Lid?n wrote:
>> Hi,
>>
>> Please have a look at the zServiceability class, which is where code
>> like this is intended to live.
>>
>> /Per
>>
>>> On 26 Jul 2018, at 14:30, Yasumasa Suenaga wrote:
>>>
>>> PING: Could you review it?
>>>
>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/
>>>
>>>
>>> Yasumasa
>>>
>>>
>>>> On 2018/07/18 11:59, Yasumasa Suenaga wrote:
>>>> Hi all,
>>>> Please review this change:
>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8207756
>>>> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/
>>>> jstat shows CGC STW phases as CGC and CGCT fields since JDK-8153333 .
>>>> However, ZGC does not adapt to it yet as following:
>>>> ```
>>>> $ jstat -gc 1234 1 1
>>>> S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT CGC
>>>> CGCT GCT
>>>> - - - - - - 514048.0 12288.0 4352.0 3731.2 0.0 0.0 - - - - - -
>>>> 0.000
>>>> ```
>>>> I uploaded webrev which is for jdk/jdk repo. Please tell me if it
>>>> should be for zgc repo.
>>>> I'm not an author of ZGC. So I need a sponsor.
>>>> (I'm a Reviewer of jdk. If this change should be pushed to jdk/jdk,
>>>> please tell me. I will push it after reviewing.)
>>>> Thanks,
>>>> Yasumasa
>>
From yasuenag at gmail.com Fri Jul 27 14:31:03 2018
From: yasuenag at gmail.com (Yasumasa Suenaga)
Date: Fri, 27 Jul 2018 23:31:03 +0900
Subject: ZGC: RFR: 8207756: jstat should show CGC STW phases of ZGC
In-Reply-To: <5B59D88E.3040708@oracle.com>
References:
<11866bdb-9e9d-019c-c64e-8548a79a7617@gmail.com>
<107AD8D6-1E1F-4469-9352-8CCD3C84AA67@oracle.com>
<51cc06a1-b027-4208-f2d9-327cbd009564@gmail.com>
<5B59D88E.3040708@oracle.com>
Message-ID: <5858d3d5-7eed-18ae-a34a-5fbefd0a90ed@gmail.com>
Hi Erik, Per,
Thank you for your comment.
I uploaded new webrev. It has CGC counter in ZServiceability.
http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.01/
I know ZGC has three STW phases. They are called by VM_ZOperation. So I had added CGC counter in it at first. However I agree with you if ZServiceability should have it.
Yasumasa
On 2018/07/26 23:19, Erik ?sterlund wrote:
> Hi Yasumasa,
>
> There is only one global instance of CollectorCounters. Do we really have to pass it around into constructors and dump it in ZDriver?
> I don't see why it would be in any way problematic to just put this global instance of counters in ZServiceabilityCounters. And put the TraceCollectorStats tcs(_cgc_counters); stuff in ZServiceabilityCountersTracer or something like that.
>
> Also I saw this: _cgc_counters = new CollectorCounters("Z stop-the-world phases", 2);
>
> Does this assume there are two pauses for a ZGC collection cycle? There are at least 3, but possibly more:
>
> 1) Mark start
> 2) Mark end (this could recur multiple times due to unlucky races with mutators resurrecting weak references right before hitting mark end, causing a new concurrent phase if a large object graph was reachable from this weak ref). So we really can't know for sure how many pauses there will be.
> 3) Relocate start (done after reference processing and concurrent selection of relocaiton set)
>
> Thanks,
> /Erik
>
> On 2018-07-26 15:44, Yasumasa Suenaga wrote:
>> CC'ing to hotspot-gc-dev
>>
>> Hi Per,
>>
>> I've looked at ZServiceabilityCounters in zServiceability.cpp .
>> IMHO it is unsuitable because CGC counter means STW in concurrent GC. So I think it is suitable to implement in VMOperation for ZGC.
>>
>> Thanks,
>>
>> Yasumasa
>>
>>
>> On 2018/07/26 22:30, Per Lid?n wrote:
>>> Hi,
>>>
>>> Please have a look at the zServiceability class, which is where code like this is intended to live.
>>>
>>> /Per
>>>
>>>> On 26 Jul 2018, at 14:30, Yasumasa Suenaga wrote:
>>>>
>>>> PING: Could you review it?
>>>>
>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/
>>>>
>>>>
>>>> Yasumasa
>>>>
>>>>
>>>>> On 2018/07/18 11:59, Yasumasa Suenaga wrote:
>>>>> Hi all,
>>>>> Please review this change:
>>>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8207756
>>>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8207756/webrev.00/
>>>>> jstat shows CGC STW phases as CGC and CGCT fields since JDK-8153333 .
>>>>> However, ZGC does not adapt to it yet as following:
>>>>> ```
>>>>> $ jstat -gc 1234 1 1
>>>>> ? S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT CGC CGCT GCT
>>>>> ?? - - - - - - 514048.0 12288.0 4352.0 3731.2 0.0 0.0 - - - - - - 0.000
>>>>> ```
>>>>> I uploaded webrev which is for jdk/jdk repo. Please tell me if it
>>>>> should be for zgc repo.
>>>>> I'm not an author of ZGC. So I need a sponsor.
>>>>> (I'm a Reviewer of jdk. If this change should be pushed to jdk/jdk,
>>>>> please tell me. I will push it after reviewing.)
>>>>> Thanks,
>>>>> Yasumasa
>>>
>