From hohensee at amazon.com Sun Sep 1 00:05:24 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Sun, 1 Sep 2019 00:05:24 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> Message-ID: <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> Thanks, Mandy. I?ve finalized the CSR. New webrev at http://cr.openjdk.java.net/~phh/8207266/webrev.04/. In management.cpp, I now have if (THREAD->is_Java_thread()) { return ((JavaThread*)THREAD)->cooked_allocated_bytes(); } In ThreadImpl.java, using requireNonNull would produce a different and less informative message, so I?d like to leave it as is. I changed throwIfNullThreadIds to ensureNonNullThreadIds, and throwIfThreadAllocatedMemoryNotSupported to ensureThreadAllocatedMemorySupported. I dropped the ?java.lang.? prefix from all uses of UnsupportedOperationException in both c.s.m.ThreadMXBean.java and j.l.m.ThreadMXBean.java, and did the same with SecurityException. ?@since 14? added to c.s.m.ThreadMXBean.java and the CSR. Do I need another reviewer? Paul From: Mandy Chung Date: Friday, August 30, 2019 at 4:26 PM To: "Hohensee, Paul" Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread CSR reviewed. management.cpp 2083 java_thread = (JavaThread*)THREAD; 2084 if (java_thread->is_Java_thread()) { 2085 return java_thread->cooked_allocated_bytes(); 2086 } The cast should be done after is_Java_thread() test. ThreadImpl.java 162 private void throwIfNullThreadIds(long[] ids) { Even better: simply use Objects::requiresNonNull and this method can be removed. This suggests positive naming alternative to throwIfThreadAllocatedMemoryNotSupported - "ensureThreadAllocatedMemorySupported" (sorry I should have suggested that) ThreadMXBean.java 130 * @throws java.lang.UnsupportedOperationException if the Java virtual Nit: "java.lang." can be dropped. @since 14 is missing. Mandy On 8/30/19 3:33 PM, Hohensee, Paul wrote: Thanks for your review, Mandy. Revised webrev at http://cr.openjdk.java.net/~phh/8207266/webrev.02/. I updated the CSR with your suggested javadoc for getCurrentThreadAllocatedBytes. It now matches that for getCurrentThreadUserTime and getCurrentThreadCputime. I also fixed the ?convenient? -> ?convenience? typos in j.l.m.ThreadMXBean.java. I meant GetOneThreads to be the possessive, but don?t feel strongly either way so I?m fine with GetOneThread. I updated ThreadImpl.java as you suggested, though in getThreadAllocatedBytes(long[] ids) I had to add a redundant-in-the-not-length-1-case check for a null ids reference. Would someone take a look at the Hotspot side and the test please? Paul From: Mandy Chung Date: Friday, August 30, 2019 at 10:22 AM To: "Hohensee, Paul" Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread OK. That's better. Some review comments: The javadoc of getCurrentThreadAllocatedBytes() can simply say: "Returns an approximation of the total amount of memory, in bytes, allocated in heap memory for the current thread. This is a convenient method for local management use and is equivalent to calling getThreadAllocatedBytes(Thread.currentThread().getId()). src/hotspot/share/include/jmm.h GetOneThreadsAllocatedMemory: s/OneThreads/OneThread/ sun/management/ThreadImpl.java 43 private static final String THREAD_ALLOCATED_MEMORY_NOT_SUPPORTED = 44 "Thread allocated memory measurement is not supported."; if (!isThreadAllocatedMemorySupported()) { throw new UnsupportedOperationException(THREAD_ALLOCATED_MEMORY_NOT_SUPPORTED); } Perhaps the above can be refactored as throwIfAllocatedMemoryUnsupported() method. 391 if (ids.length == 1) { 392 sizes[0] = -1; : 398 if (ids.length == 1) { 399 long id = ids[0]; 400 sizes[0] = getThreadAllocatedMemory0( 401 Thread.currentThread().getId() == id ? 0 : id); 402 } else { It seems cleaner to handle the 1-element array case at the beginning of this method: if (ids.length == 1) { long size = getThreadAllocatedBytes(ids[0]); return new long[] { size }; } I didn't review the hotspot implementation and the test. Mandy On 8/29/19 10:01 AM, Hohensee, Paul wrote: My bad, Mandy. The webrev puts getCurrentThreadAllocatedBytes in com.sun.management.ThreadMXBean along with the current two getThreadAllocatedBytes methods for the reasons you list. I?ve updated the CSR to specify com.sun.management and added a rationale. AllocatedBytes is currently enabled by Hotspot by default because the overhead of recording TLAB occupancy is negligible. There?s no new GC code, nor will there be, so imo we don?t have to involve the GC folks. I.e., the new JMM method GetOneThreadsAllocatedBytes uses the existing cooked_allocated_bytes JavaThread method, and getCurrentThreadAllocatedBytes is the same as getThreadAllocatedBytes: it just bypasses the thread lookup code. I hadn?t tracked down what happens when getCurrentThreadUserTime and getCurrentThreadCpuTime are called before, but if I?m not mistaken, it the code in jcmd() in attachListener.cpp will call GetThreadCpuTimeWithKind in management.cpp, and it will ultimately use Thread::current() as the subject of the call, see os::current_thread_cpu_time in os_linux.cpp. That means that the CurrentThread methods should work remotely the same way they do locally. GetOneThreadsAllocatedBytes in management.cpp uses THREAD as its subject when called on behalf of getCurrentThreadAllocatedBytes, so it will also uses the current remote Java thread. Even if these methods only worked locally, there are many setups where apps are self-monitoring that could use the performance improvement. Thanks, Paul From: Mandy Chung Date: Wednesday, August 28, 2019 at 3:59 PM To: "Hohensee, Paul" Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread Hi Paul, The CSR proposes this method in java.lang.management.ThreadMXBean as a Java SE feature. Has this been discussed with the GC team to commit measuring current thread's allocated bytes as Java SE feature? Can this be supported by all JVM implementation? What is the overhead if this is enabled by default? Does it need to be disabled? This metric is from TLAB that might be okay. This needs advice/discussion with GC experts. I see that CSR mentions it can be disabled and link to isThreadAllocatedMemoryEnabled() and setThreadAllocatedMemoryEnabled() methods but these methods are defined in com.sun.management.ThreadMXBean. As Alan points out, current thread makes sense only in local VM management. When this is monitored from a JMX client (e.g. jconsole to connect to a running JVM, "currentThreadAllowcatedBytes" attribute is the current thread in jconsole process which invoking Thread::currentThread? Mandy On 8/28/19 12:22 PM, Hohensee, Paul wrote: Please review a performance improvement for ThreadMXBean.getThreadAllocatedBytes and the addition of getCurrentThreadAllocatedBytes. JBS issue: https://bugs.openjdk.java.net/browse/JDK-8207266 Webrev: http://cr.openjdk.java.net/~phh/8207266/webrev.00/ CSR: https://bugs.openjdk.java.net/browse/JDK-8230311 Previous email threads: https://mail.openjdk.java.net/pipermail/serviceability-dev/2018-July/024441.html https://mail.openjdk.java.net/pipermail/serviceability-dev/2018-August/024763.html The CSR is for adding ThreadMXBean.getCurrentThreadAllocatedBytes. I?d be great for someone to review it. I took Mandy?s advice and put the fast paths in the library code. I added a new JMM method GetOneThreadsAllocatedBytes that works the same as GetThreadCpuTime: it uses a thread_id value of zero to distinguish the current thread. On my Mac laptop, the result runs 47x faster for the current thread than the old implementation. The 3 tests in test/jdk/com/sun/management/ThreadMXBean all pass. I added code to ThreadAllocatedMemory.java to test getCurrentThreadAllocatedBytes as well as variations on getThreadAllocatedBytes(id). A submit repo job is in progress. Thanks, Paul From kim.barrett at oracle.com Sun Sep 1 05:01:52 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 1 Sep 2019 01:01:52 -0400 Subject: RFR: JDK-8073188: Remove disable of old MSVC++ warning C4786 In-Reply-To: References: Message-ID: > On Aug 31, 2019, at 7:57 PM, Evgeny Mandrikov wrote: > > Hello! > > Please review patch [1] for JDK-8073188 [2]. Also it needs a sponsor since > I have only author status in OpenJDK Census [3]. > > After this change build passes without warnings on my machine. > > > With best regards, > Evgeny Mandrikov > > [1] http://cr.openjdk.java.net/~godin/8073188/webrev.00/ > [2] https://bugs.openjdk.java.net/browse/JDK-8073188 > [3] https://openjdk.java.net/census#godin CMS has been deprecated for more than two years: https://bugs.openjdk.java.net/browse/JDK-8179013 and there is a proposal to remove it entirely: https://bugs.openjdk.java.net/browse/JDK-8229049 which is winding its way through the process, so making small cleanups to it like this is not really worthwhile. From david.holmes at oracle.com Sun Sep 1 22:52:57 2019 From: david.holmes at oracle.com (David Holmes) Date: Mon, 2 Sep 2019 08:52:57 +1000 Subject: RFR(S): 8229422: Taskqueue: Outdated selection of weak memory model platforms In-Reply-To: References: <9d9819fe-560f-13f0-1907-794e063ee687@oracle.com> <7035ccb8-000c-3a58-b5ac-fb0a3b949784@oracle.com> <381f185e-ca2e-50c4-fe35-1e5e62ff88f6@oracle.com> <95f94a8d-d32f-c2e4-25a0-9d7471f74e08@oracle.com> Message-ID: Hi Martin, On 30/08/2019 9:14 pm, Doerr, Martin wrote: > Hi Thomas, > > good proposal. > > Here's the minimal version: > http://cr.openjdk.java.net/~mdoerr/8229422_multi-copy-atomic/webrev.02/ > > I've removed the compiler part. I can create a separate issue for making C1 and C2 consistent. > > Arm32/aarch64 folks can create new issues if they like further changes. > I don't have any further requirements for s390 and PPC64 at the moment. > > Can I consider it as reviewed by Thomas, David and Derek? The Aarch64 comment begs the question as to why it is not defining CPU_MULTI_COPY_ATOMIC, but otherwise the changes are okay. My suggestion for Aarch64 would be: // Aarch64 was not originally defined as multi-copy-atomic, but now is. // See: "Simplifying ARM Concurrency: Multicopy-atomic Axiomatic and Operational Models for ARMv8" // So we could #define CPU_MULTI_COPY_ATOMIC but historically we have not done so. Thanks, David > Best regards, > Martin > > >> -----Original Message----- >> From: Thomas Schatzl >> Sent: Freitag, 30. August 2019 10:33 >> To: Doerr, Martin ; hotspot-runtime- >> dev at openjdk.java.net; 'hotspot-compiler-dev at openjdk.java.net' > compiler-dev at openjdk.java.net> >> Cc: hotspot-gc-dev at openjdk.java.net; David Holmes >> (david.holmes at oracle.com) ; Derek White >> >> Subject: Re: RFR(S): 8229422: Taskqueue: Outdated selection of weak >> memory model platforms >> >> Hi, >> >> On 26.08.19 15:04, Doerr, Martin wrote: >>> Hi all, >>> >>> I had noticed that the platforms selection which need a fence in >> taskqueue.inline.hpp should get updated. >>> >>> My initial webrev >>> http://cr.openjdk.java.net/~mdoerr/8229422_multi-copy- >> atomic/webrev.00/ >>> was already reviewed on hotspot-gc-dev. It is an attempt to make things >> more consistent, especially the property "CPU_MULTI_COPY_ATOMIC". >>> Also the compiler constant >> "support_IRIW_for_not_multiple_copy_atomic_cpu" depends on this >> property (currently only used on PPC64). >>> >>> We could go one step further and move even more #defines into the >> platform files to give platform maintainers more control. >>> I haven't got feedback from arm/aarch64 folks about this addition, yet: >>> http://cr.openjdk.java.net/~mdoerr/8229422_multi-copy- >> atomic/webrev.01/ >>> With this proposal, each platform which is "CPU_MULTI_COPY_ATOMIC" is >> supposed to define this macro. >>> Other platforms must define >> SUPPORT_IRIW_FOR_NOT_MULTI_COPY_ATOMIC_CPU and >> IRIW_WITH_RELEASE_VOLATILE_IN_CONSTRUCTOR for fine-grained control >> of the memory ordering behavior. >>> We can even control them dynamically (added an experimental switch for >> PPC64 as an example). >>> >>> Note that neither webrev.00 nor webrev.01 contain any functional changes >> other than the taskqueue update for s390 (and the experimental switch for >> PPC64 in webrev.01). >>> >>> Feedback is welcome. Also if you have a preference wrt. webrev.00 vs. >> webrev.01. >> >> for pushing I would prefer the minimal amount of changes to solve the >> original issue, and move all other changes to a different CR. >> >> Also, I would prefer if all globalDefinitions files contained all >> defines, commented out if needed. I.e. to try to show that not defining >> a particular macro has been deliberate and not an oversight. >> >> (Like in the 00 webrev where the code at least states for aarch64: >> 37 // aarch64 is not CPU_MULTI_COPY_ATOMIC >> >> I am aware that this is not correct given new information, but in >> context of the CR it is/was) >> >> Further, let's avoid "TODOs" in the sources, the correct place for those >> is JIRA imho. :) >> >> Thanks, >> Thomas >> > From shade at redhat.com Mon Sep 2 08:31:26 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 2 Sep 2019 10:31:26 +0200 Subject: RFR (XS) 8230425: Shenandoah forces +UseNUMAInterleaving even after explicitly disabled Message-ID: <1be69480-6b62-4f73-7e09-4a74d794a74e@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8230425 Shenandoah enables NUMA when possible, but it also unconditionally sets UseNUMAInterleaving, which is not correct and overrides user setting. Fix: diff -r a333fdeb8de0 src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp --- a/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Fri Aug 30 09:37:41 2019 +0200 +++ b/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Mon Sep 02 10:29:49 2019 +0200 @@ -65,5 +65,7 @@ if (FLAG_IS_DEFAULT(UseNUMA)) { FLAG_SET_DEFAULT(UseNUMA, true); - FLAG_SET_DEFAULT(UseNUMAInterleaving, true); + if (FLAG_IS_DEFAULT(UseNUMAInterleaving)) { + FLAG_SET_DEFAULT(UseNUMAInterleaving, true); + } } Testing: manual testing, hotspot_gc_shenandoah -- Thanks, -Aleksey From shade at redhat.com Mon Sep 2 08:47:42 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 2 Sep 2019 10:47:42 +0200 Subject: RFR (XS) 8230425: Shenandoah forces +UseNUMAInterleaving even after explicitly disabled In-Reply-To: <1be69480-6b62-4f73-7e09-4a74d794a74e@redhat.com> References: <1be69480-6b62-4f73-7e09-4a74d794a74e@redhat.com> Message-ID: <08772877-880c-2d15-a661-603402606eea@redhat.com> On 9/2/19 10:31 AM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8230425 > > Shenandoah enables NUMA when possible, but it also unconditionally sets UseNUMAInterleaving, which > is not correct and overrides user setting. > > Fix: > > diff -r a333fdeb8de0 src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp > --- a/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Fri Aug 30 09:37:41 2019 +0200 > +++ b/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Mon Sep 02 10:29:49 2019 +0200 > @@ -65,5 +65,7 @@ > if (FLAG_IS_DEFAULT(UseNUMA)) { > FLAG_SET_DEFAULT(UseNUMA, true); > - FLAG_SET_DEFAULT(UseNUMAInterleaving, true); > + if (FLAG_IS_DEFAULT(UseNUMAInterleaving)) { > + FLAG_SET_DEFAULT(UseNUMAInterleaving, true); > + } > } Even better, after observing that Arguments::adjust_after_os does enable UseNUMAInterleaving once OS-specific code had chance to act: diff -r a333fdeb8de0 src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp --- a/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Fri Aug 30 09:37:41 2019 +0200 +++ b/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Mon Sep 02 10:38:49 2019 +0200 @@ -60,11 +60,9 @@ } // Enable NUMA by default. While Shenandoah is not NUMA-aware, enabling NUMA makes - // storage allocation code NUMA-aware, and NUMA interleaving makes the storage - // allocated in consistent manner (interleaving) to minimize run-to-run variance. + // storage allocation code NUMA-aware. if (FLAG_IS_DEFAULT(UseNUMA)) { FLAG_SET_DEFAULT(UseNUMA, true); - FLAG_SET_DEFAULT(UseNUMAInterleaving, true); } Testing: more manual testing on NUMA-enabled machines -- Thanks, -Aleksey From rkennke at redhat.com Mon Sep 2 10:06:56 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 02 Sep 2019 12:06:56 +0200 Subject: RFR (XS) 8230425: Shenandoah forces +UseNUMAInterleaving even after explicitly disabled In-Reply-To: <08772877-880c-2d15-a661-603402606eea@redhat.com> References: <1be69480-6b62-4f73-7e09-4a74d794a74e@redhat.com> <08772877-880c-2d15-a661-603402606eea@redhat.com> Message-ID: <1E66F814-6270-4901-857F-CB8FD3851DB9@redhat.com> Looks good. Thanks! Am 2. September 2019 10:47:42 MESZ schrieb Aleksey Shipilev : >On 9/2/19 10:31 AM, Aleksey Shipilev wrote: >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8230425 >> >> Shenandoah enables NUMA when possible, but it also unconditionally >sets UseNUMAInterleaving, which >> is not correct and overrides user setting. >> >> Fix: >> >> diff -r a333fdeb8de0 >src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp >> --- a/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Fri >Aug 30 09:37:41 2019 +0200 >> +++ b/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Mon >Sep 02 10:29:49 2019 +0200 >> @@ -65,5 +65,7 @@ >> if (FLAG_IS_DEFAULT(UseNUMA)) { >> FLAG_SET_DEFAULT(UseNUMA, true); >> - FLAG_SET_DEFAULT(UseNUMAInterleaving, true); >> + if (FLAG_IS_DEFAULT(UseNUMAInterleaving)) { >> + FLAG_SET_DEFAULT(UseNUMAInterleaving, true); >> + } >> } > >Even better, after observing that Arguments::adjust_after_os does >enable UseNUMAInterleaving once >OS-specific code had chance to act: > >diff -r a333fdeb8de0 >src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp >--- a/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Fri Aug >30 09:37:41 2019 +0200 >+++ b/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Mon Sep >02 10:38:49 2019 +0200 >@@ -60,11 +60,9 @@ > } > >// Enable NUMA by default. While Shenandoah is not NUMA-aware, enabling >NUMA makes >- // storage allocation code NUMA-aware, and NUMA interleaving makes >the storage >- // allocated in consistent manner (interleaving) to minimize >run-to-run variance. >+ // storage allocation code NUMA-aware. > if (FLAG_IS_DEFAULT(UseNUMA)) { > FLAG_SET_DEFAULT(UseNUMA, true); >- FLAG_SET_DEFAULT(UseNUMAInterleaving, true); > } > >Testing: more manual testing on NUMA-enabled machines > >-- >Thanks, >-Aleksey -- Diese Nachricht wurde von meinem Android-Ger?t mit K-9 Mail gesendet. From shade at redhat.com Mon Sep 2 10:10:48 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 2 Sep 2019 12:10:48 +0200 Subject: RFR (XS) 8230425: Shenandoah forces +UseNUMAInterleaving even after explicitly disabled In-Reply-To: <1E66F814-6270-4901-857F-CB8FD3851DB9@redhat.com> References: <1be69480-6b62-4f73-7e09-4a74d794a74e@redhat.com> <08772877-880c-2d15-a661-603402606eea@redhat.com> <1E66F814-6270-4901-857F-CB8FD3851DB9@redhat.com> Message-ID: <0a48f1bb-0b55-64db-d161-124bc953f9a0@redhat.com> On 9/2/19 12:06 PM, Roman Kennke wrote: > Looks good. Thanks! Thanks, pushed. -- -Aleksey From shade at redhat.com Mon Sep 2 10:49:26 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 2 Sep 2019 12:49:26 +0200 Subject: RFR (XS) 8230425: Shenandoah forces +UseNUMAInterleaving even after explicitly disabled In-Reply-To: <0a48f1bb-0b55-64db-d161-124bc953f9a0@redhat.com> References: <1be69480-6b62-4f73-7e09-4a74d794a74e@redhat.com> <08772877-880c-2d15-a661-603402606eea@redhat.com> <1E66F814-6270-4901-857F-CB8FD3851DB9@redhat.com> <0a48f1bb-0b55-64db-d161-124bc953f9a0@redhat.com> Message-ID: <81767a29-61d7-9581-6b7b-82a10888c986@redhat.com> On 9/2/19 12:10 PM, Aleksey Shipilev wrote: > On 9/2/19 12:06 PM, Roman Kennke wrote: >> Looks good. Thanks! > > Thanks, pushed. I would also backport it to sh/jdk8 and sh/jdk11 immediately. The patches apply cleanly there (with reshuffling in 8u), and pass my manual tests. Ok? -- Thanks, -Aleksey From martin.doerr at sap.com Mon Sep 2 11:04:31 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 2 Sep 2019 11:04:31 +0000 Subject: RFR(S): 8229422: Taskqueue: Outdated selection of weak memory model platforms In-Reply-To: References: <9d9819fe-560f-13f0-1907-794e063ee687@oracle.com> <7035ccb8-000c-3a58-b5ac-fb0a3b949784@oracle.com> <381f185e-ca2e-50c4-fe35-1e5e62ff88f6@oracle.com> <95f94a8d-d32f-c2e4-25a0-9d7471f74e08@oracle.com> Message-ID: Hi all, @David: Thanks for the more verbose description for aarch64. I've pushed it with this version. @Thomas: Thanks for creating JDK-8230392. I've created JDK-8230434 for making C1 and C2 consistent. > Somebody take my keyboard from me! @The person who has potentially taken Derek's keyboard: Please give it back to him. We miss him. ? But I had already an ok from him for webrev.00. Best regards, Martin From thomas.schatzl at oracle.com Mon Sep 2 09:02:44 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 2 Sep 2019 11:02:44 +0200 Subject: RFR(S): 8229422: Taskqueue: Outdated selection of weak memory model platforms In-Reply-To: References: <9d9819fe-560f-13f0-1907-794e063ee687@oracle.com> <7035ccb8-000c-3a58-b5ac-fb0a3b949784@oracle.com> <381f185e-ca2e-50c4-fe35-1e5e62ff88f6@oracle.com> <95f94a8d-d32f-c2e4-25a0-9d7471f74e08@oracle.com> Message-ID: <0cd245b5-34cb-48e0-4155-3d19e62a3f6a@oracle.com> Hi, On 02.09.19 00:52, David Holmes wrote: > Hi Martin, > > On 30/08/2019 9:14 pm, Doerr, Martin wrote: >> Hi Thomas, >> >> good proposal. >> >> Here's the minimal version: >> http://cr.openjdk.java.net/~mdoerr/8229422_multi-copy-atomic/webrev.02/ >> >> I've removed the compiler part. I can create a separate issue for >> making C1 and C2 consistent. >> >> Arm32/aarch64 folks can create new issues if they like further changes. >> I don't have any further requirements for s390 and PPC64 at the moment. >> >> Can I consider it as reviewed by Thomas, David and Derek? > > The Aarch64 comment begs the question as to why it is not defining > CPU_MULTI_COPY_ATOMIC, but otherwise the changes are okay. My suggestion > for Aarch64 would be: > > // Aarch64 was not originally defined as multi-copy-atomic, but now is. > // See: "Simplifying ARM Concurrency: Multicopy-atomic Axiomatic and > Operational Models for ARMv8" > // So we could #define CPU_MULTI_COPY_ATOMIC but historically we have > not done so. > Fwiw, I am good with this comment too. Thanks, Thomas From stefan.johansson at oracle.com Mon Sep 2 18:53:01 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 2 Sep 2019 20:53:01 +0200 Subject: RFR: 8230431: Move G1 trace code from gcTrace* to G1 directory Message-ID: <5CF52FE0-40D4-45B2-BD11-3894CE194A8F@oracle.com> Hi all, Please review this cleanup to prepare for fixing JDK-8209802. JBS: https://bugs.openjdk.java.net/browse/JDK-8230431 Webrev: http://cr.openjdk.java.net/~sjohanss/8230431/00/ Summary: Currently a lot of JFR related gc code is located in share/gcTrace.* and share/gcTraceSend.cpp. To fix JDK-8209802, we need to have a way to initialize and register a few JFR constants specific to G1 and to prepare for this I suggest that we move the G1 parts from the above files into g1/g1Trace.* to create a place where this can be done. This is something that probably should have been done regardless of this bug, but now is a good to get it done. Testing: Mach5 tier 1-3 Cheers, Stefan From stefan.johansson at oracle.com Mon Sep 2 19:18:58 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 2 Sep 2019 21:18:58 +0200 Subject: RFR: 8209802: Garbage collectors should register JFR types themselves to avoid build errors. Message-ID: <6C8A6F4E-E5BE-459F-944F-BD69F5CBEB59@oracle.com> Hi, Please review this patch to fix JDK-8209802. JBS: https://bugs.openjdk.java.net/browse/JDK-8209802 Webrev: http://cr.openjdk.java.net/~sjohanss/8209802/00/ Summary: This patch builds on the cleanup made in JDK-JDK-8230431, and adds an initialize method to G1NewTracer, that takes care of registering serializers needed for the two G1-specific constants in JFR. There is also a G1 periodic event but there is no similar way to register this from G1 code so it needs to be solved by guarding the G1 specific code in jfrPeriodic.cpp. Testing: Mach5 tier 1-3 Thanks, Stefan From kim.barrett at oracle.com Mon Sep 2 20:46:23 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 2 Sep 2019 16:46:23 -0400 Subject: RFR: 8230431: Move G1 trace code from gcTrace* to G1 directory In-Reply-To: <5CF52FE0-40D4-45B2-BD11-3894CE194A8F@oracle.com> References: <5CF52FE0-40D4-45B2-BD11-3894CE194A8F@oracle.com> Message-ID: <226187A4-8949-4154-87B1-C6EEA57925B6@oracle.com> > On Sep 2, 2019, at 2:53 PM, Stefan Johansson wrote: > > Hi all, > > Please review this cleanup to prepare for fixing JDK-8209802. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8230431 > Webrev: http://cr.openjdk.java.net/~sjohanss/8230431/00/ > > Summary: > Currently a lot of JFR related gc code is located in share/gcTrace.* and share/gcTraceSend.cpp. To fix JDK-8209802, we need to have a way to initialize and register a few JFR constants specific to G1 and to prepare for this I suggest that we move the G1 parts from the above files into g1/g1Trace.* to create a place where this can be done. This is something that probably should have been done regardless of this bug, but now is a good to get it done. > > Testing: > Mach5 tier 1-3 > > Cheers, > Stefan Looks good. From kim.barrett at oracle.com Mon Sep 2 20:51:52 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 2 Sep 2019 16:51:52 -0400 Subject: RFR: 8209802: Garbage collectors should register JFR types themselves to avoid build errors. In-Reply-To: <6C8A6F4E-E5BE-459F-944F-BD69F5CBEB59@oracle.com> References: <6C8A6F4E-E5BE-459F-944F-BD69F5CBEB59@oracle.com> Message-ID: > On Sep 2, 2019, at 3:18 PM, Stefan Johansson wrote: > > Hi, > > Please review this patch to fix JDK-8209802. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8209802 > Webrev: http://cr.openjdk.java.net/~sjohanss/8209802/00/ > > Summary: > This patch builds on the cleanup made in JDK-JDK-8230431, and adds an initialize method to G1NewTracer, that takes care of registering serializers needed for the two G1-specific constants in JFR. There is also a G1 periodic event but there is no similar way to register this from G1 code so it needs to be solved by guarding the G1 specific code in jfrPeriodic.cpp. > > Testing: > Mach5 tier 1-3 > > Thanks, > Stefan Looks good. From ioi.lam at oracle.com Mon Sep 2 23:41:21 2019 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 2 Sep 2019 16:41:21 -0700 Subject: Likely a bug in G1BarrierSetAssembler::oop_store_at In-Reply-To: <0b228b16-4e73-88db-379b-bf766bb5fd42@samersoff.net> References: <0b228b16-4e73-88db-379b-bf766bb5fd42@samersoff.net> Message-ID: <296450f0-0f3d-e180-e12f-e63b738ac180@oracle.com> CC-ing hotspot-gc-dev at openjdk.java.net. Hi Dmitry, The code you mentioned also exists in the mainline repo: http://hg.openjdk.java.net/jdk/jdk/file/7cf02b2c1455/src/hotspot/cpu/x86/gc/g1/g1BarrierSetAssembler_x86.cpp#l392 I think the code is OK, but maybe some from GC team can take a look at it, too. The movptr is necessary because BarrierSetAssembler::store_at may modify the contents of val, so we save its value into a temp register, tmp2. ??? if (needs_post_barrier) { ????? // G1 barrier needs uncompressed oop for region cross check. ????? if (UseCompressedOops) { ??????? new_val = tmp2; ??????? __ movptr(new_val, val); ????? } ??? } ??? BarrierSetAssembler::store_at(masm, decorators, type, Address(tmp1, 0), val, noreg, noreg, noreg); Later, inside g1_write_barrier_post, tmp2 is written only here, but we never read new_value after writing into tmp2, so we should be safe. ? __ jcc(Assembler::equal, runtime); ? __ subl(queue_index, wordSize); ? __ movptr(tmp2, buffer); #ifdef _LP64 ? __ movslq(rscratch1, queue_index); ? __ addq(tmp2, rscratch1); ? __ movq(Address(tmp2, 0), card_addr); #else ? __ addl(tmp2, queue_index); ? __ movl(Address(tmp2, 0), card_addr); #endif ? __ jmp(done); ? __ bind(runtime); ? // save the live input values ? __ push(store_addr); ? __ push(new_val); #ifdef _LP64 ? __ call_VM_leaf(CAST_FROM_FN_PTR(address, G1BarrierSetRuntime::write_ref_field_post_entry), card_addr, r15_thread); #else ? __ push(thread); ? __ call_VM_leaf(CAST_FROM_FN_PTR(address, G1BarrierSetRuntime::write_ref_field_post_entry), card_addr, thread); ? __ pop(thread); #endif ? __ pop(new_val); ? __ pop(store_addr); ? __ bind(done); My question is --- why are we saving new_value when making the runtime call? new_val can only be val or tmp2, but it's OK to scratch both registers in G1BarrierSetAssembler::oop_store_at(), and neither registers are used after the runtime call. I commented out the push/pop of new_val and ran all tests under test/hotspot/jtreg/gc without problem. Thanks - Ioi On 9/1/19 9:06 AM, Dmitry Samersoff wrote: > Hello Everybody, > > I found a following code in x86 G1BarrierSetAssembler::oop_store_at > > It looks like we pass the same register as both new_val and tmp2 to > g1_write_barrier_post. > > I don't have x86 setup in hands so can't say how critical it is. > > > // G1 barrier needs uncompressed oop for region cross check. > if (UseCompressedOops) { > new_val = tmp2; > __ movptr(new_val, val); > } > } > > BarrierSetAssembler::store_at(masm, decorators, type, Address(tmp1, > 0), val, noreg, noreg); > > if (needs_post_barrier) { > g1_write_barrier_post(masm /*masm*/, > tmp1 /* store_adr */, > new_val /* new_val */, > rthread /* thread */, > tmp3 /* tmp */, > tmp2 /* tmp2 */); > } > > -Dmitry > From leo.korinth at oracle.com Tue Sep 3 08:15:08 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Tue, 3 Sep 2019 10:15:08 +0200 Subject: RFR: 8230431: Move G1 trace code from gcTrace* to G1 directory In-Reply-To: <5CF52FE0-40D4-45B2-BD11-3894CE194A8F@oracle.com> References: <5CF52FE0-40D4-45B2-BD11-3894CE194A8F@oracle.com> Message-ID: <6374594f-e47b-479a-429c-1f4a6d2f1e7f@oracle.com> On 02/09/2019 20:53, Stefan Johansson wrote: > Hi all, > > Please review this cleanup to prepare for fixing JDK-8209802. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8230431 > Webrev: http://cr.openjdk.java.net/~sjohanss/8230431/00/ > > Summary: > Currently a lot of JFR related gc code is located in share/gcTrace.* and share/gcTraceSend.cpp. To fix JDK-8209802, we need to have a way to initialize and register a few JFR constants specific to G1 and to prepare for this I suggest that we move the G1 parts from the above files into g1/g1Trace.* to create a place where this can be done. This is something that probably should have been done regardless of this bug, but now is a good to get it done. > > Testing: > Mach5 tier 1-3 > > Cheers, > Stefan > Looks good, nice cleanup! Fix the #include sorting in g1Trace.hpp before pushing. No need for another webrev. Thanks, Leo From stefan.johansson at oracle.com Tue Sep 3 08:37:04 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 3 Sep 2019 10:37:04 +0200 Subject: RFR: 8230431: Move G1 trace code from gcTrace* to G1 directory In-Reply-To: <6374594f-e47b-479a-429c-1f4a6d2f1e7f@oracle.com> References: <5CF52FE0-40D4-45B2-BD11-3894CE194A8F@oracle.com> <6374594f-e47b-479a-429c-1f4a6d2f1e7f@oracle.com> Message-ID: <3f26c299-f6d9-139c-8673-4fa5795c0fae@oracle.com> Thanks Leo and Kim for the reviews. I will fix the sorting before pushing, Stefan On 2019-09-03 10:15, Leo Korinth wrote: > On 02/09/2019 20:53, Stefan Johansson wrote: >> Hi all, >> >> Please review this cleanup to prepare for fixing JDK-8209802. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8230431 >> Webrev: http://cr.openjdk.java.net/~sjohanss/8230431/00/ >> >> Summary: >> Currently a lot of JFR related gc code is located in share/gcTrace.* >> and share/gcTraceSend.cpp. To fix JDK-8209802, we need to have a way >> to initialize and register a few JFR constants specific to G1 and to >> prepare for this I suggest that we move the G1 parts from the above >> files into g1/g1Trace.* to create a place where this can be done. This >> is something that probably should have been done regardless of this >> bug, but now is a good to get it done. >> >> Testing: >> Mach5 tier 1-3 >> >> Cheers, >> Stefan >> > > Looks good, nice cleanup! > > Fix the #include sorting in g1Trace.hpp before pushing. No need for > another webrev. > > Thanks, > Leo From thomas.schatzl at oracle.com Tue Sep 3 13:40:43 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 03 Sep 2019 15:40:43 +0200 Subject: Likely a bug in G1BarrierSetAssembler::oop_store_at In-Reply-To: <296450f0-0f3d-e180-e12f-e63b738ac180@oracle.com> References: <0b228b16-4e73-88db-379b-bf766bb5fd42@samersoff.net> <296450f0-0f3d-e180-e12f-e63b738ac180@oracle.com> Message-ID: <91552df43e882d84774261774f7d2f995fe661fd.camel@oracle.com> Hi, On Mon, 2019-09-02 at 16:41 -0700, Ioi Lam wrote: > CC-ing hotspot-gc-dev at openjdk.java.net. > > > Hi Dmitry, > > The code you mentioned also exists in the mainline repo: > > http://hg.openjdk.java.net/jdk/jdk/file/7cf02b2c1455/src/hotspot/cpu/x86/gc/g1/g1BarrierSetAssembler_x86.cpp#l392 > > I think the code is OK, but maybe some from GC team can take a look > at it, too. The code is okay. The only problem is that > > The movptr is necessary because BarrierSetAssembler::store_at may > modify the contents of val, so we save its value into a temp To be a little more exact, BarrierSetAssembler::store_at modifies the value in val if it is a reference, i.e. it compresses it with compressed oops enabled. For the inter-region filter we need the uncompressed value though, saving it in new_val/tmp2. > register, tmp2. > > if (needs_post_barrier) { > // G1 barrier needs uncompressed oop for region cross check. > if (UseCompressedOops) { > new_val = tmp2; > __ movptr(new_val, val); > } > } > BarrierSetAssembler::store_at(masm, decorators, type, > Address(tmp1, 0), val, noreg, noreg, noreg); > > > Later, inside g1_write_barrier_post, tmp2 is written only here, but > we never read new_value after writing into tmp2, so we should be > safe. > > __ jcc(Assembler::equal, runtime); > __ subl(queue_index, wordSize); > __ movptr(tmp2, buffer); > #ifdef _LP64 > __ movslq(rscratch1, queue_index); > __ addq(tmp2, rscratch1); > __ movq(Address(tmp2, 0), card_addr); > #else > __ addl(tmp2, queue_index); > __ movl(Address(tmp2, 0), card_addr); > #endif > __ jmp(done); > > __ bind(runtime); > // save the live input values > __ push(store_addr); > __ push(new_val); > #ifdef _LP64 > __ call_VM_leaf(CAST_FROM_FN_PTR(address, > G1BarrierSetRuntime::write_ref_field_post_entry), card_addr, > r15_thread); > #else > __ push(thread); > __ call_VM_leaf(CAST_FROM_FN_PTR(address, > G1BarrierSetRuntime::write_ref_field_post_entry), card_addr, thread); > __ pop(thread); > #endif > __ pop(new_val); > __ pop(store_addr); > > __ bind(done); > > > My question is --- why are we saving new_value when making the > runtime call? new_val can only be val or tmp2, but it's OK to scratch > both registers in G1BarrierSetAssembler::oop_store_at(), and neither > registers are used after the runtime call. I can imagine that sometime/somewhere there has been an implicit contract that "val" should not be modified. Apparently G1 never did that anyway... I will file a CR to look at this some more. > > I commented out the push/pop of new_val and ran all tests under > test/hotspot/jtreg/gc without problem. Thanks, Thomas From thomas.schatzl at oracle.com Tue Sep 3 13:48:12 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 03 Sep 2019 15:48:12 +0200 Subject: RFR: 8209802: Garbage collectors should register JFR types themselves to avoid build errors. In-Reply-To: <6C8A6F4E-E5BE-459F-944F-BD69F5CBEB59@oracle.com> References: <6C8A6F4E-E5BE-459F-944F-BD69F5CBEB59@oracle.com> Message-ID: <4f436f8cd7f7476251216e1634ecc3d4470453f1.camel@oracle.com> Hi, On Mon, 2019-09-02 at 21:18 +0200, Stefan Johansson wrote: > Hi, > > Please review this patch to fix JDK-8209802. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8209802 > Webrev: http://cr.openjdk.java.net/~sjohanss/8209802/00/ > > Summary: > This patch builds on the cleanup made in JDK-JDK-8230431, and adds an > initialize method to G1NewTracer, that takes care of registering > serializers needed for the two G1-specific constants in JFR. There is > also a G1 periodic event but there is no similar way to register this > from G1 code so it needs to be solved by guarding the G1 specific > code in jfrPeriodic.cpp. > > Testing: > Mach5 tier 1-3 > looks good. Thomas From martin.doerr at sap.com Tue Sep 3 13:50:16 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 3 Sep 2019 13:50:16 +0000 Subject: Likely a bug in G1BarrierSetAssembler::oop_store_at In-Reply-To: <296450f0-0f3d-e180-e12f-e63b738ac180@oracle.com> References: <0b228b16-4e73-88db-379b-bf766bb5fd42@samersoff.net> <296450f0-0f3d-e180-e12f-e63b738ac180@oracle.com> Message-ID: Hi, I agree with Ioi, tmp2 only gets modified after new_val was read for the last time. So I can't see any bug here. > My question is --- why are we saving new_value when making the runtime > call? new_val can only be val or tmp2, but it's OK to scratch both > registers in G1BarrierSetAssembler::oop_store_at(), and neither > registers are used after the runtime call. I think we could skip saving and restoring it if we use CompressedOops (because it's just a temp register). If running with -XX:-UseCompressedOops, I think we should preserve the register "val". Users of store_at may not expect val to get killed. Best regards, Martin > -----Original Message----- > From: hotspot-gc-dev On > Behalf Of Ioi Lam > Sent: Dienstag, 3. September 2019 01:41 > To: valhalla-dev at openjdk.java.net; hotspot-gc-dev dev at openjdk.java.net> > Subject: Re: Likely a bug in G1BarrierSetAssembler::oop_store_at > > CC-ing hotspot-gc-dev at openjdk.java.net. > > > Hi Dmitry, > > The code you mentioned also exists in the mainline repo: > > http://hg.openjdk.java.net/jdk/jdk/file/7cf02b2c1455/src/hotspot/cpu/x86/ > gc/g1/g1BarrierSetAssembler_x86.cpp#l392 > > I think the code is OK, but maybe some from GC team can take a look at > it, too. > > The movptr is necessary because BarrierSetAssembler::store_at may modify > the contents of val, so we save its value into a temp register, tmp2. > > ??? if (needs_post_barrier) { > ????? // G1 barrier needs uncompressed oop for region cross check. > ????? if (UseCompressedOops) { > ??????? new_val = tmp2; > ??????? __ movptr(new_val, val); > ????? } > ??? } > ??? BarrierSetAssembler::store_at(masm, decorators, type, Address(tmp1, > 0), val, noreg, noreg, noreg); > > > Later, inside g1_write_barrier_post, tmp2 is written only here, but we > never read new_value after writing into tmp2, so we should be safe. > > ? __ jcc(Assembler::equal, runtime); > ? __ subl(queue_index, wordSize); > ? __ movptr(tmp2, buffer); > #ifdef _LP64 > ? __ movslq(rscratch1, queue_index); > ? __ addq(tmp2, rscratch1); > ? __ movq(Address(tmp2, 0), card_addr); > #else > ? __ addl(tmp2, queue_index); > ? __ movl(Address(tmp2, 0), card_addr); > #endif > ? __ jmp(done); > > ? __ bind(runtime); > ? // save the live input values > ? __ push(store_addr); > ? __ push(new_val); > #ifdef _LP64 > ? __ call_VM_leaf(CAST_FROM_FN_PTR(address, > G1BarrierSetRuntime::write_ref_field_post_entry), card_addr, r15_thread); > #else > ? __ push(thread); > ? __ call_VM_leaf(CAST_FROM_FN_PTR(address, > G1BarrierSetRuntime::write_ref_field_post_entry), card_addr, thread); > ? __ pop(thread); > #endif > ? __ pop(new_val); > ? __ pop(store_addr); > > ? __ bind(done); > > > My question is --- why are we saving new_value when making the runtime > call? new_val can only be val or tmp2, but it's OK to scratch both > registers in G1BarrierSetAssembler::oop_store_at(), and neither > registers are used after the runtime call. > > I commented out the push/pop of new_val and ran all tests under > test/hotspot/jtreg/gc without problem. > > > Thanks > - Ioi > > On 9/1/19 9:06 AM, Dmitry Samersoff wrote: > > Hello Everybody, > > > > I found a following code in x86 G1BarrierSetAssembler::oop_store_at > > > > It looks like we pass the same register as both new_val and tmp2 to > > g1_write_barrier_post. > > > > I don't have x86 setup in hands so can't say how critical it is. > > > > > > // G1 barrier needs uncompressed oop for region cross check. > > if (UseCompressedOops) { > > new_val = tmp2; > > __ movptr(new_val, val); > > } > > } > > > > BarrierSetAssembler::store_at(masm, decorators, type, Address(tmp1, > > 0), val, noreg, noreg); > > > > if (needs_post_barrier) { > > g1_write_barrier_post(masm /*masm*/, > > tmp1 /* store_adr */, > > new_val /* new_val */, > > rthread /* thread */, > > tmp3 /* tmp */, > > tmp2 /* tmp2 */); > > } > > > > -Dmitry > > From zgu at redhat.com Tue Sep 3 16:56:39 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 3 Sep 2019 12:56:39 -0400 Subject: RFR 8230483: Shenandoah: Shenandoah assert_correct failed; Object klass pointer should not be NULL Message-ID: SAP reported shenandoah assertion failure since 2019-07-14. I could not reproduce locally. I did find an instance of failure in our nightlies dated back to 2019-08-27. I believe it is caused by JDK-8226757, which changed behavior of SH:is_concurrent_traversal_in_progress(). We do have the fix in shenandoah/jdk, but has yet upstreamed. Webrev: http://cr.openjdk.java.net/~zgu/JDK-8230483/webev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) on Linux x86_64. Thanks, -Zhengyu From shade at redhat.com Tue Sep 3 17:00:51 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 3 Sep 2019 19:00:51 +0200 Subject: RFR 8230483: Shenandoah: Shenandoah assert_correct failed; Object klass pointer should not be NULL In-Reply-To: References: Message-ID: On 9/3/19 6:56 PM, Zhengyu Gu wrote: > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8230483/webev.00/ D'uh. Looks good! Maybe change the synopsis? E.g.: "Shenandoah: consistently disable concurrent roots for Traversal mode" -- Thanks, -Aleksey From zgu at redhat.com Tue Sep 3 17:57:17 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 3 Sep 2019 13:57:17 -0400 Subject: RFR 8230483: Shenandoah: Shenandoah assert_correct failed; Object klass pointer should not be NULL In-Reply-To: References: Message-ID: <7a0444d3-7955-8c50-0715-0237642e1964@redhat.com> Thanks for reviewing. On 9/3/19 1:00 PM, Aleksey Shipilev wrote: > On 9/3/19 6:56 PM, Zhengyu Gu wrote: >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8230483/webev.00/ > > D'uh. Looks good! > > Maybe change the synopsis? E.g.: > "Shenandoah: consistently disable concurrent roots for Traversal mode" > Changed and pushed. -Zhengyu From hohensee at amazon.com Tue Sep 3 19:38:08 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Tue, 3 Sep 2019 19:38:08 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> Message-ID: Minor update in new webrev http://cr.openjdk.java.net/~phh/8207266/webrev.05/. I removed ensureNonNullThreadIds() in favor of Objects.requireNonNull(ids). Thanks, Mandy, for your through reviews. May I get another reviewer to weigh in? Paul ?On 8/31/19, 5:06 PM, "hotspot-gc-dev on behalf of Hohensee, Paul" wrote: Thanks, Mandy. I?ve finalized the CSR. New webrev at http://cr.openjdk.java.net/~phh/8207266/webrev.04/. In management.cpp, I now have if (THREAD->is_Java_thread()) { return ((JavaThread*)THREAD)->cooked_allocated_bytes(); } In ThreadImpl.java, using requireNonNull would produce a different and less informative message, so I?d like to leave it as is. I changed throwIfNullThreadIds to ensureNonNullThreadIds, and throwIfThreadAllocatedMemoryNotSupported to ensureThreadAllocatedMemorySupported. I dropped the ?java.lang.? prefix from all uses of UnsupportedOperationException in both c.s.m.ThreadMXBean.java and j.l.m.ThreadMXBean.java, and did the same with SecurityException. ?@since 14? added to c.s.m.ThreadMXBean.java and the CSR. Do I need another reviewer? Paul From: Mandy Chung Date: Friday, August 30, 2019 at 4:26 PM To: "Hohensee, Paul" Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread CSR reviewed. management.cpp 2083 java_thread = (JavaThread*)THREAD; 2084 if (java_thread->is_Java_thread()) { 2085 return java_thread->cooked_allocated_bytes(); 2086 } The cast should be done after is_Java_thread() test. ThreadImpl.java 162 private void throwIfNullThreadIds(long[] ids) { Even better: simply use Objects::requiresNonNull and this method can be removed. This suggests positive naming alternative to throwIfThreadAllocatedMemoryNotSupported - "ensureThreadAllocatedMemorySupported" (sorry I should have suggested that) ThreadMXBean.java 130 * @throws java.lang.UnsupportedOperationException if the Java virtual Nit: "java.lang." can be dropped. @since 14 is missing. Mandy On 8/30/19 3:33 PM, Hohensee, Paul wrote: Thanks for your review, Mandy. Revised webrev at http://cr.openjdk.java.net/~phh/8207266/webrev.02/. I updated the CSR with your suggested javadoc for getCurrentThreadAllocatedBytes. It now matches that for getCurrentThreadUserTime and getCurrentThreadCputime. I also fixed the ?convenient? -> ?convenience? typos in j.l.m.ThreadMXBean.java. I meant GetOneThreads to be the possessive, but don?t feel strongly either way so I?m fine with GetOneThread. I updated ThreadImpl.java as you suggested, though in getThreadAllocatedBytes(long[] ids) I had to add a redundant-in-the-not-length-1-case check for a null ids reference. Would someone take a look at the Hotspot side and the test please? Paul From: Mandy Chung Date: Friday, August 30, 2019 at 10:22 AM To: "Hohensee, Paul" Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread OK. That's better. Some review comments: The javadoc of getCurrentThreadAllocatedBytes() can simply say: "Returns an approximation of the total amount of memory, in bytes, allocated in heap memory for the current thread. This is a convenient method for local management use and is equivalent to calling getThreadAllocatedBytes(Thread.currentThread().getId()). src/hotspot/share/include/jmm.h GetOneThreadsAllocatedMemory: s/OneThreads/OneThread/ sun/management/ThreadImpl.java 43 private static final String THREAD_ALLOCATED_MEMORY_NOT_SUPPORTED = 44 "Thread allocated memory measurement is not supported."; if (!isThreadAllocatedMemorySupported()) { throw new UnsupportedOperationException(THREAD_ALLOCATED_MEMORY_NOT_SUPPORTED); } Perhaps the above can be refactored as throwIfAllocatedMemoryUnsupported() method. 391 if (ids.length == 1) { 392 sizes[0] = -1; : 398 if (ids.length == 1) { 399 long id = ids[0]; 400 sizes[0] = getThreadAllocatedMemory0( 401 Thread.currentThread().getId() == id ? 0 : id); 402 } else { It seems cleaner to handle the 1-element array case at the beginning of this method: if (ids.length == 1) { long size = getThreadAllocatedBytes(ids[0]); return new long[] { size }; } I didn't review the hotspot implementation and the test. Mandy On 8/29/19 10:01 AM, Hohensee, Paul wrote: My bad, Mandy. The webrev puts getCurrentThreadAllocatedBytes in com.sun.management.ThreadMXBean along with the current two getThreadAllocatedBytes methods for the reasons you list. I?ve updated the CSR to specify com.sun.management and added a rationale. AllocatedBytes is currently enabled by Hotspot by default because the overhead of recording TLAB occupancy is negligible. There?s no new GC code, nor will there be, so imo we don?t have to involve the GC folks. I.e., the new JMM method GetOneThreadsAllocatedBytes uses the existing cooked_allocated_bytes JavaThread method, and getCurrentThreadAllocatedBytes is the same as getThreadAllocatedBytes: it just bypasses the thread lookup code. I hadn?t tracked down what happens when getCurrentThreadUserTime and getCurrentThreadCpuTime are called before, but if I?m not mistaken, it the code in jcmd() in attachListener.cpp will call GetThreadCpuTimeWithKind in management.cpp, and it will ultimately use Thread::current() as the subject of the call, see os::current_thread_cpu_time in os_linux.cpp. That means that the CurrentThread methods should work remotely the same way they do locally. GetOneThreadsAllocatedBytes in management.cpp uses THREAD as its subject when called on behalf of getCurrentThreadAllocatedBytes, so it will also uses the current remote Java thread. Even if these methods only worked locally, there are many setups where apps are self-monitoring that could use the performance improvement. Thanks, Paul From: Mandy Chung Date: Wednesday, August 28, 2019 at 3:59 PM To: "Hohensee, Paul" Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread Hi Paul, The CSR proposes this method in java.lang.management.ThreadMXBean as a Java SE feature. Has this been discussed with the GC team to commit measuring current thread's allocated bytes as Java SE feature? Can this be supported by all JVM implementation? What is the overhead if this is enabled by default? Does it need to be disabled? This metric is from TLAB that might be okay. This needs advice/discussion with GC experts. I see that CSR mentions it can be disabled and link to isThreadAllocatedMemoryEnabled() and setThreadAllocatedMemoryEnabled() methods but these methods are defined in com.sun.management.ThreadMXBean. As Alan points out, current thread makes sense only in local VM management. When this is monitored from a JMX client (e.g. jconsole to connect to a running JVM, "currentThreadAllowcatedBytes" attribute is the current thread in jconsole process which invoking Thread::currentThread? Mandy On 8/28/19 12:22 PM, Hohensee, Paul wrote: Please review a performance improvement for ThreadMXBean.getThreadAllocatedBytes and the addition of getCurrentThreadAllocatedBytes. JBS issue: https://bugs.openjdk.java.net/browse/JDK-8207266 Webrev: http://cr.openjdk.java.net/~phh/8207266/webrev.00/ CSR: https://bugs.openjdk.java.net/browse/JDK-8230311 Previous email threads: https://mail.openjdk.java.net/pipermail/serviceability-dev/2018-July/024441.html https://mail.openjdk.java.net/pipermail/serviceability-dev/2018-August/024763.html The CSR is for adding ThreadMXBean.getCurrentThreadAllocatedBytes. I?d be great for someone to review it. I took Mandy?s advice and put the fast paths in the library code. I added a new JMM method GetOneThreadsAllocatedBytes that works the same as GetThreadCpuTime: it uses a thread_id value of zero to distinguish the current thread. On my Mac laptop, the result runs 47x faster for the current thread than the old implementation. The 3 tests in test/jdk/com/sun/management/ThreadMXBean all pass. I added code to ThreadAllocatedMemory.java to test getCurrentThreadAllocatedBytes as well as variations on getThreadAllocatedBytes(id). A submit repo job is in progress. Thanks, Paul From sangheon.kim at oracle.com Wed Sep 4 07:15:53 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Wed, 4 Sep 2019 00:15:53 -0700 Subject: RFC / RFR: 8210473: JEP 345: NUMA-Aware Memory Allocation for G1 Message-ID: <4f92b0f2-45bb-8fc1-e1dd-26e35bdb9594@oracle.com> Hi all, Could I have some comments / reviews for these implementations for making G1 NUMA aware? 1. Concept The given Java Heap regions will be split and touched by memory touch threads to make the regions belong to active NUMA nodes. And then when mutators request memory, the mutators will get memory from local node. Not only mutators, but also survivors also will have same node memory during copying to survivor region state. For more details, please refer to the JEP 345: JDK-8210473 . FYI, if we enable -XX:+UseNUMA on current G1, OS will try to interleave memory, so we are already getting some benefit from it. I. e. G1 code is not NUMA aware but UseNUMAInterleaving is automatically enabled via UseNUMA. 2. Improvements - I compared "legacy +UseNUMA (without the patch)" with "new +UseNUMA (with patch)" and I could see some improvements like below. ?? a. Benchmark scores: max-JOPS on SpecJBB2015, score on SpecJBB2005 around 1 ~ 6%. SpecJVM2008: 1 ~ 5% ?? b. +UseNUMA shows better performance than -UseNUMA on above benchmarks. - Current scope (NUMA aware memory allocation) gives limited improvements but I expect following patches will make it better. 3. Known limitations - More GCs: current G1 is already wasting the last survivor region but the new implementation makes it worse as there are survivor region per NUMA node. JDK-8220089 is to address this problem and its working patch makes less frequent GC. - No pause time improvement: this is expected as worker threads are still may process foreign node allocated objects. And this is not scope of this JEP. 4. Follow-up enhancements -? JDK-8220089 G1 reuse survivor region as eden region at the end of gc: this enhancement minimizes wasting survivor region which the NUMA implementation makes this wasting situation worse. Initially I tried to push 8220089 before this JEP but there are still dependent / blocking issues at 8220089. Since all related CRs are targeting to JDK-14, I'm processing NUMA JEP first. - JDK-8230411 NUMA aware work gang: current memory touch threads will be replaced with NUMA aware work gang. - JDK-8230412 NUMA aware task processing: when gang workers do their works, threads may process foreign node allocated objects. This CR is to minimize this case. - Make G1 code more and more NUMA aware. For easier review, I separated into 3 pieces of patches. But I would like to push all together. 1/3: Mutator allocation 2/3: Survivor allocation 3/3: Stat / logging Thanks, Sangheon From sangheon.kim at oracle.com Wed Sep 4 07:16:07 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Wed, 4 Sep 2019 00:16:07 -0700 Subject: RFR(XL): 8220310: Implementation: NUMA-Aware Memory Allocation for G1, Mutator (1/3) Message-ID: Hi all, Please review this patch, making G1 NUMA aware. This is the first part of G1 NUMA implementation. - At G1 CollectedHeap initialization time, the given heap regions will be split and touched. - Internally node index(uint) is used instead of node id(int type) for easier access to arrays. - Only Linux is supported. - New logs are under gc+heap+numa - Current memory touch implementation will be replaced with WorkGang by JDK-8230411: NUMA aware work gang. CR: https://bugs.openjdk.java.net/browse/JDK-8220310 Webrev: http://cr.openjdk.java.net/~sangheki/8220310/webrev.0 Testing: hs-tier 1 ~ 5 with +- UseNUMA. Thanks, Sangheon From sangheon.kim at oracle.com Wed Sep 4 07:16:26 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Wed, 4 Sep 2019 00:16:26 -0700 Subject: RFR(M): 8220311: Implementation: NUMA-Aware Memory Allocation for G1, Survivor (2/3) Message-ID: Hi all, Please review this patch making G1 NUMA aware. This is the second part of G1 NUMA implementation: - Making Survivor region NUMA aware. CR: https://bugs.openjdk.java.net/browse/JDK-8220311 Webrev: http://cr.openjdk.java.net/~sangheki/8220311/webrev.0 Testing: hs-tier 1 ~ 5 with +- UseNUMA Thanks, Sangheon From sangheon.kim at oracle.com Wed Sep 4 07:16:35 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Wed, 4 Sep 2019 00:16:35 -0700 Subject: RFR(L): 8220312: Implementation: NUMA-Aware Memory Allocation for G1, Logging (3/3) Message-ID: Hi all, Please review this patch making G1 NUMA aware. This is the last part of G1 NUMA implementation: - Adding logs and stat. CR: https://bugs.openjdk.java.net/browse/JDK-8220312 Webrev: http://cr.openjdk.java.net/~sangheki/8220312/webrev.0 Testing: hs-tier 1 ~ 8 with +- UseNUMA Thanks, Sangheon From stefan.johansson at oracle.com Wed Sep 4 07:42:50 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 4 Sep 2019 09:42:50 +0200 Subject: RFR: 8209802: Garbage collectors should register JFR types themselves to avoid build errors. In-Reply-To: <4f436f8cd7f7476251216e1634ecc3d4470453f1.camel@oracle.com> References: <6C8A6F4E-E5BE-459F-944F-BD69F5CBEB59@oracle.com> <4f436f8cd7f7476251216e1634ecc3d4470453f1.camel@oracle.com> Message-ID: Thanks Kim and Thomas for the reviews, Stefan On 2019-09-03 15:48, Thomas Schatzl wrote: > Hi, > > On Mon, 2019-09-02 at 21:18 +0200, Stefan Johansson wrote: >> Hi, >> >> Please review this patch to fix JDK-8209802. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8209802 >> Webrev: http://cr.openjdk.java.net/~sjohanss/8209802/00/ >> >> Summary: >> This patch builds on the cleanup made in JDK-JDK-8230431, and adds an >> initialize method to G1NewTracer, that takes care of registering >> serializers needed for the two G1-specific constants in JFR. There is >> also a G1 periodic event but there is no similar way to register this >> from G1 code so it needs to be solved by guarding the G1 specific >> code in jfrPeriodic.cpp. >> >> Testing: >> Mach5 tier 1-3 >> > > looks good. > > Thomas > From per.liden at oracle.com Wed Sep 4 12:02:52 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 4 Sep 2019 14:02:52 +0200 Subject: RFR: 8230566: ZGC: Don't substitute klass pointer during array clearing Message-ID: <37bbb51a-40e4-8d22-0432-287ea2e2e4a6@oracle.com> When allocating large object arrays, ZObjArrayAllocator instantiates a non-cleared array of longs and later installs the correct klass pointer once the array has been cleared. While this might work it's also error prone. For example, there are at least two asserts() that will re-loaded the klass pointer and fail. It's also easy for someone in the future to make the innocent mistake of re-loading the klass pointer in some sensitive path, which would lead to new problems. We can avoid all these problems by not substituting with the klass pointer, and instead have another mechanism to tell GC marker threads to not follow the elements in not-yet-cleared arrays. This patch adds a new bit to ZMarkStackEntry, to signal to marking threads that the object array should not be followed, just mark it as live and visit its klass. The patch is large-ish because of the need to propagate the "follow" flag down through a few of layers. We might want to think about converting the flags (follow/finalizable/publish) into a single ZBarrierFlags-thing (perhaps similar to DecoratorSet) to reduce the noise and make it easier to add/adjust/remove flags in the future. But such an enhancements could come later. Bug: https://bugs.openjdk.java.net/browse/JDK-8230566 Webrev: http://cr.openjdk.java.net/~pliden/8230566/webrev.0 Testing: tier1-6 on linux + ad hoc runs of tests that provoked the problem /Per From erik.osterlund at oracle.com Wed Sep 4 15:12:31 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 4 Sep 2019 17:12:31 +0200 Subject: RFR: 8230566: ZGC: Don't substitute klass pointer during array clearing In-Reply-To: <37bbb51a-40e4-8d22-0432-287ea2e2e4a6@oracle.com> References: <37bbb51a-40e4-8d22-0432-287ea2e2e4a6@oracle.com> Message-ID: <5c80fbc8-1202-a4da-43ed-d82888b6d6af@oracle.com> Hi Per, In? src/hotspot/share/gc/z/zHeapIterator.cpp the only change made here is to mess up the indentation: 195 // Push roots to visit 196 push_roots(); 197 push_roots(); Should probably change that back. One possible solution to the problem of passing around all this context information in the templates, is to wrap them in a template config type, like this: template struct ZMarkConfig { static const bool _follow = follow; static const bool _finalizable = finalizable; static const bool _publish = publish; }; Then you can just pass around the ZMarkConfig in the templates all over the place, instead of the 3 (at this moment) bools. That way, adding and removing bools becomes easier in the future; no need to change the passing around code to pass more things around. Only the producer and consumer of the flags need changing. However, I'd propose we do that in a follow-up RFE so we can fix the bug faster. Having said that, looks good, and don't need a new webrev. Thanks, /Erik On 2019-09-04 14:02, Per Liden wrote: > When allocating large object arrays, ZObjArrayAllocator instantiates a > non-cleared array of longs and later installs the correct klass > pointer once the array has been cleared. While this might work it's > also error prone. For example, there are at least two asserts() that > will re-loaded the klass pointer and fail. It's also easy for someone > in the future to make the innocent mistake of re-loading the klass > pointer in some sensitive path, which would lead to new problems. We > can avoid all these problems by not substituting with the klass > pointer, and instead have another mechanism to tell GC marker threads > to not follow the elements in not-yet-cleared arrays. > > This patch adds a new bit to ZMarkStackEntry, to signal to marking > threads that the object array should not be followed, just mark it as > live and visit its klass. > > The patch is large-ish because of the need to propagate the "follow" > flag down through a few of layers. We might want to think about > converting the flags (follow/finalizable/publish) into a single > ZBarrierFlags-thing (perhaps similar to DecoratorSet) to reduce the > noise and make it easier to add/adjust/remove flags in the future. But > such an enhancements could come later. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8230566 > Webrev: http://cr.openjdk.java.net/~pliden/8230566/webrev.0 > > Testing: tier1-6 on linux + ad hoc runs of tests that provoked the > problem > > /Per From per.liden at oracle.com Wed Sep 4 17:15:20 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 4 Sep 2019 19:15:20 +0200 Subject: RFR: 8230566: ZGC: Don't substitute klass pointer during array clearing In-Reply-To: <5c80fbc8-1202-a4da-43ed-d82888b6d6af@oracle.com> References: <37bbb51a-40e4-8d22-0432-287ea2e2e4a6@oracle.com> <5c80fbc8-1202-a4da-43ed-d82888b6d6af@oracle.com> Message-ID: <9d9cfbd6-5409-226b-31c4-91ade0f3c054@oracle.com> Hi Erik, On 9/4/19 5:12 PM, Erik ?sterlund wrote: > Hi Per, > > In? src/hotspot/share/gc/z/zHeapIterator.cpp the only change made here > is to mess up the indentation: > > 195 // Push roots to visit > 196 push_roots(); > 197 push_roots(); > > Should probably change that back. Will fix! > > One possible solution to the problem of passing around all this context information in the templates, > is to wrap them in a template config type, like this: > > template > struct ZMarkConfig { > static const bool _follow = follow; > static const bool _finalizable = finalizable; > static const bool _publish = publish; > }; > > Then you can just pass around the ZMarkConfig in the templates all over the place, instead of the 3 (at this > moment) bools. That way, adding and removing bools becomes easier in the future; no need to change the passing > around code to pass more things around. Only the producer and consumer of the flags need changing. > > However, I'd propose we do that in a follow-up RFE so we can fix the bug faster. Having said that, looks good, > and don't need a new webrev. I'll prototype and see how it turns out. If it seems to be worth doing I'll file a separate RFE. Thanks for reviewing! cheers, Per > > Thanks, > /Erik > > On 2019-09-04 14:02, Per Liden wrote: >> When allocating large object arrays, ZObjArrayAllocator instantiates a >> non-cleared array of longs and later installs the correct klass >> pointer once the array has been cleared. While this might work it's >> also error prone. For example, there are at least two asserts() that >> will re-loaded the klass pointer and fail. It's also easy for someone >> in the future to make the innocent mistake of re-loading the klass >> pointer in some sensitive path, which would lead to new problems. We >> can avoid all these problems by not substituting with the klass >> pointer, and instead have another mechanism to tell GC marker threads >> to not follow the elements in not-yet-cleared arrays. >> >> This patch adds a new bit to ZMarkStackEntry, to signal to marking >> threads that the object array should not be followed, just mark it as >> live and visit its klass. >> >> The patch is large-ish because of the need to propagate the "follow" >> flag down through a few of layers. We might want to think about >> converting the flags (follow/finalizable/publish) into a single >> ZBarrierFlags-thing (perhaps similar to DecoratorSet) to reduce the >> noise and make it easier to add/adjust/remove flags in the future. But >> such an enhancements could come later. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8230566 >> Webrev: http://cr.openjdk.java.net/~pliden/8230566/webrev.0 >> >> Testing: tier1-6 on linux + ad hoc runs of tests that provoked the >> problem >> >> /Per > From per.liden at oracle.com Thu Sep 5 06:43:38 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 5 Sep 2019 08:43:38 +0200 Subject: RFR: 8219724: ZGC: Make inline cache cleaning more robust In-Reply-To: <29e3a713-113a-1486-bffc-c7c2e6a988f7@oracle.com> References: <29e3a713-113a-1486-bffc-c7c2e6a988f7@oracle.com> Message-ID: <6afeabee-0980-6f12-1e6d-2636150df602@oracle.com> Hi Erik, On 8/30/19 5:17 PM, Erik ?sterlund wrote: > Hi, > > Today, during the nmethod unlinking phase, the per-nmethod lock is held > across first an is_unloading() call on the nmethod and then inline cache > cleaning, which may take the nmethod locks of all nmethods referred to > from the inline caches. > If care is not taken, an nmethod A can have an inline cache pointing at > nmethod B, and B can have an inline cache pointing back at A. This could > potentially cause a deadlock. Today it is subtly safe, because between > calling is_unloading() and cleaning the inline caches, the nmethod entry > barrier is disarmed, which causes an mfence in the patching code. This > ensures that the racing threads do not enter a deadlock situation, > because they will observe the is_unloading state that was published as a > cache by the other thread in the race, causing the locks that would > cause the deadlock to not be taken. > > I would like to move the locks so that this becomes more robust, and > does not rely on the implicit fencing between is_unloading() and > cleaning the inline caches. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8219724 > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8219724/webrev.00/ Looks good, but please add an empty line between these: 302 ZLocker locker(ZNMethod::lock_for_nmethod(nm)); 303 // Heal oops and disarm cheers, Per From erik.osterlund at oracle.com Thu Sep 5 06:57:12 2019 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Thu, 5 Sep 2019 08:57:12 +0200 Subject: RFR: 8219724: ZGC: Make inline cache cleaning more robust In-Reply-To: <6afeabee-0980-6f12-1e6d-2636150df602@oracle.com> References: <29e3a713-113a-1486-bffc-c7c2e6a988f7@oracle.com> <6afeabee-0980-6f12-1e6d-2636150df602@oracle.com> Message-ID: Hi Per, Thanks for the review. Will add another newline before pushing. /Erik > On 5 Sep 2019, at 08:43, Per Liden wrote: > > Hi Erik, > >> On 8/30/19 5:17 PM, Erik ?sterlund wrote: >> Hi, >> Today, during the nmethod unlinking phase, the per-nmethod lock is held across first an is_unloading() call on the nmethod and then inline cache cleaning, which may take the nmethod locks of all nmethods referred to from the inline caches. >> If care is not taken, an nmethod A can have an inline cache pointing at nmethod B, and B can have an inline cache pointing back at A. This could potentially cause a deadlock. Today it is subtly safe, because between calling is_unloading() and cleaning the inline caches, the nmethod entry barrier is disarmed, which causes an mfence in the patching code. This ensures that the racing threads do not enter a deadlock situation, because they will observe the is_unloading state that was published as a cache by the other thread in the race, causing the locks that would cause the deadlock to not be taken. >> I would like to move the locks so that this becomes more robust, and does not rely on the implicit fencing between is_unloading() and cleaning the inline caches. >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8219724 >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8219724/webrev.00/ > > Looks good, but please add an empty line between these: > > 302 ZLocker locker(ZNMethod::lock_for_nmethod(nm)); > 303 // Heal oops and disarm > > cheers, > Per From thomas.stuefe at gmail.com Thu Sep 5 08:19:29 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 5 Sep 2019 10:19:29 +0200 Subject: RFR(T): 8230642: 8224815 broke Shenandoah build Message-ID: Hi, may I get a review for this trivial build error: https://bugs.openjdk.java.net/browse/JDK-8230642 http://cr.openjdk.java.net/~stuefe/webrevs/8224815-broke-shenandoah-build/8224815-broke-shenandoah-build.patch Thanks, Thomas From shade at redhat.com Thu Sep 5 08:57:03 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 5 Sep 2019 10:57:03 +0200 Subject: RFR(T): 8230642: 8224815 broke Shenandoah build In-Reply-To: References: Message-ID: <7f89f5f9-9858-548a-deac-b115fc7d15a0@redhat.com> On 9/5/19 10:19 AM, Thomas St?fe wrote: > https://bugs.openjdk.java.net/browse/JDK-8230642 > http://cr.openjdk.java.net/~stuefe/webrevs/8224815-broke-shenandoah-build/8224815-broke-shenandoah-build.patch Thanks for fixing this. The change looks good, but to match the style, I would go with: diff -r 66bb1335f8c5 src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp --- a/src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp Mon Sep 02 21:12:56 2019 +0200 +++ b/src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp Thu Sep 05 10:29:17 2019 +0200 @@ -212,10 +212,12 @@ size_t _num_regions; ShenandoahHeapRegion** _regions; ShenandoahRegionIterator _update_refs_iterator; public: + inline HeapWord* base() const { return _heap_region.start(); } + inline size_t num_regions() const { return _num_regions; } inline bool is_heap_region_special() { return _heap_region_special; } This passes hotspot_gc_shenandoah. -- Thanks, -Aleksey From martin.doerr at sap.com Thu Sep 5 09:08:38 2019 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 5 Sep 2019 09:08:38 +0000 Subject: RFR(T): 8230642: 8224815 broke Shenandoah build In-Reply-To: <7f89f5f9-9858-548a-deac-b115fc7d15a0@redhat.com> References: <7f89f5f9-9858-548a-deac-b115fc7d15a0@redhat.com> Message-ID: Hi Thomas, feel free to push Aleksey's version. Looks good and fixes the build. Thanks and best regards, Martin > -----Original Message----- > From: hotspot-gc-dev On > Behalf Of Aleksey Shipilev > Sent: Donnerstag, 5. September 2019 10:57 > To: Thomas St?fe ; Hotspot-Gc-Dev gc-dev at openjdk.java.net> > Subject: Re: RFR(T): 8230642: 8224815 broke Shenandoah build > > On 9/5/19 10:19 AM, Thomas St?fe wrote: > > https://bugs.openjdk.java.net/browse/JDK-8230642 > > http://cr.openjdk.java.net/~stuefe/webrevs/8224815-broke-shenandoah- > build/8224815-broke-shenandoah-build.patch > > Thanks for fixing this. > > The change looks good, but to match the style, I would go with: > > diff -r 66bb1335f8c5 > src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp > --- a/src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp Mon Sep > 02 21:12:56 2019 +0200 > +++ b/src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp Thu Sep > 05 10:29:17 2019 +0200 > @@ -212,10 +212,12 @@ > size_t _num_regions; > ShenandoahHeapRegion** _regions; > ShenandoahRegionIterator _update_refs_iterator; > > public: > + inline HeapWord* base() const { return _heap_region.start(); } > + > inline size_t num_regions() const { return _num_regions; } > inline bool is_heap_region_special() { return _heap_region_special; } > > This passes hotspot_gc_shenandoah. > > -- > Thanks, > -Aleksey From thomas.stuefe at gmail.com Thu Sep 5 09:17:20 2019 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 5 Sep 2019 11:17:20 +0200 Subject: RFR(T): 8230642: 8224815 broke Shenandoah build In-Reply-To: References: <7f89f5f9-9858-548a-deac-b115fc7d15a0@redhat.com> Message-ID: Okay thanks, pushed. ..Thomas On Thu, Sep 5, 2019 at 11:08 AM Doerr, Martin wrote: > Hi Thomas, > > feel free to push Aleksey's version. Looks good and fixes the build. > > Thanks and best regards, > Martin > > > > -----Original Message----- > > From: hotspot-gc-dev On > > Behalf Of Aleksey Shipilev > > Sent: Donnerstag, 5. September 2019 10:57 > > To: Thomas St?fe ; Hotspot-Gc-Dev > gc-dev at openjdk.java.net> > > Subject: Re: RFR(T): 8230642: 8224815 broke Shenandoah build > > > > On 9/5/19 10:19 AM, Thomas St?fe wrote: > > > https://bugs.openjdk.java.net/browse/JDK-8230642 > > > http://cr.openjdk.java.net/~stuefe/webrevs/8224815-broke-shenandoah- > > build/8224815-broke-shenandoah-build.patch > > > > Thanks for fixing this. > > > > The change looks good, but to match the style, I would go with: > > > > diff -r 66bb1335f8c5 > > src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp > > --- a/src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp Mon Sep > > 02 21:12:56 2019 +0200 > > +++ b/src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp Thu Sep > > 05 10:29:17 2019 +0200 > > @@ -212,10 +212,12 @@ > > size_t _num_regions; > > ShenandoahHeapRegion** _regions; > > ShenandoahRegionIterator _update_refs_iterator; > > > > public: > > + inline HeapWord* base() const { return _heap_region.start(); } > > + > > inline size_t num_regions() const { return _num_regions; } > > inline bool is_heap_region_special() { return _heap_region_special; } > > > > This passes hotspot_gc_shenandoah. > > > > -- > > Thanks, > > -Aleksey > > From shade at redhat.com Thu Sep 5 11:33:52 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 5 Sep 2019 13:33:52 +0200 Subject: RFR (XS) 8230646: Epsilon does not extend TLABs to max size Message-ID: <91226e24-4f8f-bbd4-9be7-c169ef6185c1@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8230646 I was playing with performance benchmarks and noticed that Epsilon does not extend issued TLABs to max tlab size, because of the simple overlook. TLAB allocation path asks GC about the max tlab size currently available with unsafe_max_tlab_alloc(). But, in Epsilon code, there is discrepancy: the caller interprets the returned value in bytes, while Epsilon code has the size in words. Every other GC adjusts accordingly, Epsilon should too. Fix: http://cr.openjdk.java.net/~shade/8230646/webrev.01/ Testing: gc/epsilon, jdk-submit -- Thanks, -Aleksey From thomas.schatzl at oracle.com Thu Sep 5 13:12:33 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 5 Sep 2019 15:12:33 +0200 Subject: RFR (XS) 8230646: Epsilon does not extend TLABs to max size In-Reply-To: <91226e24-4f8f-bbd4-9be7-c169ef6185c1@redhat.com> References: <91226e24-4f8f-bbd4-9be7-c169ef6185c1@redhat.com> Message-ID: <1eb95c4c-600b-4d02-dd10-d3305c595114@oracle.com> Hi, On 9/5/19 1:33 PM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8230646 > > I was playing with performance benchmarks and noticed that Epsilon does not extend issued TLABs to > max tlab size, because of the simple overlook. TLAB allocation path asks GC about the max tlab size > currently available with unsafe_max_tlab_alloc(). But, in Epsilon code, there is discrepancy: the > caller interprets the returned value in bytes, while Epsilon code has the size in words. Every other > GC adjusts accordingly, Epsilon should too. > > Fix: > http://cr.openjdk.java.net/~shade/8230646/webrev.01/ > > Testing: gc/epsilon, jdk-submit > looks good. Thomas From stefan.johansson at oracle.com Thu Sep 5 13:31:14 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 5 Sep 2019 15:31:14 +0200 Subject: RFR: 8230404: Refactor logged card refinement support in G1DirtyCardQueueSet In-Reply-To: <9794C377-E574-43FC-9AD1-6A4F5BA8EE00@oracle.com> References: <9794C377-E574-43FC-9AD1-6A4F5BA8EE00@oracle.com> Message-ID: Hi Kim, On 2019-08-31 08:09, Kim Barrett wrote: > Please review this refactoring of logged card refinement. > > We presently have two cases embedded in G1DirtyCardQueueSet. There is > support for both concurrent refinement and during GC (STW) refinement. > These cases share some code, but that code sharing causes more > problems than it solves. > > The yield to safepoint request handling in the concurrent case is > embedded in the closure and reported via the bool return value, making > for some awkwardness and ambiguity in the shared helper code as to > whether the current card was processed or not. > > STW refinement needs to go through the DCQS lock and deal with the > head/tail/count in the DCQS list representation. It performs a useless > stop_at check. And it needs to have a closure that returns bool but > must always return true. > > By separating these two cases we end up removing more code than we > add, and substantially simplify both. This also fixes a bug in > counting buffers processed concurrently: _processed_buffers_rs_thread > was being incremented for both concurrent refinement and STW > refinement, but being reported as being for concurrent refinement. > (The only use of that information currently is in logging.) This > might get a small performance benefit from STW refinement dealing with > a lock-free stack rather than dealing with the DCQS lock, but I didn't > try to measure that. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8230404 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8230404/open.00/ The change looks good, indeed easier to follow the code now. Had one thought around G1CardTableEntryClosure::apply_to_buffer, and if that should be in a separate closure that wraps the G1CardTableEntryClosure where needed. Not a big thing and if you prefer it to keep the code as is I don't mind. Thanks, Stefan > > Testing: > mach5 tier1-5 > From per.liden at oracle.com Thu Sep 5 13:51:32 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 5 Sep 2019 15:51:32 +0200 Subject: RFR: 8224820: ZGC: Support discontiguous heap reservations In-Reply-To: <5567c0b8-00e6-29f0-12c6-e067c924fdce@oracle.com> References: <5567c0b8-00e6-29f0-12c6-e067c924fdce@oracle.com> Message-ID: <8767693d-13c1-e6c8-4114-f2d73775e8e6@oracle.com> On 8/9/19 11:59 AM, Erik ?sterlund wrote: > Hi, > > Today ZGC reserves a huge chunk of virtual address space when the JVM > starts. This typically succeeds because we grab the VA before anyone > else has time to do so. But if some ASLR library or something was to > grab a tiny part of the desired VA, ZGC can't start. We should support > discontiguous heap reservations to support this. > > On linux, by default, this does not happen. But on OS X, it does happen > relatively frequently. So we need to fix this to allow a mac port. > > This patch implements a recursive algorithm for finding holes at 2MB > granularities in the normally contiguous reservation when initializing > the heap, removing them from our VA. > > This patch depends on 8224815, which depends on 8229189 and 8229278. > They are all out for review. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8224820/webrev.00/ Looks good. However, please pass down max_capacity from ZPageAllocator to ZVirtualMemory's constructor, instead of reading MaxHeapSize. cheers, Per > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8224820 > > Thanks, > /Erik From zgu at redhat.com Thu Sep 5 13:52:32 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 5 Sep 2019 09:52:32 -0400 Subject: RFR (XS) 8230646: Epsilon does not extend TLABs to max size In-Reply-To: <91226e24-4f8f-bbd4-9be7-c169ef6185c1@redhat.com> References: <91226e24-4f8f-bbd4-9be7-c169ef6185c1@redhat.com> Message-ID: <31ab2eae-dec7-cb19-e097-5d0562108c46@redhat.com> Looks good to me. -Zhengyu On 9/5/19 7:33 AM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8230646 > > I was playing with performance benchmarks and noticed that Epsilon does not extend issued TLABs to > max tlab size, because of the simple overlook. TLAB allocation path asks GC about the max tlab size > currently available with unsafe_max_tlab_alloc(). But, in Epsilon code, there is discrepancy: the > caller interprets the returned value in bytes, while Epsilon code has the size in words. Every other > GC adjusts accordingly, Epsilon should too. > > Fix: > http://cr.openjdk.java.net/~shade/8230646/webrev.01/ > > Testing: gc/epsilon, jdk-submit > From erik.osterlund at oracle.com Thu Sep 5 14:02:13 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 5 Sep 2019 16:02:13 +0200 Subject: RFR: 8224820: ZGC: Support discontiguous heap reservations In-Reply-To: <8767693d-13c1-e6c8-4114-f2d73775e8e6@oracle.com> References: <5567c0b8-00e6-29f0-12c6-e067c924fdce@oracle.com> <8767693d-13c1-e6c8-4114-f2d73775e8e6@oracle.com> Message-ID: <253106a9-dd8d-7c67-20cc-c1f3d2cad834@oracle.com> Hi Per, Thanks for the review. Will poke in the argument before pushing. /Erik On 2019-09-05 15:51, Per Liden wrote: > On 8/9/19 11:59 AM, Erik ?sterlund wrote: >> Hi, >> >> Today ZGC reserves a huge chunk of virtual address space when the JVM >> starts. This typically succeeds because we grab the VA before anyone >> else has time to do so. But if some ASLR library or something was to >> grab a tiny part of the desired VA, ZGC can't start. We should >> support discontiguous heap reservations to support this. >> >> On linux, by default, this does not happen. But on OS X, it does >> happen relatively frequently. So we need to fix this to allow a mac >> port. >> >> This patch implements a recursive algorithm for finding holes at 2MB >> granularities in the normally contiguous reservation when >> initializing the heap, removing them from our VA. >> >> This patch depends on 8224815, which depends on 8229189 and 8229278. >> They are all out for review. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8224820/webrev.00/ > > Looks good. However, please pass down max_capacity from ZPageAllocator > to ZVirtualMemory's constructor, instead of reading MaxHeapSize. > > cheers, > Per > >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8224820 >> >> Thanks, >> /Erik From kim.barrett at oracle.com Thu Sep 5 19:14:28 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 5 Sep 2019 15:14:28 -0400 Subject: RFR: 8230404: Refactor logged card refinement support in G1DirtyCardQueueSet In-Reply-To: References: <9794C377-E574-43FC-9AD1-6A4F5BA8EE00@oracle.com> Message-ID: > On Sep 5, 2019, at 9:31 AM, Stefan Johansson wrote: > > Hi Kim, > > On 2019-08-31 08:09, Kim Barrett wrote: >> [?] >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8230404 >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8230404/open.00/ > > The change looks good, indeed easier to follow the code now. Thanks. > Had one thought around G1CardTableEntryClosure::apply_to_buffer, and if that should be in a separate closure that wraps the G1CardTableEntryClosure where needed. Not a big thing and if you prefer it to keep the code as is I don't mind. I took a look at this idea. It didn't seem to provide any benefit; it increased source code size and required additional indirections. So I'm leaving it as is. > Thanks, > Stefan > >> Testing: >> mach5 tier1-5 From shade at redhat.com Thu Sep 5 19:17:18 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 5 Sep 2019 21:17:18 +0200 Subject: RFR (XS) 8230646: Epsilon does not extend TLABs to max size In-Reply-To: <31ab2eae-dec7-cb19-e097-5d0562108c46@redhat.com> References: <91226e24-4f8f-bbd4-9be7-c169ef6185c1@redhat.com> <31ab2eae-dec7-cb19-e097-5d0562108c46@redhat.com> Message-ID: <5c9bf099-eb9e-17fe-a020-99aeda088477@redhat.com> Thanks, pushed. On 9/5/19 3:52 PM, Zhengyu Gu wrote: > Looks good to me. > > -Zhengyu > > On 9/5/19 7:33 AM, Aleksey Shipilev wrote: >> Bug: >> ?? https://bugs.openjdk.java.net/browse/JDK-8230646 >> >> I was playing with performance benchmarks and noticed that Epsilon does not extend issued TLABs to >> max tlab size, because of the simple overlook. TLAB allocation path asks GC about the max tlab size >> currently available with unsafe_max_tlab_alloc(). But, in Epsilon code, there is discrepancy: the >> caller interprets the returned value in bytes, while Epsilon code has the size in words. Every other >> GC adjusts accordingly, Epsilon should too. >> >> Fix: >> ?? http://cr.openjdk.java.net/~shade/8230646/webrev.01/ >> >> Testing: gc/epsilon, jdk-submit From stefan.johansson at oracle.com Thu Sep 5 19:46:07 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 5 Sep 2019 21:46:07 +0200 Subject: RFR: 8230404: Refactor logged card refinement support in G1DirtyCardQueueSet In-Reply-To: References: <9794C377-E574-43FC-9AD1-6A4F5BA8EE00@oracle.com> Message-ID: <42A25F94-0B62-41DE-9170-ED2BFF338633@oracle.com> > 5 sep. 2019 kl. 21:14 skrev Kim Barrett : > >> On Sep 5, 2019, at 9:31 AM, Stefan Johansson wrote: >> >> Hi Kim, >> >> On 2019-08-31 08:09, Kim Barrett wrote: >>> [?] >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8230404 >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8230404/open.00/ >> >> The change looks good, indeed easier to follow the code now. > > Thanks. > >> Had one thought around G1CardTableEntryClosure::apply_to_buffer, and if that should be in a separate closure that wraps the G1CardTableEntryClosure where needed. Not a big thing and if you prefer it to keep the code as is I don't mind. > > I took a look at this idea. It didn't seem to provide any benefit; it > increased source code size and required additional indirections. So > I'm leaving it as is. > Sounds good, Stefan >> Thanks, >> Stefan >> >>> Testing: >>> mach5 tier1-5 > > From thomas.schatzl at oracle.com Fri Sep 6 11:23:30 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 06 Sep 2019 13:23:30 +0200 Subject: RFR: 8230404: Refactor logged card refinement support in G1DirtyCardQueueSet In-Reply-To: References: <9794C377-E574-43FC-9AD1-6A4F5BA8EE00@oracle.com> Message-ID: <8b470eaec7acc8b0b203d1143a29d6828796af90.camel@oracle.com> Hi, On Thu, 2019-09-05 at 15:14 -0400, Kim Barrett wrote: > > On Sep 5, 2019, at 9:31 AM, Stefan Johansson < > > stefan.johansson at oracle.com> wrote: > > > > Hi Kim, > > > > On 2019-08-31 08:09, Kim Barrett wrote: > > > [?] > > > > > > CR: > > > https://bugs.openjdk.java.net/browse/JDK-8230404 > > > Webrev: > > > http://cr.openjdk.java.net/~kbarrett/8230404/open.00/ > > > > The change looks good, indeed easier to follow the code now. > Looks good. Thomas From stefan.karlsson at oracle.com Fri Sep 6 13:51:42 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 6 Sep 2019 15:51:42 +0200 Subject: RFR: 8230566: ZGC: Don't substitute klass pointer during array clearing In-Reply-To: <9d9cfbd6-5409-226b-31c4-91ade0f3c054@oracle.com> References: <37bbb51a-40e4-8d22-0432-287ea2e2e4a6@oracle.com> <5c80fbc8-1202-a4da-43ed-d82888b6d6af@oracle.com> <9d9cfbd6-5409-226b-31c4-91ade0f3c054@oracle.com> Message-ID: Talked to Per and Erik offline about follow_array_object. We should be able to skip following the klass pointer when follow is false. Just like objects in "allocating" regions don't get their klass pointer followed. Other than that, this looks good to me. Thanks, StefanK On 2019-09-04 19:15, Per Liden wrote: > Hi Erik, > > On 9/4/19 5:12 PM, Erik ?sterlund wrote: >> Hi Per, >> >> In? src/hotspot/share/gc/z/zHeapIterator.cpp the only change made here >> is to mess up the indentation: >> >> ? 195?? // Push roots to visit >> ? 196?? push_roots> Concurrent */, false /* Weak */>(); >> ? 197?? push_roots> Concurrent */, false /* Weak */>(); >> >> Should probably change that back. > > Will fix! > >> >> One possible solution to the problem of passing around all this >> context information in the templates, >> is to wrap them in a template config type, like this: >> >> template >> struct ZMarkConfig { >> ?? static const bool _follow = follow; >> ?? static const bool _finalizable = finalizable; >> ?? static const bool _publish? = publish; >> }; >> >> Then you can just pass around the ZMarkConfig in the templates all >> over the place, instead of the 3 (at this >> moment) bools. That way, adding and removing bools becomes easier in >> the future; no need to change the passing >> around code to pass more things around. Only the producer and consumer >> of the flags need changing. >> >> However, I'd propose we do that in a follow-up RFE so we can fix the >> bug faster. Having said that, looks good, >> and don't need a new webrev. > > I'll prototype and see how it turns out. If it seems to be worth doing > I'll file a separate RFE. > > Thanks for reviewing! > > cheers, > Per > >> >> Thanks, >> /Erik >> >> On 2019-09-04 14:02, Per Liden wrote: >>> When allocating large object arrays, ZObjArrayAllocator instantiates >>> a non-cleared array of longs and later installs the correct klass >>> pointer once the array has been cleared. While this might work it's >>> also error prone. For example, there are at least two asserts() that >>> will re-loaded the klass pointer and fail. It's also easy for someone >>> in the future to make the innocent mistake of re-loading the klass >>> pointer in some sensitive path, which would lead to new problems. We >>> can avoid all these problems by not substituting with the klass >>> pointer, and instead have another mechanism to tell GC marker threads >>> to not follow the elements in not-yet-cleared arrays. >>> >>> This patch adds a new bit to ZMarkStackEntry, to signal to marking >>> threads that the object array should not be followed, just mark it as >>> live and visit its klass. >>> >>> The patch is large-ish because of the need to propagate the "follow" >>> flag down through a few of layers. We might want to think about >>> converting the flags (follow/finalizable/publish) into a single >>> ZBarrierFlags-thing (perhaps similar to DecoratorSet) to reduce the >>> noise and make it easier to add/adjust/remove flags in the future. >>> But such an enhancements could come later. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8230566 >>> Webrev: http://cr.openjdk.java.net/~pliden/8230566/webrev.0 >>> >>> Testing: tier1-6 on linux + ad hoc runs of tests that provoked the >>> problem >>> >>> /Per >> From kim.barrett at oracle.com Fri Sep 6 17:00:33 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 6 Sep 2019 13:00:33 -0400 Subject: RFR: 8230404: Refactor logged card refinement support in G1DirtyCardQueueSet In-Reply-To: <8b470eaec7acc8b0b203d1143a29d6828796af90.camel@oracle.com> References: <9794C377-E574-43FC-9AD1-6A4F5BA8EE00@oracle.com> <8b470eaec7acc8b0b203d1143a29d6828796af90.camel@oracle.com> Message-ID: > On Sep 6, 2019, at 7:23 AM, Thomas Schatzl wrote: > > Hi, > > On Thu, 2019-09-05 at 15:14 -0400, Kim Barrett wrote: >>> On Sep 5, 2019, at 9:31 AM, Stefan Johansson < >>> stefan.johansson at oracle.com> wrote: >>> >>> Hi Kim, >>> >>> On 2019-08-31 08:09, Kim Barrett wrote: >>>> [?] >>>> >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8230404 >>>> Webrev: >>>> http://cr.openjdk.java.net/~kbarrett/8230404/open.00/ >>> >>> The change looks good, indeed easier to follow the code now. >> > > Looks good. > > Thomas Thanks. From hohensee at amazon.com Fri Sep 6 18:08:06 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 6 Sep 2019 18:08:06 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> Message-ID: <23ED3BD7-3B7E-43DE-84B8-DB52D44D362B@amazon.com> Ping. Anyone? ( Thanks, ?On 9/3/19, 12:39 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: Minor update in new webrev http://cr.openjdk.java.net/~phh/8207266/webrev.05/. I removed ensureNonNullThreadIds() in favor of Objects.requireNonNull(ids). Thanks, Mandy, for your through reviews. May I get another reviewer to weigh in? Paul On 8/31/19, 5:06 PM, "hotspot-gc-dev on behalf of Hohensee, Paul" wrote: Thanks, Mandy. I?ve finalized the CSR. New webrev at http://cr.openjdk.java.net/~phh/8207266/webrev.04/. In management.cpp, I now have if (THREAD->is_Java_thread()) { return ((JavaThread*)THREAD)->cooked_allocated_bytes(); } In ThreadImpl.java, using requireNonNull would produce a different and less informative message, so I?d like to leave it as is. I changed throwIfNullThreadIds to ensureNonNullThreadIds, and throwIfThreadAllocatedMemoryNotSupported to ensureThreadAllocatedMemorySupported. I dropped the ?java.lang.? prefix from all uses of UnsupportedOperationException in both c.s.m.ThreadMXBean.java and j.l.m.ThreadMXBean.java, and did the same with SecurityException. ?@since 14? added to c.s.m.ThreadMXBean.java and the CSR. Do I need another reviewer? Paul From: Mandy Chung Date: Friday, August 30, 2019 at 4:26 PM To: "Hohensee, Paul" Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread CSR reviewed. management.cpp 2083 java_thread = (JavaThread*)THREAD; 2084 if (java_thread->is_Java_thread()) { 2085 return java_thread->cooked_allocated_bytes(); 2086 } The cast should be done after is_Java_thread() test. ThreadImpl.java 162 private void throwIfNullThreadIds(long[] ids) { Even better: simply use Objects::requiresNonNull and this method can be removed. This suggests positive naming alternative to throwIfThreadAllocatedMemoryNotSupported - "ensureThreadAllocatedMemorySupported" (sorry I should have suggested that) ThreadMXBean.java 130 * @throws java.lang.UnsupportedOperationException if the Java virtual Nit: "java.lang." can be dropped. @since 14 is missing. Mandy On 8/30/19 3:33 PM, Hohensee, Paul wrote: Thanks for your review, Mandy. Revised webrev at http://cr.openjdk.java.net/~phh/8207266/webrev.02/. I updated the CSR with your suggested javadoc for getCurrentThreadAllocatedBytes. It now matches that for getCurrentThreadUserTime and getCurrentThreadCputime. I also fixed the ?convenient? -> ?convenience? typos in j.l.m.ThreadMXBean.java. I meant GetOneThreads to be the possessive, but don?t feel strongly either way so I?m fine with GetOneThread. I updated ThreadImpl.java as you suggested, though in getThreadAllocatedBytes(long[] ids) I had to add a redundant-in-the-not-length-1-case check for a null ids reference. Would someone take a look at the Hotspot side and the test please? Paul From: Mandy Chung Date: Friday, August 30, 2019 at 10:22 AM To: "Hohensee, Paul" Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread OK. That's better. Some review comments: The javadoc of getCurrentThreadAllocatedBytes() can simply say: "Returns an approximation of the total amount of memory, in bytes, allocated in heap memory for the current thread. This is a convenient method for local management use and is equivalent to calling getThreadAllocatedBytes(Thread.currentThread().getId()). src/hotspot/share/include/jmm.h GetOneThreadsAllocatedMemory: s/OneThreads/OneThread/ sun/management/ThreadImpl.java 43 private static final String THREAD_ALLOCATED_MEMORY_NOT_SUPPORTED = 44 "Thread allocated memory measurement is not supported."; if (!isThreadAllocatedMemorySupported()) { throw new UnsupportedOperationException(THREAD_ALLOCATED_MEMORY_NOT_SUPPORTED); } Perhaps the above can be refactored as throwIfAllocatedMemoryUnsupported() method. 391 if (ids.length == 1) { 392 sizes[0] = -1; : 398 if (ids.length == 1) { 399 long id = ids[0]; 400 sizes[0] = getThreadAllocatedMemory0( 401 Thread.currentThread().getId() == id ? 0 : id); 402 } else { It seems cleaner to handle the 1-element array case at the beginning of this method: if (ids.length == 1) { long size = getThreadAllocatedBytes(ids[0]); return new long[] { size }; } I didn't review the hotspot implementation and the test. Mandy On 8/29/19 10:01 AM, Hohensee, Paul wrote: My bad, Mandy. The webrev puts getCurrentThreadAllocatedBytes in com.sun.management.ThreadMXBean along with the current two getThreadAllocatedBytes methods for the reasons you list. I?ve updated the CSR to specify com.sun.management and added a rationale. AllocatedBytes is currently enabled by Hotspot by default because the overhead of recording TLAB occupancy is negligible. There?s no new GC code, nor will there be, so imo we don?t have to involve the GC folks. I.e., the new JMM method GetOneThreadsAllocatedBytes uses the existing cooked_allocated_bytes JavaThread method, and getCurrentThreadAllocatedBytes is the same as getThreadAllocatedBytes: it just bypasses the thread lookup code. I hadn?t tracked down what happens when getCurrentThreadUserTime and getCurrentThreadCpuTime are called before, but if I?m not mistaken, it the code in jcmd() in attachListener.cpp will call GetThreadCpuTimeWithKind in management.cpp, and it will ultimately use Thread::current() as the subject of the call, see os::current_thread_cpu_time in os_linux.cpp. That means that the CurrentThread methods should work remotely the same way they do locally. GetOneThreadsAllocatedBytes in management.cpp uses THREAD as its subject when called on behalf of getCurrentThreadAllocatedBytes, so it will also uses the current remote Java thread. Even if these methods only worked locally, there are many setups where apps are self-monitoring that could use the performance improvement. Thanks, Paul From: Mandy Chung Date: Wednesday, August 28, 2019 at 3:59 PM To: "Hohensee, Paul" Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread Hi Paul, The CSR proposes this method in java.lang.management.ThreadMXBean as a Java SE feature. Has this been discussed with the GC team to commit measuring current thread's allocated bytes as Java SE feature? Can this be supported by all JVM implementation? What is the overhead if this is enabled by default? Does it need to be disabled? This metric is from TLAB that might be okay. This needs advice/discussion with GC experts. I see that CSR mentions it can be disabled and link to isThreadAllocatedMemoryEnabled() and setThreadAllocatedMemoryEnabled() methods but these methods are defined in com.sun.management.ThreadMXBean. As Alan points out, current thread makes sense only in local VM management. When this is monitored from a JMX client (e.g. jconsole to connect to a running JVM, "currentThreadAllowcatedBytes" attribute is the current thread in jconsole process which invoking Thread::currentThread? Mandy On 8/28/19 12:22 PM, Hohensee, Paul wrote: Please review a performance improvement for ThreadMXBean.getThreadAllocatedBytes and the addition of getCurrentThreadAllocatedBytes. JBS issue: https://bugs.openjdk.java.net/browse/JDK-8207266 Webrev: http://cr.openjdk.java.net/~phh/8207266/webrev.00/ CSR: https://bugs.openjdk.java.net/browse/JDK-8230311 Previous email threads: https://mail.openjdk.java.net/pipermail/serviceability-dev/2018-July/024441.html https://mail.openjdk.java.net/pipermail/serviceability-dev/2018-August/024763.html The CSR is for adding ThreadMXBean.getCurrentThreadAllocatedBytes. I?d be great for someone to review it. I took Mandy?s advice and put the fast paths in the library code. I added a new JMM method GetOneThreadsAllocatedBytes that works the same as GetThreadCpuTime: it uses a thread_id value of zero to distinguish the current thread. On my Mac laptop, the result runs 47x faster for the current thread than the old implementation. The 3 tests in test/jdk/com/sun/management/ThreadMXBean all pass. I added code to ThreadAllocatedMemory.java to test getCurrentThreadAllocatedBytes as well as variations on getThreadAllocatedBytes(id). A submit repo job is in progress. Thanks, Paul From kim.barrett at oracle.com Fri Sep 6 21:25:24 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 6 Sep 2019 17:25:24 -0400 Subject: RFR: 8221361: Eliminate two-phase initialization for PtrQueueSet classes Message-ID: <801F537C-948E-4460-B353-E55898AAD2FC@oracle.com> Please review this refactoring of PtrQueueSet to eliminate two-phase construction. The objects are now fully constructed by the constructor and there is no longer a need for an initialize() function. In particular, the allocator is now a constructor argument. And in the case of G1DirtyCardQueueSet, the CBL monitor is also a constructor argument. This means that PtrQueueSet objects are always fully initialized. As a consequence, we can eliminate the bootstrapping kludge in PtrQueue for initializing the _capacity_in_bytes member. This change required a little bit of refactoring for Shenandoah. The SATB buffer allocator has been moved from the SATB pqset to the barrier set. When constructing the barrier set, the allocator is constructed first and then passed as an argument to the pqset constructor. This is all consistent with G1's arrangement. Someone from the Shenandoah team should check this. CR: https://bugs.openjdk.java.net/browse/JDK-8221361 Webrev: http://cr.openjdk.java.net/~kbarrett/8221361/open.00/ Testing: mach5 tier1. Local (linux-x64) build with Shenandoah and ran hotspot:tier1 tests with -XX:+UseShenandoahGC, ignoring failures for tests that don't support Shenandoah (anything involving graal or aot, for example). From shade at redhat.com Fri Sep 6 21:53:26 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 6 Sep 2019 23:53:26 +0200 Subject: RFR: 8221361: Eliminate two-phase initialization for PtrQueueSet classes In-Reply-To: <801F537C-948E-4460-B353-E55898AAD2FC@oracle.com> References: <801F537C-948E-4460-B353-E55898AAD2FC@oracle.com> Message-ID: On 9/6/19 11:25 PM, Kim Barrett wrote: > Webrev: > http://cr.openjdk.java.net/~kbarrett/8221361/open.00/ Shenandoah parts look good. Thank you for going extra mile there. It is a bit unfortunate we have lost the ability to use cached _heap in ShenandoahSATBMarkQueueSet::filter, but this should not be important. It passes hotspot_gc_shenandoah for me locally. -- Thanks, -Aleksey From kim.barrett at oracle.com Fri Sep 6 23:13:16 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 6 Sep 2019 19:13:16 -0400 Subject: RFR: 8221361: Eliminate two-phase initialization for PtrQueueSet classes In-Reply-To: References: <801F537C-948E-4460-B353-E55898AAD2FC@oracle.com> Message-ID: <9BB58A52-C880-4905-9996-1A2B7033D289@oracle.com> > On Sep 6, 2019, at 5:53 PM, Aleksey Shipilev wrote: > > On 9/6/19 11:25 PM, Kim Barrett wrote: >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8221361/open.00/ > > Shenandoah parts look good. Thanks. > Thank you for going extra mile there. It is a bit unfortunate we have > lost the ability to use cached _heap in ShenandoahSATBMarkQueueSet::filter, but this should not be > important. Same thing happend for G1. I thought about having a lazily initialized cached _heap, but decided that wouldn?t really be any better. We could have it be late assigned, breaking the ?fully initialized at construction time? invariant, but I also don?t think it should be important. We could also move {G1CollectedHeap, ShenandoahHeap}::heap() to their respective .inline.hpp files... I also note that it used to work this way (for G1) back before we restructured the SATB buffer filtering into its current form for sharing between G1 and Shenandoah. That is, G1?s SATBMarkQueue::filter obtained the g1h by calling G1CollectedHeap::heap(). From zgu at redhat.com Sat Sep 7 02:45:25 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 6 Sep 2019 22:45:25 -0400 Subject: [14] RFR(T) 8230730: UseCompressedOops test crash with assertion failure Message-ID: <3a909b83-e616-ad77-9555-3c8dfca59194@redhat.com> JDK-8224815 changed the assertion from: assert((intptr_t)base() <= (intptr_t)(Universe::heap()->base() - os::vm_page_size()) || base() == NULL, "invalid value"); to assert((intptr_t)base() <= (intptr_t)(_heap_address_range.start() - os::vm_page_size()) || base() == NULL, "invalid value"); while Universe::heap()->base() returns "address" type, _heap_address_range.start() returns "HeapWord*" type, therefore, new assertion subtracts vm_page_size * sizeof(HeapWord*) from base address, that triggers assertion failure. This bug is only exposed when JVM can not obtain zero-based heap. Bug: https://bugs.openjdk.java.net/browse/JDK-8230730 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8230730/webrev.00/ Test: hotspot_gc (fastdebug) on Linux x86_64 and aarch64. Submit test in progress. Thanks, -Zhengyu From stefan.karlsson at oracle.com Sat Sep 7 05:44:16 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Sat, 7 Sep 2019 07:44:16 +0200 Subject: [14] RFR(T) 8230730: UseCompressedOops test crash with assertion failure In-Reply-To: <3a909b83-e616-ad77-9555-3c8dfca59194@redhat.com> References: <3a909b83-e616-ad77-9555-3c8dfca59194@redhat.com> Message-ID: Looks good. Thanks, StefanK On 2019-09-07 04:45, Zhengyu Gu wrote: > JDK-8224815 changed the assertion from: > > assert((intptr_t)base() <= (intptr_t)(Universe::heap()->base() - > os::vm_page_size()) || base() == NULL, "invalid value"); > > to > > assert((intptr_t)base() <= (intptr_t)(_heap_address_range.start() - > os::vm_page_size()) || base() == NULL, "invalid value"); > > while Universe::heap()->base() returns "address" type, > _heap_address_range.start() returns "HeapWord*" type, therefore, new > assertion subtracts vm_page_size * sizeof(HeapWord*) from base > address, that triggers assertion failure. > > This bug is only exposed when JVM can not obtain zero-based heap. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8230730 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8230730/webrev.00/ > > Test: > ? hotspot_gc (fastdebug) on Linux x86_64 and aarch64. > ? Submit test in progress. > > Thanks, > > -Zhengyu From erik.osterlund at oracle.com Sat Sep 7 07:27:16 2019 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Sat, 7 Sep 2019 09:27:16 +0200 Subject: [14] RFR(T) 8230730: UseCompressedOops test crash with assertion failure In-Reply-To: <3a909b83-e616-ad77-9555-3c8dfca59194@redhat.com> References: <3a909b83-e616-ad77-9555-3c8dfca59194@redhat.com> Message-ID: <2A210FE4-C0BF-46EE-8FF1-DD1E33BC55D0@oracle.com> Hi Zhengyu, Looks good. Thanks, /Erik > On 7 Sep 2019, at 04:45, Zhengyu Gu wrote: > > JDK-8224815 changed the assertion from: > > assert((intptr_t)base() <= (intptr_t)(Universe::heap()->base() - os::vm_page_size()) || base() == NULL, "invalid value"); > > to > > assert((intptr_t)base() <= (intptr_t)(_heap_address_range.start() - os::vm_page_size()) || base() == NULL, "invalid value"); > > while Universe::heap()->base() returns "address" type, _heap_address_range.start() returns "HeapWord*" type, therefore, new assertion subtracts vm_page_size * sizeof(HeapWord*) from base address, that triggers assertion failure. > > This bug is only exposed when JVM can not obtain zero-based heap. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8230730 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8230730/webrev.00/ > > Test: > hotspot_gc (fastdebug) on Linux x86_64 and aarch64. > Submit test in progress. > > Thanks, > > -Zhengyu From zgu at redhat.com Sat Sep 7 11:21:03 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Sat, 7 Sep 2019 07:21:03 -0400 Subject: [14] RFR(T) 8230730: UseCompressedOops test crash with assertion failure In-Reply-To: References: <3a909b83-e616-ad77-9555-3c8dfca59194@redhat.com> Message-ID: Thanks, Stefan. -Zhengyu On 9/7/19 1:44 AM, Stefan Karlsson wrote: > Looks good. > > Thanks, > StefanK > > On 2019-09-07 04:45, Zhengyu Gu wrote: >> JDK-8224815 changed the assertion from: >> >> assert((intptr_t)base() <= (intptr_t)(Universe::heap()->base() - >> os::vm_page_size()) || base() == NULL, "invalid value"); >> >> to >> >> assert((intptr_t)base() <= (intptr_t)(_heap_address_range.start() - >> os::vm_page_size()) || base() == NULL, "invalid value"); >> >> while Universe::heap()->base() returns "address" type, >> _heap_address_range.start() returns "HeapWord*" type, therefore, new >> assertion subtracts vm_page_size * sizeof(HeapWord*) from base >> address, that triggers assertion failure. >> >> This bug is only exposed when JVM can not obtain zero-based heap. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8230730 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8230730/webrev.00/ >> >> Test: >> ? hotspot_gc (fastdebug) on Linux x86_64 and aarch64. >> ? Submit test in progress. >> >> Thanks, >> >> -Zhengyu > From zgu at redhat.com Sat Sep 7 11:21:18 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Sat, 7 Sep 2019 07:21:18 -0400 Subject: [14] RFR(T) 8230730: UseCompressedOops test crash with assertion failure In-Reply-To: <2A210FE4-C0BF-46EE-8FF1-DD1E33BC55D0@oracle.com> References: <3a909b83-e616-ad77-9555-3c8dfca59194@redhat.com> <2A210FE4-C0BF-46EE-8FF1-DD1E33BC55D0@oracle.com> Message-ID: <1ccf4663-b7d2-e2af-f1d2-3d0dc5998a42@redhat.com> Thanks, Erik. -Zhengyu On 9/7/19 3:27 AM, Erik Osterlund wrote: > Hi Zhengyu, > > Looks good. > > Thanks, > /Erik > >> On 7 Sep 2019, at 04:45, Zhengyu Gu wrote: >> >> JDK-8224815 changed the assertion from: >> >> assert((intptr_t)base() <= (intptr_t)(Universe::heap()->base() - os::vm_page_size()) || base() == NULL, "invalid value"); >> >> to >> >> assert((intptr_t)base() <= (intptr_t)(_heap_address_range.start() - os::vm_page_size()) || base() == NULL, "invalid value"); >> >> while Universe::heap()->base() returns "address" type, _heap_address_range.start() returns "HeapWord*" type, therefore, new assertion subtracts vm_page_size * sizeof(HeapWord*) from base address, that triggers assertion failure. >> >> This bug is only exposed when JVM can not obtain zero-based heap. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8230730 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8230730/webrev.00/ >> >> Test: >> hotspot_gc (fastdebug) on Linux x86_64 and aarch64. >> Submit test in progress. >> >> Thanks, >> >> -Zhengyu > From mandrikov at gmail.com Sat Sep 7 19:09:00 2019 From: mandrikov at gmail.com (Evgeny Mandrikov) Date: Sat, 7 Sep 2019 21:09:00 +0200 Subject: RFR: JDK-8073188: Remove disable of old MSVC++ warning C4786 In-Reply-To: References: Message-ID: On Sun, Sep 1, 2019 at 7:03 AM Kim Barrett wrote: > CMS has been deprecated for more than two years: > https://bugs.openjdk.java.net/browse/JDK-8179013 > > and there is a proposal to remove it entirely: > https://bugs.openjdk.java.net/browse/JDK-8229049 > > which is winding its way through the process, so making small cleanups > to it like this is not really worthwhile. > Hi Kim, I know that CMS has been deprecated. Anyway thank you for the comments. I'll close JDK-8073188 with reference to this thread. From mandrikov at gmail.com Sun Sep 8 21:51:06 2019 From: mandrikov at gmail.com (Evgeny Mandrikov) Date: Sun, 8 Sep 2019 23:51:06 +0200 Subject: RFR: JDK-8207800: always_do_update_barrier is unused Message-ID: Hello! Please review patch [1] for JDK-8207800 [2]. Also it needs a sponsor since I have only author status in OpenJDK Census [3]. After this change tier1 tests pass on my machine. With best regards, Evgeny Mandrikov [1] http://cr.openjdk.java.net/~godin/8207800/webrev.00/ [2] https://bugs.openjdk.java.net/browse/JDK-8207800 [3] https://openjdk.java.net/census#godin From shade at redhat.com Mon Sep 9 07:37:44 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 9 Sep 2019 09:37:44 +0200 Subject: RFR: JDK-8207800: always_do_update_barrier is unused In-Reply-To: References: Message-ID: On 9/8/19 11:51 PM, Evgeny Mandrikov wrote: > [1] http://cr.openjdk.java.net/~godin/8207800/webrev.00/ This change looks good. Let's wait for more reviews. I can sponsor it then. Now running jdk-submit on it. -- Thanks, -Aleksey From shade at redhat.com Mon Sep 9 08:34:24 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 9 Sep 2019 10:34:24 +0200 Subject: RFR: JDK-8207800: always_do_update_barrier is unused In-Reply-To: References: Message-ID: On 9/9/19 9:37 AM, Aleksey Shipilev wrote: > On 9/8/19 11:51 PM, Evgeny Mandrikov wrote: >> [1] http://cr.openjdk.java.net/~godin/8207800/webrev.00/ > > This change looks good. > > Let's wait for more reviews. I can sponsor it then. Now running jdk-submit on it. jdk-submit passed. -- Thanks, -Aleksey From thomas.schatzl at oracle.com Mon Sep 9 09:39:05 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 9 Sep 2019 11:39:05 +0200 Subject: RFR: JDK-8207800: always_do_update_barrier is unused In-Reply-To: References: Message-ID: <27a416a9-2f4e-4ac9-970e-6d3d0dbf40f1@oracle.com> Hi, On 9/8/19 11:51 PM, Evgeny Mandrikov wrote: > Hello! > > Please review patch [1] for JDK-8207800 [2]. Also it needs a sponsor since > I have only author status in OpenJDK Census [3]. > > After this change tier1 tests pass on my machine. > looks good. Thanks. Thomas From stefan.karlsson at oracle.com Mon Sep 9 10:11:09 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 9 Sep 2019 12:11:09 +0200 Subject: RFR: 8230756: ZGC: Remove redundant memset in ZStatValue Message-ID: Hi all, Please review this patch to remove a redundant memset in ZStatValue. https://cr.openjdk.java.net/~stefank/8230756/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8230756 ZUtils::alloc_aligned alread clears the memory, so there's no need to clear the returned memory. Thanks, StefanK From stefan.karlsson at oracle.com Mon Sep 9 10:13:02 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 9 Sep 2019 12:13:02 +0200 Subject: RFR: 8230758: ZGC: Add missing precompiled include and fix friend declaration Message-ID: Hi all, Please review this small patch to add a missing include of precompiled.hpp and fix a friend declaration to use class instead of struct. https://cr.openjdk.java.net/~stefank/8230758/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8230758 Thanks, StefanK From thomas.schatzl at oracle.com Mon Sep 9 10:14:26 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 9 Sep 2019 12:14:26 +0200 Subject: RFR: 8230758: ZGC: Add missing precompiled include and fix friend declaration In-Reply-To: References: Message-ID: On 9/9/19 12:13 PM, Stefan Karlsson wrote: > Hi all, > > Please review this small patch to add a missing include of > precompiled.hpp and fix a friend declaration to use class instead of > struct. > > https://cr.openjdk.java.net/~stefank/8230758/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8230758 > > Thanks, > StefanK looks good. Thomas From thomas.schatzl at oracle.com Mon Sep 9 10:15:52 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 9 Sep 2019 12:15:52 +0200 Subject: RFR: 8230756: ZGC: Remove redundant memset in ZStatValue In-Reply-To: References: Message-ID: On 9/9/19 12:11 PM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to remove a redundant memset in ZStatValue. > > https://cr.openjdk.java.net/~stefank/8230756/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8230756 > > ZUtils::alloc_aligned alread clears the memory, so there's no need to > clear the returned memory. > > Thanks, > StefanK looks good. Thomas From stefan.karlsson at oracle.com Mon Sep 9 10:16:25 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 9 Sep 2019 12:16:25 +0200 Subject: RFR: 8230759: ZGC: Fix integer types Message-ID: <1a6cc4b1-b177-a934-18b3-844341312149@oracle.com> Hi all, Please review this patch to fix some narrowing and inconsistencies in our usage of different integer types. https://cr.openjdk.java.net/~stefank/8230759/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8230759 Thanks, StefanK From thomas.schatzl at oracle.com Mon Sep 9 10:28:58 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 9 Sep 2019 12:28:58 +0200 Subject: RFR: 8221361: Eliminate two-phase initialization for PtrQueueSet classes In-Reply-To: <801F537C-948E-4460-B353-E55898AAD2FC@oracle.com> References: <801F537C-948E-4460-B353-E55898AAD2FC@oracle.com> Message-ID: <6130c460-dd64-aa1d-76e0-ddc711413eed@oracle.com> Hi, On 9/6/19 11:25 PM, Kim Barrett wrote: > Please review this refactoring of PtrQueueSet to eliminate two-phase > construction. The objects are now fully constructed by the constructor and > there is no longer a need for an initialize() function. > > In particular, the allocator is now a constructor argument. And in the case > of G1DirtyCardQueueSet, the CBL monitor is also a constructor argument. > > This means that PtrQueueSet objects are always fully initialized. As a > consequence, we can eliminate the bootstrapping kludge in PtrQueue for > initializing the _capacity_in_bytes member. > > This change required a little bit of refactoring for Shenandoah. The SATB > buffer allocator has been moved from the SATB pqset to the barrier set. When > constructing the barrier set, the allocator is constructed first and then > passed as an argument to the pqset constructor. This is all consistent with > G1's arrangement. Someone from the Shenandoah team should check this. > This looks good. Great refactoring. Thanks, Thomas From kim.barrett at oracle.com Mon Sep 9 18:28:19 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 9 Sep 2019 14:28:19 -0400 Subject: RFR: 8221361: Eliminate two-phase initialization for PtrQueueSet classes In-Reply-To: <6130c460-dd64-aa1d-76e0-ddc711413eed@oracle.com> References: <801F537C-948E-4460-B353-E55898AAD2FC@oracle.com> <6130c460-dd64-aa1d-76e0-ddc711413eed@oracle.com> Message-ID: > On Sep 9, 2019, at 6:28 AM, Thomas Schatzl wrote: > > Hi, > > On 9/6/19 11:25 PM, Kim Barrett wrote: >> Please review this refactoring of PtrQueueSet to eliminate two-phase >> construction. The objects are now fully constructed by the constructor and >> there is no longer a need for an initialize() function. >> In particular, the allocator is now a constructor argument. And in the case >> of G1DirtyCardQueueSet, the CBL monitor is also a constructor argument. >> This means that PtrQueueSet objects are always fully initialized. As a >> consequence, we can eliminate the bootstrapping kludge in PtrQueue for >> initializing the _capacity_in_bytes member. >> This change required a little bit of refactoring for Shenandoah. The SATB >> buffer allocator has been moved from the SATB pqset to the barrier set. When >> constructing the barrier set, the allocator is constructed first and then >> passed as an argument to the pqset constructor. This is all consistent with >> G1's arrangement. Someone from the Shenandoah team should check this. > > This looks good. Great refactoring. > > Thanks, > Thomas Thanks. From shade at redhat.com Mon Sep 9 18:31:17 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 9 Sep 2019 20:31:17 +0200 Subject: RFR: 8221361: Eliminate two-phase initialization for PtrQueueSet classes In-Reply-To: References: <801F537C-948E-4460-B353-E55898AAD2FC@oracle.com> Message-ID: <555e0ff9-2c0d-61af-d5d5-fbd6251bde94@redhat.com> On 9/6/19 11:53 PM, Aleksey Shipilev wrote: > On 9/6/19 11:25 PM, Kim Barrett wrote: >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8221361/open.00/ > > Shenandoah parts look good. Thank you for going extra mile there. It is a bit unfortunate we have > lost the ability to use cached _heap in ShenandoahSATBMarkQueueSet::filter, but this should not be > important. > > It passes hotspot_gc_shenandoah for me locally. The entire patch looks good too. -- Thanks, -Aleksey From kim.barrett at oracle.com Mon Sep 9 20:49:09 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 9 Sep 2019 16:49:09 -0400 Subject: RFR: 8221361: Eliminate two-phase initialization for PtrQueueSet classes In-Reply-To: <555e0ff9-2c0d-61af-d5d5-fbd6251bde94@redhat.com> References: <801F537C-948E-4460-B353-E55898AAD2FC@oracle.com> <555e0ff9-2c0d-61af-d5d5-fbd6251bde94@redhat.com> Message-ID: > On Sep 9, 2019, at 2:31 PM, Aleksey Shipilev wrote: > > On 9/6/19 11:53 PM, Aleksey Shipilev wrote: >> On 9/6/19 11:25 PM, Kim Barrett wrote: >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8221361/open.00/ >> >> Shenandoah parts look good. Thank you for going extra mile there. It is a bit unfortunate we have >> lost the ability to use cached _heap in ShenandoahSATBMarkQueueSet::filter, but this should not be >> important. >> >> It passes hotspot_gc_shenandoah for me locally. > > The entire patch looks good too. > > -- > Thanks, > -Aleksey Thanks. From shade at redhat.com Tue Sep 10 06:06:34 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 10 Sep 2019 08:06:34 +0200 Subject: RFR: JDK-8207800: always_do_update_barrier is unused In-Reply-To: References: Message-ID: On 9/9/19 10:34 AM, Aleksey Shipilev wrote: > On 9/9/19 9:37 AM, Aleksey Shipilev wrote: >> On 9/8/19 11:51 PM, Evgeny Mandrikov wrote: >>> [1] http://cr.openjdk.java.net/~godin/8207800/webrev.00/ >> >> This change looks good. >> >> Let's wait for more reviews. I can sponsor it then. Now running jdk-submit on it. > > jdk-submit passed. Pushed. -- Thanks, -Aleksey From per.liden at oracle.com Tue Sep 10 07:16:36 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 10 Sep 2019 09:16:36 +0200 Subject: RFR: 8230566: ZGC: Don't substitute klass pointer during array clearing In-Reply-To: References: <37bbb51a-40e4-8d22-0432-287ea2e2e4a6@oracle.com> <5c80fbc8-1202-a4da-43ed-d82888b6d6af@oracle.com> <9d9cfbd6-5409-226b-31c4-91ade0f3c054@oracle.com> Message-ID: <5111edeb-a26f-9e86-7fe5-42eb174f2b57@oracle.com> Thanks for reviewing. Updated webrev: http://cr.openjdk.java.net/~pliden/8230566/webrev.1-diff http://cr.openjdk.java.net/~pliden/8230566/webrev.1 /Per On 9/6/19 3:51 PM, Stefan Karlsson wrote: > Talked to Per and Erik offline about follow_array_object. We should be > able to skip following the klass pointer when follow is false. Just like > objects in "allocating" regions don't get their klass pointer followed. > Other than that, this looks good to me. > > Thanks, > StefanK > > > > On 2019-09-04 19:15, Per Liden wrote: >> Hi Erik, >> >> On 9/4/19 5:12 PM, Erik ?sterlund wrote: >>> Hi Per, >>> >>> In? src/hotspot/share/gc/z/zHeapIterator.cpp the only change made >>> here is to mess up the indentation: >>> >>> ? 195?? // Push roots to visit >>> ? 196?? push_roots>> Concurrent */, false /* Weak */>(); >>> ? 197?? push_roots>> Concurrent */, false /* Weak */>(); >>> >>> Should probably change that back. >> >> Will fix! >> >>> >>> One possible solution to the problem of passing around all this >>> context information in the templates, >>> is to wrap them in a template config type, like this: >>> >>> template >>> struct ZMarkConfig { >>> ?? static const bool _follow = follow; >>> ?? static const bool _finalizable = finalizable; >>> ?? static const bool _publish? = publish; >>> }; >>> >>> Then you can just pass around the ZMarkConfig in the templates all >>> over the place, instead of the 3 (at this >>> moment) bools. That way, adding and removing bools becomes easier in >>> the future; no need to change the passing >>> around code to pass more things around. Only the producer and >>> consumer of the flags need changing. >>> >>> However, I'd propose we do that in a follow-up RFE so we can fix the >>> bug faster. Having said that, looks good, >>> and don't need a new webrev. >> >> I'll prototype and see how it turns out. If it seems to be worth doing >> I'll file a separate RFE. >> >> Thanks for reviewing! >> >> cheers, >> Per >> >>> >>> Thanks, >>> /Erik >>> >>> On 2019-09-04 14:02, Per Liden wrote: >>>> When allocating large object arrays, ZObjArrayAllocator instantiates >>>> a non-cleared array of longs and later installs the correct klass >>>> pointer once the array has been cleared. While this might work it's >>>> also error prone. For example, there are at least two asserts() that >>>> will re-loaded the klass pointer and fail. It's also easy for >>>> someone in the future to make the innocent mistake of re-loading the >>>> klass pointer in some sensitive path, which would lead to new >>>> problems. We can avoid all these problems by not substituting with >>>> the klass pointer, and instead have another mechanism to tell GC >>>> marker threads to not follow the elements in not-yet-cleared arrays. >>>> >>>> This patch adds a new bit to ZMarkStackEntry, to signal to marking >>>> threads that the object array should not be followed, just mark it >>>> as live and visit its klass. >>>> >>>> The patch is large-ish because of the need to propagate the "follow" >>>> flag down through a few of layers. We might want to think about >>>> converting the flags (follow/finalizable/publish) into a single >>>> ZBarrierFlags-thing (perhaps similar to DecoratorSet) to reduce the >>>> noise and make it easier to add/adjust/remove flags in the future. >>>> But such an enhancements could come later. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8230566 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8230566/webrev.0 >>>> >>>> Testing: tier1-6 on linux + ad hoc runs of tests that provoked the >>>> problem >>>> >>>> /Per >>> From stefan.karlsson at oracle.com Tue Sep 10 07:20:39 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 10 Sep 2019 09:20:39 +0200 Subject: RFR: 8230566: ZGC: Don't substitute klass pointer during array clearing In-Reply-To: <5111edeb-a26f-9e86-7fe5-42eb174f2b57@oracle.com> References: <37bbb51a-40e4-8d22-0432-287ea2e2e4a6@oracle.com> <5c80fbc8-1202-a4da-43ed-d82888b6d6af@oracle.com> <9d9cfbd6-5409-226b-31c4-91ade0f3c054@oracle.com> <5111edeb-a26f-9e86-7fe5-42eb174f2b57@oracle.com> Message-ID: Looks good. StefanK On 2019-09-10 09:16, Per Liden wrote: > Thanks for reviewing. Updated webrev: > > http://cr.openjdk.java.net/~pliden/8230566/webrev.1-diff > http://cr.openjdk.java.net/~pliden/8230566/webrev.1 > > /Per > > On 9/6/19 3:51 PM, Stefan Karlsson wrote: >> Talked to Per and Erik offline about follow_array_object. We should be >> able to skip following the klass pointer when follow is false. Just >> like objects in "allocating" regions don't get their klass pointer >> followed. Other than that, this looks good to me. >> >> Thanks, >> StefanK >> >> >> >> On 2019-09-04 19:15, Per Liden wrote: >>> Hi Erik, >>> >>> On 9/4/19 5:12 PM, Erik ?sterlund wrote: >>>> Hi Per, >>>> >>>> In? src/hotspot/share/gc/z/zHeapIterator.cpp the only change made >>>> here is to mess up the indentation: >>>> >>>> ? 195?? // Push roots to visit >>>> ? 196?? push_roots>>> Concurrent */, false /* Weak */>(); >>>> ? 197?? push_roots>>> Concurrent */, false /* Weak */>(); >>>> >>>> Should probably change that back. >>> >>> Will fix! >>> >>>> >>>> One possible solution to the problem of passing around all this >>>> context information in the templates, >>>> is to wrap them in a template config type, like this: >>>> >>>> template >>>> struct ZMarkConfig { >>>> ?? static const bool _follow = follow; >>>> ?? static const bool _finalizable = finalizable; >>>> ?? static const bool _publish? = publish; >>>> }; >>>> >>>> Then you can just pass around the ZMarkConfig in the templates all >>>> over the place, instead of the 3 (at this >>>> moment) bools. That way, adding and removing bools becomes easier in >>>> the future; no need to change the passing >>>> around code to pass more things around. Only the producer and >>>> consumer of the flags need changing. >>>> >>>> However, I'd propose we do that in a follow-up RFE so we can fix the >>>> bug faster. Having said that, looks good, >>>> and don't need a new webrev. >>> >>> I'll prototype and see how it turns out. If it seems to be worth >>> doing I'll file a separate RFE. >>> >>> Thanks for reviewing! >>> >>> cheers, >>> Per >>> >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2019-09-04 14:02, Per Liden wrote: >>>>> When allocating large object arrays, ZObjArrayAllocator >>>>> instantiates a non-cleared array of longs and later installs the >>>>> correct klass pointer once the array has been cleared. While this >>>>> might work it's also error prone. For example, there are at least >>>>> two asserts() that will re-loaded the klass pointer and fail. It's >>>>> also easy for someone in the future to make the innocent mistake of >>>>> re-loading the klass pointer in some sensitive path, which would >>>>> lead to new problems. We can avoid all these problems by not >>>>> substituting with the klass pointer, and instead have another >>>>> mechanism to tell GC marker threads to not follow the elements in >>>>> not-yet-cleared arrays. >>>>> >>>>> This patch adds a new bit to ZMarkStackEntry, to signal to marking >>>>> threads that the object array should not be followed, just mark it >>>>> as live and visit its klass. >>>>> >>>>> The patch is large-ish because of the need to propagate the >>>>> "follow" flag down through a few of layers. We might want to think >>>>> about converting the flags (follow/finalizable/publish) into a >>>>> single ZBarrierFlags-thing (perhaps similar to DecoratorSet) to >>>>> reduce the noise and make it easier to add/adjust/remove flags in >>>>> the future. But such an enhancements could come later. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8230566 >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8230566/webrev.0 >>>>> >>>>> Testing: tier1-6 on linux + ad hoc runs of tests that provoked the >>>>> problem >>>>> >>>>> /Per >>>> From per.liden at oracle.com Tue Sep 10 07:23:56 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 10 Sep 2019 09:23:56 +0200 Subject: RFR: 8230566: ZGC: Don't substitute klass pointer during array clearing In-Reply-To: References: <37bbb51a-40e4-8d22-0432-287ea2e2e4a6@oracle.com> <5c80fbc8-1202-a4da-43ed-d82888b6d6af@oracle.com> <9d9cfbd6-5409-226b-31c4-91ade0f3c054@oracle.com> <5111edeb-a26f-9e86-7fe5-42eb174f2b57@oracle.com> Message-ID: <9fee3afb-9abb-5c50-5fcc-9a45c14feb19@oracle.com> Thanks Stefan! /Per On 9/10/19 9:20 AM, Stefan Karlsson wrote: > Looks good. > > StefanK > > On 2019-09-10 09:16, Per Liden wrote: >> Thanks for reviewing. Updated webrev: >> >> http://cr.openjdk.java.net/~pliden/8230566/webrev.1-diff >> http://cr.openjdk.java.net/~pliden/8230566/webrev.1 >> >> /Per >> >> On 9/6/19 3:51 PM, Stefan Karlsson wrote: >>> Talked to Per and Erik offline about follow_array_object. We should >>> be able to skip following the klass pointer when follow is false. >>> Just like objects in "allocating" regions don't get their klass >>> pointer followed. Other than that, this looks good to me. >>> >>> Thanks, >>> StefanK >>> >>> >>> >>> On 2019-09-04 19:15, Per Liden wrote: >>>> Hi Erik, >>>> >>>> On 9/4/19 5:12 PM, Erik ?sterlund wrote: >>>>> Hi Per, >>>>> >>>>> In? src/hotspot/share/gc/z/zHeapIterator.cpp the only change made >>>>> here is to mess up the indentation: >>>>> >>>>> ? 195?? // Push roots to visit >>>>> ? 196?? push_roots>>>> Concurrent */, false /* Weak */>(); >>>>> ? 197?? push_roots>>>> Concurrent */, false /* Weak */>(); >>>>> >>>>> Should probably change that back. >>>> >>>> Will fix! >>>> >>>>> >>>>> One possible solution to the problem of passing around all this >>>>> context information in the templates, >>>>> is to wrap them in a template config type, like this: >>>>> >>>>> template >>>>> struct ZMarkConfig { >>>>> ?? static const bool _follow = follow; >>>>> ?? static const bool _finalizable = finalizable; >>>>> ?? static const bool _publish? = publish; >>>>> }; >>>>> >>>>> Then you can just pass around the ZMarkConfig in the templates all >>>>> over the place, instead of the 3 (at this >>>>> moment) bools. That way, adding and removing bools becomes easier >>>>> in the future; no need to change the passing >>>>> around code to pass more things around. Only the producer and >>>>> consumer of the flags need changing. >>>>> >>>>> However, I'd propose we do that in a follow-up RFE so we can fix >>>>> the bug faster. Having said that, looks good, >>>>> and don't need a new webrev. >>>> >>>> I'll prototype and see how it turns out. If it seems to be worth >>>> doing I'll file a separate RFE. >>>> >>>> Thanks for reviewing! >>>> >>>> cheers, >>>> Per >>>> >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2019-09-04 14:02, Per Liden wrote: >>>>>> When allocating large object arrays, ZObjArrayAllocator >>>>>> instantiates a non-cleared array of longs and later installs the >>>>>> correct klass pointer once the array has been cleared. While this >>>>>> might work it's also error prone. For example, there are at least >>>>>> two asserts() that will re-loaded the klass pointer and fail. It's >>>>>> also easy for someone in the future to make the innocent mistake >>>>>> of re-loading the klass pointer in some sensitive path, which >>>>>> would lead to new problems. We can avoid all these problems by not >>>>>> substituting with the klass pointer, and instead have another >>>>>> mechanism to tell GC marker threads to not follow the elements in >>>>>> not-yet-cleared arrays. >>>>>> >>>>>> This patch adds a new bit to ZMarkStackEntry, to signal to marking >>>>>> threads that the object array should not be followed, just mark it >>>>>> as live and visit its klass. >>>>>> >>>>>> The patch is large-ish because of the need to propagate the >>>>>> "follow" flag down through a few of layers. We might want to think >>>>>> about converting the flags (follow/finalizable/publish) into a >>>>>> single ZBarrierFlags-thing (perhaps similar to DecoratorSet) to >>>>>> reduce the noise and make it easier to add/adjust/remove flags in >>>>>> the future. But such an enhancements could come later. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8230566 >>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8230566/webrev.0 >>>>>> >>>>>> Testing: tier1-6 on linux + ad hoc runs of tests that provoked the >>>>>> problem >>>>>> >>>>>> /Per >>>>> From per.liden at oracle.com Tue Sep 10 08:04:36 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 10 Sep 2019 10:04:36 +0200 Subject: RFR: 8230758: ZGC: Add missing precompiled include and fix friend declaration In-Reply-To: References: Message-ID: <7b2c96ba-6811-7e3f-917d-ed5ed7e587e0@oracle.com> Looks good. /Per On 9/9/19 12:13 PM, Stefan Karlsson wrote: > Hi all, > > Please review this small patch to add a missing include of > precompiled.hpp and fix a friend declaration to use class instead of > struct. > > https://cr.openjdk.java.net/~stefank/8230758/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8230758 > > Thanks, > StefanK From per.liden at oracle.com Tue Sep 10 08:06:42 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 10 Sep 2019 10:06:42 +0200 Subject: RFR: 8230756: ZGC: Remove redundant memset in ZStatValue In-Reply-To: References: Message-ID: <0b9597d1-a431-50f4-e7fb-cb57dbcdcac1@oracle.com> Looks good. /Per On 9/9/19 12:11 PM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to remove a redundant memset in ZStatValue. > > https://cr.openjdk.java.net/~stefank/8230756/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8230756 > > ZUtils::alloc_aligned alread clears the memory, so there's no need to > clear the returned memory. > > Thanks, > StefanK From thomas.schatzl at oracle.com Tue Sep 10 10:20:23 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 10 Sep 2019 12:20:23 +0200 Subject: RFR (S): 8230794: Improve assert to get more information about the JDK-8227695 failure Message-ID: <01ef728bbdb1efc22e2bca28bb5d2141da4642cc.camel@oracle.com> Hi all, can I have reviews for this change that improves the assert that is failing in JDK-8227695? We have been unsuccessfully trying to reproduce the problem locally and doing tens of thousands runs with the original code to get a better idea of the cause. (Or finding the issue with code inspection or by tweaking the code) This change augments the assert to show more information, and adds it in several places to get a better idea of the cause. It looks like this issue is caused by extremely specific environmental conditions we were unable to reproduce. We will remove this debug code later. CR: https://bugs.openjdk.java.net/browse/JDK-8230794 Webrev: http://cr.openjdk.java.net/~tschatzl/8230794/webrev/ Testing: hs-tier1-5 without failures :( Thanks, Thomas From stefan.johansson at oracle.com Tue Sep 10 10:25:07 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 10 Sep 2019 12:25:07 +0200 Subject: RFR (S): 8230794: Improve assert to get more information about the JDK-8227695 failure In-Reply-To: <01ef728bbdb1efc22e2bca28bb5d2141da4642cc.camel@oracle.com> References: <01ef728bbdb1efc22e2bca28bb5d2141da4642cc.camel@oracle.com> Message-ID: <090d52da-e9e1-656b-590a-947d86497765@oracle.com> Hi Thomas, On 2019-09-10 12:20, Thomas Schatzl wrote: > Hi all, > > can I have reviews for this change that improves the assert that is > failing in JDK-8227695? > > We have been unsuccessfully trying to reproduce the problem locally and > doing tens of thousands runs with the original code to get a better > idea of the cause. (Or finding the issue with code inspection or by > tweaking the code) > > This change augments the assert to show more information, and adds it > in several places to get a better idea of the cause. > > It looks like this issue is caused by extremely specific environmental > conditions we were unable to reproduce. > > We will remove this debug code later. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8230794 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8230794/webrev/ Looks good, let's hope this helps us pin point the issue. Thanks, Stefan > Testing: > hs-tier1-5 without failures :( > > Thanks, > Thomas > From per.liden at oracle.com Tue Sep 10 10:47:25 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 10 Sep 2019 12:47:25 +0200 Subject: RFR: 8230796: Remove BarrierSet::oop_equals_operator_allowed() Message-ID: <0e93041c-6f79-89c9-3c41-008fc33fa7a9@oracle.com> BarrierSet::oop_equals_operator_allowed() is no longer specialized by any barrier set and can be removed. Bug: https://bugs.openjdk.java.net/browse/JDK-8230796 Webrev: http://cr.openjdk.java.net/~pliden/8230796/webrev.0 /Per From shade at redhat.com Tue Sep 10 10:50:12 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 10 Sep 2019 12:50:12 +0200 Subject: RFR: 8230796: Remove BarrierSet::oop_equals_operator_allowed() In-Reply-To: <0e93041c-6f79-89c9-3c41-008fc33fa7a9@oracle.com> References: <0e93041c-6f79-89c9-3c41-008fc33fa7a9@oracle.com> Message-ID: <3647643a-3233-5618-a7fb-6bd3a1e8f5c2@redhat.com> On 9/10/19 12:47 PM, Per Liden wrote: > BarrierSet::oop_equals_operator_allowed() is no longer specialized by any barrier set and can be > removed. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8230796 > Webrev: http://cr.openjdk.java.net/~pliden/8230796/webrev.0 Yes, the only use was Shenandoah, and it is not need it anymore. Looks good. -- Thanks, -Aleksey From per.liden at oracle.com Tue Sep 10 10:57:17 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 10 Sep 2019 12:57:17 +0200 Subject: RFR: 8230796: Remove BarrierSet::oop_equals_operator_allowed() In-Reply-To: <3647643a-3233-5618-a7fb-6bd3a1e8f5c2@redhat.com> References: <0e93041c-6f79-89c9-3c41-008fc33fa7a9@oracle.com> <3647643a-3233-5618-a7fb-6bd3a1e8f5c2@redhat.com> Message-ID: On 9/10/19 12:50 PM, Aleksey Shipilev wrote: > On 9/10/19 12:47 PM, Per Liden wrote: >> BarrierSet::oop_equals_operator_allowed() is no longer specialized by any barrier set and can be >> removed. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8230796 >> Webrev: http://cr.openjdk.java.net/~pliden/8230796/webrev.0 > > Yes, the only use was Shenandoah, and it is not need it anymore. > > Looks good. Thanks for reviewing. Btw, as a follow up, I'm looking at removing oopDesc::equals()/equals_raw() and the related Access infrastructure, which should also be the same for all GCs now. /Per From mandrikov at gmail.com Tue Sep 10 10:58:30 2019 From: mandrikov at gmail.com (Evgeny Mandrikov) Date: Tue, 10 Sep 2019 12:58:30 +0200 Subject: RFR: JDK-8207800: always_do_update_barrier is unused In-Reply-To: References: Message-ID: On Tue, Sep 10, 2019 at 8:06 AM Aleksey Shipilev wrote: > Pushed. > Thanks. Regards, Evgeny From shade at redhat.com Tue Sep 10 10:59:35 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 10 Sep 2019 12:59:35 +0200 Subject: RFR: 8230796: Remove BarrierSet::oop_equals_operator_allowed() In-Reply-To: References: <0e93041c-6f79-89c9-3c41-008fc33fa7a9@oracle.com> <3647643a-3233-5618-a7fb-6bd3a1e8f5c2@redhat.com> Message-ID: <105cb690-0403-caf3-cd2e-90aa59b73b72@redhat.com> On 9/10/19 12:57 PM, Per Liden wrote: > On 9/10/19 12:50 PM, Aleksey Shipilev wrote: >> On 9/10/19 12:47 PM, Per Liden wrote: >>> BarrierSet::oop_equals_operator_allowed() is no longer specialized by any barrier set and can be >>> removed. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8230796 >>> Webrev: http://cr.openjdk.java.net/~pliden/8230796/webrev.0 >> >> Yes, the only use was Shenandoah, and it is not need it anymore. >> >> Looks good. > > Thanks for reviewing. > > Btw, as a follow up, I'm looking at removing oopDesc::equals()/equals_raw() and the related Access > infrastructure, which should also be the same for all GCs now. I'd talk with Valhalla folks first. IIRC, early Vahalla prototypes involved overloading acmp for properly handling value obj comparisons. oopDesc::equals(_raw) might come handy there. -- Thanks, -Aleksey From per.liden at oracle.com Tue Sep 10 13:27:22 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 10 Sep 2019 15:27:22 +0200 Subject: RFR: 8230808: Remove Access::equals() Message-ID: Access::equals() is not specialized by any barrier set and can be removed. This patch doesn't remove oopDesc::equals()/equals_raw(), but they don't use Access::equals() to do the work anymore. Removing oopDesc::equals() is a more intrusive change, which can be done as the next step. Bug: https://bugs.openjdk.java.net/browse/JDK-8230808 Webrev: http://cr.openjdk.java.net/~pliden/8230808/webrev.0 /Per From per.liden at oracle.com Tue Sep 10 13:27:30 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 10 Sep 2019 15:27:30 +0200 Subject: RFR: 8230796: Remove BarrierSet::oop_equals_operator_allowed() In-Reply-To: <105cb690-0403-caf3-cd2e-90aa59b73b72@redhat.com> References: <0e93041c-6f79-89c9-3c41-008fc33fa7a9@oracle.com> <3647643a-3233-5618-a7fb-6bd3a1e8f5c2@redhat.com> <105cb690-0403-caf3-cd2e-90aa59b73b72@redhat.com> Message-ID: On 9/10/19 12:59 PM, Aleksey Shipilev wrote: > On 9/10/19 12:57 PM, Per Liden wrote: >> On 9/10/19 12:50 PM, Aleksey Shipilev wrote: >>> On 9/10/19 12:47 PM, Per Liden wrote: >>>> BarrierSet::oop_equals_operator_allowed() is no longer specialized by any barrier set and can be >>>> removed. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8230796 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8230796/webrev.0 >>> >>> Yes, the only use was Shenandoah, and it is not need it anymore. >>> >>> Looks good. >> >> Thanks for reviewing. >> >> Btw, as a follow up, I'm looking at removing oopDesc::equals()/equals_raw() and the related Access >> infrastructure, which should also be the same for all GCs now. > > I'd talk with Valhalla folks first. IIRC, early Vahalla prototypes involved overloading acmp for > properly handling value obj comparisons. oopDesc::equals(_raw) might come handy there. > I've talked David, no need for it in Valhalla. cheers, Per From kim.barrett at oracle.com Tue Sep 10 13:36:15 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 10 Sep 2019 09:36:15 -0400 Subject: RFR (S): 8230794: Improve assert to get more information about the JDK-8227695 failure In-Reply-To: <01ef728bbdb1efc22e2bca28bb5d2141da4642cc.camel@oracle.com> References: <01ef728bbdb1efc22e2bca28bb5d2141da4642cc.camel@oracle.com> Message-ID: <1180B605-41EF-49D8-B416-F53CDF006F7F@oracle.com> > On Sep 10, 2019, at 6:20 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this change that improves the assert that is > failing in JDK-8227695? > > We have been unsuccessfully trying to reproduce the problem locally and > doing tens of thousands runs with the original code to get a better > idea of the cause. (Or finding the issue with code inspection or by > tweaking the code) > > This change augments the assert to show more information, and adds it > in several places to get a better idea of the cause. > > It looks like this issue is caused by extremely specific environmental > conditions we were unable to reproduce. > > We will remove this debug code later. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8230794 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8230794/webrev/ > Testing: > hs-tier1-5 without failures :( > > Thanks, > Thomas Looks good. From thomas.schatzl at oracle.com Tue Sep 10 13:44:24 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 10 Sep 2019 15:44:24 +0200 Subject: RFR (S): 8230794: Improve assert to get more information about the JDK-8227695 failure In-Reply-To: <1180B605-41EF-49D8-B416-F53CDF006F7F@oracle.com> References: <01ef728bbdb1efc22e2bca28bb5d2141da4642cc.camel@oracle.com> <1180B605-41EF-49D8-B416-F53CDF006F7F@oracle.com> Message-ID: <166753bb-206b-7c17-af0b-d9aa206144e7@oracle.com> Hi Stefan, Kim, thanks for your reviews. Thomas On 9/10/19 3:36 PM, Kim Barrett wrote: >> On Sep 10, 2019, at 6:20 AM, Thomas Schatzl wrote: >> >> Hi all, >> >> can I have reviews for this change that improves the assert that is >> failing in JDK-8227695? >> >> We have been unsuccessfully trying to reproduce the problem locally and >> doing tens of thousands runs with the original code to get a better >> idea of the cause. (Or finding the issue with code inspection or by >> tweaking the code) >> >> This change augments the assert to show more information, and adds it >> in several places to get a better idea of the cause. >> >> It looks like this issue is caused by extremely specific environmental >> conditions we were unable to reproduce. >> >> We will remove this debug code later. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8230794 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8230794/webrev/ >> Testing: >> hs-tier1-5 without failures :( >> >> Thanks, >> Thomas > > Looks good. > From stefan.karlsson at oracle.com Tue Sep 10 15:55:24 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 10 Sep 2019 17:55:24 +0200 Subject: RFR: 8230759: ZGC: Fix integer types In-Reply-To: <1a6cc4b1-b177-a934-18b3-844341312149@oracle.com> References: <1a6cc4b1-b177-a934-18b3-844341312149@oracle.com> Message-ID: Per wanted to change some of our usages of uint32_t to use size_t instead. New webrev: https://cr.openjdk.java.net/~stefank/8230759/webrev.02 Thanks, StefanK On 2019-09-09 12:16, Stefan Karlsson wrote: > Hi all, > > Please review this patch to fix some narrowing and inconsistencies in > our usage of different integer types. > > https://cr.openjdk.java.net/~stefank/8230759/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8230759 > > Thanks, > StefanK From per.liden at oracle.com Wed Sep 11 07:35:00 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 11 Sep 2019 09:35:00 +0200 Subject: RFR: 8230841: Remove oopDesc::equals() Message-ID: <9946de4d-8001-e1e3-fa80-633280f31483@oracle.com> With the removal of Access::equals() in JDK-8230808, there's no longer any need to treat oop equality in a special way. Therefore, we can remove oopDesc::equals() and oopDesc::equals_raw(). Bug: https://bugs.openjdk.java.net/browse/JDK-8230841 Webrev: http://cr.openjdk.java.net/~pliden/8230841/webrev.0 Testing: Currently running through tier1-3 in all platforms /Per From per.liden at oracle.com Wed Sep 11 07:37:18 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 11 Sep 2019 09:37:18 +0200 Subject: RFR: 8230759: ZGC: Fix integer types In-Reply-To: References: <1a6cc4b1-b177-a934-18b3-844341312149@oracle.com> Message-ID: <5d304f68-7db8-7878-6d4a-f4f22a28b2de@oracle.com> Looks good! /Per On 9/10/19 5:55 PM, Stefan Karlsson wrote: > Per wanted to change some of our usages of uint32_t to use size_t > instead. New webrev: > https://cr.openjdk.java.net/~stefank/8230759/webrev.02 > > Thanks, > StefanK > > On 2019-09-09 12:16, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to fix some narrowing and inconsistencies in >> our usage of different integer types. >> >> https://cr.openjdk.java.net/~stefank/8230759/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8230759 >> >> Thanks, >> StefanK > From per.liden at oracle.com Wed Sep 11 08:40:54 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 11 Sep 2019 10:40:54 +0200 Subject: RFR: 8230845: ZGC: Implement ZLock using os::PlatformMutex Message-ID: <6728fd66-410d-ccaa-b3e7-91d236454500@oracle.com> ZLock is currently shared code but with a POSIX-specific implementation. We now have os::PlatformMutex, so ZLock could just use that instead and be portable. We still want to keep the ZLock class, even though it's now just a wrapper around PlatformMutex, to have the freedom to easily change lock implementation, if such a need should arise. Bug: https://bugs.openjdk.java.net/browse/JDK-8230845 Webrev: http://cr.openjdk.java.net/~pliden/8230845/webrev.0 /Per From per.liden at oracle.com Wed Sep 11 08:40:57 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 11 Sep 2019 10:40:57 +0200 Subject: RFR: 8230846: ZGC: Make ZUtils::alloc_aligned() posix-specific Message-ID: <859f2a07-df70-9f57-71fd-bc98bc063834@oracle.com> ZUtils::alloc_aligned() is currently shared code, but with a POSIX-specific implementation. We should move that function in under src/hotspot/os/posix. Bug: https://bugs.openjdk.java.net/browse/JDK-8230846 Webrev: http://cr.openjdk.java.net/~pliden/8230846/webrev.0 /Per From stefan.karlsson at oracle.com Wed Sep 11 09:10:28 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 11 Sep 2019 11:10:28 +0200 Subject: RFR: 8230845: ZGC: Implement ZLock using os::PlatformMutex In-Reply-To: <6728fd66-410d-ccaa-b3e7-91d236454500@oracle.com> References: <6728fd66-410d-ccaa-b3e7-91d236454500@oracle.com> Message-ID: <41b95475-67c5-9ecd-0ff5-79f325be4fd6@oracle.com> Looks good. StefanK On 2019-09-11 10:40, Per Liden wrote: > ZLock is currently shared code but with a POSIX-specific implementation. > We now have os::PlatformMutex, so ZLock could just use that instead and > be portable. We still want to keep the ZLock class, even though it's now > just a wrapper around PlatformMutex, to have the freedom to easily > change lock implementation, if such a need should arise. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8230845 > Webrev: http://cr.openjdk.java.net/~pliden/8230845/webrev.0 > > /Per From stefan.karlsson at oracle.com Wed Sep 11 09:12:13 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 11 Sep 2019 11:12:13 +0200 Subject: RFR: 8230846: ZGC: Make ZUtils::alloc_aligned() posix-specific In-Reply-To: <859f2a07-df70-9f57-71fd-bc98bc063834@oracle.com> References: <859f2a07-df70-9f57-71fd-bc98bc063834@oracle.com> Message-ID: Looks good. StefanK On 2019-09-11 10:40, Per Liden wrote: > ZUtils::alloc_aligned() is currently shared code, but with a > POSIX-specific implementation. We should move that function in under > src/hotspot/os/posix. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8230846 > Webrev: http://cr.openjdk.java.net/~pliden/8230846/webrev.0 > > /Per From per.liden at oracle.com Wed Sep 11 09:13:15 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 11 Sep 2019 11:13:15 +0200 Subject: RFR: 8230845: ZGC: Implement ZLock using os::PlatformMutex In-Reply-To: <41b95475-67c5-9ecd-0ff5-79f325be4fd6@oracle.com> References: <6728fd66-410d-ccaa-b3e7-91d236454500@oracle.com> <41b95475-67c5-9ecd-0ff5-79f325be4fd6@oracle.com> Message-ID: Thanks! /Per On 9/11/19 11:10 AM, Stefan Karlsson wrote: > Looks good. > > StefanK > > On 2019-09-11 10:40, Per Liden wrote: >> ZLock is currently shared code but with a POSIX-specific >> implementation. We now have os::PlatformMutex, so ZLock could just use >> that instead and be portable. We still want to keep the ZLock class, >> even though it's now just a wrapper around PlatformMutex, to have the >> freedom to easily change lock implementation, if such a need should >> arise. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8230845 >> Webrev: http://cr.openjdk.java.net/~pliden/8230845/webrev.0 >> >> /Per From per.liden at oracle.com Wed Sep 11 09:13:21 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 11 Sep 2019 11:13:21 +0200 Subject: RFR: 8230846: ZGC: Make ZUtils::alloc_aligned() posix-specific In-Reply-To: References: <859f2a07-df70-9f57-71fd-bc98bc063834@oracle.com> Message-ID: Thanks! /Per On 9/11/19 11:12 AM, Stefan Karlsson wrote: > Looks good. > > StefanK > > On 2019-09-11 10:40, Per Liden wrote: >> ZUtils::alloc_aligned() is currently shared code, but with a >> POSIX-specific implementation. We should move that function in under >> src/hotspot/os/posix. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8230846 >> Webrev: http://cr.openjdk.java.net/~pliden/8230846/webrev.0 >> >> /Per From rkennke at redhat.com Wed Sep 11 19:43:39 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 11 Sep 2019 21:43:39 +0200 Subject: RFR: 8230841: Remove oopDesc::equals() In-Reply-To: <9946de4d-8001-e1e3-fa80-633280f31483@oracle.com> References: <9946de4d-8001-e1e3-fa80-633280f31483@oracle.com> Message-ID: <1e5bc1b3-50b2-2dcd-02e4-7527e4271fb5@redhat.com> Hello Per, Thanks for cleaning this up! I meant to do this eventually, but haven't gotten to it, yet. So you beat me :-) I looked over the webrev, and looks good to me. I also checked Shenandoah tests (tier1+2+3) and looks good to. Thumbs up! Thanks, Roman > With the removal of Access::equals() in JDK-8230808, there's no longer > any need to treat oop equality in a special way. Therefore, we can > remove oopDesc::equals() and oopDesc::equals_raw(). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8230841 > Webrev: http://cr.openjdk.java.net/~pliden/8230841/webrev.0 > > Testing: Currently running through tier1-3 in all platforms > > /Per From per.liden at oracle.com Wed Sep 11 20:48:03 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 11 Sep 2019 22:48:03 +0200 Subject: RFR: 8230878: ZGC: Use thread_local instead of __thread Message-ID: <783cc8dc-c98d-a477-b08a-01911354c43d@oracle.com> We're currently using the gcc specific __thread syntax for thread local data. We should make those declarations compiler agnostic. This patch depends on the proposal to rename THREAD_LOCAL_DECL to thread_local (JDK-8230877), which is currently out for review on hotspot-dev. Bug: https://bugs.openjdk.java.net/browse/JDK-8230878 Webrev: http://cr.openjdk.java.net/~pliden/8230878/webrev.0 /Per From per.liden at oracle.com Wed Sep 11 20:52:46 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 11 Sep 2019 22:52:46 +0200 Subject: RFR: 8230841: Remove oopDesc::equals() In-Reply-To: <1e5bc1b3-50b2-2dcd-02e4-7527e4271fb5@redhat.com> References: <9946de4d-8001-e1e3-fa80-633280f31483@oracle.com> <1e5bc1b3-50b2-2dcd-02e4-7527e4271fb5@redhat.com> Message-ID: <1e250d65-ea00-f627-733b-8e974a24fbdb@oracle.com> Thanks Roman! /Per On 9/11/19 9:43 PM, Roman Kennke wrote: > Hello Per, > > Thanks for cleaning this up! I meant to do this eventually, but haven't > gotten to it, yet. So you beat me :-) > > I looked over the webrev, and looks good to me. > > I also checked Shenandoah tests (tier1+2+3) and looks good to. > > Thumbs up! > > Thanks, > Roman > > >> With the removal of Access::equals() in JDK-8230808, there's no longer >> any need to treat oop equality in a special way. Therefore, we can >> remove oopDesc::equals() and oopDesc::equals_raw(). >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8230841 >> Webrev: http://cr.openjdk.java.net/~pliden/8230841/webrev.0 >> >> Testing: Currently running through tier1-3 in all platforms >> >> /Per > From stefan.johansson at oracle.com Wed Sep 11 21:34:29 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 11 Sep 2019 23:34:29 +0200 Subject: [PATCH] Exploit Empty Regions in Young Gen to Enhance PS Full GC Performance In-Reply-To: References: <4F02DD53-EA98-4A1A-B871-C6E9D9610B2C@oracle.com> <9B69AFD1-7AE2-4B50-BFCF-C9C6E2594240@oracle.com> Message-ID: Hi Haoyu, I recently came across your patch and I would like to pick up on some of the things Kim mentioned in his mails. I especially want evaluate and investigate if this is a technique we can use to improve the other GCs as well. To start that work I want to take the patch for a spin in our internal performance testing. The patch doesn?t apply clean to the latest JDK repository, so if you could provide an updated patch that would be very helpful. It would also be great if you could share some more information around the results presented in the paper. For example, it would be good to get the full command lines for the different benchmarks so we can run them locally and reproduce the results you?ve seen. Thanks, Stefan > 12 mars 2019 kl. 03:21 skrev Haoyu Li : > > Hi Kim, > > Thanks for reviewing and testing the patch. If there are any failures or performance degradation relevant to the work, please let me know and I'll be very happy to keep improving it. Also, any suggestions about code improvements are well appreciated. > > I'm not quite sure if both G1 and Shenandoah have the similar region dependency issue, since I haven't studied their GC behaviors before. If they have, I'm also willing to propose a more general optimization. > > As to the memory overhead, I believe it will be low because this patch exploits empty regions in the young space rather than off-heap memory to allocate shadow regions, and also reuses the _source_region field of each RegionData to record the correspongding shadow region index. We only introduce a new integer filed _shadow in the RegionData class to indicate the status of a region, a global GrowableArray _free_shadow to store the indices of shadow regions, and a global Monitor to protect the array. These information might help if the memory overhead need to be evaluated. > > Looking forward to your insight. > > Best Regrads, > Haoyu Li, > Institute of Parallel and Distributed Systems(IPADS), > School of Software, > Shanghai Jiao Tong University > > > Kim Barrett > ?2019?3?12??? ??6:11??? > > On Mar 11, 2019, at 1:45 AM, Kim Barrett > wrote: > > > >> On Jan 24, 2019, at 3:58 AM, Haoyu Li > wrote: > >> > >> Hi Kim, > >> > >> I have ported my patch to OpenJDK 13 according to your instructions in your last mail, and the patch is attached in this mail. The patch does not change much since PSGC is indeed pretty stable. > >> > >> Also, I evaluate the correctness and performance of PS full GC with benchmarks from DaCapo, SPECjvm2008, and JOlden suits on a machine with dual Intel Xeon E5-2618L v3 CPUs(16 physical cores), 64G DRAM and linux kernel 4.17. The evaluation result, indicating 1.9X GC throughput improvement on average, is attached, too. > >> > >> However, I have no idea how to further test this patch for both correctness and performance. Can I please get any guidance from you or some sponsor? > > > > Sorry I missed that you had sent an updated version of the patch. > > > > I?ve run the full regression suite across Oracle-supported platforms. There are some > > failures, but there are almost always some failures in the later tiers right now. I?ll start > > looking at them tomorrow to figure out whether any of them are relevant. > > > > I?m also planning to run some of our performance benchmarks. > > > > I?ve lightly skimmed the proposed changes. There might be some code improvements > > to be made. > > > > I?m also wondering if this technique applies to other collectors. It seems like both G1 and > > Shenandoah full gc?s might have similar issues? If so, a solution that is ParallelGC-specific > > is less interesting than one that has broader applicability. Though maybe this optimization > > is less important for G1 and Shenandoah, since they actively try to avoid full gc?s. > > > > I?m also not clear on how much additional memory might be temporarily allocated by this > > mechanism. > > I?ve created a CR for this: https://bugs.openjdk.java.net/browse/JDK-8220465 > From kim.barrett at oracle.com Wed Sep 11 22:59:01 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 11 Sep 2019 18:59:01 -0400 Subject: RFR: 8230878: ZGC: Use thread_local instead of __thread In-Reply-To: <783cc8dc-c98d-a477-b08a-01911354c43d@oracle.com> References: <783cc8dc-c98d-a477-b08a-01911354c43d@oracle.com> Message-ID: > On Sep 11, 2019, at 4:48 PM, Per Liden wrote: > > We're currently using the gcc specific __thread syntax for thread local data. We should make those declarations compiler agnostic. > > This patch depends on the proposal to rename THREAD_LOCAL_DECL to thread_local (JDK-8230877), which is currently out for review on hotspot-dev. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8230878 > Webrev: http://cr.openjdk.java.net/~pliden/8230878/webrev.0 > > /Per Looks good. From kim.barrett at oracle.com Thu Sep 12 00:53:59 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 11 Sep 2019 20:53:59 -0400 Subject: RFR: 8230878: ZGC: Use thread_local instead of __thread In-Reply-To: References: <783cc8dc-c98d-a477-b08a-01911354c43d@oracle.com> Message-ID: <70CED89D-0500-4A93-8446-370F33976D7C@oracle.com> > On Sep 11, 2019, at 6:59 PM, Kim Barrett wrote: > >> On Sep 11, 2019, at 4:48 PM, Per Liden wrote: >> >> We're currently using the gcc specific __thread syntax for thread local data. We should make those declarations compiler agnostic. >> >> This patch depends on the proposal to rename THREAD_LOCAL_DECL to thread_local (JDK-8230877), which is currently out for review on hotspot-dev. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8230878 >> Webrev: http://cr.openjdk.java.net/~pliden/8230878/webrev.0 >> >> /Per > > Looks good. Retracting that, because of 8230877 discussion. From maoliang.ml at alibaba-inc.com Thu Sep 12 09:01:33 2019 From: maoliang.ml at alibaba-inc.com (Liang Mao) Date: Thu, 12 Sep 2019 17:01:33 +0800 Subject: =?UTF-8?B?UmU6IEcxIHBhdGNoIG9mIGVsYXN0aWMgSmF2YSBoZWFw?= In-Reply-To: <1267a5dd2cf6cc1d03df64d07a06ba0f45195951.camel@oracle.com> References: <6270ce59-4a8e-431e-9ccf-f6d2c0f927eb.maoliang.ml@alibaba-inc.com> , , <1267a5dd2cf6cc1d03df64d07a06ba0f45195951.camel@oracle.com> Message-ID: <3140197d-8cab-4a86-af92-58431c74cb6b.maoliang.ml@alibaba-inc.com> Hi Thomas, Sorry for some late to prepare the code. Now you can see the code in following webrev: http://cr.openjdk.java.net/~luchsh/elasticHeap/ It contains 2 patchs of hotspot and jdk against OpenJDK 8u222-ga. Most of the core source code is in src/share/vm/gc_implementation/g1/elasticHeap.cpp I want to explain some brief ideas of the patch: 1. G1ElasticHeap provides no addtional STW time to existent GC pauses. The memory commit/uncommit(mmap/munmap) will be executed in a concurrent thread because mmap/munmap costs significant time(more than 100ms easily on GB memory). 2. We provides 3 different evaluation modes to do the memory saving. a. Generation limit The young generation and old generation can be limited respectively by jcmd/MXBean. A typical scenario is that we have a lot of applications which have large young generation(default G1 is 60% as well) which is prepared for peak traffic. We don't need the large young generation memory all the time if not in a peak traffic. b. Periodic uncommit The young/old generation will be uncommit respectively based on young/old GC interval. The idea is similar to the original JEP 346 but treats young/old generation separately. For example, an application doesn't need to have large young generation all the time as described above. Periodic uncommit will save young generation memory to keep an acceptable young GC interval. c. Softmx It adjust the capacity of heap in runtime by jcmd/MXBean. 3.We still have some limitations that we need to make sure Xms=Xmx and save the memory in different policy. I made it a single patch so it may not be friendly for reading because of a lot of code and funcionalities. We can discuss a single feature like young generation uncommit or concurrent uncommit first. Please let me know if you want me to explain more details on some specific points. Thanks, Liang ------------------------------------------------------------------ From:Thomas Schatzl Send Time:2019 Jun. 19 (Wed.) 18:04 To:"MAO, Liang" ; hotspot-gc-dev Subject:Re: G1 patch of elastic Java heap Hi Liang, On Wed, 2019-06-19 at 17:38 +0800, ??(??) wrote: > Hi Thomas, > > Thank you for your quick reply. > As we implemented this feature in AlibabaJDK8, do you think if it is > feasible to just provide a CR based on JDK8? > So that you guys can have a look and then we can continue discussing > on the possible JEP and implementations in upstream. > > Otherwise we need spend time on preparing patches based on tip for > further discussion... > > BTW, I'm surely covered by the Alibaba OCA entry. for discussion a jdk8 patch/webrev is fine I guess. I would prefer a webrev because it can be referenced more easily though. Thanks, Thomas From hohensee at amazon.com Thu Sep 12 15:13:56 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 12 Sep 2019 15:13:56 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <23ED3BD7-3B7E-43DE-84B8-DB52D44D362B@amazon.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <23ED3BD7-3B7E-43DE-84B8-DB52D44D362B@amazon.com> Message-ID: Ping once more :) Need a confirmatory review to push this. If I understand the rules correctly, it doesn't need a Reviewer review since Mandy's already reviewed it, it just needs a Committer review. Thanks, Paul ?On 9/6/19, 11:08 AM, "hotspot-gc-dev on behalf of Hohensee, Paul" wrote: Ping. Anyone? ( Thanks, On 9/3/19, 12:39 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: Minor update in new webrev http://cr.openjdk.java.net/~phh/8207266/webrev.05/. I removed ensureNonNullThreadIds() in favor of Objects.requireNonNull(ids). Thanks, Mandy, for your through reviews. May I get another reviewer to weigh in? Paul On 8/31/19, 5:06 PM, "hotspot-gc-dev on behalf of Hohensee, Paul" wrote: Thanks, Mandy. I?ve finalized the CSR. New webrev at http://cr.openjdk.java.net/~phh/8207266/webrev.04/. In management.cpp, I now have if (THREAD->is_Java_thread()) { return ((JavaThread*)THREAD)->cooked_allocated_bytes(); } In ThreadImpl.java, using requireNonNull would produce a different and less informative message, so I?d like to leave it as is. I changed throwIfNullThreadIds to ensureNonNullThreadIds, and throwIfThreadAllocatedMemoryNotSupported to ensureThreadAllocatedMemorySupported. I dropped the ?java.lang.? prefix from all uses of UnsupportedOperationException in both c.s.m.ThreadMXBean.java and j.l.m.ThreadMXBean.java, and did the same with SecurityException. ?@since 14? added to c.s.m.ThreadMXBean.java and the CSR. Do I need another reviewer? Paul From: Mandy Chung Date: Friday, August 30, 2019 at 4:26 PM To: "Hohensee, Paul" Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread CSR reviewed. management.cpp 2083 java_thread = (JavaThread*)THREAD; 2084 if (java_thread->is_Java_thread()) { 2085 return java_thread->cooked_allocated_bytes(); 2086 } The cast should be done after is_Java_thread() test. ThreadImpl.java 162 private void throwIfNullThreadIds(long[] ids) { Even better: simply use Objects::requiresNonNull and this method can be removed. This suggests positive naming alternative to throwIfThreadAllocatedMemoryNotSupported - "ensureThreadAllocatedMemorySupported" (sorry I should have suggested that) ThreadMXBean.java 130 * @throws java.lang.UnsupportedOperationException if the Java virtual Nit: "java.lang." can be dropped. @since 14 is missing. Mandy On 8/30/19 3:33 PM, Hohensee, Paul wrote: Thanks for your review, Mandy. Revised webrev at http://cr.openjdk.java.net/~phh/8207266/webrev.02/. I updated the CSR with your suggested javadoc for getCurrentThreadAllocatedBytes. It now matches that for getCurrentThreadUserTime and getCurrentThreadCputime. I also fixed the ?convenient? -> ?convenience? typos in j.l.m.ThreadMXBean.java. I meant GetOneThreads to be the possessive, but don?t feel strongly either way so I?m fine with GetOneThread. I updated ThreadImpl.java as you suggested, though in getThreadAllocatedBytes(long[] ids) I had to add a redundant-in-the-not-length-1-case check for a null ids reference. Would someone take a look at the Hotspot side and the test please? Paul From: Mandy Chung Date: Friday, August 30, 2019 at 10:22 AM To: "Hohensee, Paul" Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread OK. That's better. Some review comments: The javadoc of getCurrentThreadAllocatedBytes() can simply say: "Returns an approximation of the total amount of memory, in bytes, allocated in heap memory for the current thread. This is a convenient method for local management use and is equivalent to calling getThreadAllocatedBytes(Thread.currentThread().getId()). src/hotspot/share/include/jmm.h GetOneThreadsAllocatedMemory: s/OneThreads/OneThread/ sun/management/ThreadImpl.java 43 private static final String THREAD_ALLOCATED_MEMORY_NOT_SUPPORTED = 44 "Thread allocated memory measurement is not supported."; if (!isThreadAllocatedMemorySupported()) { throw new UnsupportedOperationException(THREAD_ALLOCATED_MEMORY_NOT_SUPPORTED); } Perhaps the above can be refactored as throwIfAllocatedMemoryUnsupported() method. 391 if (ids.length == 1) { 392 sizes[0] = -1; : 398 if (ids.length == 1) { 399 long id = ids[0]; 400 sizes[0] = getThreadAllocatedMemory0( 401 Thread.currentThread().getId() == id ? 0 : id); 402 } else { It seems cleaner to handle the 1-element array case at the beginning of this method: if (ids.length == 1) { long size = getThreadAllocatedBytes(ids[0]); return new long[] { size }; } I didn't review the hotspot implementation and the test. Mandy On 8/29/19 10:01 AM, Hohensee, Paul wrote: My bad, Mandy. The webrev puts getCurrentThreadAllocatedBytes in com.sun.management.ThreadMXBean along with the current two getThreadAllocatedBytes methods for the reasons you list. I?ve updated the CSR to specify com.sun.management and added a rationale. AllocatedBytes is currently enabled by Hotspot by default because the overhead of recording TLAB occupancy is negligible. There?s no new GC code, nor will there be, so imo we don?t have to involve the GC folks. I.e., the new JMM method GetOneThreadsAllocatedBytes uses the existing cooked_allocated_bytes JavaThread method, and getCurrentThreadAllocatedBytes is the same as getThreadAllocatedBytes: it just bypasses the thread lookup code. I hadn?t tracked down what happens when getCurrentThreadUserTime and getCurrentThreadCpuTime are called before, but if I?m not mistaken, it the code in jcmd() in attachListener.cpp will call GetThreadCpuTimeWithKind in management.cpp, and it will ultimately use Thread::current() as the subject of the call, see os::current_thread_cpu_time in os_linux.cpp. That means that the CurrentThread methods should work remotely the same way they do locally. GetOneThreadsAllocatedBytes in management.cpp uses THREAD as its subject when called on behalf of getCurrentThreadAllocatedBytes, so it will also uses the current remote Java thread. Even if these methods only worked locally, there are many setups where apps are self-monitoring that could use the performance improvement. Thanks, Paul From: Mandy Chung Date: Wednesday, August 28, 2019 at 3:59 PM To: "Hohensee, Paul" Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread Hi Paul, The CSR proposes this method in java.lang.management.ThreadMXBean as a Java SE feature. Has this been discussed with the GC team to commit measuring current thread's allocated bytes as Java SE feature? Can this be supported by all JVM implementation? What is the overhead if this is enabled by default? Does it need to be disabled? This metric is from TLAB that might be okay. This needs advice/discussion with GC experts. I see that CSR mentions it can be disabled and link to isThreadAllocatedMemoryEnabled() and setThreadAllocatedMemoryEnabled() methods but these methods are defined in com.sun.management.ThreadMXBean. As Alan points out, current thread makes sense only in local VM management. When this is monitored from a JMX client (e.g. jconsole to connect to a running JVM, "currentThreadAllowcatedBytes" attribute is the current thread in jconsole process which invoking Thread::currentThread? Mandy On 8/28/19 12:22 PM, Hohensee, Paul wrote: Please review a performance improvement for ThreadMXBean.getThreadAllocatedBytes and the addition of getCurrentThreadAllocatedBytes. JBS issue: https://bugs.openjdk.java.net/browse/JDK-8207266 Webrev: http://cr.openjdk.java.net/~phh/8207266/webrev.00/ CSR: https://bugs.openjdk.java.net/browse/JDK-8230311 Previous email threads: https://mail.openjdk.java.net/pipermail/serviceability-dev/2018-July/024441.html https://mail.openjdk.java.net/pipermail/serviceability-dev/2018-August/024763.html The CSR is for adding ThreadMXBean.getCurrentThreadAllocatedBytes. I?d be great for someone to review it. I took Mandy?s advice and put the fast paths in the library code. I added a new JMM method GetOneThreadsAllocatedBytes that works the same as GetThreadCpuTime: it uses a thread_id value of zero to distinguish the current thread. On my Mac laptop, the result runs 47x faster for the current thread than the old implementation. The 3 tests in test/jdk/com/sun/management/ThreadMXBean all pass. I added code to ThreadAllocatedMemory.java to test getCurrentThreadAllocatedBytes as well as variations on getThreadAllocatedBytes(id). A submit repo job is in progress. Thanks, Paul From mandy.chung at oracle.com Thu Sep 12 17:06:48 2019 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 12 Sep 2019 10:06:48 -0700 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> Message-ID: <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> On 9/3/19 12:38 PM, Hohensee, Paul wrote: > Minor update in new webrev http://cr.openjdk.java.net/~phh/8207266/webrev.05/. I only reviewed the library side implementation that looks good.? I expect the serviceability team to review the test and hotspot change. > Need a confirmatory review to push this. If I understand the rules correctly, it doesn't need a Reviewer review since Mandy's already reviewed it, it just needs a Committer review. You need another reviewer to advice the following because I was not close to the ThreadsList work. 2087 ThreadsListHandle tlh; 2088 JavaThread* java_thread = tlh.list()->find_JavaThread_from_java_tid(thread_id); 2089 2090 if (java_thread != NULL) { 2091 return java_thread->cooked_allocated_bytes(); 2092 } This looks right to me. test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java - "ThreadAllocatedMemory is expected to be disabled"); + "TEST FAILED: ThreadAllocatedMemory is expected to be disabled"); Prepending "TEST FAILED" in exception message (in several places) seems redundant since such RuntimeException is thrown and expected a test failure. + // back-to-back calls shouldn't allocate any memory + size = mbean.getThreadAllocatedBytes(id); + size1 = mbean.getThreadAllocatedBytes(id); + if (size1 != size) { Is there anything in the test can do to help guarantee this? I didn't closely review this test. The main thing I advice is to improve the reliability of this test. Put it in another way, we want to ensure that this test change will pass all the time in various test configuration. Mandy From per.liden at oracle.com Thu Sep 12 18:54:13 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 12 Sep 2019 20:54:13 +0200 Subject: RFR: 8230878: ZGC: Use thread_local instead of __thread In-Reply-To: <70CED89D-0500-4A93-8446-370F33976D7C@oracle.com> References: <783cc8dc-c98d-a477-b08a-01911354c43d@oracle.com> <70CED89D-0500-4A93-8446-370F33976D7C@oracle.com> Message-ID: <02b465e4-28b2-7808-4842-b85000c73a6a@oracle.com> On 9/12/19 2:53 AM, Kim Barrett wrote: >> On Sep 11, 2019, at 6:59 PM, Kim Barrett wrote: >> >>> On Sep 11, 2019, at 4:48 PM, Per Liden wrote: >>> >>> We're currently using the gcc specific __thread syntax for thread local data. We should make those declarations compiler agnostic. >>> >>> This patch depends on the proposal to rename THREAD_LOCAL_DECL to thread_local (JDK-8230877), which is currently out for review on hotspot-dev. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8230878 >>> Webrev: http://cr.openjdk.java.net/~pliden/8230878/webrev.0 >>> >>> /Per >> >> Looks good. > > Retracting that, because of 8230877 discussion. Updated webrev, using THREAD_LOCAL, as discussed in the JDK-8230877 review thread. http://cr.openjdk.java.net/~pliden/8230878/webrev.1 /Per From kim.barrett at oracle.com Thu Sep 12 19:35:51 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 12 Sep 2019 15:35:51 -0400 Subject: RFR: 8230878: ZGC: Use thread_local instead of __thread In-Reply-To: <02b465e4-28b2-7808-4842-b85000c73a6a@oracle.com> References: <783cc8dc-c98d-a477-b08a-01911354c43d@oracle.com> <70CED89D-0500-4A93-8446-370F33976D7C@oracle.com> <02b465e4-28b2-7808-4842-b85000c73a6a@oracle.com> Message-ID: <6E341F7F-1C1F-439D-A895-4801434604B1@oracle.com> > On Sep 12, 2019, at 2:54 PM, Per Liden wrote: > > On 9/12/19 2:53 AM, Kim Barrett wrote: >>> On Sep 11, 2019, at 6:59 PM, Kim Barrett wrote: >>> >>>> On Sep 11, 2019, at 4:48 PM, Per Liden wrote: >>>> >>>> We're currently using the gcc specific __thread syntax for thread local data. We should make those declarations compiler agnostic. >>>> >>>> This patch depends on the proposal to rename THREAD_LOCAL_DECL to thread_local (JDK-8230877), which is currently out for review on hotspot-dev. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8230878 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8230878/webrev.0 >>>> >>>> /Per >>> >>> Looks good. >> Retracting that, because of 8230877 discussion. > > Updated webrev, using THREAD_LOCAL, as discussed in the JDK-8230877 review thread. > > http://cr.openjdk.java.net/~pliden/8230878/webrev.1 > > /Per Looks good. From per.liden at oracle.com Thu Sep 12 19:48:34 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 12 Sep 2019 21:48:34 +0200 Subject: RFR: 8230878: ZGC: Use thread_local instead of __thread In-Reply-To: <6E341F7F-1C1F-439D-A895-4801434604B1@oracle.com> References: <783cc8dc-c98d-a477-b08a-01911354c43d@oracle.com> <70CED89D-0500-4A93-8446-370F33976D7C@oracle.com> <02b465e4-28b2-7808-4842-b85000c73a6a@oracle.com> <6E341F7F-1C1F-439D-A895-4801434604B1@oracle.com> Message-ID: On 9/12/19 9:35 PM, Kim Barrett wrote: >> On Sep 12, 2019, at 2:54 PM, Per Liden wrote: >> >> On 9/12/19 2:53 AM, Kim Barrett wrote: >>>> On Sep 11, 2019, at 6:59 PM, Kim Barrett wrote: >>>> >>>>> On Sep 11, 2019, at 4:48 PM, Per Liden wrote: >>>>> >>>>> We're currently using the gcc specific __thread syntax for thread local data. We should make those declarations compiler agnostic. >>>>> >>>>> This patch depends on the proposal to rename THREAD_LOCAL_DECL to thread_local (JDK-8230877), which is currently out for review on hotspot-dev. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8230878 >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8230878/webrev.0 >>>>> >>>>> /Per >>>> >>>> Looks good. >>> Retracting that, because of 8230877 discussion. >> >> Updated webrev, using THREAD_LOCAL, as discussed in the JDK-8230877 review thread. >> >> http://cr.openjdk.java.net/~pliden/8230878/webrev.1 >> >> /Per > > Looks good. > Thanks Kim! /Per From hohensee at amazon.com Fri Sep 13 00:29:49 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 13 Sep 2019 00:29:49 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> Message-ID: Thanks for clarifying the review rules. Would someone from the serviceability team please review? New webrev at http://cr.openjdk.java.net/~phh/8207266/webrev.07/ I didn?t disturb the existing checks in the test, just added code to check the result of getThreadAllocatedBytes(long) on a non-current thread, plus the back-to-back no-allocation checks. The former wasn?t needed before because getThreadAllocatedBytes(long) was just a wrapper around getThreadAllocatedBytes(long []). This patch changes that, so I added a separate test. The latter is supposed to fail if there?s object allocation on calls to getCurrentThreadAllocatedBytes and getThreadAllocatedBytes(long). I.e., a feature, not a bug, because accumulation of transient small objects can be a performance problem. Thanks to your review, I noticed that the back-to-back check on the current thread was using getThreadAllocatedBytes(long) instead of getCurrentThreadAllocatedBytes and fixed it. I also removed all instances of ?TEST FAILED: ?. Paul From: Mandy Chung Date: Thursday, September 12, 2019 at 10:09 AM To: "Hohensee, Paul" Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread On 9/3/19 12:38 PM, Hohensee, Paul wrote: Minor update in new webrev http://cr.openjdk.java.net/~phh/8207266/webrev.05/. I only reviewed the library side implementation that looks good. I expect the serviceability team to review the test and hotspot change. Need a confirmatory review to push this. If I understand the rules correctly, it doesn't need a Reviewer review since Mandy's already reviewed it, it just needs a Committer review. You need another reviewer to advice the following because I was not close to the ThreadsList work. 2087 ThreadsListHandle tlh; 2088 JavaThread* java_thread = tlh.list()->find_JavaThread_from_java_tid(thread_id); 2089 2090 if (java_thread != NULL) { 2091 return java_thread->cooked_allocated_bytes(); 2092 } This looks right to me. test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java - "ThreadAllocatedMemory is expected to be disabled"); + "TEST FAILED: ThreadAllocatedMemory is expected to be disabled"); Prepending "TEST FAILED" in exception message (in several places) seems redundant since such RuntimeException is thrown and expected a test failure. + // back-to-back calls shouldn't allocate any memory + size = mbean.getThreadAllocatedBytes(id); + size1 = mbean.getThreadAllocatedBytes(id); + if (size1 != size) { Is there anything in the test can do to help guarantee this? I didn't closely review this test. The main thing I advice is to improve the reliability of this test. Put it in another way, we want to ensure that this test change will pass all the time in various test configuration. Mandy From hohensee at amazon.com Fri Sep 13 01:52:50 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 13 Sep 2019 01:52:50 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> Message-ID: And of course the back-to-back check is more or less accurate depending on the accuracy of the underlying Hotspot mechanism. So it?s possible (indeed likely with the current TLAB refill interval update) that it?ll return false negatives, but imo better to keep it anyway. From: "Hohensee, Paul" Date: Thursday, September 12, 2019 at 5:29 PM To: Mandy Chung Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread Thanks for clarifying the review rules. Would someone from the serviceability team please review? New webrev at http://cr.openjdk.java.net/~phh/8207266/webrev.07/ I didn?t disturb the existing checks in the test, just added code to check the result of getThreadAllocatedBytes(long) on a non-current thread, plus the back-to-back no-allocation checks. The former wasn?t needed before because getThreadAllocatedBytes(long) was just a wrapper around getThreadAllocatedBytes(long []). This patch changes that, so I added a separate test. The latter is supposed to fail if there?s object allocation on calls to getCurrentThreadAllocatedBytes and getThreadAllocatedBytes(long). I.e., a feature, not a bug, because accumulation of transient small objects can be a performance problem. Thanks to your review, I noticed that the back-to-back check on the current thread was using getThreadAllocatedBytes(long) instead of getCurrentThreadAllocatedBytes and fixed it. I also removed all instances of ?TEST FAILED: ?. Paul From: Mandy Chung Date: Thursday, September 12, 2019 at 10:09 AM To: "Hohensee, Paul" Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread On 9/3/19 12:38 PM, Hohensee, Paul wrote: Minor update in new webrev http://cr.openjdk.java.net/~phh/8207266/webrev.05/. I only reviewed the library side implementation that looks good. I expect the serviceability team to review the test and hotspot change. Need a confirmatory review to push this. If I understand the rules correctly, it doesn't need a Reviewer review since Mandy's already reviewed it, it just needs a Committer review. You need another reviewer to advice the following because I was not close to the ThreadsList work. 2087 ThreadsListHandle tlh; 2088 JavaThread* java_thread = tlh.list()->find_JavaThread_from_java_tid(thread_id); 2089 2090 if (java_thread != NULL) { 2091 return java_thread->cooked_allocated_bytes(); 2092 } This looks right to me. test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java - "ThreadAllocatedMemory is expected to be disabled"); + "TEST FAILED: ThreadAllocatedMemory is expected to be disabled"); Prepending "TEST FAILED" in exception message (in several places) seems redundant since such RuntimeException is thrown and expected a test failure. + // back-to-back calls shouldn't allocate any memory + size = mbean.getThreadAllocatedBytes(id); + size1 = mbean.getThreadAllocatedBytes(id); + if (size1 != size) { Is there anything in the test can do to help guarantee this? I didn't closely review this test. The main thing I advice is to improve the reliability of this test. Put it in another way, we want to ensure that this test change will pass all the time in various test configuration. Mandy From david.holmes at oracle.com Fri Sep 13 07:50:27 2019 From: david.holmes at oracle.com (David Holmes) Date: Fri, 13 Sep 2019 17:50:27 +1000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> Message-ID: <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> Hi Paul, On 13/09/2019 10:29 am, Hohensee, Paul wrote: > Thanks for clarifying the review rules. Would someone from the > serviceability team please review? New webrev at > > http://cr.openjdk.java.net/~phh/8207266/webrev.07/ One aspect of the functional change needs clarification for me - and apologies if this has been covered in the past. It seems to me that currently we only check isThreadAllocatedMemorySupported for these operations, but if I read things correctly the updated code additionally checks isThreadAllocatedMemoryEnabled, which is a behaviour change not mentioned in the CSR. > I didn?t disturb the existing checks in the test, just added code to > check the result of getThreadAllocatedBytes(long) on a non-current > thread, plus the back-to-back no-allocation checks. The former wasn?t > needed before because getThreadAllocatedBytes(long) was just a wrapper > around getThreadAllocatedBytes(long []). This patch changes that, so I > added a separate test. The latter is supposed to fail if there?s object > allocation on calls to getCurrentThreadAllocatedBytes and > getThreadAllocatedBytes(long). I.e., a feature, not a bug, because > accumulation of transient small objects can be a performance problem. > Thanks to your review, I noticed that the back-to-back check on the > current thread was using getThreadAllocatedBytes(long) instead of > getCurrentThreadAllocatedBytes and fixed it. I also removed all > instances of ?TEST FAILED: ?. The back-to-back check is not valid in general. You don't know if the first check might trigger some class loading on the return path after it has obtained the first memory value. The check might also fail if using JVMCI and some compilation related activity occurs in the current thread on the second call. Also with the introduction of handshakes its possible the current thread might hit a safepoint checks that results in it executing a handshake operation that performs allocation. Potentially there could be numerous non-deterministic actions that might occur leading to unanticipated allocation. I understand what you want to test here, I just don't think it is reliably doable. Thanks, David ----- > > Paul > > *From: *Mandy Chung > *Date: *Thursday, September 12, 2019 at 10:09 AM > *To: *"Hohensee, Paul" > *Cc: *OpenJDK Serviceability , > "hotspot-gc-dev at openjdk.java.net" > *Subject: *Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() > can be quicker for self thread > > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > > Minor update in new webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > > > I only reviewed the library side implementation that looks good.? I > expect the serviceability team to review the test and hotspot change. > > > Need a confirmatory review to push this. If I understand the rules correctly, it doesn't need a Reviewer review since Mandy's already reviewed it, it just needs a Committer review. > > > You need another reviewer to advice the following because I was not > close to the ThreadsList work. > > 2087?? ThreadsListHandle tlh; > > 2088?? JavaThread* java_thread = tlh.list()->find_JavaThread_from_java_tid(thread_id); > > 2089 > > 2090?? if (java_thread != NULL) { > > 2091???? return java_thread->cooked_allocated_bytes(); > > 2092?? } > > This looks right to me. > > test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > > -??????????????? "ThreadAllocatedMemory is expected to be disabled"); > > +??????????????? "TEST FAILED: ThreadAllocatedMemory is expected to be > disabled"); > > Prepending "TEST FAILED" in exception message (in several places) > > seems redundant since such RuntimeException is thrown and expected > > a test failure. > > +??????? // back-to-back calls shouldn't allocate any memory > > +??????? size = mbean.getThreadAllocatedBytes(id); > > +??????? size1 = mbean.getThreadAllocatedBytes(id); > > +??????? if (size1 != size) { > > Is there anything in the test can do to help guarantee this? I didn't > > closely review this test.? The main thing I advice is to improve > > the reliability of this test.? Put it in another way, we want to > > ensure that this test change will pass all the time in various > > test configuration. > > Mandy > From thomas.schatzl at oracle.com Fri Sep 13 11:29:43 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 13 Sep 2019 13:29:43 +0200 Subject: RFR: 8230796: Remove BarrierSet::oop_equals_operator_allowed() In-Reply-To: <0e93041c-6f79-89c9-3c41-008fc33fa7a9@oracle.com> References: <0e93041c-6f79-89c9-3c41-008fc33fa7a9@oracle.com> Message-ID: <9a653039-2335-26ba-0c3d-f0c58a3636d6@oracle.com> Hi, On 10.09.19 12:47, Per Liden wrote: > BarrierSet::oop_equals_operator_allowed() is no longer specialized by > any barrier set and can be removed. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8230796 > Webrev: http://cr.openjdk.java.net/~pliden/8230796/webrev.0 > > /Per looks good. Thomas From thomas.schatzl at oracle.com Fri Sep 13 11:42:24 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 13 Sep 2019 13:42:24 +0200 Subject: RFR: 8230808: Remove Access::equals() In-Reply-To: References: Message-ID: <7ae7afed-94d9-3e5f-7168-049aeb29a765@oracle.com> Hi, On 10.09.19 15:27, Per Liden wrote: > Access::equals() is not specialized by any barrier set and can be > removed. This patch doesn't remove oopDesc::equals()/equals_raw(), but > they don't use Access::equals() to do the work anymore. Removing > oopDesc::equals() is a more intrusive change, which can be done as the > next step. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8230808 > Webrev: http://cr.openjdk.java.net/~pliden/8230808/webrev.0 > > /Per looks good to me. Thomas From thomas.schatzl at oracle.com Fri Sep 13 11:50:14 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 13 Sep 2019 13:50:14 +0200 Subject: RFR: 8230841: Remove oopDesc::equals() In-Reply-To: <9946de4d-8001-e1e3-fa80-633280f31483@oracle.com> References: <9946de4d-8001-e1e3-fa80-633280f31483@oracle.com> Message-ID: Hi, On 11.09.19 09:35, Per Liden wrote: > With the removal of Access::equals() in JDK-8230808, there's no longer > any need to treat oop equality in a special way. Therefore, we can > remove oopDesc::equals() and oopDesc::equals_raw(). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8230841 > Webrev: http://cr.openjdk.java.net/~pliden/8230841/webrev.0 > > Testing: Currently running through tier1-3 in all platforms > > /Per looks good. Thomas From per.liden at oracle.com Fri Sep 13 11:54:29 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 13 Sep 2019 13:54:29 +0200 Subject: RFR: 8230841: Remove oopDesc::equals() In-Reply-To: References: <9946de4d-8001-e1e3-fa80-633280f31483@oracle.com> Message-ID: Thanks Thomas! /Per On 9/13/19 1:50 PM, Thomas Schatzl wrote: > Hi, > > On 11.09.19 09:35, Per Liden wrote: >> With the removal of Access::equals() in JDK-8230808, there's no longer >> any need to treat oop equality in a special way. Therefore, we can >> remove oopDesc::equals() and oopDesc::equals_raw(). >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8230841 >> Webrev: http://cr.openjdk.java.net/~pliden/8230841/webrev.0 >> >> Testing: Currently running through tier1-3 in all platforms >> >> /Per > > ? looks good. > > Thomas From per.liden at oracle.com Fri Sep 13 11:55:03 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 13 Sep 2019 13:55:03 +0200 Subject: RFR: 8230808: Remove Access::equals() In-Reply-To: <7ae7afed-94d9-3e5f-7168-049aeb29a765@oracle.com> References: <7ae7afed-94d9-3e5f-7168-049aeb29a765@oracle.com> Message-ID: <1306aba1-2312-456d-a5c1-d4f3c5abaf58@oracle.com> Thanks Thomas! /Per On 9/13/19 1:42 PM, Thomas Schatzl wrote: > Hi, > > On 10.09.19 15:27, Per Liden wrote: >> Access::equals() is not specialized by any barrier set and can be >> removed. This patch doesn't remove oopDesc::equals()/equals_raw(), but >> they don't use Access::equals() to do the work anymore. Removing >> oopDesc::equals() is a more intrusive change, which can be done as the >> next step. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8230808 >> Webrev: http://cr.openjdk.java.net/~pliden/8230808/webrev.0 >> >> /Per > > ? looks good to me. > > Thomas From per.liden at oracle.com Fri Sep 13 11:54:47 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 13 Sep 2019 13:54:47 +0200 Subject: RFR: 8230796: Remove BarrierSet::oop_equals_operator_allowed() In-Reply-To: <9a653039-2335-26ba-0c3d-f0c58a3636d6@oracle.com> References: <0e93041c-6f79-89c9-3c41-008fc33fa7a9@oracle.com> <9a653039-2335-26ba-0c3d-f0c58a3636d6@oracle.com> Message-ID: <9260cd64-bc37-92ea-6462-e766516c3aac@oracle.com> Thanks Thomas! /Per On 9/13/19 1:29 PM, Thomas Schatzl wrote: > Hi, > > On 10.09.19 12:47, Per Liden wrote: >> BarrierSet::oop_equals_operator_allowed() is no longer specialized by >> any barrier set and can be removed. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8230796 >> Webrev: http://cr.openjdk.java.net/~pliden/8230796/webrev.0 >> >> /Per > > ? looks good. > > Thomas From hohensee at amazon.com Fri Sep 13 19:11:30 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Fri, 13 Sep 2019 19:11:30 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> Message-ID: <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> Hi David, thanks for your comments. New webrev in http://cr.openjdk.java.net/~phh/8207266/webrev.08/ Both the old and new versions of the code check that thread allocated memory is both supported and enabled. The existing version of getThreadAllocatedBytes(long []) calls verifyThreadAllocatedMemory(long []), which checks inline to make sure thread allocated memory is supported, then calls isThreadAllocatedMemoryEnabled() to verify that it's enabled. isThreadAllocatedMemoryEnabled() duplicates (!) the support check and returns the enabled flag. I removed the redundant check in the new version. You're of course correct about the back-to-back check. Application code can't know when the runtime will hijack a thread for its own purposes. I've removed the check. Paul ?On 9/13/19, 12:50 AM, "David Holmes" wrote: Hi Paul, On 13/09/2019 10:29 am, Hohensee, Paul wrote: > Thanks for clarifying the review rules. Would someone from the > serviceability team please review? New webrev at > > http://cr.openjdk.java.net/~phh/8207266/webrev.07/ One aspect of the functional change needs clarification for me - and apologies if this has been covered in the past. It seems to me that currently we only check isThreadAllocatedMemorySupported for these operations, but if I read things correctly the updated code additionally checks isThreadAllocatedMemoryEnabled, which is a behaviour change not mentioned in the CSR. > I didn?t disturb the existing checks in the test, just added code to > check the result of getThreadAllocatedBytes(long) on a non-current > thread, plus the back-to-back no-allocation checks. The former wasn?t > needed before because getThreadAllocatedBytes(long) was just a wrapper > around getThreadAllocatedBytes(long []). This patch changes that, so I > added a separate test. The latter is supposed to fail if there?s object > allocation on calls to getCurrentThreadAllocatedBytes and > getThreadAllocatedBytes(long). I.e., a feature, not a bug, because > accumulation of transient small objects can be a performance problem. > Thanks to your review, I noticed that the back-to-back check on the > current thread was using getThreadAllocatedBytes(long) instead of > getCurrentThreadAllocatedBytes and fixed it. I also removed all > instances of ?TEST FAILED: ?. The back-to-back check is not valid in general. You don't know if the first check might trigger some class loading on the return path after it has obtained the first memory value. The check might also fail if using JVMCI and some compilation related activity occurs in the current thread on the second call. Also with the introduction of handshakes its possible the current thread might hit a safepoint checks that results in it executing a handshake operation that performs allocation. Potentially there could be numerous non-deterministic actions that might occur leading to unanticipated allocation. I understand what you want to test here, I just don't think it is reliably doable. Thanks, David ----- > > Paul > > *From: *Mandy Chung > *Date: *Thursday, September 12, 2019 at 10:09 AM > *To: *"Hohensee, Paul" > *Cc: *OpenJDK Serviceability , > "hotspot-gc-dev at openjdk.java.net" > *Subject: *Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() > can be quicker for self thread > > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > > Minor update in new webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > > > I only reviewed the library side implementation that looks good. I > expect the serviceability team to review the test and hotspot change. > > > Need a confirmatory review to push this. If I understand the rules correctly, it doesn't need a Reviewer review since Mandy's already reviewed it, it just needs a Committer review. > > > You need another reviewer to advice the following because I was not > close to the ThreadsList work. > > 2087 ThreadsListHandle tlh; > > 2088 JavaThread* java_thread = tlh.list()->find_JavaThread_from_java_tid(thread_id); > > 2089 > > 2090 if (java_thread != NULL) { > > 2091 return java_thread->cooked_allocated_bytes(); > > 2092 } > > This looks right to me. > > test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > > - "ThreadAllocatedMemory is expected to be disabled"); > > + "TEST FAILED: ThreadAllocatedMemory is expected to be > disabled"); > > Prepending "TEST FAILED" in exception message (in several places) > > seems redundant since such RuntimeException is thrown and expected > > a test failure. > > + // back-to-back calls shouldn't allocate any memory > > + size = mbean.getThreadAllocatedBytes(id); > > + size1 = mbean.getThreadAllocatedBytes(id); > > + if (size1 != size) { > > Is there anything in the test can do to help guarantee this? I didn't > > closely review this test. The main thing I advice is to improve > > the reliability of this test. Put it in another way, we want to > > ensure that this test change will pass all the time in various > > test configuration. > > Mandy > From david.holmes at oracle.com Fri Sep 13 22:34:38 2019 From: david.holmes at oracle.com (David Holmes) Date: Sat, 14 Sep 2019 08:34:38 +1000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> Message-ID: <28a1cc57-dee3-7beb-19ef-d434ddf038ca@oracle.com> Hi Paul, On 14/09/2019 5:11 am, Hohensee, Paul wrote: > Hi David, thanks for your comments. New webrev in > > http://cr.openjdk.java.net/~phh/8207266/webrev.08/ > > Both the old and new versions of the code check that thread allocated memory is both supported and enabled. The existing version of getThreadAllocatedBytes(long []) calls verifyThreadAllocatedMemory(long []), which checks inline to make sure thread allocated memory is supported, then calls isThreadAllocatedMemoryEnabled() to verify that it's enabled. isThreadAllocatedMemoryEnabled() duplicates (!) the support check and returns the enabled flag. I removed the redundant check in the new version. Thanks for clarifying. > You're of course correct about the back-to-back check. Application code can't know when the runtime will hijack a thread for its own purposes. I've removed the check. Updated test looks fine. Nothing further from me. Thanks, David ----- > Paul > > ?On 9/13/19, 12:50 AM, "David Holmes" wrote: > > Hi Paul, > > On 13/09/2019 10:29 am, Hohensee, Paul wrote: > > Thanks for clarifying the review rules. Would someone from the > > serviceability team please review? New webrev at > > > > http://cr.openjdk.java.net/~phh/8207266/webrev.07/ > > One aspect of the functional change needs clarification for me - and > apologies if this has been covered in the past. It seems to me that > currently we only check isThreadAllocatedMemorySupported for these > operations, but if I read things correctly the updated code additionally > checks isThreadAllocatedMemoryEnabled, which is a behaviour change not > mentioned in the CSR. > > > I didn?t disturb the existing checks in the test, just added code to > > check the result of getThreadAllocatedBytes(long) on a non-current > > thread, plus the back-to-back no-allocation checks. The former wasn?t > > needed before because getThreadAllocatedBytes(long) was just a wrapper > > around getThreadAllocatedBytes(long []). This patch changes that, so I > > added a separate test. The latter is supposed to fail if there?s object > > allocation on calls to getCurrentThreadAllocatedBytes and > > getThreadAllocatedBytes(long). I.e., a feature, not a bug, because > > accumulation of transient small objects can be a performance problem. > > Thanks to your review, I noticed that the back-to-back check on the > > current thread was using getThreadAllocatedBytes(long) instead of > > getCurrentThreadAllocatedBytes and fixed it. I also removed all > > instances of ?TEST FAILED: ?. > > The back-to-back check is not valid in general. You don't know if the > first check might trigger some class loading on the return path after it > has obtained the first memory value. The check might also fail if using > JVMCI and some compilation related activity occurs in the current thread > on the second call. Also with the introduction of handshakes its > possible the current thread might hit a safepoint checks that results in > it executing a handshake operation that performs allocation. Potentially > there could be numerous non-deterministic actions that might occur > leading to unanticipated allocation. > > I understand what you want to test here, I just don't think it is > reliably doable. > > Thanks, > David > ----- > > > > > Paul > > > > *From: *Mandy Chung > > *Date: *Thursday, September 12, 2019 at 10:09 AM > > *To: *"Hohensee, Paul" > > *Cc: *OpenJDK Serviceability , > > "hotspot-gc-dev at openjdk.java.net" > > *Subject: *Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() > > can be quicker for self thread > > > > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > > > > Minor update in new webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > > > > > > I only reviewed the library side implementation that looks good. I > > expect the serviceability team to review the test and hotspot change. > > > > > > Need a confirmatory review to push this. If I understand the rules correctly, it doesn't need a Reviewer review since Mandy's already reviewed it, it just needs a Committer review. > > > > > > You need another reviewer to advice the following because I was not > > close to the ThreadsList work. > > > > 2087 ThreadsListHandle tlh; > > > > 2088 JavaThread* java_thread = tlh.list()->find_JavaThread_from_java_tid(thread_id); > > > > 2089 > > > > 2090 if (java_thread != NULL) { > > > > 2091 return java_thread->cooked_allocated_bytes(); > > > > 2092 } > > > > This looks right to me. > > > > test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > > > > - "ThreadAllocatedMemory is expected to be disabled"); > > > > + "TEST FAILED: ThreadAllocatedMemory is expected to be > > disabled"); > > > > Prepending "TEST FAILED" in exception message (in several places) > > > > seems redundant since such RuntimeException is thrown and expected > > > > a test failure. > > > > + // back-to-back calls shouldn't allocate any memory > > > > + size = mbean.getThreadAllocatedBytes(id); > > > > + size1 = mbean.getThreadAllocatedBytes(id); > > > > + if (size1 != size) { > > > > Is there anything in the test can do to help guarantee this? I didn't > > > > closely review this test. The main thing I advice is to improve > > > > the reliability of this test. Put it in another way, we want to > > > > ensure that this test change will pass all the time in various > > > > test configuration. > > > > Mandy > > > > From serguei.spitsyn at oracle.com Sat Sep 14 00:48:51 2019 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 13 Sep 2019 17:48:51 -0700 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> Message-ID: <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> Hi Paul, It looks pretty good in general. http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html It would be nice to refactor the java main() method as it becomes too big. Two ways ofgetCurrentThreadAllocatedBytes() testing are good candidates to become separate methods. 98 long size1 = mbean.getThreadAllocatedBytes(id); Just wanted to double check if you wanted to invoke the getCurrentThreadAllocatedBytes() instead as it is a part of: 85 // First way, getCurrentThreadAllocatedBytes Thanks, Serguei On 9/13/19 12:11 PM, Hohensee, Paul wrote: > Hi David, thanks for your comments. New webrev in > > http://cr.openjdk.java.net/~phh/8207266/webrev.08/ > > Both the old and new versions of the code check that thread allocated memory is both supported and enabled. The existing version of getThreadAllocatedBytes(long []) calls verifyThreadAllocatedMemory(long []), which checks inline to make sure thread allocated memory is supported, then calls isThreadAllocatedMemoryEnabled() to verify that it's enabled. isThreadAllocatedMemoryEnabled() duplicates (!) the support check and returns the enabled flag. I removed the redundant check in the new version. > > You're of course correct about the back-to-back check. Application code can't know when the runtime will hijack a thread for its own purposes. I've removed the check. > > Paul > > ?On 9/13/19, 12:50 AM, "David Holmes" wrote: > > Hi Paul, > > On 13/09/2019 10:29 am, Hohensee, Paul wrote: > > Thanks for clarifying the review rules. Would someone from the > > serviceability team please review? New webrev at > > > > http://cr.openjdk.java.net/~phh/8207266/webrev.07/ > > One aspect of the functional change needs clarification for me - and > apologies if this has been covered in the past. It seems to me that > currently we only check isThreadAllocatedMemorySupported for these > operations, but if I read things correctly the updated code additionally > checks isThreadAllocatedMemoryEnabled, which is a behaviour change not > mentioned in the CSR. > > > I didn?t disturb the existing checks in the test, just added code to > > check the result of getThreadAllocatedBytes(long) on a non-current > > thread, plus the back-to-back no-allocation checks. The former wasn?t > > needed before because getThreadAllocatedBytes(long) was just a wrapper > > around getThreadAllocatedBytes(long []). This patch changes that, so I > > added a separate test. The latter is supposed to fail if there?s object > > allocation on calls to getCurrentThreadAllocatedBytes and > > getThreadAllocatedBytes(long). I.e., a feature, not a bug, because > > accumulation of transient small objects can be a performance problem. > > Thanks to your review, I noticed that the back-to-back check on the > > current thread was using getThreadAllocatedBytes(long) instead of > > getCurrentThreadAllocatedBytes and fixed it. I also removed all > > instances of ?TEST FAILED: ?. > > The back-to-back check is not valid in general. You don't know if the > first check might trigger some class loading on the return path after it > has obtained the first memory value. The check might also fail if using > JVMCI and some compilation related activity occurs in the current thread > on the second call. Also with the introduction of handshakes its > possible the current thread might hit a safepoint checks that results in > it executing a handshake operation that performs allocation. Potentially > there could be numerous non-deterministic actions that might occur > leading to unanticipated allocation. > > I understand what you want to test here, I just don't think it is > reliably doable. > > Thanks, > David > ----- > > > > > Paul > > > > *From: *Mandy Chung > > *Date: *Thursday, September 12, 2019 at 10:09 AM > > *To: *"Hohensee, Paul" > > *Cc: *OpenJDK Serviceability , > > "hotspot-gc-dev at openjdk.java.net" > > *Subject: *Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() > > can be quicker for self thread > > > > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > > > > Minor update in new webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > > > > > > I only reviewed the library side implementation that looks good. I > > expect the serviceability team to review the test and hotspot change. > > > > > > Need a confirmatory review to push this. If I understand the rules correctly, it doesn't need a Reviewer review since Mandy's already reviewed it, it just needs a Committer review. > > > > > > You need another reviewer to advice the following because I was not > > close to the ThreadsList work. > > > > 2087 ThreadsListHandle tlh; > > > > 2088 JavaThread* java_thread = tlh.list()->find_JavaThread_from_java_tid(thread_id); > > > > 2089 > > > > 2090 if (java_thread != NULL) { > > > > 2091 return java_thread->cooked_allocated_bytes(); > > > > 2092 } > > > > This looks right to me. > > > > test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > > > > - "ThreadAllocatedMemory is expected to be disabled"); > > > > + "TEST FAILED: ThreadAllocatedMemory is expected to be > > disabled"); > > > > Prepending "TEST FAILED" in exception message (in several places) > > > > seems redundant since such RuntimeException is thrown and expected > > > > a test failure. > > > > + // back-to-back calls shouldn't allocate any memory > > > > + size = mbean.getThreadAllocatedBytes(id); > > > > + size1 = mbean.getThreadAllocatedBytes(id); > > > > + if (size1 != size) { > > > > Is there anything in the test can do to help guarantee this? I didn't > > > > closely review this test. The main thing I advice is to improve > > > > the reliability of this test. Put it in another way, we want to > > > > ensure that this test change will pass all the time in various > > > > test configuration. > > > > Mandy > > > > From hohensee at amazon.com Sun Sep 15 09:52:30 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Sun, 15 Sep 2019 09:52:30 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> Message-ID: Hi, Serguei, thanks for the review. New webrev at http://cr.openjdk.java.net/~phh/8207266/webrev.09/ I refactored the test?s main() method, and you?re correct, getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in that context: fixed. Paul From: "serguei.spitsyn at oracle.com" Organization: Oracle Corporation Date: Friday, September 13, 2019 at 5:50 PM To: "Hohensee, Paul" , David Holmes , Mandy Chung Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread Hi Paul, It looks pretty good in general. http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html It would be nice to refactor the java main() method as it becomes too big. Two ways of getCurrentThreadAllocatedBytes() testing are good candidates to become separate methods. 98 long size1 = mbean.getThreadAllocatedBytes(id); Just wanted to double check if you wanted to invoke the getCurrentThreadAllocatedBytes() instead as it is a part of: 85 // First way, getCurrentThreadAllocatedBytes Thanks, Serguei On 9/13/19 12:11 PM, Hohensee, Paul wrote: Hi David, thanks for your comments. New webrev in http://cr.openjdk.java.net/~phh/8207266/webrev.08/ Both the old and new versions of the code check that thread allocated memory is both supported and enabled. The existing version of getThreadAllocatedBytes(long []) calls verifyThreadAllocatedMemory(long []), which checks inline to make sure thread allocated memory is supported, then calls isThreadAllocatedMemoryEnabled() to verify that it's enabled. isThreadAllocatedMemoryEnabled() duplicates (!) the support check and returns the enabled flag. I removed the redundant check in the new version. You're of course correct about the back-to-back check. Application code can't know when the runtime will hijack a thread for its own purposes. I've removed the check. Paul On 9/13/19, 12:50 AM, "David Holmes" wrote: Hi Paul, On 13/09/2019 10:29 am, Hohensee, Paul wrote: > Thanks for clarifying the review rules. Would someone from the > serviceability team please review? New webrev at > > http://cr.openjdk.java.net/~phh/8207266/webrev.07/ One aspect of the functional change needs clarification for me - and apologies if this has been covered in the past. It seems to me that currently we only check isThreadAllocatedMemorySupported for these operations, but if I read things correctly the updated code additionally checks isThreadAllocatedMemoryEnabled, which is a behaviour change not mentioned in the CSR. > I didn?t disturb the existing checks in the test, just added code to > check the result of getThreadAllocatedBytes(long) on a non-current > thread, plus the back-to-back no-allocation checks. The former wasn?t > needed before because getThreadAllocatedBytes(long) was just a wrapper > around getThreadAllocatedBytes(long []). This patch changes that, so I > added a separate test. The latter is supposed to fail if there?s object > allocation on calls to getCurrentThreadAllocatedBytes and > getThreadAllocatedBytes(long). I.e., a feature, not a bug, because > accumulation of transient small objects can be a performance problem. > Thanks to your review, I noticed that the back-to-back check on the > current thread was using getThreadAllocatedBytes(long) instead of > getCurrentThreadAllocatedBytes and fixed it. I also removed all > instances of ?TEST FAILED: ?. The back-to-back check is not valid in general. You don't know if the first check might trigger some class loading on the return path after it has obtained the first memory value. The check might also fail if using JVMCI and some compilation related activity occurs in the current thread on the second call. Also with the introduction of handshakes its possible the current thread might hit a safepoint checks that results in it executing a handshake operation that performs allocation. Potentially there could be numerous non-deterministic actions that might occur leading to unanticipated allocation. I understand what you want to test here, I just don't think it is reliably doable. Thanks, David ----- > > Paul > > *From: *Mandy Chung > *Date: *Thursday, September 12, 2019 at 10:09 AM > *To: *"Hohensee, Paul" > *Cc: *OpenJDK Serviceability , > "hotspot-gc-dev at openjdk.java.net" > *Subject: *Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() > can be quicker for self thread > > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > > Minor update in new webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > > > I only reviewed the library side implementation that looks good. I > expect the serviceability team to review the test and hotspot change. > > > Need a confirmatory review to push this. If I understand the rules correctly, it doesn't need a Reviewer review since Mandy's already reviewed it, it just needs a Committer review. > > > You need another reviewer to advice the following because I was not > close to the ThreadsList work. > > 2087 ThreadsListHandle tlh; > > 2088 JavaThread* java_thread = tlh.list()->find_JavaThread_from_java_tid(thread_id); > > 2089 > > 2090 if (java_thread != NULL) { > > 2091 return java_thread->cooked_allocated_bytes(); > > 2092 } > > This looks right to me. > > test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > > - "ThreadAllocatedMemory is expected to be disabled"); > > + "TEST FAILED: ThreadAllocatedMemory is expected to be > disabled"); > > Prepending "TEST FAILED" in exception message (in several places) > > seems redundant since such RuntimeException is thrown and expected > > a test failure. > > + // back-to-back calls shouldn't allocate any memory > > + size = mbean.getThreadAllocatedBytes(id); > > + size1 = mbean.getThreadAllocatedBytes(id); > > + if (size1 != size) { > > Is there anything in the test can do to help guarantee this? I didn't > > closely review this test. The main thing I advice is to improve > > the reliability of this test. Put it in another way, we want to > > ensure that this test change will pass all the time in various > > test configuration. > > Mandy > From mandrikov at gmail.com Sun Sep 15 22:14:35 2019 From: mandrikov at gmail.com (Evgeny Mandrikov) Date: Mon, 16 Sep 2019 00:14:35 +0200 Subject: RFR: JDK-8066774: Rename the annotations arrays names in ClassFileParser Message-ID: Hello! Please review patch [1] for JDK-8066774 [2]. Also it needs a sponsor since I have only author status in OpenJDK Census [3]. After this change tier1 tests pass on my machine. With best regards, Evgeny Mandrikov [1] http://cr.openjdk.java.net/~godin/8066774/webrev.00/ [2] https://bugs.openjdk.java.net/browse/JDK-8066774 [3] https://openjdk.java.net/census#godin From per.liden at oracle.com Mon Sep 16 05:54:53 2019 From: per.liden at oracle.com (Per Liden) Date: Mon, 16 Sep 2019 07:54:53 +0200 Subject: RFR: 8230808: Remove Access::equals() In-Reply-To: References: Message-ID: Can I please have a second review of this? Thanks, Per On 9/10/19 3:27 PM, Per Liden wrote: > Access::equals() is not specialized by any barrier set and can be > removed. This patch doesn't remove oopDesc::equals()/equals_raw(), but > they don't use Access::equals() to do the work anymore. Removing > oopDesc::equals() is a more intrusive change, which can be done as the > next step. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8230808 > Webrev: http://cr.openjdk.java.net/~pliden/8230808/webrev.0 > > /Per From shade at redhat.com Mon Sep 16 08:49:45 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 16 Sep 2019 10:49:45 +0200 Subject: RFR: 8230808: Remove Access::equals() In-Reply-To: References: Message-ID: <7d45b757-951f-29e2-89b6-a3c579e32716@redhat.com> On 9/16/19 7:54 AM, Per Liden wrote: > On 9/10/19 3:27 PM, Per Liden wrote: >> Access::equals() is not specialized by any barrier set and can be removed. This patch doesn't >> remove oopDesc::equals()/equals_raw(), but they don't use Access::equals() to do the work anymore. >> Removing oopDesc::equals() is a more intrusive change, which can be done as the next step. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8230808 >> Webrev: http://cr.openjdk.java.net/~pliden/8230808/webrev.0 Looks fine. -- Thanks, -Aleksey From shade at redhat.com Mon Sep 16 08:57:09 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 16 Sep 2019 10:57:09 +0200 Subject: RFR: JDK-8066774: Rename the annotations arrays names in ClassFileParser In-Reply-To: References: Message-ID: On 9/16/19 12:14 AM, Evgeny Mandrikov wrote: > After this change tier1 tests pass on my machine. Would you mind passing patches like these via jdk-submit? https://wiki.openjdk.java.net/display/Build/Submit+Repo > [1] http://cr.openjdk.java.net/~godin/8066774/webrev.00/ > [2] https://bugs.openjdk.java.net/browse/JDK-8066774 This looks good. I note that ClassFileParser is in runtime purview, cc'ing hotspot-runtime-dev at . -- Thanks, -Aleksey From mandrikov at gmail.com Mon Sep 16 10:33:52 2019 From: mandrikov at gmail.com (Evgeny Mandrikov) Date: Mon, 16 Sep 2019 12:33:52 +0200 Subject: RFR: JDK-8066774: Rename the annotations arrays names in ClassFileParser In-Reply-To: References: Message-ID: On Mon, Sep 16, 2019 at 10:57 AM Aleksey Shipilev wrote: > On 9/16/19 12:14 AM, Evgeny Mandrikov wrote: > > After this change tier1 tests pass on my machine. > > Would you mind passing patches like these via jdk-submit? > https://wiki.openjdk.java.net/display/Build/Submit+Repo > I'd be glad to, but seems that author status is not enough and committer is required - right now tried to push and got: hg push --branch=JDK-8066774 --new-branch -v pushing to ssh://godin at hg.openjdk.java.net/jdk/submit searching for changes 1 changesets found uncompressed size of bundle content: 422 (changelog) 278 (manifests) 1700 src/hotspot/share/classfile/classFileParser.cpp 234 src/hotspot/share/classfile/classFileParser.hpp remote: abort: could not lock repository jdk/submit: Read-only file system abort: unexpected response: empty string Regards, Evgeny From shade at redhat.com Mon Sep 16 10:39:37 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 16 Sep 2019 12:39:37 +0200 Subject: RFR: JDK-8066774: Rename the annotations arrays names in ClassFileParser In-Reply-To: References: Message-ID: <17427f69-c936-ab49-c22f-7df9aacac82a@redhat.com> On 9/16/19 12:33 PM, Evgeny Mandrikov wrote: > On Mon, Sep 16, 2019 at 10:57 AM Aleksey Shipilev > wrote: > > On 9/16/19 12:14 AM, Evgeny Mandrikov wrote: > > After this change tier1 tests pass on my machine. > > Would you mind passing patches like these via jdk-submit? > ?https://wiki.openjdk.java.net/display/Build/Submit+Repo > > I'd be glad to, but seems that author status is not enough and committer is required - right now > tried to push and got: Right, right... I submitted it myself. -- Thanks, -Aleksey From per.liden at oracle.com Mon Sep 16 11:08:10 2019 From: per.liden at oracle.com (Per Liden) Date: Mon, 16 Sep 2019 13:08:10 +0200 Subject: RFR: 8230808: Remove Access::equals() In-Reply-To: <7d45b757-951f-29e2-89b6-a3c579e32716@redhat.com> References: <7d45b757-951f-29e2-89b6-a3c579e32716@redhat.com> Message-ID: <89C1F663-6C9E-4B8B-BD1D-5E86296D1663@oracle.com> Thanks Aleksey! /Per > On 16 Sep 2019, at 10:49, Aleksey Shipilev wrote: > >> On 9/16/19 7:54 AM, Per Liden wrote: >>> On 9/10/19 3:27 PM, Per Liden wrote: >>> Access::equals() is not specialized by any barrier set and can be removed. This patch doesn't >>> remove oopDesc::equals()/equals_raw(), but they don't use Access::equals() to do the work anymore. >>> Removing oopDesc::equals() is a more intrusive change, which can be done as the next step. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8230808 >>> Webrev: http://cr.openjdk.java.net/~pliden/8230808/webrev.0 > > Looks fine. > > -- > Thanks, > -Aleksey > From leihouyju at gmail.com Mon Sep 16 13:54:19 2019 From: leihouyju at gmail.com (Haoyu Li) Date: Mon, 16 Sep 2019 21:54:19 +0800 Subject: [PATCH] Exploit Empty Regions in Young Gen to Enhance PS Full GC Performance In-Reply-To: References: <4F02DD53-EA98-4A1A-B871-C6E9D9610B2C@oracle.com> <9B69AFD1-7AE2-4B50-BFCF-C9C6E2594240@oracle.com> Message-ID: Hi Stefan, Thanks for getting back to me! I have ported the optimization to JDK 14 and the new patch is attached in this mail. As to the command lines in our evaluation, basically, we run the benchmarks with flags including *-Xmx -XX:ParallelGCThreads=32 -XX:+UseParallelGC -XX:-ScavengeBeforeFullGC -Xlog:gc.* We set the maximum heap size for each benchmark to 3X of their minimum heap size and the amount of GC threads to 32 because our machine has 32 physical cores. Full command lines for all benchmarks can be found in the attached file *evaluation.sh*. I am more than happy to have any feedback. Thanks for reviewing this patch! Best Regrads, Haoyu Li, Institute of Parallel and Distributed Systems(IPADS), School of Software, Shanghai Jiao Tong University Stefan Johansson ?2019?9?12??? ??5:34??? > Hi Haoyu, > > I recently came across your patch and I would like to pick up on some of > the things Kim mentioned in his mails. I especially want evaluate > and investigate if this is a technique we can use to improve the other GCs > as well. To start that work I want to take the patch for a spin in our > internal performance testing. The patch doesn?t apply clean to the latest > JDK repository, so if you could provide an updated patch that would be very > helpful. > > It would also be great if you could share some more information around the > results presented in the paper. For example, it would be good to get the > full command lines for the different benchmarks so we can run them locally > and reproduce the results you?ve seen. > > Thanks, > Stefan > > 12 mars 2019 kl. 03:21 skrev Haoyu Li : > > Hi Kim, > > Thanks for reviewing and testing the patch. If there are any failures or > performance degradation relevant to the work, please let me know and I'll > be very happy to keep improving it. Also, any suggestions about code > improvements are well appreciated. > > I'm not quite sure if both G1 and Shenandoah have the similar region > dependency issue, since I haven't studied their GC behaviors before. If > they have, I'm also willing to propose a more general optimization. > > As to the memory overhead, I believe it will be low because this patch > exploits empty regions in the young space rather than off-heap memory to > allocate shadow regions, and also reuses the *_source_region* field of > each *RegionData *to record the correspongding shadow region index. We > only introduce a new integer filed *_shadow *in the RegionData class to > indicate the status of a region, a global *GrowableArray _free_shadow* to > store the indices of shadow regions, and a global *Monitor* to protect > the array. These information might help if the memory overhead need to be > evaluated. > > Looking forward to your insight. > > Best Regrads, > Haoyu Li, > Institute of Parallel and Distributed Systems(IPADS), > School of Software, > Shanghai Jiao Tong University > > > Kim Barrett ?2019?3?12??? ??6:11??? > >> > On Mar 11, 2019, at 1:45 AM, Kim Barrett >> wrote: >> > >> >> On Jan 24, 2019, at 3:58 AM, Haoyu Li wrote: >> >> >> >> Hi Kim, >> >> >> >> I have ported my patch to OpenJDK 13 according to your instructions in >> your last mail, and the patch is attached in this mail. The patch does not >> change much since PSGC is indeed pretty stable. >> >> >> >> Also, I evaluate the correctness and performance of PS full GC with >> benchmarks from DaCapo, SPECjvm2008, and JOlden suits on a machine with >> dual Intel Xeon E5-2618L v3 CPUs(16 physical cores), 64G DRAM and linux >> kernel 4.17. The evaluation result, indicating 1.9X GC throughput >> improvement on average, is attached, too. >> >> >> >> However, I have no idea how to further test this patch for both >> correctness and performance. Can I please get any guidance from you or some >> sponsor? >> > >> > Sorry I missed that you had sent an updated version of the patch. >> > >> > I?ve run the full regression suite across Oracle-supported platforms. >> There are some >> > failures, but there are almost always some failures in the later tiers >> right now. I?ll start >> > looking at them tomorrow to figure out whether any of them are relevant. >> > >> > I?m also planning to run some of our performance benchmarks. >> > >> > I?ve lightly skimmed the proposed changes. There might be some code >> improvements >> > to be made. >> > >> > I?m also wondering if this technique applies to other collectors. It >> seems like both G1 and >> > Shenandoah full gc?s might have similar issues? If so, a solution that >> is ParallelGC-specific >> > is less interesting than one that has broader applicability. Though >> maybe this optimization >> > is less important for G1 and Shenandoah, since they actively try to >> avoid full gc?s. >> > >> > I?m also not clear on how much additional memory might be temporarily >> allocated by this >> > mechanism. >> >> I?ve created a CR for this: >> https://bugs.openjdk.java.net/browse/JDK-8220465 >> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: shadow-region.patch Type: text/x-patch Size: 23652 bytes Desc: not available URL: From leihouyju at gmail.com Mon Sep 16 14:02:40 2019 From: leihouyju at gmail.com (Haoyu Li) Date: Mon, 16 Sep 2019 22:02:40 +0800 Subject: [PATCH] Exploit Empty Regions in Young Gen to Enhance PS Full GC Performance In-Reply-To: References: <4F02DD53-EA98-4A1A-B871-C6E9D9610B2C@oracle.com> <9B69AFD1-7AE2-4B50-BFCF-C9C6E2594240@oracle.com> Message-ID: FYI, the evaluation results on OpenJDK 14 are plotted in the attachment. I compute the full GC throughput by dividing the heap size before full GC by the GC pause time, and the results are arithmetic mean values of ten runs after a warm-up run. The evaluation is conducted on a machine with dual Intel ?XeonTM E5-2618L v3 CPUs (2 sockets, 16 physical cores with SMT enabled) and 64G DRAM. Best Regrads, Haoyu Li, Institute of Parallel and Distributed Systems(IPADS), School of Software, Shanghai Jiao Tong University Stefan Johansson ?2019?9?12??? ??5:34??? > Hi Haoyu, > > I recently came across your patch and I would like to pick up on some of > the things Kim mentioned in his mails. I especially want evaluate > and investigate if this is a technique we can use to improve the other GCs > as well. To start that work I want to take the patch for a spin in our > internal performance testing. The patch doesn?t apply clean to the latest > JDK repository, so if you could provide an updated patch that would be very > helpful. > > It would also be great if you could share some more information around the > results presented in the paper. For example, it would be good to get the > full command lines for the different benchmarks so we can run them locally > and reproduce the results you?ve seen. > > Thanks, > Stefan > > 12 mars 2019 kl. 03:21 skrev Haoyu Li : > > Hi Kim, > > Thanks for reviewing and testing the patch. If there are any failures or > performance degradation relevant to the work, please let me know and I'll > be very happy to keep improving it. Also, any suggestions about code > improvements are well appreciated. > > I'm not quite sure if both G1 and Shenandoah have the similar region > dependency issue, since I haven't studied their GC behaviors before. If > they have, I'm also willing to propose a more general optimization. > > As to the memory overhead, I believe it will be low because this patch > exploits empty regions in the young space rather than off-heap memory to > allocate shadow regions, and also reuses the *_source_region* field of > each *RegionData *to record the correspongding shadow region index. We > only introduce a new integer filed *_shadow *in the RegionData class to > indicate the status of a region, a global *GrowableArray _free_shadow* to > store the indices of shadow regions, and a global *Monitor* to protect > the array. These information might help if the memory overhead need to be > evaluated. > > Looking forward to your insight. > > Best Regrads, > Haoyu Li, > Institute of Parallel and Distributed Systems(IPADS), > School of Software, > Shanghai Jiao Tong University > > > Kim Barrett ?2019?3?12??? ??6:11??? > >> > On Mar 11, 2019, at 1:45 AM, Kim Barrett >> wrote: >> > >> >> On Jan 24, 2019, at 3:58 AM, Haoyu Li wrote: >> >> >> >> Hi Kim, >> >> >> >> I have ported my patch to OpenJDK 13 according to your instructions in >> your last mail, and the patch is attached in this mail. The patch does not >> change much since PSGC is indeed pretty stable. >> >> >> >> Also, I evaluate the correctness and performance of PS full GC with >> benchmarks from DaCapo, SPECjvm2008, and JOlden suits on a machine with >> dual Intel Xeon E5-2618L v3 CPUs(16 physical cores), 64G DRAM and linux >> kernel 4.17. The evaluation result, indicating 1.9X GC throughput >> improvement on average, is attached, too. >> >> >> >> However, I have no idea how to further test this patch for both >> correctness and performance. Can I please get any guidance from you or some >> sponsor? >> > >> > Sorry I missed that you had sent an updated version of the patch. >> > >> > I?ve run the full regression suite across Oracle-supported platforms. >> There are some >> > failures, but there are almost always some failures in the later tiers >> right now. I?ll start >> > looking at them tomorrow to figure out whether any of them are relevant. >> > >> > I?m also planning to run some of our performance benchmarks. >> > >> > I?ve lightly skimmed the proposed changes. There might be some code >> improvements >> > to be made. >> > >> > I?m also wondering if this technique applies to other collectors. It >> seems like both G1 and >> > Shenandoah full gc?s might have similar issues? If so, a solution that >> is ParallelGC-specific >> > is less interesting than one that has broader applicability. Though >> maybe this optimization >> > is less important for G1 and Shenandoah, since they actively try to >> avoid full gc?s. >> > >> > I?m also not clear on how much additional memory might be temporarily >> allocated by this >> > mechanism. >> >> I?ve created a CR for this: >> https://bugs.openjdk.java.net/browse/JDK-8220465 >> >> > From coleen.phillimore at oracle.com Mon Sep 16 16:54:17 2019 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 16 Sep 2019 12:54:17 -0400 Subject: RFR: JDK-8066774: Rename the annotations arrays names in ClassFileParser In-Reply-To: References: Message-ID: <743bce0d-b4c5-b517-7565-699d08ea5999@oracle.com> CCing all lists. This looks good to me. Coleen On 9/16/19 4:57 AM, Aleksey Shipilev wrote: > On 9/16/19 12:14 AM, Evgeny Mandrikov wrote: >> After this change tier1 tests pass on my machine. > Would you mind passing patches like these via jdk-submit? > https://wiki.openjdk.java.net/display/Build/Submit+Repo > > >> [1] http://cr.openjdk.java.net/~godin/8066774/webrev.00/ >> [2] https://bugs.openjdk.java.net/browse/JDK-8066774 > This looks good. > > I note that ClassFileParser is in runtime purview, cc'ing hotspot-runtime-dev at . > From kim.barrett at oracle.com Mon Sep 16 23:12:46 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 16 Sep 2019 19:12:46 -0400 Subject: RFR(XL): 8220310: Implementation: NUMA-Aware Memory Allocation for G1, Mutator (1/3) In-Reply-To: References: Message-ID: <06ACBF87-ADBE-499F-B668-0274E4925B26@oracle.com> > On Sep 4, 2019, at 3:16 AM, sangheon.kim at oracle.com wrote: > > Hi all, > > Please review this patch, making G1 NUMA aware. > This is the first part of G1 NUMA implementation. > > - At G1 CollectedHeap initialization time, the given heap regions will be split and touched. > - Internally node index(uint) is used instead of node id(int type) for easier access to arrays. > - Only Linux is supported. > - New logs are under gc+heap+numa > - Current memory touch implementation will be replaced with WorkGang by JDK-8230411: NUMA aware work gang. > > CR: https://bugs.openjdk.java.net/browse/JDK-8220310 > Webrev: http://cr.openjdk.java.net/~sangheki/8220310/webrev.0 > Testing: hs-tier 1 ~ 5 with +- UseNUMA. > > Thanks, > Sangheon Here is an initial set of comments. Some of these might be nonsense; I'm not all that familiar with NUMA support. ------------------------------------------------------------------------------ Just in general, it's kind of disappointing that NUMA support for G1 apparently involves this much code, and is completely different from ParallelGC NUMA support, and both are completely different from ZGC NUMA support. (And Shenandoah is probably yet another still.) Oh well... ------------------------------------------------------------------------------ According to off-line discussion, the intent is that a HeapRegion only contains memory from one NUMA node. That isn't obvious though, and there are API and data structure choices that obscure that. It seems like some things could be simplified with that constraint in mind. I think some of that is a result of trying to leave open support for NUMA aware off-heap data structures, but that isn't part of this change, and might be better kept separate from NUMA aware Java heap support. I'll try to point out places where I think there are opportunities for simplification. ------------------------------------------------------------------------------ According to off-line discussion, the touching mechanism might not be necessary. If that's true, that would be a significant simplification. For now I won't comment on the code for the touch threads, though I think I found some problems there before the discussion suggesting we might not need them happened. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1NUMA.hpp 64 // Manages NUMA node related information and memory touch threads. ... I found the comment describing G1NUMA pretty hard to parse. Even after looking at the code I'm having trouble figuring out what's being said. This part I was able to figure out, but think it could be worded better. 70 // Also provides a way to convert NUMA id and NUMA index(starting from zero and sequatial). 71 // Actual NUMA ids would not be sequential nor not start from zero, so a conversion is necessary 72 // between id and index. Something like Also provides a mapping between NUMA indices (sequential, starting from zero) and NUMA ids (which may not start at zero and may not be sequential). Indices are generally used by clients, and mapped to ids when dealing with the OS/hardware layer. (I *think* that's true.) But then, why expose node_ids at all in G1NUMA? It seems like node_ids could be completely relegated to an implementation detail that clients never need to deal with. As evidence for not needing to expose ids to clients, there are no callers of numa_id_of_index in any of the series of 3 patches. And G1NUMA::numa_ids() and G1MemoryMultNodeManager::node_ids() are only used in a couple of places for printing/logging and by WB_G1MemoryNodeIds. I'm not sure what the last is for, but the printing and logging code could be isolated into G1NUMA. Also, it looks like the index to node_id mapping provided here is duplicating the information provided by os::Linux::nindex_to_node. Not that this necessarily helps with anything in generic code... ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp 57 virtual int* node_ids() const { static int dummy_id = 0; return &dummy_id; } Can the return type here be "const int*" ? That would be better if possible. [This is probably moot if clients only deal with numa indexes and not node_ids, as suggested earlier.] ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp 63 virtual uint guarantee_valid_node_index(uint node_index) const { return 0; } Given naming conventions elsewhere, the name of this function makes me expect it to be a verification function. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1NUMA.inline.hpp 29 #include "gc/shared/taskqueue.inline.hpp" This #include seems to be just to obtain randomParkAndMiller. It would be better to refactor that function into its own file(s). Or make it part of some component providing various random number generators. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1NUMA.inline.hpp 45 int seed = randomParkAndMiller(_seed); 46 Atomic::store(seed, &_seed); This sequence may lose updates. I don't know if that's important. And why isn't this just using os::random(), or maybe refactoring that slightly to gain access to the external seed version. I don't want this change to be made: http://cr.openjdk.java.net/~sangheki/8220310/webrev.0/src/hotspot/share/gc/shared/taskqueue.inline.hpp.udiff.html And do we *really* need randomness here at all? Is there a reason not to simply cycle through the nodes? ------------------------------------------------------------------------------ src/hotspot/os/linux/os_linux.cpp 3016 size_t num_pages_left = size_in_bytes / page_size; Is size_in_bytes required to be a multiple of page_size? I believe it is, and an assert of that would be useful. ------------------------------------------------------------------------------ src/hotspot/os/linux/os_linux.cpp 3016 size_t num_pages_left = size_in_bytes / page_size; 3017 const size_t num_pages_total = num_pages_left; 3018 size_t num_buffer_entries = MIN2(num_pages_left, max_pages); I think clearer would be 3017 const size_t num_pages_total = size_in_bytes / page_size; 3018 size_t num_buffer_entries = MIN2(num_pages_total, max_pages); and move this down to the loop setup: 3016 size_t num_pages_left = num_pages_total; ------------------------------------------------------------------------------ src/hotspot/os/linux/os_linux.cpp 3020 void** pages = (void**)alloca(sizeof(void*) * num_buffer_entries); 3021 int* status = (int*)alloca(sizeof(int) * num_buffer_entries); I think a ResourceMark and NEW_RESOURCE_ARRAYs would be preferable to using alloca() here. alloca() has some dangers and I think is currently only used in some very specialized places where it really is needed. ------------------------------------------------------------------------------ src/hotspot/os/linux/os_linux.cpp 3061 } 3062 // Moves current thread on a specific node and it will not migrate to Missing blank line between functions. ------------------------------------------------------------------------------ src/hotspot/os/linux/os_linux.cpp 3031 pages[i] = (void*)base; Unnecessary cast. ------------------------------------------------------------------------------ src/hotspot/os/linux/os_linux.cpp 3072 case AFFINITY_NONE: 3073 // Do nothing. 3074 return true; 3075 break; Shouldn't this be calling numa_run_on_node with -1 node id? The documentation for numa_run_on_node() says "Passing -1 permits the kernel to schedule on all nodes again." But see also next comment about numa_affinity_set. ------------------------------------------------------------------------------ src/hotspot/share/runtime/os.hpp 401 // Available affinities. 402 enum NumaAffinity { 403 AFFINITY_STRONG, 404 AFFINITY_NONE 405 }; 406 static bool numa_affinity_set(Thread* thread, int numa_id, NumaAffinity affinity); Could there ever be other affinities? This API doesn't seem right. It seems to me it should be something like static bool numa_set_affinity(Thread* thread, int numa_id); static bool numa_clear_affinity(Thread* thread); No need for NumaAffinity enum. (I think numa_set/clear_affinity is better than numa_affinity_set/clear; numa is a "namespace" and we usually set/clear/get_something rather than something_set/clear/get.) Or maybe this should just be static bool numa_set_affinity(Thread* thread, int numa_id); with "clearing" provided by passing AnyId as the numa_id? Still don't need the NumaAffinity enum for this. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1NUMA.cpp 124 guarantee(rs.size() % page_size == 0, "Reserved space size (" SIZE_FORMAT ") should be aligned " 141 guarantee(size_in_bytes % page_size == 0, "The given range (" SIZE_FORMAT ") should be aligned " 209 guarantee((uintptr_t)G1HeapRegionSize % page_size == 0, "G1HeapRegionSize (" SIZE_FORMAT ") and page size (" 234 guarantee((uintptr_t)_page_size % _heap_region_size == 0, "G1HeapRegionSize (" SIZE_FORMAT ") and page size (" Use is_aligned. I think casts shouldn't be needed. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1NUMA.cpp 43 _numa_id_to_index_map = NEW_C_HEAP_ARRAY_RETURN_NULL(uint, _len_numa_id_to_index_map, mtGC); 44 memset(_numa_id_to_index_map, 0, sizeof(uint) * _len_numa_id_to_index_map); Using NEW_xxx_RETURN_NULL and then ignoring the possibility of a NULL result. It would be better to use the non-_RETURN_NULL variant and get a decent error message rather than the segfault from attemptint to write to NULL. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1NUMA.cpp 34 void G1NUMA::init_numa_id_to_index_map(int* numa_ids, uint num_numa_ids) { int* numa_ids => const int* numa_ids. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1NUMA.cpp 34 void G1NUMA::init_numa_id_to_index_map(int* numa_ids, uint num_numa_ids) { ... 44 memset(_numa_id_to_index_map, 0, sizeof(uint) * _len_numa_id_to_index_map); Rather than this memset and then writing to all of them later, instead initialize all entries to InvalidNodeIndex and then assign from numa_ids. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1MemoryTouchThread.cpp 261 _wakeup_monitor = new Monitor(Mutex::nonleaf, "Wakeup Monitor for Touch Control", true, Monitor::_safepoint_check_never); 262 _sync_monitor = new Monitor(Mutex::nonleaf, "Sync Monitor for Touch Control", true, Monitor::_safepoint_check_never); For singleton mutex/monitor, we generally prefer to put them in mutexLocker. Probably passed as constructor arguments to G1NUMAThreadsControl. And I know there are counter-examples. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp 30 extern const uint AnyNodeIndex; 31 extern const uint InvalidNodeIndex; Why are these at namespace scope. And they seem misplaced here. Seems like they should belong to the G1NUMA class, which is responsible for indexing nodes and mapping between indexes and ids. 30 const uint AnyNodeIndex = (uint)os::AnyId; 31 const uint InvalidNodeIndex = (uint)os::InvalidId; And why these values? That seems like a units mismatch. Just use UINT_MAX and UINT_MAX - 1. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1NUMA.hpp 80 // For invalid numa id, return InvalidNodeIndex. So the caller need exception handling. "need exception handling"? ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegionManager.cpp 114 uint allocated_node_index; This variable gets assigned via an out parameter here: 116 hr = _free_list.remove_region_with_node_index(from_head, valid_node_index, &allocated_node_index); and by assignment here: 122 allocated_node_index = mgr->node_index_of_address(hr->bottom()); but is never used. If the lack of use is not itself a mistake, then the out parameter of remove_region_with_node_index appears to be unnecessary. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegionSet.inline.hpp 151 inline HeapRegion* FreeRegionList::remove_region_with_node_index(bool from_head, As written, I think this might always return NULL if requested_node_index is not a valid index. I think callers ensure that it's valid, but an assert of the argument would be help. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegionSet.inline.hpp 175 if (cur == NULL || cur_depth >= max_search_depth) { // not found There's an off-by-one issue here. The max_search_depth time through the while-control-expression may have found a matching region, checked via the numa_index_of_address lookup. Either the test should be "current_numa_index != requested_numa_index" or the loop should be changed. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegionSet.inline.hpp 151 inline HeapRegion* FreeRegionList::remove_region_with_node_index(bool from_head, I think the list manipulation could be simplified quite a bit. I think we don't need the "last" variable. And the splicing out of cur can be unified between the from_head and !from_head cases, e.g. something like this: // Find the region to use, searching from _head or _tail as requested. if (from_head) { for (cur = _head; cur != NULL && cur_depth < max_search_depth; cur = cur->next(), ++cur_depth) { if (current_numa_index == numa->numa_index_of_address(cur->bottom)) { break; } } } else { for (cur = _tail; cur != NULL && cur_depth < max_search_depth; cur = cur->prev(), ++cur_depth) { if (current_numa_index == numa->numa_index_of_address(cur->bottom)) { break; } } } if (cur == NULL || cur_depth >= max_search_depth) { return NULL; // Didn't find a region to use } // Splice the region out of the list. HeapRegion* prev = cur->prev(); HeapRegion* next = cur->next(); if (prev == NULL) { _head = next; } else { prev->set_next(next); } if (next == NULL) { _tail = prev; } else { next->set_prev(prev); } cur->set_prev(NULL); cur->set_next(NULL); ... ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1NUMA.cpp 166 class DumpIterator : public StackObj { Polluting global namespace. Suggest making DumpIterator and the classes derived from it (G1HeapRegion{Larger,Smaller}Iterator) private nested classes of G1NUMA, making the names unimportant outside of that context. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heterogeneousHeapRegionManager.hpp 122 virtual HeapRegion* allocate_free_region(HeapRegionType type); 123 virtual HeapRegion* allocate_free_region(HeapRegionType type, uint node_index); The base class (HeapRegionManager) no longer has the one-arg function. So this one-arg function is no longer overriding the base class definition. (Too bad it wasn't declared as C++11 "override".) I don't see any one-arg callers, so seems like it can just be removed. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1PageBasedNUMAVirtualSpace.cpp I dislike the new structure here, with G1PageBasedVirtualSpace now being both a base class and a concrete class (which is often a problematic design). And the new derived class doesn't really do all that much. It seems like it would be much simpler to just have G1PageBasedVirtualSpace check UseNUMA to conditionalize it's commit and uncommit functions. Especially since it's commit already needs to have some knowlege of NUMA because of the new numa_index argument. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1PageBasedNUMAVirtualSpace.cpp 52 void G1PageBasedNUMAVirtualSpace::uncommit(size_t start_page, size_t size_in_pages) { Do we actually need to do anything here? We're resetting the G1NUMA::_numa_id_of_pages_table (to AnyId), but it's not clear we really need to do that. When we later recommit any of those pages, that table will be updated appropriately. Maybe rather than having G1NUMA expose quite so much detail it should instead have post_commit and a pre_uncommit functions. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1NUMA.hpp 75 G1NUMAPageIdTable _numa_id_of_pages_table; I was surprised by the need for this, but I can't find anything in the NUMA API that provides this information. That seems strange, because one would think that information must exist somewhere in the NUMA implementation. ------------------------------------------------------------------------------ From rkennke at redhat.com Tue Sep 17 09:40:36 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 17 Sep 2019 11:40:36 +0200 Subject: RFR: JDK-8231085: C2/GC: Better GC-interface for expanding clone Message-ID: <7f810767-32ed-cbb7-b090-03bc525c29df@redhat.com> The current GC-interface for inserting barriers after expanded clones is rather rigid: it only allows to insert a barrier after the expanded clone. It does, for example, not allow a barrier *before* the clone. Also, a possible optimization would be to generate only a single call for both the barrier and the actual clone, i.e. replace the whole clone-expansion with a GC-specific call. I propose that the expansion code for clone is fully owned by GC. This is also in the spirit of other GC interfaces where the GC owns the whole path. However, it requires relaxing visibility of some methods. Jira issue: https://bugs.openjdk.java.net/browse/JDK-8231085 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8231085/webrev.00/ Testing: tier1,2 locally without regressions For an example how I intend to use it, look here (in shenandoahBarrierSetC2.cpp): http://cr.openjdk.java.net/~rkennke/JDK-8231086/webrev.00/ I intend to push this together with JDK-8231086 (otherwise Shenandoah might be temporarily broken). Can I please get a review? Thanks, Roman From per.liden at oracle.com Tue Sep 17 10:05:23 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 17 Sep 2019 12:05:23 +0200 Subject: RFR: 8224820: ZGC: Support discontiguous heap reservations In-Reply-To: <8767693d-13c1-e6c8-4114-f2d73775e8e6@oracle.com> References: <5567c0b8-00e6-29f0-12c6-e067c924fdce@oracle.com> <8767693d-13c1-e6c8-4114-f2d73775e8e6@oracle.com> Message-ID: Hi, Discussed with Erik off-line. In light of things like future Windows support, we probably don't want to make the reservation code shared, instead just move it to os/posix. Here's a proposed webrev for doing just that: http://cr.openjdk.java.net/~pliden/8224820/webrev.0 /Per On 9/5/19 3:51 PM, Per Liden wrote: > On 8/9/19 11:59 AM, Erik ?sterlund wrote: >> Hi, >> >> Today ZGC reserves a huge chunk of virtual address space when the JVM >> starts. This typically succeeds because we grab the VA before anyone >> else has time to do so. But if some ASLR library or something was to >> grab a tiny part of the desired VA, ZGC can't start. We should support >> discontiguous heap reservations to support this. >> >> On linux, by default, this does not happen. But on OS X, it does >> happen relatively frequently. So we need to fix this to allow a mac port. >> >> This patch implements a recursive algorithm for finding holes at 2MB >> granularities in the normally contiguous reservation when initializing >> the heap, removing them from our VA. >> >> This patch depends on 8224815, which depends on 8229189 and 8229278. >> They are all out for review. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8224820/webrev.00/ > > Looks good. However, please pass down max_capacity from ZPageAllocator > to ZVirtualMemory's constructor, instead of reading MaxHeapSize. > > cheers, > Per > >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8224820 >> >> Thanks, >> /Erik From erik.osterlund at oracle.com Tue Sep 17 11:06:13 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 17 Sep 2019 13:06:13 +0200 Subject: RFC: JEP JDK-8229358: ZGC: Support for macOS Message-ID: Hi, I prepared a new JEP [1], about adding support in ZGC for macOS. Implementing this is fairly straight forward now, after the work to have HotSpot deal with discontiguous heap reservations has completed. The reason this discontiguous heap reservation support was necessary is that macOS ASLR may put things in "our" address space, making the previously large and contiguous memory reservation fail. So now that the carpet has been rolled out, it will be a matter of using POSIX shared memory objects for ZGC bindings on macOS. Feedback is more than welcome. Thanks, /Erik [1] https://bugs.openjdk.java.net/browse/JDK-8229358 From erik.osterlund at oracle.com Tue Sep 17 11:42:33 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 17 Sep 2019 13:42:33 +0200 Subject: RFR: JDK-8231085: C2/GC: Better GC-interface for expanding clone In-Reply-To: <7f810767-32ed-cbb7-b090-03bc525c29df@redhat.com> References: <7f810767-32ed-cbb7-b090-03bc525c29df@redhat.com> Message-ID: <4d696b13-1672-437b-32f4-12f08f9ec1c2@oracle.com> Hi Roman, Nice cleanup. This looks good to me. I agree it is more in line with how we have done other things. /Erik On 2019-09-17 11:40, Roman Kennke wrote: > The current GC-interface for inserting barriers after expanded clones is > rather rigid: it only allows to insert a barrier after the expanded > clone. It does, for example, not allow a barrier *before* the clone. > Also, a possible optimization would be to generate only a single call > for both the barrier and the actual clone, i.e. replace the whole > clone-expansion with a GC-specific call. > > I propose that the expansion code for clone is fully owned by GC. This > is also in the spirit of other GC interfaces where the GC owns the whole > path. However, it requires relaxing visibility of some methods. > > Jira issue: > https://bugs.openjdk.java.net/browse/JDK-8231085 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8231085/webrev.00/ > Testing: tier1,2 locally without regressions > > For an example how I intend to use it, look here (in > shenandoahBarrierSetC2.cpp): > http://cr.openjdk.java.net/~rkennke/JDK-8231086/webrev.00/ > > I intend to push this together with JDK-8231086 (otherwise Shenandoah > might be temporarily broken). > > Can I please get a review? > > Thanks, > Roman > From per.liden at oracle.com Tue Sep 17 13:02:32 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 17 Sep 2019 15:02:32 +0200 Subject: RFR: 8231113: Adjust CollectedHeap::check_oop_location() Message-ID: I propose we adjust and rename CollectedHeap::check_oop_location() into CollectedHeap::is_oop_location() that returns a bool, to better match its sibling CollectedHeap::is_oop() and how that is used. Bug: https://bugs.openjdk.java.net/browse/JDK-8231113 Webrev: http://cr.openjdk.java.net/~pliden/8231113/webrev.0 /Per From stefan.johansson at oracle.com Tue Sep 17 13:52:53 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 17 Sep 2019 15:52:53 +0200 Subject: [PATCH] Exploit Empty Regions in Young Gen to Enhance PS Full GC Performance In-Reply-To: References: <4F02DD53-EA98-4A1A-B871-C6E9D9610B2C@oracle.com> <9B69AFD1-7AE2-4B50-BFCF-C9C6E2594240@oracle.com> Message-ID: Thanks, I will try to find time the coming weeks to do some evaluation and I'll get back to you if I have any questions or comments. Thanks, Stefan On 2019-09-16 15:54, Haoyu Li wrote: > Hi Stefan, > > Thanks for getting?back to me! I have ported the optimization to JDK 14 > and the new patch is attached in this mail. > > As to the command lines in our evaluation, basically, we run the > benchmarks with flags including *-Xmx > -XX:ParallelGCThreads=32?-XX:+UseParallelGC -XX:-ScavengeBeforeFullGC > -Xlog:gc.*?We set the maximum heap size for each benchmark to 3X of > their minimum heap size and the amount of GC threads to 32 because our > machine has 32 physical cores. Full command lines for all benchmarks can > be found in the attached file /evaluation.sh/. > > I am more than happy to have any feedback. Thanks for reviewing this patch! > > Best Regrads, > Haoyu Li, > Institute of Parallel and Distributed Systems(IPADS), > School of Software, > Shanghai Jiao Tong University > > > Stefan Johansson > ?2019?9?12??? ??5:34??? > > Hi Haoyu, > > I recently came across your patch and I would like to pick up on > some of the things Kim mentioned in his mails. I especially want > evaluate and?investigate if this is a technique we can use to > improve the other?GCs as well. To start?that work I want to take the > patch for a spin in our internal performance testing. The patch > doesn?t apply clean to the latest JDK repository, so if you could > provide an updated patch that would be very helpful. > > It would also be great if you could share some more information > around the results presented in the paper. For example, it would be > good to get the full?command lines for the different benchmarks so > we can run them locally and reproduce the results?you?ve?seen. > > Thanks, > Stefan > >> 12 mars 2019 kl. 03:21 skrev Haoyu Li > >: >> >> Hi Kim, >> >> Thanks for reviewing and testing the patch. If there are any >> failures or performance degradation relevant to the work, please >> let me know and I'll be very happy to keep improving it. Also, any >> suggestions about code improvements are well appreciated. >> >> I'm not quite sure if both G1 and Shenandoah have the similar >> region dependency issue, since I haven't studied their GC >> behaviors before. If they have, I'm also willing to propose a more >> general optimization. >> >> As to the memory overhead, I believe it will be low because this >> patch exploits empty regions in the young space rather than >> off-heap memory to allocate shadow regions, and also reuses the >> /_source_region/ field of each /RegionData /to record the >> correspongding shadow region index. We only introduce a new >> integer filed /_shadow /in the RegionData class to indicate the >> status of a region, a global /GrowableArray _free_shadow/?to store >> the indices of shadow regions, and a global /Monitor/?to protect >> the array. These information might help if the memory overhead >> need to be evaluated. >> >> Looking forward to your insight. >> >> Best Regrads, >> Haoyu Li, >> Institute of Parallel and Distributed Systems(IPADS), >> School of Software, >> Shanghai Jiao Tong University >> >> >> Kim Barrett > > ?2019?3?12??? ??6:11??? >> >> > On Mar 11, 2019, at 1:45 AM, Kim Barrett >> > wrote: >> > >> >> On Jan 24, 2019, at 3:58 AM, Haoyu Li > > wrote: >> >> >> >> Hi Kim, >> >> >> >> I have ported my patch to OpenJDK 13 according to your >> instructions in your last mail, and the patch is attached in >> this mail. The patch does not change much since PSGC is indeed >> pretty stable. >> >> >> >> Also, I evaluate the correctness and performance of PS full >> GC with benchmarks from DaCapo, SPECjvm2008, and JOlden suits >> on a machine with dual Intel Xeon E5-2618L v3 CPUs(16 physical >> cores), 64G DRAM and linux kernel 4.17. The evaluation result, >> indicating 1.9X GC throughput improvement on average, is >> attached, too. >> >> >> >> However, I have no idea how to further test this patch for >> both correctness and performance. Can I please get any >> guidance from you or some sponsor? >> > >> > Sorry I missed that you had sent an updated version of the >> patch. >> > >> > I?ve run the full regression suite across Oracle-supported >> platforms.? There are some >> > failures, but there are almost always some failures in the >> later tiers right now.? I?ll start >> > looking at them tomorrow to figure out whether any of them >> are relevant. >> > >> > I?m also planning to run some of our performance benchmarks. >> > >> > I?ve lightly skimmed the proposed changes.? There might be >> some code improvements >> > to be made. >> > >> > I?m also wondering if this technique applies to other >> collectors.? It seems like both G1 and >> > Shenandoah full gc?s might have similar issues?? If so, a >> solution that is ParallelGC-specific >> > is less interesting than one that has broader >> applicability.? Though maybe this optimization >> > is less important for G1 and Shenandoah, since they actively >> try to avoid full gc?s. >> > >> > I?m also not clear on how much additional memory might be >> temporarily allocated by this >> > mechanism. >> >> I?ve created a CR for this: >> https://bugs.openjdk.java.net/browse/JDK-8220465 >> > From rkennke at redhat.com Tue Sep 17 14:02:42 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 17 Sep 2019 16:02:42 +0200 Subject: RFR: JDK-8231086: Shenandoah: Stronger invariant for object-arraycopy Message-ID: <5891bb2e-52ba-b3af-c89a-dfb60ac4e4ac@redhat.com> Currently, arraycopy operations can cause temporary from-space-references in the destination array. Those would be subsequently updated by an arraycopy-post-barrier, but it's awkward because this temporary from-space references need to be dealt with, e.g. in the CAS-obj barrier, and a few other places. We can update the src array before copying, and therefore ensure to never see a from-space-reference in the dst array, ever. This opens up a couple of opportunities for simplification and optimization. The change depends on JDK-8231085, which is under review too: http://cr.openjdk.java.net/~rkennke/JDK-8231085/webrev.00/ It removes the huge single-loop that has been used for runtime and interpreter arraycopy, and makes all (c2, c1, interpreter, runtime) use the same pre-barrier. Jira issue: https://bugs.openjdk.java.net/browse/JDK-8231086 Webrev: https://bugs.openjdk.java.net/browse/JDK-8231086 Testing: hotspot_gc_shenandoah Performance testing with specjvm showed no significant pessimizations, and in few cases we came out around 7% improved. I intend to push this together with JDK-8231085, once both are reviewed. Can I please get a review? Thanks, Roman From rkennke at redhat.com Tue Sep 17 14:03:06 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 17 Sep 2019 16:03:06 +0200 Subject: RFR: JDK-8231085: C2/GC: Better GC-interface for expanding clone In-Reply-To: <4d696b13-1672-437b-32f4-12f08f9ec1c2@oracle.com> References: <7f810767-32ed-cbb7-b090-03bc525c29df@redhat.com> <4d696b13-1672-437b-32f4-12f08f9ec1c2@oracle.com> Message-ID: <6dd61a84-5f61-77fd-7d1a-337ff0249b3b@redhat.com> Thanks, Erik! Roman > Hi Roman, > > Nice cleanup. This looks good to me. I agree it is more in line with how > we have done other things. > > /Erik > > On 2019-09-17 11:40, Roman Kennke wrote: >> The current GC-interface for inserting barriers after expanded clones is >> rather rigid: it only allows to insert a barrier after the expanded >> clone. It does, for example, not allow a barrier *before* the clone. >> Also, a possible optimization would be to generate only a single call >> for both the barrier and the actual clone, i.e. replace the whole >> clone-expansion with a GC-specific call. >> >> I propose that the expansion code for clone is fully owned by GC. This >> is also in the spirit of other GC interfaces where the GC owns the whole >> path. However, it requires relaxing visibility of some methods. >> >> Jira issue: >> https://bugs.openjdk.java.net/browse/JDK-8231085 >> Webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8231085/webrev.00/ >> Testing: tier1,2 locally without regressions >> >> For an example how I intend to use it, look here (in >> shenandoahBarrierSetC2.cpp): >> http://cr.openjdk.java.net/~rkennke/JDK-8231086/webrev.00/ >> >> I intend to push this together with JDK-8231086 (otherwise Shenandoah >> might be temporarily broken). >> >> Can I please get a review? >> >> Thanks, >> Roman >> > From hohensee at amazon.com Tue Sep 17 14:10:26 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Tue, 17 Sep 2019 14:10:26 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> Message-ID: <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> Thanks, Serguei. :) David, are you ok with the patch? Paul From: "serguei.spitsyn at oracle.com" Date: Tuesday, September 17, 2019 at 2:26 AM To: "Hohensee, Paul" , David Holmes , Mandy Chung Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread Hi Paul, Thank you for refactoring and fixing the test. It looks great now! Thanks, Serguei On 9/15/19 02:52, Hohensee, Paul wrote: Hi, Serguei, thanks for the review. New webrev at http://cr.openjdk.java.net/~phh/8207266/webrev.09/ I refactored the test?s main() method, and you?re correct, getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in that context: fixed. Paul From: "serguei.spitsyn at oracle.com" Organization: Oracle Corporation Date: Friday, September 13, 2019 at 5:50 PM To: "Hohensee, Paul" , David Holmes , Mandy Chung Cc: OpenJDK Serviceability , "hotspot-gc-dev at openjdk.java.net" Subject: Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread Hi Paul, It looks pretty good in general. http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html It would be nice to refactor the java main() method as it becomes too big. Two ways of getCurrentThreadAllocatedBytes() testing are good candidates to become separate methods. 98 long size1 = mbean.getThreadAllocatedBytes(id); Just wanted to double check if you wanted to invoke the getCurrentThreadAllocatedBytes() instead as it is a part of: 85 // First way, getCurrentThreadAllocatedBytes Thanks, Serguei On 9/13/19 12:11 PM, Hohensee, Paul wrote: Hi David, thanks for your comments. New webrev in http://cr.openjdk.java.net/~phh/8207266/webrev.08/ Both the old and new versions of the code check that thread allocated memory is both supported and enabled. The existing version of getThreadAllocatedBytes(long []) calls verifyThreadAllocatedMemory(long []), which checks inline to make sure thread allocated memory is supported, then calls isThreadAllocatedMemoryEnabled() to verify that it's enabled. isThreadAllocatedMemoryEnabled() duplicates (!) the support check and returns the enabled flag. I removed the redundant check in the new version. You're of course correct about the back-to-back check. Application code can't know when the runtime will hijack a thread for its own purposes. I've removed the check. Paul On 9/13/19, 12:50 AM, "David Holmes" wrote: Hi Paul, On 13/09/2019 10:29 am, Hohensee, Paul wrote: > Thanks for clarifying the review rules. Would someone from the > serviceability team please review? New webrev at > > http://cr.openjdk.java.net/~phh/8207266/webrev.07/ One aspect of the functional change needs clarification for me - and apologies if this has been covered in the past. It seems to me that currently we only check isThreadAllocatedMemorySupported for these operations, but if I read things correctly the updated code additionally checks isThreadAllocatedMemoryEnabled, which is a behaviour change not mentioned in the CSR. > I didn?t disturb the existing checks in the test, just added code to > check the result of getThreadAllocatedBytes(long) on a non-current > thread, plus the back-to-back no-allocation checks. The former wasn?t > needed before because getThreadAllocatedBytes(long) was just a wrapper > around getThreadAllocatedBytes(long []). This patch changes that, so I > added a separate test. The latter is supposed to fail if there?s object > allocation on calls to getCurrentThreadAllocatedBytes and > getThreadAllocatedBytes(long). I.e., a feature, not a bug, because > accumulation of transient small objects can be a performance problem. > Thanks to your review, I noticed that the back-to-back check on the > current thread was using getThreadAllocatedBytes(long) instead of > getCurrentThreadAllocatedBytes and fixed it. I also removed all > instances of ?TEST FAILED: ?. The back-to-back check is not valid in general. You don't know if the first check might trigger some class loading on the return path after it has obtained the first memory value. The check might also fail if using JVMCI and some compilation related activity occurs in the current thread on the second call. Also with the introduction of handshakes its possible the current thread might hit a safepoint checks that results in it executing a handshake operation that performs allocation. Potentially there could be numerous non-deterministic actions that might occur leading to unanticipated allocation. I understand what you want to test here, I just don't think it is reliably doable. Thanks, David ----- > > Paul > > *From: *Mandy Chung > *Date: *Thursday, September 12, 2019 at 10:09 AM > *To: *"Hohensee, Paul" > *Cc: *OpenJDK Serviceability , > "hotspot-gc-dev at openjdk.java.net" > *Subject: *Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() > can be quicker for self thread > > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > > Minor update in new webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > > > I only reviewed the library side implementation that looks good. I > expect the serviceability team to review the test and hotspot change. > > > Need a confirmatory review to push this. If I understand the rules correctly, it doesn't need a Reviewer review since Mandy's already reviewed it, it just needs a Committer review. > > > You need another reviewer to advice the following because I was not > close to the ThreadsList work. > > 2087 ThreadsListHandle tlh; > > 2088 JavaThread* java_thread = tlh.list()->find_JavaThread_from_java_tid(thread_id); > > 2089 > > 2090 if (java_thread != NULL) { > > 2091 return java_thread->cooked_allocated_bytes(); > > 2092 } > > This looks right to me. > > test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > > - "ThreadAllocatedMemory is expected to be disabled"); > > + "TEST FAILED: ThreadAllocatedMemory is expected to be > disabled"); > > Prepending "TEST FAILED" in exception message (in several places) > > seems redundant since such RuntimeException is thrown and expected > > a test failure. > > + // back-to-back calls shouldn't allocate any memory > > + size = mbean.getThreadAllocatedBytes(id); > > + size1 = mbean.getThreadAllocatedBytes(id); > > + if (size1 != size) { > > Is there anything in the test can do to help guarantee this? I didn't > > closely review this test. The main thing I advice is to improve > > the reliability of this test. Put it in another way, we want to > > ensure that this test change will pass all the time in various > > test configuration. > > Mandy > From zgu at redhat.com Tue Sep 17 14:13:47 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 17 Sep 2019 10:13:47 -0400 Subject: RFR(T) 8230350: Shenandoah: Assertion failed when GC is cancelled by a worker thread Message-ID: <0841b841-863a-cab1-d0a1-53de76bb11da@redhat.com> This assertion failure rarely happens, because ShenandoahSuspendibleWorkers is off by default, so this path is rarely executed in jdk/jdk. Now, ShenandoahSuspendibleWorkers is on when concurrent class unloading is enabled in shenandoah/jdk, but I only saw this once, so it is still rare. Bug: https://bugs.openjdk.java.net/browse/JDK-8230350 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8230350/webrev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) Thanks, -Zhengyu From erik.osterlund at oracle.com Tue Sep 17 14:25:55 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 17 Sep 2019 16:25:55 +0200 Subject: RFR: 8231113: Adjust CollectedHeap::check_oop_location() In-Reply-To: References: Message-ID: <7d4fc145-14eb-7305-6690-2545ab5e61e6@oracle.com> Hi Per, Looks good. /Erik On 2019-09-17 15:02, Per Liden wrote: > I propose we adjust and rename CollectedHeap::check_oop_location() > into CollectedHeap::is_oop_location() that returns a bool, to better > match its sibling CollectedHeap::is_oop() and how that is used. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231113 > Webrev: http://cr.openjdk.java.net/~pliden/8231113/webrev.0 > > /Per From rkennke at redhat.com Tue Sep 17 15:40:32 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 17 Sep 2019 17:40:32 +0200 Subject: RFR(T) 8230350: Shenandoah: Assertion failed when GC is cancelled by a worker thread In-Reply-To: <0841b841-863a-cab1-d0a1-53de76bb11da@redhat.com> References: <0841b841-863a-cab1-d0a1-53de76bb11da@redhat.com> Message-ID: Looks good. Thanks! Roman > This assertion failure rarely happens, because > ShenandoahSuspendibleWorkers is off by default, so this path is rarely > executed in jdk/jdk. > > Now, ShenandoahSuspendibleWorkers is on when concurrent class unloading > is enabled in shenandoah/jdk, but I only saw this once, so it is still > rare. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8230350 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8230350/webrev.00/ > > > Test: > ? hotspot_gc_shenandoah (fastdebug and release) > > Thanks, > > -Zhengyu > From per.liden at oracle.com Tue Sep 17 15:41:38 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 17 Sep 2019 17:41:38 +0200 Subject: RFR: 8231113: Adjust CollectedHeap::check_oop_location() In-Reply-To: <7d4fc145-14eb-7305-6690-2545ab5e61e6@oracle.com> References: <7d4fc145-14eb-7305-6690-2545ab5e61e6@oracle.com> Message-ID: Thanks Erik! /Per On 9/17/19 4:25 PM, Erik ?sterlund wrote: > Hi Per, > > Looks good. > > /Erik > > On 2019-09-17 15:02, Per Liden wrote: >> I propose we adjust and rename CollectedHeap::check_oop_location() >> into CollectedHeap::is_oop_location() that returns a bool, to better >> match its sibling CollectedHeap::is_oop() and how that is used. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231113 >> Webrev: http://cr.openjdk.java.net/~pliden/8231113/webrev.0 >> >> /Per > From thomas.schatzl at oracle.com Tue Sep 17 16:04:11 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 17 Sep 2019 18:04:11 +0200 Subject: RFR: 8231113: Adjust CollectedHeap::check_oop_location() In-Reply-To: References: Message-ID: <5ce0836c-c78e-c857-8c2b-ee1965d7a767@oracle.com> Hi, On 17.09.19 15:02, Per Liden wrote: > I propose we adjust and rename CollectedHeap::check_oop_location() into > CollectedHeap::is_oop_location() that returns a bool, to better match > its sibling CollectedHeap::is_oop() and how that is used. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231113 > Webrev: http://cr.openjdk.java.net/~pliden/8231113/webrev.0 > > /Per looks good. Thomas From rkennke at redhat.com Tue Sep 17 17:31:58 2019 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 17 Sep 2019 19:31:58 +0200 Subject: RFR: JDK-8231087: Shenandoah: Self-fixing references Message-ID: One significant contributor to Shenandoah's barrier performance is the LRB-midpath performance: when heap is unstable *and* the loaded reference is in cset, we then check if the object is forwarded and if not, call out into the slow-path. A way to avoid this is to update references right in the slow-path. This avoids triggering the cset-check on next access of the same field. Depending on the workload, this provides throughput speedup of up to 15%. On the other hand, the additional work of updating the field doesn't seem to impact noticably, presumably because it's on the slow-path. An additional advantage is faster update-refs phase, because fewer references are remaining that point to from-space. Jira issue: https://bugs.openjdk.java.net/browse/JDK-8231087 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8231087/webrev.00/ Testing: This has baked for several weeks in sh/jdk and undergone deep CI testing for a while, including several related bugfixes. The rebased patch has been tested with hotspot_gc_shenandoah locally Can I please get a review? Thanks, Roman From kim.barrett at oracle.com Tue Sep 17 18:49:10 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 17 Sep 2019 14:49:10 -0400 Subject: RFR(XL): 8220310: Implementation: NUMA-Aware Memory Allocation for G1, Mutator (1/3) In-Reply-To: <06ACBF87-ADBE-499F-B668-0274E4925B26@oracle.com> References: <06ACBF87-ADBE-499F-B668-0274E4925B26@oracle.com> Message-ID: > On Sep 16, 2019, at 7:12 PM, Kim Barrett wrote: > >> On Sep 4, 2019, at 3:16 AM, sangheon.kim at oracle.com wrote: >> >> Hi all, >> >> Please review this patch, making G1 NUMA aware. >> This is the first part of G1 NUMA implementation. >> >> - At G1 CollectedHeap initialization time, the given heap regions will be split and touched. >> - Internally node index(uint) is used instead of node id(int type) for easier access to arrays. >> - Only Linux is supported. >> - New logs are under gc+heap+numa >> - Current memory touch implementation will be replaced with WorkGang by JDK-8230411: NUMA aware work gang. >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8220310 >> Webrev: http://cr.openjdk.java.net/~sangheki/8220310/webrev.0 >> Testing: hs-tier 1 ~ 5 with +- UseNUMA. >> >> Thanks, >> Sangheon > > Here is an initial set of comments. Some of these might be nonsense; I'm > not all that familiar with NUMA support. Here are a few more. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp 64 // Returns node index of the given address. 65 // If the node id of the address is invalid return randomly selected node index. 66 virtual uint valid_index_of_address(HeapWord* o) const { return 0; } This function is unused. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp 70 virtual uint node_index_of_address(HeapWord* o) const { return 0; } The only caller of this function is HeapRegionManager::allocate_free_region. As noted in an earlier comment, the result of that call is unused. ------------------------------------------------------------------------------ As pointed out to me offline, there *are* ways to query the address -> associated numa node mapping. So I'm going back and questioning the need for _numa_id_of_pages_table and the code for managing it. The above two comments (valid_index_of_address and node_index_of_address) arise from that question. If those functions were removed then the only remaining caller of num_index_of_address is FreeRegionList::remove_region_with_node_index, to get the numa index for a heap region by querying the numa index of the region's bottom. It seems like it would be much easier to maintain that information in the region, since that's the level at which we're interested anyway. Among other things, that might make G1PageBasedNUMAVirtualSpace basically moot. ------------------------------------------------------------------------------ From stefan.karlsson at oracle.com Tue Sep 17 19:48:25 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 17 Sep 2019 21:48:25 +0200 Subject: RFR: 8231113: Adjust CollectedHeap::check_oop_location() In-Reply-To: References: Message-ID: Hi Per, I think both names are bad because the functions return false positives but the names give the impression that they won't. For example: // CMS forwards some non-heap value into the mark oop to reserve oops during // promotion, so we can't assert about obj alignment or that the forwardee is in heap - virtual void check_oop_location(void* addr) const {} + virtual bool is_oop_location(void* addr) const { return true; } It's obvious that this function returns true for pointers outside of the heap. Other implementations are not as bad, but they also return false positives. Users can't use is_oop_location to drive code logic, only to do sanity checks. The check_oop_location functions didn't have that problem, but they couldn't be used in asserts. Did you consider changing the check_oop_location functions to return a bool instead of asserting? G1CollectedHeap* g1h = G1CollectedHeap::heap(); // can't do because of races // assert(oopDesc::is_oop_or_null(obj), "expected an oop"); - g1h->check_oop_location(obj); + assert(g1h->check_oop_location(obj), "invalid oop location"); I think that would be less misleading to readers of the code. Thanks, StefanK On 2019-09-17 15:02, Per Liden wrote: > I propose we adjust and rename CollectedHeap::check_oop_location() > into CollectedHeap::is_oop_location() that returns a bool, to better > match its sibling CollectedHeap::is_oop() and how that is used. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231113 > Webrev: http://cr.openjdk.java.net/~pliden/8231113/webrev.0 > > /Per From david.holmes at oracle.com Tue Sep 17 22:50:24 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 18 Sep 2019 08:50:24 +1000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> Message-ID: <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> On 18/09/2019 12:10 am, Hohensee, Paul wrote: > Thanks, Serguei. :) > > David, are you ok with the patch? Yep, nothing further from me. David > Paul > > *From: *"serguei.spitsyn at oracle.com" > *Date: *Tuesday, September 17, 2019 at 2:26 AM > *To: *"Hohensee, Paul" , David Holmes > , Mandy Chung > *Cc: *OpenJDK Serviceability , > "hotspot-gc-dev at openjdk.java.net" > *Subject: *Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() > can be quicker for self thread > > Hi Paul, > > Thank you for refactoring and fixing the test. > It looks great now! > > Thanks, > Serguei > > > On 9/15/19 02:52, Hohensee, Paul wrote: > > Hi, Serguei, thanks for the review. New webrev at > > http://cr.openjdk.java.net/~phh/8207266/webrev.09/ > > I refactored the test?s main() method, and you?re correct, > getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in > that context: fixed. > > Paul > > *From: *"serguei.spitsyn at oracle.com" > > > *Organization: *Oracle Corporation > *Date: *Friday, September 13, 2019 at 5:50 PM > *To: *"Hohensee, Paul" > , David Holmes > , Mandy Chung > > *Cc: *OpenJDK Serviceability > , > "hotspot-gc-dev at openjdk.java.net" > > > > *Subject: *Re: RFR (M): 8207266: > ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread > > Hi Paul, > > It looks pretty good in general. > > http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html > > It would be nice to refactor the java main() method as it becomes > too big. > Two ways ofgetCurrentThreadAllocatedBytes() testing are good candidates > to become separate methods. > > ? 98?? ??????long size1 = mbean.getThreadAllocatedBytes(id); > > Just wanted to double check if you wanted to invoke > the getCurrentThreadAllocatedBytes() instead as it is > a part of: > > ? 85???????? // First way, getCurrentThreadAllocatedBytes > > > Thanks, > Serguei > > On 9/13/19 12:11 PM, Hohensee, Paul wrote: > > Hi David, thanks for your comments. New webrev in > > > > http://cr.openjdk.java.net/~phh/8207266/webrev.08/ > > > > Both the old and new versions of the code check that thread allocated memory is both supported and enabled. The existing version of getThreadAllocatedBytes(long []) calls verifyThreadAllocatedMemory(long []), which checks inline to make sure thread allocated memory is supported, then calls isThreadAllocatedMemoryEnabled() to verify that it's enabled. isThreadAllocatedMemoryEnabled() duplicates (!) the support check and returns the enabled flag. I removed the redundant check in the new version. > > > > You're of course correct about the back-to-back check. Application code can't know when the runtime will hijack a thread for its own purposes. I've removed the check. > > > > Paul > > > > On 9/13/19, 12:50 AM, "David Holmes" wrote: > > > > ??? Hi Paul, > > > > ????On 13/09/2019 10:29 am, Hohensee, Paul wrote: > > ??? > Thanks for clarifying the review rules. Would someone from the > > ????> serviceability team please review? New webrev at > > ??? > > > ????>http://cr.openjdk.java.net/~phh/8207266/webrev.07/ > > > > ????One aspect of the functional change needs clarification for me - and > > ????apologies if this has been covered in the past. It seems to me that > > ????currently we only check isThreadAllocatedMemorySupported for these > > ????operations, but if I read things correctly the updated code additionally > > ????checks isThreadAllocatedMemoryEnabled, which is a behaviour change not > > ????mentioned in the CSR. > > > > ????> I didn?t disturb the existing checks in the test, just added code to > > ????> check the result of getThreadAllocatedBytes(long) on a non-current > > ????> thread, plus the back-to-back no-allocation checks. The former wasn?t > > ????> needed before because getThreadAllocatedBytes(long) was just a wrapper > > ????> around getThreadAllocatedBytes(long []). This patch changes that, so I > > ????> added a separate test. The latter is supposed to fail if there?s object > > ????> allocation on calls to getCurrentThreadAllocatedBytes and > > ????> getThreadAllocatedBytes(long). I.e., a feature, not a bug, because > > ????> accumulation of transient small objects can be a performance problem. > > ????> Thanks to your review, I noticed that the back-to-back check on the > > ????> current thread was using getThreadAllocatedBytes(long) instead of > > ????> getCurrentThreadAllocatedBytes and fixed it. I also removed all > > ????> instances of ?TEST FAILED: ?. > > > > ????The back-to-back check is not valid in general. You don't know if the > > ????first check might trigger some class loading on the return path after it > > ????has obtained the first memory value. The check might also fail if using > > ????JVMCI and some compilation related activity occurs in the current thread > > ????on the second call. Also with the introduction of handshakes its > > ????possible the current thread might hit a safepoint checks that results in > > ????it executing a handshake operation that performs allocation. Potentially > > ????there could be numerous non-deterministic actions that might occur > > ????leading to unanticipated allocation. > > > > ????I understand what you want to test here, I just don't think it is > > ????reliably doable. > > > > ????Thanks, > > ??? David > > ??? ----- > > > > ????> > > ????> Paul > > ??? > > > ????> *From: *Mandy Chung > > ??? > *Date: *Thursday, September 12, 2019 at 10:09 AM > > ??? > *To: *"Hohensee, Paul" > > ??? > *Cc: *OpenJDK Serviceability , > > ????>"hotspot-gc-dev at openjdk.java.net" > > ??? > *Subject: *Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() > > ????> can be quicker for self thread > > ??? > > > ????> On 9/3/19 12:38 PM, Hohensee, Paul wrote: > > ??? > > > ????>???? Minor update in new webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > > ??? > > > ????> > > ????> I only reviewed the library side implementation that looks good.? I > > ????> expect the serviceability team to review the test and hotspot change. > > ??? > > > ????> > > ????>???? Need a confirmatory review to push this. If I understand the rules correctly, it doesn't need a Reviewer review since Mandy's already reviewed it, it just needs a Committer review. > > ??? > > > ????> > > ????> You need another reviewer to advice the following because I was not > > ????> close to the ThreadsList work. > > ??? > > > ????> 2087?? ThreadsListHandle tlh; > > ??? > > > ????> 2088?? JavaThread* java_thread = tlh.list()->find_JavaThread_from_java_tid(thread_id); > > ??? > > > ????> 2089 > > ??? > > > ????> 2090?? if (java_thread != NULL) { > > ??? > > > ????> 2091???? return java_thread->cooked_allocated_bytes(); > > ??? > > > ????> 2092?? } > > ??? > > > ????> This looks right to me. > > ??? > > > ????> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > > ??? > > > ????> -??????????????? "ThreadAllocatedMemory is expected to be disabled"); > > ??? > > > ????> +??????????????? "TEST FAILED: ThreadAllocatedMemory is expected to be > > ????> disabled"); > > ??? > > > ????> Prepending "TEST FAILED" in exception message (in several places) > > ??? > > > ????> seems redundant since such RuntimeException is thrown and expected > > ??? > > > ????> a test failure. > > ??? > > > ????> +??????? // back-to-back calls shouldn't allocate any memory > > ??? > > > ????> +??????? size = mbean.getThreadAllocatedBytes(id); > > ??? > > > ????> +??????? size1 = mbean.getThreadAllocatedBytes(id); > > ??? > > > ????> +??????? if (size1 != size) { > > ??? > > > ????> Is there anything in the test can do to help guarantee this? I didn't > > ??? > > > ????> closely review this test.? The main thing I advice is to improve > > ??? > > > ????> the reliability of this test.? Put it in another way, we want to > > ??? > > > ????> ensure that this test change will pass all the time in various > > ??? > > > ????> test configuration. > > ??? > > > ????> Mandy > > ??? > > > > > > > > From mandrikov at gmail.com Wed Sep 18 07:42:28 2019 From: mandrikov at gmail.com (Evgeny Mandrikov) Date: Wed, 18 Sep 2019 09:42:28 +0200 Subject: RFR: JDK-8066774: Rename the annotations arrays names in ClassFileParser In-Reply-To: <17427f69-c936-ab49-c22f-7df9aacac82a@redhat.com> References: <17427f69-c936-ab49-c22f-7df9aacac82a@redhat.com> Message-ID: On Mon, Sep 16, 2019 at 12:39 PM Aleksey Shipilev wrote: > Right, right... I submitted it myself. > I'm guessing that jdk-submit passed and you just forgot to commit this? ;) Regards, Evgeny From shade at redhat.com Wed Sep 18 08:09:05 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 18 Sep 2019 10:09:05 +0200 Subject: RFR: JDK-8066774: Rename the annotations arrays names in ClassFileParser In-Reply-To: References: <17427f69-c936-ab49-c22f-7df9aacac82a@redhat.com> Message-ID: <5c1afffe-922d-2de0-d633-d9729a277ac2@redhat.com> On 9/18/19 9:42 AM, Evgeny Mandrikov wrote: > On Mon, Sep 16, 2019 at 12:39 PM Aleksey Shipilev > wrote: > > Right, right... I submitted it myself. > > > I'm guessing that jdk-submit passed and you just forgot to commit this? ;) Still waiting for reply from jdk-submit, actually... -- Thanks, -Aleksey From rkennke at redhat.com Wed Sep 18 09:40:47 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 18 Sep 2019 11:40:47 +0200 Subject: RFC: JEP JDK-8229358: ZGC: Support for macOS In-Reply-To: References: Message-ID: <44cbe8b1-7665-79b7-46a5-41dea96b2fe3@redhat.com> I personally wouldn't bother to file a JEP for it, and go through The Process. We certainly didn't for Shenandoah new features in JDK13 (elimination of fwdptr, new arches, maybe LRB, maybe concurrent roots), and likely won't do for JDK14 (concurrent class-unloading,...) either. A whole new GC: sure. New GC features: not so sure. It just seems to much hassle for too little benefit. Time's better spent on actual coding and fixing ;-) my 2c Cheers, Roman > Hi, > > I prepared a new JEP [1], about adding support in ZGC for macOS. > Implementing this is fairly straight forward now, after the work to have > HotSpot deal with discontiguous heap reservations has completed. The > reason this discontiguous heap reservation support was necessary is that > macOS ASLR may put things in "our" address space, making the previously > large and contiguous memory reservation fail. So now that the carpet has > been rolled out, it will be a matter of using POSIX shared memory > objects for ZGC bindings on macOS. > > Feedback is more than welcome. > > Thanks, > /Erik > > [1] https://bugs.openjdk.java.net/browse/JDK-8229358 From david.holmes at oracle.com Wed Sep 18 10:14:39 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 18 Sep 2019 20:14:39 +1000 Subject: RFR: JDK-8066774: Rename the annotations arrays names in ClassFileParser In-Reply-To: <5c1afffe-922d-2de0-d633-d9729a277ac2@redhat.com> References: <17427f69-c936-ab49-c22f-7df9aacac82a@redhat.com> <5c1afffe-922d-2de0-d633-d9729a277ac2@redhat.com> Message-ID: On 18/09/2019 6:09 pm, Aleksey Shipilev wrote: > On 9/18/19 9:42 AM, Evgeny Mandrikov wrote: >> On Mon, Sep 16, 2019 at 12:39 PM Aleksey Shipilev > wrote: >> >> Right, right... I submitted it myself. >> >> >> I'm guessing that jdk-submit passed and you just forgot to commit this? ;) > > Still waiting for reply from jdk-submit, actually... There was an email outage for a couple of days at least. Do you have a job reference? David From shade at redhat.com Wed Sep 18 10:21:38 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 18 Sep 2019 12:21:38 +0200 Subject: RFR: JDK-8066774: Rename the annotations arrays names in ClassFileParser In-Reply-To: References: <17427f69-c936-ab49-c22f-7df9aacac82a@redhat.com> <5c1afffe-922d-2de0-d633-d9729a277ac2@redhat.com> Message-ID: On 9/18/19 12:14 PM, David Holmes wrote: > On 18/09/2019 6:09 pm, Aleksey Shipilev wrote: >> On 9/18/19 9:42 AM, Evgeny Mandrikov wrote: >>> On Mon, Sep 16, 2019 at 12:39 PM Aleksey Shipilev > >>> wrote: >>> >>> ???? Right, right... I submitted it myself. >>> >>> >>> I'm guessing that jdk-submit passed and you just forgot to commit this? ;) >> >> Still waiting for reply from jdk-submit, actually... > > There was an email outage for a couple of days at least. Do you have a job reference? No, because non-Oracle submitters only see the end result. Or see nothing. I guess you could search by bug ID? It is posted from the named branch that includes the bug ID. -- Thanks, -Aleksey From shade at redhat.com Wed Sep 18 10:50:30 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 18 Sep 2019 12:50:30 +0200 Subject: RFR: JDK-8231086: Shenandoah: Stronger invariant for object-arraycopy In-Reply-To: <5891bb2e-52ba-b3af-c89a-dfb60ac4e4ac@redhat.com> References: <5891bb2e-52ba-b3af-c89a-dfb60ac4e4ac@redhat.com> Message-ID: <4bb05092-ca60-af20-0c40-f7175e97541f@redhat.com> On 9/17/19 4:02 PM, Roman Kennke wrote: > Webrev: > https://bugs.openjdk.java.net/browse/JDK-8231086 The webrev seems to be actually here: http://cr.openjdk.java.net/~rkennke/JDK-8231086/webrev.00/ *) src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp Indenting is bad at L63-65 here: 60 if (dest_uninitialized) { 61 __ tbz(rscratch2, ShenandoahHeap::HAS_FORWARDED_BITPOS, done); 62 } else { 63 __ mov(rscratch2, ShenandoahHeap::HAS_FORWARDED | ShenandoahHeap::MARKING); 64 __ tst(rscratch1, rscratch2); 65 __ br(Assembler::EQ, done); 66 } *) src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp I think we can do this to simplify the code and expose x86_32 path early: #ifdef _LP64 assert(src == rdi, "expected"); assert(dst == rsi, "expected"); assert(count == rdx, "expected"); if (UseCompressedOops) { if (dest_uninitialized) { ....call write_ref_array_pre_duinit_narrow_oop_entry } else { ... call write_ref_array_pre_narrow_oop_entry } } else #endif { if (dest_uninitialized) { ... call write_ref_array_pre_duinit_oop_entry } else { ... call write_ref_array_pre_oop_entry } } *) src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp Do these locals serve any purpose? const char* copyfunc_name = "shenandoah_clone"; address copyfunc_addr = CAST_FROM_FN_PTR(address, ShenandoahRuntime::shenandoah_clone_barrier); const TypePtr* raw_adr_type = TypeRawPtr::BOTTOM; const TypeFunc* call_type = ShenandoahBarrierSetC2::shenandoah_clone_barrier_Type(); Node* call = phase->make_leaf_call(ctrl, mem, call_type, copyfunc_addr, copyfunc_name, raw_adr_type, src, dest, length); Seems to be cleaner to inline them: Node* call = phase->make_leaf_call(ctrl, mem, ShenandoahBarrierSetC2::shenandoah_clone_barrier_Type(), CAST_FROM_FN_PTR(address, ShenandoahRuntime::shenandoah_clone_barrier), "shenandoah_clone", TypeRawPtr::BOTTOM, src, dest, length); *) src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp Having this one is a cute trick: static inline oop cas_oop(oop n, narrowOop* addr, narrowOop c); Otherwise looks good to me. -- Thanks, -Aleksey From shade at redhat.com Wed Sep 18 11:17:29 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 18 Sep 2019 13:17:29 +0200 Subject: RFR: JDK-8231087: Shenandoah: Self-fixing references In-Reply-To: References: Message-ID: On 9/17/19 7:31 PM, Roman Kennke wrote: > Jira issue: > https://bugs.openjdk.java.net/browse/JDK-8231087 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8231087/webrev.00/ Looks cool. We are still doing the full non-fixing LRB from assembler, because it is much harder to do there, or? And that version still does resolve-fwdptr path, so it does not call to slowpath all that often? *) gc/shenandoah/shenandoahRuntime.(cpp|hpp) I think your squashed patch misses this important follow-up: https://hg.openjdk.java.net/shenandoah/jdk/rev/7175c0025de1 In other words, we should not extend SHR::load_reference_barrier_native without changing all the uses too, because it wrecks up calling convention when called from assembler. *) Should be diagnostic. Also, description: "Fix references with load reference barrier. Disabling this might degrade performance". (It would, because we remove the resolve-fwdptr parts from everywhere): 395 experimental(bool, ShenandoahSelfFixing, true, \ 396 "Load-reference-barrier also fixes references") \ -- Thanks, -Aleksey From shade at redhat.com Wed Sep 18 11:19:32 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 18 Sep 2019 13:19:32 +0200 Subject: RFR: JDK-8231087: Shenandoah: Self-fixing references In-Reply-To: References: Message-ID: <6a3c2ccf-b73d-88c2-0a9b-9a04fd0a30f5@redhat.com> On 9/18/19 1:17 PM, Aleksey Shipilev wrote: > On 9/17/19 7:31 PM, Roman Kennke wrote: >> Jira issue: >> https://bugs.openjdk.java.net/browse/JDK-8231087 >> Webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8231087/webrev.00/ > > Looks cool. Oh, and synopsis: "Shenandoah: self-fixing load reference barriers for C1/C2"? Because "self-fixing references" sounds odd. Maybe drop "C1/C2", but it seems to catch that assembler paths are untouched. -- Thanks, -Aleksey From david.holmes at oracle.com Wed Sep 18 12:12:40 2019 From: david.holmes at oracle.com (David Holmes) Date: Wed, 18 Sep 2019 22:12:40 +1000 Subject: RFR: JDK-8066774: Rename the annotations arrays names in ClassFileParser In-Reply-To: References: <17427f69-c936-ab49-c22f-7df9aacac82a@redhat.com> <5c1afffe-922d-2de0-d633-d9729a277ac2@redhat.com> Message-ID: <11734050-657e-ba56-1851-95c8b5c22922@oracle.com> On 18/09/2019 8:21 pm, Aleksey Shipilev wrote: > On 9/18/19 12:14 PM, David Holmes wrote: >> On 18/09/2019 6:09 pm, Aleksey Shipilev wrote: >>> On 9/18/19 9:42 AM, Evgeny Mandrikov wrote: >>>> On Mon, Sep 16, 2019 at 12:39 PM Aleksey Shipilev > >>>> wrote: >>>> >>>> ???? Right, right... I submitted it myself. >>>> >>>> >>>> I'm guessing that jdk-submit passed and you just forgot to commit this? ;) >>> >>> Still waiting for reply from jdk-submit, actually... >> >> There was an email outage for a couple of days at least. Do you have a job reference? > > No, because non-Oracle submitters only see the end result. Or see nothing. I guess you could search > by bug ID? It is posted from the named branch that includes the bug ID. Email will be re-sent. Job passed. David From rkennke at redhat.com Wed Sep 18 14:00:06 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 18 Sep 2019 16:00:06 +0200 Subject: RFR: JDK-8231087: Shenandoah: Self-fixing references In-Reply-To: References: Message-ID: <6db05eeb-dcde-24f3-168b-510e66283079@redhat.com> >> Jira issue: >> https://bugs.openjdk.java.net/browse/JDK-8231087 >> Webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8231087/webrev.00/ > > Looks cool. > > We are still doing the full non-fixing LRB from assembler, because it is much harder to do there, > or? And that version still does resolve-fwdptr path, so it does not call to slowpath all that often? Correct. The register shuffling in interpreter to get the load-addr to the runtime call would be aweful, especially in x86_32. E.g. some loads override the base register of the load-addr, it would have to be lea'd and saved acrossed the load, plus we need an actual register for that, etc. Sorry, I forgot that in the original description. > *) gc/shenandoah/shenandoahRuntime.(cpp|hpp) > > I think your squashed patch misses this important follow-up: > https://hg.openjdk.java.net/shenandoah/jdk/rev/7175c0025de1 > > In other words, we should not extend SHR::load_reference_barrier_native without changing all the > uses too, because it wrecks up calling convention when called from assembler. > Ok, fixed. > *) Should be diagnostic. Also, description: "Fix references with load reference barrier. Disabling > this might degrade performance". (It would, because we remove the resolve-fwdptr parts from everywhere): > > 395 experimental(bool, ShenandoahSelfFixing, true, \ > 396 "Load-reference-barrier also fixes references") \ Fixed. Incremental diff: http://cr.openjdk.java.net/~rkennke/JDK-8231087/webrev.01.diff/ Full: http://cr.openjdk.java.net/~rkennke/JDK-8231087/webrev.01/ Ok now? Roman From rkennke at redhat.com Wed Sep 18 14:16:37 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 18 Sep 2019 16:16:37 +0200 Subject: RFR: JDK-8231086: Shenandoah: Stronger invariant for object-arraycopy In-Reply-To: <4bb05092-ca60-af20-0c40-f7175e97541f@redhat.com> References: <5891bb2e-52ba-b3af-c89a-dfb60ac4e4ac@redhat.com> <4bb05092-ca60-af20-0c40-f7175e97541f@redhat.com> Message-ID: <621178e0-b4f8-1a24-09e9-3d7852ef56f1@redhat.com> Hi Aleksey, >> Webrev: >> https://bugs.openjdk.java.net/browse/JDK-8231086 > > The webrev seems to be actually here: > http://cr.openjdk.java.net/~rkennke/JDK-8231086/webrev.00/ > > *) src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp > > Indenting is bad at L63-65 here: > > 60 if (dest_uninitialized) { > 61 __ tbz(rscratch2, ShenandoahHeap::HAS_FORWARDED_BITPOS, done); > 62 } else { > 63 __ mov(rscratch2, ShenandoahHeap::HAS_FORWARDED | ShenandoahHeap::MARKING); > 64 __ tst(rscratch1, rscratch2); > 65 __ br(Assembler::EQ, done); > 66 } Right, fixed! > *) src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp > > I think we can do this to simplify the code and expose x86_32 path early: > > #ifdef _LP64 > assert(src == rdi, "expected"); > assert(dst == rsi, "expected"); > assert(count == rdx, "expected"); > if (UseCompressedOops) { > if (dest_uninitialized) { > ....call write_ref_array_pre_duinit_narrow_oop_entry > } else { > ... call write_ref_array_pre_narrow_oop_entry > } > } else > #endif > { > if (dest_uninitialized) { > ... call write_ref_array_pre_duinit_oop_entry > } else { > ... call write_ref_array_pre_oop_entry > } > } Good point! Fixed. > *) src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp > > Do these locals serve any purpose? No. Probably left-over from somewhere during dev. > const char* copyfunc_name = "shenandoah_clone"; > address copyfunc_addr = CAST_FROM_FN_PTR(address, ShenandoahRuntime::shenandoah_clone_barrier); > const TypePtr* raw_adr_type = TypeRawPtr::BOTTOM; > const TypeFunc* call_type = ShenandoahBarrierSetC2::shenandoah_clone_barrier_Type(); > Node* call = phase->make_leaf_call(ctrl, mem, call_type, copyfunc_addr, copyfunc_name, > raw_adr_type, src, dest, length); > > Seems to be cleaner to inline them: > > Node* call = phase->make_leaf_call(ctrl, mem, > ShenandoahBarrierSetC2::shenandoah_clone_barrier_Type(), > CAST_FROM_FN_PTR(address, ShenandoahRuntime::shenandoah_clone_barrier), > "shenandoah_clone", > TypeRawPtr::BOTTOM, > src, dest, length); Yeah, fixed. > *) src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp > > Having this one is a cute trick: > static inline oop cas_oop(oop n, narrowOop* addr, narrowOop c); Right? :-) Incremental diff: http://cr.openjdk.java.net/~rkennke/JDK-8231086/webrev.01.diff/ Full: http://cr.openjdk.java.net/~rkennke/JDK-8231086/webrev.01/ Ok? Roman From shade at redhat.com Wed Sep 18 14:24:45 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 18 Sep 2019 16:24:45 +0200 Subject: RFR: JDK-8231086: Shenandoah: Stronger invariant for object-arraycopy In-Reply-To: <621178e0-b4f8-1a24-09e9-3d7852ef56f1@redhat.com> References: <5891bb2e-52ba-b3af-c89a-dfb60ac4e4ac@redhat.com> <4bb05092-ca60-af20-0c40-f7175e97541f@redhat.com> <621178e0-b4f8-1a24-09e9-3d7852ef56f1@redhat.com> Message-ID: On 9/18/19 4:16 PM, Roman Kennke wrote: > Incremental diff: > http://cr.openjdk.java.net/~rkennke/JDK-8231086/webrev.01.diff/ > Full: > http://cr.openjdk.java.net/~rkennke/JDK-8231086/webrev.01/ Looks fine to me. -- Thanks, -Aleksey From hohensee at amazon.com Wed Sep 18 15:03:07 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Wed, 18 Sep 2019 15:03:07 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> Message-ID: <7D09AF54-54DC-4064-83F9-29EA671C3484@amazon.com> Thanks David (and Mandy and Serguei). Pushed. ?On 9/17/19, 3:51 PM, "David Holmes" wrote: On 18/09/2019 12:10 am, Hohensee, Paul wrote: > Thanks, Serguei. :) > > David, are you ok with the patch? Yep, nothing further from me. David > Paul > > *From: *"serguei.spitsyn at oracle.com" > *Date: *Tuesday, September 17, 2019 at 2:26 AM > *To: *"Hohensee, Paul" , David Holmes > , Mandy Chung > *Cc: *OpenJDK Serviceability , > "hotspot-gc-dev at openjdk.java.net" > *Subject: *Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() > can be quicker for self thread > > Hi Paul, > > Thank you for refactoring and fixing the test. > It looks great now! > > Thanks, > Serguei > > > On 9/15/19 02:52, Hohensee, Paul wrote: > > Hi, Serguei, thanks for the review. New webrev at > > http://cr.openjdk.java.net/~phh/8207266/webrev.09/ > > I refactored the test?s main() method, and you?re correct, > getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in > that context: fixed. > > Paul > > *From: *"serguei.spitsyn at oracle.com" > > > *Organization: *Oracle Corporation > *Date: *Friday, September 13, 2019 at 5:50 PM > *To: *"Hohensee, Paul" > , David Holmes > , Mandy Chung > > *Cc: *OpenJDK Serviceability > , > "hotspot-gc-dev at openjdk.java.net" > > > > *Subject: *Re: RFR (M): 8207266: > ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread > > Hi Paul, > > It looks pretty good in general. > > http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html > > It would be nice to refactor the java main() method as it becomes > too big. > Two ways ofgetCurrentThreadAllocatedBytes() testing are good candidates > to become separate methods. > > 98 long size1 = mbean.getThreadAllocatedBytes(id); > > Just wanted to double check if you wanted to invoke > the getCurrentThreadAllocatedBytes() instead as it is > a part of: > > 85 // First way, getCurrentThreadAllocatedBytes > > > Thanks, > Serguei > > On 9/13/19 12:11 PM, Hohensee, Paul wrote: > > Hi David, thanks for your comments. New webrev in > > > > http://cr.openjdk.java.net/~phh/8207266/webrev.08/ > > > > Both the old and new versions of the code check that thread allocated memory is both supported and enabled. The existing version of getThreadAllocatedBytes(long []) calls verifyThreadAllocatedMemory(long []), which checks inline to make sure thread allocated memory is supported, then calls isThreadAllocatedMemoryEnabled() to verify that it's enabled. isThreadAllocatedMemoryEnabled() duplicates (!) the support check and returns the enabled flag. I removed the redundant check in the new version. > > > > You're of course correct about the back-to-back check. Application code can't know when the runtime will hijack a thread for its own purposes. I've removed the check. > > > > Paul > > > > On 9/13/19, 12:50 AM, "David Holmes" wrote: > > > > Hi Paul, > > > > On 13/09/2019 10:29 am, Hohensee, Paul wrote: > > > Thanks for clarifying the review rules. Would someone from the > > > serviceability team please review? New webrev at > > > > > >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ > > > > One aspect of the functional change needs clarification for me - and > > apologies if this has been covered in the past. It seems to me that > > currently we only check isThreadAllocatedMemorySupported for these > > operations, but if I read things correctly the updated code additionally > > checks isThreadAllocatedMemoryEnabled, which is a behaviour change not > > mentioned in the CSR. > > > > > I didn?t disturb the existing checks in the test, just added code to > > > check the result of getThreadAllocatedBytes(long) on a non-current > > > thread, plus the back-to-back no-allocation checks. The former wasn?t > > > needed before because getThreadAllocatedBytes(long) was just a wrapper > > > around getThreadAllocatedBytes(long []). This patch changes that, so I > > > added a separate test. The latter is supposed to fail if there?s object > > > allocation on calls to getCurrentThreadAllocatedBytes and > > > getThreadAllocatedBytes(long). I.e., a feature, not a bug, because > > > accumulation of transient small objects can be a performance problem. > > > Thanks to your review, I noticed that the back-to-back check on the > > > current thread was using getThreadAllocatedBytes(long) instead of > > > getCurrentThreadAllocatedBytes and fixed it. I also removed all > > > instances of ?TEST FAILED: ?. > > > > The back-to-back check is not valid in general. You don't know if the > > first check might trigger some class loading on the return path after it > > has obtained the first memory value. The check might also fail if using > > JVMCI and some compilation related activity occurs in the current thread > > on the second call. Also with the introduction of handshakes its > > possible the current thread might hit a safepoint checks that results in > > it executing a handshake operation that performs allocation. Potentially > > there could be numerous non-deterministic actions that might occur > > leading to unanticipated allocation. > > > > I understand what you want to test here, I just don't think it is > > reliably doable. > > > > Thanks, > > David > > ----- > > > > > > > > Paul > > > > > > *From: *Mandy Chung > > > *Date: *Thursday, September 12, 2019 at 10:09 AM > > > *To: *"Hohensee, Paul" > > > *Cc: *OpenJDK Serviceability , > > >"hotspot-gc-dev at openjdk.java.net" > > > *Subject: *Re: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() > > > can be quicker for self thread > > > > > > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > > > > > > Minor update in new webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > > > > > > > > > I only reviewed the library side implementation that looks good. I > > > expect the serviceability team to review the test and hotspot change. > > > > > > > > > Need a confirmatory review to push this. If I understand the rules correctly, it doesn't need a Reviewer review since Mandy's already reviewed it, it just needs a Committer review. > > > > > > > > > You need another reviewer to advice the following because I was not > > > close to the ThreadsList work. > > > > > > 2087 ThreadsListHandle tlh; > > > > > > 2088 JavaThread* java_thread = tlh.list()->find_JavaThread_from_java_tid(thread_id); > > > > > > 2089 > > > > > > 2090 if (java_thread != NULL) { > > > > > > 2091 return java_thread->cooked_allocated_bytes(); > > > > > > 2092 } > > > > > > This looks right to me. > > > > > > test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > > > > > > - "ThreadAllocatedMemory is expected to be disabled"); > > > > > > + "TEST FAILED: ThreadAllocatedMemory is expected to be > > > disabled"); > > > > > > Prepending "TEST FAILED" in exception message (in several places) > > > > > > seems redundant since such RuntimeException is thrown and expected > > > > > > a test failure. > > > > > > + // back-to-back calls shouldn't allocate any memory > > > > > > + size = mbean.getThreadAllocatedBytes(id); > > > > > > + size1 = mbean.getThreadAllocatedBytes(id); > > > > > > + if (size1 != size) { > > > > > > Is there anything in the test can do to help guarantee this? I didn't > > > > > > closely review this test. The main thing I advice is to improve > > > > > > the reliability of this test. Put it in another way, we want to > > > > > > ensure that this test change will pass all the time in various > > > > > > test configuration. > > > > > > Mandy > > > > > > > > > > From thomas.schatzl at oracle.com Wed Sep 18 15:02:36 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 18 Sep 2019 17:02:36 +0200 Subject: RFR (S): 8231117: Remove G1EvacuationRootClosures::raw_strong_oops() Message-ID: Hi all, can I have reviews for this small cleanup that removes the use of G1EvacuationRootClosures::raw_strong_oops() as its implementation always corresponds to a call to G1EvacuationRootClosures::strong_oops()? Also the comment referenced some non-existent flushing, which even more indicates that this is only some dead code. CR: https://bugs.openjdk.java.net/browse/JDK-8231117 Webrev: http://cr.openjdk.java.net/~tschatzl/8231117/webrev Testing: hs-tier1-5 Thanks, Thomas From shade at redhat.com Wed Sep 18 15:07:32 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 18 Sep 2019 17:07:32 +0200 Subject: RFR: JDK-8231087: Shenandoah: Self-fixing references In-Reply-To: <6db05eeb-dcde-24f3-168b-510e66283079@redhat.com> References: <6db05eeb-dcde-24f3-168b-510e66283079@redhat.com> Message-ID: <603f1c75-ddee-59cb-24b8-b6f128cad29e@redhat.com> On 9/18/19 4:00 PM, Roman Kennke wrote: > Incremental diff: > http://cr.openjdk.java.net/~rkennke/JDK-8231087/webrev.01.diff/ > Full: > http://cr.openjdk.java.net/~rkennke/JDK-8231087/webrev.01/ *) shenandoah_globals.hpp: Please make sure indenting on L396-397 matches other options: 395 diagnostic(bool, ShenandoahSelfFixing, true, \ 396 "Fix references with load reference barrier. Disabling this " \ 397 "might degrade performance.") \ ...for example: 381 diagnostic(bool, ShenandoahOptimizeStaticFinals, true, \ 382 "Optimize barriers on static final fields. " \ 383 "Turn it off for maximum compatibility with reflection or JNI " \ 384 "code that manipulates final fields.") \ Otherwise looks good. No need for another review. -- Thanks, -Aleksey From thomas.schatzl at oracle.com Wed Sep 18 15:22:37 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 18 Sep 2019 17:22:37 +0200 Subject: RFR (M): 8159984: Remove call to ClassLoaderDataGraph::clear_claimed_marks during the initial mark pause Message-ID: <4e851c12-ca88-5d2d-f636-767437ad5a01@oracle.com> Hi all, can I have reviews for this change that, given the observation that during root processing, there is only one kind of root that walks CLDs, we do not need to a) claim CLDs during iteration, obsoleting the need to clear the CLD claims a) wait for "strong" CLD iteration before we can continue with evacuating the heap any more (during the concurrent start pause). We *still* need to wait for "strong" iteration of oops from nmethods as later during the evacuation phase we use the per-region nmethod remembered sets. We need to make sure that all "strong" nmethod iteration is finished at that time. There is the possibility to change the "strong" nmethod iteration to mark their oops outside the iteration claim, then the wait barrier is unnecessary. This is a separate issue though. Thanks go to StefanK for explaining me the root processing issue over and over again until I finally got it. This change obsoletes JDK-8163213 and JDK-8231073. CR: https://bugs.openjdk.java.net/browse/JDK-8159984 Webrev: http://cr.openjdk.java.net/~tschatzl/8159984/webrev/ Testing: hs-tier1-5 (based on JDK-8231117) Thanks, Thomas From thomas.schatzl at oracle.com Wed Sep 18 15:51:53 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 18 Sep 2019 17:51:53 +0200 Subject: RFR (S): 8231189: Rename worker_i parameters to worker_id Message-ID: Hi all, can I have reviews for this change that renames all remaining "worker_i"s to "worker_id"s? In the jdk9 timeframe we did quite a few renamings of similarly used variables to "worker_id", but either some "worker_i" crept in or were forgotten. This rectifies this. CR: https://bugs.openjdk.java.net/browse/JDK-8231189 Webrev: http://cr.openjdk.java.net/~tschatzl/8231189/webrev/ Testing: Local compilation (Patch based on JDK-8159984 also out for review) Thanks, Thomas From kim.barrett at oracle.com Wed Sep 18 16:27:27 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 18 Sep 2019 12:27:27 -0400 Subject: RFR (S): 8231189: Rename worker_i parameters to worker_id In-Reply-To: References: Message-ID: > On Sep 18, 2019, at 11:51 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this change that renames all remaining "worker_i"s to "worker_id"s? > > In the jdk9 timeframe we did quite a few renamings of similarly used variables to "worker_id", but either some "worker_i" crept in or were forgotten. > > This rectifies this. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8231189 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8231189/webrev/ > Testing: > Local compilation > > (Patch based on JDK-8159984 also out for review) > > Thanks, > Thomas Looks good, and trivial. From thomas.schatzl at oracle.com Wed Sep 18 16:29:07 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 18 Sep 2019 18:29:07 +0200 Subject: RFR (S): 8231189: Rename worker_i parameters to worker_id In-Reply-To: References: Message-ID: <9e0c947a-6931-d83b-f430-fae032bfcb84@oracle.com> Hi Kim, thanks for your review. I agree about triviality; I will push this as soon as the dependent CRs have been reviewed even if nobody else gives his approval. Thomas On 18.09.19 18:27, Kim Barrett wrote: >> On Sep 18, 2019, at 11:51 AM, Thomas Schatzl wrote: >> >> Hi all, >> >> can I have reviews for this change that renames all remaining "worker_i"s to "worker_id"s? >> >> In the jdk9 timeframe we did quite a few renamings of similarly used variables to "worker_id", but either some "worker_i" crept in or were forgotten. >> >> This rectifies this. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8231189 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8231189/webrev/ >> Testing: >> Local compilation >> >> (Patch based on JDK-8159984 also out for review) >> >> Thanks, >> Thomas > > Looks good, and trivial. > From kim.barrett at oracle.com Wed Sep 18 16:46:32 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 18 Sep 2019 12:46:32 -0400 Subject: RFR (S): 8231117: Remove G1EvacuationRootClosures::raw_strong_oops() In-Reply-To: References: Message-ID: <40AAD0B9-2269-42F2-9D50-9CEB81E2DD05@oracle.com> > On Sep 18, 2019, at 11:02 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this small cleanup that removes the use of G1EvacuationRootClosures::raw_strong_oops() as its implementation always corresponds to a call to G1EvacuationRootClosures::strong_oops()? > > Also the comment referenced some non-existent flushing, which even more indicates that this is only some dead code. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8231117 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8231117/webrev > Testing: > hs-tier1-5 > > Thanks, > Thomas Looks good. From thomas.schatzl at oracle.com Wed Sep 18 17:15:39 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 18 Sep 2019 19:15:39 +0200 Subject: RFR (S): 8231117: Remove G1EvacuationRootClosures::raw_strong_oops() In-Reply-To: <40AAD0B9-2269-42F2-9D50-9CEB81E2DD05@oracle.com> References: <40AAD0B9-2269-42F2-9D50-9CEB81E2DD05@oracle.com> Message-ID: <2b98c927-bddd-3fe8-071c-6d666b49461f@oracle.com> Hi Kim, On 18.09.19 18:46, Kim Barrett wrote: >> On Sep 18, 2019, at 11:02 AM, Thomas Schatzl wrote: >> >> Hi all, >> >> can I have reviews for this small cleanup that removes the use of G1EvacuationRootClosures::raw_strong_oops() as its implementation always corresponds to a call to G1EvacuationRootClosures::strong_oops()? >> >> Also the comment referenced some non-existent flushing, which even more indicates that this is only some dead code. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8231117 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8231117/webrev >> Testing: >> hs-tier1-5 >> >> Thanks, >> Thomas > > Looks good. > thanks for your review. thomas From kim.barrett at oracle.com Wed Sep 18 18:36:58 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 18 Sep 2019 14:36:58 -0400 Subject: RFR (M): 8159984: Remove call to ClassLoaderDataGraph::clear_claimed_marks during the initial mark pause In-Reply-To: <4e851c12-ca88-5d2d-f636-767437ad5a01@oracle.com> References: <4e851c12-ca88-5d2d-f636-767437ad5a01@oracle.com> Message-ID: > On Sep 18, 2019, at 11:22 AM, Thomas Schatzl wrote: > > Hi all, > > can I have reviews for this change that, given the observation that during root processing, there is only one kind of root that walks CLDs, we do not need to > > a) claim CLDs during iteration, obsoleting the need to clear the CLD claims > a) wait for "strong" CLD iteration before we can continue with evacuating the heap > > any more (during the concurrent start pause). > > We *still* need to wait for "strong" iteration of oops from nmethods as later during the evacuation phase we use the per-region nmethod remembered sets. We need to make sure that all "strong" nmethod iteration is finished at that time. > > There is the possibility to change the "strong" nmethod iteration to mark their oops outside the iteration claim, then the wait barrier is unnecessary. This is a separate issue though. > > Thanks go to StefanK for explaining me the root processing issue over and over again until I finally got it. > > This change obsoletes JDK-8163213 and JDK-8231073. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8159984 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8159984/webrev/ > Testing: > hs-tier1-5 > > (based on JDK-8231117) > > Thanks, > Thomas I was just looking at the root scanning code recently, trying to figure out what worker_has_discovered_all_strong_classes() was about. The naming of worker_has_discovered_all_strong_classes() and wait_until_all_strong_classes_discovered() seems wrong now. These are about discovering/processing strong nmethods now. A better comment at the call to the wait function about why it's called there would be helpful. Obviously we want it as late as possible, to allow threads to get past the notification. But why not later? (We talked about this off-line; this is just a reminder and request for some commentary capturing that discussion. Though hopefully your idea for getting rid of the barrier completly will pan out.) Other than that, looks good. From shade at redhat.com Wed Sep 18 19:21:28 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 18 Sep 2019 21:21:28 +0200 Subject: RFR: JDK-8066774: Rename the annotations arrays names in ClassFileParser In-Reply-To: <11734050-657e-ba56-1851-95c8b5c22922@oracle.com> References: <17427f69-c936-ab49-c22f-7df9aacac82a@redhat.com> <5c1afffe-922d-2de0-d633-d9729a277ac2@redhat.com> <11734050-657e-ba56-1851-95c8b5c22922@oracle.com> Message-ID: <53b0ab37-b70f-176c-0ce0-421ec4045df4@redhat.com> On 9/18/19 2:12 PM, David Holmes wrote: > On 18/09/2019 8:21 pm, Aleksey Shipilev wrote: >> On 9/18/19 12:14 PM, David Holmes wrote: >>> On 18/09/2019 6:09 pm, Aleksey Shipilev wrote: >>>> On 9/18/19 9:42 AM, Evgeny Mandrikov wrote: >>>>> On Mon, Sep 16, 2019 at 12:39 PM Aleksey Shipilev > >>>>> wrote: >>>>> >>>>> ????? Right, right... I submitted it myself. >>>>> >>>>> >>>>> I'm guessing that jdk-submit passed and you just forgot to commit this? ;) >>>> >>>> Still waiting for reply from jdk-submit, actually... >>> >>> There was an email outage for a couple of days at least. Do you have a job reference? >> >> No, because non-Oracle submitters only see the end result. Or see nothing. I guess you could search >> by bug ID? It is posted from the named branch that includes the bug ID. > > Email will be re-sent. Job passed. Got the notification. Thanks! Pushed. -- -Aleksey From david.holmes at oracle.com Wed Sep 18 23:40:32 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 19 Sep 2019 09:40:32 +1000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> Message-ID: <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> Paul, Unfortunately this patch has broken the vmTestbase/nsk/monitoring tests: [2019-09-18T22:59:32,349Z] /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: error: ServerThreadMXBeanNew is not abstract and does not override abstract method getCurrentThreadAllocatedBytes() in ThreadMXBean and possibly other issues as we are seeing hundreds of failures. David On 18/09/2019 8:50 am, David Holmes wrote: > On 18/09/2019 12:10 am, Hohensee, Paul wrote: >> Thanks, Serguei. :) >> >> David, are you ok with the patch? > > Yep, nothing further from me. > > David > >> Paul >> >> *From: *"serguei.spitsyn at oracle.com" >> *Date: *Tuesday, September 17, 2019 at 2:26 AM >> *To: *"Hohensee, Paul" , David Holmes >> , Mandy Chung >> *Cc: *OpenJDK Serviceability , >> "hotspot-gc-dev at openjdk.java.net" >> *Subject: *Re: RFR (M): 8207266: >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread >> >> Hi Paul, >> >> Thank you for refactoring and fixing the test. >> It looks great now! >> >> Thanks, >> Serguei >> >> >> On 9/15/19 02:52, Hohensee, Paul wrote: >> >> ??? Hi, Serguei, thanks for the review. New webrev at >> >> ??? http://cr.openjdk.java.net/~phh/8207266/webrev.09/ >> >> ??? I refactored the test?s main() method, and you?re correct, >> ??? getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in >> ??? that context: fixed. >> >> ??? Paul >> >> ??? *From: *"serguei.spitsyn at oracle.com" >> ??? >> ??? >> ??? *Organization: *Oracle Corporation >> ??? *Date: *Friday, September 13, 2019 at 5:50 PM >> ??? *To: *"Hohensee, Paul" >> ??? , David Holmes >> ??? , Mandy Chung >> ??? >> ??? *Cc: *OpenJDK Serviceability >> ??? , >> ??? "hotspot-gc-dev at openjdk.java.net" >> ??? >> ??? >> ??? >> ??? *Subject: *Re: RFR (M): 8207266: >> ??? ThreadMXBean::getThreadAllocatedBytes() can be quicker for self >> thread >> >> ??? Hi Paul, >> >> ??? It looks pretty good in general. >> >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html >> >> >> ??? It would be nice to refactor the java main() method as it becomes >> ??? too big. >> ??? Two ways ofgetCurrentThreadAllocatedBytes() testing are good >> candidates >> ??? to become separate methods. >> >> ???? ? 98?? ??????long size1 = mbean.getThreadAllocatedBytes(id); >> >> ??? Just wanted to double check if you wanted to invoke >> ??? the getCurrentThreadAllocatedBytes() instead as it is >> ??? a part of: >> >> ???? ? 85???????? // First way, getCurrentThreadAllocatedBytes >> >> >> ??? Thanks, >> ??? Serguei >> >> ??? On 9/13/19 12:11 PM, Hohensee, Paul wrote: >> >> ??????? Hi David, thanks for your comments. New webrev in >> >> >> ??????? http://cr.openjdk.java.net/~phh/8207266/webrev.08/ >> >> >> ??????? Both the old and new versions of the code check that thread >> allocated memory is both supported and enabled. The existing version >> of getThreadAllocatedBytes(long []) calls >> verifyThreadAllocatedMemory(long []), which checks inline to make sure >> thread allocated memory is supported, then calls >> isThreadAllocatedMemoryEnabled() to verify that it's enabled. >> isThreadAllocatedMemoryEnabled() duplicates (!) the support check and >> returns the enabled flag. I removed the redundant check in the new >> version. >> >> >> ??????? You're of course correct about the back-to-back check. >> Application code can't know when the runtime will hijack a thread for >> its own purposes. I've removed the check. >> >> >> ??????? Paul >> >> >> ??????? On 9/13/19, 12:50 AM, "David Holmes" >> ? wrote: >> >> >> ???????? ??? Hi Paul, >> >> >> ???????? ????On 13/09/2019 10:29 am, Hohensee, Paul wrote: >> >> ???????? ??? > Thanks for clarifying the review rules. Would someone >> from the >> >> ???????? ????> serviceability team please review? New webrev at >> >> ???????? ??? > >> >> ???????? ????>http://cr.openjdk.java.net/~phh/8207266/webrev.07/ >> >> >> ???????? ????One aspect of the functional change needs clarification >> for me - and >> >> ???????? ????apologies if this has been covered in the past. It seems >> to me that >> >> ???????? ????currently we only check isThreadAllocatedMemorySupported >> for these >> >> ???????? ????operations, but if I read things correctly the updated >> code additionally >> >> ???????? ????checks isThreadAllocatedMemoryEnabled, which is a >> behaviour change not >> >> ???????? ????mentioned in the CSR. >> >> >> ???????? ????> I didn?t disturb the existing checks in the test, just >> added code to >> >> ???????? ????> check the result of getThreadAllocatedBytes(long) on a >> non-current >> >> ???????? ????> thread, plus the back-to-back no-allocation checks. The >> former wasn?t >> >> ???????? ????> needed before because getThreadAllocatedBytes(long) was >> just a wrapper >> >> ???????? ????> around getThreadAllocatedBytes(long []). This patch >> changes that, so I >> >> ???????? ????> added a separate test. The latter is supposed to fail >> if there?s object >> >> ???????? ????> allocation on calls to getCurrentThreadAllocatedBytes and >> >> ???????? ????> getThreadAllocatedBytes(long). I.e., a feature, not a >> bug, because >> >> ???????? ????> accumulation of transient small objects can be a >> performance problem. >> >> ???????? ????> Thanks to your review, I noticed that the back-to-back >> check on the >> >> ???????? ????> current thread was using getThreadAllocatedBytes(long) >> instead of >> >> ???????? ????> getCurrentThreadAllocatedBytes and fixed it. I also >> removed all >> >> ???????? ????> instances of ?TEST FAILED: ?. >> >> >> ???????? ????The back-to-back check is not valid in general. You don't >> know if the >> >> ???????? ????first check might trigger some class loading on the >> return path after it >> >> ???????? ????has obtained the first memory value. The check might also >> fail if using >> >> ???????? ????JVMCI and some compilation related activity occurs in the >> current thread >> >> ???????? ????on the second call. Also with the introduction of >> handshakes its >> >> ???????? ????possible the current thread might hit a safepoint checks >> that results in >> >> ???????? ????it executing a handshake operation that performs >> allocation. Potentially >> >> ???????? ????there could be numerous non-deterministic actions that >> might occur >> >> ???????? ????leading to unanticipated allocation. >> >> >> ???????? ????I understand what you want to test here, I just don't >> think it is >> >> ???????? ????reliably doable. >> >> >> ???????? ????Thanks, >> >> ???????? ??? David >> >> ???????? ??? ----- >> >> >> ???????? ????> >> >> ???????? ????> Paul >> >> ???????? ??? > >> >> ???????? ????> *From: *Mandy Chung >> >> >> ???????? ??? > *Date: *Thursday, September 12, 2019 at 10:09 AM >> >> ???????? ??? > *To: *"Hohensee, Paul" >> >> >> ???????? ??? > *Cc: *OpenJDK >> Serviceability >> , >> >> ???????? ????>"hotspot-gc-dev at openjdk.java.net" >> >> >> >> >> ???????? ??? > *Subject: *Re: RFR (M): 8207266: >> ThreadMXBean::getThreadAllocatedBytes() >> >> ???????? ????> can be quicker for self thread >> >> ???????? ??? > >> >> ???????? ????> On 9/3/19 12:38 PM, Hohensee, Paul wrote: >> >> ???????? ??? > >> >> ???????? ????>???? Minor update in new >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. >> >> ???????? ??? > >> >> ???????? ????> >> >> ???????? ????> I only reviewed the library side implementation that >> looks good.? I >> >> ???????? ????> expect the serviceability team to review the test and >> hotspot change. >> >> ???????? ??? > >> >> ???????? ????> >> >> ???????? ????>???? Need a confirmatory review to push this. If I >> understand the rules correctly, it doesn't need a Reviewer review >> since Mandy's already reviewed it, it just needs a Committer review. >> >> ???????? ??? > >> >> ???????? ????> >> >> ???????? ????> You need another reviewer to advice the following >> because I was not >> >> ???????? ????> close to the ThreadsList work. >> >> ???????? ??? > >> >> ???????? ????> 2087?? ThreadsListHandle tlh; >> >> ???????? ??? > >> >> ???????? ????> 2088?? JavaThread* java_thread = >> tlh.list()->find_JavaThread_from_java_tid(thread_id); >> >> ???????? ??? > >> >> ???????? ????> 2089 >> >> ???????? ??? > >> >> ???????? ????> 2090?? if (java_thread != NULL) { >> >> ???????? ??? > >> >> ???????? ????> 2091???? return java_thread->cooked_allocated_bytes(); >> >> ???????? ??? > >> >> ???????? ????> 2092?? } >> >> ???????? ??? > >> >> ???????? ????> This looks right to me. >> >> ???????? ??? > >> >> ???????? ????> >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java >> >> ???????? ??? > >> >> ???????? ????> -??????????????? "ThreadAllocatedMemory is expected to >> be disabled"); >> >> ???????? ??? > >> >> ???????? ????> +??????????????? "TEST FAILED: ThreadAllocatedMemory is >> expected to be >> >> ???????? ????> disabled"); >> >> ???????? ??? > >> >> ???????? ????> Prepending "TEST FAILED" in exception message (in >> several places) >> >> ???????? ??? > >> >> ???????? ????> seems redundant since such RuntimeException is thrown >> and expected >> >> ???????? ??? > >> >> ???????? ????> a test failure. >> >> ???????? ??? > >> >> ???????? ????> +??????? // back-to-back calls shouldn't allocate any >> memory >> >> ???????? ??? > >> >> ???????? ????> +??????? size = mbean.getThreadAllocatedBytes(id); >> >> ???????? ??? > >> >> ???????? ????> +??????? size1 = mbean.getThreadAllocatedBytes(id); >> >> ???????? ??? > >> >> ???????? ????> +??????? if (size1 != size) { >> >> ???????? ??? > >> >> ???????? ????> Is there anything in the test can do to help guarantee >> this? I didn't >> >> ???????? ??? > >> >> ???????? ????> closely review this test.? The main thing I advice is >> to improve >> >> ???????? ??? > >> >> ???????? ????> the reliability of this test.? Put it in another way, >> we want to >> >> ???????? ??? > >> >> ???????? ????> ensure that this test change will pass all the time in >> various >> >> ???????? ??? > >> >> ???????? ????> test configuration. >> >> ???????? ??? > >> >> ???????? ????> Mandy >> >> ???????? ??? > >> >> >> >> >> From hohensee at amazon.com Wed Sep 18 23:47:02 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Wed, 18 Sep 2019 23:47:02 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> Message-ID: I'll take a look. ?On 9/18/19, 4:40 PM, "David Holmes" wrote: Paul, Unfortunately this patch has broken the vmTestbase/nsk/monitoring tests: [2019-09-18T22:59:32,349Z] /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: error: ServerThreadMXBeanNew is not abstract and does not override abstract method getCurrentThreadAllocatedBytes() in ThreadMXBean and possibly other issues as we are seeing hundreds of failures. David On 18/09/2019 8:50 am, David Holmes wrote: > On 18/09/2019 12:10 am, Hohensee, Paul wrote: >> Thanks, Serguei. :) >> >> David, are you ok with the patch? > > Yep, nothing further from me. > > David > >> Paul >> >> *From: *"serguei.spitsyn at oracle.com" >> *Date: *Tuesday, September 17, 2019 at 2:26 AM >> *To: *"Hohensee, Paul" , David Holmes >> , Mandy Chung >> *Cc: *OpenJDK Serviceability , >> "hotspot-gc-dev at openjdk.java.net" >> *Subject: *Re: RFR (M): 8207266: >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread >> >> Hi Paul, >> >> Thank you for refactoring and fixing the test. >> It looks great now! >> >> Thanks, >> Serguei >> >> >> On 9/15/19 02:52, Hohensee, Paul wrote: >> >> Hi, Serguei, thanks for the review. New webrev at >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ >> >> I refactored the test?s main() method, and you?re correct, >> getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in >> that context: fixed. >> >> Paul >> >> *From: *"serguei.spitsyn at oracle.com" >> >> >> *Organization: *Oracle Corporation >> *Date: *Friday, September 13, 2019 at 5:50 PM >> *To: *"Hohensee, Paul" >> , David Holmes >> , Mandy Chung >> >> *Cc: *OpenJDK Serviceability >> , >> "hotspot-gc-dev at openjdk.java.net" >> >> >> >> *Subject: *Re: RFR (M): 8207266: >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self >> thread >> >> Hi Paul, >> >> It looks pretty good in general. >> >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html >> >> >> It would be nice to refactor the java main() method as it becomes >> too big. >> Two ways ofgetCurrentThreadAllocatedBytes() testing are good >> candidates >> to become separate methods. >> >> 98 long size1 = mbean.getThreadAllocatedBytes(id); >> >> Just wanted to double check if you wanted to invoke >> the getCurrentThreadAllocatedBytes() instead as it is >> a part of: >> >> 85 // First way, getCurrentThreadAllocatedBytes >> >> >> Thanks, >> Serguei >> >> On 9/13/19 12:11 PM, Hohensee, Paul wrote: >> >> Hi David, thanks for your comments. New webrev in >> >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ >> >> >> Both the old and new versions of the code check that thread >> allocated memory is both supported and enabled. The existing version >> of getThreadAllocatedBytes(long []) calls >> verifyThreadAllocatedMemory(long []), which checks inline to make sure >> thread allocated memory is supported, then calls >> isThreadAllocatedMemoryEnabled() to verify that it's enabled. >> isThreadAllocatedMemoryEnabled() duplicates (!) the support check and >> returns the enabled flag. I removed the redundant check in the new >> version. >> >> >> You're of course correct about the back-to-back check. >> Application code can't know when the runtime will hijack a thread for >> its own purposes. I've removed the check. >> >> >> Paul >> >> >> On 9/13/19, 12:50 AM, "David Holmes" >> wrote: >> >> >> Hi Paul, >> >> >> On 13/09/2019 10:29 am, Hohensee, Paul wrote: >> >> > Thanks for clarifying the review rules. Would someone >> from the >> >> > serviceability team please review? New webrev at >> >> > >> >> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ >> >> >> One aspect of the functional change needs clarification >> for me - and >> >> apologies if this has been covered in the past. It seems >> to me that >> >> currently we only check isThreadAllocatedMemorySupported >> for these >> >> operations, but if I read things correctly the updated >> code additionally >> >> checks isThreadAllocatedMemoryEnabled, which is a >> behaviour change not >> >> mentioned in the CSR. >> >> >> > I didn?t disturb the existing checks in the test, just >> added code to >> >> > check the result of getThreadAllocatedBytes(long) on a >> non-current >> >> > thread, plus the back-to-back no-allocation checks. The >> former wasn?t >> >> > needed before because getThreadAllocatedBytes(long) was >> just a wrapper >> >> > around getThreadAllocatedBytes(long []). This patch >> changes that, so I >> >> > added a separate test. The latter is supposed to fail >> if there?s object >> >> > allocation on calls to getCurrentThreadAllocatedBytes and >> >> > getThreadAllocatedBytes(long). I.e., a feature, not a >> bug, because >> >> > accumulation of transient small objects can be a >> performance problem. >> >> > Thanks to your review, I noticed that the back-to-back >> check on the >> >> > current thread was using getThreadAllocatedBytes(long) >> instead of >> >> > getCurrentThreadAllocatedBytes and fixed it. I also >> removed all >> >> > instances of ?TEST FAILED: ?. >> >> >> The back-to-back check is not valid in general. You don't >> know if the >> >> first check might trigger some class loading on the >> return path after it >> >> has obtained the first memory value. The check might also >> fail if using >> >> JVMCI and some compilation related activity occurs in the >> current thread >> >> on the second call. Also with the introduction of >> handshakes its >> >> possible the current thread might hit a safepoint checks >> that results in >> >> it executing a handshake operation that performs >> allocation. Potentially >> >> there could be numerous non-deterministic actions that >> might occur >> >> leading to unanticipated allocation. >> >> >> I understand what you want to test here, I just don't >> think it is >> >> reliably doable. >> >> >> Thanks, >> >> David >> >> ----- >> >> >> > >> >> > Paul >> >> > >> >> > *From: *Mandy Chung >> >> >> > *Date: *Thursday, September 12, 2019 at 10:09 AM >> >> > *To: *"Hohensee, Paul" >> >> >> > *Cc: *OpenJDK >> Serviceability >> , >> >> >"hotspot-gc-dev at openjdk.java.net" >> >> >> >> >> > *Subject: *Re: RFR (M): 8207266: >> ThreadMXBean::getThreadAllocatedBytes() >> >> > can be quicker for self thread >> >> > >> >> > On 9/3/19 12:38 PM, Hohensee, Paul wrote: >> >> > >> >> > Minor update in new >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. >> >> > >> >> > >> >> > I only reviewed the library side implementation that >> looks good. I >> >> > expect the serviceability team to review the test and >> hotspot change. >> >> > >> >> > >> >> > Need a confirmatory review to push this. If I >> understand the rules correctly, it doesn't need a Reviewer review >> since Mandy's already reviewed it, it just needs a Committer review. >> >> > >> >> > >> >> > You need another reviewer to advice the following >> because I was not >> >> > close to the ThreadsList work. >> >> > >> >> > 2087 ThreadsListHandle tlh; >> >> > >> >> > 2088 JavaThread* java_thread = >> tlh.list()->find_JavaThread_from_java_tid(thread_id); >> >> > >> >> > 2089 >> >> > >> >> > 2090 if (java_thread != NULL) { >> >> > >> >> > 2091 return java_thread->cooked_allocated_bytes(); >> >> > >> >> > 2092 } >> >> > >> >> > This looks right to me. >> >> > >> >> > >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java >> >> > >> >> > - "ThreadAllocatedMemory is expected to >> be disabled"); >> >> > >> >> > + "TEST FAILED: ThreadAllocatedMemory is >> expected to be >> >> > disabled"); >> >> > >> >> > Prepending "TEST FAILED" in exception message (in >> several places) >> >> > >> >> > seems redundant since such RuntimeException is thrown >> and expected >> >> > >> >> > a test failure. >> >> > >> >> > + // back-to-back calls shouldn't allocate any >> memory >> >> > >> >> > + size = mbean.getThreadAllocatedBytes(id); >> >> > >> >> > + size1 = mbean.getThreadAllocatedBytes(id); >> >> > >> >> > + if (size1 != size) { >> >> > >> >> > Is there anything in the test can do to help guarantee >> this? I didn't >> >> > >> >> > closely review this test. The main thing I advice is >> to improve >> >> > >> >> > the reliability of this test. Put it in another way, >> we want to >> >> > >> >> > ensure that this test change will pass all the time in >> various >> >> > >> >> > test configuration. >> >> > >> >> > Mandy >> >> > >> >> >> >> >> From hohensee at amazon.com Thu Sep 19 00:00:58 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 19 Sep 2019 00:00:58 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> Message-ID: <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> They all implement com.sun.management.ThreadMXBean, so adding a getCurrentThreadAllocatedBytes broke them. Potential fix is to give it a default implementation, vis public default long getCurrentThreadAllocatedBytes() { return -1; } Shall I go with that, or reverse the original patch? ?On 9/18/19, 4:48 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: I'll take a look. On 9/18/19, 4:40 PM, "David Holmes" wrote: Paul, Unfortunately this patch has broken the vmTestbase/nsk/monitoring tests: [2019-09-18T22:59:32,349Z] /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: error: ServerThreadMXBeanNew is not abstract and does not override abstract method getCurrentThreadAllocatedBytes() in ThreadMXBean and possibly other issues as we are seeing hundreds of failures. David On 18/09/2019 8:50 am, David Holmes wrote: > On 18/09/2019 12:10 am, Hohensee, Paul wrote: >> Thanks, Serguei. :) >> >> David, are you ok with the patch? > > Yep, nothing further from me. > > David > >> Paul >> >> *From: *"serguei.spitsyn at oracle.com" >> *Date: *Tuesday, September 17, 2019 at 2:26 AM >> *To: *"Hohensee, Paul" , David Holmes >> , Mandy Chung >> *Cc: *OpenJDK Serviceability , >> "hotspot-gc-dev at openjdk.java.net" >> *Subject: *Re: RFR (M): 8207266: >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread >> >> Hi Paul, >> >> Thank you for refactoring and fixing the test. >> It looks great now! >> >> Thanks, >> Serguei >> >> >> On 9/15/19 02:52, Hohensee, Paul wrote: >> >> Hi, Serguei, thanks for the review. New webrev at >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ >> >> I refactored the test?s main() method, and you?re correct, >> getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in >> that context: fixed. >> >> Paul >> >> *From: *"serguei.spitsyn at oracle.com" >> >> >> *Organization: *Oracle Corporation >> *Date: *Friday, September 13, 2019 at 5:50 PM >> *To: *"Hohensee, Paul" >> , David Holmes >> , Mandy Chung >> >> *Cc: *OpenJDK Serviceability >> , >> "hotspot-gc-dev at openjdk.java.net" >> >> >> >> *Subject: *Re: RFR (M): 8207266: >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self >> thread >> >> Hi Paul, >> >> It looks pretty good in general. >> >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html >> >> >> It would be nice to refactor the java main() method as it becomes >> too big. >> Two ways ofgetCurrentThreadAllocatedBytes() testing are good >> candidates >> to become separate methods. >> >> 98 long size1 = mbean.getThreadAllocatedBytes(id); >> >> Just wanted to double check if you wanted to invoke >> the getCurrentThreadAllocatedBytes() instead as it is >> a part of: >> >> 85 // First way, getCurrentThreadAllocatedBytes >> >> >> Thanks, >> Serguei >> >> On 9/13/19 12:11 PM, Hohensee, Paul wrote: >> >> Hi David, thanks for your comments. New webrev in >> >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ >> >> >> Both the old and new versions of the code check that thread >> allocated memory is both supported and enabled. The existing version >> of getThreadAllocatedBytes(long []) calls >> verifyThreadAllocatedMemory(long []), which checks inline to make sure >> thread allocated memory is supported, then calls >> isThreadAllocatedMemoryEnabled() to verify that it's enabled. >> isThreadAllocatedMemoryEnabled() duplicates (!) the support check and >> returns the enabled flag. I removed the redundant check in the new >> version. >> >> >> You're of course correct about the back-to-back check. >> Application code can't know when the runtime will hijack a thread for >> its own purposes. I've removed the check. >> >> >> Paul >> >> >> On 9/13/19, 12:50 AM, "David Holmes" >> wrote: >> >> >> Hi Paul, >> >> >> On 13/09/2019 10:29 am, Hohensee, Paul wrote: >> >> > Thanks for clarifying the review rules. Would someone >> from the >> >> > serviceability team please review? New webrev at >> >> > >> >> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ >> >> >> One aspect of the functional change needs clarification >> for me - and >> >> apologies if this has been covered in the past. It seems >> to me that >> >> currently we only check isThreadAllocatedMemorySupported >> for these >> >> operations, but if I read things correctly the updated >> code additionally >> >> checks isThreadAllocatedMemoryEnabled, which is a >> behaviour change not >> >> mentioned in the CSR. >> >> >> > I didn?t disturb the existing checks in the test, just >> added code to >> >> > check the result of getThreadAllocatedBytes(long) on a >> non-current >> >> > thread, plus the back-to-back no-allocation checks. The >> former wasn?t >> >> > needed before because getThreadAllocatedBytes(long) was >> just a wrapper >> >> > around getThreadAllocatedBytes(long []). This patch >> changes that, so I >> >> > added a separate test. The latter is supposed to fail >> if there?s object >> >> > allocation on calls to getCurrentThreadAllocatedBytes and >> >> > getThreadAllocatedBytes(long). I.e., a feature, not a >> bug, because >> >> > accumulation of transient small objects can be a >> performance problem. >> >> > Thanks to your review, I noticed that the back-to-back >> check on the >> >> > current thread was using getThreadAllocatedBytes(long) >> instead of >> >> > getCurrentThreadAllocatedBytes and fixed it. I also >> removed all >> >> > instances of ?TEST FAILED: ?. >> >> >> The back-to-back check is not valid in general. You don't >> know if the >> >> first check might trigger some class loading on the >> return path after it >> >> has obtained the first memory value. The check might also >> fail if using >> >> JVMCI and some compilation related activity occurs in the >> current thread >> >> on the second call. Also with the introduction of >> handshakes its >> >> possible the current thread might hit a safepoint checks >> that results in >> >> it executing a handshake operation that performs >> allocation. Potentially >> >> there could be numerous non-deterministic actions that >> might occur >> >> leading to unanticipated allocation. >> >> >> I understand what you want to test here, I just don't >> think it is >> >> reliably doable. >> >> >> Thanks, >> >> David >> >> ----- >> >> >> > >> >> > Paul >> >> > >> >> > *From: *Mandy Chung >> >> >> > *Date: *Thursday, September 12, 2019 at 10:09 AM >> >> > *To: *"Hohensee, Paul" >> >> >> > *Cc: *OpenJDK >> Serviceability >> , >> >> >"hotspot-gc-dev at openjdk.java.net" >> >> >> >> >> > *Subject: *Re: RFR (M): 8207266: >> ThreadMXBean::getThreadAllocatedBytes() >> >> > can be quicker for self thread >> >> > >> >> > On 9/3/19 12:38 PM, Hohensee, Paul wrote: >> >> > >> >> > Minor update in new >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. >> >> > >> >> > >> >> > I only reviewed the library side implementation that >> looks good. I >> >> > expect the serviceability team to review the test and >> hotspot change. >> >> > >> >> > >> >> > Need a confirmatory review to push this. If I >> understand the rules correctly, it doesn't need a Reviewer review >> since Mandy's already reviewed it, it just needs a Committer review. >> >> > >> >> > >> >> > You need another reviewer to advice the following >> because I was not >> >> > close to the ThreadsList work. >> >> > >> >> > 2087 ThreadsListHandle tlh; >> >> > >> >> > 2088 JavaThread* java_thread = >> tlh.list()->find_JavaThread_from_java_tid(thread_id); >> >> > >> >> > 2089 >> >> > >> >> > 2090 if (java_thread != NULL) { >> >> > >> >> > 2091 return java_thread->cooked_allocated_bytes(); >> >> > >> >> > 2092 } >> >> > >> >> > This looks right to me. >> >> > >> >> > >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java >> >> > >> >> > - "ThreadAllocatedMemory is expected to >> be disabled"); >> >> > >> >> > + "TEST FAILED: ThreadAllocatedMemory is >> expected to be >> >> > disabled"); >> >> > >> >> > Prepending "TEST FAILED" in exception message (in >> several places) >> >> > >> >> > seems redundant since such RuntimeException is thrown >> and expected >> >> > >> >> > a test failure. >> >> > >> >> > + // back-to-back calls shouldn't allocate any >> memory >> >> > >> >> > + size = mbean.getThreadAllocatedBytes(id); >> >> > >> >> > + size1 = mbean.getThreadAllocatedBytes(id); >> >> > >> >> > + if (size1 != size) { >> >> > >> >> > Is there anything in the test can do to help guarantee >> this? I didn't >> >> > >> >> > closely review this test. The main thing I advice is >> to improve >> >> > >> >> > the reliability of this test. Put it in another way, >> we want to >> >> > >> >> > ensure that this test change will pass all the time in >> various >> >> > >> >> > test configuration. >> >> > >> >> > Mandy >> >> > >> >> >> >> >> From hohensee at amazon.com Thu Sep 19 00:10:02 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 19 Sep 2019 00:10:02 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> Message-ID: I've filed https://bugs.openjdk.java.net/browse/JDK-8231209 for this quick fix. A better fix is to support getCurrentThreadAllocatedBytes in these tests. ?On 9/18/19, 5:02 PM, "hotspot-gc-dev on behalf of Hohensee, Paul" wrote: They all implement com.sun.management.ThreadMXBean, so adding a getCurrentThreadAllocatedBytes broke them. Potential fix is to give it a default implementation, vis public default long getCurrentThreadAllocatedBytes() { return -1; } Shall I go with that, or reverse the original patch? On 9/18/19, 4:48 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: I'll take a look. On 9/18/19, 4:40 PM, "David Holmes" wrote: Paul, Unfortunately this patch has broken the vmTestbase/nsk/monitoring tests: [2019-09-18T22:59:32,349Z] /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: error: ServerThreadMXBeanNew is not abstract and does not override abstract method getCurrentThreadAllocatedBytes() in ThreadMXBean and possibly other issues as we are seeing hundreds of failures. David On 18/09/2019 8:50 am, David Holmes wrote: > On 18/09/2019 12:10 am, Hohensee, Paul wrote: >> Thanks, Serguei. :) >> >> David, are you ok with the patch? > > Yep, nothing further from me. > > David > >> Paul >> >> *From: *"serguei.spitsyn at oracle.com" >> *Date: *Tuesday, September 17, 2019 at 2:26 AM >> *To: *"Hohensee, Paul" , David Holmes >> , Mandy Chung >> *Cc: *OpenJDK Serviceability , >> "hotspot-gc-dev at openjdk.java.net" >> *Subject: *Re: RFR (M): 8207266: >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread >> >> Hi Paul, >> >> Thank you for refactoring and fixing the test. >> It looks great now! >> >> Thanks, >> Serguei >> >> >> On 9/15/19 02:52, Hohensee, Paul wrote: >> >> Hi, Serguei, thanks for the review. New webrev at >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ >> >> I refactored the test?s main() method, and you?re correct, >> getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in >> that context: fixed. >> >> Paul >> >> *From: *"serguei.spitsyn at oracle.com" >> >> >> *Organization: *Oracle Corporation >> *Date: *Friday, September 13, 2019 at 5:50 PM >> *To: *"Hohensee, Paul" >> , David Holmes >> , Mandy Chung >> >> *Cc: *OpenJDK Serviceability >> , >> "hotspot-gc-dev at openjdk.java.net" >> >> >> >> *Subject: *Re: RFR (M): 8207266: >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self >> thread >> >> Hi Paul, >> >> It looks pretty good in general. >> >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html >> >> >> It would be nice to refactor the java main() method as it becomes >> too big. >> Two ways ofgetCurrentThreadAllocatedBytes() testing are good >> candidates >> to become separate methods. >> >> 98 long size1 = mbean.getThreadAllocatedBytes(id); >> >> Just wanted to double check if you wanted to invoke >> the getCurrentThreadAllocatedBytes() instead as it is >> a part of: >> >> 85 // First way, getCurrentThreadAllocatedBytes >> >> >> Thanks, >> Serguei >> >> On 9/13/19 12:11 PM, Hohensee, Paul wrote: >> >> Hi David, thanks for your comments. New webrev in >> >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ >> >> >> Both the old and new versions of the code check that thread >> allocated memory is both supported and enabled. The existing version >> of getThreadAllocatedBytes(long []) calls >> verifyThreadAllocatedMemory(long []), which checks inline to make sure >> thread allocated memory is supported, then calls >> isThreadAllocatedMemoryEnabled() to verify that it's enabled. >> isThreadAllocatedMemoryEnabled() duplicates (!) the support check and >> returns the enabled flag. I removed the redundant check in the new >> version. >> >> >> You're of course correct about the back-to-back check. >> Application code can't know when the runtime will hijack a thread for >> its own purposes. I've removed the check. >> >> >> Paul >> >> >> On 9/13/19, 12:50 AM, "David Holmes" >> wrote: >> >> >> Hi Paul, >> >> >> On 13/09/2019 10:29 am, Hohensee, Paul wrote: >> >> > Thanks for clarifying the review rules. Would someone >> from the >> >> > serviceability team please review? New webrev at >> >> > >> >> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ >> >> >> One aspect of the functional change needs clarification >> for me - and >> >> apologies if this has been covered in the past. It seems >> to me that >> >> currently we only check isThreadAllocatedMemorySupported >> for these >> >> operations, but if I read things correctly the updated >> code additionally >> >> checks isThreadAllocatedMemoryEnabled, which is a >> behaviour change not >> >> mentioned in the CSR. >> >> >> > I didn?t disturb the existing checks in the test, just >> added code to >> >> > check the result of getThreadAllocatedBytes(long) on a >> non-current >> >> > thread, plus the back-to-back no-allocation checks. The >> former wasn?t >> >> > needed before because getThreadAllocatedBytes(long) was >> just a wrapper >> >> > around getThreadAllocatedBytes(long []). This patch >> changes that, so I >> >> > added a separate test. The latter is supposed to fail >> if there?s object >> >> > allocation on calls to getCurrentThreadAllocatedBytes and >> >> > getThreadAllocatedBytes(long). I.e., a feature, not a >> bug, because >> >> > accumulation of transient small objects can be a >> performance problem. >> >> > Thanks to your review, I noticed that the back-to-back >> check on the >> >> > current thread was using getThreadAllocatedBytes(long) >> instead of >> >> > getCurrentThreadAllocatedBytes and fixed it. I also >> removed all >> >> > instances of ?TEST FAILED: ?. >> >> >> The back-to-back check is not valid in general. You don't >> know if the >> >> first check might trigger some class loading on the >> return path after it >> >> has obtained the first memory value. The check might also >> fail if using >> >> JVMCI and some compilation related activity occurs in the >> current thread >> >> on the second call. Also with the introduction of >> handshakes its >> >> possible the current thread might hit a safepoint checks >> that results in >> >> it executing a handshake operation that performs >> allocation. Potentially >> >> there could be numerous non-deterministic actions that >> might occur >> >> leading to unanticipated allocation. >> >> >> I understand what you want to test here, I just don't >> think it is >> >> reliably doable. >> >> >> Thanks, >> >> David >> >> ----- >> >> >> > >> >> > Paul >> >> > >> >> > *From: *Mandy Chung >> >> >> > *Date: *Thursday, September 12, 2019 at 10:09 AM >> >> > *To: *"Hohensee, Paul" >> >> >> > *Cc: *OpenJDK >> Serviceability >> , >> >> >"hotspot-gc-dev at openjdk.java.net" >> >> >> >> >> > *Subject: *Re: RFR (M): 8207266: >> ThreadMXBean::getThreadAllocatedBytes() >> >> > can be quicker for self thread >> >> > >> >> > On 9/3/19 12:38 PM, Hohensee, Paul wrote: >> >> > >> >> > Minor update in new >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. >> >> > >> >> > >> >> > I only reviewed the library side implementation that >> looks good. I >> >> > expect the serviceability team to review the test and >> hotspot change. >> >> > >> >> > >> >> > Need a confirmatory review to push this. If I >> understand the rules correctly, it doesn't need a Reviewer review >> since Mandy's already reviewed it, it just needs a Committer review. >> >> > >> >> > >> >> > You need another reviewer to advice the following >> because I was not >> >> > close to the ThreadsList work. >> >> > >> >> > 2087 ThreadsListHandle tlh; >> >> > >> >> > 2088 JavaThread* java_thread = >> tlh.list()->find_JavaThread_from_java_tid(thread_id); >> >> > >> >> > 2089 >> >> > >> >> > 2090 if (java_thread != NULL) { >> >> > >> >> > 2091 return java_thread->cooked_allocated_bytes(); >> >> > >> >> > 2092 } >> >> > >> >> > This looks right to me. >> >> > >> >> > >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java >> >> > >> >> > - "ThreadAllocatedMemory is expected to >> be disabled"); >> >> > >> >> > + "TEST FAILED: ThreadAllocatedMemory is >> expected to be >> >> > disabled"); >> >> > >> >> > Prepending "TEST FAILED" in exception message (in >> several places) >> >> > >> >> > seems redundant since such RuntimeException is thrown >> and expected >> >> > >> >> > a test failure. >> >> > >> >> > + // back-to-back calls shouldn't allocate any >> memory >> >> > >> >> > + size = mbean.getThreadAllocatedBytes(id); >> >> > >> >> > + size1 = mbean.getThreadAllocatedBytes(id); >> >> > >> >> > + if (size1 != size) { >> >> > >> >> > Is there anything in the test can do to help guarantee >> this? I didn't >> >> > >> >> > closely review this test. The main thing I advice is >> to improve >> >> > >> >> > the reliability of this test. Put it in another way, >> we want to >> >> > >> >> > ensure that this test change will pass all the time in >> various >> >> > >> >> > test configuration. >> >> > >> >> > Mandy >> >> > >> >> >> >> >> From daniel.daugherty at oracle.com Thu Sep 19 00:14:13 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 18 Sep 2019 20:14:13 -0400 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> Message-ID: <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> > Shall I go with that, or reverse the original patch? I'm a bit worried about what else might show up since the NSK monitoring tests were not run prior to this push. I vote for backing out the fix until proper testing has been done (and at least the one problem fixed...) Dan On 9/18/19 8:00 PM, Hohensee, Paul wrote: > They all implement com.sun.management.ThreadMXBean, so adding a getCurrentThreadAllocatedBytes broke them. Potential fix is to give it a default implementation, vis > > public default long getCurrentThreadAllocatedBytes() { > return -1; > } > > Shall I go with that, or reverse the original patch? > > ?On 9/18/19, 4:48 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > I'll take a look. > > On 9/18/19, 4:40 PM, "David Holmes" wrote: > > Paul, > > Unfortunately this patch has broken the vmTestbase/nsk/monitoring tests: > > [2019-09-18T22:59:32,349Z] > /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: > error: ServerThreadMXBeanNew is not abstract and does not override > abstract method getCurrentThreadAllocatedBytes() in ThreadMXBean > > and possibly other issues as we are seeing hundreds of failures. > > David > > On 18/09/2019 8:50 am, David Holmes wrote: > > On 18/09/2019 12:10 am, Hohensee, Paul wrote: > >> Thanks, Serguei. :) > >> > >> David, are you ok with the patch? > > > > Yep, nothing further from me. > > > > David > > > >> Paul > >> > >> *From: *"serguei.spitsyn at oracle.com" > >> *Date: *Tuesday, September 17, 2019 at 2:26 AM > >> *To: *"Hohensee, Paul" , David Holmes > >> , Mandy Chung > >> *Cc: *OpenJDK Serviceability , > >> "hotspot-gc-dev at openjdk.java.net" > >> *Subject: *Re: RFR (M): 8207266: > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread > >> > >> Hi Paul, > >> > >> Thank you for refactoring and fixing the test. > >> It looks great now! > >> > >> Thanks, > >> Serguei > >> > >> > >> On 9/15/19 02:52, Hohensee, Paul wrote: > >> > >> Hi, Serguei, thanks for the review. New webrev at > >> > >> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ > >> > >> I refactored the test?s main() method, and you?re correct, > >> getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in > >> that context: fixed. > >> > >> Paul > >> > >> *From: *"serguei.spitsyn at oracle.com" > >> > >> > >> *Organization: *Oracle Corporation > >> *Date: *Friday, September 13, 2019 at 5:50 PM > >> *To: *"Hohensee, Paul" > >> , David Holmes > >> , Mandy Chung > >> > >> *Cc: *OpenJDK Serviceability > >> , > >> "hotspot-gc-dev at openjdk.java.net" > >> > >> > >> > >> *Subject: *Re: RFR (M): 8207266: > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self > >> thread > >> > >> Hi Paul, > >> > >> It looks pretty good in general. > >> > >> > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html > >> > >> > >> It would be nice to refactor the java main() method as it becomes > >> too big. > >> Two ways ofgetCurrentThreadAllocatedBytes() testing are good > >> candidates > >> to become separate methods. > >> > >> 98 long size1 = mbean.getThreadAllocatedBytes(id); > >> > >> Just wanted to double check if you wanted to invoke > >> the getCurrentThreadAllocatedBytes() instead as it is > >> a part of: > >> > >> 85 // First way, getCurrentThreadAllocatedBytes > >> > >> > >> Thanks, > >> Serguei > >> > >> On 9/13/19 12:11 PM, Hohensee, Paul wrote: > >> > >> Hi David, thanks for your comments. New webrev in > >> > >> > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ > >> > >> > >> Both the old and new versions of the code check that thread > >> allocated memory is both supported and enabled. The existing version > >> of getThreadAllocatedBytes(long []) calls > >> verifyThreadAllocatedMemory(long []), which checks inline to make sure > >> thread allocated memory is supported, then calls > >> isThreadAllocatedMemoryEnabled() to verify that it's enabled. > >> isThreadAllocatedMemoryEnabled() duplicates (!) the support check and > >> returns the enabled flag. I removed the redundant check in the new > >> version. > >> > >> > >> You're of course correct about the back-to-back check. > >> Application code can't know when the runtime will hijack a thread for > >> its own purposes. I've removed the check. > >> > >> > >> Paul > >> > >> > >> On 9/13/19, 12:50 AM, "David Holmes" > >> wrote: > >> > >> > >> Hi Paul, > >> > >> > >> On 13/09/2019 10:29 am, Hohensee, Paul wrote: > >> > >> > Thanks for clarifying the review rules. Would someone > >> from the > >> > >> > serviceability team please review? New webrev at > >> > >> > > >> > >> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ > >> > >> > >> One aspect of the functional change needs clarification > >> for me - and > >> > >> apologies if this has been covered in the past. It seems > >> to me that > >> > >> currently we only check isThreadAllocatedMemorySupported > >> for these > >> > >> operations, but if I read things correctly the updated > >> code additionally > >> > >> checks isThreadAllocatedMemoryEnabled, which is a > >> behaviour change not > >> > >> mentioned in the CSR. > >> > >> > >> > I didn?t disturb the existing checks in the test, just > >> added code to > >> > >> > check the result of getThreadAllocatedBytes(long) on a > >> non-current > >> > >> > thread, plus the back-to-back no-allocation checks. The > >> former wasn?t > >> > >> > needed before because getThreadAllocatedBytes(long) was > >> just a wrapper > >> > >> > around getThreadAllocatedBytes(long []). This patch > >> changes that, so I > >> > >> > added a separate test. The latter is supposed to fail > >> if there?s object > >> > >> > allocation on calls to getCurrentThreadAllocatedBytes and > >> > >> > getThreadAllocatedBytes(long). I.e., a feature, not a > >> bug, because > >> > >> > accumulation of transient small objects can be a > >> performance problem. > >> > >> > Thanks to your review, I noticed that the back-to-back > >> check on the > >> > >> > current thread was using getThreadAllocatedBytes(long) > >> instead of > >> > >> > getCurrentThreadAllocatedBytes and fixed it. I also > >> removed all > >> > >> > instances of ?TEST FAILED: ?. > >> > >> > >> The back-to-back check is not valid in general. You don't > >> know if the > >> > >> first check might trigger some class loading on the > >> return path after it > >> > >> has obtained the first memory value. The check might also > >> fail if using > >> > >> JVMCI and some compilation related activity occurs in the > >> current thread > >> > >> on the second call. Also with the introduction of > >> handshakes its > >> > >> possible the current thread might hit a safepoint checks > >> that results in > >> > >> it executing a handshake operation that performs > >> allocation. Potentially > >> > >> there could be numerous non-deterministic actions that > >> might occur > >> > >> leading to unanticipated allocation. > >> > >> > >> I understand what you want to test here, I just don't > >> think it is > >> > >> reliably doable. > >> > >> > >> Thanks, > >> > >> David > >> > >> ----- > >> > >> > >> > > >> > >> > Paul > >> > >> > > >> > >> > *From: *Mandy Chung > >> > >> > >> > *Date: *Thursday, September 12, 2019 at 10:09 AM > >> > >> > *To: *"Hohensee, Paul" > >> > >> > >> > *Cc: *OpenJDK > >> Serviceability > >> , > >> > >> >"hotspot-gc-dev at openjdk.java.net" > >> > >> > >> > >> > >> > *Subject: *Re: RFR (M): 8207266: > >> ThreadMXBean::getThreadAllocatedBytes() > >> > >> > can be quicker for self thread > >> > >> > > >> > >> > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > >> > >> > > >> > >> > Minor update in new > >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > >> > >> > > >> > >> > > >> > >> > I only reviewed the library side implementation that > >> looks good. I > >> > >> > expect the serviceability team to review the test and > >> hotspot change. > >> > >> > > >> > >> > > >> > >> > Need a confirmatory review to push this. If I > >> understand the rules correctly, it doesn't need a Reviewer review > >> since Mandy's already reviewed it, it just needs a Committer review. > >> > >> > > >> > >> > > >> > >> > You need another reviewer to advice the following > >> because I was not > >> > >> > close to the ThreadsList work. > >> > >> > > >> > >> > 2087 ThreadsListHandle tlh; > >> > >> > > >> > >> > 2088 JavaThread* java_thread = > >> tlh.list()->find_JavaThread_from_java_tid(thread_id); > >> > >> > > >> > >> > 2089 > >> > >> > > >> > >> > 2090 if (java_thread != NULL) { > >> > >> > > >> > >> > 2091 return java_thread->cooked_allocated_bytes(); > >> > >> > > >> > >> > 2092 } > >> > >> > > >> > >> > This looks right to me. > >> > >> > > >> > >> > > >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > >> > >> > > >> > >> > - "ThreadAllocatedMemory is expected to > >> be disabled"); > >> > >> > > >> > >> > + "TEST FAILED: ThreadAllocatedMemory is > >> expected to be > >> > >> > disabled"); > >> > >> > > >> > >> > Prepending "TEST FAILED" in exception message (in > >> several places) > >> > >> > > >> > >> > seems redundant since such RuntimeException is thrown > >> and expected > >> > >> > > >> > >> > a test failure. > >> > >> > > >> > >> > + // back-to-back calls shouldn't allocate any > >> memory > >> > >> > > >> > >> > + size = mbean.getThreadAllocatedBytes(id); > >> > >> > > >> > >> > + size1 = mbean.getThreadAllocatedBytes(id); > >> > >> > > >> > >> > + if (size1 != size) { > >> > >> > > >> > >> > Is there anything in the test can do to help guarantee > >> this? I didn't > >> > >> > > >> > >> > closely review this test. The main thing I advice is > >> to improve > >> > >> > > >> > >> > the reliability of this test. Put it in another way, > >> we want to > >> > >> > > >> > >> > ensure that this test change will pass all the time in > >> various > >> > >> > > >> > >> > test configuration. > >> > >> > > >> > >> > Mandy > >> > >> > > >> > >> > >> > >> > >> > > > > From hohensee at amazon.com Thu Sep 19 00:17:04 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 19 Sep 2019 00:17:04 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> Message-ID: <9A151A35-DA66-4B0F-B67F-E7F5BECF205A@amazon.com> Is there a tool that will generate a reversal patch? ?On 9/18/19, 5:14 PM, "Daniel D. Daugherty" wrote: > Shall I go with that, or reverse the original patch? I'm a bit worried about what else might show up since the NSK monitoring tests were not run prior to this push. I vote for backing out the fix until proper testing has been done (and at least the one problem fixed...) Dan On 9/18/19 8:00 PM, Hohensee, Paul wrote: > They all implement com.sun.management.ThreadMXBean, so adding a getCurrentThreadAllocatedBytes broke them. Potential fix is to give it a default implementation, vis > > public default long getCurrentThreadAllocatedBytes() { > return -1; > } > > Shall I go with that, or reverse the original patch? > > On 9/18/19, 4:48 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > I'll take a look. > > On 9/18/19, 4:40 PM, "David Holmes" wrote: > > Paul, > > Unfortunately this patch has broken the vmTestbase/nsk/monitoring tests: > > [2019-09-18T22:59:32,349Z] > /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: > error: ServerThreadMXBeanNew is not abstract and does not override > abstract method getCurrentThreadAllocatedBytes() in ThreadMXBean > > and possibly other issues as we are seeing hundreds of failures. > > David > > On 18/09/2019 8:50 am, David Holmes wrote: > > On 18/09/2019 12:10 am, Hohensee, Paul wrote: > >> Thanks, Serguei. :) > >> > >> David, are you ok with the patch? > > > > Yep, nothing further from me. > > > > David > > > >> Paul > >> > >> *From: *"serguei.spitsyn at oracle.com" > >> *Date: *Tuesday, September 17, 2019 at 2:26 AM > >> *To: *"Hohensee, Paul" , David Holmes > >> , Mandy Chung > >> *Cc: *OpenJDK Serviceability , > >> "hotspot-gc-dev at openjdk.java.net" > >> *Subject: *Re: RFR (M): 8207266: > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread > >> > >> Hi Paul, > >> > >> Thank you for refactoring and fixing the test. > >> It looks great now! > >> > >> Thanks, > >> Serguei > >> > >> > >> On 9/15/19 02:52, Hohensee, Paul wrote: > >> > >> Hi, Serguei, thanks for the review. New webrev at > >> > >> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ > >> > >> I refactored the test?s main() method, and you?re correct, > >> getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in > >> that context: fixed. > >> > >> Paul > >> > >> *From: *"serguei.spitsyn at oracle.com" > >> > >> > >> *Organization: *Oracle Corporation > >> *Date: *Friday, September 13, 2019 at 5:50 PM > >> *To: *"Hohensee, Paul" > >> , David Holmes > >> , Mandy Chung > >> > >> *Cc: *OpenJDK Serviceability > >> , > >> "hotspot-gc-dev at openjdk.java.net" > >> > >> > >> > >> *Subject: *Re: RFR (M): 8207266: > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self > >> thread > >> > >> Hi Paul, > >> > >> It looks pretty good in general. > >> > >> > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html > >> > >> > >> It would be nice to refactor the java main() method as it becomes > >> too big. > >> Two ways ofgetCurrentThreadAllocatedBytes() testing are good > >> candidates > >> to become separate methods. > >> > >> 98 long size1 = mbean.getThreadAllocatedBytes(id); > >> > >> Just wanted to double check if you wanted to invoke > >> the getCurrentThreadAllocatedBytes() instead as it is > >> a part of: > >> > >> 85 // First way, getCurrentThreadAllocatedBytes > >> > >> > >> Thanks, > >> Serguei > >> > >> On 9/13/19 12:11 PM, Hohensee, Paul wrote: > >> > >> Hi David, thanks for your comments. New webrev in > >> > >> > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ > >> > >> > >> Both the old and new versions of the code check that thread > >> allocated memory is both supported and enabled. The existing version > >> of getThreadAllocatedBytes(long []) calls > >> verifyThreadAllocatedMemory(long []), which checks inline to make sure > >> thread allocated memory is supported, then calls > >> isThreadAllocatedMemoryEnabled() to verify that it's enabled. > >> isThreadAllocatedMemoryEnabled() duplicates (!) the support check and > >> returns the enabled flag. I removed the redundant check in the new > >> version. > >> > >> > >> You're of course correct about the back-to-back check. > >> Application code can't know when the runtime will hijack a thread for > >> its own purposes. I've removed the check. > >> > >> > >> Paul > >> > >> > >> On 9/13/19, 12:50 AM, "David Holmes" > >> wrote: > >> > >> > >> Hi Paul, > >> > >> > >> On 13/09/2019 10:29 am, Hohensee, Paul wrote: > >> > >> > Thanks for clarifying the review rules. Would someone > >> from the > >> > >> > serviceability team please review? New webrev at > >> > >> > > >> > >> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ > >> > >> > >> One aspect of the functional change needs clarification > >> for me - and > >> > >> apologies if this has been covered in the past. It seems > >> to me that > >> > >> currently we only check isThreadAllocatedMemorySupported > >> for these > >> > >> operations, but if I read things correctly the updated > >> code additionally > >> > >> checks isThreadAllocatedMemoryEnabled, which is a > >> behaviour change not > >> > >> mentioned in the CSR. > >> > >> > >> > I didn?t disturb the existing checks in the test, just > >> added code to > >> > >> > check the result of getThreadAllocatedBytes(long) on a > >> non-current > >> > >> > thread, plus the back-to-back no-allocation checks. The > >> former wasn?t > >> > >> > needed before because getThreadAllocatedBytes(long) was > >> just a wrapper > >> > >> > around getThreadAllocatedBytes(long []). This patch > >> changes that, so I > >> > >> > added a separate test. The latter is supposed to fail > >> if there?s object > >> > >> > allocation on calls to getCurrentThreadAllocatedBytes and > >> > >> > getThreadAllocatedBytes(long). I.e., a feature, not a > >> bug, because > >> > >> > accumulation of transient small objects can be a > >> performance problem. > >> > >> > Thanks to your review, I noticed that the back-to-back > >> check on the > >> > >> > current thread was using getThreadAllocatedBytes(long) > >> instead of > >> > >> > getCurrentThreadAllocatedBytes and fixed it. I also > >> removed all > >> > >> > instances of ?TEST FAILED: ?. > >> > >> > >> The back-to-back check is not valid in general. You don't > >> know if the > >> > >> first check might trigger some class loading on the > >> return path after it > >> > >> has obtained the first memory value. The check might also > >> fail if using > >> > >> JVMCI and some compilation related activity occurs in the > >> current thread > >> > >> on the second call. Also with the introduction of > >> handshakes its > >> > >> possible the current thread might hit a safepoint checks > >> that results in > >> > >> it executing a handshake operation that performs > >> allocation. Potentially > >> > >> there could be numerous non-deterministic actions that > >> might occur > >> > >> leading to unanticipated allocation. > >> > >> > >> I understand what you want to test here, I just don't > >> think it is > >> > >> reliably doable. > >> > >> > >> Thanks, > >> > >> David > >> > >> ----- > >> > >> > >> > > >> > >> > Paul > >> > >> > > >> > >> > *From: *Mandy Chung > >> > >> > >> > *Date: *Thursday, September 12, 2019 at 10:09 AM > >> > >> > *To: *"Hohensee, Paul" > >> > >> > >> > *Cc: *OpenJDK > >> Serviceability > >> , > >> > >> >"hotspot-gc-dev at openjdk.java.net" > >> > >> > >> > >> > >> > *Subject: *Re: RFR (M): 8207266: > >> ThreadMXBean::getThreadAllocatedBytes() > >> > >> > can be quicker for self thread > >> > >> > > >> > >> > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > >> > >> > > >> > >> > Minor update in new > >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > >> > >> > > >> > >> > > >> > >> > I only reviewed the library side implementation that > >> looks good. I > >> > >> > expect the serviceability team to review the test and > >> hotspot change. > >> > >> > > >> > >> > > >> > >> > Need a confirmatory review to push this. If I > >> understand the rules correctly, it doesn't need a Reviewer review > >> since Mandy's already reviewed it, it just needs a Committer review. > >> > >> > > >> > >> > > >> > >> > You need another reviewer to advice the following > >> because I was not > >> > >> > close to the ThreadsList work. > >> > >> > > >> > >> > 2087 ThreadsListHandle tlh; > >> > >> > > >> > >> > 2088 JavaThread* java_thread = > >> tlh.list()->find_JavaThread_from_java_tid(thread_id); > >> > >> > > >> > >> > 2089 > >> > >> > > >> > >> > 2090 if (java_thread != NULL) { > >> > >> > > >> > >> > 2091 return java_thread->cooked_allocated_bytes(); > >> > >> > > >> > >> > 2092 } > >> > >> > > >> > >> > This looks right to me. > >> > >> > > >> > >> > > >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > >> > >> > > >> > >> > - "ThreadAllocatedMemory is expected to > >> be disabled"); > >> > >> > > >> > >> > + "TEST FAILED: ThreadAllocatedMemory is > >> expected to be > >> > >> > disabled"); > >> > >> > > >> > >> > Prepending "TEST FAILED" in exception message (in > >> several places) > >> > >> > > >> > >> > seems redundant since such RuntimeException is thrown > >> and expected > >> > >> > > >> > >> > a test failure. > >> > >> > > >> > >> > + // back-to-back calls shouldn't allocate any > >> memory > >> > >> > > >> > >> > + size = mbean.getThreadAllocatedBytes(id); > >> > >> > > >> > >> > + size1 = mbean.getThreadAllocatedBytes(id); > >> > >> > > >> > >> > + if (size1 != size) { > >> > >> > > >> > >> > Is there anything in the test can do to help guarantee > >> this? I didn't > >> > >> > > >> > >> > closely review this test. The main thing I advice is > >> to improve > >> > >> > > >> > >> > the reliability of this test. Put it in another way, > >> we want to > >> > >> > > >> > >> > ensure that this test change will pass all the time in > >> various > >> > >> > > >> > >> > test configuration. > >> > >> > > >> > >> > Mandy > >> > >> > > >> > >> > >> > >> > >> > > > > From daniel.daugherty at oracle.com Thu Sep 19 00:18:43 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 18 Sep 2019 20:18:43 -0400 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <9A151A35-DA66-4B0F-B67F-E7F5BECF205A@amazon.com> References: <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> <9A151A35-DA66-4B0F-B67F-E7F5BECF205A@amazon.com> Message-ID: % hg backout is the usual way to do this... Dan On 9/18/19 8:17 PM, Hohensee, Paul wrote: > Is there a tool that will generate a reversal patch? > > ?On 9/18/19, 5:14 PM, "Daniel D. Daugherty" wrote: > > > Shall I go with that, or reverse the original patch? > > I'm a bit worried about what else might show up since the > NSK monitoring tests were not run prior to this push. > > I vote for backing out the fix until proper testing has > been done (and at least the one problem fixed...) > > Dan > > > On 9/18/19 8:00 PM, Hohensee, Paul wrote: > > They all implement com.sun.management.ThreadMXBean, so adding a getCurrentThreadAllocatedBytes broke them. Potential fix is to give it a default implementation, vis > > > > public default long getCurrentThreadAllocatedBytes() { > > return -1; > > } > > > > Shall I go with that, or reverse the original patch? > > > > On 9/18/19, 4:48 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > I'll take a look. > > > > On 9/18/19, 4:40 PM, "David Holmes" wrote: > > > > Paul, > > > > Unfortunately this patch has broken the vmTestbase/nsk/monitoring tests: > > > > [2019-09-18T22:59:32,349Z] > > /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: > > error: ServerThreadMXBeanNew is not abstract and does not override > > abstract method getCurrentThreadAllocatedBytes() in ThreadMXBean > > > > and possibly other issues as we are seeing hundreds of failures. > > > > David > > > > On 18/09/2019 8:50 am, David Holmes wrote: > > > On 18/09/2019 12:10 am, Hohensee, Paul wrote: > > >> Thanks, Serguei. :) > > >> > > >> David, are you ok with the patch? > > > > > > Yep, nothing further from me. > > > > > > David > > > > > >> Paul > > >> > > >> *From: *"serguei.spitsyn at oracle.com" > > >> *Date: *Tuesday, September 17, 2019 at 2:26 AM > > >> *To: *"Hohensee, Paul" , David Holmes > > >> , Mandy Chung > > >> *Cc: *OpenJDK Serviceability , > > >> "hotspot-gc-dev at openjdk.java.net" > > >> *Subject: *Re: RFR (M): 8207266: > > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread > > >> > > >> Hi Paul, > > >> > > >> Thank you for refactoring and fixing the test. > > >> It looks great now! > > >> > > >> Thanks, > > >> Serguei > > >> > > >> > > >> On 9/15/19 02:52, Hohensee, Paul wrote: > > >> > > >> Hi, Serguei, thanks for the review. New webrev at > > >> > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ > > >> > > >> I refactored the test?s main() method, and you?re correct, > > >> getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in > > >> that context: fixed. > > >> > > >> Paul > > >> > > >> *From: *"serguei.spitsyn at oracle.com" > > >> > > >> > > >> *Organization: *Oracle Corporation > > >> *Date: *Friday, September 13, 2019 at 5:50 PM > > >> *To: *"Hohensee, Paul" > > >> , David Holmes > > >> , Mandy Chung > > >> > > >> *Cc: *OpenJDK Serviceability > > >> , > > >> "hotspot-gc-dev at openjdk.java.net" > > >> > > >> > > >> > > >> *Subject: *Re: RFR (M): 8207266: > > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self > > >> thread > > >> > > >> Hi Paul, > > >> > > >> It looks pretty good in general. > > >> > > >> > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html > > >> > > >> > > >> It would be nice to refactor the java main() method as it becomes > > >> too big. > > >> Two ways ofgetCurrentThreadAllocatedBytes() testing are good > > >> candidates > > >> to become separate methods. > > >> > > >> 98 long size1 = mbean.getThreadAllocatedBytes(id); > > >> > > >> Just wanted to double check if you wanted to invoke > > >> the getCurrentThreadAllocatedBytes() instead as it is > > >> a part of: > > >> > > >> 85 // First way, getCurrentThreadAllocatedBytes > > >> > > >> > > >> Thanks, > > >> Serguei > > >> > > >> On 9/13/19 12:11 PM, Hohensee, Paul wrote: > > >> > > >> Hi David, thanks for your comments. New webrev in > > >> > > >> > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ > > >> > > >> > > >> Both the old and new versions of the code check that thread > > >> allocated memory is both supported and enabled. The existing version > > >> of getThreadAllocatedBytes(long []) calls > > >> verifyThreadAllocatedMemory(long []), which checks inline to make sure > > >> thread allocated memory is supported, then calls > > >> isThreadAllocatedMemoryEnabled() to verify that it's enabled. > > >> isThreadAllocatedMemoryEnabled() duplicates (!) the support check and > > >> returns the enabled flag. I removed the redundant check in the new > > >> version. > > >> > > >> > > >> You're of course correct about the back-to-back check. > > >> Application code can't know when the runtime will hijack a thread for > > >> its own purposes. I've removed the check. > > >> > > >> > > >> Paul > > >> > > >> > > >> On 9/13/19, 12:50 AM, "David Holmes" > > >> wrote: > > >> > > >> > > >> Hi Paul, > > >> > > >> > > >> On 13/09/2019 10:29 am, Hohensee, Paul wrote: > > >> > > >> > Thanks for clarifying the review rules. Would someone > > >> from the > > >> > > >> > serviceability team please review? New webrev at > > >> > > >> > > > >> > > >> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ > > >> > > >> > > >> One aspect of the functional change needs clarification > > >> for me - and > > >> > > >> apologies if this has been covered in the past. It seems > > >> to me that > > >> > > >> currently we only check isThreadAllocatedMemorySupported > > >> for these > > >> > > >> operations, but if I read things correctly the updated > > >> code additionally > > >> > > >> checks isThreadAllocatedMemoryEnabled, which is a > > >> behaviour change not > > >> > > >> mentioned in the CSR. > > >> > > >> > > >> > I didn?t disturb the existing checks in the test, just > > >> added code to > > >> > > >> > check the result of getThreadAllocatedBytes(long) on a > > >> non-current > > >> > > >> > thread, plus the back-to-back no-allocation checks. The > > >> former wasn?t > > >> > > >> > needed before because getThreadAllocatedBytes(long) was > > >> just a wrapper > > >> > > >> > around getThreadAllocatedBytes(long []). This patch > > >> changes that, so I > > >> > > >> > added a separate test. The latter is supposed to fail > > >> if there?s object > > >> > > >> > allocation on calls to getCurrentThreadAllocatedBytes and > > >> > > >> > getThreadAllocatedBytes(long). I.e., a feature, not a > > >> bug, because > > >> > > >> > accumulation of transient small objects can be a > > >> performance problem. > > >> > > >> > Thanks to your review, I noticed that the back-to-back > > >> check on the > > >> > > >> > current thread was using getThreadAllocatedBytes(long) > > >> instead of > > >> > > >> > getCurrentThreadAllocatedBytes and fixed it. I also > > >> removed all > > >> > > >> > instances of ?TEST FAILED: ?. > > >> > > >> > > >> The back-to-back check is not valid in general. You don't > > >> know if the > > >> > > >> first check might trigger some class loading on the > > >> return path after it > > >> > > >> has obtained the first memory value. The check might also > > >> fail if using > > >> > > >> JVMCI and some compilation related activity occurs in the > > >> current thread > > >> > > >> on the second call. Also with the introduction of > > >> handshakes its > > >> > > >> possible the current thread might hit a safepoint checks > > >> that results in > > >> > > >> it executing a handshake operation that performs > > >> allocation. Potentially > > >> > > >> there could be numerous non-deterministic actions that > > >> might occur > > >> > > >> leading to unanticipated allocation. > > >> > > >> > > >> I understand what you want to test here, I just don't > > >> think it is > > >> > > >> reliably doable. > > >> > > >> > > >> Thanks, > > >> > > >> David > > >> > > >> ----- > > >> > > >> > > >> > > > >> > > >> > Paul > > >> > > >> > > > >> > > >> > *From: *Mandy Chung > > >> > > >> > > >> > *Date: *Thursday, September 12, 2019 at 10:09 AM > > >> > > >> > *To: *"Hohensee, Paul" > > >> > > >> > > >> > *Cc: *OpenJDK > > >> Serviceability > > >> , > > >> > > >> >"hotspot-gc-dev at openjdk.java.net" > > >> > > >> > > >> > > >> > > >> > *Subject: *Re: RFR (M): 8207266: > > >> ThreadMXBean::getThreadAllocatedBytes() > > >> > > >> > can be quicker for self thread > > >> > > >> > > > >> > > >> > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > > >> > > >> > > > >> > > >> > Minor update in new > > >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > > >> > > >> > > > >> > > >> > > > >> > > >> > I only reviewed the library side implementation that > > >> looks good. I > > >> > > >> > expect the serviceability team to review the test and > > >> hotspot change. > > >> > > >> > > > >> > > >> > > > >> > > >> > Need a confirmatory review to push this. If I > > >> understand the rules correctly, it doesn't need a Reviewer review > > >> since Mandy's already reviewed it, it just needs a Committer review. > > >> > > >> > > > >> > > >> > > > >> > > >> > You need another reviewer to advice the following > > >> because I was not > > >> > > >> > close to the ThreadsList work. > > >> > > >> > > > >> > > >> > 2087 ThreadsListHandle tlh; > > >> > > >> > > > >> > > >> > 2088 JavaThread* java_thread = > > >> tlh.list()->find_JavaThread_from_java_tid(thread_id); > > >> > > >> > > > >> > > >> > 2089 > > >> > > >> > > > >> > > >> > 2090 if (java_thread != NULL) { > > >> > > >> > > > >> > > >> > 2091 return java_thread->cooked_allocated_bytes(); > > >> > > >> > > > >> > > >> > 2092 } > > >> > > >> > > > >> > > >> > This looks right to me. > > >> > > >> > > > >> > > >> > > > >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > > >> > > >> > > > >> > > >> > - "ThreadAllocatedMemory is expected to > > >> be disabled"); > > >> > > >> > > > >> > > >> > + "TEST FAILED: ThreadAllocatedMemory is > > >> expected to be > > >> > > >> > disabled"); > > >> > > >> > > > >> > > >> > Prepending "TEST FAILED" in exception message (in > > >> several places) > > >> > > >> > > > >> > > >> > seems redundant since such RuntimeException is thrown > > >> and expected > > >> > > >> > > > >> > > >> > a test failure. > > >> > > >> > > > >> > > >> > + // back-to-back calls shouldn't allocate any > > >> memory > > >> > > >> > > > >> > > >> > + size = mbean.getThreadAllocatedBytes(id); > > >> > > >> > > > >> > > >> > + size1 = mbean.getThreadAllocatedBytes(id); > > >> > > >> > > > >> > > >> > + if (size1 != size) { > > >> > > >> > > > >> > > >> > Is there anything in the test can do to help guarantee > > >> this? I didn't > > >> > > >> > > > >> > > >> > closely review this test. The main thing I advice is > > >> to improve > > >> > > >> > > > >> > > >> > the reliability of this test. Put it in another way, > > >> we want to > > >> > > >> > > > >> > > >> > ensure that this test change will pass all the time in > > >> various > > >> > > >> > > > >> > > >> > test configuration. > > >> > > >> > > > >> > > >> > Mandy > > >> > > >> > > > >> > > >> > > >> > > >> > > >> > > > > > > > > > > > From hohensee at amazon.com Thu Sep 19 00:21:30 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 19 Sep 2019 00:21:30 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: References: <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> <9A151A35-DA66-4B0F-B67F-E7F5BECF205A@amazon.com> Message-ID: <586EB17C-1974-4FEF-A751-34B5B359BA61@amazon.com> Never having done this before, is it hg backout -r ? Do I file a JBS issue for the reversion? Seems necessary. ?On 9/18/19, 5:18 PM, "Daniel D. Daugherty" wrote: % hg backout is the usual way to do this... Dan On 9/18/19 8:17 PM, Hohensee, Paul wrote: > Is there a tool that will generate a reversal patch? > > On 9/18/19, 5:14 PM, "Daniel D. Daugherty" wrote: > > > Shall I go with that, or reverse the original patch? > > I'm a bit worried about what else might show up since the > NSK monitoring tests were not run prior to this push. > > I vote for backing out the fix until proper testing has > been done (and at least the one problem fixed...) > > Dan > > > On 9/18/19 8:00 PM, Hohensee, Paul wrote: > > They all implement com.sun.management.ThreadMXBean, so adding a getCurrentThreadAllocatedBytes broke them. Potential fix is to give it a default implementation, vis > > > > public default long getCurrentThreadAllocatedBytes() { > > return -1; > > } > > > > Shall I go with that, or reverse the original patch? > > > > On 9/18/19, 4:48 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > I'll take a look. > > > > On 9/18/19, 4:40 PM, "David Holmes" wrote: > > > > Paul, > > > > Unfortunately this patch has broken the vmTestbase/nsk/monitoring tests: > > > > [2019-09-18T22:59:32,349Z] > > /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: > > error: ServerThreadMXBeanNew is not abstract and does not override > > abstract method getCurrentThreadAllocatedBytes() in ThreadMXBean > > > > and possibly other issues as we are seeing hundreds of failures. > > > > David > > > > On 18/09/2019 8:50 am, David Holmes wrote: > > > On 18/09/2019 12:10 am, Hohensee, Paul wrote: > > >> Thanks, Serguei. :) > > >> > > >> David, are you ok with the patch? > > > > > > Yep, nothing further from me. > > > > > > David > > > > > >> Paul > > >> > > >> *From: *"serguei.spitsyn at oracle.com" > > >> *Date: *Tuesday, September 17, 2019 at 2:26 AM > > >> *To: *"Hohensee, Paul" , David Holmes > > >> , Mandy Chung > > >> *Cc: *OpenJDK Serviceability , > > >> "hotspot-gc-dev at openjdk.java.net" > > >> *Subject: *Re: RFR (M): 8207266: > > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread > > >> > > >> Hi Paul, > > >> > > >> Thank you for refactoring and fixing the test. > > >> It looks great now! > > >> > > >> Thanks, > > >> Serguei > > >> > > >> > > >> On 9/15/19 02:52, Hohensee, Paul wrote: > > >> > > >> Hi, Serguei, thanks for the review. New webrev at > > >> > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ > > >> > > >> I refactored the test?s main() method, and you?re correct, > > >> getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in > > >> that context: fixed. > > >> > > >> Paul > > >> > > >> *From: *"serguei.spitsyn at oracle.com" > > >> > > >> > > >> *Organization: *Oracle Corporation > > >> *Date: *Friday, September 13, 2019 at 5:50 PM > > >> *To: *"Hohensee, Paul" > > >> , David Holmes > > >> , Mandy Chung > > >> > > >> *Cc: *OpenJDK Serviceability > > >> , > > >> "hotspot-gc-dev at openjdk.java.net" > > >> > > >> > > >> > > >> *Subject: *Re: RFR (M): 8207266: > > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self > > >> thread > > >> > > >> Hi Paul, > > >> > > >> It looks pretty good in general. > > >> > > >> > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html > > >> > > >> > > >> It would be nice to refactor the java main() method as it becomes > > >> too big. > > >> Two ways ofgetCurrentThreadAllocatedBytes() testing are good > > >> candidates > > >> to become separate methods. > > >> > > >> 98 long size1 = mbean.getThreadAllocatedBytes(id); > > >> > > >> Just wanted to double check if you wanted to invoke > > >> the getCurrentThreadAllocatedBytes() instead as it is > > >> a part of: > > >> > > >> 85 // First way, getCurrentThreadAllocatedBytes > > >> > > >> > > >> Thanks, > > >> Serguei > > >> > > >> On 9/13/19 12:11 PM, Hohensee, Paul wrote: > > >> > > >> Hi David, thanks for your comments. New webrev in > > >> > > >> > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ > > >> > > >> > > >> Both the old and new versions of the code check that thread > > >> allocated memory is both supported and enabled. The existing version > > >> of getThreadAllocatedBytes(long []) calls > > >> verifyThreadAllocatedMemory(long []), which checks inline to make sure > > >> thread allocated memory is supported, then calls > > >> isThreadAllocatedMemoryEnabled() to verify that it's enabled. > > >> isThreadAllocatedMemoryEnabled() duplicates (!) the support check and > > >> returns the enabled flag. I removed the redundant check in the new > > >> version. > > >> > > >> > > >> You're of course correct about the back-to-back check. > > >> Application code can't know when the runtime will hijack a thread for > > >> its own purposes. I've removed the check. > > >> > > >> > > >> Paul > > >> > > >> > > >> On 9/13/19, 12:50 AM, "David Holmes" > > >> wrote: > > >> > > >> > > >> Hi Paul, > > >> > > >> > > >> On 13/09/2019 10:29 am, Hohensee, Paul wrote: > > >> > > >> > Thanks for clarifying the review rules. Would someone > > >> from the > > >> > > >> > serviceability team please review? New webrev at > > >> > > >> > > > >> > > >> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ > > >> > > >> > > >> One aspect of the functional change needs clarification > > >> for me - and > > >> > > >> apologies if this has been covered in the past. It seems > > >> to me that > > >> > > >> currently we only check isThreadAllocatedMemorySupported > > >> for these > > >> > > >> operations, but if I read things correctly the updated > > >> code additionally > > >> > > >> checks isThreadAllocatedMemoryEnabled, which is a > > >> behaviour change not > > >> > > >> mentioned in the CSR. > > >> > > >> > > >> > I didn?t disturb the existing checks in the test, just > > >> added code to > > >> > > >> > check the result of getThreadAllocatedBytes(long) on a > > >> non-current > > >> > > >> > thread, plus the back-to-back no-allocation checks. The > > >> former wasn?t > > >> > > >> > needed before because getThreadAllocatedBytes(long) was > > >> just a wrapper > > >> > > >> > around getThreadAllocatedBytes(long []). This patch > > >> changes that, so I > > >> > > >> > added a separate test. The latter is supposed to fail > > >> if there?s object > > >> > > >> > allocation on calls to getCurrentThreadAllocatedBytes and > > >> > > >> > getThreadAllocatedBytes(long). I.e., a feature, not a > > >> bug, because > > >> > > >> > accumulation of transient small objects can be a > > >> performance problem. > > >> > > >> > Thanks to your review, I noticed that the back-to-back > > >> check on the > > >> > > >> > current thread was using getThreadAllocatedBytes(long) > > >> instead of > > >> > > >> > getCurrentThreadAllocatedBytes and fixed it. I also > > >> removed all > > >> > > >> > instances of ?TEST FAILED: ?. > > >> > > >> > > >> The back-to-back check is not valid in general. You don't > > >> know if the > > >> > > >> first check might trigger some class loading on the > > >> return path after it > > >> > > >> has obtained the first memory value. The check might also > > >> fail if using > > >> > > >> JVMCI and some compilation related activity occurs in the > > >> current thread > > >> > > >> on the second call. Also with the introduction of > > >> handshakes its > > >> > > >> possible the current thread might hit a safepoint checks > > >> that results in > > >> > > >> it executing a handshake operation that performs > > >> allocation. Potentially > > >> > > >> there could be numerous non-deterministic actions that > > >> might occur > > >> > > >> leading to unanticipated allocation. > > >> > > >> > > >> I understand what you want to test here, I just don't > > >> think it is > > >> > > >> reliably doable. > > >> > > >> > > >> Thanks, > > >> > > >> David > > >> > > >> ----- > > >> > > >> > > >> > > > >> > > >> > Paul > > >> > > >> > > > >> > > >> > *From: *Mandy Chung > > >> > > >> > > >> > *Date: *Thursday, September 12, 2019 at 10:09 AM > > >> > > >> > *To: *"Hohensee, Paul" > > >> > > >> > > >> > *Cc: *OpenJDK > > >> Serviceability > > >> , > > >> > > >> >"hotspot-gc-dev at openjdk.java.net" > > >> > > >> > > >> > > >> > > >> > *Subject: *Re: RFR (M): 8207266: > > >> ThreadMXBean::getThreadAllocatedBytes() > > >> > > >> > can be quicker for self thread > > >> > > >> > > > >> > > >> > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > > >> > > >> > > > >> > > >> > Minor update in new > > >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > > >> > > >> > > > >> > > >> > > > >> > > >> > I only reviewed the library side implementation that > > >> looks good. I > > >> > > >> > expect the serviceability team to review the test and > > >> hotspot change. > > >> > > >> > > > >> > > >> > > > >> > > >> > Need a confirmatory review to push this. If I > > >> understand the rules correctly, it doesn't need a Reviewer review > > >> since Mandy's already reviewed it, it just needs a Committer review. > > >> > > >> > > > >> > > >> > > > >> > > >> > You need another reviewer to advice the following > > >> because I was not > > >> > > >> > close to the ThreadsList work. > > >> > > >> > > > >> > > >> > 2087 ThreadsListHandle tlh; > > >> > > >> > > > >> > > >> > 2088 JavaThread* java_thread = > > >> tlh.list()->find_JavaThread_from_java_tid(thread_id); > > >> > > >> > > > >> > > >> > 2089 > > >> > > >> > > > >> > > >> > 2090 if (java_thread != NULL) { > > >> > > >> > > > >> > > >> > 2091 return java_thread->cooked_allocated_bytes(); > > >> > > >> > > > >> > > >> > 2092 } > > >> > > >> > > > >> > > >> > This looks right to me. > > >> > > >> > > > >> > > >> > > > >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > > >> > > >> > > > >> > > >> > - "ThreadAllocatedMemory is expected to > > >> be disabled"); > > >> > > >> > > > >> > > >> > + "TEST FAILED: ThreadAllocatedMemory is > > >> expected to be > > >> > > >> > disabled"); > > >> > > >> > > > >> > > >> > Prepending "TEST FAILED" in exception message (in > > >> several places) > > >> > > >> > > > >> > > >> > seems redundant since such RuntimeException is thrown > > >> and expected > > >> > > >> > > > >> > > >> > a test failure. > > >> > > >> > > > >> > > >> > + // back-to-back calls shouldn't allocate any > > >> memory > > >> > > >> > > > >> > > >> > + size = mbean.getThreadAllocatedBytes(id); > > >> > > >> > > > >> > > >> > + size1 = mbean.getThreadAllocatedBytes(id); > > >> > > >> > > > >> > > >> > + if (size1 != size) { > > >> > > >> > > > >> > > >> > Is there anything in the test can do to help guarantee > > >> this? I didn't > > >> > > >> > > > >> > > >> > closely review this test. The main thing I advice is > > >> to improve > > >> > > >> > > > >> > > >> > the reliability of this test. Put it in another way, > > >> we want to > > >> > > >> > > > >> > > >> > ensure that this test change will pass all the time in > > >> various > > >> > > >> > > > >> > > >> > test configuration. > > >> > > >> > > > >> > > >> > Mandy > > >> > > >> > > > >> > > >> > > >> > > >> > > >> > > > > > > > > > > > From david.holmes at oracle.com Thu Sep 19 00:21:40 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 19 Sep 2019 10:21:40 +1000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> Message-ID: <67eb44c3-a9e9-7377-a4e5-01f07ccc5462@oracle.com> Hi Paul, On 19/09/2019 10:00 am, Hohensee, Paul wrote: > They all implement com.sun.management.ThreadMXBean, so adding a getCurrentThreadAllocatedBytes broke them. Potential fix is to give it a default implementation, vis > > public default long getCurrentThreadAllocatedBytes() { > return -1; > } > > Shall I go with that, or reverse the original patch? Any default implementation must follow the specification for the new method: * equivalent to calling: * 
  *   {@link #getThreadAllocatedBytes 
getThreadAllocatedBytes}(Thread.currentThread().getId());
  *
The CSR request for this seems to be quite misleading in the compatibility section: "Adds a method with well-understood semantics and leaves existing method semantics unchanged. Existing programs will continue to compile and execute as before. " It may have been assumed, based on the wording, that there would be a default implementation, otherwise the second sentence is obviously not correct. But it's clear from the specification section that this was not going to be a default method. It may be better to back this out and redo with a revised CSR request. Thanks, David > ?On 9/18/19, 4:48 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > I'll take a look. > > On 9/18/19, 4:40 PM, "David Holmes" wrote: > > Paul, > > Unfortunately this patch has broken the vmTestbase/nsk/monitoring tests: > > [2019-09-18T22:59:32,349Z] > /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: > error: ServerThreadMXBeanNew is not abstract and does not override > abstract method getCurrentThreadAllocatedBytes() in ThreadMXBean > > and possibly other issues as we are seeing hundreds of failures. > > David > > On 18/09/2019 8:50 am, David Holmes wrote: > > On 18/09/2019 12:10 am, Hohensee, Paul wrote: > >> Thanks, Serguei. :) > >> > >> David, are you ok with the patch? > > > > Yep, nothing further from me. > > > > David > > > >> Paul > >> > >> *From: *"serguei.spitsyn at oracle.com" > >> *Date: *Tuesday, September 17, 2019 at 2:26 AM > >> *To: *"Hohensee, Paul" , David Holmes > >> , Mandy Chung > >> *Cc: *OpenJDK Serviceability , > >> "hotspot-gc-dev at openjdk.java.net" > >> *Subject: *Re: RFR (M): 8207266: > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread > >> > >> Hi Paul, > >> > >> Thank you for refactoring and fixing the test. > >> It looks great now! > >> > >> Thanks, > >> Serguei > >> > >> > >> On 9/15/19 02:52, Hohensee, Paul wrote: > >> > >> Hi, Serguei, thanks for the review. New webrev at > >> > >> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ > >> > >> I refactored the test?s main() method, and you?re correct, > >> getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in > >> that context: fixed. > >> > >> Paul > >> > >> *From: *"serguei.spitsyn at oracle.com" > >> > >> > >> *Organization: *Oracle Corporation > >> *Date: *Friday, September 13, 2019 at 5:50 PM > >> *To: *"Hohensee, Paul" > >> , David Holmes > >> , Mandy Chung > >> > >> *Cc: *OpenJDK Serviceability > >> , > >> "hotspot-gc-dev at openjdk.java.net" > >> > >> > >> > >> *Subject: *Re: RFR (M): 8207266: > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self > >> thread > >> > >> Hi Paul, > >> > >> It looks pretty good in general. > >> > >> > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html > >> > >> > >> It would be nice to refactor the java main() method as it becomes > >> too big. > >> Two ways ofgetCurrentThreadAllocatedBytes() testing are good > >> candidates > >> to become separate methods. > >> > >> 98 long size1 = mbean.getThreadAllocatedBytes(id); > >> > >> Just wanted to double check if you wanted to invoke > >> the getCurrentThreadAllocatedBytes() instead as it is > >> a part of: > >> > >> 85 // First way, getCurrentThreadAllocatedBytes > >> > >> > >> Thanks, > >> Serguei > >> > >> On 9/13/19 12:11 PM, Hohensee, Paul wrote: > >> > >> Hi David, thanks for your comments. New webrev in > >> > >> > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ > >> > >> > >> Both the old and new versions of the code check that thread > >> allocated memory is both supported and enabled. The existing version > >> of getThreadAllocatedBytes(long []) calls > >> verifyThreadAllocatedMemory(long []), which checks inline to make sure > >> thread allocated memory is supported, then calls > >> isThreadAllocatedMemoryEnabled() to verify that it's enabled. > >> isThreadAllocatedMemoryEnabled() duplicates (!) the support check and > >> returns the enabled flag. I removed the redundant check in the new > >> version. > >> > >> > >> You're of course correct about the back-to-back check. > >> Application code can't know when the runtime will hijack a thread for > >> its own purposes. I've removed the check. > >> > >> > >> Paul > >> > >> > >> On 9/13/19, 12:50 AM, "David Holmes" > >> wrote: > >> > >> > >> Hi Paul, > >> > >> > >> On 13/09/2019 10:29 am, Hohensee, Paul wrote: > >> > >> > Thanks for clarifying the review rules. Would someone > >> from the > >> > >> > serviceability team please review? New webrev at > >> > >> > > >> > >> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ > >> > >> > >> One aspect of the functional change needs clarification > >> for me - and > >> > >> apologies if this has been covered in the past. It seems > >> to me that > >> > >> currently we only check isThreadAllocatedMemorySupported > >> for these > >> > >> operations, but if I read things correctly the updated > >> code additionally > >> > >> checks isThreadAllocatedMemoryEnabled, which is a > >> behaviour change not > >> > >> mentioned in the CSR. > >> > >> > >> > I didn?t disturb the existing checks in the test, just > >> added code to > >> > >> > check the result of getThreadAllocatedBytes(long) on a > >> non-current > >> > >> > thread, plus the back-to-back no-allocation checks. The > >> former wasn?t > >> > >> > needed before because getThreadAllocatedBytes(long) was > >> just a wrapper > >> > >> > around getThreadAllocatedBytes(long []). This patch > >> changes that, so I > >> > >> > added a separate test. The latter is supposed to fail > >> if there?s object > >> > >> > allocation on calls to getCurrentThreadAllocatedBytes and > >> > >> > getThreadAllocatedBytes(long). I.e., a feature, not a > >> bug, because > >> > >> > accumulation of transient small objects can be a > >> performance problem. > >> > >> > Thanks to your review, I noticed that the back-to-back > >> check on the > >> > >> > current thread was using getThreadAllocatedBytes(long) > >> instead of > >> > >> > getCurrentThreadAllocatedBytes and fixed it. I also > >> removed all > >> > >> > instances of ?TEST FAILED: ?. > >> > >> > >> The back-to-back check is not valid in general. You don't > >> know if the > >> > >> first check might trigger some class loading on the > >> return path after it > >> > >> has obtained the first memory value. The check might also > >> fail if using > >> > >> JVMCI and some compilation related activity occurs in the > >> current thread > >> > >> on the second call. Also with the introduction of > >> handshakes its > >> > >> possible the current thread might hit a safepoint checks > >> that results in > >> > >> it executing a handshake operation that performs > >> allocation. Potentially > >> > >> there could be numerous non-deterministic actions that > >> might occur > >> > >> leading to unanticipated allocation. > >> > >> > >> I understand what you want to test here, I just don't > >> think it is > >> > >> reliably doable. > >> > >> > >> Thanks, > >> > >> David > >> > >> ----- > >> > >> > >> > > >> > >> > Paul > >> > >> > > >> > >> > *From: *Mandy Chung > >> > >> > >> > *Date: *Thursday, September 12, 2019 at 10:09 AM > >> > >> > *To: *"Hohensee, Paul" > >> > >> > >> > *Cc: *OpenJDK > >> Serviceability > >> , > >> > >> >"hotspot-gc-dev at openjdk.java.net" > >> > >> > >> > >> > >> > *Subject: *Re: RFR (M): 8207266: > >> ThreadMXBean::getThreadAllocatedBytes() > >> > >> > can be quicker for self thread > >> > >> > > >> > >> > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > >> > >> > > >> > >> > Minor update in new > >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > >> > >> > > >> > >> > > >> > >> > I only reviewed the library side implementation that > >> looks good. I > >> > >> > expect the serviceability team to review the test and > >> hotspot change. > >> > >> > > >> > >> > > >> > >> > Need a confirmatory review to push this. If I > >> understand the rules correctly, it doesn't need a Reviewer review > >> since Mandy's already reviewed it, it just needs a Committer review. > >> > >> > > >> > >> > > >> > >> > You need another reviewer to advice the following > >> because I was not > >> > >> > close to the ThreadsList work. > >> > >> > > >> > >> > 2087 ThreadsListHandle tlh; > >> > >> > > >> > >> > 2088 JavaThread* java_thread = > >> tlh.list()->find_JavaThread_from_java_tid(thread_id); > >> > >> > > >> > >> > 2089 > >> > >> > > >> > >> > 2090 if (java_thread != NULL) { > >> > >> > > >> > >> > 2091 return java_thread->cooked_allocated_bytes(); > >> > >> > > >> > >> > 2092 } > >> > >> > > >> > >> > This looks right to me. > >> > >> > > >> > >> > > >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > >> > >> > > >> > >> > - "ThreadAllocatedMemory is expected to > >> be disabled"); > >> > >> > > >> > >> > + "TEST FAILED: ThreadAllocatedMemory is > >> expected to be > >> > >> > disabled"); > >> > >> > > >> > >> > Prepending "TEST FAILED" in exception message (in > >> several places) > >> > >> > > >> > >> > seems redundant since such RuntimeException is thrown > >> and expected > >> > >> > > >> > >> > a test failure. > >> > >> > > >> > >> > + // back-to-back calls shouldn't allocate any > >> memory > >> > >> > > >> > >> > + size = mbean.getThreadAllocatedBytes(id); > >> > >> > > >> > >> > + size1 = mbean.getThreadAllocatedBytes(id); > >> > >> > > >> > >> > + if (size1 != size) { > >> > >> > > >> > >> > Is there anything in the test can do to help guarantee > >> this? I didn't > >> > >> > > >> > >> > closely review this test. The main thing I advice is > >> to improve > >> > >> > > >> > >> > the reliability of this test. Put it in another way, > >> we want to > >> > >> > > >> > >> > ensure that this test change will pass all the time in > >> various > >> > >> > > >> > >> > test configuration. > >> > >> > > >> > >> > Mandy > >> > >> > > >> > >> > >> > >> > >> > > > > From daniel.daugherty at oracle.com Thu Sep 19 00:25:19 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 18 Sep 2019 20:25:19 -0400 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <586EB17C-1974-4FEF-A751-34B5B359BA61@amazon.com> References: <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> <9A151A35-DA66-4B0F-B67F-E7F5BECF205A@amazon.com> <586EB17C-1974-4FEF-A751-34B5B359BA61@amazon.com> Message-ID: <6139732f-a914-c59c-2119-517e480a279c@oracle.com> I created this sub-task for you: JDK-8231210 [BACKOUT] JDK-8207266 ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread https://bugs.openjdk.java.net/browse/JDK-8231210 If you would prefer, I can handle this backout for you. Dan On 9/18/19 8:21 PM, Hohensee, Paul wrote: > Never having done this before, is it > > hg backout -r > > ? Do I file a JBS issue for the reversion? Seems necessary. > > ?On 9/18/19, 5:18 PM, "Daniel D. Daugherty" wrote: > > % hg backout > > is the usual way to do this... > > Dan > > > On 9/18/19 8:17 PM, Hohensee, Paul wrote: > > Is there a tool that will generate a reversal patch? > > > > On 9/18/19, 5:14 PM, "Daniel D. Daugherty" wrote: > > > > > Shall I go with that, or reverse the original patch? > > > > I'm a bit worried about what else might show up since the > > NSK monitoring tests were not run prior to this push. > > > > I vote for backing out the fix until proper testing has > > been done (and at least the one problem fixed...) > > > > Dan > > > > > > On 9/18/19 8:00 PM, Hohensee, Paul wrote: > > > They all implement com.sun.management.ThreadMXBean, so adding a getCurrentThreadAllocatedBytes broke them. Potential fix is to give it a default implementation, vis > > > > > > public default long getCurrentThreadAllocatedBytes() { > > > return -1; > > > } > > > > > > Shall I go with that, or reverse the original patch? > > > > > > On 9/18/19, 4:48 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > > > I'll take a look. > > > > > > On 9/18/19, 4:40 PM, "David Holmes" wrote: > > > > > > Paul, > > > > > > Unfortunately this patch has broken the vmTestbase/nsk/monitoring tests: > > > > > > [2019-09-18T22:59:32,349Z] > > > /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: > > > error: ServerThreadMXBeanNew is not abstract and does not override > > > abstract method getCurrentThreadAllocatedBytes() in ThreadMXBean > > > > > > and possibly other issues as we are seeing hundreds of failures. > > > > > > David > > > > > > On 18/09/2019 8:50 am, David Holmes wrote: > > > > On 18/09/2019 12:10 am, Hohensee, Paul wrote: > > > >> Thanks, Serguei. :) > > > >> > > > >> David, are you ok with the patch? > > > > > > > > Yep, nothing further from me. > > > > > > > > David > > > > > > > >> Paul > > > >> > > > >> *From: *"serguei.spitsyn at oracle.com" > > > >> *Date: *Tuesday, September 17, 2019 at 2:26 AM > > > >> *To: *"Hohensee, Paul" , David Holmes > > > >> , Mandy Chung > > > >> *Cc: *OpenJDK Serviceability , > > > >> "hotspot-gc-dev at openjdk.java.net" > > > >> *Subject: *Re: RFR (M): 8207266: > > > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread > > > >> > > > >> Hi Paul, > > > >> > > > >> Thank you for refactoring and fixing the test. > > > >> It looks great now! > > > >> > > > >> Thanks, > > > >> Serguei > > > >> > > > >> > > > >> On 9/15/19 02:52, Hohensee, Paul wrote: > > > >> > > > >> Hi, Serguei, thanks for the review. New webrev at > > > >> > > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ > > > >> > > > >> I refactored the test?s main() method, and you?re correct, > > > >> getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in > > > >> that context: fixed. > > > >> > > > >> Paul > > > >> > > > >> *From: *"serguei.spitsyn at oracle.com" > > > >> > > > >> > > > >> *Organization: *Oracle Corporation > > > >> *Date: *Friday, September 13, 2019 at 5:50 PM > > > >> *To: *"Hohensee, Paul" > > > >> , David Holmes > > > >> , Mandy Chung > > > >> > > > >> *Cc: *OpenJDK Serviceability > > > >> , > > > >> "hotspot-gc-dev at openjdk.java.net" > > > >> > > > >> > > > >> > > > >> *Subject: *Re: RFR (M): 8207266: > > > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self > > > >> thread > > > >> > > > >> Hi Paul, > > > >> > > > >> It looks pretty good in general. > > > >> > > > >> > > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html > > > >> > > > >> > > > >> It would be nice to refactor the java main() method as it becomes > > > >> too big. > > > >> Two ways ofgetCurrentThreadAllocatedBytes() testing are good > > > >> candidates > > > >> to become separate methods. > > > >> > > > >> 98 long size1 = mbean.getThreadAllocatedBytes(id); > > > >> > > > >> Just wanted to double check if you wanted to invoke > > > >> the getCurrentThreadAllocatedBytes() instead as it is > > > >> a part of: > > > >> > > > >> 85 // First way, getCurrentThreadAllocatedBytes > > > >> > > > >> > > > >> Thanks, > > > >> Serguei > > > >> > > > >> On 9/13/19 12:11 PM, Hohensee, Paul wrote: > > > >> > > > >> Hi David, thanks for your comments. New webrev in > > > >> > > > >> > > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ > > > >> > > > >> > > > >> Both the old and new versions of the code check that thread > > > >> allocated memory is both supported and enabled. The existing version > > > >> of getThreadAllocatedBytes(long []) calls > > > >> verifyThreadAllocatedMemory(long []), which checks inline to make sure > > > >> thread allocated memory is supported, then calls > > > >> isThreadAllocatedMemoryEnabled() to verify that it's enabled. > > > >> isThreadAllocatedMemoryEnabled() duplicates (!) the support check and > > > >> returns the enabled flag. I removed the redundant check in the new > > > >> version. > > > >> > > > >> > > > >> You're of course correct about the back-to-back check. > > > >> Application code can't know when the runtime will hijack a thread for > > > >> its own purposes. I've removed the check. > > > >> > > > >> > > > >> Paul > > > >> > > > >> > > > >> On 9/13/19, 12:50 AM, "David Holmes" > > > >> wrote: > > > >> > > > >> > > > >> Hi Paul, > > > >> > > > >> > > > >> On 13/09/2019 10:29 am, Hohensee, Paul wrote: > > > >> > > > >> > Thanks for clarifying the review rules. Would someone > > > >> from the > > > >> > > > >> > serviceability team please review? New webrev at > > > >> > > > >> > > > > >> > > > >> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ > > > >> > > > >> > > > >> One aspect of the functional change needs clarification > > > >> for me - and > > > >> > > > >> apologies if this has been covered in the past. It seems > > > >> to me that > > > >> > > > >> currently we only check isThreadAllocatedMemorySupported > > > >> for these > > > >> > > > >> operations, but if I read things correctly the updated > > > >> code additionally > > > >> > > > >> checks isThreadAllocatedMemoryEnabled, which is a > > > >> behaviour change not > > > >> > > > >> mentioned in the CSR. > > > >> > > > >> > > > >> > I didn?t disturb the existing checks in the test, just > > > >> added code to > > > >> > > > >> > check the result of getThreadAllocatedBytes(long) on a > > > >> non-current > > > >> > > > >> > thread, plus the back-to-back no-allocation checks. The > > > >> former wasn?t > > > >> > > > >> > needed before because getThreadAllocatedBytes(long) was > > > >> just a wrapper > > > >> > > > >> > around getThreadAllocatedBytes(long []). This patch > > > >> changes that, so I > > > >> > > > >> > added a separate test. The latter is supposed to fail > > > >> if there?s object > > > >> > > > >> > allocation on calls to getCurrentThreadAllocatedBytes and > > > >> > > > >> > getThreadAllocatedBytes(long). I.e., a feature, not a > > > >> bug, because > > > >> > > > >> > accumulation of transient small objects can be a > > > >> performance problem. > > > >> > > > >> > Thanks to your review, I noticed that the back-to-back > > > >> check on the > > > >> > > > >> > current thread was using getThreadAllocatedBytes(long) > > > >> instead of > > > >> > > > >> > getCurrentThreadAllocatedBytes and fixed it. I also > > > >> removed all > > > >> > > > >> > instances of ?TEST FAILED: ?. > > > >> > > > >> > > > >> The back-to-back check is not valid in general. You don't > > > >> know if the > > > >> > > > >> first check might trigger some class loading on the > > > >> return path after it > > > >> > > > >> has obtained the first memory value. The check might also > > > >> fail if using > > > >> > > > >> JVMCI and some compilation related activity occurs in the > > > >> current thread > > > >> > > > >> on the second call. Also with the introduction of > > > >> handshakes its > > > >> > > > >> possible the current thread might hit a safepoint checks > > > >> that results in > > > >> > > > >> it executing a handshake operation that performs > > > >> allocation. Potentially > > > >> > > > >> there could be numerous non-deterministic actions that > > > >> might occur > > > >> > > > >> leading to unanticipated allocation. > > > >> > > > >> > > > >> I understand what you want to test here, I just don't > > > >> think it is > > > >> > > > >> reliably doable. > > > >> > > > >> > > > >> Thanks, > > > >> > > > >> David > > > >> > > > >> ----- > > > >> > > > >> > > > >> > > > > >> > > > >> > Paul > > > >> > > > >> > > > > >> > > > >> > *From: *Mandy Chung > > > >> > > > >> > > > >> > *Date: *Thursday, September 12, 2019 at 10:09 AM > > > >> > > > >> > *To: *"Hohensee, Paul" > > > >> > > > >> > > > >> > *Cc: *OpenJDK > > > >> Serviceability > > > >> , > > > >> > > > >> >"hotspot-gc-dev at openjdk.java.net" > > > >> > > > >> > > > >> > > > >> > > > >> > *Subject: *Re: RFR (M): 8207266: > > > >> ThreadMXBean::getThreadAllocatedBytes() > > > >> > > > >> > can be quicker for self thread > > > >> > > > >> > > > > >> > > > >> > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > > > >> > > > >> > > > > >> > > > >> > Minor update in new > > > >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > > > >> > > > >> > > > > >> > > > >> > > > > >> > > > >> > I only reviewed the library side implementation that > > > >> looks good. I > > > >> > > > >> > expect the serviceability team to review the test and > > > >> hotspot change. > > > >> > > > >> > > > > >> > > > >> > > > > >> > > > >> > Need a confirmatory review to push this. If I > > > >> understand the rules correctly, it doesn't need a Reviewer review > > > >> since Mandy's already reviewed it, it just needs a Committer review. > > > >> > > > >> > > > > >> > > > >> > > > > >> > > > >> > You need another reviewer to advice the following > > > >> because I was not > > > >> > > > >> > close to the ThreadsList work. > > > >> > > > >> > > > > >> > > > >> > 2087 ThreadsListHandle tlh; > > > >> > > > >> > > > > >> > > > >> > 2088 JavaThread* java_thread = > > > >> tlh.list()->find_JavaThread_from_java_tid(thread_id); > > > >> > > > >> > > > > >> > > > >> > 2089 > > > >> > > > >> > > > > >> > > > >> > 2090 if (java_thread != NULL) { > > > >> > > > >> > > > > >> > > > >> > 2091 return java_thread->cooked_allocated_bytes(); > > > >> > > > >> > > > > >> > > > >> > 2092 } > > > >> > > > >> > > > > >> > > > >> > This looks right to me. > > > >> > > > >> > > > > >> > > > >> > > > > >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > > > >> > > > >> > > > > >> > > > >> > - "ThreadAllocatedMemory is expected to > > > >> be disabled"); > > > >> > > > >> > > > > >> > > > >> > + "TEST FAILED: ThreadAllocatedMemory is > > > >> expected to be > > > >> > > > >> > disabled"); > > > >> > > > >> > > > > >> > > > >> > Prepending "TEST FAILED" in exception message (in > > > >> several places) > > > >> > > > >> > > > > >> > > > >> > seems redundant since such RuntimeException is thrown > > > >> and expected > > > >> > > > >> > > > > >> > > > >> > a test failure. > > > >> > > > >> > > > > >> > > > >> > + // back-to-back calls shouldn't allocate any > > > >> memory > > > >> > > > >> > > > > >> > > > >> > + size = mbean.getThreadAllocatedBytes(id); > > > >> > > > >> > > > > >> > > > >> > + size1 = mbean.getThreadAllocatedBytes(id); > > > >> > > > >> > > > > >> > > > >> > + if (size1 != size) { > > > >> > > > >> > > > > >> > > > >> > Is there anything in the test can do to help guarantee > > > >> this? I didn't > > > >> > > > >> > > > > >> > > > >> > closely review this test. The main thing I advice is > > > >> to improve > > > >> > > > >> > > > > >> > > > >> > the reliability of this test. Put it in another way, > > > >> we want to > > > >> > > > >> > > > > >> > > > >> > ensure that this test change will pass all the time in > > > >> various > > > >> > > > >> > > > > >> > > > >> > test configuration. > > > >> > > > >> > > > > >> > > > >> > Mandy > > > >> > > > >> > > > > >> > > > >> > > > >> > > > >> > > > >> > > > > > > > > > > > > > > > > > > > > > From daniel.daugherty at oracle.com Thu Sep 19 00:34:41 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 18 Sep 2019 20:34:41 -0400 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <6139732f-a914-c59c-2119-517e480a279c@oracle.com> References: <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> <9A151A35-DA66-4B0F-B67F-E7F5BECF205A@amazon.com> <586EB17C-1974-4FEF-A751-34B5B359BA61@amazon.com> <6139732f-a914-c59c-2119-517e480a279c@oracle.com> Message-ID: I have the backout ready to go. About to send out a webrev... Dan On 9/18/19 8:25 PM, Daniel D. Daugherty wrote: > I created this sub-task for you: > > JDK-8231210 [BACKOUT] JDK-8207266 > ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread > https://bugs.openjdk.java.net/browse/JDK-8231210 > > If you would prefer, I can handle this backout for you. > > Dan > > > On 9/18/19 8:21 PM, Hohensee, Paul wrote: >> Never having done this before, is it >> >> hg backout -r >> >> ? Do I file a JBS issue for the reversion? Seems necessary. >> >> ?On 9/18/19, 5:18 PM, "Daniel D. Daugherty" >> wrote: >> >> ???? % hg backout >> ???? ???? is the usual way to do this... >> ???? ???? Dan >> ???? ???? ???? On 9/18/19 8:17 PM, Hohensee, Paul wrote: >> ???? > Is there a tool that will generate a reversal patch? >> ???? > >> ???? > On 9/18/19, 5:14 PM, "Daniel D. Daugherty" >> wrote: >> ???? > >> ???? >?????? > Shall I go with that, or reverse the original patch? >> ???? > >> ???? >????? I'm a bit worried about what else might show up since the >> ???? >????? NSK monitoring tests were not run prior to this push. >> ???? > >> ???? >????? I vote for backing out the fix until proper testing has >> ???? >????? been done (and at least the one problem fixed...) >> ???? > >> ???? >????? Dan >> ???? > >> ???? > >> ???? >????? On 9/18/19 8:00 PM, Hohensee, Paul wrote: >> ???? >????? > They all implement com.sun.management.ThreadMXBean, so >> adding a getCurrentThreadAllocatedBytes broke them. Potential fix is >> to give it a default implementation, vis >> ???? >????? > >> ???? >????? >????? public default long >> getCurrentThreadAllocatedBytes() { >> ???? >????? >????????? return -1; >> ???? >????? >????? } >> ???? >????? > >> ???? >????? > Shall I go with that, or reverse the original patch? >> ???? >????? > >> ???? >????? > On 9/18/19, 4:48 PM, "serviceability-dev on behalf of >> Hohensee, Paul" > behalf of hohensee at amazon.com> wrote: >> ???? >????? > >> ???? >????? >????? I'll take a look. >> ???? >????? > >> ???? >????? >????? On 9/18/19, 4:40 PM, "David Holmes" >> wrote: >> ???? >????? > >> ???? >????? >????????? Paul, >> ???? >????? > >> ???? >????? >????????? Unfortunately this patch has broken the >> vmTestbase/nsk/monitoring tests: >> ???? >????? > >> ???? >????? >????????? [2019-09-18T22:59:32,349Z] >> ???? >????? > >> /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: >> ???? >????? >????????? error: ServerThreadMXBeanNew is not abstract >> and does not override >> ???? >????? >????????? abstract method >> getCurrentThreadAllocatedBytes() in ThreadMXBean >> ???? >????? > >> ???? >????? >????????? and possibly other issues as we are seeing >> hundreds of failures. >> ???? >????? > >> ???? >????? >????????? David >> ???? >????? > >> ???? >????? >????????? On 18/09/2019 8:50 am, David Holmes wrote: >> ???? >????? >????????? > On 18/09/2019 12:10 am, Hohensee, Paul wrote: >> ???? >????? >????????? >> Thanks, Serguei. :) >> ???? >????? >????????? >> >> ???? >????? >????????? >> David, are you ok with the patch? >> ???? >????? >????????? > >> ???? >????? >????????? > Yep, nothing further from me. >> ???? >????? >????????? > >> ???? >????? >????????? > David >> ???? >????? >????????? > >> ???? >????? >????????? >> Paul >> ???? >????? >????????? >> >> ???? >????? >????????? >> *From: *"serguei.spitsyn at oracle.com" >> >> ???? >????? >????????? >> *Date: *Tuesday, September 17, 2019 at 2:26 AM >> ???? >????? >????????? >> *To: *"Hohensee, Paul" >> , David Holmes >> ???? >????? >????????? >> , Mandy Chung >> >> ???? >????? >????????? >> *Cc: *OpenJDK Serviceability >> , >> ???? >????? >????????? >> "hotspot-gc-dev at openjdk.java.net" >> >> ???? >????? >????????? >> *Subject: *Re: RFR (M): 8207266: >> ???? >????? >????????? >> ThreadMXBean::getThreadAllocatedBytes() can >> be quicker for self thread >> ???? >????? >????????? >> >> ???? >????? >????????? >> Hi Paul, >> ???? >????? >????????? >> >> ???? >????? >????????? >> Thank you for refactoring and fixing the test. >> ???? >????? >????????? >> It looks great now! >> ???? >????? >????????? >> >> ???? >????? >????????? >> Thanks, >> ???? >????? >????????? >> Serguei >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >> On 9/15/19 02:52, Hohensee, Paul wrote: >> ???? >????? >????????? >> >> ???? >????? >????????? >>???? Hi, Serguei, thanks for the review. New >> webrev at >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ >> ???? >????? >????????? >> >> ???? >????? >????????? >>???? I refactored the test?s main() method, >> and you?re correct, >> ???? >????? >????????? >> getThreadAllocatedBytes should be >> getCurrentThreadAllocatedBytes in >> ???? >????? >????????? >>???? that context: fixed. >> ???? >????? >????????? >> >> ???? >????? >????????? >>???? Paul >> ???? >????? >????????? >> >> ???? >????? >????????? >>???? *From: *"serguei.spitsyn at oracle.com" >> ???? >????? >????????? >> >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>???? *Organization: *Oracle Corporation >> ???? >????? >????????? >>???? *Date: *Friday, September 13, 2019 at >> 5:50 PM >> ???? >????? >????????? >>???? *To: *"Hohensee, Paul" >> >> ???? >????? >????????? >> , David Holmes >> >> ???? >????? >????????? >> , Mandy Chung >> ???? >????? >????????? >> >> >> ???? >????? >????????? >>???? *Cc: *OpenJDK Serviceability >> >> ???? >????? >????????? >> , >> ???? >????? >????????? >> "hotspot-gc-dev at openjdk.java.net" >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>???? *Subject: *Re: RFR (M): 8207266: >> ???? >????? >????????? >> ThreadMXBean::getThreadAllocatedBytes() can >> be quicker for self >> ???? >????? >????????? >> thread >> ???? >????? >????????? >> >> ???? >????? >????????? >>???? Hi Paul, >> ???? >????? >????????? >> >> ???? >????? >????????? >>???? It looks pretty good in general. >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>???? It would be nice to refactor the java >> main() method as it becomes >> ???? >????? >????????? >>???? too big. >> ???? >????? >????????? >>???? Two ways >> ofgetCurrentThreadAllocatedBytes() testing are good >> ???? >????? >????????? >> candidates >> ???? >????? >????????? >>???? to become separate methods. >> ???? >????? >????????? >> >> ???? >????? >????????? >>??????? 98???????? long size1 = >> mbean.getThreadAllocatedBytes(id); >> ???? >????? >????????? >> >> ???? >????? >????????? >>???? Just wanted to double check if you >> wanted to invoke >> ???? >????? >????????? >>???? the getCurrentThreadAllocatedBytes() >> instead as it is >> ???? >????? >????????? >>???? a part of: >> ???? >????? >????????? >> >> ???? >????? >????????? >>??????? 85???????? // First way, >> getCurrentThreadAllocatedBytes >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>???? Thanks, >> ???? >????? >????????? >>???? Serguei >> ???? >????? >????????? >> >> ???? >????? >????????? >>???? On 9/13/19 12:11 PM, Hohensee, Paul wrote: >> ???? >????? >????????? >> >> ???? >????? >????????? >>???????? Hi David, thanks for your comments. >> New webrev in >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>???????? Both the old and new versions of >> the code check that thread >> ???? >????? >????????? >> allocated memory is both supported and >> enabled. The existing version >> ???? >????? >????????? >> of getThreadAllocatedBytes(long []) calls >> ???? >????? >????????? >> verifyThreadAllocatedMemory(long []), which >> checks inline to make sure >> ???? >????? >????????? >> thread allocated memory is supported, then >> calls >> ???? >????? >????????? >> isThreadAllocatedMemoryEnabled() to verify >> that it's enabled. >> ???? >????? >????????? >> isThreadAllocatedMemoryEnabled() duplicates >> (!) the support check and >> ???? >????? >????????? >> returns the enabled flag. I removed the >> redundant check in the new >> ???? >????? >????????? >> version. >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>???????? You're of course correct about the >> back-to-back check. >> ???? >????? >????????? >> Application code can't know when the >> runtime will hijack a thread for >> ???? >????? >????????? >> its own purposes. I've removed the check. >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>???????? Paul >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>???????? On 9/13/19, 12:50 AM, "David >> Holmes" >> ???? >????? >????????? >> ? wrote: >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? Hi Paul, >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? On 13/09/2019 10:29 am, >> Hohensee, Paul wrote: >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > Thanks for clarifying the >> review rules. Would someone >> ???? >????? >????????? >> from the >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > serviceability team please >> review? New webrev at >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? One aspect of the functional >> change needs clarification >> ???? >????? >????????? >> for me - and >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? apologies if this has been >> covered in the past. It seems >> ???? >????? >????????? >> to me that >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? currently we only check >> isThreadAllocatedMemorySupported >> ???? >????? >????????? >> for these >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? operations, but if I read >> things correctly the updated >> ???? >????? >????????? >> code additionally >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? checks >> isThreadAllocatedMemoryEnabled, which is a >> ???? >????? >????????? >> behaviour change not >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? mentioned in the CSR. >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > I didn?t disturb the >> existing checks in the test, just >> ???? >????? >????????? >> added code to >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > check the result of >> getThreadAllocatedBytes(long) on a >> ???? >????? >????????? >> non-current >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > thread, plus the >> back-to-back no-allocation checks. The >> ???? >????? >????????? >> former wasn?t >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > needed before because >> getThreadAllocatedBytes(long) was >> ???? >????? >????????? >> just a wrapper >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > around >> getThreadAllocatedBytes(long []). This patch >> ???? >????? >????????? >> changes that, so I >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > added a separate test. The >> latter is supposed to fail >> ???? >????? >????????? >> if there?s object >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > allocation on calls to >> getCurrentThreadAllocatedBytes and >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> getThreadAllocatedBytes(long). I.e., a feature, not a >> ???? >????? >????????? >> bug, because >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > accumulation of transient >> small objects can be a >> ???? >????? >????????? >> performance problem. >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > Thanks to your review, I >> noticed that the back-to-back >> ???? >????? >????????? >> check on the >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > current thread was using >> getThreadAllocatedBytes(long) >> ???? >????? >????????? >> instead of >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> getCurrentThreadAllocatedBytes and fixed it. I also >> ???? >????? >????????? >> removed all >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > instances of ?TEST FAILED: ?. >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? The back-to-back check is not >> valid in general. You don't >> ???? >????? >????????? >> know if the >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? first check might trigger some >> class loading on the >> ???? >????? >????????? >> return path after it >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? has obtained the first memory >> value. The check might also >> ???? >????? >????????? >> fail if using >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? JVMCI and some compilation >> related activity occurs in the >> ???? >????? >????????? >> current thread >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? on the second call. Also with >> the introduction of >> ???? >????? >????????? >> handshakes its >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? possible the current thread >> might hit a safepoint checks >> ???? >????? >????????? >> that results in >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? it executing a handshake >> operation that performs >> ???? >????? >????????? >> allocation. Potentially >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? there could be numerous >> non-deterministic actions that >> ???? >????? >????????? >> might occur >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? leading to unanticipated >> allocation. >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? I understand what you want to >> test here, I just don't >> ???? >????? >????????? >> think it is >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? reliably doable. >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? Thanks, >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? David >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? ----- >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > Paul >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > *From: *Mandy >> Chung >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > *Date: *Thursday, September >> 12, 2019 at 10:09 AM >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > *To: *"Hohensee, >> Paul" >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > *Cc: *OpenJDK >> ???? >????? >????????? >> >> Serviceability >> ???? >????? >????????? >> , >> ???? >????? >????????? >> >> ???? >????? >????????? >> >"hotspot-gc-dev at openjdk.java.net" >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > *Subject: *Re: RFR (M): >> 8207266: >> ???? >????? >????????? >> ThreadMXBean::getThreadAllocatedBytes() >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > can be quicker for self thread >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > On 9/3/19 12:38 PM, >> Hohensee, Paul wrote: >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > Minor update in new >> ???? >????? >????????? >> >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > I only reviewed the library >> side implementation that >> ???? >????? >????????? >> looks good.? I >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > expect the serviceability >> team to review the test and >> ???? >????? >????????? >> hotspot change. >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? >???? Need a confirmatory >> review to push this. If I >> ???? >????? >????????? >> understand the rules correctly, it doesn't >> need a Reviewer review >> ???? >????? >????????? >> since Mandy's already reviewed it, it just >> needs a Committer review. >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > You need another reviewer to >> advice the following >> ???? >????? >????????? >> because I was not >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > close to the ThreadsList work. >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > 2087 ThreadsListHandle tlh; >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > 2088 JavaThread* java_thread = >> ???? >????? >????????? >> >> tlh.list()->find_JavaThread_from_java_tid(thread_id); >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > 2089 >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > 2090 if (java_thread != NULL) { >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > 2091 return >> java_thread->cooked_allocated_bytes(); >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > 2092?? } >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > This looks right to me. >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > -??????????????? >> "ThreadAllocatedMemory is expected to >> ???? >????? >????????? >> be disabled"); >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > +??????????????? "TEST >> FAILED: ThreadAllocatedMemory is >> ???? >????? >????????? >> expected to be >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > disabled"); >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > Prepending "TEST FAILED" in >> exception message (in >> ???? >????? >????????? >> several places) >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > seems redundant since such >> RuntimeException is thrown >> ???? >????? >????????? >> and expected >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > a test failure. >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > + // back-to-back calls >> shouldn't allocate any >> ???? >????? >????????? >> memory >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > + size = >> mbean.getThreadAllocatedBytes(id); >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > + size1 = >> mbean.getThreadAllocatedBytes(id); >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > + if (size1 != size) { >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > Is there anything in the >> test can do to help guarantee >> ???? >????? >????????? >> this? I didn't >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > closely review this test.? >> The main thing I advice is >> ???? >????? >????????? >> to improve >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > the reliability of this >> test.? Put it in another way, >> ???? >????? >????????? >> we want to >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > ensure that this test change >> will pass all the time in >> ???? >????? >????????? >> various >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > test configuration. >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > Mandy >> ???? >????? >????????? >> >> ???? >????? >????????? >>????????????? > >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? >????????? >> >> ???? >????? > >> ???? >????? > >> ???? >????? > >> ???? >????? > >> ???? > >> ???? > >> ???? > >> > From hohensee at amazon.com Thu Sep 19 00:36:26 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 19 Sep 2019 00:36:26 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <6139732f-a914-c59c-2119-517e480a279c@oracle.com> References: <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> <9A151A35-DA66-4B0F-B67F-E7F5BECF205A@amazon.com> <586EB17C-1974-4FEF-A751-34B5B359BA61@amazon.com> <6139732f-a914-c59c-2119-517e480a279c@oracle.com> Message-ID: <8E8AE26F-5A7B-499C-BB6A-5EC954E42BD7@amazon.com> And I filed 8231211 for the same thing. :) Yes, please handle it, because it will go faster since I don't have access to a fast machine (just my laptop). Webrev here: http://cr.openjdk.java.net/~phh/8231211/webrev.00/ Thanks, ?On 9/18/19, 5:25 PM, "Daniel D. Daugherty" wrote: I created this sub-task for you: JDK-8231210 [BACKOUT] JDK-8207266 ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread https://bugs.openjdk.java.net/browse/JDK-8231210 If you would prefer, I can handle this backout for you. Dan On 9/18/19 8:21 PM, Hohensee, Paul wrote: > Never having done this before, is it > > hg backout -r > > ? Do I file a JBS issue for the reversion? Seems necessary. > > On 9/18/19, 5:18 PM, "Daniel D. Daugherty" wrote: > > % hg backout > > is the usual way to do this... > > Dan > > > On 9/18/19 8:17 PM, Hohensee, Paul wrote: > > Is there a tool that will generate a reversal patch? > > > > On 9/18/19, 5:14 PM, "Daniel D. Daugherty" wrote: > > > > > Shall I go with that, or reverse the original patch? > > > > I'm a bit worried about what else might show up since the > > NSK monitoring tests were not run prior to this push. > > > > I vote for backing out the fix until proper testing has > > been done (and at least the one problem fixed...) > > > > Dan > > > > > > On 9/18/19 8:00 PM, Hohensee, Paul wrote: > > > They all implement com.sun.management.ThreadMXBean, so adding a getCurrentThreadAllocatedBytes broke them. Potential fix is to give it a default implementation, vis > > > > > > public default long getCurrentThreadAllocatedBytes() { > > > return -1; > > > } > > > > > > Shall I go with that, or reverse the original patch? > > > > > > On 9/18/19, 4:48 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > > > I'll take a look. > > > > > > On 9/18/19, 4:40 PM, "David Holmes" wrote: > > > > > > Paul, > > > > > > Unfortunately this patch has broken the vmTestbase/nsk/monitoring tests: > > > > > > [2019-09-18T22:59:32,349Z] > > > /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: > > > error: ServerThreadMXBeanNew is not abstract and does not override > > > abstract method getCurrentThreadAllocatedBytes() in ThreadMXBean > > > > > > and possibly other issues as we are seeing hundreds of failures. > > > > > > David > > > > > > On 18/09/2019 8:50 am, David Holmes wrote: > > > > On 18/09/2019 12:10 am, Hohensee, Paul wrote: > > > >> Thanks, Serguei. :) > > > >> > > > >> David, are you ok with the patch? > > > > > > > > Yep, nothing further from me. > > > > > > > > David > > > > > > > >> Paul > > > >> > > > >> *From: *"serguei.spitsyn at oracle.com" > > > >> *Date: *Tuesday, September 17, 2019 at 2:26 AM > > > >> *To: *"Hohensee, Paul" , David Holmes > > > >> , Mandy Chung > > > >> *Cc: *OpenJDK Serviceability , > > > >> "hotspot-gc-dev at openjdk.java.net" > > > >> *Subject: *Re: RFR (M): 8207266: > > > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread > > > >> > > > >> Hi Paul, > > > >> > > > >> Thank you for refactoring and fixing the test. > > > >> It looks great now! > > > >> > > > >> Thanks, > > > >> Serguei > > > >> > > > >> > > > >> On 9/15/19 02:52, Hohensee, Paul wrote: > > > >> > > > >> Hi, Serguei, thanks for the review. New webrev at > > > >> > > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ > > > >> > > > >> I refactored the test?s main() method, and you?re correct, > > > >> getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in > > > >> that context: fixed. > > > >> > > > >> Paul > > > >> > > > >> *From: *"serguei.spitsyn at oracle.com" > > > >> > > > >> > > > >> *Organization: *Oracle Corporation > > > >> *Date: *Friday, September 13, 2019 at 5:50 PM > > > >> *To: *"Hohensee, Paul" > > > >> , David Holmes > > > >> , Mandy Chung > > > >> > > > >> *Cc: *OpenJDK Serviceability > > > >> , > > > >> "hotspot-gc-dev at openjdk.java.net" > > > >> > > > >> > > > >> > > > >> *Subject: *Re: RFR (M): 8207266: > > > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self > > > >> thread > > > >> > > > >> Hi Paul, > > > >> > > > >> It looks pretty good in general. > > > >> > > > >> > > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html > > > >> > > > >> > > > >> It would be nice to refactor the java main() method as it becomes > > > >> too big. > > > >> Two ways ofgetCurrentThreadAllocatedBytes() testing are good > > > >> candidates > > > >> to become separate methods. > > > >> > > > >> 98 long size1 = mbean.getThreadAllocatedBytes(id); > > > >> > > > >> Just wanted to double check if you wanted to invoke > > > >> the getCurrentThreadAllocatedBytes() instead as it is > > > >> a part of: > > > >> > > > >> 85 // First way, getCurrentThreadAllocatedBytes > > > >> > > > >> > > > >> Thanks, > > > >> Serguei > > > >> > > > >> On 9/13/19 12:11 PM, Hohensee, Paul wrote: > > > >> > > > >> Hi David, thanks for your comments. New webrev in > > > >> > > > >> > > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ > > > >> > > > >> > > > >> Both the old and new versions of the code check that thread > > > >> allocated memory is both supported and enabled. The existing version > > > >> of getThreadAllocatedBytes(long []) calls > > > >> verifyThreadAllocatedMemory(long []), which checks inline to make sure > > > >> thread allocated memory is supported, then calls > > > >> isThreadAllocatedMemoryEnabled() to verify that it's enabled. > > > >> isThreadAllocatedMemoryEnabled() duplicates (!) the support check and > > > >> returns the enabled flag. I removed the redundant check in the new > > > >> version. > > > >> > > > >> > > > >> You're of course correct about the back-to-back check. > > > >> Application code can't know when the runtime will hijack a thread for > > > >> its own purposes. I've removed the check. > > > >> > > > >> > > > >> Paul > > > >> > > > >> > > > >> On 9/13/19, 12:50 AM, "David Holmes" > > > >> wrote: > > > >> > > > >> > > > >> Hi Paul, > > > >> > > > >> > > > >> On 13/09/2019 10:29 am, Hohensee, Paul wrote: > > > >> > > > >> > Thanks for clarifying the review rules. Would someone > > > >> from the > > > >> > > > >> > serviceability team please review? New webrev at > > > >> > > > >> > > > > >> > > > >> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ > > > >> > > > >> > > > >> One aspect of the functional change needs clarification > > > >> for me - and > > > >> > > > >> apologies if this has been covered in the past. It seems > > > >> to me that > > > >> > > > >> currently we only check isThreadAllocatedMemorySupported > > > >> for these > > > >> > > > >> operations, but if I read things correctly the updated > > > >> code additionally > > > >> > > > >> checks isThreadAllocatedMemoryEnabled, which is a > > > >> behaviour change not > > > >> > > > >> mentioned in the CSR. > > > >> > > > >> > > > >> > I didn?t disturb the existing checks in the test, just > > > >> added code to > > > >> > > > >> > check the result of getThreadAllocatedBytes(long) on a > > > >> non-current > > > >> > > > >> > thread, plus the back-to-back no-allocation checks. The > > > >> former wasn?t > > > >> > > > >> > needed before because getThreadAllocatedBytes(long) was > > > >> just a wrapper > > > >> > > > >> > around getThreadAllocatedBytes(long []). This patch > > > >> changes that, so I > > > >> > > > >> > added a separate test. The latter is supposed to fail > > > >> if there?s object > > > >> > > > >> > allocation on calls to getCurrentThreadAllocatedBytes and > > > >> > > > >> > getThreadAllocatedBytes(long). I.e., a feature, not a > > > >> bug, because > > > >> > > > >> > accumulation of transient small objects can be a > > > >> performance problem. > > > >> > > > >> > Thanks to your review, I noticed that the back-to-back > > > >> check on the > > > >> > > > >> > current thread was using getThreadAllocatedBytes(long) > > > >> instead of > > > >> > > > >> > getCurrentThreadAllocatedBytes and fixed it. I also > > > >> removed all > > > >> > > > >> > instances of ?TEST FAILED: ?. > > > >> > > > >> > > > >> The back-to-back check is not valid in general. You don't > > > >> know if the > > > >> > > > >> first check might trigger some class loading on the > > > >> return path after it > > > >> > > > >> has obtained the first memory value. The check might also > > > >> fail if using > > > >> > > > >> JVMCI and some compilation related activity occurs in the > > > >> current thread > > > >> > > > >> on the second call. Also with the introduction of > > > >> handshakes its > > > >> > > > >> possible the current thread might hit a safepoint checks > > > >> that results in > > > >> > > > >> it executing a handshake operation that performs > > > >> allocation. Potentially > > > >> > > > >> there could be numerous non-deterministic actions that > > > >> might occur > > > >> > > > >> leading to unanticipated allocation. > > > >> > > > >> > > > >> I understand what you want to test here, I just don't > > > >> think it is > > > >> > > > >> reliably doable. > > > >> > > > >> > > > >> Thanks, > > > >> > > > >> David > > > >> > > > >> ----- > > > >> > > > >> > > > >> > > > > >> > > > >> > Paul > > > >> > > > >> > > > > >> > > > >> > *From: *Mandy Chung > > > >> > > > >> > > > >> > *Date: *Thursday, September 12, 2019 at 10:09 AM > > > >> > > > >> > *To: *"Hohensee, Paul" > > > >> > > > >> > > > >> > *Cc: *OpenJDK > > > >> Serviceability > > > >> , > > > >> > > > >> >"hotspot-gc-dev at openjdk.java.net" > > > >> > > > >> > > > >> > > > >> > > > >> > *Subject: *Re: RFR (M): 8207266: > > > >> ThreadMXBean::getThreadAllocatedBytes() > > > >> > > > >> > can be quicker for self thread > > > >> > > > >> > > > > >> > > > >> > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > > > >> > > > >> > > > > >> > > > >> > Minor update in new > > > >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > > > >> > > > >> > > > > >> > > > >> > > > > >> > > > >> > I only reviewed the library side implementation that > > > >> looks good. I > > > >> > > > >> > expect the serviceability team to review the test and > > > >> hotspot change. > > > >> > > > >> > > > > >> > > > >> > > > > >> > > > >> > Need a confirmatory review to push this. If I > > > >> understand the rules correctly, it doesn't need a Reviewer review > > > >> since Mandy's already reviewed it, it just needs a Committer review. > > > >> > > > >> > > > > >> > > > >> > > > > >> > > > >> > You need another reviewer to advice the following > > > >> because I was not > > > >> > > > >> > close to the ThreadsList work. > > > >> > > > >> > > > > >> > > > >> > 2087 ThreadsListHandle tlh; > > > >> > > > >> > > > > >> > > > >> > 2088 JavaThread* java_thread = > > > >> tlh.list()->find_JavaThread_from_java_tid(thread_id); > > > >> > > > >> > > > > >> > > > >> > 2089 > > > >> > > > >> > > > > >> > > > >> > 2090 if (java_thread != NULL) { > > > >> > > > >> > > > > >> > > > >> > 2091 return java_thread->cooked_allocated_bytes(); > > > >> > > > >> > > > > >> > > > >> > 2092 } > > > >> > > > >> > > > > >> > > > >> > This looks right to me. > > > >> > > > >> > > > > >> > > > >> > > > > >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > > > >> > > > >> > > > > >> > > > >> > - "ThreadAllocatedMemory is expected to > > > >> be disabled"); > > > >> > > > >> > > > > >> > > > >> > + "TEST FAILED: ThreadAllocatedMemory is > > > >> expected to be > > > >> > > > >> > disabled"); > > > >> > > > >> > > > > >> > > > >> > Prepending "TEST FAILED" in exception message (in > > > >> several places) > > > >> > > > >> > > > > >> > > > >> > seems redundant since such RuntimeException is thrown > > > >> and expected > > > >> > > > >> > > > > >> > > > >> > a test failure. > > > >> > > > >> > > > > >> > > > >> > + // back-to-back calls shouldn't allocate any > > > >> memory > > > >> > > > >> > > > > >> > > > >> > + size = mbean.getThreadAllocatedBytes(id); > > > >> > > > >> > > > > >> > > > >> > + size1 = mbean.getThreadAllocatedBytes(id); > > > >> > > > >> > > > > >> > > > >> > + if (size1 != size) { > > > >> > > > >> > > > > >> > > > >> > Is there anything in the test can do to help guarantee > > > >> this? I didn't > > > >> > > > >> > > > > >> > > > >> > closely review this test. The main thing I advice is > > > >> to improve > > > >> > > > >> > > > > >> > > > >> > the reliability of this test. Put it in another way, > > > >> we want to > > > >> > > > >> > > > > >> > > > >> > ensure that this test change will pass all the time in > > > >> various > > > >> > > > >> > > > > >> > > > >> > test configuration. > > > >> > > > >> > > > > >> > > > >> > Mandy > > > >> > > > >> > > > > >> > > > >> > > > >> > > > >> > > > >> > > > > > > > > > > > > > > > > > > > > > From daniel.daugherty at oracle.com Thu Sep 19 00:44:00 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 18 Sep 2019 20:44:00 -0400 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <8E8AE26F-5A7B-499C-BB6A-5EC954E42BD7@amazon.com> References: <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> <9A151A35-DA66-4B0F-B67F-E7F5BECF205A@amazon.com> <586EB17C-1974-4FEF-A751-34B5B359BA61@amazon.com> <6139732f-a914-c59c-2119-517e480a279c@oracle.com> <8E8AE26F-5A7B-499C-BB6A-5EC954E42BD7@amazon.com> Message-ID: <0d316e76-44cc-4f65-a96d-e1207b1c94e6@oracle.com> For some reason, the backout that I did does not match the backout that you did so I'm trying to figure that out. Dan On 9/18/19 8:36 PM, Hohensee, Paul wrote: > And I filed 8231211 for the same thing. :) > > Yes, please handle it, because it will go faster since I don't have access to a fast machine (just my laptop). > > Webrev here: > > http://cr.openjdk.java.net/~phh/8231211/webrev.00/ > > Thanks, > > ?On 9/18/19, 5:25 PM, "Daniel D. Daugherty" wrote: > > I created this sub-task for you: > > JDK-8231210 [BACKOUT] JDK-8207266 > ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread > https://bugs.openjdk.java.net/browse/JDK-8231210 > > If you would prefer, I can handle this backout for you. > > Dan > > > On 9/18/19 8:21 PM, Hohensee, Paul wrote: > > Never having done this before, is it > > > > hg backout -r > > > > ? Do I file a JBS issue for the reversion? Seems necessary. > > > > On 9/18/19, 5:18 PM, "Daniel D. Daugherty" wrote: > > > > % hg backout > > > > is the usual way to do this... > > > > Dan > > > > > > On 9/18/19 8:17 PM, Hohensee, Paul wrote: > > > Is there a tool that will generate a reversal patch? > > > > > > On 9/18/19, 5:14 PM, "Daniel D. Daugherty" wrote: > > > > > > > Shall I go with that, or reverse the original patch? > > > > > > I'm a bit worried about what else might show up since the > > > NSK monitoring tests were not run prior to this push. > > > > > > I vote for backing out the fix until proper testing has > > > been done (and at least the one problem fixed...) > > > > > > Dan > > > > > > > > > On 9/18/19 8:00 PM, Hohensee, Paul wrote: > > > > They all implement com.sun.management.ThreadMXBean, so adding a getCurrentThreadAllocatedBytes broke them. Potential fix is to give it a default implementation, vis > > > > > > > > public default long getCurrentThreadAllocatedBytes() { > > > > return -1; > > > > } > > > > > > > > Shall I go with that, or reverse the original patch? > > > > > > > > On 9/18/19, 4:48 PM, "serviceability-dev on behalf of Hohensee, Paul" wrote: > > > > > > > > I'll take a look. > > > > > > > > On 9/18/19, 4:40 PM, "David Holmes" wrote: > > > > > > > > Paul, > > > > > > > > Unfortunately this patch has broken the vmTestbase/nsk/monitoring tests: > > > > > > > > [2019-09-18T22:59:32,349Z] > > > > /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: > > > > error: ServerThreadMXBeanNew is not abstract and does not override > > > > abstract method getCurrentThreadAllocatedBytes() in ThreadMXBean > > > > > > > > and possibly other issues as we are seeing hundreds of failures. > > > > > > > > David > > > > > > > > On 18/09/2019 8:50 am, David Holmes wrote: > > > > > On 18/09/2019 12:10 am, Hohensee, Paul wrote: > > > > >> Thanks, Serguei. :) > > > > >> > > > > >> David, are you ok with the patch? > > > > > > > > > > Yep, nothing further from me. > > > > > > > > > > David > > > > > > > > > >> Paul > > > > >> > > > > >> *From: *"serguei.spitsyn at oracle.com" > > > > >> *Date: *Tuesday, September 17, 2019 at 2:26 AM > > > > >> *To: *"Hohensee, Paul" , David Holmes > > > > >> , Mandy Chung > > > > >> *Cc: *OpenJDK Serviceability , > > > > >> "hotspot-gc-dev at openjdk.java.net" > > > > >> *Subject: *Re: RFR (M): 8207266: > > > > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread > > > > >> > > > > >> Hi Paul, > > > > >> > > > > >> Thank you for refactoring and fixing the test. > > > > >> It looks great now! > > > > >> > > > > >> Thanks, > > > > >> Serguei > > > > >> > > > > >> > > > > >> On 9/15/19 02:52, Hohensee, Paul wrote: > > > > >> > > > > >> Hi, Serguei, thanks for the review. New webrev at > > > > >> > > > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ > > > > >> > > > > >> I refactored the test?s main() method, and you?re correct, > > > > >> getThreadAllocatedBytes should be getCurrentThreadAllocatedBytes in > > > > >> that context: fixed. > > > > >> > > > > >> Paul > > > > >> > > > > >> *From: *"serguei.spitsyn at oracle.com" > > > > >> > > > > >> > > > > >> *Organization: *Oracle Corporation > > > > >> *Date: *Friday, September 13, 2019 at 5:50 PM > > > > >> *To: *"Hohensee, Paul" > > > > >> , David Holmes > > > > >> , Mandy Chung > > > > >> > > > > >> *Cc: *OpenJDK Serviceability > > > > >> , > > > > >> "hotspot-gc-dev at openjdk.java.net" > > > > >> > > > > >> > > > > >> > > > > >> *Subject: *Re: RFR (M): 8207266: > > > > >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self > > > > >> thread > > > > >> > > > > >> Hi Paul, > > > > >> > > > > >> It looks pretty good in general. > > > > >> > > > > >> > > > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html > > > > >> > > > > >> > > > > >> It would be nice to refactor the java main() method as it becomes > > > > >> too big. > > > > >> Two ways ofgetCurrentThreadAllocatedBytes() testing are good > > > > >> candidates > > > > >> to become separate methods. > > > > >> > > > > >> 98 long size1 = mbean.getThreadAllocatedBytes(id); > > > > >> > > > > >> Just wanted to double check if you wanted to invoke > > > > >> the getCurrentThreadAllocatedBytes() instead as it is > > > > >> a part of: > > > > >> > > > > >> 85 // First way, getCurrentThreadAllocatedBytes > > > > >> > > > > >> > > > > >> Thanks, > > > > >> Serguei > > > > >> > > > > >> On 9/13/19 12:11 PM, Hohensee, Paul wrote: > > > > >> > > > > >> Hi David, thanks for your comments. New webrev in > > > > >> > > > > >> > > > > >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ > > > > >> > > > > >> > > > > >> Both the old and new versions of the code check that thread > > > > >> allocated memory is both supported and enabled. The existing version > > > > >> of getThreadAllocatedBytes(long []) calls > > > > >> verifyThreadAllocatedMemory(long []), which checks inline to make sure > > > > >> thread allocated memory is supported, then calls > > > > >> isThreadAllocatedMemoryEnabled() to verify that it's enabled. > > > > >> isThreadAllocatedMemoryEnabled() duplicates (!) the support check and > > > > >> returns the enabled flag. I removed the redundant check in the new > > > > >> version. > > > > >> > > > > >> > > > > >> You're of course correct about the back-to-back check. > > > > >> Application code can't know when the runtime will hijack a thread for > > > > >> its own purposes. I've removed the check. > > > > >> > > > > >> > > > > >> Paul > > > > >> > > > > >> > > > > >> On 9/13/19, 12:50 AM, "David Holmes" > > > > >> wrote: > > > > >> > > > > >> > > > > >> Hi Paul, > > > > >> > > > > >> > > > > >> On 13/09/2019 10:29 am, Hohensee, Paul wrote: > > > > >> > > > > >> > Thanks for clarifying the review rules. Would someone > > > > >> from the > > > > >> > > > > >> > serviceability team please review? New webrev at > > > > >> > > > > >> > > > > > >> > > > > >> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ > > > > >> > > > > >> > > > > >> One aspect of the functional change needs clarification > > > > >> for me - and > > > > >> > > > > >> apologies if this has been covered in the past. It seems > > > > >> to me that > > > > >> > > > > >> currently we only check isThreadAllocatedMemorySupported > > > > >> for these > > > > >> > > > > >> operations, but if I read things correctly the updated > > > > >> code additionally > > > > >> > > > > >> checks isThreadAllocatedMemoryEnabled, which is a > > > > >> behaviour change not > > > > >> > > > > >> mentioned in the CSR. > > > > >> > > > > >> > > > > >> > I didn?t disturb the existing checks in the test, just > > > > >> added code to > > > > >> > > > > >> > check the result of getThreadAllocatedBytes(long) on a > > > > >> non-current > > > > >> > > > > >> > thread, plus the back-to-back no-allocation checks. The > > > > >> former wasn?t > > > > >> > > > > >> > needed before because getThreadAllocatedBytes(long) was > > > > >> just a wrapper > > > > >> > > > > >> > around getThreadAllocatedBytes(long []). This patch > > > > >> changes that, so I > > > > >> > > > > >> > added a separate test. The latter is supposed to fail > > > > >> if there?s object > > > > >> > > > > >> > allocation on calls to getCurrentThreadAllocatedBytes and > > > > >> > > > > >> > getThreadAllocatedBytes(long). I.e., a feature, not a > > > > >> bug, because > > > > >> > > > > >> > accumulation of transient small objects can be a > > > > >> performance problem. > > > > >> > > > > >> > Thanks to your review, I noticed that the back-to-back > > > > >> check on the > > > > >> > > > > >> > current thread was using getThreadAllocatedBytes(long) > > > > >> instead of > > > > >> > > > > >> > getCurrentThreadAllocatedBytes and fixed it. I also > > > > >> removed all > > > > >> > > > > >> > instances of ?TEST FAILED: ?. > > > > >> > > > > >> > > > > >> The back-to-back check is not valid in general. You don't > > > > >> know if the > > > > >> > > > > >> first check might trigger some class loading on the > > > > >> return path after it > > > > >> > > > > >> has obtained the first memory value. The check might also > > > > >> fail if using > > > > >> > > > > >> JVMCI and some compilation related activity occurs in the > > > > >> current thread > > > > >> > > > > >> on the second call. Also with the introduction of > > > > >> handshakes its > > > > >> > > > > >> possible the current thread might hit a safepoint checks > > > > >> that results in > > > > >> > > > > >> it executing a handshake operation that performs > > > > >> allocation. Potentially > > > > >> > > > > >> there could be numerous non-deterministic actions that > > > > >> might occur > > > > >> > > > > >> leading to unanticipated allocation. > > > > >> > > > > >> > > > > >> I understand what you want to test here, I just don't > > > > >> think it is > > > > >> > > > > >> reliably doable. > > > > >> > > > > >> > > > > >> Thanks, > > > > >> > > > > >> David > > > > >> > > > > >> ----- > > > > >> > > > > >> > > > > >> > > > > > >> > > > > >> > Paul > > > > >> > > > > >> > > > > > >> > > > > >> > *From: *Mandy Chung > > > > >> > > > > >> > > > > >> > *Date: *Thursday, September 12, 2019 at 10:09 AM > > > > >> > > > > >> > *To: *"Hohensee, Paul" > > > > >> > > > > >> > > > > >> > *Cc: *OpenJDK > > > > >> Serviceability > > > > >> , > > > > >> > > > > >> >"hotspot-gc-dev at openjdk.java.net" > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > *Subject: *Re: RFR (M): 8207266: > > > > >> ThreadMXBean::getThreadAllocatedBytes() > > > > >> > > > > >> > can be quicker for self thread > > > > >> > > > > >> > > > > > >> > > > > >> > On 9/3/19 12:38 PM, Hohensee, Paul wrote: > > > > >> > > > > >> > > > > > >> > > > > >> > Minor update in new > > > > >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > > > > >> > > > > >> > > > > > >> > > > > >> > > > > > >> > > > > >> > I only reviewed the library side implementation that > > > > >> looks good. I > > > > >> > > > > >> > expect the serviceability team to review the test and > > > > >> hotspot change. > > > > >> > > > > >> > > > > > >> > > > > >> > > > > > >> > > > > >> > Need a confirmatory review to push this. If I > > > > >> understand the rules correctly, it doesn't need a Reviewer review > > > > >> since Mandy's already reviewed it, it just needs a Committer review. > > > > >> > > > > >> > > > > > >> > > > > >> > > > > > >> > > > > >> > You need another reviewer to advice the following > > > > >> because I was not > > > > >> > > > > >> > close to the ThreadsList work. > > > > >> > > > > >> > > > > > >> > > > > >> > 2087 ThreadsListHandle tlh; > > > > >> > > > > >> > > > > > >> > > > > >> > 2088 JavaThread* java_thread = > > > > >> tlh.list()->find_JavaThread_from_java_tid(thread_id); > > > > >> > > > > >> > > > > > >> > > > > >> > 2089 > > > > >> > > > > >> > > > > > >> > > > > >> > 2090 if (java_thread != NULL) { > > > > >> > > > > >> > > > > > >> > > > > >> > 2091 return java_thread->cooked_allocated_bytes(); > > > > >> > > > > >> > > > > > >> > > > > >> > 2092 } > > > > >> > > > > >> > > > > > >> > > > > >> > This looks right to me. > > > > >> > > > > >> > > > > > >> > > > > >> > > > > > >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > > > > >> > > > > >> > > > > > >> > > > > >> > - "ThreadAllocatedMemory is expected to > > > > >> be disabled"); > > > > >> > > > > >> > > > > > >> > > > > >> > + "TEST FAILED: ThreadAllocatedMemory is > > > > >> expected to be > > > > >> > > > > >> > disabled"); > > > > >> > > > > >> > > > > > >> > > > > >> > Prepending "TEST FAILED" in exception message (in > > > > >> several places) > > > > >> > > > > >> > > > > > >> > > > > >> > seems redundant since such RuntimeException is thrown > > > > >> and expected > > > > >> > > > > >> > > > > > >> > > > > >> > a test failure. > > > > >> > > > > >> > > > > > >> > > > > >> > + // back-to-back calls shouldn't allocate any > > > > >> memory > > > > >> > > > > >> > > > > > >> > > > > >> > + size = mbean.getThreadAllocatedBytes(id); > > > > >> > > > > >> > > > > > >> > > > > >> > + size1 = mbean.getThreadAllocatedBytes(id); > > > > >> > > > > >> > > > > > >> > > > > >> > + if (size1 != size) { > > > > >> > > > > >> > > > > > >> > > > > >> > Is there anything in the test can do to help guarantee > > > > >> this? I didn't > > > > >> > > > > >> > > > > > >> > > > > >> > closely review this test. The main thing I advice is > > > > >> to improve > > > > >> > > > > >> > > > > > >> > > > > >> > the reliability of this test. Put it in another way, > > > > >> we want to > > > > >> > > > > >> > > > > > >> > > > > >> > ensure that this test change will pass all the time in > > > > >> various > > > > >> > > > > >> > > > > > >> > > > > >> > test configuration. > > > > >> > > > > >> > > > > > >> > > > > >> > Mandy > > > > >> > > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From daniel.daugherty at oracle.com Thu Sep 19 00:53:42 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 18 Sep 2019 20:53:42 -0400 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <0d316e76-44cc-4f65-a96d-e1207b1c94e6@oracle.com> References: <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> <9A151A35-DA66-4B0F-B67F-E7F5BECF205A@amazon.com> <586EB17C-1974-4FEF-A751-34B5B359BA61@amazon.com> <6139732f-a914-c59c-2119-517e480a279c@oracle.com> <8E8AE26F-5A7B-499C-BB6A-5EC954E42BD7@amazon.com> <0d316e76-44cc-4f65-a96d-e1207b1c94e6@oracle.com> Message-ID: Looks like the issue is different versions of 'hg' in use. When I import Paul's patch from his webrev using my 'hg' and then export it again, it matches my version of the backout. I have done a mechanical verification that the backout is an exact reversal for Paul's original changeset. I'm planning to push the changeset with the following info: 8231210: [BACKOUT] JDK-8207266 ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread Reviewed-by: phh, dholmes Everyone good with this? Dan On 9/18/19 8:44 PM, Daniel D. Daugherty wrote: > For some reason, the backout that I did does not match the backout > that you did so I'm trying to figure that out. > > Dan > > > > On 9/18/19 8:36 PM, Hohensee, Paul wrote: >> And I filed 8231211 for the same thing. :) >> >> Yes, please handle it, because it will go faster since I don't have >> access to a fast machine (just my laptop). >> >> Webrev here: >> >> http://cr.openjdk.java.net/~phh/8231211/webrev.00/ >> >> Thanks, >> >> ?On 9/18/19, 5:25 PM, "Daniel D. Daugherty" >> wrote: >> >> ???? I created this sub-task for you: >> ???? ???? JDK-8231210 [BACKOUT] JDK-8207266 >> ???? ThreadMXBean::getThreadAllocatedBytes() can be quicker for self >> thread >> ???? https://bugs.openjdk.java.net/browse/JDK-8231210 >> ???? ???? If you would prefer, I can handle this backout for you. >> ???? ???? Dan >> ???? ???? ???? On 9/18/19 8:21 PM, Hohensee, Paul wrote: >> ???? > Never having done this before, is it >> ???? > >> ???? > hg backout -r >> ???? > >> ???? > ? Do I file a JBS issue for the reversion? Seems necessary. >> ???? > >> ???? > On 9/18/19, 5:18 PM, "Daniel D. Daugherty" >> wrote: >> ???? > >> ???? >????? % hg backout >> ???? > >> ???? >????? is the usual way to do this... >> ???? > >> ???? >????? Dan >> ???? > >> ???? > >> ???? >????? On 9/18/19 8:17 PM, Hohensee, Paul wrote: >> ???? >????? > Is there a tool that will generate a reversal patch? >> ???? >????? > >> ???? >????? > On 9/18/19, 5:14 PM, "Daniel D. Daugherty" >> wrote: >> ???? >????? > >> ???? >????? >?????? > Shall I go with that, or reverse the original >> patch? >> ???? >????? > >> ???? >????? >????? I'm a bit worried about what else might show up >> since the >> ???? >????? >????? NSK monitoring tests were not run prior to this push. >> ???? >????? > >> ???? >????? >????? I vote for backing out the fix until proper >> testing has >> ???? >????? >????? been done (and at least the one problem fixed...) >> ???? >????? > >> ???? >????? >????? Dan >> ???? >????? > >> ???? >????? > >> ???? >????? >????? On 9/18/19 8:00 PM, Hohensee, Paul wrote: >> ???? >????? >????? > They all implement >> com.sun.management.ThreadMXBean, so adding a >> getCurrentThreadAllocatedBytes broke them. Potential fix is to give >> it a default implementation, vis >> ???? >????? >????? > >> ???? >????? >????? >????? public default long >> getCurrentThreadAllocatedBytes() { >> ???? >????? >????? >????????? return -1; >> ???? >????? >????? >????? } >> ???? >????? >????? > >> ???? >????? >????? > Shall I go with that, or reverse the original >> patch? >> ???? >????? >????? > >> ???? >????? >????? > On 9/18/19, 4:48 PM, "serviceability-dev on >> behalf of Hohensee, Paul" >> > hohensee at amazon.com> wrote: >> ???? >????? >????? > >> ???? >????? >????? >????? I'll take a look. >> ???? >????? >????? > >> ???? >????? >????? >????? On 9/18/19, 4:40 PM, "David Holmes" >> wrote: >> ???? >????? >????? > >> ???? >????? >????? >????????? Paul, >> ???? >????? >????? > >> ???? >????? >????? >????????? Unfortunately this patch has broken the >> vmTestbase/nsk/monitoring tests: >> ???? >????? >????? > >> ???? >????? >????? > [2019-09-18T22:59:32,349Z] >> ???? >????? >????? > >> /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: >> ???? >????? >????? >????????? error: ServerThreadMXBeanNew is not >> abstract and does not override >> ???? >????? >????? >????????? abstract method >> getCurrentThreadAllocatedBytes() in ThreadMXBean >> ???? >????? >????? > >> ???? >????? >????? >????????? and possibly other issues as we are >> seeing hundreds of failures. >> ???? >????? >????? > >> ???? >????? >????? >????????? David >> ???? >????? >????? > >> ???? >????? >????? >????????? On 18/09/2019 8:50 am, David Holmes wrote: >> ???? >????? >????? >????????? > On 18/09/2019 12:10 am, Hohensee, >> Paul wrote: >> ???? >????? >????? >????????? >> Thanks, Serguei. :) >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> David, are you ok with the patch? >> ???? >????? >????? >????????? > >> ???? >????? >????? >????????? > Yep, nothing further from me. >> ???? >????? >????? >????????? > >> ???? >????? >????? >????????? > David >> ???? >????? >????? >????????? > >> ???? >????? >????? >????????? >> Paul >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> *From: *"serguei.spitsyn at oracle.com" >> >> ???? >????? >????? >????????? >> *Date: *Tuesday, September 17, 2019 >> at 2:26 AM >> ???? >????? >????? >????????? >> *To: *"Hohensee, Paul" >> , David Holmes >> ???? >????? >????? >????????? >> , Mandy >> Chung >> ???? >????? >????? >????????? >> *Cc: *OpenJDK Serviceability >> , >> ???? >????? >????? >????????? >> "hotspot-gc-dev at openjdk.java.net" >> >> ???? >????? >????? >????????? >> *Subject: *Re: RFR (M): 8207266: >> ???? >????? >????? >????????? >> >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> Hi Paul, >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> Thank you for refactoring and fixing >> the test. >> ???? >????? >????? >????????? >> It looks great now! >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> Thanks, >> ???? >????? >????? >????????? >> Serguei >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> On 9/15/19 02:52, Hohensee, Paul wrote: >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???? Hi, Serguei, thanks for the >> review. New webrev at >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???? I refactored the test?s main() >> method, and you?re correct, >> ???? >????? >????? >????????? >> getThreadAllocatedBytes should be >> getCurrentThreadAllocatedBytes in >> ???? >????? >????? >????????? >>???? that context: fixed. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???? Paul >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???? *From: >> *"serguei.spitsyn at oracle.com" >> ???? >????? >????? >????????? >> >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> *Organization: *Oracle Corporation >> ???? >????? >????? >????????? >>???? *Date: *Friday, September 13, >> 2019 at 5:50 PM >> ???? >????? >????? >????????? >>???? *To: *"Hohensee, Paul" >> >> ???? >????? >????? >????????? >> , David >> Holmes >> ???? >????? >????? >????????? >> , >> Mandy Chung >> ???? >????? >????? >????????? >> >> >> ???? >????? >????? >????????? >>???? *Cc: *OpenJDK Serviceability >> >> ???? >????? >????? >????????? >> >> , >> ???? >????? >????? >????????? >> "hotspot-gc-dev at openjdk.java.net" >> ???? >????? >????? >????????? >> >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> >> ???? >????? >????? >????????? >>???? *Subject: *Re: RFR (M): 8207266: >> ???? >????? >????? >????????? >> >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self >> ???? >????? >????? >????????? >> thread >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???? Hi Paul, >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???? It looks pretty good in general. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???? It would be nice to refactor the >> java main() method as it becomes >> ???? >????? >????? >????????? >>???? too big. >> ???? >????? >????? >????????? >>???? Two ways >> ofgetCurrentThreadAllocatedBytes() testing are good >> ???? >????? >????? >????????? >> candidates >> ???? >????? >????? >????????? >>???? to become separate methods. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> 98???????? long size1 = >> mbean.getThreadAllocatedBytes(id); >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???? Just wanted to double check if >> you wanted to invoke >> ???? >????? >????? >????????? >>???? the >> getCurrentThreadAllocatedBytes() instead as it is >> ???? >????? >????? >????????? >>???? a part of: >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> 85???????? // First way, >> getCurrentThreadAllocatedBytes >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???? Thanks, >> ???? >????? >????? >????????? >>???? Serguei >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???? On 9/13/19 12:11 PM, Hohensee, >> Paul wrote: >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???????? Hi David, thanks for your >> comments. New webrev in >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???????? Both the old and new >> versions of the code check that thread >> ???? >????? >????? >????????? >> allocated memory is both supported >> and enabled. The existing version >> ???? >????? >????? >????????? >> of getThreadAllocatedBytes(long []) >> calls >> ???? >????? >????? >????????? >> verifyThreadAllocatedMemory(long >> []), which checks inline to make sure >> ???? >????? >????? >????????? >> thread allocated memory is >> supported, then calls >> ???? >????? >????? >????????? >> isThreadAllocatedMemoryEnabled() to >> verify that it's enabled. >> ???? >????? >????? >????????? >> isThreadAllocatedMemoryEnabled() >> duplicates (!) the support check and >> ???? >????? >????? >????????? >> returns the enabled flag. I removed >> the redundant check in the new >> ???? >????? >????? >????????? >> version. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???????? You're of course correct >> about the back-to-back check. >> ???? >????? >????? >????????? >> Application code can't know when the >> runtime will hijack a thread for >> ???? >????? >????? >????????? >> its own purposes. I've removed the >> check. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???????? Paul >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>???????? On 9/13/19, 12:50 AM, "David >> Holmes" >> ???? >????? >????? >????????? >> ? >> wrote: >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>????????????? Hi Paul, >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>????????????? On 13/09/2019 10:29 am, >> Hohensee, Paul wrote: >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > Thanks for clarifying the review >> rules. Would someone >> ???? >????? >????? >????????? >> from the >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > serviceability team please review? >> New webrev at >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>????????????? One aspect of the >> functional change needs clarification >> ???? >????? >????? >????????? >> for me - and >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> apologies if this has been covered >> in the past. It seems >> ???? >????? >????? >????????? >> to me that >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> currently we only check >> isThreadAllocatedMemorySupported >> ???? >????? >????? >????????? >> for these >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> operations, but if I read things >> correctly the updated >> ???? >????? >????? >????????? >> code additionally >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> checks >> isThreadAllocatedMemoryEnabled, which is a >> ???? >????? >????? >????????? >> behaviour change not >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> mentioned in the CSR. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > I didn?t disturb the existing >> checks in the test, just >> ???? >????? >????? >????????? >> added code to >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > check the result of >> getThreadAllocatedBytes(long) on a >> ???? >????? >????? >????????? >> non-current >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > thread, plus the back-to-back >> no-allocation checks. The >> ???? >????? >????? >????????? >> former wasn?t >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > needed before because >> getThreadAllocatedBytes(long) was >> ???? >????? >????? >????????? >> just a wrapper >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > around >> getThreadAllocatedBytes(long []). This patch >> ???? >????? >????? >????????? >> changes that, so I >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > added a separate test. The latter >> is supposed to fail >> ???? >????? >????? >????????? >> if there?s object >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > allocation on calls to >> getCurrentThreadAllocatedBytes and >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > getThreadAllocatedBytes(long). >> I.e., a feature, not a >> ???? >????? >????? >????????? >> bug, because >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > accumulation of transient small >> objects can be a >> ???? >????? >????? >????????? >> performance problem. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > Thanks to your review, I noticed >> that the back-to-back >> ???? >????? >????? >????????? >> check on the >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > current thread was using >> getThreadAllocatedBytes(long) >> ???? >????? >????? >????????? >> instead of >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > getCurrentThreadAllocatedBytes and >> fixed it. I also >> ???? >????? >????? >????????? >> removed all >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > instances of ?TEST FAILED: ?. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>????????????? The back-to-back check >> is not valid in general. You don't >> ???? >????? >????? >????????? >> know if the >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> first check might trigger some class >> loading on the >> ???? >????? >????? >????????? >> return path after it >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>????????????? has obtained the first >> memory value. The check might also >> ???? >????? >????? >????????? >> fail if using >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> JVMCI and some compilation related >> activity occurs in the >> ???? >????? >????? >????????? >> current thread >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>????????????? on the second call. >> Also with the introduction of >> ???? >????? >????? >????????? >> handshakes its >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> possible the current thread might >> hit a safepoint checks >> ???? >????? >????? >????????? >> that results in >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>????????????? it executing a >> handshake operation that performs >> ???? >????? >????? >????????? >> allocation. Potentially >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> there could be numerous >> non-deterministic actions that >> ???? >????? >????? >????????? >> might occur >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> leading to unanticipated allocation. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >>????????????? I understand what you >> want to test here, I just don't >> ???? >????? >????? >????????? >> think it is >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> reliably doable. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> Thanks, >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> David >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> ----- >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > Paul >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > *From: *Mandy >> Chung >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > *Date: *Thursday, September 12, >> 2019 at 10:09 AM >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > *To: *"Hohensee, >> Paul" >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > *Cc: *OpenJDK >> ???? >????? >????? >????????? >> >> Serviceability >> ???? >????? >????? >????????? >> >> , >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >"hotspot-gc-dev at openjdk.java.net" >> ???? >????? >????? >????????? >> >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > *Subject: *Re: RFR (M): 8207266: >> ???? >????? >????? >????????? >> ThreadMXBean::getThreadAllocatedBytes() >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > can be quicker for self thread >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > On 9/3/19 12:38 PM, Hohensee, Paul >> wrote: >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >???? Minor update in new >> ???? >????? >????? >????????? >> >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > I only reviewed the library side >> implementation that >> ???? >????? >????? >????????? >> looks good.? I >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > expect the serviceability team to >> review the test and >> ???? >????? >????? >????????? >> hotspot change. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >???? Need a confirmatory review to >> push this. If I >> ???? >????? >????? >????????? >> understand the rules correctly, it >> doesn't need a Reviewer review >> ???? >????? >????? >????????? >> since Mandy's already reviewed it, >> it just needs a Committer review. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > You need another reviewer to >> advice the following >> ???? >????? >????? >????????? >> because I was not >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > close to the ThreadsList work. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > 2087?? ThreadsListHandle tlh; >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > 2088?? JavaThread* java_thread = >> ???? >????? >????? >????????? >> >> tlh.list()->find_JavaThread_from_java_tid(thread_id); >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > 2089 >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > 2090?? if (java_thread != NULL) { >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > 2091???? return >> java_thread->cooked_allocated_bytes(); >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > 2092?? } >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > This looks right to me. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > -??????????????? >> "ThreadAllocatedMemory is expected to >> ???? >????? >????? >????????? >> be disabled"); >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > +??????????????? "TEST FAILED: >> ThreadAllocatedMemory is >> ???? >????? >????? >????????? >> expected to be >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > disabled"); >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > Prepending "TEST FAILED" in >> exception message (in >> ???? >????? >????? >????????? >> several places) >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > seems redundant since such >> RuntimeException is thrown >> ???? >????? >????? >????????? >> and expected >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > a test failure. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > +??????? // back-to-back calls >> shouldn't allocate any >> ???? >????? >????? >????????? >> memory >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > +??????? size = >> mbean.getThreadAllocatedBytes(id); >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > +??????? size1 = >> mbean.getThreadAllocatedBytes(id); >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > +??????? if (size1 != size) { >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > Is there anything in the test can >> do to help guarantee >> ???? >????? >????? >????????? >> this? I didn't >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > closely review this test.? The >> main thing I advice is >> ???? >????? >????? >????????? >> to improve >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > the reliability of this test.? Put >> it in another way, >> ???? >????? >????? >????????? >> we want to >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > ensure that this test change will >> pass all the time in >> ???? >????? >????? >????????? >> various >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > test configuration. >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > Mandy >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> > >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? >????????? >> >> ???? >????? >????? > >> ???? >????? >????? > >> ???? >????? >????? > >> ???? >????? >????? > >> ???? >????? > >> ???? >????? > >> ???? >????? > >> ???? > >> ???? > >> ???? > >> > From hohensee at amazon.com Thu Sep 19 00:55:26 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 19 Sep 2019 00:55:26 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: References: <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> <9A151A35-DA66-4B0F-B67F-E7F5BECF205A@amazon.com> <586EB17C-1974-4FEF-A751-34B5B359BA61@amazon.com> <6139732f-a914-c59c-2119-517e480a279c@oracle.com> <8E8AE26F-5A7B-499C-BB6A-5EC954E42BD7@amazon.com> <0d316e76-44cc-4f65-a96d-e1207b1c94e6@oracle.com> Message-ID: I am. ?On 9/18/19, 5:54 PM, "Daniel D. Daugherty" wrote: Looks like the issue is different versions of 'hg' in use. When I import Paul's patch from his webrev using my 'hg' and then export it again, it matches my version of the backout. I have done a mechanical verification that the backout is an exact reversal for Paul's original changeset. I'm planning to push the changeset with the following info: 8231210: [BACKOUT] JDK-8207266 ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread Reviewed-by: phh, dholmes Everyone good with this? Dan On 9/18/19 8:44 PM, Daniel D. Daugherty wrote: > For some reason, the backout that I did does not match the backout > that you did so I'm trying to figure that out. > > Dan > > > > On 9/18/19 8:36 PM, Hohensee, Paul wrote: >> And I filed 8231211 for the same thing. :) >> >> Yes, please handle it, because it will go faster since I don't have >> access to a fast machine (just my laptop). >> >> Webrev here: >> >> http://cr.openjdk.java.net/~phh/8231211/webrev.00/ >> >> Thanks, >> >> On 9/18/19, 5:25 PM, "Daniel D. Daugherty" >> wrote: >> >> I created this sub-task for you: >> JDK-8231210 [BACKOUT] JDK-8207266 >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self >> thread >> https://bugs.openjdk.java.net/browse/JDK-8231210 >> If you would prefer, I can handle this backout for you. >> Dan >> On 9/18/19 8:21 PM, Hohensee, Paul wrote: >> > Never having done this before, is it >> > >> > hg backout -r >> > >> > ? Do I file a JBS issue for the reversion? Seems necessary. >> > >> > On 9/18/19, 5:18 PM, "Daniel D. Daugherty" >> wrote: >> > >> > % hg backout >> > >> > is the usual way to do this... >> > >> > Dan >> > >> > >> > On 9/18/19 8:17 PM, Hohensee, Paul wrote: >> > > Is there a tool that will generate a reversal patch? >> > > >> > > On 9/18/19, 5:14 PM, "Daniel D. Daugherty" >> wrote: >> > > >> > > > Shall I go with that, or reverse the original >> patch? >> > > >> > > I'm a bit worried about what else might show up >> since the >> > > NSK monitoring tests were not run prior to this push. >> > > >> > > I vote for backing out the fix until proper >> testing has >> > > been done (and at least the one problem fixed...) >> > > >> > > Dan >> > > >> > > >> > > On 9/18/19 8:00 PM, Hohensee, Paul wrote: >> > > > They all implement >> com.sun.management.ThreadMXBean, so adding a >> getCurrentThreadAllocatedBytes broke them. Potential fix is to give >> it a default implementation, vis >> > > > >> > > > public default long >> getCurrentThreadAllocatedBytes() { >> > > > return -1; >> > > > } >> > > > >> > > > Shall I go with that, or reverse the original >> patch? >> > > > >> > > > On 9/18/19, 4:48 PM, "serviceability-dev on >> behalf of Hohensee, Paul" >> > hohensee at amazon.com> wrote: >> > > > >> > > > I'll take a look. >> > > > >> > > > On 9/18/19, 4:40 PM, "David Holmes" >> wrote: >> > > > >> > > > Paul, >> > > > >> > > > Unfortunately this patch has broken the >> vmTestbase/nsk/monitoring tests: >> > > > >> > > > [2019-09-18T22:59:32,349Z] >> > > > >> /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: >> > > > error: ServerThreadMXBeanNew is not >> abstract and does not override >> > > > abstract method >> getCurrentThreadAllocatedBytes() in ThreadMXBean >> > > > >> > > > and possibly other issues as we are >> seeing hundreds of failures. >> > > > >> > > > David >> > > > >> > > > On 18/09/2019 8:50 am, David Holmes wrote: >> > > > > On 18/09/2019 12:10 am, Hohensee, >> Paul wrote: >> > > > >> Thanks, Serguei. :) >> > > > >> >> > > > >> David, are you ok with the patch? >> > > > > >> > > > > Yep, nothing further from me. >> > > > > >> > > > > David >> > > > > >> > > > >> Paul >> > > > >> >> > > > >> *From: *"serguei.spitsyn at oracle.com" >> >> > > > >> *Date: *Tuesday, September 17, 2019 >> at 2:26 AM >> > > > >> *To: *"Hohensee, Paul" >> , David Holmes >> > > > >> , Mandy >> Chung >> > > > >> *Cc: *OpenJDK Serviceability >> , >> > > > >> "hotspot-gc-dev at openjdk.java.net" >> >> > > > >> *Subject: *Re: RFR (M): 8207266: >> > > > >> >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread >> > > > >> >> > > > >> Hi Paul, >> > > > >> >> > > > >> Thank you for refactoring and fixing >> the test. >> > > > >> It looks great now! >> > > > >> >> > > > >> Thanks, >> > > > >> Serguei >> > > > >> >> > > > >> >> > > > >> On 9/15/19 02:52, Hohensee, Paul wrote: >> > > > >> >> > > > >> Hi, Serguei, thanks for the >> review. New webrev at >> > > > >> >> > > > >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ >> > > > >> >> > > > >> I refactored the test?s main() >> method, and you?re correct, >> > > > >> getThreadAllocatedBytes should be >> getCurrentThreadAllocatedBytes in >> > > > >> that context: fixed. >> > > > >> >> > > > >> Paul >> > > > >> >> > > > >> *From: >> *"serguei.spitsyn at oracle.com" >> > > > >> >> >> > > > >> >> > > > >> *Organization: *Oracle Corporation >> > > > >> *Date: *Friday, September 13, >> 2019 at 5:50 PM >> > > > >> *To: *"Hohensee, Paul" >> >> > > > >> , David >> Holmes >> > > > >> , >> Mandy Chung >> > > > >> >> >> > > > >> *Cc: *OpenJDK Serviceability >> >> > > > >> >> , >> > > > >> "hotspot-gc-dev at openjdk.java.net" >> > > > >> >> >> > > > >> >> > > > >> >> >> > > > >> *Subject: *Re: RFR (M): 8207266: >> > > > >> >> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self >> > > > >> thread >> > > > >> >> > > > >> Hi Paul, >> > > > >> >> > > > >> It looks pretty good in general. >> > > > >> >> > > > >> >> > > > >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html >> > > > >> >> > > > >> >> > > > >> It would be nice to refactor the >> java main() method as it becomes >> > > > >> too big. >> > > > >> Two ways >> ofgetCurrentThreadAllocatedBytes() testing are good >> > > > >> candidates >> > > > >> to become separate methods. >> > > > >> >> > > > >> 98 long size1 = >> mbean.getThreadAllocatedBytes(id); >> > > > >> >> > > > >> Just wanted to double check if >> you wanted to invoke >> > > > >> the >> getCurrentThreadAllocatedBytes() instead as it is >> > > > >> a part of: >> > > > >> >> > > > >> 85 // First way, >> getCurrentThreadAllocatedBytes >> > > > >> >> > > > >> >> > > > >> Thanks, >> > > > >> Serguei >> > > > >> >> > > > >> On 9/13/19 12:11 PM, Hohensee, >> Paul wrote: >> > > > >> >> > > > >> Hi David, thanks for your >> comments. New webrev in >> > > > >> >> > > > >> >> > > > >> >> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ >> > > > >> >> > > > >> >> > > > >> Both the old and new >> versions of the code check that thread >> > > > >> allocated memory is both supported >> and enabled. The existing version >> > > > >> of getThreadAllocatedBytes(long []) >> calls >> > > > >> verifyThreadAllocatedMemory(long >> []), which checks inline to make sure >> > > > >> thread allocated memory is >> supported, then calls >> > > > >> isThreadAllocatedMemoryEnabled() to >> verify that it's enabled. >> > > > >> isThreadAllocatedMemoryEnabled() >> duplicates (!) the support check and >> > > > >> returns the enabled flag. I removed >> the redundant check in the new >> > > > >> version. >> > > > >> >> > > > >> >> > > > >> You're of course correct >> about the back-to-back check. >> > > > >> Application code can't know when the >> runtime will hijack a thread for >> > > > >> its own purposes. I've removed the >> check. >> > > > >> >> > > > >> >> > > > >> Paul >> > > > >> >> > > > >> >> > > > >> On 9/13/19, 12:50 AM, "David >> Holmes" >> > > > >> >> wrote: >> > > > >> >> > > > >> >> > > > >> Hi Paul, >> > > > >> >> > > > >> >> > > > >> On 13/09/2019 10:29 am, >> Hohensee, Paul wrote: >> > > > >> >> > > > >> > Thanks for clarifying the review >> rules. Would someone >> > > > >> from the >> > > > >> >> > > > >> > serviceability team please review? >> New webrev at >> > > > >> >> > > > >> > >> > > > >> >> > > > >> >> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ >> > > > >> >> > > > >> >> > > > >> One aspect of the >> functional change needs clarification >> > > > >> for me - and >> > > > >> >> > > > >> apologies if this has been covered >> in the past. It seems >> > > > >> to me that >> > > > >> >> > > > >> currently we only check >> isThreadAllocatedMemorySupported >> > > > >> for these >> > > > >> >> > > > >> operations, but if I read things >> correctly the updated >> > > > >> code additionally >> > > > >> >> > > > >> checks >> isThreadAllocatedMemoryEnabled, which is a >> > > > >> behaviour change not >> > > > >> >> > > > >> mentioned in the CSR. >> > > > >> >> > > > >> >> > > > >> > I didn?t disturb the existing >> checks in the test, just >> > > > >> added code to >> > > > >> >> > > > >> > check the result of >> getThreadAllocatedBytes(long) on a >> > > > >> non-current >> > > > >> >> > > > >> > thread, plus the back-to-back >> no-allocation checks. The >> > > > >> former wasn?t >> > > > >> >> > > > >> > needed before because >> getThreadAllocatedBytes(long) was >> > > > >> just a wrapper >> > > > >> >> > > > >> > around >> getThreadAllocatedBytes(long []). This patch >> > > > >> changes that, so I >> > > > >> >> > > > >> > added a separate test. The latter >> is supposed to fail >> > > > >> if there?s object >> > > > >> >> > > > >> > allocation on calls to >> getCurrentThreadAllocatedBytes and >> > > > >> >> > > > >> > getThreadAllocatedBytes(long). >> I.e., a feature, not a >> > > > >> bug, because >> > > > >> >> > > > >> > accumulation of transient small >> objects can be a >> > > > >> performance problem. >> > > > >> >> > > > >> > Thanks to your review, I noticed >> that the back-to-back >> > > > >> check on the >> > > > >> >> > > > >> > current thread was using >> getThreadAllocatedBytes(long) >> > > > >> instead of >> > > > >> >> > > > >> > getCurrentThreadAllocatedBytes and >> fixed it. I also >> > > > >> removed all >> > > > >> >> > > > >> > instances of ?TEST FAILED: ?. >> > > > >> >> > > > >> >> > > > >> The back-to-back check >> is not valid in general. You don't >> > > > >> know if the >> > > > >> >> > > > >> first check might trigger some class >> loading on the >> > > > >> return path after it >> > > > >> >> > > > >> has obtained the first >> memory value. The check might also >> > > > >> fail if using >> > > > >> >> > > > >> JVMCI and some compilation related >> activity occurs in the >> > > > >> current thread >> > > > >> >> > > > >> on the second call. >> Also with the introduction of >> > > > >> handshakes its >> > > > >> >> > > > >> possible the current thread might >> hit a safepoint checks >> > > > >> that results in >> > > > >> >> > > > >> it executing a >> handshake operation that performs >> > > > >> allocation. Potentially >> > > > >> >> > > > >> there could be numerous >> non-deterministic actions that >> > > > >> might occur >> > > > >> >> > > > >> leading to unanticipated allocation. >> > > > >> >> > > > >> >> > > > >> I understand what you >> want to test here, I just don't >> > > > >> think it is >> > > > >> >> > > > >> reliably doable. >> > > > >> >> > > > >> >> > > > >> Thanks, >> > > > >> >> > > > >> David >> > > > >> >> > > > >> ----- >> > > > >> >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > Paul >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > *From: *Mandy >> Chung >> > > > >> >> > > > >> >> > > > >> > *Date: *Thursday, September 12, >> 2019 at 10:09 AM >> > > > >> >> > > > >> > *To: *"Hohensee, >> Paul" >> > > > >> >> > > > >> >> > > > >> > *Cc: *OpenJDK >> > > > >> >> Serviceability >> > > > >> >> , >> > > > >> >> > > > >> >"hotspot-gc-dev at openjdk.java.net" >> > > > >> >> >> > > > >> >> > > > >> >> >> > > > >> >> > > > >> > *Subject: *Re: RFR (M): 8207266: >> > > > >> ThreadMXBean::getThreadAllocatedBytes() >> > > > >> >> > > > >> > can be quicker for self thread >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > On 9/3/19 12:38 PM, Hohensee, Paul >> wrote: >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > Minor update in new >> > > > >> >> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > I only reviewed the library side >> implementation that >> > > > >> looks good. I >> > > > >> >> > > > >> > expect the serviceability team to >> review the test and >> > > > >> hotspot change. >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > Need a confirmatory review to >> push this. If I >> > > > >> understand the rules correctly, it >> doesn't need a Reviewer review >> > > > >> since Mandy's already reviewed it, >> it just needs a Committer review. >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > You need another reviewer to >> advice the following >> > > > >> because I was not >> > > > >> >> > > > >> > close to the ThreadsList work. >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > 2087 ThreadsListHandle tlh; >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > 2088 JavaThread* java_thread = >> > > > >> >> tlh.list()->find_JavaThread_from_java_tid(thread_id); >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > 2089 >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > 2090 if (java_thread != NULL) { >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > 2091 return >> java_thread->cooked_allocated_bytes(); >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > 2092 } >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > This looks right to me. >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > >> > > > >> >> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > - >> "ThreadAllocatedMemory is expected to >> > > > >> be disabled"); >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > + "TEST FAILED: >> ThreadAllocatedMemory is >> > > > >> expected to be >> > > > >> >> > > > >> > disabled"); >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > Prepending "TEST FAILED" in >> exception message (in >> > > > >> several places) >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > seems redundant since such >> RuntimeException is thrown >> > > > >> and expected >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > a test failure. >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > + // back-to-back calls >> shouldn't allocate any >> > > > >> memory >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > + size = >> mbean.getThreadAllocatedBytes(id); >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > + size1 = >> mbean.getThreadAllocatedBytes(id); >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > + if (size1 != size) { >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > Is there anything in the test can >> do to help guarantee >> > > > >> this? I didn't >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > closely review this test. The >> main thing I advice is >> > > > >> to improve >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > the reliability of this test. Put >> it in another way, >> > > > >> we want to >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > ensure that this test change will >> pass all the time in >> > > > >> various >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > test configuration. >> > > > >> >> > > > >> > >> > > > >> >> > > > >> > Mandy >> > > > >> >> > > > >> > >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> > > > >> > > > >> > > > >> > > >> > > >> > > >> > >> > >> > >> > From david.holmes at oracle.com Thu Sep 19 01:00:07 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 19 Sep 2019 11:00:07 +1000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: References: <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> <9A151A35-DA66-4B0F-B67F-E7F5BECF205A@amazon.com> <586EB17C-1974-4FEF-A751-34B5B359BA61@amazon.com> <6139732f-a914-c59c-2119-517e480a279c@oracle.com> <8E8AE26F-5A7B-499C-BB6A-5EC954E42BD7@amazon.com> <0d316e76-44cc-4f65-a96d-e1207b1c94e6@oracle.com> Message-ID: <114ed65b-4b13-22aa-1849-f2737f7a9779@oracle.com> Ship it! Thanks Dan! David On 19/09/2019 10:53 am, Daniel D. Daugherty wrote: > Looks like the issue is different versions of 'hg' in use. > > When I import Paul's patch from his webrev using my 'hg' and > then export it again, it matches my version of the backout. > > I have done a mechanical verification that the backout is an > exact reversal for Paul's original changeset. > > I'm planning to push the changeset with the following info: > > > 8231210: [BACKOUT] JDK-8207266 ThreadMXBean::getThreadAllocatedBytes() > can be quicker for self thread > Reviewed-by: phh, dholmes > > Everyone good with this? > > Dan > > On 9/18/19 8:44 PM, Daniel D. Daugherty wrote: >> For some reason, the backout that I did does not match the backout >> that you did so I'm trying to figure that out. >> >> Dan >> >> >> >> On 9/18/19 8:36 PM, Hohensee, Paul wrote: >>> And I filed 8231211 for the same thing. :) >>> >>> Yes, please handle it, because it will go faster since I don't have >>> access to a fast machine (just my laptop). >>> >>> Webrev here: >>> >>> http://cr.openjdk.java.net/~phh/8231211/webrev.00/ >>> >>> Thanks, >>> >>> ?On 9/18/19, 5:25 PM, "Daniel D. Daugherty" >>> wrote: >>> >>> ???? I created this sub-task for you: >>> ???? ???? JDK-8231210 [BACKOUT] JDK-8207266 >>> ???? ThreadMXBean::getThreadAllocatedBytes() can be quicker for self >>> thread >>> ???? https://bugs.openjdk.java.net/browse/JDK-8231210 >>> ???? ???? If you would prefer, I can handle this backout for you. >>> ???? ???? Dan >>> ???? ???? ???? On 9/18/19 8:21 PM, Hohensee, Paul wrote: >>> ???? > Never having done this before, is it >>> ???? > >>> ???? > hg backout -r >>> ???? > >>> ???? > ? Do I file a JBS issue for the reversion? Seems necessary. >>> ???? > >>> ???? > On 9/18/19, 5:18 PM, "Daniel D. Daugherty" >>> wrote: >>> ???? > >>> ???? >????? % hg backout >>> ???? > >>> ???? >????? is the usual way to do this... >>> ???? > >>> ???? >????? Dan >>> ???? > >>> ???? > >>> ???? >????? On 9/18/19 8:17 PM, Hohensee, Paul wrote: >>> ???? >????? > Is there a tool that will generate a reversal patch? >>> ???? >????? > >>> ???? >????? > On 9/18/19, 5:14 PM, "Daniel D. Daugherty" >>> wrote: >>> ???? >????? > >>> ???? >????? >?????? > Shall I go with that, or reverse the original >>> patch? >>> ???? >????? > >>> ???? >????? >????? I'm a bit worried about what else might show up >>> since the >>> ???? >????? >????? NSK monitoring tests were not run prior to this push. >>> ???? >????? > >>> ???? >????? >????? I vote for backing out the fix until proper >>> testing has >>> ???? >????? >????? been done (and at least the one problem fixed...) >>> ???? >????? > >>> ???? >????? >????? Dan >>> ???? >????? > >>> ???? >????? > >>> ???? >????? >????? On 9/18/19 8:00 PM, Hohensee, Paul wrote: >>> ???? >????? >????? > They all implement >>> com.sun.management.ThreadMXBean, so adding a >>> getCurrentThreadAllocatedBytes broke them. Potential fix is to give >>> it a default implementation, vis >>> ???? >????? >????? > >>> ???? >????? >????? >????? public default long >>> getCurrentThreadAllocatedBytes() { >>> ???? >????? >????? >????????? return -1; >>> ???? >????? >????? >????? } >>> ???? >????? >????? > >>> ???? >????? >????? > Shall I go with that, or reverse the original >>> patch? >>> ???? >????? >????? > >>> ???? >????? >????? > On 9/18/19, 4:48 PM, "serviceability-dev on >>> behalf of Hohensee, Paul" >>> >> hohensee at amazon.com> wrote: >>> ???? >????? >????? > >>> ???? >????? >????? >????? I'll take a look. >>> ???? >????? >????? > >>> ???? >????? >????? >????? On 9/18/19, 4:40 PM, "David Holmes" >>> wrote: >>> ???? >????? >????? > >>> ???? >????? >????? >????????? Paul, >>> ???? >????? >????? > >>> ???? >????? >????? >????????? Unfortunately this patch has broken the >>> vmTestbase/nsk/monitoring tests: >>> ???? >????? >????? > >>> ???? >????? >????? > [2019-09-18T22:59:32,349Z] >>> ???? >????? >????? > >>> /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: >>> >>> ???? >????? >????? >????????? error: ServerThreadMXBeanNew is not >>> abstract and does not override >>> ???? >????? >????? >????????? abstract method >>> getCurrentThreadAllocatedBytes() in ThreadMXBean >>> ???? >????? >????? > >>> ???? >????? >????? >????????? and possibly other issues as we are >>> seeing hundreds of failures. >>> ???? >????? >????? > >>> ???? >????? >????? >????????? David >>> ???? >????? >????? > >>> ???? >????? >????? >????????? On 18/09/2019 8:50 am, David Holmes wrote: >>> ???? >????? >????? >????????? > On 18/09/2019 12:10 am, Hohensee, >>> Paul wrote: >>> ???? >????? >????? >????????? >> Thanks, Serguei. :) >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> David, are you ok with the patch? >>> ???? >????? >????? >????????? > >>> ???? >????? >????? >????????? > Yep, nothing further from me. >>> ???? >????? >????? >????????? > >>> ???? >????? >????? >????????? > David >>> ???? >????? >????? >????????? > >>> ???? >????? >????? >????????? >> Paul >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> *From: *"serguei.spitsyn at oracle.com" >>> >>> ???? >????? >????? >????????? >> *Date: *Tuesday, September 17, 2019 >>> at 2:26 AM >>> ???? >????? >????? >????????? >> *To: *"Hohensee, Paul" >>> , David Holmes >>> ???? >????? >????? >????????? >> , Mandy >>> Chung >>> ???? >????? >????? >????????? >> *Cc: *OpenJDK Serviceability >>> , >>> ???? >????? >????? >????????? >> "hotspot-gc-dev at openjdk.java.net" >>> >>> ???? >????? >????? >????????? >> *Subject: *Re: RFR (M): 8207266: >>> ???? >????? >????? >????????? >> >>> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> Hi Paul, >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> Thank you for refactoring and fixing >>> the test. >>> ???? >????? >????? >????????? >> It looks great now! >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> Thanks, >>> ???? >????? >????? >????????? >> Serguei >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> On 9/15/19 02:52, Hohensee, Paul wrote: >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???? Hi, Serguei, thanks for the >>> review. New webrev at >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???? I refactored the test?s main() >>> method, and you?re correct, >>> ???? >????? >????? >????????? >> getThreadAllocatedBytes should be >>> getCurrentThreadAllocatedBytes in >>> ???? >????? >????? >????????? >>???? that context: fixed. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???? Paul >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???? *From: >>> *"serguei.spitsyn at oracle.com" >>> ???? >????? >????? >????????? >> >>> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> *Organization: *Oracle Corporation >>> ???? >????? >????? >????????? >>???? *Date: *Friday, September 13, >>> 2019 at 5:50 PM >>> ???? >????? >????? >????????? >>???? *To: *"Hohensee, Paul" >>> >>> ???? >????? >????? >????????? >> , David >>> Holmes >>> ???? >????? >????? >????????? >> , >>> Mandy Chung >>> ???? >????? >????? >????????? >> >>> >>> ???? >????? >????? >????????? >>???? *Cc: *OpenJDK Serviceability >>> >>> ???? >????? >????? >????????? >> >>> , >>> ???? >????? >????? >????????? >> "hotspot-gc-dev at openjdk.java.net" >>> ???? >????? >????? >????????? >> >>> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> >>> ???? >????? >????? >????????? >>???? *Subject: *Re: RFR (M): 8207266: >>> ???? >????? >????? >????????? >> >>> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self >>> ???? >????? >????? >????????? >> thread >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???? Hi Paul, >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???? It looks pretty good in general. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html >>> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???? It would be nice to refactor the >>> java main() method as it becomes >>> ???? >????? >????? >????????? >>???? too big. >>> ???? >????? >????? >????????? >>???? Two ways >>> ofgetCurrentThreadAllocatedBytes() testing are good >>> ???? >????? >????? >????????? >> candidates >>> ???? >????? >????? >????????? >>???? to become separate methods. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> 98???????? long size1 = >>> mbean.getThreadAllocatedBytes(id); >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???? Just wanted to double check if >>> you wanted to invoke >>> ???? >????? >????? >????????? >>???? the >>> getCurrentThreadAllocatedBytes() instead as it is >>> ???? >????? >????? >????????? >>???? a part of: >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> 85???????? // First way, >>> getCurrentThreadAllocatedBytes >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???? Thanks, >>> ???? >????? >????? >????????? >>???? Serguei >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???? On 9/13/19 12:11 PM, Hohensee, >>> Paul wrote: >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???????? Hi David, thanks for your >>> comments. New webrev in >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???????? Both the old and new >>> versions of the code check that thread >>> ???? >????? >????? >????????? >> allocated memory is both supported >>> and enabled. The existing version >>> ???? >????? >????? >????????? >> of getThreadAllocatedBytes(long []) >>> calls >>> ???? >????? >????? >????????? >> verifyThreadAllocatedMemory(long >>> []), which checks inline to make sure >>> ???? >????? >????? >????????? >> thread allocated memory is >>> supported, then calls >>> ???? >????? >????? >????????? >> isThreadAllocatedMemoryEnabled() to >>> verify that it's enabled. >>> ???? >????? >????? >????????? >> isThreadAllocatedMemoryEnabled() >>> duplicates (!) the support check and >>> ???? >????? >????? >????????? >> returns the enabled flag. I removed >>> the redundant check in the new >>> ???? >????? >????? >????????? >> version. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???????? You're of course correct >>> about the back-to-back check. >>> ???? >????? >????? >????????? >> Application code can't know when the >>> runtime will hijack a thread for >>> ???? >????? >????? >????????? >> its own purposes. I've removed the >>> check. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???????? Paul >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>???????? On 9/13/19, 12:50 AM, "David >>> Holmes" >>> ???? >????? >????? >????????? >> wrote: >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>????????????? Hi Paul, >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>????????????? On 13/09/2019 10:29 am, >>> Hohensee, Paul wrote: >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > Thanks for clarifying the review >>> rules. Would someone >>> ???? >????? >????? >????????? >> from the >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > serviceability team please review? >>> New webrev at >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>????????????? One aspect of the >>> functional change needs clarification >>> ???? >????? >????? >????????? >> for me - and >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> apologies if this has been covered >>> in the past. It seems >>> ???? >????? >????? >????????? >> to me that >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> currently we only check >>> isThreadAllocatedMemorySupported >>> ???? >????? >????? >????????? >> for these >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> operations, but if I read things >>> correctly the updated >>> ???? >????? >????? >????????? >> code additionally >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> checks >>> isThreadAllocatedMemoryEnabled, which is a >>> ???? >????? >????? >????????? >> behaviour change not >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> mentioned in the CSR. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > I didn?t disturb the existing >>> checks in the test, just >>> ???? >????? >????? >????????? >> added code to >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > check the result of >>> getThreadAllocatedBytes(long) on a >>> ???? >????? >????? >????????? >> non-current >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > thread, plus the back-to-back >>> no-allocation checks. The >>> ???? >????? >????? >????????? >> former wasn?t >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > needed before because >>> getThreadAllocatedBytes(long) was >>> ???? >????? >????? >????????? >> just a wrapper >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > around >>> getThreadAllocatedBytes(long []). This patch >>> ???? >????? >????? >????????? >> changes that, so I >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > added a separate test. The latter >>> is supposed to fail >>> ???? >????? >????? >????????? >> if there?s object >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > allocation on calls to >>> getCurrentThreadAllocatedBytes and >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > getThreadAllocatedBytes(long). >>> I.e., a feature, not a >>> ???? >????? >????? >????????? >> bug, because >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > accumulation of transient small >>> objects can be a >>> ???? >????? >????? >????????? >> performance problem. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > Thanks to your review, I noticed >>> that the back-to-back >>> ???? >????? >????? >????????? >> check on the >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > current thread was using >>> getThreadAllocatedBytes(long) >>> ???? >????? >????? >????????? >> instead of >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > getCurrentThreadAllocatedBytes and >>> fixed it. I also >>> ???? >????? >????? >????????? >> removed all >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > instances of ?TEST FAILED: ?. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>????????????? The back-to-back check >>> is not valid in general. You don't >>> ???? >????? >????? >????????? >> know if the >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> first check might trigger some class >>> loading on the >>> ???? >????? >????? >????????? >> return path after it >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>????????????? has obtained the first >>> memory value. The check might also >>> ???? >????? >????? >????????? >> fail if using >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> JVMCI and some compilation related >>> activity occurs in the >>> ???? >????? >????? >????????? >> current thread >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>????????????? on the second call. >>> Also with the introduction of >>> ???? >????? >????? >????????? >> handshakes its >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> possible the current thread might >>> hit a safepoint checks >>> ???? >????? >????? >????????? >> that results in >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>????????????? it executing a >>> handshake operation that performs >>> ???? >????? >????? >????????? >> allocation. Potentially >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> there could be numerous >>> non-deterministic actions that >>> ???? >????? >????? >????????? >> might occur >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> leading to unanticipated allocation. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >>????????????? I understand what you >>> want to test here, I just don't >>> ???? >????? >????? >????????? >> think it is >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> reliably doable. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> Thanks, >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> David >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> ----- >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > Paul >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > *From: *Mandy >>> Chung >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > *Date: *Thursday, September 12, >>> 2019 at 10:09 AM >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > *To: *"Hohensee, >>> Paul" >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > *Cc: *OpenJDK >>> ???? >????? >????? >????????? >> >>> Serviceability >>> ???? >????? >????? >????????? >> >>> , >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >"hotspot-gc-dev at openjdk.java.net" >>> ???? >????? >????? >????????? >> >>> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > *Subject: *Re: RFR (M): 8207266: >>> ???? >????? >????? >????????? >> ThreadMXBean::getThreadAllocatedBytes() >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > can be quicker for self thread >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > On 9/3/19 12:38 PM, Hohensee, Paul >>> wrote: >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >???? Minor update in new >>> ???? >????? >????? >????????? >> >>> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > I only reviewed the library side >>> implementation that >>> ???? >????? >????? >????????? >> looks good.? I >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > expect the serviceability team to >>> review the test and >>> ???? >????? >????? >????????? >> hotspot change. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >???? Need a confirmatory review to >>> push this. If I >>> ???? >????? >????? >????????? >> understand the rules correctly, it >>> doesn't need a Reviewer review >>> ???? >????? >????? >????????? >> since Mandy's already reviewed it, >>> it just needs a Committer review. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > You need another reviewer to >>> advice the following >>> ???? >????? >????? >????????? >> because I was not >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > close to the ThreadsList work. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > 2087?? ThreadsListHandle tlh; >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > 2088?? JavaThread* java_thread = >>> ???? >????? >????? >????????? >> >>> tlh.list()->find_JavaThread_from_java_tid(thread_id); >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > 2089 >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > 2090?? if (java_thread != NULL) { >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > 2091???? return >>> java_thread->cooked_allocated_bytes(); >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > 2092?? } >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > This looks right to me. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > - "ThreadAllocatedMemory is >>> expected to >>> ???? >????? >????? >????????? >> be disabled"); >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > +??????????????? "TEST FAILED: >>> ThreadAllocatedMemory is >>> ???? >????? >????? >????????? >> expected to be >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > disabled"); >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > Prepending "TEST FAILED" in >>> exception message (in >>> ???? >????? >????? >????????? >> several places) >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > seems redundant since such >>> RuntimeException is thrown >>> ???? >????? >????? >????????? >> and expected >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > a test failure. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > +??????? // back-to-back calls >>> shouldn't allocate any >>> ???? >????? >????? >????????? >> memory >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > +??????? size = >>> mbean.getThreadAllocatedBytes(id); >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > +??????? size1 = >>> mbean.getThreadAllocatedBytes(id); >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > +??????? if (size1 != size) { >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > Is there anything in the test can >>> do to help guarantee >>> ???? >????? >????? >????????? >> this? I didn't >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > closely review this test.? The >>> main thing I advice is >>> ???? >????? >????? >????????? >> to improve >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > the reliability of this test.? Put >>> it in another way, >>> ???? >????? >????? >????????? >> we want to >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > ensure that this test change will >>> pass all the time in >>> ???? >????? >????? >????????? >> various >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > test configuration. >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > Mandy >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> > >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? >????????? >> >>> ???? >????? >????? > >>> ???? >????? >????? > >>> ???? >????? >????? > >>> ???? >????? >????? > >>> ???? >????? > >>> ???? >????? > >>> ???? >????? > >>> ???? > >>> ???? > >>> ???? > >>> >> > From hohensee at amazon.com Thu Sep 19 01:05:40 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 19 Sep 2019 01:05:40 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <114ed65b-4b13-22aa-1849-f2737f7a9779@oracle.com> References: <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> <9A151A35-DA66-4B0F-B67F-E7F5BECF205A@amazon.com> <586EB17C-1974-4FEF-A751-34B5B359BA61@amazon.com> <6139732f-a914-c59c-2119-517e480a279c@oracle.com> <8E8AE26F-5A7B-499C-BB6A-5EC954E42BD7@amazon.com> <0d316e76-44cc-4f65-a96d-e1207b1c94e6@oracle.com> <114ed65b-4b13-22aa-1849-f2737f7a9779@oracle.com> Message-ID: <5DE94C8C-B398-48A4-AB0C-2CEE51A99DA0@amazon.com> +1, thanks! My apologies for the bad patch. I'll file another issue and run every test that mentions ThreadMXBean. At least, I know how to revert a patch now. Paul ?On 9/18/19, 6:00 PM, "David Holmes" wrote: Ship it! Thanks Dan! David On 19/09/2019 10:53 am, Daniel D. Daugherty wrote: > Looks like the issue is different versions of 'hg' in use. > > When I import Paul's patch from his webrev using my 'hg' and > then export it again, it matches my version of the backout. > > I have done a mechanical verification that the backout is an > exact reversal for Paul's original changeset. > > I'm planning to push the changeset with the following info: > > > 8231210: [BACKOUT] JDK-8207266 ThreadMXBean::getThreadAllocatedBytes() > can be quicker for self thread > Reviewed-by: phh, dholmes > > Everyone good with this? > > Dan > > On 9/18/19 8:44 PM, Daniel D. Daugherty wrote: >> For some reason, the backout that I did does not match the backout >> that you did so I'm trying to figure that out. >> >> Dan >> >> >> >> On 9/18/19 8:36 PM, Hohensee, Paul wrote: >>> And I filed 8231211 for the same thing. :) >>> >>> Yes, please handle it, because it will go faster since I don't have >>> access to a fast machine (just my laptop). >>> >>> Webrev here: >>> >>> http://cr.openjdk.java.net/~phh/8231211/webrev.00/ >>> >>> Thanks, >>> >>> On 9/18/19, 5:25 PM, "Daniel D. Daugherty" >>> wrote: >>> >>> I created this sub-task for you: >>> JDK-8231210 [BACKOUT] JDK-8207266 >>> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self >>> thread >>> https://bugs.openjdk.java.net/browse/JDK-8231210 >>> If you would prefer, I can handle this backout for you. >>> Dan >>> On 9/18/19 8:21 PM, Hohensee, Paul wrote: >>> > Never having done this before, is it >>> > >>> > hg backout -r >>> > >>> > ? Do I file a JBS issue for the reversion? Seems necessary. >>> > >>> > On 9/18/19, 5:18 PM, "Daniel D. Daugherty" >>> wrote: >>> > >>> > % hg backout >>> > >>> > is the usual way to do this... >>> > >>> > Dan >>> > >>> > >>> > On 9/18/19 8:17 PM, Hohensee, Paul wrote: >>> > > Is there a tool that will generate a reversal patch? >>> > > >>> > > On 9/18/19, 5:14 PM, "Daniel D. Daugherty" >>> wrote: >>> > > >>> > > > Shall I go with that, or reverse the original >>> patch? >>> > > >>> > > I'm a bit worried about what else might show up >>> since the >>> > > NSK monitoring tests were not run prior to this push. >>> > > >>> > > I vote for backing out the fix until proper >>> testing has >>> > > been done (and at least the one problem fixed...) >>> > > >>> > > Dan >>> > > >>> > > >>> > > On 9/18/19 8:00 PM, Hohensee, Paul wrote: >>> > > > They all implement >>> com.sun.management.ThreadMXBean, so adding a >>> getCurrentThreadAllocatedBytes broke them. Potential fix is to give >>> it a default implementation, vis >>> > > > >>> > > > public default long >>> getCurrentThreadAllocatedBytes() { >>> > > > return -1; >>> > > > } >>> > > > >>> > > > Shall I go with that, or reverse the original >>> patch? >>> > > > >>> > > > On 9/18/19, 4:48 PM, "serviceability-dev on >>> behalf of Hohensee, Paul" >>> >> hohensee at amazon.com> wrote: >>> > > > >>> > > > I'll take a look. >>> > > > >>> > > > On 9/18/19, 4:40 PM, "David Holmes" >>> wrote: >>> > > > >>> > > > Paul, >>> > > > >>> > > > Unfortunately this patch has broken the >>> vmTestbase/nsk/monitoring tests: >>> > > > >>> > > > [2019-09-18T22:59:32,349Z] >>> > > > >>> /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: >>> >>> > > > error: ServerThreadMXBeanNew is not >>> abstract and does not override >>> > > > abstract method >>> getCurrentThreadAllocatedBytes() in ThreadMXBean >>> > > > >>> > > > and possibly other issues as we are >>> seeing hundreds of failures. >>> > > > >>> > > > David >>> > > > >>> > > > On 18/09/2019 8:50 am, David Holmes wrote: >>> > > > > On 18/09/2019 12:10 am, Hohensee, >>> Paul wrote: >>> > > > >> Thanks, Serguei. :) >>> > > > >> >>> > > > >> David, are you ok with the patch? >>> > > > > >>> > > > > Yep, nothing further from me. >>> > > > > >>> > > > > David >>> > > > > >>> > > > >> Paul >>> > > > >> >>> > > > >> *From: *"serguei.spitsyn at oracle.com" >>> >>> > > > >> *Date: *Tuesday, September 17, 2019 >>> at 2:26 AM >>> > > > >> *To: *"Hohensee, Paul" >>> , David Holmes >>> > > > >> , Mandy >>> Chung >>> > > > >> *Cc: *OpenJDK Serviceability >>> , >>> > > > >> "hotspot-gc-dev at openjdk.java.net" >>> >>> > > > >> *Subject: *Re: RFR (M): 8207266: >>> > > > >> >>> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread >>> > > > >> >>> > > > >> Hi Paul, >>> > > > >> >>> > > > >> Thank you for refactoring and fixing >>> the test. >>> > > > >> It looks great now! >>> > > > >> >>> > > > >> Thanks, >>> > > > >> Serguei >>> > > > >> >>> > > > >> >>> > > > >> On 9/15/19 02:52, Hohensee, Paul wrote: >>> > > > >> >>> > > > >> Hi, Serguei, thanks for the >>> review. New webrev at >>> > > > >> >>> > > > >> >>> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ >>> > > > >> >>> > > > >> I refactored the test?s main() >>> method, and you?re correct, >>> > > > >> getThreadAllocatedBytes should be >>> getCurrentThreadAllocatedBytes in >>> > > > >> that context: fixed. >>> > > > >> >>> > > > >> Paul >>> > > > >> >>> > > > >> *From: >>> *"serguei.spitsyn at oracle.com" >>> > > > >> >>> >>> > > > >> >>> > > > >> *Organization: *Oracle Corporation >>> > > > >> *Date: *Friday, September 13, >>> 2019 at 5:50 PM >>> > > > >> *To: *"Hohensee, Paul" >>> >>> > > > >> , David >>> Holmes >>> > > > >> , >>> Mandy Chung >>> > > > >> >>> >>> > > > >> *Cc: *OpenJDK Serviceability >>> >>> > > > >> >>> , >>> > > > >> "hotspot-gc-dev at openjdk.java.net" >>> > > > >> >>> >>> > > > >> >>> > > > >> >>> >>> > > > >> *Subject: *Re: RFR (M): 8207266: >>> > > > >> >>> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self >>> > > > >> thread >>> > > > >> >>> > > > >> Hi Paul, >>> > > > >> >>> > > > >> It looks pretty good in general. >>> > > > >> >>> > > > >> >>> > > > >> >>> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html >>> >>> > > > >> >>> > > > >> >>> > > > >> It would be nice to refactor the >>> java main() method as it becomes >>> > > > >> too big. >>> > > > >> Two ways >>> ofgetCurrentThreadAllocatedBytes() testing are good >>> > > > >> candidates >>> > > > >> to become separate methods. >>> > > > >> >>> > > > >> 98 long size1 = >>> mbean.getThreadAllocatedBytes(id); >>> > > > >> >>> > > > >> Just wanted to double check if >>> you wanted to invoke >>> > > > >> the >>> getCurrentThreadAllocatedBytes() instead as it is >>> > > > >> a part of: >>> > > > >> >>> > > > >> 85 // First way, >>> getCurrentThreadAllocatedBytes >>> > > > >> >>> > > > >> >>> > > > >> Thanks, >>> > > > >> Serguei >>> > > > >> >>> > > > >> On 9/13/19 12:11 PM, Hohensee, >>> Paul wrote: >>> > > > >> >>> > > > >> Hi David, thanks for your >>> comments. New webrev in >>> > > > >> >>> > > > >> >>> > > > >> >>> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ >>> > > > >> >>> > > > >> >>> > > > >> Both the old and new >>> versions of the code check that thread >>> > > > >> allocated memory is both supported >>> and enabled. The existing version >>> > > > >> of getThreadAllocatedBytes(long []) >>> calls >>> > > > >> verifyThreadAllocatedMemory(long >>> []), which checks inline to make sure >>> > > > >> thread allocated memory is >>> supported, then calls >>> > > > >> isThreadAllocatedMemoryEnabled() to >>> verify that it's enabled. >>> > > > >> isThreadAllocatedMemoryEnabled() >>> duplicates (!) the support check and >>> > > > >> returns the enabled flag. I removed >>> the redundant check in the new >>> > > > >> version. >>> > > > >> >>> > > > >> >>> > > > >> You're of course correct >>> about the back-to-back check. >>> > > > >> Application code can't know when the >>> runtime will hijack a thread for >>> > > > >> its own purposes. I've removed the >>> check. >>> > > > >> >>> > > > >> >>> > > > >> Paul >>> > > > >> >>> > > > >> >>> > > > >> On 9/13/19, 12:50 AM, "David >>> Holmes" >>> > > > >> wrote: >>> > > > >> >>> > > > >> >>> > > > >> Hi Paul, >>> > > > >> >>> > > > >> >>> > > > >> On 13/09/2019 10:29 am, >>> Hohensee, Paul wrote: >>> > > > >> >>> > > > >> > Thanks for clarifying the review >>> rules. Would someone >>> > > > >> from the >>> > > > >> >>> > > > >> > serviceability team please review? >>> New webrev at >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> >>> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ >>> > > > >> >>> > > > >> >>> > > > >> One aspect of the >>> functional change needs clarification >>> > > > >> for me - and >>> > > > >> >>> > > > >> apologies if this has been covered >>> in the past. It seems >>> > > > >> to me that >>> > > > >> >>> > > > >> currently we only check >>> isThreadAllocatedMemorySupported >>> > > > >> for these >>> > > > >> >>> > > > >> operations, but if I read things >>> correctly the updated >>> > > > >> code additionally >>> > > > >> >>> > > > >> checks >>> isThreadAllocatedMemoryEnabled, which is a >>> > > > >> behaviour change not >>> > > > >> >>> > > > >> mentioned in the CSR. >>> > > > >> >>> > > > >> >>> > > > >> > I didn?t disturb the existing >>> checks in the test, just >>> > > > >> added code to >>> > > > >> >>> > > > >> > check the result of >>> getThreadAllocatedBytes(long) on a >>> > > > >> non-current >>> > > > >> >>> > > > >> > thread, plus the back-to-back >>> no-allocation checks. The >>> > > > >> former wasn?t >>> > > > >> >>> > > > >> > needed before because >>> getThreadAllocatedBytes(long) was >>> > > > >> just a wrapper >>> > > > >> >>> > > > >> > around >>> getThreadAllocatedBytes(long []). This patch >>> > > > >> changes that, so I >>> > > > >> >>> > > > >> > added a separate test. The latter >>> is supposed to fail >>> > > > >> if there?s object >>> > > > >> >>> > > > >> > allocation on calls to >>> getCurrentThreadAllocatedBytes and >>> > > > >> >>> > > > >> > getThreadAllocatedBytes(long). >>> I.e., a feature, not a >>> > > > >> bug, because >>> > > > >> >>> > > > >> > accumulation of transient small >>> objects can be a >>> > > > >> performance problem. >>> > > > >> >>> > > > >> > Thanks to your review, I noticed >>> that the back-to-back >>> > > > >> check on the >>> > > > >> >>> > > > >> > current thread was using >>> getThreadAllocatedBytes(long) >>> > > > >> instead of >>> > > > >> >>> > > > >> > getCurrentThreadAllocatedBytes and >>> fixed it. I also >>> > > > >> removed all >>> > > > >> >>> > > > >> > instances of ?TEST FAILED: ?. >>> > > > >> >>> > > > >> >>> > > > >> The back-to-back check >>> is not valid in general. You don't >>> > > > >> know if the >>> > > > >> >>> > > > >> first check might trigger some class >>> loading on the >>> > > > >> return path after it >>> > > > >> >>> > > > >> has obtained the first >>> memory value. The check might also >>> > > > >> fail if using >>> > > > >> >>> > > > >> JVMCI and some compilation related >>> activity occurs in the >>> > > > >> current thread >>> > > > >> >>> > > > >> on the second call. >>> Also with the introduction of >>> > > > >> handshakes its >>> > > > >> >>> > > > >> possible the current thread might >>> hit a safepoint checks >>> > > > >> that results in >>> > > > >> >>> > > > >> it executing a >>> handshake operation that performs >>> > > > >> allocation. Potentially >>> > > > >> >>> > > > >> there could be numerous >>> non-deterministic actions that >>> > > > >> might occur >>> > > > >> >>> > > > >> leading to unanticipated allocation. >>> > > > >> >>> > > > >> >>> > > > >> I understand what you >>> want to test here, I just don't >>> > > > >> think it is >>> > > > >> >>> > > > >> reliably doable. >>> > > > >> >>> > > > >> >>> > > > >> Thanks, >>> > > > >> >>> > > > >> David >>> > > > >> >>> > > > >> ----- >>> > > > >> >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > Paul >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > *From: *Mandy >>> Chung >>> > > > >> >>> > > > >> >>> > > > >> > *Date: *Thursday, September 12, >>> 2019 at 10:09 AM >>> > > > >> >>> > > > >> > *To: *"Hohensee, >>> Paul" >>> > > > >> >>> > > > >> >>> > > > >> > *Cc: *OpenJDK >>> > > > >> >>> Serviceability >>> > > > >> >>> , >>> > > > >> >>> > > > >> >"hotspot-gc-dev at openjdk.java.net" >>> > > > >> >>> >>> > > > >> >>> > > > >> >>> >>> > > > >> >>> > > > >> > *Subject: *Re: RFR (M): 8207266: >>> > > > >> ThreadMXBean::getThreadAllocatedBytes() >>> > > > >> >>> > > > >> > can be quicker for self thread >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > On 9/3/19 12:38 PM, Hohensee, Paul >>> wrote: >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > Minor update in new >>> > > > >> >>> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > I only reviewed the library side >>> implementation that >>> > > > >> looks good. I >>> > > > >> >>> > > > >> > expect the serviceability team to >>> review the test and >>> > > > >> hotspot change. >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > Need a confirmatory review to >>> push this. If I >>> > > > >> understand the rules correctly, it >>> doesn't need a Reviewer review >>> > > > >> since Mandy's already reviewed it, >>> it just needs a Committer review. >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > You need another reviewer to >>> advice the following >>> > > > >> because I was not >>> > > > >> >>> > > > >> > close to the ThreadsList work. >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > 2087 ThreadsListHandle tlh; >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > 2088 JavaThread* java_thread = >>> > > > >> >>> tlh.list()->find_JavaThread_from_java_tid(thread_id); >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > 2089 >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > 2090 if (java_thread != NULL) { >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > 2091 return >>> java_thread->cooked_allocated_bytes(); >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > 2092 } >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > This looks right to me. >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > >>> > > > >> >>> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > - "ThreadAllocatedMemory is >>> expected to >>> > > > >> be disabled"); >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > + "TEST FAILED: >>> ThreadAllocatedMemory is >>> > > > >> expected to be >>> > > > >> >>> > > > >> > disabled"); >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > Prepending "TEST FAILED" in >>> exception message (in >>> > > > >> several places) >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > seems redundant since such >>> RuntimeException is thrown >>> > > > >> and expected >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > a test failure. >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > + // back-to-back calls >>> shouldn't allocate any >>> > > > >> memory >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > + size = >>> mbean.getThreadAllocatedBytes(id); >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > + size1 = >>> mbean.getThreadAllocatedBytes(id); >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > + if (size1 != size) { >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > Is there anything in the test can >>> do to help guarantee >>> > > > >> this? I didn't >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > closely review this test. The >>> main thing I advice is >>> > > > >> to improve >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > the reliability of this test. Put >>> it in another way, >>> > > > >> we want to >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > ensure that this test change will >>> pass all the time in >>> > > > >> various >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > test configuration. >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> > Mandy >>> > > > >> >>> > > > >> > >>> > > > >> >>> > > > >> >>> > > > >> >>> > > > >> >>> > > > >> >>> > > > >>> > > > >>> > > > >>> > > > >>> > > >>> > > >>> > > >>> > >>> > >>> > >>> >> > From daniel.daugherty at oracle.com Thu Sep 19 01:08:11 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 18 Sep 2019 21:08:11 -0400 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <5DE94C8C-B398-48A4-AB0C-2CEE51A99DA0@amazon.com> References: <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> <9A151A35-DA66-4B0F-B67F-E7F5BECF205A@amazon.com> <586EB17C-1974-4FEF-A751-34B5B359BA61@amazon.com> <6139732f-a914-c59c-2119-517e480a279c@oracle.com> <8E8AE26F-5A7B-499C-BB6A-5EC954E42BD7@amazon.com> <0d316e76-44cc-4f65-a96d-e1207b1c94e6@oracle.com> <114ed65b-4b13-22aa-1849-f2737f7a9779@oracle.com> <5DE94C8C-B398-48A4-AB0C-2CEE51A99DA0@amazon.com> Message-ID: <35e2e812-5341-e193-7783-5d45ec77ca89@oracle.com> We should probably repurpose ??? JDK-8231209 Many com.sun.management.ThreadMXBean test failures after 8207266 as your REDO bug. Dan On 9/18/19 9:05 PM, Hohensee, Paul wrote: > +1, thanks! > > My apologies for the bad patch. I'll file another issue and run every test that mentions ThreadMXBean. At least, I know how to revert a patch now. > > Paul > > ?On 9/18/19, 6:00 PM, "David Holmes" wrote: > > Ship it! > > Thanks Dan! > > David > > On 19/09/2019 10:53 am, Daniel D. Daugherty wrote: > > Looks like the issue is different versions of 'hg' in use. > > > > When I import Paul's patch from his webrev using my 'hg' and > > then export it again, it matches my version of the backout. > > > > I have done a mechanical verification that the backout is an > > exact reversal for Paul's original changeset. > > > > I'm planning to push the changeset with the following info: > > > > > > 8231210: [BACKOUT] JDK-8207266 ThreadMXBean::getThreadAllocatedBytes() > > can be quicker for self thread > > Reviewed-by: phh, dholmes > > > > Everyone good with this? > > > > Dan > > > > On 9/18/19 8:44 PM, Daniel D. Daugherty wrote: > >> For some reason, the backout that I did does not match the backout > >> that you did so I'm trying to figure that out. > >> > >> Dan > >> > >> > >> > >> On 9/18/19 8:36 PM, Hohensee, Paul wrote: > >>> And I filed 8231211 for the same thing. :) > >>> > >>> Yes, please handle it, because it will go faster since I don't have > >>> access to a fast machine (just my laptop). > >>> > >>> Webrev here: > >>> > >>> http://cr.openjdk.java.net/~phh/8231211/webrev.00/ > >>> > >>> Thanks, > >>> > >>> On 9/18/19, 5:25 PM, "Daniel D. Daugherty" > >>> wrote: > >>> > >>> I created this sub-task for you: > >>> JDK-8231210 [BACKOUT] JDK-8207266 > >>> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self > >>> thread > >>> https://bugs.openjdk.java.net/browse/JDK-8231210 > >>> If you would prefer, I can handle this backout for you. > >>> Dan > >>> On 9/18/19 8:21 PM, Hohensee, Paul wrote: > >>> > Never having done this before, is it > >>> > > >>> > hg backout -r > >>> > > >>> > ? Do I file a JBS issue for the reversion? Seems necessary. > >>> > > >>> > On 9/18/19, 5:18 PM, "Daniel D. Daugherty" > >>> wrote: > >>> > > >>> > % hg backout > >>> > > >>> > is the usual way to do this... > >>> > > >>> > Dan > >>> > > >>> > > >>> > On 9/18/19 8:17 PM, Hohensee, Paul wrote: > >>> > > Is there a tool that will generate a reversal patch? > >>> > > > >>> > > On 9/18/19, 5:14 PM, "Daniel D. Daugherty" > >>> wrote: > >>> > > > >>> > > > Shall I go with that, or reverse the original > >>> patch? > >>> > > > >>> > > I'm a bit worried about what else might show up > >>> since the > >>> > > NSK monitoring tests were not run prior to this push. > >>> > > > >>> > > I vote for backing out the fix until proper > >>> testing has > >>> > > been done (and at least the one problem fixed...) > >>> > > > >>> > > Dan > >>> > > > >>> > > > >>> > > On 9/18/19 8:00 PM, Hohensee, Paul wrote: > >>> > > > They all implement > >>> com.sun.management.ThreadMXBean, so adding a > >>> getCurrentThreadAllocatedBytes broke them. Potential fix is to give > >>> it a default implementation, vis > >>> > > > > >>> > > > public default long > >>> getCurrentThreadAllocatedBytes() { > >>> > > > return -1; > >>> > > > } > >>> > > > > >>> > > > Shall I go with that, or reverse the original > >>> patch? > >>> > > > > >>> > > > On 9/18/19, 4:48 PM, "serviceability-dev on > >>> behalf of Hohensee, Paul" > >>> >>> hohensee at amazon.com> wrote: > >>> > > > > >>> > > > I'll take a look. > >>> > > > > >>> > > > On 9/18/19, 4:40 PM, "David Holmes" > >>> wrote: > >>> > > > > >>> > > > Paul, > >>> > > > > >>> > > > Unfortunately this patch has broken the > >>> vmTestbase/nsk/monitoring tests: > >>> > > > > >>> > > > [2019-09-18T22:59:32,349Z] > >>> > > > > >>> /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: > >>> > >>> > > > error: ServerThreadMXBeanNew is not > >>> abstract and does not override > >>> > > > abstract method > >>> getCurrentThreadAllocatedBytes() in ThreadMXBean > >>> > > > > >>> > > > and possibly other issues as we are > >>> seeing hundreds of failures. > >>> > > > > >>> > > > David > >>> > > > > >>> > > > On 18/09/2019 8:50 am, David Holmes wrote: > >>> > > > > On 18/09/2019 12:10 am, Hohensee, > >>> Paul wrote: > >>> > > > >> Thanks, Serguei. :) > >>> > > > >> > >>> > > > >> David, are you ok with the patch? > >>> > > > > > >>> > > > > Yep, nothing further from me. > >>> > > > > > >>> > > > > David > >>> > > > > > >>> > > > >> Paul > >>> > > > >> > >>> > > > >> *From: *"serguei.spitsyn at oracle.com" > >>> > >>> > > > >> *Date: *Tuesday, September 17, 2019 > >>> at 2:26 AM > >>> > > > >> *To: *"Hohensee, Paul" > >>> , David Holmes > >>> > > > >> , Mandy > >>> Chung > >>> > > > >> *Cc: *OpenJDK Serviceability > >>> , > >>> > > > >> "hotspot-gc-dev at openjdk.java.net" > >>> > >>> > > > >> *Subject: *Re: RFR (M): 8207266: > >>> > > > >> > >>> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread > >>> > > > >> > >>> > > > >> Hi Paul, > >>> > > > >> > >>> > > > >> Thank you for refactoring and fixing > >>> the test. > >>> > > > >> It looks great now! > >>> > > > >> > >>> > > > >> Thanks, > >>> > > > >> Serguei > >>> > > > >> > >>> > > > >> > >>> > > > >> On 9/15/19 02:52, Hohensee, Paul wrote: > >>> > > > >> > >>> > > > >> Hi, Serguei, thanks for the > >>> review. New webrev at > >>> > > > >> > >>> > > > >> > >>> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ > >>> > > > >> > >>> > > > >> I refactored the test?s main() > >>> method, and you?re correct, > >>> > > > >> getThreadAllocatedBytes should be > >>> getCurrentThreadAllocatedBytes in > >>> > > > >> that context: fixed. > >>> > > > >> > >>> > > > >> Paul > >>> > > > >> > >>> > > > >> *From: > >>> *"serguei.spitsyn at oracle.com" > >>> > > > >> > >>> > >>> > > > >> > >>> > > > >> *Organization: *Oracle Corporation > >>> > > > >> *Date: *Friday, September 13, > >>> 2019 at 5:50 PM > >>> > > > >> *To: *"Hohensee, Paul" > >>> > >>> > > > >> , David > >>> Holmes > >>> > > > >> , > >>> Mandy Chung > >>> > > > >> > >>> > >>> > > > >> *Cc: *OpenJDK Serviceability > >>> > >>> > > > >> > >>> , > >>> > > > >> "hotspot-gc-dev at openjdk.java.net" > >>> > > > >> > >>> > >>> > > > >> > >>> > > > >> > >>> > >>> > > > >> *Subject: *Re: RFR (M): 8207266: > >>> > > > >> > >>> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self > >>> > > > >> thread > >>> > > > >> > >>> > > > >> Hi Paul, > >>> > > > >> > >>> > > > >> It looks pretty good in general. > >>> > > > >> > >>> > > > >> > >>> > > > >> > >>> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html > >>> > >>> > > > >> > >>> > > > >> > >>> > > > >> It would be nice to refactor the > >>> java main() method as it becomes > >>> > > > >> too big. > >>> > > > >> Two ways > >>> ofgetCurrentThreadAllocatedBytes() testing are good > >>> > > > >> candidates > >>> > > > >> to become separate methods. > >>> > > > >> > >>> > > > >> 98 long size1 = > >>> mbean.getThreadAllocatedBytes(id); > >>> > > > >> > >>> > > > >> Just wanted to double check if > >>> you wanted to invoke > >>> > > > >> the > >>> getCurrentThreadAllocatedBytes() instead as it is > >>> > > > >> a part of: > >>> > > > >> > >>> > > > >> 85 // First way, > >>> getCurrentThreadAllocatedBytes > >>> > > > >> > >>> > > > >> > >>> > > > >> Thanks, > >>> > > > >> Serguei > >>> > > > >> > >>> > > > >> On 9/13/19 12:11 PM, Hohensee, > >>> Paul wrote: > >>> > > > >> > >>> > > > >> Hi David, thanks for your > >>> comments. New webrev in > >>> > > > >> > >>> > > > >> > >>> > > > >> > >>> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ > >>> > > > >> > >>> > > > >> > >>> > > > >> Both the old and new > >>> versions of the code check that thread > >>> > > > >> allocated memory is both supported > >>> and enabled. The existing version > >>> > > > >> of getThreadAllocatedBytes(long []) > >>> calls > >>> > > > >> verifyThreadAllocatedMemory(long > >>> []), which checks inline to make sure > >>> > > > >> thread allocated memory is > >>> supported, then calls > >>> > > > >> isThreadAllocatedMemoryEnabled() to > >>> verify that it's enabled. > >>> > > > >> isThreadAllocatedMemoryEnabled() > >>> duplicates (!) the support check and > >>> > > > >> returns the enabled flag. I removed > >>> the redundant check in the new > >>> > > > >> version. > >>> > > > >> > >>> > > > >> > >>> > > > >> You're of course correct > >>> about the back-to-back check. > >>> > > > >> Application code can't know when the > >>> runtime will hijack a thread for > >>> > > > >> its own purposes. I've removed the > >>> check. > >>> > > > >> > >>> > > > >> > >>> > > > >> Paul > >>> > > > >> > >>> > > > >> > >>> > > > >> On 9/13/19, 12:50 AM, "David > >>> Holmes" > >>> > > > >> wrote: > >>> > > > >> > >>> > > > >> > >>> > > > >> Hi Paul, > >>> > > > >> > >>> > > > >> > >>> > > > >> On 13/09/2019 10:29 am, > >>> Hohensee, Paul wrote: > >>> > > > >> > >>> > > > >> > Thanks for clarifying the review > >>> rules. Would someone > >>> > > > >> from the > >>> > > > >> > >>> > > > >> > serviceability team please review? > >>> New webrev at > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > >>> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ > >>> > > > >> > >>> > > > >> > >>> > > > >> One aspect of the > >>> functional change needs clarification > >>> > > > >> for me - and > >>> > > > >> > >>> > > > >> apologies if this has been covered > >>> in the past. It seems > >>> > > > >> to me that > >>> > > > >> > >>> > > > >> currently we only check > >>> isThreadAllocatedMemorySupported > >>> > > > >> for these > >>> > > > >> > >>> > > > >> operations, but if I read things > >>> correctly the updated > >>> > > > >> code additionally > >>> > > > >> > >>> > > > >> checks > >>> isThreadAllocatedMemoryEnabled, which is a > >>> > > > >> behaviour change not > >>> > > > >> > >>> > > > >> mentioned in the CSR. > >>> > > > >> > >>> > > > >> > >>> > > > >> > I didn?t disturb the existing > >>> checks in the test, just > >>> > > > >> added code to > >>> > > > >> > >>> > > > >> > check the result of > >>> getThreadAllocatedBytes(long) on a > >>> > > > >> non-current > >>> > > > >> > >>> > > > >> > thread, plus the back-to-back > >>> no-allocation checks. The > >>> > > > >> former wasn?t > >>> > > > >> > >>> > > > >> > needed before because > >>> getThreadAllocatedBytes(long) was > >>> > > > >> just a wrapper > >>> > > > >> > >>> > > > >> > around > >>> getThreadAllocatedBytes(long []). This patch > >>> > > > >> changes that, so I > >>> > > > >> > >>> > > > >> > added a separate test. The latter > >>> is supposed to fail > >>> > > > >> if there?s object > >>> > > > >> > >>> > > > >> > allocation on calls to > >>> getCurrentThreadAllocatedBytes and > >>> > > > >> > >>> > > > >> > getThreadAllocatedBytes(long). > >>> I.e., a feature, not a > >>> > > > >> bug, because > >>> > > > >> > >>> > > > >> > accumulation of transient small > >>> objects can be a > >>> > > > >> performance problem. > >>> > > > >> > >>> > > > >> > Thanks to your review, I noticed > >>> that the back-to-back > >>> > > > >> check on the > >>> > > > >> > >>> > > > >> > current thread was using > >>> getThreadAllocatedBytes(long) > >>> > > > >> instead of > >>> > > > >> > >>> > > > >> > getCurrentThreadAllocatedBytes and > >>> fixed it. I also > >>> > > > >> removed all > >>> > > > >> > >>> > > > >> > instances of ?TEST FAILED: ?. > >>> > > > >> > >>> > > > >> > >>> > > > >> The back-to-back check > >>> is not valid in general. You don't > >>> > > > >> know if the > >>> > > > >> > >>> > > > >> first check might trigger some class > >>> loading on the > >>> > > > >> return path after it > >>> > > > >> > >>> > > > >> has obtained the first > >>> memory value. The check might also > >>> > > > >> fail if using > >>> > > > >> > >>> > > > >> JVMCI and some compilation related > >>> activity occurs in the > >>> > > > >> current thread > >>> > > > >> > >>> > > > >> on the second call. > >>> Also with the introduction of > >>> > > > >> handshakes its > >>> > > > >> > >>> > > > >> possible the current thread might > >>> hit a safepoint checks > >>> > > > >> that results in > >>> > > > >> > >>> > > > >> it executing a > >>> handshake operation that performs > >>> > > > >> allocation. Potentially > >>> > > > >> > >>> > > > >> there could be numerous > >>> non-deterministic actions that > >>> > > > >> might occur > >>> > > > >> > >>> > > > >> leading to unanticipated allocation. > >>> > > > >> > >>> > > > >> > >>> > > > >> I understand what you > >>> want to test here, I just don't > >>> > > > >> think it is > >>> > > > >> > >>> > > > >> reliably doable. > >>> > > > >> > >>> > > > >> > >>> > > > >> Thanks, > >>> > > > >> > >>> > > > >> David > >>> > > > >> > >>> > > > >> ----- > >>> > > > >> > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > Paul > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > *From: *Mandy > >>> Chung > >>> > > > >> > >>> > > > >> > >>> > > > >> > *Date: *Thursday, September 12, > >>> 2019 at 10:09 AM > >>> > > > >> > >>> > > > >> > *To: *"Hohensee, > >>> Paul" > >>> > > > >> > >>> > > > >> > >>> > > > >> > *Cc: *OpenJDK > >>> > > > >> > >>> Serviceability > >>> > > > >> > >>> , > >>> > > > >> > >>> > > > >> >"hotspot-gc-dev at openjdk.java.net" > >>> > > > >> > >>> > >>> > > > >> > >>> > > > >> > >>> > >>> > > > >> > >>> > > > >> > *Subject: *Re: RFR (M): 8207266: > >>> > > > >> ThreadMXBean::getThreadAllocatedBytes() > >>> > > > >> > >>> > > > >> > can be quicker for self thread > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > On 9/3/19 12:38 PM, Hohensee, Paul > >>> wrote: > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > Minor update in new > >>> > > > >> > >>> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > I only reviewed the library side > >>> implementation that > >>> > > > >> looks good. I > >>> > > > >> > >>> > > > >> > expect the serviceability team to > >>> review the test and > >>> > > > >> hotspot change. > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > Need a confirmatory review to > >>> push this. If I > >>> > > > >> understand the rules correctly, it > >>> doesn't need a Reviewer review > >>> > > > >> since Mandy's already reviewed it, > >>> it just needs a Committer review. > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > You need another reviewer to > >>> advice the following > >>> > > > >> because I was not > >>> > > > >> > >>> > > > >> > close to the ThreadsList work. > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > 2087 ThreadsListHandle tlh; > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > 2088 JavaThread* java_thread = > >>> > > > >> > >>> tlh.list()->find_JavaThread_from_java_tid(thread_id); > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > 2089 > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > 2090 if (java_thread != NULL) { > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > 2091 return > >>> java_thread->cooked_allocated_bytes(); > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > 2092 } > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > This looks right to me. > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > - "ThreadAllocatedMemory is > >>> expected to > >>> > > > >> be disabled"); > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > + "TEST FAILED: > >>> ThreadAllocatedMemory is > >>> > > > >> expected to be > >>> > > > >> > >>> > > > >> > disabled"); > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > Prepending "TEST FAILED" in > >>> exception message (in > >>> > > > >> several places) > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > seems redundant since such > >>> RuntimeException is thrown > >>> > > > >> and expected > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > a test failure. > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > + // back-to-back calls > >>> shouldn't allocate any > >>> > > > >> memory > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > + size = > >>> mbean.getThreadAllocatedBytes(id); > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > + size1 = > >>> mbean.getThreadAllocatedBytes(id); > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > + if (size1 != size) { > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > Is there anything in the test can > >>> do to help guarantee > >>> > > > >> this? I didn't > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > closely review this test. The > >>> main thing I advice is > >>> > > > >> to improve > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > the reliability of this test. Put > >>> it in another way, > >>> > > > >> we want to > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > ensure that this test change will > >>> pass all the time in > >>> > > > >> various > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > test configuration. > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > Mandy > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > >>> > > > >> > >>> > > > >> > >>> > > > >> > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > >>> > > > >>> > > > >>> > > >>> > > >>> > > >>> > >> > > > > From hohensee at amazon.com Thu Sep 19 01:09:14 2019 From: hohensee at amazon.com (Hohensee, Paul) Date: Thu, 19 Sep 2019 01:09:14 +0000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <35e2e812-5341-e193-7783-5d45ec77ca89@oracle.com> References: <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> <64069096-23e3-efd5-44cd-2fff0618d28f@oracle.com> <9A151A35-DA66-4B0F-B67F-E7F5BECF205A@amazon.com> <586EB17C-1974-4FEF-A751-34B5B359BA61@amazon.com> <6139732f-a914-c59c-2119-517e480a279c@oracle.com> <8E8AE26F-5A7B-499C-BB6A-5EC954E42BD7@amazon.com> <0d316e76-44cc-4f65-a96d-e1207b1c94e6@oracle.com> <114ed65b-4b13-22aa-1849-f2737f7a9779@oracle.com> <5DE94C8C-B398-48A4-AB0C-2CEE51A99DA0@amazon.com> <35e2e812-5341-e193-7783-5d45ec77ca89@oracle.com> Message-ID: <3234DE48-C3C7-4E92-BE23-B3B638EB8BC4@amazon.com> Good idea, will do. ?On 9/18/19, 6:08 PM, "Daniel D. Daugherty" wrote: We should probably repurpose JDK-8231209 Many com.sun.management.ThreadMXBean test failures after 8207266 as your REDO bug. Dan On 9/18/19 9:05 PM, Hohensee, Paul wrote: > +1, thanks! > > My apologies for the bad patch. I'll file another issue and run every test that mentions ThreadMXBean. At least, I know how to revert a patch now. > > Paul > > On 9/18/19, 6:00 PM, "David Holmes" wrote: > > Ship it! > > Thanks Dan! > > David > > On 19/09/2019 10:53 am, Daniel D. Daugherty wrote: > > Looks like the issue is different versions of 'hg' in use. > > > > When I import Paul's patch from his webrev using my 'hg' and > > then export it again, it matches my version of the backout. > > > > I have done a mechanical verification that the backout is an > > exact reversal for Paul's original changeset. > > > > I'm planning to push the changeset with the following info: > > > > > > 8231210: [BACKOUT] JDK-8207266 ThreadMXBean::getThreadAllocatedBytes() > > can be quicker for self thread > > Reviewed-by: phh, dholmes > > > > Everyone good with this? > > > > Dan > > > > On 9/18/19 8:44 PM, Daniel D. Daugherty wrote: > >> For some reason, the backout that I did does not match the backout > >> that you did so I'm trying to figure that out. > >> > >> Dan > >> > >> > >> > >> On 9/18/19 8:36 PM, Hohensee, Paul wrote: > >>> And I filed 8231211 for the same thing. :) > >>> > >>> Yes, please handle it, because it will go faster since I don't have > >>> access to a fast machine (just my laptop). > >>> > >>> Webrev here: > >>> > >>> http://cr.openjdk.java.net/~phh/8231211/webrev.00/ > >>> > >>> Thanks, > >>> > >>> On 9/18/19, 5:25 PM, "Daniel D. Daugherty" > >>> wrote: > >>> > >>> I created this sub-task for you: > >>> JDK-8231210 [BACKOUT] JDK-8207266 > >>> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self > >>> thread > >>> https://bugs.openjdk.java.net/browse/JDK-8231210 > >>> If you would prefer, I can handle this backout for you. > >>> Dan > >>> On 9/18/19 8:21 PM, Hohensee, Paul wrote: > >>> > Never having done this before, is it > >>> > > >>> > hg backout -r > >>> > > >>> > ? Do I file a JBS issue for the reversion? Seems necessary. > >>> > > >>> > On 9/18/19, 5:18 PM, "Daniel D. Daugherty" > >>> wrote: > >>> > > >>> > % hg backout > >>> > > >>> > is the usual way to do this... > >>> > > >>> > Dan > >>> > > >>> > > >>> > On 9/18/19 8:17 PM, Hohensee, Paul wrote: > >>> > > Is there a tool that will generate a reversal patch? > >>> > > > >>> > > On 9/18/19, 5:14 PM, "Daniel D. Daugherty" > >>> wrote: > >>> > > > >>> > > > Shall I go with that, or reverse the original > >>> patch? > >>> > > > >>> > > I'm a bit worried about what else might show up > >>> since the > >>> > > NSK monitoring tests were not run prior to this push. > >>> > > > >>> > > I vote for backing out the fix until proper > >>> testing has > >>> > > been done (and at least the one problem fixed...) > >>> > > > >>> > > Dan > >>> > > > >>> > > > >>> > > On 9/18/19 8:00 PM, Hohensee, Paul wrote: > >>> > > > They all implement > >>> com.sun.management.ThreadMXBean, so adding a > >>> getCurrentThreadAllocatedBytes broke them. Potential fix is to give > >>> it a default implementation, vis > >>> > > > > >>> > > > public default long > >>> getCurrentThreadAllocatedBytes() { > >>> > > > return -1; > >>> > > > } > >>> > > > > >>> > > > Shall I go with that, or reverse the original > >>> patch? > >>> > > > > >>> > > > On 9/18/19, 4:48 PM, "serviceability-dev on > >>> behalf of Hohensee, Paul" > >>> >>> hohensee at amazon.com> wrote: > >>> > > > > >>> > > > I'll take a look. > >>> > > > > >>> > > > On 9/18/19, 4:40 PM, "David Holmes" > >>> wrote: > >>> > > > > >>> > > > Paul, > >>> > > > > >>> > > > Unfortunately this patch has broken the > >>> vmTestbase/nsk/monitoring tests: > >>> > > > > >>> > > > [2019-09-18T22:59:32,349Z] > >>> > > > > >>> /scratch/mesos/jib-master/install/jdk-14+15-615/src.full/open/test/hotspot/jtreg/vmTestbase/nsk/monitoring/share/server/ServerThreadMXBeanNew.java:32: > >>> > >>> > > > error: ServerThreadMXBeanNew is not > >>> abstract and does not override > >>> > > > abstract method > >>> getCurrentThreadAllocatedBytes() in ThreadMXBean > >>> > > > > >>> > > > and possibly other issues as we are > >>> seeing hundreds of failures. > >>> > > > > >>> > > > David > >>> > > > > >>> > > > On 18/09/2019 8:50 am, David Holmes wrote: > >>> > > > > On 18/09/2019 12:10 am, Hohensee, > >>> Paul wrote: > >>> > > > >> Thanks, Serguei. :) > >>> > > > >> > >>> > > > >> David, are you ok with the patch? > >>> > > > > > >>> > > > > Yep, nothing further from me. > >>> > > > > > >>> > > > > David > >>> > > > > > >>> > > > >> Paul > >>> > > > >> > >>> > > > >> *From: *"serguei.spitsyn at oracle.com" > >>> > >>> > > > >> *Date: *Tuesday, September 17, 2019 > >>> at 2:26 AM > >>> > > > >> *To: *"Hohensee, Paul" > >>> , David Holmes > >>> > > > >> , Mandy > >>> Chung > >>> > > > >> *Cc: *OpenJDK Serviceability > >>> , > >>> > > > >> "hotspot-gc-dev at openjdk.java.net" > >>> > >>> > > > >> *Subject: *Re: RFR (M): 8207266: > >>> > > > >> > >>> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread > >>> > > > >> > >>> > > > >> Hi Paul, > >>> > > > >> > >>> > > > >> Thank you for refactoring and fixing > >>> the test. > >>> > > > >> It looks great now! > >>> > > > >> > >>> > > > >> Thanks, > >>> > > > >> Serguei > >>> > > > >> > >>> > > > >> > >>> > > > >> On 9/15/19 02:52, Hohensee, Paul wrote: > >>> > > > >> > >>> > > > >> Hi, Serguei, thanks for the > >>> review. New webrev at > >>> > > > >> > >>> > > > >> > >>> http://cr.openjdk.java.net/~phh/8207266/webrev.09/ > >>> > > > >> > >>> > > > >> I refactored the test?s main() > >>> method, and you?re correct, > >>> > > > >> getThreadAllocatedBytes should be > >>> getCurrentThreadAllocatedBytes in > >>> > > > >> that context: fixed. > >>> > > > >> > >>> > > > >> Paul > >>> > > > >> > >>> > > > >> *From: > >>> *"serguei.spitsyn at oracle.com" > >>> > > > >> > >>> > >>> > > > >> > >>> > > > >> *Organization: *Oracle Corporation > >>> > > > >> *Date: *Friday, September 13, > >>> 2019 at 5:50 PM > >>> > > > >> *To: *"Hohensee, Paul" > >>> > >>> > > > >> , David > >>> Holmes > >>> > > > >> , > >>> Mandy Chung > >>> > > > >> > >>> > >>> > > > >> *Cc: *OpenJDK Serviceability > >>> > >>> > > > >> > >>> , > >>> > > > >> "hotspot-gc-dev at openjdk.java.net" > >>> > > > >> > >>> > >>> > > > >> > >>> > > > >> > >>> > >>> > > > >> *Subject: *Re: RFR (M): 8207266: > >>> > > > >> > >>> ThreadMXBean::getThreadAllocatedBytes() can be quicker for self > >>> > > > >> thread > >>> > > > >> > >>> > > > >> Hi Paul, > >>> > > > >> > >>> > > > >> It looks pretty good in general. > >>> > > > >> > >>> > > > >> > >>> > > > >> > >>> http://cr.openjdk.java.net/~phh/8207266/webrev.08/test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java.frames.html > >>> > >>> > > > >> > >>> > > > >> > >>> > > > >> It would be nice to refactor the > >>> java main() method as it becomes > >>> > > > >> too big. > >>> > > > >> Two ways > >>> ofgetCurrentThreadAllocatedBytes() testing are good > >>> > > > >> candidates > >>> > > > >> to become separate methods. > >>> > > > >> > >>> > > > >> 98 long size1 = > >>> mbean.getThreadAllocatedBytes(id); > >>> > > > >> > >>> > > > >> Just wanted to double check if > >>> you wanted to invoke > >>> > > > >> the > >>> getCurrentThreadAllocatedBytes() instead as it is > >>> > > > >> a part of: > >>> > > > >> > >>> > > > >> 85 // First way, > >>> getCurrentThreadAllocatedBytes > >>> > > > >> > >>> > > > >> > >>> > > > >> Thanks, > >>> > > > >> Serguei > >>> > > > >> > >>> > > > >> On 9/13/19 12:11 PM, Hohensee, > >>> Paul wrote: > >>> > > > >> > >>> > > > >> Hi David, thanks for your > >>> comments. New webrev in > >>> > > > >> > >>> > > > >> > >>> > > > >> > >>> http://cr.openjdk.java.net/~phh/8207266/webrev.08/ > >>> > > > >> > >>> > > > >> > >>> > > > >> Both the old and new > >>> versions of the code check that thread > >>> > > > >> allocated memory is both supported > >>> and enabled. The existing version > >>> > > > >> of getThreadAllocatedBytes(long []) > >>> calls > >>> > > > >> verifyThreadAllocatedMemory(long > >>> []), which checks inline to make sure > >>> > > > >> thread allocated memory is > >>> supported, then calls > >>> > > > >> isThreadAllocatedMemoryEnabled() to > >>> verify that it's enabled. > >>> > > > >> isThreadAllocatedMemoryEnabled() > >>> duplicates (!) the support check and > >>> > > > >> returns the enabled flag. I removed > >>> the redundant check in the new > >>> > > > >> version. > >>> > > > >> > >>> > > > >> > >>> > > > >> You're of course correct > >>> about the back-to-back check. > >>> > > > >> Application code can't know when the > >>> runtime will hijack a thread for > >>> > > > >> its own purposes. I've removed the > >>> check. > >>> > > > >> > >>> > > > >> > >>> > > > >> Paul > >>> > > > >> > >>> > > > >> > >>> > > > >> On 9/13/19, 12:50 AM, "David > >>> Holmes" > >>> > > > >> wrote: > >>> > > > >> > >>> > > > >> > >>> > > > >> Hi Paul, > >>> > > > >> > >>> > > > >> > >>> > > > >> On 13/09/2019 10:29 am, > >>> Hohensee, Paul wrote: > >>> > > > >> > >>> > > > >> > Thanks for clarifying the review > >>> rules. Would someone > >>> > > > >> from the > >>> > > > >> > >>> > > > >> > serviceability team please review? > >>> New webrev at > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > >>> >http://cr.openjdk.java.net/~phh/8207266/webrev.07/ > >>> > > > >> > >>> > > > >> > >>> > > > >> One aspect of the > >>> functional change needs clarification > >>> > > > >> for me - and > >>> > > > >> > >>> > > > >> apologies if this has been covered > >>> in the past. It seems > >>> > > > >> to me that > >>> > > > >> > >>> > > > >> currently we only check > >>> isThreadAllocatedMemorySupported > >>> > > > >> for these > >>> > > > >> > >>> > > > >> operations, but if I read things > >>> correctly the updated > >>> > > > >> code additionally > >>> > > > >> > >>> > > > >> checks > >>> isThreadAllocatedMemoryEnabled, which is a > >>> > > > >> behaviour change not > >>> > > > >> > >>> > > > >> mentioned in the CSR. > >>> > > > >> > >>> > > > >> > >>> > > > >> > I didn?t disturb the existing > >>> checks in the test, just > >>> > > > >> added code to > >>> > > > >> > >>> > > > >> > check the result of > >>> getThreadAllocatedBytes(long) on a > >>> > > > >> non-current > >>> > > > >> > >>> > > > >> > thread, plus the back-to-back > >>> no-allocation checks. The > >>> > > > >> former wasn?t > >>> > > > >> > >>> > > > >> > needed before because > >>> getThreadAllocatedBytes(long) was > >>> > > > >> just a wrapper > >>> > > > >> > >>> > > > >> > around > >>> getThreadAllocatedBytes(long []). This patch > >>> > > > >> changes that, so I > >>> > > > >> > >>> > > > >> > added a separate test. The latter > >>> is supposed to fail > >>> > > > >> if there?s object > >>> > > > >> > >>> > > > >> > allocation on calls to > >>> getCurrentThreadAllocatedBytes and > >>> > > > >> > >>> > > > >> > getThreadAllocatedBytes(long). > >>> I.e., a feature, not a > >>> > > > >> bug, because > >>> > > > >> > >>> > > > >> > accumulation of transient small > >>> objects can be a > >>> > > > >> performance problem. > >>> > > > >> > >>> > > > >> > Thanks to your review, I noticed > >>> that the back-to-back > >>> > > > >> check on the > >>> > > > >> > >>> > > > >> > current thread was using > >>> getThreadAllocatedBytes(long) > >>> > > > >> instead of > >>> > > > >> > >>> > > > >> > getCurrentThreadAllocatedBytes and > >>> fixed it. I also > >>> > > > >> removed all > >>> > > > >> > >>> > > > >> > instances of ?TEST FAILED: ?. > >>> > > > >> > >>> > > > >> > >>> > > > >> The back-to-back check > >>> is not valid in general. You don't > >>> > > > >> know if the > >>> > > > >> > >>> > > > >> first check might trigger some class > >>> loading on the > >>> > > > >> return path after it > >>> > > > >> > >>> > > > >> has obtained the first > >>> memory value. The check might also > >>> > > > >> fail if using > >>> > > > >> > >>> > > > >> JVMCI and some compilation related > >>> activity occurs in the > >>> > > > >> current thread > >>> > > > >> > >>> > > > >> on the second call. > >>> Also with the introduction of > >>> > > > >> handshakes its > >>> > > > >> > >>> > > > >> possible the current thread might > >>> hit a safepoint checks > >>> > > > >> that results in > >>> > > > >> > >>> > > > >> it executing a > >>> handshake operation that performs > >>> > > > >> allocation. Potentially > >>> > > > >> > >>> > > > >> there could be numerous > >>> non-deterministic actions that > >>> > > > >> might occur > >>> > > > >> > >>> > > > >> leading to unanticipated allocation. > >>> > > > >> > >>> > > > >> > >>> > > > >> I understand what you > >>> want to test here, I just don't > >>> > > > >> think it is > >>> > > > >> > >>> > > > >> reliably doable. > >>> > > > >> > >>> > > > >> > >>> > > > >> Thanks, > >>> > > > >> > >>> > > > >> David > >>> > > > >> > >>> > > > >> ----- > >>> > > > >> > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > Paul > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > *From: *Mandy > >>> Chung > >>> > > > >> > >>> > > > >> > >>> > > > >> > *Date: *Thursday, September 12, > >>> 2019 at 10:09 AM > >>> > > > >> > >>> > > > >> > *To: *"Hohensee, > >>> Paul" > >>> > > > >> > >>> > > > >> > >>> > > > >> > *Cc: *OpenJDK > >>> > > > >> > >>> Serviceability > >>> > > > >> > >>> , > >>> > > > >> > >>> > > > >> >"hotspot-gc-dev at openjdk.java.net" > >>> > > > >> > >>> > >>> > > > >> > >>> > > > >> > >>> > >>> > > > >> > >>> > > > >> > *Subject: *Re: RFR (M): 8207266: > >>> > > > >> ThreadMXBean::getThreadAllocatedBytes() > >>> > > > >> > >>> > > > >> > can be quicker for self thread > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > On 9/3/19 12:38 PM, Hohensee, Paul > >>> wrote: > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > Minor update in new > >>> > > > >> > >>> webrevhttp://cr.openjdk.java.net/~phh/8207266/webrev.05/. > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > I only reviewed the library side > >>> implementation that > >>> > > > >> looks good. I > >>> > > > >> > >>> > > > >> > expect the serviceability team to > >>> review the test and > >>> > > > >> hotspot change. > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > Need a confirmatory review to > >>> push this. If I > >>> > > > >> understand the rules correctly, it > >>> doesn't need a Reviewer review > >>> > > > >> since Mandy's already reviewed it, > >>> it just needs a Committer review. > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > You need another reviewer to > >>> advice the following > >>> > > > >> because I was not > >>> > > > >> > >>> > > > >> > close to the ThreadsList work. > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > 2087 ThreadsListHandle tlh; > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > 2088 JavaThread* java_thread = > >>> > > > >> > >>> tlh.list()->find_JavaThread_from_java_tid(thread_id); > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > 2089 > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > 2090 if (java_thread != NULL) { > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > 2091 return > >>> java_thread->cooked_allocated_bytes(); > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > 2092 } > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > This looks right to me. > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> test/jdk/com/sun/management/ThreadMXBean/ThreadAllocatedMemory.java > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > - "ThreadAllocatedMemory is > >>> expected to > >>> > > > >> be disabled"); > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > + "TEST FAILED: > >>> ThreadAllocatedMemory is > >>> > > > >> expected to be > >>> > > > >> > >>> > > > >> > disabled"); > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > Prepending "TEST FAILED" in > >>> exception message (in > >>> > > > >> several places) > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > seems redundant since such > >>> RuntimeException is thrown > >>> > > > >> and expected > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > a test failure. > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > + // back-to-back calls > >>> shouldn't allocate any > >>> > > > >> memory > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > + size = > >>> mbean.getThreadAllocatedBytes(id); > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > + size1 = > >>> mbean.getThreadAllocatedBytes(id); > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > + if (size1 != size) { > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > Is there anything in the test can > >>> do to help guarantee > >>> > > > >> this? I didn't > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > closely review this test. The > >>> main thing I advice is > >>> > > > >> to improve > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > the reliability of this test. Put > >>> it in another way, > >>> > > > >> we want to > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > ensure that this test change will > >>> pass all the time in > >>> > > > >> various > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > test configuration. > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > Mandy > >>> > > > >> > >>> > > > >> > > >>> > > > >> > >>> > > > >> > >>> > > > >> > >>> > > > >> > >>> > > > >> > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > >>> > > > >>> > > > >>> > > >>> > > >>> > > >>> > >> > > > > From mandy.chung at oracle.com Thu Sep 19 02:57:18 2019 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 18 Sep 2019 19:57:18 -0700 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> Message-ID: On 9/18/19 5:00 PM, Hohensee, Paul wrote: > They all implement com.sun.management.ThreadMXBean, so adding a getCurrentThreadAllocatedBytes broke them. Potential fix is to give it a default implementation, vis > > public default long getCurrentThreadAllocatedBytes() { > return -1; > } > com.sun.management.ThreadMXBean (and other platform MXBeans) is a "sealed" interface which should only be implemented by JDK. Unfortunately we don't have the sealed type feature yet.? Yes it needs to be a default method.? I think it should throw UOE. ???? * @implSpec ???? * The default implementation throws {@code UnsupportedOperationException}. The @throw UOE can make it clear that it does not support current thread memory allocation measurement. Mandy From david.holmes at oracle.com Thu Sep 19 03:15:39 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 19 Sep 2019 13:15:39 +1000 Subject: RFR (M): 8207266: ThreadMXBean::getThreadAllocatedBytes() can be quicker for self thread In-Reply-To: References: <588a91ec-8d4a-1157-5d72-88bb1eef1e6e@oracle.com> <30EA5D0C-1AEC-4242-B17B-CA4D39ECAF71@amazon.com> <0d42d653-d158-a6e4-45b6-84f087c7e592@oracle.com> <03A2509C-5587-448A-82F8-9240EA040326@amazon.com> <6f674d71-58f6-bc79-7d08-7bcc24e3b0fa@oracle.com> <5252a51d-4217-000b-1444-a088bb8a6a58@oracle.com> <873119A8-C595-4B73-AD0B-1625D6CAC47D@amazon.com> <56ea7c5f-8c91-9a05-6d95-255bfd0c154d@oracle.com> <5417BEA4-AECD-4130-B269-19847C0092B3@amazon.com> <1561d09b-68ff-55fa-128a-045798a3d6a9@oracle.com> <6732ee43-532f-377d-f37c-fe4e0f9becfe@oracle.com> <93FFE1B3-C1BA-4568-9402-48EB74BB089B@amazon.com> Message-ID: On 19/09/2019 12:57 pm, Mandy Chung wrote: > On 9/18/19 5:00 PM, Hohensee, Paul wrote: >> They all implement com.sun.management.ThreadMXBean, so adding a >> getCurrentThreadAllocatedBytes broke them. Potential fix is to give it >> a default implementation, vis >> >> ???? public default long getCurrentThreadAllocatedBytes() { >> ???????? return -1; >> ???? } >> > > com.sun.management.ThreadMXBean (and other platform MXBeans) is a > "sealed" interface which should only be implemented by JDK. Didn't realize that. I don't recall knowing about PlatformManagedObject. Sealed types will at least allow this to be enforced, though I have to wonder what the tests are doing here. > Unfortunately we don't have the sealed type feature yet.? Yes it needs > to be a default method.? I think it should throw UOE. > > ???? * @implSpec > ???? * The default implementation throws {@code > UnsupportedOperationException}. > > The @throw UOE can make it clear that it does not support current thread > memory allocation measurement. Yes that seems a reasonable default if we don't want this to be implemented outside the platform. Thanks, David > Mandy From thomas.schatzl at oracle.com Thu Sep 19 09:42:46 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 19 Sep 2019 11:42:46 +0200 Subject: RFR (M): 8159984: Remove call to ClassLoaderDataGraph::clear_claimed_marks during the initial mark pause In-Reply-To: References: <4e851c12-ca88-5d2d-f636-767437ad5a01@oracle.com> Message-ID: <52dd1037-00b6-86a6-0222-3a7807c5d2c9@oracle.com> Hi Kim, thanks for your review. On 18.09.19 20:36, Kim Barrett wrote: >> On Sep 18, 2019, at 11:22 AM, Thomas Schatzl wrote: >> >> Hi all, >> >> can I have reviews for this change that, given the observation that during root processing, there is only one kind of root that walks CLDs, we do not need to >> >> [...] >> >> (based on JDK-8231117) >> >> Thanks, >> Thomas > > I was just looking at the root scanning code recently, trying to > figure out what worker_has_discovered_all_strong_classes() was about. > > The naming of worker_has_discovered_all_strong_classes() and > wait_until_all_strong_classes_discovered() seems wrong now. These are > about discovering/processing strong nmethods now. > > A better comment at the call to the wait function about why it's > called there would be helpful. Obviously we want it as late as > possible, to allow threads to get past the notification. But why not > later? We could at the latest do the waiting when we do the nmethod weak root scanning. Given that I think I now have a working prototype that works without the barrier I did not care optimizing that for this change. If there are any unforeseen issues, or it does not pan out, I can improve that (in a separate CR if needed). I fixed the naming of the two methods and tried to improve the comment in a new webrev. New webrevs: http://cr.openjdk.java.net/~tschatzl/8159984/webrev.0_to_1/ (diff) http://cr.openjdk.java.net/~tschatzl/8159984/webrev.1/ (full) > (We talked about this off-line; this is just a reminder and > request for some commentary capturing that discussion. Though > hopefully your idea for getting rid of the barrier completly will pan > out. > > Other than that, looks good. > Thanks! Thomas From huangjia at loongson.cn Thu Sep 19 10:56:56 2019 From: huangjia at loongson.cn (huangjia) Date: Thu, 19 Sep 2019 18:56:56 +0800 Subject: RFR: 8231242: G1CollectedHeap::print_regions_on lost description for "OA" and "CA" Message-ID: Hi all, JBS: https://bugs.openjdk.java.net/browse/JDK-8231242 When dumping the info of heap region, the description for "OA" and "CA" seems to be lost. Please review the following fix which improves the readability of the heap region log. ----------------------------------- diff -r 778fc2dcbdaa src/hotspot/share/gc/g1/g1CollectedHeap.cpp --- a/src/hotspot/share/gc/g1/g1CollectedHeap.cpp Wed Sep 18 20:49:13 2019 -0400 +++ b/src/hotspot/share/gc/g1/g1CollectedHeap.cpp Thu Sep 19 11:47:07 2019 +0800 @@ -2378,6 +2378,7 @@ st->print_cr("Heap Regions: E=young(eden), S=young(survivor), O=old, " "HS=humongous(starts), HC=humongous(continues), " "CS=collection set, F=free, A=archive, " + "OA=open archive, CA=closed archive, " "TAMS=top-at-mark-start (previous, next)"); PrintRegionClosure blk(st); heap_region_iterate(&blk); ----------------------------------- For more info, please see the JBS. Testing: - make test TEST="jtreg:hotspot_gc" CONF=release - make test TEST="jtreg:hotspot_gc" CONF=fastdebug Thanks a lot Best regards, Jia From per.liden at oracle.com Thu Sep 19 11:23:50 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 19 Sep 2019 13:23:50 +0200 Subject: RFR: 8231113: Adjust CollectedHeap::check_oop_location() In-Reply-To: References: Message-ID: Hi Stefan, On 2019-09-17 21:48, Stefan Karlsson wrote: > Hi Per, > > I think both names are bad because the functions return false positives > but the names give the impression that they won't. For example: > > ?? // CMS forwards some non-heap value into the mark oop to reserve > oops during > ?? // promotion, so we can't assert about obj alignment or that the > forwardee is in heap > - virtual void check_oop_location(void* addr) const {} > + virtual bool is_oop_location(void* addr) const { return true; } > > It's obvious that this function returns true for pointers outside of the > heap. Other implementations are not as bad, but they also return false > positives. Users can't use is_oop_location to drive code logic, only to > do sanity checks. > > The check_oop_location functions didn't have that problem, but they > couldn't be used in asserts. Did you consider changing the > check_oop_location functions to return a bool instead of asserting? > > ?? G1CollectedHeap* g1h = G1CollectedHeap::heap(); > ?? // can't do because of races > ?? // assert(oopDesc::is_oop_or_null(obj), "expected an oop"); > - g1h->check_oop_location(obj); > + assert(g1h->check_oop_location(obj), "invalid oop location"); > > I think that would be less misleading to readers of the code. I see your point. After looking more at how we use check_oop_location() today, I'm more and more thinking that it's just the wrong abstraction. In CMS we're "checking oops" that we know aren't oops, so we're forced to disable check_oop_location() completely. The checks done for filler objects is sort of strange too. We're filling a space outside of the allocated part of a TLAB/region, so it becomes a philosophical discussion, where this is no "correct" answer, whether that is a valid oop location or not. I actually think I'd prefer removing check_oop_location() all together, with the rational that some of the asserts where it's used today are "conceptually wrong". In the other cases (e.g. in CompressedOops and G1), we can make valid checks using better mechanisms. So, how about this: http://cr.openjdk.java.net/~pliden/8231113/webrev.1 Passed tier1-3. cheers, Per > > Thanks, > StefanK > > On 2019-09-17 15:02, Per Liden wrote: >> I propose we adjust and rename CollectedHeap::check_oop_location() >> into CollectedHeap::is_oop_location() that returns a bool, to better >> match its sibling CollectedHeap::is_oop() and how that is used. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231113 >> Webrev: http://cr.openjdk.java.net/~pliden/8231113/webrev.0 >> >> /Per > From thomas.schatzl at oracle.com Thu Sep 19 11:24:17 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 19 Sep 2019 13:24:17 +0200 Subject: RFR: 8231242: G1CollectedHeap::print_regions_on lost description for "OA" and "CA" In-Reply-To: References: Message-ID: Hi, On Thu, 2019-09-19 at 18:56 +0800, huangjia wrote: > Hi all, > > JBS: https://bugs.openjdk.java.net/browse/JDK-8231242 > > When dumping the info of heap region, the description for "OA" and > "CA" seems to be lost. > Please review the following fix which improves the readability of > the heap region log. > ----------------------------------- > diff -r 778fc2dcbdaa src/hotspot/share/gc/g1/g1CollectedHeap.cpp > --- a/src/hotspot/share/gc/g1/g1CollectedHeap.cpp Wed Sep 18 > 20:49:13 2019 -0400 > +++ b/src/hotspot/share/gc/g1/g1CollectedHeap.cpp Thu Sep 19 > 11:47:07 2019 +0800 > @@ -2378,6 +2378,7 @@ > st->print_cr("Heap Regions: E=young(eden), S=young(survivor), > O=old, " > "HS=humongous(starts), HC=humongous(continues), " > "CS=collection set, F=free, A=archive, " > + "OA=open archive, CA=closed archive, " > "TAMS=top-at-mark-start (previous, next)"); The fix should also remove the "A=archive" description - there will afaik no plain "A" regions any more. Thanks, Thomas From stefan.karlsson at oracle.com Thu Sep 19 11:33:33 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 19 Sep 2019 13:33:33 +0200 Subject: RFR: 8231113: Adjust CollectedHeap::check_oop_location() In-Reply-To: References: Message-ID: <17dc202b-c3b9-eeef-24d6-60291e352833@oracle.com> On 2019-09-19 13:23, Per Liden wrote: > Hi Stefan, > > On 2019-09-17 21:48, Stefan Karlsson wrote: >> Hi Per, >> >> I think both names are bad because the functions return false >> positives but the names give the impression that they won't. For example: >> >> ??? // CMS forwards some non-heap value into the mark oop to reserve >> oops during >> ??? // promotion, so we can't assert about obj alignment or that the >> forwardee is in heap >> - virtual void check_oop_location(void* addr) const {} >> + virtual bool is_oop_location(void* addr) const { return true; } >> >> It's obvious that this function returns true for pointers outside of >> the heap. Other implementations are not as bad, but they also return >> false positives. Users can't use is_oop_location to drive code logic, >> only to do sanity checks. >> >> The check_oop_location functions didn't have that problem, but they >> couldn't be used in asserts. Did you consider changing the >> check_oop_location functions to return a bool instead of asserting? >> >> ??? G1CollectedHeap* g1h = G1CollectedHeap::heap(); >> ??? // can't do because of races >> ??? // assert(oopDesc::is_oop_or_null(obj), "expected an oop"); >> - g1h->check_oop_location(obj); >> + assert(g1h->check_oop_location(obj), "invalid oop location"); >> >> I think that would be less misleading to readers of the code. > > I see your point. After looking more at how we use check_oop_location() > today, I'm more and more thinking that it's just the wrong abstraction. > In CMS we're "checking oops" that we know aren't oops, so we're forced > to disable check_oop_location() completely. The checks done for filler > objects is sort of strange too. We're filling a space outside of the > allocated part of a TLAB/region, so it becomes a philosophical > discussion, where this is no "correct" answer, whether that is a valid > oop location or not. > > I actually think I'd prefer removing check_oop_location() all together, > with the rational that some of the asserts where it's used today are > "conceptually wrong". In the other cases (e.g. in CompressedOops and > G1), we can make valid checks using better mechanisms. > > So, how about this: > > http://cr.openjdk.java.net/~pliden/8231113/webrev.1 Looks good to me! StefanK > > Passed tier1-3. > > cheers, > Per > >> >> Thanks, >> StefanK >> >> On 2019-09-17 15:02, Per Liden wrote: >>> I propose we adjust and rename CollectedHeap::check_oop_location() >>> into CollectedHeap::is_oop_location() that returns a bool, to better >>> match its sibling CollectedHeap::is_oop() and how that is used. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231113 >>> Webrev: http://cr.openjdk.java.net/~pliden/8231113/webrev.0 >>> >>> /Per >> From stefan.johansson at oracle.com Thu Sep 19 11:40:54 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 19 Sep 2019 13:40:54 +0200 Subject: RFR (S): 8231117: Remove G1EvacuationRootClosures::raw_strong_oops() In-Reply-To: References: Message-ID: <0478FD96-FE67-4FC5-9D4D-D5C4CFF21B54@oracle.com> Hi Thomas, > 18 sep. 2019 kl. 17:02 skrev Thomas Schatzl : > > Hi all, > > can I have reviews for this small cleanup that removes the use of G1EvacuationRootClosures::raw_strong_oops() as its implementation always corresponds to a call to G1EvacuationRootClosures::strong_oops()? > > Also the comment referenced some non-existent flushing, which even more indicates that this is only some dead code. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8231117 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8231117/webrev Looks good, Stefan > Testing: > hs-tier1-5 > > Thanks, > Thomas From per.liden at oracle.com Thu Sep 19 11:47:33 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 19 Sep 2019 13:47:33 +0200 Subject: RFR: 8231113: Adjust CollectedHeap::check_oop_location() In-Reply-To: <17dc202b-c3b9-eeef-24d6-60291e352833@oracle.com> References: <17dc202b-c3b9-eeef-24d6-60291e352833@oracle.com> Message-ID: On 9/19/19 1:33 PM, Stefan Karlsson wrote: > > On 2019-09-19 13:23, Per Liden wrote: >> Hi Stefan, >> >> On 2019-09-17 21:48, Stefan Karlsson wrote: >>> Hi Per, >>> >>> I think both names are bad because the functions return false >>> positives but the names give the impression that they won't. For >>> example: >>> >>> ??? // CMS forwards some non-heap value into the mark oop to reserve >>> oops during >>> ??? // promotion, so we can't assert about obj alignment or that the >>> forwardee is in heap >>> - virtual void check_oop_location(void* addr) const {} >>> + virtual bool is_oop_location(void* addr) const { return true; } >>> >>> It's obvious that this function returns true for pointers outside of >>> the heap. Other implementations are not as bad, but they also return >>> false positives. Users can't use is_oop_location to drive code logic, >>> only to do sanity checks. >>> >>> The check_oop_location functions didn't have that problem, but they >>> couldn't be used in asserts. Did you consider changing the >>> check_oop_location functions to return a bool instead of asserting? >>> >>> ??? G1CollectedHeap* g1h = G1CollectedHeap::heap(); >>> ??? // can't do because of races >>> ??? // assert(oopDesc::is_oop_or_null(obj), "expected an oop"); >>> - g1h->check_oop_location(obj); >>> + assert(g1h->check_oop_location(obj), "invalid oop location"); >>> >>> I think that would be less misleading to readers of the code. >> >> I see your point. After looking more at how we use >> check_oop_location() today, I'm more and more thinking that it's just >> the wrong abstraction. In CMS we're "checking oops" that we know >> aren't oops, so we're forced to disable check_oop_location() >> completely. The checks done for filler objects is sort of strange too. >> We're filling a space outside of the allocated part of a TLAB/region, >> so it becomes a philosophical discussion, where this is no "correct" >> answer, whether that is a valid oop location or not. >> >> I actually think I'd prefer removing check_oop_location() all >> together, with the rational that some of the asserts where it's used >> today are "conceptually wrong". In the other cases (e.g. in >> CompressedOops and G1), we can make valid checks using better mechanisms. >> >> So, how about this: >> >> http://cr.openjdk.java.net/~pliden/8231113/webrev.1 > > Looks good to me! Thanks Stefan! /Per > > StefanK > >> >> Passed tier1-3. >> >> cheers, >> Per >> >>> >>> Thanks, >>> StefanK >>> >>> On 2019-09-17 15:02, Per Liden wrote: >>>> I propose we adjust and rename CollectedHeap::check_oop_location() >>>> into CollectedHeap::is_oop_location() that returns a bool, to better >>>> match its sibling CollectedHeap::is_oop() and how that is used. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231113 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8231113/webrev.0 >>>> >>>> /Per >>> From stefan.johansson at oracle.com Thu Sep 19 11:45:50 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 19 Sep 2019 13:45:50 +0200 Subject: RFR (S): 8231189: Rename worker_i parameters to worker_id In-Reply-To: <9e0c947a-6931-d83b-f430-fae032bfcb84@oracle.com> References: <9e0c947a-6931-d83b-f430-fae032bfcb84@oracle.com> Message-ID: <183a7b3f-2f3d-98c6-455a-a8145abad8da@oracle.com> On 2019-09-18 18:29, Thomas Schatzl wrote: > Hi Kim, > > ? thanks for your review. > > I agree about triviality; I will push this as soon as the dependent CRs > have been reviewed even if nobody else gives his approval. You can have one more =) Looks good, Stefan > > Thomas > > On 18.09.19 18:27, Kim Barrett wrote: >>> On Sep 18, 2019, at 11:51 AM, Thomas Schatzl >>> wrote: >>> >>> Hi all, >>> >>> ? can I have reviews for this change that renames all remaining >>> "worker_i"s to "worker_id"s? >>> >>> In the jdk9 timeframe we did quite a few renamings of similarly used >>> variables to "worker_id", but either some "worker_i" crept in or were >>> forgotten. >>> >>> This rectifies this. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8231189 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8231189/webrev/ >>> Testing: >>> Local compilation >>> >>> (Patch based on JDK-8159984 also out for review) >>> >>> Thanks, >>> ? Thomas >> >> Looks good, and trivial. >> > From erik.osterlund at oracle.com Thu Sep 19 12:35:06 2019 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 19 Sep 2019 14:35:06 +0200 Subject: RFR: 8231113: Adjust CollectedHeap::check_oop_location() In-Reply-To: References: Message-ID: Hi Per, With this approach we are essentially removing the support for catching forwarding to something not in the heap. I think the original intention of the assert might have been to catch scenarios where an adjust phase visits the same oop* twice. That would cause the second adjustment to read a random value in the mark word, likely not in the heap, and trigger the assert. Having said that, I think that such bugs will be easily found anyway as it will very likely crash or fail verification later on. So I think we will still catch any such bugs. And I can't recall this assertion ever helping. So I think I am okay with this relaxation. Thanks, /Erik On 2019-09-19 13:23, Per Liden wrote: > Hi Stefan, > > On 2019-09-17 21:48, Stefan Karlsson wrote: >> Hi Per, >> >> I think both names are bad because the functions return false >> positives but the names give the impression that they won't. For >> example: >> >> ??? // CMS forwards some non-heap value into the mark oop to reserve >> oops during >> ??? // promotion, so we can't assert about obj alignment or that the >> forwardee is in heap >> - virtual void check_oop_location(void* addr) const {} >> + virtual bool is_oop_location(void* addr) const { return true; } >> >> It's obvious that this function returns true for pointers outside of >> the heap. Other implementations are not as bad, but they also return >> false positives. Users can't use is_oop_location to drive code logic, >> only to do sanity checks. >> >> The check_oop_location functions didn't have that problem, but they >> couldn't be used in asserts. Did you consider changing the >> check_oop_location functions to return a bool instead of asserting? >> >> ??? G1CollectedHeap* g1h = G1CollectedHeap::heap(); >> ??? // can't do because of races >> ??? // assert(oopDesc::is_oop_or_null(obj), "expected an oop"); >> - g1h->check_oop_location(obj); >> + assert(g1h->check_oop_location(obj), "invalid oop location"); >> >> I think that would be less misleading to readers of the code. > > I see your point. After looking more at how we use > check_oop_location() today, I'm more and more thinking that it's just > the wrong abstraction. In CMS we're "checking oops" that we know > aren't oops, so we're forced to disable check_oop_location() > completely. The checks done for filler objects is sort of strange too. > We're filling a space outside of the allocated part of a TLAB/region, > so it becomes a philosophical discussion, where this is no "correct" > answer, whether that is a valid oop location or not. > > I actually think I'd prefer removing check_oop_location() all > together, with the rational that some of the asserts where it's used > today are "conceptually wrong". In the other cases (e.g. in > CompressedOops and G1), we can make valid checks using better mechanisms. > > So, how about this: > > http://cr.openjdk.java.net/~pliden/8231113/webrev.1 > > Passed tier1-3. > > cheers, > Per > >> >> Thanks, >> StefanK >> >> On 2019-09-17 15:02, Per Liden wrote: >>> I propose we adjust and rename CollectedHeap::check_oop_location() >>> into CollectedHeap::is_oop_location() that returns a bool, to better >>> match its sibling CollectedHeap::is_oop() and how that is used. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231113 >>> Webrev: http://cr.openjdk.java.net/~pliden/8231113/webrev.0 >>> >>> /Per >> From thomas.schatzl at oracle.com Thu Sep 19 12:54:05 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 19 Sep 2019 14:54:05 +0200 Subject: RFR (S): 8231189: Rename worker_i parameters to worker_id In-Reply-To: <183a7b3f-2f3d-98c6-455a-a8145abad8da@oracle.com> References: <9e0c947a-6931-d83b-f430-fae032bfcb84@oracle.com> <183a7b3f-2f3d-98c6-455a-a8145abad8da@oracle.com> Message-ID: <37d9801d3997f00949b4e5024e9dad22e00cd33a.camel@oracle.com> Hi Stefan, On Thu, 2019-09-19 at 13:45 +0200, Stefan Johansson wrote: > > On 2019-09-18 18:29, Thomas Schatzl wrote: > > Hi Kim, > > > > thanks for your review. > > > > I agree about triviality; I will push this as soon as the dependent > > CRs have been reviewed even if nobody else gives his approval. > > You can have one more =) Looks good, > Stefan > thanks for your review. Thomas From thomas.schatzl at oracle.com Thu Sep 19 12:53:17 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 19 Sep 2019 14:53:17 +0200 Subject: RFR: 8231113: Adjust CollectedHeap::check_oop_location() In-Reply-To: References: Message-ID: <90098a3911cfb4d62e6f5d15c1e84231ec43ba8b.camel@oracle.com> Hi Per, On Thu, 2019-09-19 at 13:23 +0200, Per Liden wrote: > Hi Stefan, > > On 2019-09-17 21:48, Stefan Karlsson wrote: > > Hi Per, > > > > I think both names are bad because the functions return false > > positives but the names give the impression that they won't. For > > example: > > [...] > I actually think I'd prefer removing check_oop_location() all > together, with the rational that some of the asserts where it's used > today are "conceptually wrong". In the other cases (e.g. in > CompressedOops and G1), we can make valid checks using better > mechanisms. > > So, how about this: > > http://cr.openjdk.java.net/~pliden/8231113/webrev.1 Looks good. Thomas From thomas.schatzl at oracle.com Thu Sep 19 12:54:46 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 19 Sep 2019 14:54:46 +0200 Subject: RFR (S): 8231117: Remove G1EvacuationRootClosures::raw_strong_oops() In-Reply-To: <0478FD96-FE67-4FC5-9D4D-D5C4CFF21B54@oracle.com> References: <0478FD96-FE67-4FC5-9D4D-D5C4CFF21B54@oracle.com> Message-ID: <70d8ca8573fd6e84d55735f8ac1db24b2ceeb3a4.camel@oracle.com> Hi Stefan, On Thu, 2019-09-19 at 13:40 +0200, Stefan Johansson wrote: > Hi Thomas, > > > 18 sep. 2019 kl. 17:02 skrev Thomas Schatzl < > > thomas.schatzl at oracle.com>: > > > > Hi all, > > > > can I have reviews [...] > > > > CR: > > https://bugs.openjdk.java.net/browse/JDK-8231117 > > Webrev: > > http://cr.openjdk.java.net/~tschatzl/8231117/webrev > > Looks good, > Stefan Thanks for your review. Thomas From per.liden at oracle.com Thu Sep 19 13:51:35 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 19 Sep 2019 15:51:35 +0200 Subject: RFR: 8231113: Adjust CollectedHeap::check_oop_location() In-Reply-To: References: Message-ID: <810a6634-d54a-7390-79a7-2e8470a9d427@oracle.com> Thanks for reviewing! /Per On 9/19/19 2:35 PM, Erik ?sterlund wrote: > Hi Per, > > With this approach we are essentially removing the support for catching > forwarding to something not in the heap. I think the original intention > of the assert might have been to catch scenarios where an adjust phase > visits the same oop* twice. That would cause the second adjustment to > read a random value in the mark word, likely not in the heap, and > trigger the assert. Having said that, I think that such bugs will be > easily found anyway as it will very likely crash or fail verification > later on. So I think we will still catch any such bugs. And I can't > recall this assertion ever helping. So I think I am okay with this > relaxation. > > Thanks, > /Erik > > On 2019-09-19 13:23, Per Liden wrote: >> Hi Stefan, >> >> On 2019-09-17 21:48, Stefan Karlsson wrote: >>> Hi Per, >>> >>> I think both names are bad because the functions return false >>> positives but the names give the impression that they won't. For >>> example: >>> >>> ??? // CMS forwards some non-heap value into the mark oop to reserve >>> oops during >>> ??? // promotion, so we can't assert about obj alignment or that the >>> forwardee is in heap >>> - virtual void check_oop_location(void* addr) const {} >>> + virtual bool is_oop_location(void* addr) const { return true; } >>> >>> It's obvious that this function returns true for pointers outside of >>> the heap. Other implementations are not as bad, but they also return >>> false positives. Users can't use is_oop_location to drive code logic, >>> only to do sanity checks. >>> >>> The check_oop_location functions didn't have that problem, but they >>> couldn't be used in asserts. Did you consider changing the >>> check_oop_location functions to return a bool instead of asserting? >>> >>> ??? G1CollectedHeap* g1h = G1CollectedHeap::heap(); >>> ??? // can't do because of races >>> ??? // assert(oopDesc::is_oop_or_null(obj), "expected an oop"); >>> - g1h->check_oop_location(obj); >>> + assert(g1h->check_oop_location(obj), "invalid oop location"); >>> >>> I think that would be less misleading to readers of the code. >> >> I see your point. After looking more at how we use >> check_oop_location() today, I'm more and more thinking that it's just >> the wrong abstraction. In CMS we're "checking oops" that we know >> aren't oops, so we're forced to disable check_oop_location() >> completely. The checks done for filler objects is sort of strange too. >> We're filling a space outside of the allocated part of a TLAB/region, >> so it becomes a philosophical discussion, where this is no "correct" >> answer, whether that is a valid oop location or not. >> >> I actually think I'd prefer removing check_oop_location() all >> together, with the rational that some of the asserts where it's used >> today are "conceptually wrong". In the other cases (e.g. in >> CompressedOops and G1), we can make valid checks using better mechanisms. >> >> So, how about this: >> >> http://cr.openjdk.java.net/~pliden/8231113/webrev.1 >> >> Passed tier1-3. >> >> cheers, >> Per >> >>> >>> Thanks, >>> StefanK >>> >>> On 2019-09-17 15:02, Per Liden wrote: >>>> I propose we adjust and rename CollectedHeap::check_oop_location() >>>> into CollectedHeap::is_oop_location() that returns a bool, to better >>>> match its sibling CollectedHeap::is_oop() and how that is used. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231113 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8231113/webrev.0 >>>> >>>> /Per >>> > From per.liden at oracle.com Thu Sep 19 13:51:46 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 19 Sep 2019 15:51:46 +0200 Subject: RFR: 8231113: Adjust CollectedHeap::check_oop_location() In-Reply-To: <90098a3911cfb4d62e6f5d15c1e84231ec43ba8b.camel@oracle.com> References: <90098a3911cfb4d62e6f5d15c1e84231ec43ba8b.camel@oracle.com> Message-ID: <97237af0-6689-b4b3-7136-2edc39135171@oracle.com> Thanks for reviewing! /Per On 9/19/19 2:53 PM, Thomas Schatzl wrote: > Hi Per, > > On Thu, 2019-09-19 at 13:23 +0200, Per Liden wrote: >> Hi Stefan, >> >> On 2019-09-17 21:48, Stefan Karlsson wrote: >>> Hi Per, >>> >>> I think both names are bad because the functions return false >>> positives but the names give the impression that they won't. For >>> example: >>> > [...] >> I actually think I'd prefer removing check_oop_location() all >> together, with the rational that some of the asserts where it's used >> today are "conceptually wrong". In the other cases (e.g. in >> CompressedOops and G1), we can make valid checks using better >> mechanisms. >> >> So, how about this: >> >> http://cr.openjdk.java.net/~pliden/8231113/webrev.1 > > Looks good. > > Thomas > > From huangjia at loongson.cn Thu Sep 19 14:13:18 2019 From: huangjia at loongson.cn (huangjia) Date: Thu, 19 Sep 2019 22:13:18 +0800 Subject: RFR: 8231242: G1CollectedHeap::print_regions_on lost description for "OA" and "CA" In-Reply-To: References: Message-ID: <15e63fab-f63c-e858-6795-d20a0d83f153@loongson.cn> Hi Thomas, Thank you for your review and valuable comments. Updated: http://cr.openjdk.java.net/~jiefu/8231242-huangjia/webrev.01/ Please review it and give me some advice. Thanks a lot. Best regards, Jia On Thu, 2019-09-19 at 19:24 +0800, Thomas Schatzl wrote: > The fix should also remove the "A=archive" description - there will > afaik no plain "A" regions any more. Good catch! Fixed. Thanks. From jianglizhou at google.com Thu Sep 19 14:32:38 2019 From: jianglizhou at google.com (Jiangli Zhou) Date: Thu, 19 Sep 2019 07:32:38 -0700 Subject: RFR: 8231242: G1CollectedHeap::print_regions_on lost description for "OA" and "CA" In-Reply-To: References: Message-ID: Looks good. Best, Jiangli On Thu, Sep 19, 2019 at 4:01 AM huangjia wrote: > Hi all, > > JBS: https://bugs.openjdk.java.net/browse/JDK-8231242 > > When dumping the info of heap region, the description for "OA" and "CA" > seems to be lost. > Please review the following fix which improves the readability of the > heap region log. > ----------------------------------- > diff -r 778fc2dcbdaa src/hotspot/share/gc/g1/g1CollectedHeap.cpp > --- a/src/hotspot/share/gc/g1/g1CollectedHeap.cpp Wed Sep 18 > 20:49:13 2019 -0400 > +++ b/src/hotspot/share/gc/g1/g1CollectedHeap.cpp Thu Sep 19 > 11:47:07 2019 +0800 > @@ -2378,6 +2378,7 @@ > st->print_cr("Heap Regions: E=young(eden), S=young(survivor), O=old, " > "HS=humongous(starts), HC=humongous(continues), " > "CS=collection set, F=free, A=archive, " > + "OA=open archive, CA=closed archive, " > "TAMS=top-at-mark-start (previous, next)"); > PrintRegionClosure blk(st); > heap_region_iterate(&blk); > ----------------------------------- > For more info, please see the JBS. > > Testing: > - make test TEST="jtreg:hotspot_gc" CONF=release > - make test TEST="jtreg:hotspot_gc" CONF=fastdebug > > Thanks a lot > > Best regards, > Jia > > From per.liden at oracle.com Thu Sep 19 14:41:21 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 19 Sep 2019 16:41:21 +0200 Subject: RFR: 8231251: ZGC: Fix ZHeap includes Message-ID: I noticed that the ZHeap includes needed a bit of a cleanup, as there are many includes that are not longer needed, and a few are missing. Bug: https://bugs.openjdk.java.net/browse/JDK-8231251 Webrev: http://cr.openjdk.java.net/~pliden/8231251/webrev.0 /Per From shade at redhat.com Thu Sep 19 14:54:34 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 19 Sep 2019 16:54:34 +0200 Subject: RFR (S) 8231197: Shenandoah: JVMTI heap walking cleanup crashes with NULL forwardee Message-ID: <48c5540a-7691-ae93-9a89-4d87f414a82e@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8231197 Fix: https://cr.openjdk.java.net/~shade/8231197/webrev.02/ This reliably reproduces with vmTestbase_nsk_jvmti. The reason is described in the comment there: weird clash between JVMTI-own walk of heap and Shenandoah. ShenandoahCodeRoots assert also needs relaxing, and I relaxed it only for JVMTI heap walk. Testing: hotspot_gc_shenandoah, vmTestbase_nsk_jvmti -- Thanks, -Aleksey From shade at redhat.com Thu Sep 19 14:54:59 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 19 Sep 2019 16:54:59 +0200 Subject: RFR (S) 8231198: Shenandoah: heap walking should visit all roots most of the time Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8231198 Fix: https://cr.openjdk.java.net/~shade/8231198/webrev.01/ This reliably reproduces with vmTestbase_nsk_jvmti. We have to match the objects that we walk with what JVMTI is doing there. Unfortunately, our current code sometimes discovers that unload_classes is true, and so skips some weak roots, while JVMTI visits them. As the result, the marks for objects in those weak roots is never reset, and we crash later. The easy way out is to avoid scanning weak roots only when it matters: during evacuation. Testing: hotspot_gc_shenandoah, vmTestbase_nsk_jvmti -- Thanks, -Aleksey From shade at redhat.com Thu Sep 19 14:55:06 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 19 Sep 2019 16:55:06 +0200 Subject: RFR (S) 8231244: Shenandoah: all-roots heap walking misses some weak roots Message-ID: <22183e57-1301-5a9d-dc23-66f433d2feee@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8231244 Fix: https://cr.openjdk.java.net/~shade/8231244/webrev.02/ This reliably reproduces with vmTestbase_nsk_jvmti. We miss some weak roots, apparently. New code tries to match how ShenandoahRoot{Evacuator,Adjuster}::roots_do handle it. Testing: hotspot_gc_shenandoah, vmTestbase_nsk_jvmti -- Thanks, -Aleksey From dkeenan at twitter.com Thu Sep 19 14:59:29 2019 From: dkeenan at twitter.com (David Keenan) Date: Thu, 19 Sep 2019 07:59:29 -0700 Subject: RFR: 8231242: G1CollectedHeap::print_regions_on lost description for "OA" and "CA" In-Reply-To: <15e63fab-f63c-e858-6795-d20a0d83f153@loongson.cn> References: <15e63fab-f63c-e858-6795-d20a0d83f153@loongson.cn> Message-ID: Keith McGuigan, Kirandeep Paul, Kaushik Srinivasan I?m slammed with Obj 1 and functional plans. I?ll look at the old feedback tools and see if I can find anything. He interviewed for other roles in the past as well, so you may find feedback in ats @dagskeenan | 617.755.8186 > On Sep 19, 2019, at 7:13 AM, huangjia wrote: > > Hi Thomas, > > Thank you for your review and valuable comments. > Updated: http://cr.openjdk.java.net/~jiefu/8231242-huangjia/webrev.01/ > > Please review it and give me some advice. > > Thanks a lot. > Best regards, > Jia > >> On Thu, 2019-09-19 at 19:24 +0800, Thomas Schatzl wrote: >> >> The fix should also remove the "A=archive" description - there will >> afaik no plain "A" regions any more. > > Good catch! > Fixed. Thanks. > > > From zgu at redhat.com Thu Sep 19 15:03:54 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 19 Sep 2019 11:03:54 -0400 Subject: RFR (S) 8231198: Shenandoah: heap walking should visit all roots most of the time In-Reply-To: References: Message-ID: <5f5fa982-dbeb-0790-a35b-a92ef675840f@redhat.com> Good. -Zhengyu On 9/19/19 10:54 AM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8231198 > > Fix: > https://cr.openjdk.java.net/~shade/8231198/webrev.01/ > > This reliably reproduces with vmTestbase_nsk_jvmti. We have to match the objects that we walk with > what JVMTI is doing there. Unfortunately, our current code sometimes discovers that unload_classes > is true, and so skips some weak roots, while JVMTI visits them. As the result, the marks for objects > in those weak roots is never reset, and we crash later. The easy way out is to avoid scanning weak > roots only when it matters: during evacuation. > > Testing: hotspot_gc_shenandoah, vmTestbase_nsk_jvmti > From zgu at redhat.com Thu Sep 19 15:04:06 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 19 Sep 2019 11:04:06 -0400 Subject: RFR (S) 8231244: Shenandoah: all-roots heap walking misses some weak roots In-Reply-To: <22183e57-1301-5a9d-dc23-66f433d2feee@redhat.com> References: <22183e57-1301-5a9d-dc23-66f433d2feee@redhat.com> Message-ID: Looks good. -Zhengyu On 9/19/19 10:55 AM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8231244 > > Fix: > https://cr.openjdk.java.net/~shade/8231244/webrev.02/ > > This reliably reproduces with vmTestbase_nsk_jvmti. We miss some weak roots, apparently. New code > tries to match how ShenandoahRoot{Evacuator,Adjuster}::roots_do handle it. > > Testing: hotspot_gc_shenandoah, vmTestbase_nsk_jvmti > From thomas.schatzl at oracle.com Thu Sep 19 15:12:57 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 19 Sep 2019 17:12:57 +0200 Subject: RFR: 8231242: G1CollectedHeap::print_regions_on lost description for "OA" and "CA" In-Reply-To: <15e63fab-f63c-e858-6795-d20a0d83f153@loongson.cn> References: <15e63fab-f63c-e858-6795-d20a0d83f153@loongson.cn> Message-ID: <54c6f8d9-91f1-60a9-a758-faacdb3c0d7c@oracle.com> Hi, On 19.09.19 16:13, huangjia wrote: > Hi Thomas, > > Thank you for your review and valuable comments. > Updated: http://cr.openjdk.java.net/~jiefu/8231242-huangjia/webrev.01/ > > Please review it and give me some advice. > > Thanks a lot. > Best regards, > Jia > > On Thu, 2019-09-19 at 19:24 +0800, Thomas Schatzl wrote: > >> The fix should also remove the "A=archive" description - there will >> afaik no plain "A" regions any more. > > Good catch! > Fixed. Thanks. > looks good. Thomas From zgu at redhat.com Thu Sep 19 15:03:38 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 19 Sep 2019 11:03:38 -0400 Subject: RFR (S) 8231197: Shenandoah: JVMTI heap walking cleanup crashes with NULL forwardee In-Reply-To: <48c5540a-7691-ae93-9a89-4d87f414a82e@redhat.com> References: <48c5540a-7691-ae93-9a89-4d87f414a82e@redhat.com> Message-ID: Looks good. -Zhengyu On 9/19/19 10:54 AM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8231197 > > Fix: > https://cr.openjdk.java.net/~shade/8231197/webrev.02/ > > This reliably reproduces with vmTestbase_nsk_jvmti. The reason is described in the comment there: > weird clash between JVMTI-own walk of heap and Shenandoah. ShenandoahCodeRoots assert also needs > relaxing, and I relaxed it only for JVMTI heap walk. > > Testing: hotspot_gc_shenandoah, vmTestbase_nsk_jvmti > From kim.barrett at oracle.com Thu Sep 19 17:27:20 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 19 Sep 2019 13:27:20 -0400 Subject: RFR (M): 8159984: Remove call to ClassLoaderDataGraph::clear_claimed_marks during the initial mark pause In-Reply-To: <52dd1037-00b6-86a6-0222-3a7807c5d2c9@oracle.com> References: <4e851c12-ca88-5d2d-f636-767437ad5a01@oracle.com> <52dd1037-00b6-86a6-0222-3a7807c5d2c9@oracle.com> Message-ID: <22E42951-E28C-478A-9D9B-6FBD320335B5@oracle.com> > On Sep 19, 2019, at 5:42 AM, Thomas Schatzl wrote: > > Hi Kim, > > thanks for your review. > > On 18.09.19 20:36, Kim Barrett wrote: >>> On Sep 18, 2019, at 11:22 AM, Thomas Schatzl wrote: >>> >>> Hi all, >>> >>> can I have reviews for this change that, given the observation that during root processing, there is only one kind of root that walks CLDs, we do not need to >>> >>> [...] >>> >>> (based on JDK-8231117) >>> >>> Thanks, >>> Thomas >> I was just looking at the root scanning code recently, trying to >> figure out what worker_has_discovered_all_strong_classes() was about. >> The naming of worker_has_discovered_all_strong_classes() and >> wait_until_all_strong_classes_discovered() seems wrong now. These are >> about discovering/processing strong nmethods now. >> A better comment at the call to the wait function about why it's >> called there would be helpful. Obviously we want it as late as >> possible, to allow threads to get past the notification. But why not >> later? > > We could at the latest do the waiting when we do the nmethod weak root scanning. Given that I think I now have a working prototype that works without the barrier I did not care optimizing that for this change. > If there are any unforeseen issues, or it does not pan out, I can improve that (in a separate CR if needed). > > I fixed the naming of the two methods and tried to improve the comment in a new webrev. > > New webrevs: > http://cr.openjdk.java.net/~tschatzl/8159984/webrev.0_to_1/ (diff) > http://cr.openjdk.java.net/~tschatzl/8159984/webrev.1/ (full) The additional comments look good. But shouldn?t the names of those functions refer to ?nmethods" rather than (generic) ?roots?? That is, worker_has_discovered_all_strong_nmethods wait_until_all_strong_nmethods_discovered There are certainly strong roots that get looked at after the has_discovered call. > >> (We talked about this off-line; this is just a reminder and >> request for some commentary capturing that discussion. Though >> hopefully your idea for getting rid of the barrier completly will pan >> out. >> Other than that, looks good. > > Thanks! > > Thomas From rkennke at redhat.com Thu Sep 19 17:39:38 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 19 Sep 2019 19:39:38 +0200 Subject: RFR (S) 8231197: Shenandoah: JVMTI heap walking cleanup crashes with NULL forwardee In-Reply-To: <48c5540a-7691-ae93-9a89-4d87f414a82e@redhat.com> References: <48c5540a-7691-ae93-9a89-4d87f414a82e@redhat.com> Message-ID: I think it's ok for now. Long-term, the JVMTI heapwalking should be done by GC as well to ensure consistency. I can already spot some subtle bugs there concerning concurrent roots cleaning. Roman > Bug: > https://bugs.openjdk.java.net/browse/JDK-8231197 > > Fix: > https://cr.openjdk.java.net/~shade/8231197/webrev.02/ > > This reliably reproduces with vmTestbase_nsk_jvmti. The reason is described in the comment there: > weird clash between JVMTI-own walk of heap and Shenandoah. ShenandoahCodeRoots assert also needs > relaxing, and I relaxed it only for JVMTI heap walk. > > Testing: hotspot_gc_shenandoah, vmTestbase_nsk_jvmti > From rkennke at redhat.com Thu Sep 19 17:41:06 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 19 Sep 2019 19:41:06 +0200 Subject: RFR (S) 8231198: Shenandoah: heap walking should visit all roots most of the time In-Reply-To: References: Message-ID: That makes sense. ok. For the same reason described in JDK-8231197, JVMTI should not actually do heapwalking of its own. Roman > Bug: > https://bugs.openjdk.java.net/browse/JDK-8231198 > > Fix: > https://cr.openjdk.java.net/~shade/8231198/webrev.01/ > > This reliably reproduces with vmTestbase_nsk_jvmti. We have to match the objects that we walk with > what JVMTI is doing there. Unfortunately, our current code sometimes discovers that unload_classes > is true, and so skips some weak roots, while JVMTI visits them. As the result, the marks for objects > in those weak roots is never reset, and we crash later. The easy way out is to avoid scanning weak > roots only when it matters: during evacuation. > > Testing: hotspot_gc_shenandoah, vmTestbase_nsk_jvmti > From rkennke at redhat.com Thu Sep 19 17:41:27 2019 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 19 Sep 2019 19:41:27 +0200 Subject: RFR (S) 8231244: Shenandoah: all-roots heap walking misses some weak roots In-Reply-To: <22183e57-1301-5a9d-dc23-66f433d2feee@redhat.com> References: <22183e57-1301-5a9d-dc23-66f433d2feee@redhat.com> Message-ID: <4d0ec821-d94f-3eba-7b77-62a87b73cba8@redhat.com> Looks good. Thanks! Roman > Bug: > https://bugs.openjdk.java.net/browse/JDK-8231244 > > Fix: > https://cr.openjdk.java.net/~shade/8231244/webrev.02/ > > This reliably reproduces with vmTestbase_nsk_jvmti. We miss some weak roots, apparently. New code > tries to match how ShenandoahRoot{Evacuator,Adjuster}::roots_do handle it. > > Testing: hotspot_gc_shenandoah, vmTestbase_nsk_jvmti > From per.liden at oracle.com Thu Sep 19 21:07:16 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 19 Sep 2019 23:07:16 +0200 Subject: RFR: 8231266: ZGC: Minor cleanups in ZCollectedHeap and ZHeap Message-ID: <40e2f38a-582c-8a06-670e-dd51f8b042d4@oracle.com> ZCollectedHeap's is_oop() and print_location() should delegate to ZHeap. Also adjusting a few types, so that the ZHeap interface uses uintptr_t. This keeps things consistent and avoids a few casts in ZHeap. Bug: https://bugs.openjdk.java.net/browse/JDK-8231266 Webrev: http://cr.openjdk.java.net/~pliden/8231266/webrev.0 /Per From per.liden at oracle.com Thu Sep 19 21:08:33 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 19 Sep 2019 23:08:33 +0200 Subject: RFR: 8231270: ZGC: Remove ZAddressSpace* and ZAddressReserved* Message-ID: <11eec445-9827-8150-efd9-59cb104e866c@oracle.com> After JDK-8224820, ZAddressSpace{Start,End,Size} is not used for anything other than being printed, so they can be removed. Also, the two last uses of ZAddressReserved* can easily be replaced. Bug: https://bugs.openjdk.java.net/browse/JDK-8231270 Webrev: http://cr.openjdk.java.net/~pliden/8231270/webrev.0 /Per From huangjia at loongson.cn Fri Sep 20 01:22:29 2019 From: huangjia at loongson.cn (huangjia) Date: Fri, 20 Sep 2019 09:22:29 +0800 Subject: RFR: 8231242: G1CollectedHeap::print_regions_on lost description for "OA" and "CA" In-Reply-To: <54c6f8d9-91f1-60a9-a758-faacdb3c0d7c@oracle.com> References: <15e63fab-f63c-e858-6795-d20a0d83f153@loongson.cn> <54c6f8d9-91f1-60a9-a758-faacdb3c0d7c@oracle.com> Message-ID: <6d77c7bc-95d7-6fe3-1a1b-c1aaab5a0c88@loongson.cn> Thanks Thomas. ? 2019?09?19? 23:12, Thomas Schatzl ??: > > looks good. From huangjia at loongson.cn Fri Sep 20 01:26:59 2019 From: huangjia at loongson.cn (huangjia) Date: Fri, 20 Sep 2019 09:26:59 +0800 Subject: RFR: 8231242: G1CollectedHeap::print_regions_on lost description for "OA" and "CA" In-Reply-To: References: Message-ID: Hi Jiangli, Thanks for your review. Are you Ok with http://cr.openjdk.java.net/~jiefu/8231242-huangjia/webrev.01/ ? Best regards, Jia ? 2019?09?19? 22:32, Jiangli Zhou ??: > Looks good. > > Best, > Jiangli > > On Thu, Sep 19, 2019 at 4:01 AM huangjia > wrote: > > Hi all, > > JBS: https://bugs.openjdk.java.net/browse/JDK-8231242 > > When dumping the info of heap region, the description for "OA" and > "CA" > seems to be lost. > Please review the following fix which improves the readability of the > heap region log. > ----------------------------------- > diff -r 778fc2dcbdaa src/hotspot/share/gc/g1/g1CollectedHeap.cpp > --- a/src/hotspot/share/gc/g1/g1CollectedHeap.cpp Wed Sep 18 > 20:49:13 2019 -0400 > +++ b/src/hotspot/share/gc/g1/g1CollectedHeap.cpp Thu Sep 19 > 11:47:07 2019 +0800 > @@ -2378,6 +2378,7 @@ > st->print_cr("Heap Regions: E=young(eden), S=young(survivor), > O=old, " > "HS=humongous(starts), HC=humongous(continues), " > "CS=collection set, F=free, A=archive, " > + "OA=open archive, CA=closed archive, " > "TAMS=top-at-mark-start (previous, next)"); > PrintRegionClosure blk(st); > heap_region_iterate(&blk); > ----------------------------------- > For more info, please see the JBS. > > Testing: > - make test TEST="jtreg:hotspot_gc" CONF=release > - make test TEST="jtreg:hotspot_gc" CONF=fastdebug > > Thanks a lot > > Best regards, > Jia > From jianglizhou at google.com Fri Sep 20 02:15:19 2019 From: jianglizhou at google.com (Jiangli Zhou) Date: Thu, 19 Sep 2019 19:15:19 -0700 Subject: RFR: 8231242: G1CollectedHeap::print_regions_on lost description for "OA" and "CA" In-Reply-To: References: Message-ID: Hi Jia, Yes. Glad that Thomas caught the leftover 'A' type. I missed it initially. Best regards, Jiangli On Thu, Sep 19, 2019, 6:27 PM huangjia wrote: > Hi Jiangli, > > Thanks for your review. > Are you Ok with > http://cr.openjdk.java.net/~jiefu/8231242-huangjia/webrev.01/ ? > > Best regards, > Jia > > ? 2019?09?19? 22:32, Jiangli Zhou ??: > > Looks good. > > Best, > Jiangli > > On Thu, Sep 19, 2019 at 4:01 AM huangjia wrote: > >> Hi all, >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8231242 >> >> When dumping the info of heap region, the description for "OA" and "CA" >> seems to be lost. >> Please review the following fix which improves the readability of the >> heap region log. >> ----------------------------------- >> diff -r 778fc2dcbdaa src/hotspot/share/gc/g1/g1CollectedHeap.cpp >> --- a/src/hotspot/share/gc/g1/g1CollectedHeap.cpp Wed Sep 18 >> 20:49:13 2019 -0400 >> +++ b/src/hotspot/share/gc/g1/g1CollectedHeap.cpp Thu Sep 19 >> 11:47:07 2019 +0800 >> @@ -2378,6 +2378,7 @@ >> st->print_cr("Heap Regions: E=young(eden), S=young(survivor), O=old, " >> "HS=humongous(starts), HC=humongous(continues), " >> "CS=collection set, F=free, A=archive, " >> + "OA=open archive, CA=closed archive, " >> "TAMS=top-at-mark-start (previous, next)"); >> PrintRegionClosure blk(st); >> heap_region_iterate(&blk); >> ----------------------------------- >> For more info, please see the JBS. >> >> Testing: >> - make test TEST="jtreg:hotspot_gc" CONF=release >> - make test TEST="jtreg:hotspot_gc" CONF=fastdebug >> >> Thanks a lot >> >> Best regards, >> Jia >> >> > From huangjia at loongson.cn Fri Sep 20 03:40:28 2019 From: huangjia at loongson.cn (huangjia) Date: Fri, 20 Sep 2019 11:40:28 +0800 Subject: RFR: 8231242: G1CollectedHeap::print_regions_on lost description for "OA" and "CA" In-Reply-To: References: Message-ID: Thanks Jiangli. ? 2019?09?20? 10:15, Jiangli Zhou ??: > Yes. Glad that Thomas caught the leftover 'A' type. I missed it initially. From huangjia at loongson.cn Fri Sep 20 03:48:39 2019 From: huangjia at loongson.cn (huangjia) Date: Fri, 20 Sep 2019 11:48:39 +0800 Subject: RFR: 8231242: G1CollectedHeap::print_regions_on lost description for "OA" and "CA" In-Reply-To: <54c6f8d9-91f1-60a9-a758-faacdb3c0d7c@oracle.com> References: <15e63fab-f63c-e858-6795-d20a0d83f153@loongson.cn> <54c6f8d9-91f1-60a9-a758-faacdb3c0d7c@oracle.com> Message-ID: <657b9c92-514f-c2d8-4d38-9eaf0d0db68b@loongson.cn> Hi Thomas, Updated: http://cr.openjdk.java.net/~jiefu/8231242-huangjia/webrev.02/ I had prepared the patch by adding the reviewers in it. Please note the user in the patch. Hope you wouldn't mind it. Thanks. Could you please sponsor it? Thanks a lot. Best regards, Jia ? 2019?09?19? 23:12, Thomas Schatzl ??: > Hi, > > On 19.09.19 16:13, huangjia wrote: >> Hi Thomas, >> >> Thank you for your review and valuable comments. >> Updated: http://cr.openjdk.java.net/~jiefu/8231242-huangjia/webrev.01/ >> >> Please review it and give me some advice. >> >> Thanks a lot. >> Best regards, >> Jia >> >> On Thu, 2019-09-19 at 19:24 +0800, Thomas Schatzl wrote: >> >>> The fix should also remove the "A=archive" description - there will >>> afaik no plain "A" regions any more. >> >> Good catch! >> Fixed. Thanks. >> > > looks good. > > Thomas > From thomas.schatzl at oracle.com Fri Sep 20 08:02:24 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 20 Sep 2019 10:02:24 +0200 Subject: RFR (M): 8159984: Remove call to ClassLoaderDataGraph::clear_claimed_marks during the initial mark pause In-Reply-To: <22E42951-E28C-478A-9D9B-6FBD320335B5@oracle.com> References: <4e851c12-ca88-5d2d-f636-767437ad5a01@oracle.com> <52dd1037-00b6-86a6-0222-3a7807c5d2c9@oracle.com> <22E42951-E28C-478A-9D9B-6FBD320335B5@oracle.com> Message-ID: Hi, On 19.09.19 19:27, Kim Barrett wrote: >> On Sep 19, 2019, at 5:42 AM, Thomas Schatzl wrote: >> >> Hi Kim, >> >> thanks for your review. >> >> On 18.09.19 20:36, Kim Barrett wrote: >>>> On Sep 18, 2019, at 11:22 AM, Thomas Schatzl wrote: >>>> >>>> Hi all, >>>> >>>> can I have reviews for this change that, given the observation that during root processing, there is only one kind of root that walks CLDs, we do not need to >>>> >>>> [...] >>>> >>>> (based on JDK-8231117) >>>> >>>> Thanks, >>>> Thomas >>> I was just looking at the root scanning code recently, trying to >>> figure out what worker_has_discovered_all_strong_classes() was about. >>> The naming of worker_has_discovered_all_strong_classes() and >>> wait_until_all_strong_classes_discovered() seems wrong now. These are >>> about discovering/processing strong nmethods now. >>> A better comment at the call to the wait function about why it's >>> called there would be helpful. Obviously we want it as late as >>> possible, to allow threads to get past the notification. But why not >>> later? >> >> We could at the latest do the waiting when we do the nmethod weak root scanning. Given that I think I now have a working prototype that works without the barrier I did not care optimizing that for this change. >> If there are any unforeseen issues, or it does not pan out, I can improve that (in a separate CR if needed). >> >> I fixed the naming of the two methods and tried to improve the comment in a new webrev. >> >> New webrevs: >> http://cr.openjdk.java.net/~tschatzl/8159984/webrev.0_to_1/ (diff) >> http://cr.openjdk.java.net/~tschatzl/8159984/webrev.1/ (full) > > The additional comments look good. But shouldn?t the names of those functions refer to ?nmethods" > rather than (generic) ?roots?? That is, > > worker_has_discovered_all_strong_nmethods > wait_until_all_strong_nmethods_discovered > > There are certainly strong roots that get looked at after the has_discovered call. fixed in place as suggested. The naming is indeed overloaded here; I wanted to make the names more generic than they need to be. Thanks, Thomas From thomas.schatzl at oracle.com Fri Sep 20 08:02:41 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 20 Sep 2019 10:02:41 +0200 Subject: RFR: 8231242: G1CollectedHeap::print_regions_on lost description for "OA" and "CA" In-Reply-To: <657b9c92-514f-c2d8-4d38-9eaf0d0db68b@loongson.cn> References: <15e63fab-f63c-e858-6795-d20a0d83f153@loongson.cn> <54c6f8d9-91f1-60a9-a758-faacdb3c0d7c@oracle.com> <657b9c92-514f-c2d8-4d38-9eaf0d0db68b@loongson.cn> Message-ID: Hi, On 20.09.19 05:48, huangjia wrote: > Hi Thomas, > > Updated: http://cr.openjdk.java.net/~jiefu/8231242-huangjia/webrev.02/ > > I had prepared the patch by adding the reviewers in it. > Please note the user in the patch. > Hope you wouldn't mind it. Thanks. > > Could you please sponsor it? pushed. Thomas From huangjia at loongson.cn Fri Sep 20 08:08:40 2019 From: huangjia at loongson.cn (Jia Huang) Date: Fri, 20 Sep 2019 16:08:40 +0800 Subject: RFR: 8231242: G1CollectedHeap::print_regions_on lost description for "OA" and "CA" In-Reply-To: References: <15e63fab-f63c-e858-6795-d20a0d83f153@loongson.cn> <54c6f8d9-91f1-60a9-a758-faacdb3c0d7c@oracle.com> <657b9c92-514f-c2d8-4d38-9eaf0d0db68b@loongson.cn> Message-ID: Thanks Thomas. ? 2019?09?20? 16:02, Thomas Schatzl ??: > pushed. From kim.barrett at oracle.com Fri Sep 20 08:33:19 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 20 Sep 2019 04:33:19 -0400 Subject: RFR (M): 8159984: Remove call to ClassLoaderDataGraph::clear_claimed_marks during the initial mark pause In-Reply-To: References: <4e851c12-ca88-5d2d-f636-767437ad5a01@oracle.com> <52dd1037-00b6-86a6-0222-3a7807c5d2c9@oracle.com> <22E42951-E28C-478A-9D9B-6FBD320335B5@oracle.com> Message-ID: <5C7F042E-048E-446B-B67F-46E5A7092136@oracle.com> > On Sep 20, 2019, at 4:02 AM, Thomas Schatzl wrote: >> The additional comments look good. But shouldn?t the names of those functions refer to ?nmethods" >> rather than (generic) ?roots?? That is, >> worker_has_discovered_all_strong_nmethods >> wait_until_all_strong_nmethods_discovered >> There are certainly strong roots that get looked at after the has_discovered call. > > fixed in place as suggested. The naming is indeed overloaded here; I wanted to make the names more generic than they need to be. > > Thanks, > Thomas Looks good. From zgu at redhat.com Fri Sep 20 11:29:00 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Fri, 20 Sep 2019 07:29:00 -0400 Subject: RFR 8231250: Shenandoah: Shenandoah assert_correct failed; Object klass pointer Message-ID: Many thanks to Christroph Langer (SAP) for reporting and helping to verify the problem. The initial report was recorded in JDK-8230483, unfortunately, it did not fix the problem. The root cause is that JDK-8227635 work changed behavior of traversal GC, which is required to always enqueue new value. Bug: https://bugs.openjdk.java.net/browse/JDK-8231250 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8231250/webrev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) on Linux x86_64 SAP internal tests Thanks, -Zhengyu From stefan.johansson at oracle.com Fri Sep 20 12:00:03 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 20 Sep 2019 14:00:03 +0200 Subject: RFR (M): 8159984: Remove call to ClassLoaderDataGraph::clear_claimed_marks during the initial mark pause In-Reply-To: References: <4e851c12-ca88-5d2d-f636-767437ad5a01@oracle.com> <52dd1037-00b6-86a6-0222-3a7807c5d2c9@oracle.com> <22E42951-E28C-478A-9D9B-6FBD320335B5@oracle.com> Message-ID: <6aa7ec98-a886-2318-a884-04e79744c84f@oracle.com> Hi Thomas, On 2019-09-20 10:02, Thomas Schatzl wrote: > Hi, > > On 19.09.19 19:27, Kim Barrett wrote: >>> On Sep 19, 2019, at 5:42 AM, Thomas Schatzl >>> wrote: >>> >>> Hi Kim, >>> >>> ? thanks for your review. >>> >>> On 18.09.19 20:36, Kim Barrett wrote: >>>>> On Sep 18, 2019, at 11:22 AM, Thomas Schatzl >>>>> wrote: >>>>> >>>>> Hi all, >>>>> >>>>> ? can I have reviews for this change that, given the observation >>>>> that during root processing, there is only one kind of root that >>>>> walks CLDs, we do not need to >>>>> >>>>> [...] >>>>> >>>>> (based on JDK-8231117) >>>>> >>>>> Thanks, >>>>> ? Thomas >>>> I was just looking at the root scanning code recently, trying to >>>> figure out what worker_has_discovered_all_strong_classes() was about. >>>> The naming of worker_has_discovered_all_strong_classes() and >>>> wait_until_all_strong_classes_discovered() seems wrong now.? These are >>>> about discovering/processing strong nmethods now. >>>> A better comment at the call to the wait function about why it's >>>> called there would be helpful.? Obviously we want it as late as >>>> possible, to allow threads to get past the notification.? But why not >>>> later? >>> >>> We could at the latest do the waiting when we do the nmethod weak >>> root scanning. Given that I think I now have a working prototype that >>> works without the barrier I did not care optimizing that for this >>> change. >>> If there are any unforeseen issues, or it does not pan out, I can >>> improve that (in a separate CR if needed). >>> >>> I fixed the naming of the two methods and tried to improve the >>> comment in a new webrev. >>> >>> New webrevs: >>> http://cr.openjdk.java.net/~tschatzl/8159984/webrev.0_to_1/ (diff) >>> http://cr.openjdk.java.net/~tschatzl/8159984/webrev.1/ (full) >> >> The additional comments look good.? But shouldn?t the names of those >> functions refer to ?nmethods" >> rather than (generic) ?roots??? That is, >> >> worker_has_discovered_all_strong_nmethods >> wait_until_all_strong_nmethods_discovered >> >> There are certainly strong roots that get looked at after the >> has_discovered call. > > ? fixed in place as suggested. The naming is indeed overloaded here; I > wanted to make the names more generic than they need to be. > Looks good, just one minor comment that you can ignore if you don't agree and no need for a new webrev. I think the parameter notify_claimed_roots_done should also follow the "nmethod naming scheme". Thanks, Stefan > Thanks, > ? Thomas From kim.barrett at oracle.com Fri Sep 20 18:50:08 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 20 Sep 2019 14:50:08 -0400 Subject: RFR (M): 8159984: Remove call to ClassLoaderDataGraph::clear_claimed_marks during the initial mark pause In-Reply-To: <6aa7ec98-a886-2318-a884-04e79744c84f@oracle.com> References: <4e851c12-ca88-5d2d-f636-767437ad5a01@oracle.com> <52dd1037-00b6-86a6-0222-3a7807c5d2c9@oracle.com> <22E42951-E28C-478A-9D9B-6FBD320335B5@oracle.com> <6aa7ec98-a886-2318-a884-04e79744c84f@oracle.com> Message-ID: > On Sep 20, 2019, at 8:00 AM, Stefan Johansson wrote: > Looks good, just one minor comment that you can ignore if you don't agree and no need for a new webrev. I think the parameter notify_claimed_roots_done should also follow the "nmethod naming scheme?. +1 > > Thanks, > Stefan > >> Thanks, >> Thomas From sangheon.kim at oracle.com Sat Sep 21 05:19:13 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Fri, 20 Sep 2019 22:19:13 -0700 Subject: RFR(XL): 8220310: Implementation: NUMA-Aware Memory Allocation for G1, Mutator (1/3) In-Reply-To: <06ACBF87-ADBE-499F-B668-0274E4925B26@oracle.com> References: <06ACBF87-ADBE-499F-B668-0274E4925B26@oracle.com> Message-ID: <2b37edd6-3e0f-013d-1616-9d003f8ac1ed@oracle.com> Hi Kim, Many thanks for this thorough review! On 9/16/19 4:12 PM, Kim Barrett wrote: >> On Sep 4, 2019, at 3:16 AM, sangheon.kim at oracle.com wrote: >> >> Hi all, >> >> Please review this patch, making G1 NUMA aware. >> This is the first part of G1 NUMA implementation. >> >> - At G1 CollectedHeap initialization time, the given heap regions will be split and touched. >> - Internally node index(uint) is used instead of node id(int type) for easier access to arrays. >> - Only Linux is supported. >> - New logs are under gc+heap+numa >> - Current memory touch implementation will be replaced with WorkGang by JDK-8230411: NUMA aware work gang. >> >> CR: https://bugs.openjdk.java.net/browse/JDK-8220310 >> Webrev: http://cr.openjdk.java.net/~sangheki/8220310/webrev.0 >> Testing: hs-tier 1 ~ 5 with +- UseNUMA. >> >> Thanks, >> Sangheon I had long off-line discussion with Kim, Thomas and Stefan J. And 2 major changes are: 1) Memory touch thread is removed. This was a general way to implement for both Linux and Solaris. However, Solaris is not supported anymore, so memory touch thread is just removed. And this made the code much simpler. 2) G1NUMA doesn't manage numa id of each page and we don't care about each page instead HeapRegion is only cared. i.e. we assume numa id of all pages between bottom and end of HeapRegion to be same. This also made related codes much simpler. There are some other changes as well. > Here is an initial set of comments. Some of these might be nonsense; I'm > not all that familiar with NUMA support. > > ------------------------------------------------------------------------------ > > Just in general, it's kind of disappointing that NUMA support for G1 > apparently involves this much code, and is completely different from > ParallelGC NUMA support, and both are completely different from ZGC > NUMA support. (And Shenandoah is probably yet another still.) > Oh well... Yes, I agree on that. > > ------------------------------------------------------------------------------ > > According to off-line discussion, the intent is that a HeapRegion only > contains memory from one NUMA node. That isn't obvious though, and there > are API and data structure choices that obscure that. It seems like some > things could be simplified with that constraint in mind. I think some of > that is a result of trying to leave open support for NUMA aware off-heap > data structures, but that isn't part of this change, and might be better > kept separate from NUMA aware Java heap support. > > I'll try to point out places where I think there are opportunities for > simplification. > > ------------------------------------------------------------------------------ > > According to off-line discussion, the touching mechanism might not be > necessary. If that's true, that would be a significant simplification. For > now I won't comment on the code for the touch threads, though I think > I found some problems there before the discussion suggesting we might > not need them happened. Thanks. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1NUMA.hpp > 64 // Manages NUMA node related information and memory touch threads. > ... > > I found the comment describing G1NUMA pretty hard to parse. Even > after looking at the code I'm having trouble figuring out what's being > said. > > This part I was able to figure out, but think it could be worded better. > 70 // Also provides a way to convert NUMA id and NUMA index(starting from zero and sequatial). > 71 // Actual NUMA ids would not be sequential nor not start from zero, so a conversion is necessary > 72 // between id and index. > > Something like > > Also provides a mapping between NUMA indices (sequential, starting > from zero) and NUMA ids (which may not start at zero and may not be > sequential). Indices are generally used by clients, and mapped to > ids when dealing with the OS/hardware layer. DONE > > (I *think* that's true.) > > But then, why expose node_ids at all in G1NUMA? It seems like node_ids > could be completely relegated to an implementation detail that clients > never need to deal with. ... > As evidence for not needing to expose ids to clients, there are no > callers of numa_id_of_index in any of the series of 3 patches. And > G1NUMA::numa_ids() and G1MemoryMultNodeManager::node_ids() are only > used in a couple of places for printing/logging and by > WB_G1MemoryNodeIds. I'm not sure what the last is for, but the > printing and logging code could be isolated into G1NUMA. Basically I totally agree with you for not exposing numa id. Current patches are only using node id for logging purpose at some locations because users are only interested on node id. Also Thread::lgrp_id() (renaming lgrp id, numa id etc.. is different topic that we had before) is also public, so my feeling is current try is enough. :) WB_G1MemoryNodeIds() is for jtreg testing purpose which checks whether the given memory has node id as we expected. > > Also, it looks like the index to node_id mapping provided here is > duplicating the information provided by os::Linux::nindex_to_node. > Not that this necessarily helps with anything in generic code... Yes, I'm aware of _nindex_to_node but I think maintaining such conversion at G1NUMA is okay. Other NUMA implementation such as parallel gc relies on node id so completely removing node id at general code seems different scope. Unless you have strong opinion on addressing it in this patch, I would like proceed as is. If it is the case, we can consider your suggestion as a separate enhancement. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp > 57 virtual int* node_ids() const { static int dummy_id = 0; return &dummy_id; } > > Can the return type here be "const int*" ? That would be better if possible. > > [This is probably moot if clients only deal with numa indexes and not > node_ids, as suggested earlier.] DONE > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp > 63 virtual uint guarantee_valid_node_index(uint node_index) const { return 0; } > > Given naming conventions elsewhere, the name of this function makes me > expect it to be a verification function. Agree. valid_node_index() is better? webrev.1 changed name. Or are you suggesting to remove this and then the caller do such thing? e.g. void A(uint node_index) { uint valid_node_index = node_index; if (!is_valid_node_index(valid_node_index)) { ? valid_node_index = random_numa_index(); ... > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1NUMA.inline.hpp > 29 #include "gc/shared/taskqueue.inline.hpp" > > This #include seems to be just to obtain randomParkAndMiller. It would be > better to refactor that function into its own file(s). Or make it part of > some component providing various random number generators. Not need because of below change. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1NUMA.inline.hpp > 45 int seed = randomParkAndMiller(_seed); > 46 Atomic::store(seed, &_seed); > > This sequence may lose updates. I don't know if that's important. > And why isn't this just using os::random(), or maybe refactoring that > slightly to gain access to the external seed version. I don't want this > change to be made: > http://cr.openjdk.java.net/~sangheki/8220310/webrev.0/src/hotspot/share/gc/shared/taskqueue.inline.hpp.udiff.html Thomas and Stefan J. also complained on this. Changed to get next numa index with round-robin manner. i.e. doesn't use random() at all. > And do we *really* need randomness here at all? Is there a reason not > to simply cycle through the nodes? Not really. > > ------------------------------------------------------------------------------ > src/hotspot/os/linux/os_linux.cpp > 3016 size_t num_pages_left = size_in_bytes / page_size; > > Is size_in_bytes required to be a multiple of page_size? I believe it is, > and an assert of that would be useful. This code is removed. And yes to answer to your question. > > ------------------------------------------------------------------------------ > src/hotspot/os/linux/os_linux.cpp > 3016 size_t num_pages_left = size_in_bytes / page_size; > 3017 const size_t num_pages_total = num_pages_left; > 3018 size_t num_buffer_entries = MIN2(num_pages_left, max_pages); > > I think clearer would be > > 3017 const size_t num_pages_total = size_in_bytes / page_size; > 3018 size_t num_buffer_entries = MIN2(num_pages_total, max_pages); > > and move this down to the loop setup: > 3016 size_t num_pages_left = num_pages_total; This code is removed. > > ------------------------------------------------------------------------------ > src/hotspot/os/linux/os_linux.cpp > 3020 void** pages = (void**)alloca(sizeof(void*) * num_buffer_entries); > 3021 int* status = (int*)alloca(sizeof(int) * num_buffer_entries); > > I think a ResourceMark and NEW_RESOURCE_ARRAYs would be preferable to > using alloca() here. alloca() has some dangers and I think is currently > only used in some very specialized places where it really is needed. This code is removed. > > ------------------------------------------------------------------------------ > src/hotspot/os/linux/os_linux.cpp > 3061 } > 3062 // Moves current thread on a specific node and it will not migrate to > > Missing blank line between functions. This code is removed. > ------------------------------------------------------------------------------ > src/hotspot/os/linux/os_linux.cpp > 3031 pages[i] = (void*)base; > > Unnecessary cast. This code is removed. > > ------------------------------------------------------------------------------ > src/hotspot/os/linux/os_linux.cpp > 3072 case AFFINITY_NONE: > 3073 // Do nothing. > 3074 return true; > 3075 break; > > Shouldn't this be calling numa_run_on_node with -1 node id? The > documentation for numa_run_on_node() says "Passing -1 permits the > kernel to schedule on all nodes again." My understand of that description is let kernel decide the numa node which is not actually want here. We want to move current thread to the preferred numa node. Currently we don't need this API (numa_set_affinity() ) so removed on webrev.1. However following patch which is not scope of this JEP will use it. I will add your suggestion below? when we need this. > > But see also next comment about numa_affinity_set. > > ------------------------------------------------------------------------------ > src/hotspot/share/runtime/os.hpp > 401 // Available affinities. > 402 enum NumaAffinity { > 403 AFFINITY_STRONG, > 404 AFFINITY_NONE > 405 }; > 406 static bool numa_affinity_set(Thread* thread, int numa_id, NumaAffinity affinity); > > Could there ever be other affinities? This API doesn't seem right. > It seems to me it should be something like > > static bool numa_set_affinity(Thread* thread, int numa_id); > static bool numa_clear_affinity(Thread* thread); > > No need for NumaAffinity enum. The enum is also kind of remainder after removing Solaris implementation which has more types. But I agree with you as there are only 2 types now, it is simpler to separate API without enum. > > (I think numa_set/clear_affinity is better than numa_affinity_set/clear; > numa is a "namespace" and we usually set/clear/get_something rather > than something_set/clear/get.) > > Or maybe this should just be > > static bool numa_set_affinity(Thread* thread, int numa_id); > > with "clearing" provided by passing AnyId as the numa_id? Still don't > need the NumaAffinity enum for this. Agree on this naming stuff. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1NUMA.cpp > 124 guarantee(rs.size() % page_size == 0, "Reserved space size (" SIZE_FORMAT ") should be aligned " > 141 guarantee(size_in_bytes % page_size == 0, "The given range (" SIZE_FORMAT ") should be aligned " > 209 guarantee((uintptr_t)G1HeapRegionSize % page_size == 0, "G1HeapRegionSize (" SIZE_FORMAT ") and page size (" > 234 guarantee((uintptr_t)_page_size % _heap_region_size == 0, "G1HeapRegionSize (" SIZE_FORMAT ") and page size (" > > Use is_aligned. I think casts shouldn't be needed. These lines are removed after major refactoring. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1NUMA.cpp > 43 _numa_id_to_index_map = NEW_C_HEAP_ARRAY_RETURN_NULL(uint, _len_numa_id_to_index_map, mtGC); > 44 memset(_numa_id_to_index_map, 0, sizeof(uint) * _len_numa_id_to_index_map); > > Using NEW_xxx_RETURN_NULL and then ignoring the possibility of a NULL > result. It would be better to use the non-_RETURN_NULL variant and > get a decent error message rather than the segfault from attemptint to > write to NULL. Changed to use non-_RETURN_NULL version. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1NUMA.cpp > 34 void G1NUMA::init_numa_id_to_index_map(int* numa_ids, uint num_numa_ids) { > > int* numa_ids => const int* numa_ids. DONE > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1NUMA.cpp > 34 void G1NUMA::init_numa_id_to_index_map(int* numa_ids, uint num_numa_ids) { > ... > 44 memset(_numa_id_to_index_map, 0, sizeof(uint) * _len_numa_id_to_index_map); > > Rather than this memset and then writing to all of them later, instead > initialize all entries to InvalidNodeIndex and then assign from numa_ids. DONE > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1MemoryTouchThread.cpp > 261 _wakeup_monitor = new Monitor(Mutex::nonleaf, "Wakeup Monitor for Touch Control", true, Monitor::_safepoint_check_never); > 262 _sync_monitor = new Monitor(Mutex::nonleaf, "Sync Monitor for Touch Control", true, Monitor::_safepoint_check_never); > > For singleton mutex/monitor, we generally prefer to put them in mutexLocker. > Probably passed as constructor arguments to G1NUMAThreadsControl. And > I know there are counter-examples. Not necessary as memory touch thread is gone. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp > 30 extern const uint AnyNodeIndex; > 31 extern const uint InvalidNodeIndex; > > Why are these at namespace scope. And they seem misplaced here. > Seems like they should belong to the G1NUMA class, which is > responsible for indexing nodes and mapping between indexes and ids. Changed to G1MemoryNodeManager::AnyNodeIndex. And I wanted to hide or minimize exposing G1NUMA in general. Memory node manager also has index related APIs, so let manager have such constants seems okay. > > 30 const uint AnyNodeIndex = (uint)os::AnyId; > 31 const uint InvalidNodeIndex = (uint)os::InvalidId; > > And why these values? That seems like a units mismatch. Just use > UINT_MAX and UINT_MAX - 1. I don't have strong opinion on this. But the intend is to make same value after casting for same meaning constants instead of randomly chosen ones. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1NUMA.hpp > 80 // For invalid numa id, return InvalidNodeIndex. So the caller need exception handling. > > "need exception handling"? Thomas also commented on it so reworded. But finally the class is removed. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/heapRegionManager.cpp > 114 uint allocated_node_index; > > This variable gets assigned via an out parameter here: > 116 hr = _free_list.remove_region_with_node_index(from_head, valid_node_index, &allocated_node_index); > > and by assignment here: > 122 allocated_node_index = mgr->node_index_of_address(hr->bottom()); > > but is never used. If the lack of use is not itself a mistake, then the out > parameter of remove_region_with_node_index appears to be unnecessary. The variable will be used at patch3 (JDK-8220312) but slightly changed to allow NULL at allocated_node_index parameter at remove_region_with_node_index(). > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/heapRegionSet.inline.hpp > 151 inline HeapRegion* FreeRegionList::remove_region_with_node_index(bool from_head, > > As written, I think this might always return NULL if requested_node_index is > not a valid index. I think callers ensure that it's valid, but an assert of > the argument would be help. Done > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/heapRegionSet.inline.hpp > 175 if (cur == NULL || cur_depth >= max_search_depth) { // not found > > There's an off-by-one issue here. The max_search_depth time through the > while-control-expression may have found a matching region, checked via the > numa_index_of_address lookup. > > Either the test should be "current_numa_index != requested_numa_index" or > the loop should be changed. This code is removed. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/heapRegionSet.inline.hpp > 151 inline HeapRegion* FreeRegionList::remove_region_with_node_index(bool from_head, > > I think the list manipulation could be simplified quite a bit. I think we > don't need the "last" variable. And the splicing out of cur can be unified > between the from_head and !from_head cases, e.g. something like this: > > // Find the region to use, searching from _head or _tail as requested. > if (from_head) { > for (cur = _head; > cur != NULL && cur_depth < max_search_depth; > cur = cur->next(), ++cur_depth) { > if (current_numa_index == numa->numa_index_of_address(cur->bottom)) { > break; > } > } > } else { > for (cur = _tail; > cur != NULL && cur_depth < max_search_depth; > cur = cur->prev(), ++cur_depth) { > if (current_numa_index == numa->numa_index_of_address(cur->bottom)) { > break; > } > } > } > if (cur == NULL || cur_depth >= max_search_depth) { > return NULL; // Didn't find a region to use > } > // Splice the region out of the list. > HeapRegion* prev = cur->prev(); > HeapRegion* next = cur->next(); > if (prev == NULL) { > _head = next; > } else { > prev->set_next(next); > } > if (next == NULL) { > _tail = prev; > } else { > next->set_prev(prev); > } > cur->set_prev(NULL); > cur->set_next(NULL); > ... Nice suggestion! > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1NUMA.cpp > 166 class DumpIterator : public StackObj { > > Polluting global namespace. Suggest making DumpIterator and the classes > derived from it (G1HeapRegion{Larger,Smaller}Iterator) private nested > classes of G1NUMA, making the names unimportant outside of that context. Not need any more. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/heterogeneousHeapRegionManager.hpp > 122 virtual HeapRegion* allocate_free_region(HeapRegionType type); > 123 virtual HeapRegion* allocate_free_region(HeapRegionType type, uint node_index); > > The base class (HeapRegionManager) no longer has the one-arg function. So > this one-arg function is no longer overriding the base class definition. > (Too bad it wasn't declared as C++11 "override".) > > I don't see any one-arg callers, so seems like it can just be removed. DONE Nice finding! :) > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1PageBasedNUMAVirtualSpace.cpp > > I dislike the new structure here, with G1PageBasedVirtualSpace now being > both a base class and a concrete class (which is often a problematic > design). And the new derived class doesn't really do all that much. > > It seems like it would be much simpler to just have G1PageBasedVirtualSpace > check UseNUMA to conditionalize it's commit and uncommit functions. > Especially since it's commit already needs to have some knowlege of NUMA > because of the new numa_index argument. DONE Removed NUMA version and reverted related changes to allow inherit are reverted. Now, G1PageBasedVirtualSpace does additional work after checking NUMA. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1PageBasedNUMAVirtualSpace.cpp > 52 void G1PageBasedNUMAVirtualSpace::uncommit(size_t start_page, size_t size_in_pages) { > > Do we actually need to do anything here? We're resetting the > G1NUMA::_numa_id_of_pages_table (to AnyId), but it's not clear we really > need to do that. When we later recommit any of those pages, that table will > be updated appropriately. > > Maybe rather than having G1NUMA expose quite so much detail it should > instead have post_commit and a pre_uncommit functions. I think clearing the node index may help for debugging. Unlike webrev.0, numa id/index information is located at HeapRegion which is much simpler. i.e. HeapRegionManager::uncommit_regions() does resetting node index to InvalidNodeIndex. > > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1NUMA.hpp > 75 G1NUMAPageIdTable _numa_id_of_pages_table; > > I was surprised by the need for this, but I can't find anything in the NUMA > API that provides this information. That seems strange, because one would > think that information must exist somewhere in the NUMA implementation. Removed. And this made the patch much simpler! And here are your comments from other email. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp > 64 // Returns node index of the given address. > 65 // If the node id of the address is invalid return randomly selected node index. > 66 virtual uint valid_index_of_address(HeapWord* o) const { return 0; } > > This function is unused. Removed. > ------------------------------------------------------------------------------ > src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp > 70 virtual uint node_index_of_address(HeapWord* o) const { return 0; } > > The only caller of this function is HeapRegionManager::allocate_free_region. > As noted in an earlier comment, the result of that call is unused. Removed as you commented but I had to revert as a part of _numa_id_of_pages_table removal. Previously we retrieved the page info from _numa_id_of_pages_table, but new version reads directly via system call. > > ------------------------------------------------------------------------------ > > As pointed out to me offline, there*are* ways to query the address -> > associated numa node mapping. So I'm going back and questioning the need > for _numa_id_of_pages_table and the code for managing it. As we discussed offline, I removed this table. So webrev.1 checks the bottom of HeapRegion and then saves it at HeapRegion::_node_index. However we do need a way to check node index of the given address and so I added back. From my testing, checking a numa id of one page seems okay to do on runtime. Previously I did checking for all pages of given memory so it was a bit slow. One thing that didn't include at webrev.1 is combining with existing one. The newly introduced os::numa_get_address_id() is a copy from zNUMA_linux.cpp. It may be unified but didn't try yet. webrev: http://cr.openjdk.java.net/~sangheki/8220310/webrev.1 http://cr.openjdk.java.net/~sangheki/8220310/webrev.1.inc (this may not help much! :) ) Testing: hs-tier 1 ~ 5 (with/without UseNUMA) Thanks, Sangheon > > ------------------------------------------------------------------------------ > From zgu at redhat.com Sat Sep 21 17:27:50 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Sat, 21 Sep 2019 13:27:50 -0400 Subject: RFR 8231293: Shenandoah: Traversal fails JVMTI tests Message-ID: <638b42c4-96dc-e2e8-f12d-2958ce36878c@redhat.com> The problem is caused by LRB trying to evacuate dead oops. At this point, traversal is completed, therefore, no more evacuation should happen. We should complete the marking bitmap and arm native barrier to prevent from-space oops from leaking to system. There are duplicated works among fixup roots and parallel cleaning, let's address them in DKJDK-8231324. Bug: https://bugs.openjdk.java.net/browse/JDK-8231293 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8231293/webrev.00/index.html Test: hotspot_gc_shenandoah (fastdebug and release) vmTestbase/nsk/jvmti (fastdebug and release) normal and traversal modes. Thanks, -Zhengyu From shade at redhat.com Mon Sep 23 07:49:14 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 23 Sep 2019 09:49:14 +0200 Subject: RFR 8231293: Shenandoah: Traversal fails JVMTI tests In-Reply-To: <638b42c4-96dc-e2e8-f12d-2958ce36878c@redhat.com> References: <638b42c4-96dc-e2e8-f12d-2958ce36878c@redhat.com> Message-ID: <1b6cc097-54f9-fd1c-a946-7b5a94cb0e19@redhat.com> On 9/21/19 7:27 PM, Zhengyu Gu wrote: > The problem is caused by LRB trying to evacuate dead oops. > At this point, traversal is completed, therefore, no more evacuation should happen. We should > complete the marking bitmap and arm native barrier to prevent from-space oops from leaking to system. > > There are duplicated works among fixup roots and parallel cleaning, let's address them in > DKJDK-8231324. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231293 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8231293/webrev.00/index.html Please change the synopsis to something that reflects the change being done, not the symptoms. For example: "Shenandoah: Traversal should not revive dead weak roots" Otherwise looks fine. -- Thanks, -Aleksey From stefan.karlsson at oracle.com Mon Sep 23 08:24:28 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 23 Sep 2019 10:24:28 +0200 Subject: RFR: 8231266: ZGC: Minor cleanups in ZCollectedHeap and ZHeap In-Reply-To: <40e2f38a-582c-8a06-670e-dd51f8b042d4@oracle.com> References: <40e2f38a-582c-8a06-670e-dd51f8b042d4@oracle.com> Message-ID: <1a1c916e-12c3-8d33-0e5f-0ef202d7ee97@oracle.com> Looks good. StefanK On 2019-09-19 23:07, Per Liden wrote: > ZCollectedHeap's is_oop() and print_location() should delegate to ZHeap. > Also adjusting a few types, so that the ZHeap interface uses uintptr_t. > This keeps things consistent and avoids a few casts in ZHeap. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231266 > Webrev: http://cr.openjdk.java.net/~pliden/8231266/webrev.0 > > /Per From thomas.schatzl at oracle.com Mon Sep 23 09:44:50 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 23 Sep 2019 11:44:50 +0200 Subject: RFR (M): 8159984: Remove call to ClassLoaderDataGraph::clear_claimed_marks during the initial mark pause In-Reply-To: References: <4e851c12-ca88-5d2d-f636-767437ad5a01@oracle.com> <52dd1037-00b6-86a6-0222-3a7807c5d2c9@oracle.com> <22E42951-E28C-478A-9D9B-6FBD320335B5@oracle.com> <6aa7ec98-a886-2318-a884-04e79744c84f@oracle.com> Message-ID: <45ceb0d7-7a55-c2bc-739e-bb86dea07c0b@oracle.com> Hi, On 20.09.19 20:50, Kim Barrett wrote: >> On Sep 20, 2019, at 8:00 AM, Stefan Johansson wrote: >> Looks good, just one minor comment that you can ignore if you don't agree and no need for a new webrev. I think the parameter notify_claimed_roots_done should also follow the "nmethod naming scheme?. > > +1 > pushed after fixing this and some local verification. Thanks, Thomas From thomas.schatzl at oracle.com Mon Sep 23 10:53:37 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 23 Sep 2019 12:53:37 +0200 Subject: RFC: JEP: Deprecate the Parallel Scavenge / Serial Old collector combination Message-ID: Hi all, the Oracle gc team would like to deprecate the parallel scavenge / serial old collector combination enabled by using -XX:+UseParallelGC -XX:-UseParallelOldGC. This seems to be an extremely rarely used collector combination, as at last OCW in August with ~100? Java users/devs nobody seemed to know it, but it still takes some resources away that may be used for other issues. This is only the *deprecation* effort - there is no change in functionality except for a "this collector combination is deprecated"2 at this time. I wrote up a JEP draft at https://bugs.openjdk.java.net/browse/JDK-8229492 . Comments and reviewers to move it along appreciated. Thanks, Thomas From shade at redhat.com Mon Sep 23 12:14:56 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 23 Sep 2019 14:14:56 +0200 Subject: RFR (S) 8231249: Shenandoah: GC retries are too aggressive for tests that expect OOME Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8231249 This readily reproduces with vmTestbase_nsk_jvmti. Some tests (notably vmTestbase/nsk/jvmti/GarbageCollectionStart/gcstart001) are filling up the memory until OOME happens, and even count the OOMEs happened. Unfortunately, that clashes with Shenandoah's quite aggressive retry policy. At some point, test is able to sneak past allocation failure and make progress, and less aggressive policy helps that. Fix: http://cr.openjdk.java.net/~shade/8231249/webrev.01/ Since Degenerated GC would dive into Full GC on low progress, we can just trust the flag for retries. We also need to poll the flag after at least one attempt at handling the alloc failure (thus entering degen-full sequence). This makes retry logic less aggressive, which saves us from test timeouts. Testing: vmTestbase_nsk_jvmti; hotspot_gc_shenandoah -- Thanks, -Aleksey From per.liden at oracle.com Mon Sep 23 13:15:57 2019 From: per.liden at oracle.com (Per Liden) Date: Mon, 23 Sep 2019 15:15:57 +0200 Subject: RFR: 8231266: ZGC: Minor cleanups in ZCollectedHeap and ZHeap In-Reply-To: <1a1c916e-12c3-8d33-0e5f-0ef202d7ee97@oracle.com> References: <40e2f38a-582c-8a06-670e-dd51f8b042d4@oracle.com> <1a1c916e-12c3-8d33-0e5f-0ef202d7ee97@oracle.com> Message-ID: <88491c3f-b451-3d5e-967b-eb0de9be7714@oracle.com> Thanks Stefan! /Per On 9/23/19 10:24 AM, Stefan Karlsson wrote: > Looks good. > > StefanK > > On 2019-09-19 23:07, Per Liden wrote: >> ZCollectedHeap's is_oop() and print_location() should delegate to >> ZHeap. Also adjusting a few types, so that the ZHeap interface uses >> uintptr_t. This keeps things consistent and avoids a few casts in ZHeap. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231266 >> Webrev: http://cr.openjdk.java.net/~pliden/8231266/webrev.0 >> >> /Per From rkennke at redhat.com Mon Sep 23 14:00:10 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 23 Sep 2019 16:00:10 +0200 Subject: RFR 8231293: Shenandoah: Traversal fails JVMTI tests In-Reply-To: <638b42c4-96dc-e2e8-f12d-2958ce36878c@redhat.com> References: <638b42c4-96dc-e2e8-f12d-2958ce36878c@redhat.com> Message-ID: <15c5abd9-d839-a294-fdaf-de0f428059b8@redhat.com> Why is that a problem? As far as I know, we don't concurrently clean native roots with traversal GC. Or do we? Roman > The problem is caused by LRB trying to evacuate dead oops. > At this point, traversal is completed, therefore, no more evacuation > should happen. We should complete the marking bitmap and arm native > barrier to prevent from-space oops from leaking to system. > > There are duplicated works among fixup roots and parallel cleaning, > let's address them in DKJDK-8231324. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231293 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8231293/webrev.00/index.html > > Test: > ? hotspot_gc_shenandoah (fastdebug and release) > ? vmTestbase/nsk/jvmti (fastdebug and release) normal and traversal > ? modes. > > Thanks, > > -Zhengyu > From rkennke at redhat.com Mon Sep 23 14:01:01 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 23 Sep 2019 16:01:01 +0200 Subject: RFR (S) 8231249: Shenandoah: GC retries are too aggressive for tests that expect OOME In-Reply-To: References: Message-ID: <2408462d-1911-adf2-4626-c83f971f3471@redhat.com> Ok. Thanks! Roman > Bug: > https://bugs.openjdk.java.net/browse/JDK-8231249 > > This readily reproduces with vmTestbase_nsk_jvmti. Some tests (notably > vmTestbase/nsk/jvmti/GarbageCollectionStart/gcstart001) are filling up the memory until OOME > happens, and even count the OOMEs happened. Unfortunately, that clashes with Shenandoah's quite > aggressive retry policy. At some point, test is able to sneak past allocation failure and make > progress, and less aggressive policy helps that. > > Fix: > http://cr.openjdk.java.net/~shade/8231249/webrev.01/ > > Since Degenerated GC would dive into Full GC on low progress, we can just trust the flag for > retries. We also need to poll the flag after at least one attempt at handling the alloc failure > (thus entering degen-full sequence). This makes retry logic less aggressive, which saves us from > test timeouts. > > Testing: vmTestbase_nsk_jvmti; hotspot_gc_shenandoah > From zgu at redhat.com Mon Sep 23 14:21:53 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 23 Sep 2019 10:21:53 -0400 Subject: RFR 8231293: Shenandoah: Traversal fails JVMTI tests In-Reply-To: <15c5abd9-d839-a294-fdaf-de0f428059b8@redhat.com> References: <638b42c4-96dc-e2e8-f12d-2958ce36878c@redhat.com> <15c5abd9-d839-a294-fdaf-de0f428059b8@redhat.com> Message-ID: <9547881f-5663-a6ba-c42e-f1a05fe4b6ad@redhat.com> On 9/23/19 10:00 AM, Roman Kennke wrote: > Why is that a problem? > > As far as I know, we don't concurrently clean native roots with > traversal GC. Or do we? No. But when walking jvmti weak roots (in fixup roots or parallel cleaning), it triggers load barrier. -Zhengyu > > Roman > >> The problem is caused by LRB trying to evacuate dead oops. >> At this point, traversal is completed, therefore, no more evacuation >> should happen. We should complete the marking bitmap and arm native >> barrier to prevent from-space oops from leaking to system. >> >> There are duplicated works among fixup roots and parallel cleaning, >> let's address them in DKJDK-8231324. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231293 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8231293/webrev.00/index.html >> >> Test: >> ?? hotspot_gc_shenandoah (fastdebug and release) >> ?? vmTestbase/nsk/jvmti (fastdebug and release) normal and traversal >> ?? modes. >> >> Thanks, >> >> -Zhengyu >> From zgu at redhat.com Mon Sep 23 14:27:55 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 23 Sep 2019 10:27:55 -0400 Subject: RFR 8231293: Shenandoah: Traversal fails JVMTI tests In-Reply-To: <15c5abd9-d839-a294-fdaf-de0f428059b8@redhat.com> References: <638b42c4-96dc-e2e8-f12d-2958ce36878c@redhat.com> <15c5abd9-d839-a294-fdaf-de0f428059b8@redhat.com> Message-ID: <492a9c61-761a-de2e-bfbd-29ff0866d512@redhat.com> Hi Roman, You may have a point, I wonder if it is an upstream bug. JvmtiTagMap::do_weak_oops() -> entry->object_peek() triggers native load barrier, while here should probably be a RawAccess. -Zhengyu On 9/23/19 10:00 AM, Roman Kennke wrote: > Why is that a problem? > > As far as I know, we don't concurrently clean native roots with > traversal GC. Or do we? > > Roman > >> The problem is caused by LRB trying to evacuate dead oops. >> At this point, traversal is completed, therefore, no more evacuation >> should happen. We should complete the marking bitmap and arm native >> barrier to prevent from-space oops from leaking to system. >> >> There are duplicated works among fixup roots and parallel cleaning, >> let's address them in DKJDK-8231324. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231293 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8231293/webrev.00/index.html >> >> Test: >> ?? hotspot_gc_shenandoah (fastdebug and release) >> ?? vmTestbase/nsk/jvmti (fastdebug and release) normal and traversal >> ?? modes. >> >> Thanks, >> >> -Zhengyu >> From stefan.johansson at oracle.com Mon Sep 23 15:14:36 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 23 Sep 2019 17:14:36 +0200 Subject: RFR(XL): 8220310: Implementation: NUMA-Aware Memory Allocation for G1, Mutator (1/3) In-Reply-To: <2b37edd6-3e0f-013d-1616-9d003f8ac1ed@oracle.com> References: <06ACBF87-ADBE-499F-B668-0274E4925B26@oracle.com> <2b37edd6-3e0f-013d-1616-9d003f8ac1ed@oracle.com> Message-ID: Hi Sangheon, Thanks for the updated webrev, I like where we are heading with this. I still have some comments but most of them are fairly small. The only big thing is that I still think we could simplify things a lot by not passing down node_index to the lower level commit functions and instead just focus on keeping a strict manual interleaving of regions/pages (whichever is largest). I know this wouldn't allow us to expand by a single region on a specific node, but since this is a fairly uncommon case I think we could live with that. src/hotspot/share/gc/g1/g1Allocator.* ------------------------------------- There are a few calls like this: uint node_index = _g1h->mem_node_mgr()->index_of_current_thread(); I would like a private helper to wrap this, something like current_node_index(). --- hotspot/share/gc/g1/g1Arguments.cpp ----------------------------------- 161 if (UseNUMA) { 162 if (FLAG_IS_DEFAULT(AlwaysPreTouch)) { 163 FLAG_SET_DEFAULT(AlwaysPreTouch, true); 164 } 165 if (!AlwaysPreTouch && FLAG_IS_CMDLINE(AlwaysPreTouch)) { 166 warning("Disabling AlwaysPreTouch is incompatible with UseNUMA. Disabling UseNUMA."); 167 FLAG_SET_ERGO(UseNUMA, false); 168 } 169 } I'm not sure I understand why we have to enforce AlwaysPreTouch when NUMA is enabled. This will improve performance, but I think the NUMA support should work even without it. Or am I missing something? --- src/hotspot/share/gc/g1/g1CollectedHeap.* ----------------------------------------- As we discussed offline I'm not sure we need pass the node_index down in the call to expand. Since we're not strict in giving out memory from the correct node during the mutator phase, I think we could do the same during the GC in the case we need to expand. --- src/hotspot/share/gc/g1/g1RegionToSpaceMapper.cpp ------------------------------------------------- 162 G1NodeDistributor itr(node_index); 163 164 for (uint i = start_idx; i < start_idx + num_regions; i++) { 165 // If there are many pages to touch, different node ids will be used. 166 uint processed_node_index = itr.next_node_index(); ... 179 zero_filled = _storage.commit(idx, 1, processed_node_index); 180 itr.next(); 181 } Do we really need to do this? Can't we just rely on the code in G1NUMA::touch_memory_roundrobin() to do the right thing? Or is this just to allow committing a single region on the correct17134 node? --- src/hotspot/share/gc/g1/g1_globals.hpp -------------------------------------- Instead of adding those flags I think we should use log-tags and guard the logic by using log_is_enabled(), or LogTarget::is_enable() as you do in heapRegionManager.cpp. To allow these to be excluded with other logging turned on I suggest using specific log-tag-sets for each of them something like: G1PrintNUMAIdOfHeapRegions -> gc, numa, region (debug or trace) G1VerifyNUMAIdOfHeapRegions -> gc, numa, region, verification (debug or trace) --- src/hotspot/share/gc/g1/heapRegionManager.cpp --------------------------------------------- 112 if (mgr->num_active_nodes() > 1) { 113 uint valid_node_index = mgr->valid_node_index(requested_node_index); 114 // Try to allocate with requested node index. 115 hr = _free_list.remove_region_with_node_index(from_head, valid_node_index, NULL); 116 } I've seen some problems in my testing with this code, valid_node_index() now call _numa->next_numa_index() when the index isn't valid. This means that both the lower level commit code will call this function as well as the allocation path in some cases (for example single region humongous). I'm not sure if this will lead to any real problems but it makes it harder to understand what is going on underneath and I've seen it lead to having imbalance over the nodes. To avoid this I think we could instead do: if (mgr->num_active_nodes() > 1 && mgr->is_valid_node_index(requested_node_index) { hr = _free_list.remove_region_with_node_index(from_head, requested_node_index, NULL); } This way we only remove with index if we have a valid index, otherwise we get a HeapRegion from the code below, which should be more or less the same as taking whatever index and removing with that: 118 if (hr == NULL) { ... 121 hr = _free_list.remove_region(from_head); 122 } If this is an ok approach I think valid_node_index() is not longer used and can be removed. --- src/hotspot/share/gc/g1/g1NUMA.inline.hpp ----------------------------------------- G1NUMA::next_numa_id() seems to be unused and could then be removed. --- I'll continue looking at the patch and taking it for some spins. I might come with additional feedback later on, but these are my initial findings. Thanks, Stefan On 2019-09-21 07:19, sangheon.kim at oracle.com wrote: > Hi Kim, > > Many thanks for this thorough review! > > On 9/16/19 4:12 PM, Kim Barrett wrote: >>> On Sep 4, 2019, at 3:16 AM, sangheon.kim at oracle.com wrote: >>> >>> Hi all, >>> >>> Please review this patch, making G1 NUMA aware. >>> This is the first part of G1 NUMA implementation. >>> >>> - At G1 CollectedHeap initialization time, the given heap regions >>> will be split and touched. >>> - Internally node index(uint) is used instead of node id(int type) >>> for easier access to arrays. >>> - Only Linux is supported. >>> - New logs are under gc+heap+numa >>> - Current memory touch implementation will be replaced with WorkGang >>> by JDK-8230411: NUMA aware work gang. >>> >>> CR: https://bugs.openjdk.java.net/browse/JDK-8220310 >>> Webrev: http://cr.openjdk.java.net/~sangheki/8220310/webrev.0 >>> Testing: hs-tier 1 ~ 5 with +- UseNUMA. >>> >>> Thanks, >>> Sangheon > I had long off-line discussion with Kim, Thomas and Stefan J. > And 2 major changes are: > > 1) Memory touch thread is removed. This was a general way to implement > for both Linux and Solaris. However, Solaris is not supported anymore, > so memory touch thread is just removed. > And this made the code much simpler. > > 2) G1NUMA doesn't manage numa id of each page and we don't care about > each page instead HeapRegion is only cared. i.e. we assume numa id of > all pages between bottom and end of HeapRegion to be same. This also > made related codes much simpler. > > There are some other changes as well. > > >> Here is an initial set of comments.? Some of these might be nonsense; I'm >> not all that familiar with NUMA support. >> >> ------------------------------------------------------------------------------ >> >> >> Just in general, it's kind of disappointing that NUMA support for G1 >> apparently involves this much code, and is completely different from >> ParallelGC NUMA support, and both are completely different from ZGC >> NUMA support.? (And Shenandoah is probably yet another still.) >> Oh well... > Yes, I agree on that. > >> >> ------------------------------------------------------------------------------ >> >> >> According to off-line discussion, the intent is that a HeapRegion only >> contains memory from one NUMA node.? That isn't obvious though, and there >> are API and data structure choices that obscure that.? It seems like some >> things could be simplified with that constraint in mind.? I think some of >> that is a result of trying to leave open support for NUMA aware off-heap >> data structures, but that isn't part of this change, and might be better >> kept separate from NUMA aware Java heap support. >> >> I'll try to point out places where I think there are opportunities for >> simplification. >> >> ------------------------------------------------------------------------------ >> >> >> According to off-line discussion, the touching mechanism might not be >> necessary.? If that's true, that would be a significant >> simplification.? For >> now I won't comment on the code for the touch threads, though I think >> I found some problems there before the discussion suggesting we might >> not need them happened. > Thanks. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1NUMA.hpp >> ?? 64 // Manages NUMA node related information and memory touch threads. >> ... >> >> I found the comment describing G1NUMA pretty hard to parse.? Even >> after looking at the code I'm having trouble figuring out what's being >> said. >> >> This part I was able to figure out, but think it could be worded better. >> ?? 70 // Also provides a way to convert NUMA id and NUMA >> index(starting from zero and sequatial). >> ?? 71 // Actual NUMA ids would not be sequential nor not start from >> zero, so a conversion is necessary >> ?? 72 // between id and index. >> >> Something like >> >> ?? Also provides a mapping between NUMA indices (sequential, starting >> ?? from zero) and NUMA ids (which may not start at zero and may not be >> ?? sequential).? Indices are generally used by clients, and mapped to >> ?? ids when dealing with the OS/hardware layer. > DONE > >> >> (I *think* that's true.) >> >> But then, why expose node_ids at all in G1NUMA? It seems like node_ids >> could be completely relegated to an implementation detail that clients >> never need to deal with. > ... >> As evidence for not needing to expose ids to clients, there are no >> callers of numa_id_of_index in any of the series of 3 patches.? And >> G1NUMA::numa_ids() and G1MemoryMultNodeManager::node_ids() are only >> used in a couple of places for printing/logging and by >> WB_G1MemoryNodeIds.? I'm not sure what the last is for, but the >> printing and logging code could be isolated into G1NUMA. > Basically I totally agree with you for not exposing numa id. > Current patches are only using node id for logging purpose at some > locations because users are only interested on node id. > > Also Thread::lgrp_id() (renaming lgrp id, numa id etc.. is different > topic that we had before) is also public, so my feeling is current try > is enough. :) > > WB_G1MemoryNodeIds() is for jtreg testing purpose which checks whether > the given memory has node id as we expected. > >> >> Also, it looks like the index to node_id mapping provided here is >> duplicating the information provided by os::Linux::nindex_to_node. >> Not that this necessarily helps with anything in generic code... > Yes, I'm aware of _nindex_to_node but I think maintaining such > conversion at G1NUMA is okay. Other NUMA implementation such as parallel > gc relies on node id so completely removing node id at general code > seems different scope. > Unless you have strong opinion on addressing it in this patch, I would > like proceed as is. > If it is the case, we can consider your suggestion as a separate > enhancement. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >> ?? 57?? virtual int* node_ids() const { static int dummy_id = 0; >> return &dummy_id; } >> >> Can the return type here be "const int*" ?? That would be better if >> possible. >> >> [This is probably moot if clients only deal with numa indexes and not >> node_ids, as suggested earlier.] > DONE > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >> ?? 63?? virtual uint guarantee_valid_node_index(uint node_index) const >> { return 0; } >> >> Given naming conventions elsewhere, the name of this function makes me >> expect it to be a verification function. > Agree. > valid_node_index() is better? webrev.1 changed name. > Or are you suggesting to remove this and then the caller do such thing? > e.g. > void A(uint node_index) { > uint valid_node_index = node_index; > if (!is_valid_node_index(valid_node_index)) { > ? valid_node_index = random_numa_index(); > ... > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1NUMA.inline.hpp >> ?? 29 #include "gc/shared/taskqueue.inline.hpp" >> >> This #include seems to be just to obtain randomParkAndMiller. It would be >> better to refactor that function into its own file(s). Or make it part of >> some component providing various random number generators. > Not need because of below change. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1NUMA.inline.hpp >> ?? 45?? int seed = randomParkAndMiller(_seed); >> ?? 46?? Atomic::store(seed, &_seed); >> >> This sequence may lose updates.? I don't know if that's important. >> And why isn't this just using os::random(), or maybe refactoring that >> slightly to gain access to the external seed version.? I don't want this >> change to be made: >> http://cr.openjdk.java.net/~sangheki/8220310/webrev.0/src/hotspot/share/gc/shared/taskqueue.inline.hpp.udiff.html >> > Thomas and Stefan J. also complained on this. > Changed to get next numa index with round-robin manner. i.e. doesn't use > random() at all. > >> And do we *really* need randomness here at all?? Is there a reason not >> to simply cycle through the nodes? > Not really. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/os/linux/os_linux.cpp >> 3016?? size_t num_pages_left = size_in_bytes / page_size; >> >> Is size_in_bytes required to be a multiple of page_size?? I believe it >> is, >> and an assert of that would be useful. > This code is removed. And yes to answer to your question. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/os/linux/os_linux.cpp >> 3016?? size_t num_pages_left = size_in_bytes / page_size; >> 3017?? const size_t num_pages_total = num_pages_left; >> 3018?? size_t num_buffer_entries = MIN2(num_pages_left, max_pages); >> >> I think clearer would be >> >> 3017?? const size_t num_pages_total = size_in_bytes / page_size; >> 3018?? size_t num_buffer_entries = MIN2(num_pages_total, max_pages); >> >> and move this down to the loop setup: >> 3016?? size_t num_pages_left = num_pages_total; > This code is removed. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/os/linux/os_linux.cpp >> 3020?? void** pages = (void**)alloca(sizeof(void*) * num_buffer_entries); >> 3021?? int* status = (int*)alloca(sizeof(int) * num_buffer_entries); >> >> I think a ResourceMark and NEW_RESOURCE_ARRAYs would be preferable to >> using alloca() here.? alloca() has some dangers and I think is currently >> only used in some very specialized places where it really is needed. > This code is removed. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/os/linux/os_linux.cpp >> 3061 } >> 3062 // Moves current thread on a specific node and it will not >> migrate to >> >> Missing blank line between functions. > This code is removed. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/os/linux/os_linux.cpp >> 3031?????? pages[i] = (void*)base; >> >> Unnecessary cast. > This code is removed. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/os/linux/os_linux.cpp >> 3072???? case AFFINITY_NONE: >> 3073?????? // Do nothing. >> 3074?????? return true; >> 3075?????? break; >> >> Shouldn't this be calling numa_run_on_node with -1 node id?? The >> documentation for numa_run_on_node() says "Passing -1 permits the >> kernel to schedule on all nodes again." > My understand of that description is let kernel decide the numa node > which is not actually want here. We want to move current thread to the > preferred numa node. > > Currently we don't need this API (numa_set_affinity() ) so removed on > webrev.1. > However following patch which is not scope of this JEP will use it. > I will add your suggestion below? when we need this. > >> >> But see also next comment about numa_affinity_set. >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/runtime/os.hpp >> ? 401?? // Available affinities. >> ? 402?? enum NumaAffinity { >> ? 403???? AFFINITY_STRONG, >> ? 404???? AFFINITY_NONE >> ? 405?? }; >> ? 406?? static bool numa_affinity_set(Thread* thread, int numa_id, >> NumaAffinity affinity); >> >> Could there ever be other affinities?? This API doesn't seem right. >> It seems to me it should be something like >> >> ?? static bool numa_set_affinity(Thread* thread, int numa_id); >> ?? static bool numa_clear_affinity(Thread* thread); >> >> No need for NumaAffinity enum. > The enum is also kind of remainder after removing Solaris implementation > which has more types. > But I agree with you as there are only 2 types now, it is simpler to > separate API without enum. > >> >> (I think numa_set/clear_affinity is better than numa_affinity_set/clear; >> numa is a "namespace" and we usually set/clear/get_something rather >> than something_set/clear/get.) >> >> Or maybe this should just be >> >> ?? static bool numa_set_affinity(Thread* thread, int numa_id); >> >> with "clearing" provided by passing AnyId as the numa_id?? Still don't >> need the NumaAffinity enum for this. > Agree on this naming stuff. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1NUMA.cpp >> ? 124?? guarantee(rs.size() % page_size == 0, "Reserved space size (" >> SIZE_FORMAT ") should be aligned " >> ? 141?? guarantee(size_in_bytes % page_size == 0, "The given range (" >> SIZE_FORMAT ") should be aligned " >> ? 209???? guarantee((uintptr_t)G1HeapRegionSize % page_size == 0, >> "G1HeapRegionSize (" SIZE_FORMAT ") and page size (" >> ? 234???? guarantee((uintptr_t)_page_size % _heap_region_size == 0, >> "G1HeapRegionSize (" SIZE_FORMAT ") and page size (" >> >> Use is_aligned.? I think casts shouldn't be needed. > These lines are removed after major refactoring. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1NUMA.cpp >> ?? 43?? _numa_id_to_index_map = NEW_C_HEAP_ARRAY_RETURN_NULL(uint, >> _len_numa_id_to_index_map, mtGC); >> ?? 44?? memset(_numa_id_to_index_map, 0, sizeof(uint) * >> _len_numa_id_to_index_map); >> >> Using NEW_xxx_RETURN_NULL and then ignoring the possibility of a NULL >> result.? It would be better to use the non-_RETURN_NULL variant and >> get a decent error message rather than the segfault from attemptint to >> write to NULL. > Changed to use non-_RETURN_NULL version. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1NUMA.cpp >> ?? 34 void G1NUMA::init_numa_id_to_index_map(int* numa_ids, uint >> num_numa_ids) { >> >> int* numa_ids => const int* numa_ids. > DONE > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1NUMA.cpp >> ?? 34 void G1NUMA::init_numa_id_to_index_map(int* numa_ids, uint >> num_numa_ids) { >> ... >> ?? 44?? memset(_numa_id_to_index_map, 0, sizeof(uint) * >> _len_numa_id_to_index_map); >> Rather than this memset and then writing to all of them later, instead >> initialize all entries to InvalidNodeIndex and then assign from numa_ids. > DONE > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1MemoryTouchThread.cpp >> ? 261?? _wakeup_monitor = new Monitor(Mutex::nonleaf, "Wakeup Monitor >> for Touch Control", true, Monitor::_safepoint_check_never); >> ? 262?? _sync_monitor = new Monitor(Mutex::nonleaf, "Sync Monitor for >> Touch Control", true, Monitor::_safepoint_check_never); >> >> For singleton mutex/monitor, we generally prefer to put them in >> mutexLocker. >> Probably passed as constructor arguments to G1NUMAThreadsControl.? And >> I know there are counter-examples. > Not necessary as memory touch thread is gone. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >> ?? 30 extern const uint AnyNodeIndex; >> ?? 31 extern const uint InvalidNodeIndex; >> >> Why are these at namespace scope.? And they seem misplaced here. >> Seems like they should belong to the G1NUMA class, which is >> responsible for indexing nodes and mapping between indexes and ids. > Changed to G1MemoryNodeManager::AnyNodeIndex. > And I wanted to hide or minimize exposing G1NUMA in general. > Memory node manager also has index related APIs, so let manager have > such constants seems okay. > >> >> ?? 30 const uint AnyNodeIndex = (uint)os::AnyId; >> ?? 31 const uint InvalidNodeIndex = (uint)os::InvalidId; >> And why these values?? That seems like a units mismatch.? Just use >> UINT_MAX and UINT_MAX - 1. > I don't have strong opinion on this. > But the intend is to make same value after casting for same meaning > constants instead of randomly chosen ones. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1NUMA.hpp >> ?? 80?? // For invalid numa id, return InvalidNodeIndex. So the caller >> need exception handling. >> >> "need exception handling"? > Thomas also commented on it so reworded. But finally the class is removed. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/heapRegionManager.cpp >> ? 114???? uint allocated_node_index; >> >> This variable gets assigned via an out parameter here: >> ? 116???? hr = _free_list.remove_region_with_node_index(from_head, >> valid_node_index, &allocated_node_index); >> >> and by assignment here: >> ? 122???????? allocated_node_index = >> mgr->node_index_of_address(hr->bottom()); >> >> but is never used. If the lack of use is not itself a mistake, then >> the out >> parameter of remove_region_with_node_index appears to be unnecessary. > The variable will be used at patch3 (JDK-8220312) but slightly changed > to allow NULL at allocated_node_index parameter at > remove_region_with_node_index(). > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/heapRegionSet.inline.hpp >> ? 151 inline HeapRegion* >> FreeRegionList::remove_region_with_node_index(bool from_head, >> >> As written, I think this might always return NULL if >> requested_node_index is >> not a valid index. I think callers ensure that it's valid, but an >> assert of >> the argument would be help. > Done > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/heapRegionSet.inline.hpp >> ? 175???? if (cur == NULL || cur_depth >= max_search_depth) { // not >> found >> >> There's an off-by-one issue here. The max_search_depth time through the >> while-control-expression may have found a matching region, checked via >> the >> numa_index_of_address lookup. >> >> Either the test should be "current_numa_index != requested_numa_index" or >> the loop should be changed. > This code is removed. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/heapRegionSet.inline.hpp >> ? 151 inline HeapRegion* >> FreeRegionList::remove_region_with_node_index(bool from_head, >> >> I think the list manipulation could be simplified quite a bit.? I >> think we >> don't need the "last" variable.? And the splicing out of cur can be >> unified >> between the from_head and !from_head cases, e.g. something like this: >> >> ?? // Find the region to use, searching from _head or _tail as requested. >> ?? if (from_head) { >> ???? for (cur = _head; >> ????????? cur != NULL && cur_depth < max_search_depth; >> ????????? cur = cur->next(), ++cur_depth) { >> ?????? if (current_numa_index == >> numa->numa_index_of_address(cur->bottom)) { >> ???????? break; >> ?????? } >> ???? } >> ?? } else { >> ???? for (cur = _tail; >> ????????? cur != NULL && cur_depth < max_search_depth; >> ????????? cur = cur->prev(), ++cur_depth) { >> ?????? if (current_numa_index == >> numa->numa_index_of_address(cur->bottom)) { >> ???????? break; >> ?????? } >> ???? } >> ?? } >> ?? if (cur == NULL || cur_depth >= max_search_depth) { >> ???? return NULL;????????????????????? // Didn't find a region to use >> ?? } >> ?? // Splice the region out of the list. >> ?? HeapRegion* prev = cur->prev(); >> ?? HeapRegion* next = cur->next(); >> ?? if (prev == NULL) { >> ???? _head = next; >> ?? } else { >> ???? prev->set_next(next); >> ?? } >> ?? if (next == NULL) { >> ???? _tail = prev; >> ?? } else { >> ???? next->set_prev(prev); >> ?? } >> ?? cur->set_prev(NULL); >> ?? cur->set_next(NULL); >> ?? ... > Nice suggestion! > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1NUMA.cpp >> ? 166 class DumpIterator : public StackObj { >> >> Polluting global namespace.? Suggest making DumpIterator and the classes >> derived from it (G1HeapRegion{Larger,Smaller}Iterator) private nested >> classes of G1NUMA, making the names unimportant outside of that context. > Not need any more. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/heterogeneousHeapRegionManager.hpp >> ? 122?? virtual HeapRegion* allocate_free_region(HeapRegionType type); >> ? 123?? virtual HeapRegion* allocate_free_region(HeapRegionType type, >> uint node_index); >> >> The base class (HeapRegionManager) no longer has the one-arg >> function.? So >> this one-arg function is no longer overriding the base class definition. >> (Too bad it wasn't declared as C++11 "override".) >> >> I don't see any one-arg callers, so seems like it can just be removed. > DONE > Nice finding! :) > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1PageBasedNUMAVirtualSpace.cpp >> >> I dislike the new structure here, with G1PageBasedVirtualSpace now being >> both a base class and a concrete class (which is often a problematic >> design).? And the new derived class doesn't really do all that much. >> >> It seems like it would be much simpler to just have >> G1PageBasedVirtualSpace >> check UseNUMA to conditionalize it's commit and uncommit functions. >> Especially since it's commit already needs to have some knowlege of NUMA >> because of the new numa_index argument. > DONE > Removed NUMA version and reverted related changes to allow inherit are > reverted. > Now, G1PageBasedVirtualSpace does additional work after checking NUMA. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1PageBasedNUMAVirtualSpace.cpp >> ?? 52 void G1PageBasedNUMAVirtualSpace::uncommit(size_t start_page, >> size_t size_in_pages) { >> >> Do we actually need to do anything here?? We're resetting the >> G1NUMA::_numa_id_of_pages_table (to AnyId), but it's not clear we really >> need to do that.? When we later recommit any of those pages, that >> table will >> be updated appropriately. >> >> Maybe rather than having G1NUMA expose quite so much detail it should >> instead have post_commit and a pre_uncommit functions. > I think clearing the node index may help for debugging. Unlike webrev.0, > numa id/index information is located at HeapRegion which is much > simpler. i.e. HeapRegionManager::uncommit_regions() does resetting node > index to InvalidNodeIndex. > >> >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1NUMA.hpp >> ?? 75?? G1NUMAPageIdTable _numa_id_of_pages_table; >> >> I was surprised by the need for this, but I can't find anything in the >> NUMA >> API that provides this information.? That seems strange, because one >> would >> think that information must exist somewhere in the NUMA implementation. > Removed. > And this made the patch much simpler! > > And here are your comments from other email. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >> ?? 64?? // Returns node index of the given address. >> ?? 65?? // If the node id of the address is invalid return randomly >> selected node index. >> ?? 66?? virtual uint valid_index_of_address(HeapWord* o) const { >> return 0; } >> >> This function is unused. > Removed. > >> ------------------------------------------------------------------------------ >> >> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >> ?? 70?? virtual uint node_index_of_address(HeapWord* o) const { return >> 0; } >> >> The only caller of this function is >> HeapRegionManager::allocate_free_region. >> As noted in an earlier comment, the result of that call is unused. > Removed as you commented but I had to revert as a part of > _numa_id_of_pages_table removal. > Previously we retrieved the page info from _numa_id_of_pages_table, but > new version reads directly via system call. > >> >> ------------------------------------------------------------------------------ >> >> >> As pointed out to me offline, there*are*? ways to query the address -> >> associated numa node mapping.? So I'm going back and questioning the need >> for _numa_id_of_pages_table and the code for managing it. > As we discussed offline, I removed this table. So webrev.1 checks the > bottom of HeapRegion and then saves it at HeapRegion::_node_index. > However we do need a way to check node index of the given address and so > I added back. > > From my testing, checking a numa id of one page seems okay to do on > runtime. Previously I did checking for all pages of given memory so it > was a bit slow. > > One thing that didn't include at webrev.1 is combining with existing one. > The newly introduced os::numa_get_address_id() is a copy from > zNUMA_linux.cpp. It may be unified but didn't try yet. > > webrev: > http://cr.openjdk.java.net/~sangheki/8220310/webrev.1 > http://cr.openjdk.java.net/~sangheki/8220310/webrev.1.inc (this may not > help much! :) ) > Testing: hs-tier 1 ~ 5 (with/without UseNUMA) > > Thanks, > Sangheon > >> >> ------------------------------------------------------------------------------ >> >> > From zgu at redhat.com Mon Sep 23 15:16:46 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 23 Sep 2019 11:16:46 -0400 Subject: RFR 8231250: Shenandoah: Traversal GC should keep alive weak load from heap Message-ID: Resubmit: Was: 8231250: Shenandoah: Shenandoah assert_correct failed; Object klass pointer SAP reported that this patch did not fix the assert failure in original bug report. But the fix is legit, as JDK-8227635 did change the behavior of traversal GC, resulting it may drop keepalive for heap loads. Bug: https://bugs.openjdk.java.net/browse/JDK-8231250 Webrev: http://cr.openjdk.java.net/~zgu/JDK-8231250/webrev.00/ Test: hotspot_gc_shenandoah (fastdebug and release) on Linux x86_64 Thanks, -Zhengyu From rkennke at redhat.com Mon Sep 23 16:37:01 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 23 Sep 2019 18:37:01 +0200 Subject: RFR 8231250: Shenandoah: Traversal GC should keep alive weak load from heap In-Reply-To: References: Message-ID: OK. We need to have a close look at the interaction between traversal and native barriers/conc roots soon. Roman > Resubmit: > Was:? 8231250: Shenandoah: Shenandoah assert_correct failed; Object > klass pointer > > SAP reported that this patch did not fix the assert failure in original > bug report. But the fix is legit, as JDK-8227635 did change the behavior > of traversal GC, resulting it may drop keepalive for heap loads. > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231250 > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8231250/webrev.00/ > > Test: > ? hotspot_gc_shenandoah (fastdebug and release) on Linux x86_64 > > Thanks, > > -Zhengyu > From zgu at redhat.com Mon Sep 23 18:28:43 2019 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 23 Sep 2019 14:28:43 -0400 Subject: RFR 8231250: Shenandoah: Traversal GC should keep alive weak load from heap In-Reply-To: References: Message-ID: On 9/23/19 12:37 PM, Roman Kennke wrote: > OK. Thanks for reviewing. > > We need to have a close look at the interaction between traversal and > native barriers/conc roots soon. Concurrent roots is not enabled for traversal. I think we have problem of hitting barriers on GC paths, which trigger side-effects of LRB. Thanks, -Zhengyu > > Roman > > >> Resubmit: >> Was:? 8231250: Shenandoah: Shenandoah assert_correct failed; Object >> klass pointer >> >> SAP reported that this patch did not fix the assert failure in >> original bug report. But the fix is legit, as JDK-8227635 did change >> the behavior of traversal GC, resulting it may drop keepalive for heap >> loads. >> >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231250 >> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8231250/webrev.00/ >> >> Test: >> ?? hotspot_gc_shenandoah (fastdebug and release) on Linux x86_64 >> >> Thanks, >> >> -Zhengyu >> From sangheon.kim at oracle.com Mon Sep 23 22:33:59 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Mon, 23 Sep 2019 15:33:59 -0700 Subject: RFR(XL): 8220310: Implementation: NUMA-Aware Memory Allocation for G1, Mutator (1/3) In-Reply-To: References: <06ACBF87-ADBE-499F-B668-0274E4925B26@oracle.com> <2b37edd6-3e0f-013d-1616-9d003f8ac1ed@oracle.com> Message-ID: <6ad56365-4507-8909-c13d-8ed15ab7cba9@oracle.com> Hi Stefan, Many thanks for reviewing and discussing this! And please allow me to post next webrev with more comments. On 9/23/19 8:14 AM, Stefan Johansson wrote: > Hi Sangheon, > > Thanks for the updated webrev, I like where we are heading with this. > I still have some comments but most of them are fairly small. The only > big thing is that I still think we could simplify things a lot by not > passing down node_index to the lower level commit functions and > instead just focus on keeping a strict manual interleaving of > regions/pages (whichever is largest). I know this wouldn't allow us to > expand by a single region on a specific node, but since this is a > fairly uncommon case I think we could live with that. > > src/hotspot/share/gc/g1/g1Allocator.* > ------------------------------------- > There are a few calls like this: > uint node_index = _g1h->mem_node_mgr()->index_of_current_thread(); > I would like a private helper to wrap this, something like > current_node_index(). Will do. > --- > > hotspot/share/gc/g1/g1Arguments.cpp > ----------------------------------- > ?161?? if (UseNUMA) { > ?162???? if (FLAG_IS_DEFAULT(AlwaysPreTouch)) { > ?163?????? FLAG_SET_DEFAULT(AlwaysPreTouch, true); > ?164???? } > ?165???? if (!AlwaysPreTouch && FLAG_IS_CMDLINE(AlwaysPreTouch)) { > ?166?????? warning("Disabling AlwaysPreTouch is incompatible with > UseNUMA. Disabling UseNUMA."); > ?167?????? FLAG_SET_ERGO(UseNUMA, false); > ?168???? } > ?169?? } > > I'm not sure I understand why we have to enforce AlwaysPreTouch when > NUMA is enabled. This will improve performance, but I think the NUMA > support should work even without it. Or am I missing something? When we allocate a new region, we have to know the node index of a region came from free list. However, without pretouch, we can't know whether the region is properly located or not. i.e. when checking node id via os::numa_get_address_id(), we will get invalid id even though we called numa_make_local() because the page (of the bottom of heap region) is not yet faulted. We may set HeapRegion::_node_index with preferred index after calling numa_make_local() ( without actually checking the node id). But not sure whether it is a way to go. If you are suggesting this, maybe we need more discussions and tests. So unless others have strong opinion on this, I would like to address in a separate CR. > --- > > src/hotspot/share/gc/g1/g1CollectedHeap.* > ----------------------------------------- > As we discussed offline I'm not sure we need pass the node_index down > in the call to expand. Since we're not strict in giving out memory > from the correct node during the mutator phase, I think we could do > the same during the GC in the case we need to expand. Yeah, the only difference would be: unlike the mutator case, we even don't try to get the identical node and expect to get random one. I do understand this case would be rare, but I think doing our best to match is better even though we have to sacrifice a little bit of code complexity. > --- > > src/hotspot/share/gc/g1/g1RegionToSpaceMapper.cpp > ------------------------------------------------- > ?162???? G1NodeDistributor itr(node_index); > ?163 > ?164???? for (uint i = start_idx; i < start_idx + num_regions; i++) { > ?165?????? // If there are many pages to touch, different node ids > will be used. > ?166?????? uint processed_node_index = itr.next_node_index(); > ?... > ?179???????? zero_filled = _storage.commit(idx, 1, processed_node_index); > ?180???????? itr.next(); > ?181?????? } > > Do we really need to do this? Can't we just rely on the code in > G1NUMA::touch_memory_roundrobin() to do the right thing? Or is this > just to allow committing a single region on the correct17134 node? Yes, this is to commit a single region and touch_memory_roundrobin() is not called. i.e. HeapRegion is smaller than page size, so we commit one page at once and skip for next region until the page is filled with region. e.g. 2MB page size, 1MB heap region. every 2 commit for heap regions, 1 page will be committed. However, your comment reminded me to enhance this area. Instead of starting from the first node, we can just call G1NUMA::next_numa_index() at G1NodeDistributor::next(). > --- > > src/hotspot/share/gc/g1/g1_globals.hpp > -------------------------------------- > Instead of adding those flags I think we should use log-tags and guard > the logic by using log_is_enabled(), or LogTarget::is_enable() as you > do in heapRegionManager.cpp. To allow these to be excluded with other > logging turned on I suggest using specific log-tag-sets for each of > them something like: > G1PrintNUMAIdOfHeapRegions -> gc, numa, region (debug or trace) > G1VerifyNUMAIdOfHeapRegions -> gc, numa, region, verification (debug > or trace) Will do as you suggested. Kim also suggested same one but I wanted to hear opinions. :) G1PrintNUMAIdOfHeapRegions -> gc, numa, region (debug) G1VerifyNUMAIdOfHeapRegions -> gc, numa, region, verification (debug) > --- > > src/hotspot/share/gc/g1/heapRegionManager.cpp > --------------------------------------------- > ?112?? if (mgr->num_active_nodes() > 1) { > ?113???? uint valid_node_index = > mgr->valid_node_index(requested_node_index); > ?114???? // Try to allocate with requested node index. > ?115???? hr = _free_list.remove_region_with_node_index(from_head, > valid_node_index, NULL); > ?116?? } > > I've seen some problems in my testing with this code, > valid_node_index() now call _numa->next_numa_index() when the index > isn't valid. This means that both the lower level commit code will > call this function as well as the allocation path in some cases (for > example single region humongous). I'm not sure if this will lead to > any real problems but it makes it harder to understand what is going > on underneath and I've seen it lead to having imbalance over the > nodes. To avoid this I think we could instead do: > > if (mgr->num_active_nodes() > 1 && > ??? mgr->is_valid_node_index(requested_node_index) { > ? hr = _free_list.remove_region_with_node_index(from_head, > requested_node_index, NULL); > } > > This way we only remove with index if we have a valid index, otherwise > we get a HeapRegion from the code below, which should be more or less > the same as taking whatever index and removing with that: > ?118?? if (hr == NULL) { > ?... > ?121???? hr = _free_list.remove_region(from_head); > ?122?? } > > If this is an ok approach I think valid_node_index() is not longer > used and can be removed. Looks nice suggestion. So instead of relying on G1NUMA::next_numa_index(), just using the first region would relief the node imbalance. I like this but let me back with some testing results. > --- > > src/hotspot/share/gc/g1/g1NUMA.inline.hpp > ----------------------------------------- > G1NUMA::next_numa_id() seems to be unused and could then be removed. Will remove. > --- > > I'll continue looking at the patch and taking it for some spins. I > might come with additional feedback later on, but these are my initial > findings. Okay! Thanks, Sangheon > > Thanks, > Stefan > > > On 2019-09-21 07:19, sangheon.kim at oracle.com wrote: >> Hi Kim, >> >> Many thanks for this thorough review! >> >> On 9/16/19 4:12 PM, Kim Barrett wrote: >>>> On Sep 4, 2019, at 3:16 AM, sangheon.kim at oracle.com wrote: >>>> >>>> Hi all, >>>> >>>> Please review this patch, making G1 NUMA aware. >>>> This is the first part of G1 NUMA implementation. >>>> >>>> - At G1 CollectedHeap initialization time, the given heap regions >>>> will be split and touched. >>>> - Internally node index(uint) is used instead of node id(int type) >>>> for easier access to arrays. >>>> - Only Linux is supported. >>>> - New logs are under gc+heap+numa >>>> - Current memory touch implementation will be replaced with >>>> WorkGang by JDK-8230411: NUMA aware work gang. >>>> >>>> CR: https://bugs.openjdk.java.net/browse/JDK-8220310 >>>> Webrev: http://cr.openjdk.java.net/~sangheki/8220310/webrev.0 >>>> Testing: hs-tier 1 ~ 5 with +- UseNUMA. >>>> >>>> Thanks, >>>> Sangheon >> I had long off-line discussion with Kim, Thomas and Stefan J. >> And 2 major changes are: >> >> 1) Memory touch thread is removed. This was a general way to >> implement for both Linux and Solaris. However, Solaris is not >> supported anymore, so memory touch thread is just removed. >> And this made the code much simpler. >> >> 2) G1NUMA doesn't manage numa id of each page and we don't care about >> each page instead HeapRegion is only cared. i.e. we assume numa id of >> all pages between bottom and end of HeapRegion to be same. This also >> made related codes much simpler. >> >> There are some other changes as well. >> >> >>> Here is an initial set of comments. Some of these might be nonsense; >>> I'm >>> not all that familiar with NUMA support. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> Just in general, it's kind of disappointing that NUMA support for G1 >>> apparently involves this much code, and is completely different from >>> ParallelGC NUMA support, and both are completely different from ZGC >>> NUMA support.? (And Shenandoah is probably yet another still.) >>> Oh well... >> Yes, I agree on that. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> According to off-line discussion, the intent is that a HeapRegion only >>> contains memory from one NUMA node.? That isn't obvious though, and >>> there >>> are API and data structure choices that obscure that.? It seems like >>> some >>> things could be simplified with that constraint in mind.? I think >>> some of >>> that is a result of trying to leave open support for NUMA aware >>> off-heap >>> data structures, but that isn't part of this change, and might be >>> better >>> kept separate from NUMA aware Java heap support. >>> >>> I'll try to point out places where I think there are opportunities for >>> simplification. >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> According to off-line discussion, the touching mechanism might not be >>> necessary.? If that's true, that would be a significant >>> simplification.? For >>> now I won't comment on the code for the touch threads, though I think >>> I found some problems there before the discussion suggesting we might >>> not need them happened. >> Thanks. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1NUMA.hpp >>> ?? 64 // Manages NUMA node related information and memory touch >>> threads. >>> ... >>> >>> I found the comment describing G1NUMA pretty hard to parse. Even >>> after looking at the code I'm having trouble figuring out what's being >>> said. >>> >>> This part I was able to figure out, but think it could be worded >>> better. >>> ?? 70 // Also provides a way to convert NUMA id and NUMA >>> index(starting from zero and sequatial). >>> ?? 71 // Actual NUMA ids would not be sequential nor not start from >>> zero, so a conversion is necessary >>> ?? 72 // between id and index. >>> >>> Something like >>> >>> ?? Also provides a mapping between NUMA indices (sequential, starting >>> ?? from zero) and NUMA ids (which may not start at zero and may not be >>> ?? sequential).? Indices are generally used by clients, and mapped to >>> ?? ids when dealing with the OS/hardware layer. >> DONE >> >>> >>> (I *think* that's true.) >>> >>> But then, why expose node_ids at all in G1NUMA? It seems like node_ids >>> could be completely relegated to an implementation detail that clients >>> never need to deal with. >> ... >>> As evidence for not needing to expose ids to clients, there are no >>> callers of numa_id_of_index in any of the series of 3 patches.? And >>> G1NUMA::numa_ids() and G1MemoryMultNodeManager::node_ids() are only >>> used in a couple of places for printing/logging and by >>> WB_G1MemoryNodeIds.? I'm not sure what the last is for, but the >>> printing and logging code could be isolated into G1NUMA. >> Basically I totally agree with you for not exposing numa id. >> Current patches are only using node id for logging purpose at some >> locations because users are only interested on node id. >> >> Also Thread::lgrp_id() (renaming lgrp id, numa id etc.. is different >> topic that we had before) is also public, so my feeling is current >> try is enough. :) >> >> WB_G1MemoryNodeIds() is for jtreg testing purpose which checks >> whether the given memory has node id as we expected. >> >>> >>> Also, it looks like the index to node_id mapping provided here is >>> duplicating the information provided by os::Linux::nindex_to_node. >>> Not that this necessarily helps with anything in generic code... >> Yes, I'm aware of _nindex_to_node but I think maintaining such >> conversion at G1NUMA is okay. Other NUMA implementation such as >> parallel gc relies on node id so completely removing node id at >> general code seems different scope. >> Unless you have strong opinion on addressing it in this patch, I >> would like proceed as is. >> If it is the case, we can consider your suggestion as a separate >> enhancement. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>> ?? 57?? virtual int* node_ids() const { static int dummy_id = 0; >>> return &dummy_id; } >>> >>> Can the return type here be "const int*" ?? That would be better if >>> possible. >>> >>> [This is probably moot if clients only deal with numa indexes and not >>> node_ids, as suggested earlier.] >> DONE >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>> ?? 63?? virtual uint guarantee_valid_node_index(uint node_index) >>> const { return 0; } >>> >>> Given naming conventions elsewhere, the name of this function makes me >>> expect it to be a verification function. >> Agree. >> valid_node_index() is better? webrev.1 changed name. >> Or are you suggesting to remove this and then the caller do such >> thing? e.g. >> void A(uint node_index) { >> uint valid_node_index = node_index; >> if (!is_valid_node_index(valid_node_index)) { >> ?? valid_node_index = random_numa_index(); >> ... >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1NUMA.inline.hpp >>> ?? 29 #include "gc/shared/taskqueue.inline.hpp" >>> >>> This #include seems to be just to obtain randomParkAndMiller. It >>> would be >>> better to refactor that function into its own file(s). Or make it >>> part of >>> some component providing various random number generators. >> Not need because of below change. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1NUMA.inline.hpp >>> ?? 45?? int seed = randomParkAndMiller(_seed); >>> ?? 46?? Atomic::store(seed, &_seed); >>> >>> This sequence may lose updates.? I don't know if that's important. >>> And why isn't this just using os::random(), or maybe refactoring that >>> slightly to gain access to the external seed version.? I don't want >>> this >>> change to be made: >>> http://cr.openjdk.java.net/~sangheki/8220310/webrev.0/src/hotspot/share/gc/shared/taskqueue.inline.hpp.udiff.html >>> >> Thomas and Stefan J. also complained on this. >> Changed to get next numa index with round-robin manner. i.e. doesn't >> use random() at all. >> >>> And do we *really* need randomness here at all?? Is there a reason not >>> to simply cycle through the nodes? >> Not really. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/os/linux/os_linux.cpp >>> 3016?? size_t num_pages_left = size_in_bytes / page_size; >>> >>> Is size_in_bytes required to be a multiple of page_size?? I believe >>> it is, >>> and an assert of that would be useful. >> This code is removed. And yes to answer to your question. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/os/linux/os_linux.cpp >>> 3016?? size_t num_pages_left = size_in_bytes / page_size; >>> 3017?? const size_t num_pages_total = num_pages_left; >>> 3018?? size_t num_buffer_entries = MIN2(num_pages_left, max_pages); >>> >>> I think clearer would be >>> >>> 3017?? const size_t num_pages_total = size_in_bytes / page_size; >>> 3018?? size_t num_buffer_entries = MIN2(num_pages_total, max_pages); >>> >>> and move this down to the loop setup: >>> 3016?? size_t num_pages_left = num_pages_total; >> This code is removed. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/os/linux/os_linux.cpp >>> 3020?? void** pages = (void**)alloca(sizeof(void*) * >>> num_buffer_entries); >>> 3021?? int* status = (int*)alloca(sizeof(int) * num_buffer_entries); >>> >>> I think a ResourceMark and NEW_RESOURCE_ARRAYs would be preferable to >>> using alloca() here.? alloca() has some dangers and I think is >>> currently >>> only used in some very specialized places where it really is needed. >> This code is removed. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/os/linux/os_linux.cpp >>> 3061 } >>> 3062 // Moves current thread on a specific node and it will not >>> migrate to >>> >>> Missing blank line between functions. >> This code is removed. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/os/linux/os_linux.cpp >>> 3031?????? pages[i] = (void*)base; >>> >>> Unnecessary cast. >> This code is removed. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/os/linux/os_linux.cpp >>> 3072???? case AFFINITY_NONE: >>> 3073?????? // Do nothing. >>> 3074?????? return true; >>> 3075?????? break; >>> >>> Shouldn't this be calling numa_run_on_node with -1 node id? The >>> documentation for numa_run_on_node() says "Passing -1 permits the >>> kernel to schedule on all nodes again." >> My understand of that description is let kernel decide the numa node >> which is not actually want here. We want to move current thread to >> the preferred numa node. >> >> Currently we don't need this API (numa_set_affinity() ) so removed on >> webrev.1. >> However following patch which is not scope of this JEP will use it. >> I will add your suggestion below? when we need this. >> >>> >>> But see also next comment about numa_affinity_set. >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/runtime/os.hpp >>> ? 401?? // Available affinities. >>> ? 402?? enum NumaAffinity { >>> ? 403???? AFFINITY_STRONG, >>> ? 404???? AFFINITY_NONE >>> ? 405?? }; >>> ? 406?? static bool numa_affinity_set(Thread* thread, int numa_id, >>> NumaAffinity affinity); >>> >>> Could there ever be other affinities?? This API doesn't seem right. >>> It seems to me it should be something like >>> >>> ?? static bool numa_set_affinity(Thread* thread, int numa_id); >>> ?? static bool numa_clear_affinity(Thread* thread); >>> >>> No need for NumaAffinity enum. >> The enum is also kind of remainder after removing Solaris >> implementation which has more types. >> But I agree with you as there are only 2 types now, it is simpler to >> separate API without enum. >> >>> >>> (I think numa_set/clear_affinity is better than >>> numa_affinity_set/clear; >>> numa is a "namespace" and we usually set/clear/get_something rather >>> than something_set/clear/get.) >>> >>> Or maybe this should just be >>> >>> ?? static bool numa_set_affinity(Thread* thread, int numa_id); >>> >>> with "clearing" provided by passing AnyId as the numa_id? Still don't >>> need the NumaAffinity enum for this. >> Agree on this naming stuff. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1NUMA.cpp >>> ? 124?? guarantee(rs.size() % page_size == 0, "Reserved space size >>> (" SIZE_FORMAT ") should be aligned " >>> ? 141?? guarantee(size_in_bytes % page_size == 0, "The given range >>> (" SIZE_FORMAT ") should be aligned " >>> ? 209???? guarantee((uintptr_t)G1HeapRegionSize % page_size == 0, >>> "G1HeapRegionSize (" SIZE_FORMAT ") and page size (" >>> ? 234???? guarantee((uintptr_t)_page_size % _heap_region_size == 0, >>> "G1HeapRegionSize (" SIZE_FORMAT ") and page size (" >>> >>> Use is_aligned.? I think casts shouldn't be needed. >> These lines are removed after major refactoring. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1NUMA.cpp >>> ?? 43?? _numa_id_to_index_map = NEW_C_HEAP_ARRAY_RETURN_NULL(uint, >>> _len_numa_id_to_index_map, mtGC); >>> ?? 44?? memset(_numa_id_to_index_map, 0, sizeof(uint) * >>> _len_numa_id_to_index_map); >>> >>> Using NEW_xxx_RETURN_NULL and then ignoring the possibility of a NULL >>> result.? It would be better to use the non-_RETURN_NULL variant and >>> get a decent error message rather than the segfault from attemptint to >>> write to NULL. >> Changed to use non-_RETURN_NULL version. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1NUMA.cpp >>> ?? 34 void G1NUMA::init_numa_id_to_index_map(int* numa_ids, uint >>> num_numa_ids) { >>> >>> int* numa_ids => const int* numa_ids. >> DONE >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1NUMA.cpp >>> ?? 34 void G1NUMA::init_numa_id_to_index_map(int* numa_ids, uint >>> num_numa_ids) { >>> ... >>> ?? 44?? memset(_numa_id_to_index_map, 0, sizeof(uint) * >>> _len_numa_id_to_index_map); >>> Rather than this memset and then writing to all of them later, instead >>> initialize all entries to InvalidNodeIndex and then assign from >>> numa_ids. >> DONE >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1MemoryTouchThread.cpp >>> ? 261?? _wakeup_monitor = new Monitor(Mutex::nonleaf, "Wakeup >>> Monitor for Touch Control", true, Monitor::_safepoint_check_never); >>> ? 262?? _sync_monitor = new Monitor(Mutex::nonleaf, "Sync Monitor >>> for Touch Control", true, Monitor::_safepoint_check_never); >>> >>> For singleton mutex/monitor, we generally prefer to put them in >>> mutexLocker. >>> Probably passed as constructor arguments to G1NUMAThreadsControl.? And >>> I know there are counter-examples. >> Not necessary as memory touch thread is gone. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>> ?? 30 extern const uint AnyNodeIndex; >>> ?? 31 extern const uint InvalidNodeIndex; >>> >>> Why are these at namespace scope.? And they seem misplaced here. >>> Seems like they should belong to the G1NUMA class, which is >>> responsible for indexing nodes and mapping between indexes and ids. >> Changed to G1MemoryNodeManager::AnyNodeIndex. >> And I wanted to hide or minimize exposing G1NUMA in general. >> Memory node manager also has index related APIs, so let manager have >> such constants seems okay. >> >>> >>> ?? 30 const uint AnyNodeIndex = (uint)os::AnyId; >>> ?? 31 const uint InvalidNodeIndex = (uint)os::InvalidId; >>> And why these values?? That seems like a units mismatch.? Just use >>> UINT_MAX and UINT_MAX - 1. >> I don't have strong opinion on this. >> But the intend is to make same value after casting for same meaning >> constants instead of randomly chosen ones. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1NUMA.hpp >>> ?? 80?? // For invalid numa id, return InvalidNodeIndex. So the >>> caller need exception handling. >>> >>> "need exception handling"? >> Thomas also commented on it so reworded. But finally the class is >> removed. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/heapRegionManager.cpp >>> ? 114???? uint allocated_node_index; >>> >>> This variable gets assigned via an out parameter here: >>> ? 116???? hr = _free_list.remove_region_with_node_index(from_head, >>> valid_node_index, &allocated_node_index); >>> >>> and by assignment here: >>> ? 122???????? allocated_node_index = >>> mgr->node_index_of_address(hr->bottom()); >>> >>> but is never used. If the lack of use is not itself a mistake, then >>> the out >>> parameter of remove_region_with_node_index appears to be unnecessary. >> The variable will be used at patch3 (JDK-8220312) but slightly >> changed to allow NULL at allocated_node_index parameter at >> remove_region_with_node_index(). >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/heapRegionSet.inline.hpp >>> ? 151 inline HeapRegion* >>> FreeRegionList::remove_region_with_node_index(bool from_head, >>> >>> As written, I think this might always return NULL if >>> requested_node_index is >>> not a valid index. I think callers ensure that it's valid, but an >>> assert of >>> the argument would be help. >> Done >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/heapRegionSet.inline.hpp >>> ? 175???? if (cur == NULL || cur_depth >= max_search_depth) { // not >>> found >>> >>> There's an off-by-one issue here. The max_search_depth time through the >>> while-control-expression may have found a matching region, checked >>> via the >>> numa_index_of_address lookup. >>> >>> Either the test should be "current_numa_index != >>> requested_numa_index" or >>> the loop should be changed. >> This code is removed. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/heapRegionSet.inline.hpp >>> ? 151 inline HeapRegion* >>> FreeRegionList::remove_region_with_node_index(bool from_head, >>> >>> I think the list manipulation could be simplified quite a bit.? I >>> think we >>> don't need the "last" variable.? And the splicing out of cur can be >>> unified >>> between the from_head and !from_head cases, e.g. something like this: >>> >>> ?? // Find the region to use, searching from _head or _tail as >>> requested. >>> ?? if (from_head) { >>> ???? for (cur = _head; >>> ????????? cur != NULL && cur_depth < max_search_depth; >>> ????????? cur = cur->next(), ++cur_depth) { >>> ?????? if (current_numa_index == >>> numa->numa_index_of_address(cur->bottom)) { >>> ???????? break; >>> ?????? } >>> ???? } >>> ?? } else { >>> ???? for (cur = _tail; >>> ????????? cur != NULL && cur_depth < max_search_depth; >>> ????????? cur = cur->prev(), ++cur_depth) { >>> ?????? if (current_numa_index == >>> numa->numa_index_of_address(cur->bottom)) { >>> ???????? break; >>> ?????? } >>> ???? } >>> ?? } >>> ?? if (cur == NULL || cur_depth >= max_search_depth) { >>> ???? return NULL;????????????????????? // Didn't find a region to use >>> ?? } >>> ?? // Splice the region out of the list. >>> ?? HeapRegion* prev = cur->prev(); >>> ?? HeapRegion* next = cur->next(); >>> ?? if (prev == NULL) { >>> ???? _head = next; >>> ?? } else { >>> ???? prev->set_next(next); >>> ?? } >>> ?? if (next == NULL) { >>> ???? _tail = prev; >>> ?? } else { >>> ???? next->set_prev(prev); >>> ?? } >>> ?? cur->set_prev(NULL); >>> ?? cur->set_next(NULL); >>> ?? ... >> Nice suggestion! >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1NUMA.cpp >>> ? 166 class DumpIterator : public StackObj { >>> >>> Polluting global namespace.? Suggest making DumpIterator and the >>> classes >>> derived from it (G1HeapRegion{Larger,Smaller}Iterator) private nested >>> classes of G1NUMA, making the names unimportant outside of that >>> context. >> Not need any more. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/heterogeneousHeapRegionManager.hpp >>> ? 122?? virtual HeapRegion* allocate_free_region(HeapRegionType type); >>> ? 123?? virtual HeapRegion* allocate_free_region(HeapRegionType >>> type, uint node_index); >>> >>> The base class (HeapRegionManager) no longer has the one-arg >>> function.? So >>> this one-arg function is no longer overriding the base class >>> definition. >>> (Too bad it wasn't declared as C++11 "override".) >>> >>> I don't see any one-arg callers, so seems like it can just be removed. >> DONE >> Nice finding! :) >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1PageBasedNUMAVirtualSpace.cpp >>> >>> I dislike the new structure here, with G1PageBasedVirtualSpace now >>> being >>> both a base class and a concrete class (which is often a problematic >>> design).? And the new derived class doesn't really do all that much. >>> >>> It seems like it would be much simpler to just have >>> G1PageBasedVirtualSpace >>> check UseNUMA to conditionalize it's commit and uncommit functions. >>> Especially since it's commit already needs to have some knowlege of >>> NUMA >>> because of the new numa_index argument. >> DONE >> Removed NUMA version and reverted related changes to allow inherit >> are reverted. >> Now, G1PageBasedVirtualSpace does additional work after checking NUMA. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1PageBasedNUMAVirtualSpace.cpp >>> ?? 52 void G1PageBasedNUMAVirtualSpace::uncommit(size_t start_page, >>> size_t size_in_pages) { >>> >>> Do we actually need to do anything here?? We're resetting the >>> G1NUMA::_numa_id_of_pages_table (to AnyId), but it's not clear we >>> really >>> need to do that.? When we later recommit any of those pages, that >>> table will >>> be updated appropriately. >>> >>> Maybe rather than having G1NUMA expose quite so much detail it should >>> instead have post_commit and a pre_uncommit functions. >> I think clearing the node index may help for debugging. Unlike >> webrev.0, numa id/index information is located at HeapRegion which is >> much simpler. i.e. HeapRegionManager::uncommit_regions() does >> resetting node index to InvalidNodeIndex. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1NUMA.hpp >>> ?? 75?? G1NUMAPageIdTable _numa_id_of_pages_table; >>> >>> I was surprised by the need for this, but I can't find anything in >>> the NUMA >>> API that provides this information.? That seems strange, because one >>> would >>> think that information must exist somewhere in the NUMA implementation. >> Removed. >> And this made the patch much simpler! >> >> And here are your comments from other email. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>> ?? 64?? // Returns node index of the given address. >>> ?? 65?? // If the node id of the address is invalid return randomly >>> selected node index. >>> ?? 66?? virtual uint valid_index_of_address(HeapWord* o) const { >>> return 0; } >>> >>> This function is unused. >> Removed. >> >>> ------------------------------------------------------------------------------ >>> >>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>> ?? 70?? virtual uint node_index_of_address(HeapWord* o) const { >>> return 0; } >>> >>> The only caller of this function is >>> HeapRegionManager::allocate_free_region. >>> As noted in an earlier comment, the result of that call is unused. >> Removed as you commented but I had to revert as a part of >> _numa_id_of_pages_table removal. >> Previously we retrieved the page info from _numa_id_of_pages_table, >> but new version reads directly via system call. >> >>> >>> ------------------------------------------------------------------------------ >>> >>> >>> As pointed out to me offline, there*are*? ways to query the address -> >>> associated numa node mapping.? So I'm going back and questioning the >>> need >>> for _numa_id_of_pages_table and the code for managing it. >> As we discussed offline, I removed this table. So webrev.1 checks the >> bottom of HeapRegion and then saves it at HeapRegion::_node_index. >> However we do need a way to check node index of the given address and >> so I added back. >> >> ?From my testing, checking a numa id of one page seems okay to do on >> runtime. Previously I did checking for all pages of given memory so >> it was a bit slow. >> >> One thing that didn't include at webrev.1 is combining with existing >> one. >> The newly introduced os::numa_get_address_id() is a copy from >> zNUMA_linux.cpp. It may be unified but didn't try yet. >> >> webrev: >> http://cr.openjdk.java.net/~sangheki/8220310/webrev.1 >> http://cr.openjdk.java.net/~sangheki/8220310/webrev.1.inc (this may >> not help much! :) ) >> Testing: hs-tier 1 ~ 5 (with/without UseNUMA) >> >> Thanks, >> Sangheon >> >>> >>> ------------------------------------------------------------------------------ >>> >>> >> From kim.barrett at oracle.com Tue Sep 24 02:58:41 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 23 Sep 2019 22:58:41 -0400 Subject: RFR: 8231153: Improve concurrent refinement statistics Message-ID: <1DADC595-3106-4CE7-BA5D-7B6C7EE0E81E@oracle.com> Please review this change to collection additional statistics about the generation and processing of dirty cards. These are needed for future improvements to the control of the concurrent refinement threads (JDK-8137022). This change adds tracking and prediction of the rate at which concurrent refinement threads process cards. It also adds tracking and prediction of the rate at which mutator threads generate cards. As part of that, there is now tracking of total cards processed by the concurrent refinement threads and the mutator threads. This is now used by G1RemsetSummary instead of the buffer counts previously reported. CR: https://bugs.openjdk.java.net/browse/JDK-8231153 Webrev: https://cr.openjdk.java.net/~kbarrett/8231153/open.00/ Testing: mach5 tier1-5 some local by-hand testing to look at the rate tracking. From stefan.karlsson at oracle.com Tue Sep 24 07:07:25 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 24 Sep 2019 09:07:25 +0200 Subject: RFR: 8231270: ZGC: Remove ZAddressSpace* and ZAddressReserved* In-Reply-To: <11eec445-9827-8150-efd9-59cb104e866c@oracle.com> References: <11eec445-9827-8150-efd9-59cb104e866c@oracle.com> Message-ID: Looks good. -XX:+VerifyOops has bit-rottened so I think it's OK to provide a nop implementation with: Universe::calculate_verify_data((HeapWord*)0, (HeapWord*)UINTPTR_MAX); but we might want to create an RFE to provide a better bit-mask if VerifyOops is fixed. Thanks, StefanK On 2019-09-19 23:08, Per Liden wrote: > After JDK-8224820, ZAddressSpace{Start,End,Size} is not used for > anything other than being printed, so they can be removed. Also, the two > last uses of ZAddressReserved* can easily be replaced. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231270 > Webrev: http://cr.openjdk.java.net/~pliden/8231270/webrev.0 > > /Per From stefan.karlsson at oracle.com Tue Sep 24 07:09:00 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 24 Sep 2019 09:09:00 +0200 Subject: RFR: 8231251: ZGC: Fix ZHeap includes In-Reply-To: References: Message-ID: Looks good - if it compiles :) StefanK On 2019-09-19 16:41, Per Liden wrote: > I noticed that the ZHeap includes needed a bit of a cleanup, as there > are many includes that are not longer needed, and a few are missing. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231251 > Webrev: http://cr.openjdk.java.net/~pliden/8231251/webrev.0 > > /Per From per.liden at oracle.com Tue Sep 24 07:16:19 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 24 Sep 2019 09:16:19 +0200 Subject: RFR: 8231270: ZGC: Remove ZAddressSpace* and ZAddressReserved* In-Reply-To: References: <11eec445-9827-8150-efd9-59cb104e866c@oracle.com> Message-ID: Thanks! And I agree that we could use some refined values here if/when VerifyOops start to become useful again. /Per On 9/24/19 9:07 AM, Stefan Karlsson wrote: > Looks good. > > -XX:+VerifyOops has bit-rottened so I think it's OK to provide a nop > implementation with: > Universe::calculate_verify_data((HeapWord*)0, (HeapWord*)UINTPTR_MAX); > > but we might want to create an RFE to provide a better bit-mask if > VerifyOops is fixed. > > Thanks, > StefanK > > On 2019-09-19 23:08, Per Liden wrote: >> After JDK-8224820, ZAddressSpace{Start,End,Size} is not used for >> anything other than being printed, so they can be removed. Also, the >> two last uses of ZAddressReserved* can easily be replaced. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231270 >> Webrev: http://cr.openjdk.java.net/~pliden/8231270/webrev.0 >> >> /Per From per.liden at oracle.com Tue Sep 24 07:15:08 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 24 Sep 2019 09:15:08 +0200 Subject: RFR: 8231251: ZGC: Fix ZHeap includes In-Reply-To: References: Message-ID: Thanks! /Per On 9/24/19 9:09 AM, Stefan Karlsson wrote: > Looks good - if it compiles :) > > StefanK > > On 2019-09-19 16:41, Per Liden wrote: >> I noticed that the ZHeap includes needed a bit of a cleanup, as there >> are many includes that are not longer needed, and a few are missing. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231251 >> Webrev: http://cr.openjdk.java.net/~pliden/8231251/webrev.0 >> >> /Per From shade at redhat.com Tue Sep 24 07:43:46 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 24 Sep 2019 09:43:46 +0200 Subject: RFR (T) 8231395: Backout JDK-8231249 Message-ID: <1727364f-036a-ef37-bfa5-467c1d65b842@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8231395 JDK-8231249 caused some testing failures, notably in: gc/shenandoah/oom/TestClassLoaderLeak.java failed (JDK-8231389) gc/shenandoah/TestAllocHumongousFragment.java The reason is that those tests expect us to eventually come to Full GC, and JDK-8231249 prevents that. It requires more comprehensive fix. Let's backout the bad fix meanwhile. Fix, produced by "hg backout" diff -r ef8c8cf9256a src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp --- a/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp Fri Sep 20 10:28:48 2019 +0200 +++ b/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp Tue Sep 24 09:42:57 2019 +0200 @@ -810,10 +810,20 @@ // other threads have already depleted the free storage. In this case, a better // strategy is to try again, as long as GC makes progress. - - if (result == NULL) { - do { - control_thread()->handle_alloc_failure(req.size()); - result = allocate_memory_under_lock(req, in_new_region); - } while (result == NULL && _progress_last_gc.is_set()); + // + // Then, we need to make sure the allocation was retried after at least one + // Full GC, which means we want to try more than ShenandoahFullGCThreshold times. + + size_t tries = 0; + + while (result == NULL && _progress_last_gc.is_set()) { + tries++; + control_thread()->handle_alloc_failure(req.size()); + result = allocate_memory_under_lock(req, in_new_region); + } + + while (result == NULL && tries <= ShenandoahFullGCThreshold) { + tries++; + control_thread()->handle_alloc_failure(req.size()); + result = allocate_memory_under_lock(req, in_new_region); } Testing: hotspot_gc_shenandoah -- Thanks, -Aleksey From stefan.karlsson at oracle.com Tue Sep 24 08:19:20 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 24 Sep 2019 10:19:20 +0200 Subject: RFR: 8224820: ZGC: Support discontiguous heap reservations In-Reply-To: References: <5567c0b8-00e6-29f0-12c6-e067c924fdce@oracle.com> <8767693d-13c1-e6c8-4114-f2d73775e8e6@oracle.com> Message-ID: <4e8f01be-5c0d-21f7-94bd-3210f2f4bce1@oracle.com> Looks good. StefanK On 2019-09-17 12:05, Per Liden wrote: > Hi, > > Discussed with Erik off-line. In light of things like future Windows > support, we probably don't want to make the reservation code shared, > instead just move it to os/posix. > > Here's a proposed webrev for doing just that: > > http://cr.openjdk.java.net/~pliden/8224820/webrev.0 > > /Per > > On 9/5/19 3:51 PM, Per Liden wrote: >> On 8/9/19 11:59 AM, Erik ?sterlund wrote: >>> Hi, >>> >>> Today ZGC reserves a huge chunk of virtual address space when the JVM >>> starts. This typically succeeds because we grab the VA before anyone >>> else has time to do so. But if some ASLR library or something was to >>> grab a tiny part of the desired VA, ZGC can't start. We should >>> support discontiguous heap reservations to support this. >>> >>> On linux, by default, this does not happen. But on OS X, it does >>> happen relatively frequently. So we need to fix this to allow a mac >>> port. >>> >>> This patch implements a recursive algorithm for finding holes at 2MB >>> granularities in the normally contiguous reservation when >>> initializing the heap, removing them from our VA. >>> >>> This patch depends on 8224815, which depends on 8229189 and 8229278. >>> They are all out for review. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8224820/webrev.00/ >> >> Looks good. However, please pass down max_capacity from ZPageAllocator >> to ZVirtualMemory's constructor, instead of reading MaxHeapSize. >> >> cheers, >> Per >> >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8224820 >>> >>> Thanks, >>> /Erik From thomas.schatzl at oracle.com Tue Sep 24 08:44:55 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 24 Sep 2019 10:44:55 +0200 Subject: RFR (T) 8231395: Backout JDK-8231249 In-Reply-To: <1727364f-036a-ef37-bfa5-467c1d65b842@redhat.com> References: <1727364f-036a-ef37-bfa5-467c1d65b842@redhat.com> Message-ID: Hi, On 24.09.19 09:43, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8231395 > > JDK-8231249 caused some testing failures, notably in: > gc/shenandoah/oom/TestClassLoaderLeak.java failed (JDK-8231389) > gc/shenandoah/TestAllocHumongousFragment.java > > The reason is that those tests expect us to eventually come to Full GC, and JDK-8231249 prevents > that. It requires more comprehensive fix. Let's backout the bad fix meanwhile. > > Fix, produced by "hg backout" looks like a legit backout :) Ship it. Thomas > > diff -r ef8c8cf9256a src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp > --- a/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp Fri Sep 20 10:28:48 2019 +0200 > +++ b/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp Tue Sep 24 09:42:57 2019 +0200 > @@ -810,10 +810,20 @@ > // other threads have already depleted the free storage. In this case, a better > // strategy is to try again, as long as GC makes progress. > - > - if (result == NULL) { > - do { > - control_thread()->handle_alloc_failure(req.size()); > - result = allocate_memory_under_lock(req, in_new_region); > - } while (result == NULL && _progress_last_gc.is_set()); > + // > + // Then, we need to make sure the allocation was retried after at least one > + // Full GC, which means we want to try more than ShenandoahFullGCThreshold times. > + > + size_t tries = 0; > + > + while (result == NULL && _progress_last_gc.is_set()) { > + tries++; > + control_thread()->handle_alloc_failure(req.size()); > + result = allocate_memory_under_lock(req, in_new_region); > + } > + > + while (result == NULL && tries <= ShenandoahFullGCThreshold) { > + tries++; > + control_thread()->handle_alloc_failure(req.size()); > + result = allocate_memory_under_lock(req, in_new_region); > } > > > Testing: hotspot_gc_shenandoah > From shade at redhat.com Tue Sep 24 08:56:26 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 24 Sep 2019 10:56:26 +0200 Subject: RFR (T) 8231395: Backout JDK-8231249 In-Reply-To: References: <1727364f-036a-ef37-bfa5-467c1d65b842@redhat.com> Message-ID: <3939877a-f441-d45e-1166-c2aff2adf812@redhat.com> On 9/24/19 10:44 AM, Thomas Schatzl wrote: > On 24.09.19 09:43, Aleksey Shipilev wrote: >> Bug: >> ? https://bugs.openjdk.java.net/browse/JDK-8231395 >> >> JDK-8231249 caused some testing failures, notably in: >> ?? gc/shenandoah/oom/TestClassLoaderLeak.java failed (JDK-8231389) >> ?? gc/shenandoah/TestAllocHumongousFragment.java >> >> The reason is that those tests expect us to eventually come to Full GC, and JDK-8231249 prevents >> that. It requires more comprehensive fix. Let's backout the bad fix meanwhile. >> >> Fix, produced by "hg backout" > > ? looks like a legit backout :) Ship it. Thanks, pushed. -- Thanks, -Aleksey From stefan.johansson at oracle.com Tue Sep 24 09:02:19 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 24 Sep 2019 11:02:19 +0200 Subject: RFR(XL): 8220310: Implementation: NUMA-Aware Memory Allocation for G1, Mutator (1/3) In-Reply-To: <6ad56365-4507-8909-c13d-8ed15ab7cba9@oracle.com> References: <06ACBF87-ADBE-499F-B668-0274E4925B26@oracle.com> <2b37edd6-3e0f-013d-1616-9d003f8ac1ed@oracle.com> <6ad56365-4507-8909-c13d-8ed15ab7cba9@oracle.com> Message-ID: Hi Sangheon, On 2019-09-24 00:33, sangheon.kim at oracle.com wrote: > Hi Stefan, > > Many thanks for reviewing and discussing this! > And please allow me to post next webrev with more comments. Yes, no rush. > > On 9/23/19 8:14 AM, Stefan Johansson wrote: >> Hi Sangheon, >> >> Thanks for the updated webrev, I like where we are heading with this. >> I still have some comments but most of them are fairly small. The only >> big thing is that I still think we could simplify things a lot by not >> passing down node_index to the lower level commit functions and >> instead just focus on keeping a strict manual interleaving of >> regions/pages (whichever is largest). I know this wouldn't allow us to >> expand by a single region on a specific node, but since this is a >> fairly uncommon case I think we could live with that. >> >> src/hotspot/share/gc/g1/g1Allocator.* >> ------------------------------------- >> There are a few calls like this: >> uint node_index = _g1h->mem_node_mgr()->index_of_current_thread(); >> I would like a private helper to wrap this, something like >> current_node_index(). > Will do. > >> --- >> >> hotspot/share/gc/g1/g1Arguments.cpp >> ----------------------------------- >> ?161?? if (UseNUMA) { >> ?162???? if (FLAG_IS_DEFAULT(AlwaysPreTouch)) { >> ?163?????? FLAG_SET_DEFAULT(AlwaysPreTouch, true); >> ?164???? } >> ?165???? if (!AlwaysPreTouch && FLAG_IS_CMDLINE(AlwaysPreTouch)) { >> ?166?????? warning("Disabling AlwaysPreTouch is incompatible with >> UseNUMA. Disabling UseNUMA."); >> ?167?????? FLAG_SET_ERGO(UseNUMA, false); >> ?168???? } >> ?169?? } >> >> I'm not sure I understand why we have to enforce AlwaysPreTouch when >> NUMA is enabled. This will improve performance, but I think the NUMA >> support should work even without it. Or am I missing something? > When we allocate a new region, we have to know the node index of a > region came from free list. > However, without pretouch, we can't know whether the region is properly > located or not. i.e. when checking node id via > os::numa_get_address_id(), we will get invalid id even though we called > numa_make_local() because the page (of the bottom of heap region) is not > yet faulted. > > We may set HeapRegion::_node_index with preferred index after calling > numa_make_local() ( without actually checking the node id). But not sure > whether it is a way to go. > If you are suggesting this, maybe we need more discussions and tests. So > unless others have strong opinion on this, I would like to address in a > separate CR. > I think we should go with trusting the OS and set the preferred index on the HeapRegion and then we can have our verification/check on what actual node we ended up on during a safepoint. But we can discuss this more. I'm not sure how much more testing this requires, in the end we cannot know what the OS will do underneath. >> --- >> >> src/hotspot/share/gc/g1/g1CollectedHeap.* >> ----------------------------------------- >> As we discussed offline I'm not sure we need pass the node_index down >> in the call to expand. Since we're not strict in giving out memory >> from the correct node during the mutator phase, I think we could do >> the same during the GC in the case we need to expand. > Yeah, the only difference would be: unlike the mutator case, we even > don't try to get the identical node and expect to get random one. I do > understand this case would be rare, but I think doing our best to match > is better even though we have to sacrifice a little bit of code complexity. > I see you point, but I don't fully agree. If the case is rare, or we can't show that handing out a region with the correct index is giving a benefit I think we should leave this as a separate enhancement later on. Doing so we might even be able to design it in a smarter way. >> --- >> >> src/hotspot/share/gc/g1/g1RegionToSpaceMapper.cpp >> ------------------------------------------------- >> ?162???? G1NodeDistributor itr(node_index); >> ?163 >> ?164???? for (uint i = start_idx; i < start_idx + num_regions; i++) { >> ?165?????? // If there are many pages to touch, different node ids >> will be used. >> ?166?????? uint processed_node_index = itr.next_node_index(); >> ?... >> ?179???????? zero_filled = _storage.commit(idx, 1, processed_node_index); >> ?180???????? itr.next(); >> ?181?????? } >> >> Do we really need to do this? Can't we just rely on the code in >> G1NUMA::touch_memory_roundrobin() to do the right thing? Or is this >> just to allow committing a single region on the correct17134 node? > Yes, this is to commit a single region and touch_memory_roundrobin() is > not called. > i.e. HeapRegion is smaller than page size, so we commit one page at once > and skip for next region until the page is filled with region. e.g. 2MB > page size, 1MB heap region. every 2 commit for heap regions, 1 page will > be committed. > However, your comment reminded me to enhance this area. Instead of > starting from the first node, we can just call G1NUMA::next_numa_index() > at G1NodeDistributor::next(). > Ok, I see, as long as we need to be able to commit to a specific node this is needed. >> --- >> >> src/hotspot/share/gc/g1/g1_globals.hpp >> -------------------------------------- >> Instead of adding those flags I think we should use log-tags and guard >> the logic by using log_is_enabled(), or LogTarget::is_enable() as you >> do in heapRegionManager.cpp. To allow these to be excluded with other >> logging turned on I suggest using specific log-tag-sets for each of >> them something like: >> G1PrintNUMAIdOfHeapRegions -> gc, numa, region (debug or trace) >> G1VerifyNUMAIdOfHeapRegions -> gc, numa, region, verification (debug >> or trace) > Will do as you suggested. > Kim also suggested same one but I wanted to hear opinions. :) > > G1PrintNUMAIdOfHeapRegions -> gc, numa, region (debug) > G1VerifyNUMAIdOfHeapRegions -> gc, numa, region, verification (debug) > >> --- >> >> src/hotspot/share/gc/g1/heapRegionManager.cpp >> --------------------------------------------- >> ?112?? if (mgr->num_active_nodes() > 1) { >> ?113???? uint valid_node_index = >> mgr->valid_node_index(requested_node_index); >> ?114???? // Try to allocate with requested node index. >> ?115???? hr = _free_list.remove_region_with_node_index(from_head, >> valid_node_index, NULL); >> ?116?? } >> >> I've seen some problems in my testing with this code, >> valid_node_index() now call _numa->next_numa_index() when the index >> isn't valid. This means that both the lower level commit code will >> call this function as well as the allocation path in some cases (for >> example single region humongous). I'm not sure if this will lead to >> any real problems but it makes it harder to understand what is going >> on underneath and I've seen it lead to having imbalance over the >> nodes. To avoid this I think we could instead do: >> >> if (mgr->num_active_nodes() > 1 && >> ??? mgr->is_valid_node_index(requested_node_index) { >> ? hr = _free_list.remove_region_with_node_index(from_head, >> requested_node_index, NULL); >> } >> >> This way we only remove with index if we have a valid index, otherwise >> we get a HeapRegion from the code below, which should be more or less >> the same as taking whatever index and removing with that: >> ?118?? if (hr == NULL) { >> ?... >> ?121???? hr = _free_list.remove_region(from_head); >> ?122?? } >> >> If this is an ok approach I think valid_node_index() is not longer >> used and can be removed. > Looks nice suggestion. > So instead of relying on G1NUMA::next_numa_index(), just using the first > region would relief the node imbalance. I like this but let me back with > some testing results. > >> --- >> >> src/hotspot/share/gc/g1/g1NUMA.inline.hpp >> ----------------------------------------- >> G1NUMA::next_numa_id() seems to be unused and could then be removed. > Will remove. > >> --- >> >> I'll continue looking at the patch and taking it for some spins. I >> might come with additional feedback later on, but these are my initial >> findings. > Okay! > > Thanks, > Sangheon > > >> >> Thanks, >> Stefan >> >> >> On 2019-09-21 07:19, sangheon.kim at oracle.com wrote: >>> Hi Kim, >>> >>> Many thanks for this thorough review! >>> >>> On 9/16/19 4:12 PM, Kim Barrett wrote: >>>>> On Sep 4, 2019, at 3:16 AM, sangheon.kim at oracle.com wrote: >>>>> >>>>> Hi all, >>>>> >>>>> Please review this patch, making G1 NUMA aware. >>>>> This is the first part of G1 NUMA implementation. >>>>> >>>>> - At G1 CollectedHeap initialization time, the given heap regions >>>>> will be split and touched. >>>>> - Internally node index(uint) is used instead of node id(int type) >>>>> for easier access to arrays. >>>>> - Only Linux is supported. >>>>> - New logs are under gc+heap+numa >>>>> - Current memory touch implementation will be replaced with >>>>> WorkGang by JDK-8230411: NUMA aware work gang. >>>>> >>>>> CR: https://bugs.openjdk.java.net/browse/JDK-8220310 >>>>> Webrev: http://cr.openjdk.java.net/~sangheki/8220310/webrev.0 >>>>> Testing: hs-tier 1 ~ 5 with +- UseNUMA. >>>>> >>>>> Thanks, >>>>> Sangheon >>> I had long off-line discussion with Kim, Thomas and Stefan J. >>> And 2 major changes are: >>> >>> 1) Memory touch thread is removed. This was a general way to >>> implement for both Linux and Solaris. However, Solaris is not >>> supported anymore, so memory touch thread is just removed. >>> And this made the code much simpler. >>> >>> 2) G1NUMA doesn't manage numa id of each page and we don't care about >>> each page instead HeapRegion is only cared. i.e. we assume numa id of >>> all pages between bottom and end of HeapRegion to be same. This also >>> made related codes much simpler. >>> >>> There are some other changes as well. >>> >>> >>>> Here is an initial set of comments. Some of these might be nonsense; >>>> I'm >>>> not all that familiar with NUMA support. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> Just in general, it's kind of disappointing that NUMA support for G1 >>>> apparently involves this much code, and is completely different from >>>> ParallelGC NUMA support, and both are completely different from ZGC >>>> NUMA support.? (And Shenandoah is probably yet another still.) >>>> Oh well... >>> Yes, I agree on that. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> According to off-line discussion, the intent is that a HeapRegion only >>>> contains memory from one NUMA node.? That isn't obvious though, and >>>> there >>>> are API and data structure choices that obscure that.? It seems like >>>> some >>>> things could be simplified with that constraint in mind.? I think >>>> some of >>>> that is a result of trying to leave open support for NUMA aware >>>> off-heap >>>> data structures, but that isn't part of this change, and might be >>>> better >>>> kept separate from NUMA aware Java heap support. >>>> >>>> I'll try to point out places where I think there are opportunities for >>>> simplification. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> According to off-line discussion, the touching mechanism might not be >>>> necessary.? If that's true, that would be a significant >>>> simplification.? For >>>> now I won't comment on the code for the touch threads, though I think >>>> I found some problems there before the discussion suggesting we might >>>> not need them happened. >>> Thanks. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1NUMA.hpp >>>> ?? 64 // Manages NUMA node related information and memory touch >>>> threads. >>>> ... >>>> >>>> I found the comment describing G1NUMA pretty hard to parse. Even >>>> after looking at the code I'm having trouble figuring out what's being >>>> said. >>>> >>>> This part I was able to figure out, but think it could be worded >>>> better. >>>> ?? 70 // Also provides a way to convert NUMA id and NUMA >>>> index(starting from zero and sequatial). >>>> ?? 71 // Actual NUMA ids would not be sequential nor not start from >>>> zero, so a conversion is necessary >>>> ?? 72 // between id and index. >>>> >>>> Something like >>>> >>>> ?? Also provides a mapping between NUMA indices (sequential, starting >>>> ?? from zero) and NUMA ids (which may not start at zero and may not be >>>> ?? sequential).? Indices are generally used by clients, and mapped to >>>> ?? ids when dealing with the OS/hardware layer. >>> DONE >>> >>>> >>>> (I *think* that's true.) >>>> >>>> But then, why expose node_ids at all in G1NUMA? It seems like node_ids >>>> could be completely relegated to an implementation detail that clients >>>> never need to deal with. >>> ... >>>> As evidence for not needing to expose ids to clients, there are no >>>> callers of numa_id_of_index in any of the series of 3 patches.? And >>>> G1NUMA::numa_ids() and G1MemoryMultNodeManager::node_ids() are only >>>> used in a couple of places for printing/logging and by >>>> WB_G1MemoryNodeIds.? I'm not sure what the last is for, but the >>>> printing and logging code could be isolated into G1NUMA. >>> Basically I totally agree with you for not exposing numa id. >>> Current patches are only using node id for logging purpose at some >>> locations because users are only interested on node id. >>> >>> Also Thread::lgrp_id() (renaming lgrp id, numa id etc.. is different >>> topic that we had before) is also public, so my feeling is current >>> try is enough. :) >>> >>> WB_G1MemoryNodeIds() is for jtreg testing purpose which checks >>> whether the given memory has node id as we expected. >>> >>>> >>>> Also, it looks like the index to node_id mapping provided here is >>>> duplicating the information provided by os::Linux::nindex_to_node. >>>> Not that this necessarily helps with anything in generic code... >>> Yes, I'm aware of _nindex_to_node but I think maintaining such >>> conversion at G1NUMA is okay. Other NUMA implementation such as >>> parallel gc relies on node id so completely removing node id at >>> general code seems different scope. >>> Unless you have strong opinion on addressing it in this patch, I >>> would like proceed as is. >>> If it is the case, we can consider your suggestion as a separate >>> enhancement. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>>> ?? 57?? virtual int* node_ids() const { static int dummy_id = 0; >>>> return &dummy_id; } >>>> >>>> Can the return type here be "const int*" ?? That would be better if >>>> possible. >>>> >>>> [This is probably moot if clients only deal with numa indexes and not >>>> node_ids, as suggested earlier.] >>> DONE >>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>>> ?? 63?? virtual uint guarantee_valid_node_index(uint node_index) >>>> const { return 0; } >>>> >>>> Given naming conventions elsewhere, the name of this function makes me >>>> expect it to be a verification function. >>> Agree. >>> valid_node_index() is better? webrev.1 changed name. >>> Or are you suggesting to remove this and then the caller do such >>> thing? e.g. >>> void A(uint node_index) { >>> uint valid_node_index = node_index; >>> if (!is_valid_node_index(valid_node_index)) { >>> ?? valid_node_index = random_numa_index(); >>> ... >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1NUMA.inline.hpp >>>> ?? 29 #include "gc/shared/taskqueue.inline.hpp" >>>> >>>> This #include seems to be just to obtain randomParkAndMiller. It >>>> would be >>>> better to refactor that function into its own file(s). Or make it >>>> part of >>>> some component providing various random number generators. >>> Not need because of below change. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1NUMA.inline.hpp >>>> ?? 45?? int seed = randomParkAndMiller(_seed); >>>> ?? 46?? Atomic::store(seed, &_seed); >>>> >>>> This sequence may lose updates.? I don't know if that's important. >>>> And why isn't this just using os::random(), or maybe refactoring that >>>> slightly to gain access to the external seed version.? I don't want >>>> this >>>> change to be made: >>>> http://cr.openjdk.java.net/~sangheki/8220310/webrev.0/src/hotspot/share/gc/shared/taskqueue.inline.hpp.udiff.html >>>> >>> Thomas and Stefan J. also complained on this. >>> Changed to get next numa index with round-robin manner. i.e. doesn't >>> use random() at all. >>> >>>> And do we *really* need randomness here at all?? Is there a reason not >>>> to simply cycle through the nodes? >>> Not really. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/os/linux/os_linux.cpp >>>> 3016?? size_t num_pages_left = size_in_bytes / page_size; >>>> >>>> Is size_in_bytes required to be a multiple of page_size?? I believe >>>> it is, >>>> and an assert of that would be useful. >>> This code is removed. And yes to answer to your question. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/os/linux/os_linux.cpp >>>> 3016?? size_t num_pages_left = size_in_bytes / page_size; >>>> 3017?? const size_t num_pages_total = num_pages_left; >>>> 3018?? size_t num_buffer_entries = MIN2(num_pages_left, max_pages); >>>> >>>> I think clearer would be >>>> >>>> 3017?? const size_t num_pages_total = size_in_bytes / page_size; >>>> 3018?? size_t num_buffer_entries = MIN2(num_pages_total, max_pages); >>>> >>>> and move this down to the loop setup: >>>> 3016?? size_t num_pages_left = num_pages_total; >>> This code is removed. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/os/linux/os_linux.cpp >>>> 3020?? void** pages = (void**)alloca(sizeof(void*) * >>>> num_buffer_entries); >>>> 3021?? int* status = (int*)alloca(sizeof(int) * num_buffer_entries); >>>> >>>> I think a ResourceMark and NEW_RESOURCE_ARRAYs would be preferable to >>>> using alloca() here.? alloca() has some dangers and I think is >>>> currently >>>> only used in some very specialized places where it really is needed. >>> This code is removed. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/os/linux/os_linux.cpp >>>> 3061 } >>>> 3062 // Moves current thread on a specific node and it will not >>>> migrate to >>>> >>>> Missing blank line between functions. >>> This code is removed. >>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/os/linux/os_linux.cpp >>>> 3031?????? pages[i] = (void*)base; >>>> >>>> Unnecessary cast. >>> This code is removed. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/os/linux/os_linux.cpp >>>> 3072???? case AFFINITY_NONE: >>>> 3073?????? // Do nothing. >>>> 3074?????? return true; >>>> 3075?????? break; >>>> >>>> Shouldn't this be calling numa_run_on_node with -1 node id? The >>>> documentation for numa_run_on_node() says "Passing -1 permits the >>>> kernel to schedule on all nodes again." >>> My understand of that description is let kernel decide the numa node >>> which is not actually want here. We want to move current thread to >>> the preferred numa node. >>> >>> Currently we don't need this API (numa_set_affinity() ) so removed on >>> webrev.1. >>> However following patch which is not scope of this JEP will use it. >>> I will add your suggestion below? when we need this. >>> >>>> >>>> But see also next comment about numa_affinity_set. >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/runtime/os.hpp >>>> ? 401?? // Available affinities. >>>> ? 402?? enum NumaAffinity { >>>> ? 403???? AFFINITY_STRONG, >>>> ? 404???? AFFINITY_NONE >>>> ? 405?? }; >>>> ? 406?? static bool numa_affinity_set(Thread* thread, int numa_id, >>>> NumaAffinity affinity); >>>> >>>> Could there ever be other affinities?? This API doesn't seem right. >>>> It seems to me it should be something like >>>> >>>> ?? static bool numa_set_affinity(Thread* thread, int numa_id); >>>> ?? static bool numa_clear_affinity(Thread* thread); >>>> >>>> No need for NumaAffinity enum. >>> The enum is also kind of remainder after removing Solaris >>> implementation which has more types. >>> But I agree with you as there are only 2 types now, it is simpler to >>> separate API without enum. >>> >>>> >>>> (I think numa_set/clear_affinity is better than >>>> numa_affinity_set/clear; >>>> numa is a "namespace" and we usually set/clear/get_something rather >>>> than something_set/clear/get.) >>>> >>>> Or maybe this should just be >>>> >>>> ?? static bool numa_set_affinity(Thread* thread, int numa_id); >>>> >>>> with "clearing" provided by passing AnyId as the numa_id? Still don't >>>> need the NumaAffinity enum for this. >>> Agree on this naming stuff. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1NUMA.cpp >>>> ? 124?? guarantee(rs.size() % page_size == 0, "Reserved space size >>>> (" SIZE_FORMAT ") should be aligned " >>>> ? 141?? guarantee(size_in_bytes % page_size == 0, "The given range >>>> (" SIZE_FORMAT ") should be aligned " >>>> ? 209???? guarantee((uintptr_t)G1HeapRegionSize % page_size == 0, >>>> "G1HeapRegionSize (" SIZE_FORMAT ") and page size (" >>>> ? 234???? guarantee((uintptr_t)_page_size % _heap_region_size == 0, >>>> "G1HeapRegionSize (" SIZE_FORMAT ") and page size (" >>>> >>>> Use is_aligned.? I think casts shouldn't be needed. >>> These lines are removed after major refactoring. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1NUMA.cpp >>>> ?? 43?? _numa_id_to_index_map = NEW_C_HEAP_ARRAY_RETURN_NULL(uint, >>>> _len_numa_id_to_index_map, mtGC); >>>> ?? 44?? memset(_numa_id_to_index_map, 0, sizeof(uint) * >>>> _len_numa_id_to_index_map); >>>> >>>> Using NEW_xxx_RETURN_NULL and then ignoring the possibility of a NULL >>>> result.? It would be better to use the non-_RETURN_NULL variant and >>>> get a decent error message rather than the segfault from attemptint to >>>> write to NULL. >>> Changed to use non-_RETURN_NULL version. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1NUMA.cpp >>>> ?? 34 void G1NUMA::init_numa_id_to_index_map(int* numa_ids, uint >>>> num_numa_ids) { >>>> >>>> int* numa_ids => const int* numa_ids. >>> DONE >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1NUMA.cpp >>>> ?? 34 void G1NUMA::init_numa_id_to_index_map(int* numa_ids, uint >>>> num_numa_ids) { >>>> ... >>>> ?? 44?? memset(_numa_id_to_index_map, 0, sizeof(uint) * >>>> _len_numa_id_to_index_map); >>>> Rather than this memset and then writing to all of them later, instead >>>> initialize all entries to InvalidNodeIndex and then assign from >>>> numa_ids. >>> DONE >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1MemoryTouchThread.cpp >>>> ? 261?? _wakeup_monitor = new Monitor(Mutex::nonleaf, "Wakeup >>>> Monitor for Touch Control", true, Monitor::_safepoint_check_never); >>>> ? 262?? _sync_monitor = new Monitor(Mutex::nonleaf, "Sync Monitor >>>> for Touch Control", true, Monitor::_safepoint_check_never); >>>> >>>> For singleton mutex/monitor, we generally prefer to put them in >>>> mutexLocker. >>>> Probably passed as constructor arguments to G1NUMAThreadsControl.? And >>>> I know there are counter-examples. >>> Not necessary as memory touch thread is gone. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>>> ?? 30 extern const uint AnyNodeIndex; >>>> ?? 31 extern const uint InvalidNodeIndex; >>>> >>>> Why are these at namespace scope.? And they seem misplaced here. >>>> Seems like they should belong to the G1NUMA class, which is >>>> responsible for indexing nodes and mapping between indexes and ids. >>> Changed to G1MemoryNodeManager::AnyNodeIndex. >>> And I wanted to hide or minimize exposing G1NUMA in general. >>> Memory node manager also has index related APIs, so let manager have >>> such constants seems okay. >>> >>>> >>>> ?? 30 const uint AnyNodeIndex = (uint)os::AnyId; >>>> ?? 31 const uint InvalidNodeIndex = (uint)os::InvalidId; >>>> And why these values?? That seems like a units mismatch.? Just use >>>> UINT_MAX and UINT_MAX - 1. >>> I don't have strong opinion on this. >>> But the intend is to make same value after casting for same meaning >>> constants instead of randomly chosen ones. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1NUMA.hpp >>>> ?? 80?? // For invalid numa id, return InvalidNodeIndex. So the >>>> caller need exception handling. >>>> >>>> "need exception handling"? >>> Thomas also commented on it so reworded. But finally the class is >>> removed. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/heapRegionManager.cpp >>>> ? 114???? uint allocated_node_index; >>>> >>>> This variable gets assigned via an out parameter here: >>>> ? 116???? hr = _free_list.remove_region_with_node_index(from_head, >>>> valid_node_index, &allocated_node_index); >>>> >>>> and by assignment here: >>>> ? 122???????? allocated_node_index = >>>> mgr->node_index_of_address(hr->bottom()); >>>> >>>> but is never used. If the lack of use is not itself a mistake, then >>>> the out >>>> parameter of remove_region_with_node_index appears to be unnecessary. >>> The variable will be used at patch3 (JDK-8220312) but slightly >>> changed to allow NULL at allocated_node_index parameter at >>> remove_region_with_node_index(). >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/heapRegionSet.inline.hpp >>>> ? 151 inline HeapRegion* >>>> FreeRegionList::remove_region_with_node_index(bool from_head, >>>> >>>> As written, I think this might always return NULL if >>>> requested_node_index is >>>> not a valid index. I think callers ensure that it's valid, but an >>>> assert of >>>> the argument would be help. >>> Done >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/heapRegionSet.inline.hpp >>>> ? 175???? if (cur == NULL || cur_depth >= max_search_depth) { // not >>>> found >>>> >>>> There's an off-by-one issue here. The max_search_depth time through the >>>> while-control-expression may have found a matching region, checked >>>> via the >>>> numa_index_of_address lookup. >>>> >>>> Either the test should be "current_numa_index != >>>> requested_numa_index" or >>>> the loop should be changed. >>> This code is removed. >>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/heapRegionSet.inline.hpp >>>> ? 151 inline HeapRegion* >>>> FreeRegionList::remove_region_with_node_index(bool from_head, >>>> >>>> I think the list manipulation could be simplified quite a bit.? I >>>> think we >>>> don't need the "last" variable.? And the splicing out of cur can be >>>> unified >>>> between the from_head and !from_head cases, e.g. something like this: >>>> >>>> ?? // Find the region to use, searching from _head or _tail as >>>> requested. >>>> ?? if (from_head) { >>>> ???? for (cur = _head; >>>> ????????? cur != NULL && cur_depth < max_search_depth; >>>> ????????? cur = cur->next(), ++cur_depth) { >>>> ?????? if (current_numa_index == >>>> numa->numa_index_of_address(cur->bottom)) { >>>> ???????? break; >>>> ?????? } >>>> ???? } >>>> ?? } else { >>>> ???? for (cur = _tail; >>>> ????????? cur != NULL && cur_depth < max_search_depth; >>>> ????????? cur = cur->prev(), ++cur_depth) { >>>> ?????? if (current_numa_index == >>>> numa->numa_index_of_address(cur->bottom)) { >>>> ???????? break; >>>> ?????? } >>>> ???? } >>>> ?? } >>>> ?? if (cur == NULL || cur_depth >= max_search_depth) { >>>> ???? return NULL;????????????????????? // Didn't find a region to use >>>> ?? } >>>> ?? // Splice the region out of the list. >>>> ?? HeapRegion* prev = cur->prev(); >>>> ?? HeapRegion* next = cur->next(); >>>> ?? if (prev == NULL) { >>>> ???? _head = next; >>>> ?? } else { >>>> ???? prev->set_next(next); >>>> ?? } >>>> ?? if (next == NULL) { >>>> ???? _tail = prev; >>>> ?? } else { >>>> ???? next->set_prev(prev); >>>> ?? } >>>> ?? cur->set_prev(NULL); >>>> ?? cur->set_next(NULL); >>>> ?? ... >>> Nice suggestion! >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1NUMA.cpp >>>> ? 166 class DumpIterator : public StackObj { >>>> >>>> Polluting global namespace.? Suggest making DumpIterator and the >>>> classes >>>> derived from it (G1HeapRegion{Larger,Smaller}Iterator) private nested >>>> classes of G1NUMA, making the names unimportant outside of that >>>> context. >>> Not need any more. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/heterogeneousHeapRegionManager.hpp >>>> ? 122?? virtual HeapRegion* allocate_free_region(HeapRegionType type); >>>> ? 123?? virtual HeapRegion* allocate_free_region(HeapRegionType >>>> type, uint node_index); >>>> >>>> The base class (HeapRegionManager) no longer has the one-arg >>>> function.? So >>>> this one-arg function is no longer overriding the base class >>>> definition. >>>> (Too bad it wasn't declared as C++11 "override".) >>>> >>>> I don't see any one-arg callers, so seems like it can just be removed. >>> DONE >>> Nice finding! :) >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1PageBasedNUMAVirtualSpace.cpp >>>> >>>> I dislike the new structure here, with G1PageBasedVirtualSpace now >>>> being >>>> both a base class and a concrete class (which is often a problematic >>>> design).? And the new derived class doesn't really do all that much. >>>> >>>> It seems like it would be much simpler to just have >>>> G1PageBasedVirtualSpace >>>> check UseNUMA to conditionalize it's commit and uncommit functions. >>>> Especially since it's commit already needs to have some knowlege of >>>> NUMA >>>> because of the new numa_index argument. >>> DONE >>> Removed NUMA version and reverted related changes to allow inherit >>> are reverted. >>> Now, G1PageBasedVirtualSpace does additional work after checking NUMA. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1PageBasedNUMAVirtualSpace.cpp >>>> ?? 52 void G1PageBasedNUMAVirtualSpace::uncommit(size_t start_page, >>>> size_t size_in_pages) { >>>> >>>> Do we actually need to do anything here?? We're resetting the >>>> G1NUMA::_numa_id_of_pages_table (to AnyId), but it's not clear we >>>> really >>>> need to do that.? When we later recommit any of those pages, that >>>> table will >>>> be updated appropriately. >>>> >>>> Maybe rather than having G1NUMA expose quite so much detail it should >>>> instead have post_commit and a pre_uncommit functions. >>> I think clearing the node index may help for debugging. Unlike >>> webrev.0, numa id/index information is located at HeapRegion which is >>> much simpler. i.e. HeapRegionManager::uncommit_regions() does >>> resetting node index to InvalidNodeIndex. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1NUMA.hpp >>>> ?? 75?? G1NUMAPageIdTable _numa_id_of_pages_table; >>>> >>>> I was surprised by the need for this, but I can't find anything in >>>> the NUMA >>>> API that provides this information.? That seems strange, because one >>>> would >>>> think that information must exist somewhere in the NUMA implementation. >>> Removed. >>> And this made the patch much simpler! >>> >>> And here are your comments from other email. >>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>>> ?? 64?? // Returns node index of the given address. >>>> ?? 65?? // If the node id of the address is invalid return randomly >>>> selected node index. >>>> ?? 66?? virtual uint valid_index_of_address(HeapWord* o) const { >>>> return 0; } >>>> >>>> This function is unused. >>> Removed. >>> >>>> ------------------------------------------------------------------------------ >>>> >>>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>>> ?? 70?? virtual uint node_index_of_address(HeapWord* o) const { >>>> return 0; } >>>> >>>> The only caller of this function is >>>> HeapRegionManager::allocate_free_region. >>>> As noted in an earlier comment, the result of that call is unused. >>> Removed as you commented but I had to revert as a part of >>> _numa_id_of_pages_table removal. >>> Previously we retrieved the page info from _numa_id_of_pages_table, >>> but new version reads directly via system call. >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>>> As pointed out to me offline, there*are*? ways to query the address -> >>>> associated numa node mapping.? So I'm going back and questioning the >>>> need >>>> for _numa_id_of_pages_table and the code for managing it. >>> As we discussed offline, I removed this table. So webrev.1 checks the >>> bottom of HeapRegion and then saves it at HeapRegion::_node_index. >>> However we do need a way to check node index of the given address and >>> so I added back. >>> >>> ?From my testing, checking a numa id of one page seems okay to do on >>> runtime. Previously I did checking for all pages of given memory so >>> it was a bit slow. >>> >>> One thing that didn't include at webrev.1 is combining with existing >>> one. >>> The newly introduced os::numa_get_address_id() is a copy from >>> zNUMA_linux.cpp. It may be unified but didn't try yet. >>> >>> webrev: >>> http://cr.openjdk.java.net/~sangheki/8220310/webrev.1 >>> http://cr.openjdk.java.net/~sangheki/8220310/webrev.1.inc (this may >>> not help much! :) ) >>> Testing: hs-tier 1 ~ 5 (with/without UseNUMA) >>> >>> Thanks, >>> Sangheon >>> >>>> >>>> ------------------------------------------------------------------------------ >>>> >>>> >>> > From erik.osterlund at oracle.com Tue Sep 24 09:34:38 2019 From: erik.osterlund at oracle.com (erik.osterlund at oracle.com) Date: Tue, 24 Sep 2019 11:34:38 +0200 Subject: RFR: 8231251: ZGC: Fix ZHeap includes In-Reply-To: References: Message-ID: <6f17404b-3854-dc3f-97e4-cb3e38f667e9@oracle.com> Hi Per, Looks amazing. Thanks, /Erik On 09/19/2019 04:41 PM, Per Liden wrote: > I noticed that the ZHeap includes needed a bit of a cleanup, as there > are many includes that are not longer needed, and a few are missing. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231251 > Webrev: http://cr.openjdk.java.net/~pliden/8231251/webrev.0 > > /Per From per.liden at oracle.com Tue Sep 24 09:35:40 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 24 Sep 2019 11:35:40 +0200 Subject: RFR: 8231251: ZGC: Fix ZHeap includes In-Reply-To: <6f17404b-3854-dc3f-97e4-cb3e38f667e9@oracle.com> References: <6f17404b-3854-dc3f-97e4-cb3e38f667e9@oracle.com> Message-ID: Thanks! /Per On 9/24/19 11:34 AM, erik.osterlund at oracle.com wrote: > Hi Per, > > Looks amazing. > > Thanks, > /Erik > > On 09/19/2019 04:41 PM, Per Liden wrote: >> I noticed that the ZHeap includes needed a bit of a cleanup, as there >> are many includes that are not longer needed, and a few are missing. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231251 >> Webrev: http://cr.openjdk.java.net/~pliden/8231251/webrev.0 >> >> /Per > From erik.osterlund at oracle.com Tue Sep 24 09:36:16 2019 From: erik.osterlund at oracle.com (erik.osterlund at oracle.com) Date: Tue, 24 Sep 2019 11:36:16 +0200 Subject: RFR: 8231266: ZGC: Minor cleanups in ZCollectedHeap and ZHeap In-Reply-To: <40e2f38a-582c-8a06-670e-dd51f8b042d4@oracle.com> References: <40e2f38a-582c-8a06-670e-dd51f8b042d4@oracle.com> Message-ID: Hi Per, Looks good. Thanks, /Erik On 09/19/2019 11:07 PM, Per Liden wrote: > ZCollectedHeap's is_oop() and print_location() should delegate to > ZHeap. Also adjusting a few types, so that the ZHeap interface uses > uintptr_t. This keeps things consistent and avoids a few casts in ZHeap. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231266 > Webrev: http://cr.openjdk.java.net/~pliden/8231266/webrev.0 > > /Per From erik.osterlund at oracle.com Tue Sep 24 09:37:34 2019 From: erik.osterlund at oracle.com (erik.osterlund at oracle.com) Date: Tue, 24 Sep 2019 11:37:34 +0200 Subject: RFR: 8231270: ZGC: Remove ZAddressSpace* and ZAddressReserved* In-Reply-To: <11eec445-9827-8150-efd9-59cb104e866c@oracle.com> References: <11eec445-9827-8150-efd9-59cb104e866c@oracle.com> Message-ID: <0b4ae8f7-5c70-cd31-c274-37144f3e0fb8@oracle.com> Hi Per, Looks good. Thanks, /Erik On 09/19/2019 11:08 PM, Per Liden wrote: > After JDK-8224820, ZAddressSpace{Start,End,Size} is not used for > anything other than being printed, so they can be removed. Also, the > two last uses of ZAddressReserved* can easily be replaced. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231270 > Webrev: http://cr.openjdk.java.net/~pliden/8231270/webrev.0 > > /Per From per.liden at oracle.com Tue Sep 24 11:17:56 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 24 Sep 2019 13:17:56 +0200 Subject: RFR: 8231266: ZGC: Minor cleanups in ZCollectedHeap and ZHeap In-Reply-To: References: <40e2f38a-582c-8a06-670e-dd51f8b042d4@oracle.com> Message-ID: Thanks! /Per On 9/24/19 11:36 AM, erik.osterlund at oracle.com wrote: > Hi Per, > > Looks good. > > Thanks, > /Erik > > On 09/19/2019 11:07 PM, Per Liden wrote: >> ZCollectedHeap's is_oop() and print_location() should delegate to >> ZHeap. Also adjusting a few types, so that the ZHeap interface uses >> uintptr_t. This keeps things consistent and avoids a few casts in ZHeap. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231266 >> Webrev: http://cr.openjdk.java.net/~pliden/8231266/webrev.0 >> >> /Per > From per.liden at oracle.com Tue Sep 24 11:18:02 2019 From: per.liden at oracle.com (Per Liden) Date: Tue, 24 Sep 2019 13:18:02 +0200 Subject: RFR: 8231270: ZGC: Remove ZAddressSpace* and ZAddressReserved* In-Reply-To: <0b4ae8f7-5c70-cd31-c274-37144f3e0fb8@oracle.com> References: <11eec445-9827-8150-efd9-59cb104e866c@oracle.com> <0b4ae8f7-5c70-cd31-c274-37144f3e0fb8@oracle.com> Message-ID: Thanks! /Per On 9/24/19 11:37 AM, erik.osterlund at oracle.com wrote: > Hi Per, > > Looks good. > > Thanks, > /Erik > > On 09/19/2019 11:08 PM, Per Liden wrote: >> After JDK-8224820, ZAddressSpace{Start,End,Size} is not used for >> anything other than being printed, so they can be removed. Also, the >> two last uses of ZAddressReserved* can easily be replaced. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231270 >> Webrev: http://cr.openjdk.java.net/~pliden/8231270/webrev.0 >> >> /Per > From erik.osterlund at oracle.com Tue Sep 24 11:28:26 2019 From: erik.osterlund at oracle.com (erik.osterlund at oracle.com) Date: Tue, 24 Sep 2019 13:28:26 +0200 Subject: RFR: 8224820: ZGC: Support discontiguous heap reservations In-Reply-To: <4e8f01be-5c0d-21f7-94bd-3210f2f4bce1@oracle.com> References: <5567c0b8-00e6-29f0-12c6-e067c924fdce@oracle.com> <8767693d-13c1-e6c8-4114-f2d73775e8e6@oracle.com> <4e8f01be-5c0d-21f7-94bd-3210f2f4bce1@oracle.com> Message-ID: <983e44dd-7786-f328-cf64-85308b7b2e34@oracle.com> Hi Stefan, Thanks for the review. /Erik On 09/24/2019 10:19 AM, Stefan Karlsson wrote: > Looks good. > > StefanK > > On 2019-09-17 12:05, Per Liden wrote: >> Hi, >> >> Discussed with Erik off-line. In light of things like future Windows >> support, we probably don't want to make the reservation code shared, >> instead just move it to os/posix. >> >> Here's a proposed webrev for doing just that: >> >> http://cr.openjdk.java.net/~pliden/8224820/webrev.0 >> >> /Per >> >> On 9/5/19 3:51 PM, Per Liden wrote: >>> On 8/9/19 11:59 AM, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> Today ZGC reserves a huge chunk of virtual address space when the >>>> JVM starts. This typically succeeds because we grab the VA before >>>> anyone else has time to do so. But if some ASLR library or >>>> something was to grab a tiny part of the desired VA, ZGC can't >>>> start. We should support discontiguous heap reservations to support >>>> this. >>>> >>>> On linux, by default, this does not happen. But on OS X, it does >>>> happen relatively frequently. So we need to fix this to allow a mac >>>> port. >>>> >>>> This patch implements a recursive algorithm for finding holes at >>>> 2MB granularities in the normally contiguous reservation when >>>> initializing the heap, removing them from our VA. >>>> >>>> This patch depends on 8224815, which depends on 8229189 and >>>> 8229278. They are all out for review. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8224820/webrev.00/ >>> >>> Looks good. However, please pass down max_capacity from >>> ZPageAllocator to ZVirtualMemory's constructor, instead of reading >>> MaxHeapSize. >>> >>> cheers, >>> Per >>> >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8224820 >>>> >>>> Thanks, >>>> /Erik From mark.reinhold at oracle.com Tue Sep 24 23:34:59 2019 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 24 Sep 2019 16:34:59 -0700 Subject: RFR: JDK-8231086: Shenandoah: Stronger invariant for object-arraycopy In-Reply-To: <5891bb2e-52ba-b3af-c89a-dfb60ac4e4ac@redhat.com> References: <5891bb2e-52ba-b3af-c89a-dfb60ac4e4ac@redhat.com> Message-ID: <20190924163459.120313957@eggemoggin.niobe.net> Perhaps you?re aware of this already, but after this change and its predecessor (8231085) it takes significantly longer to compile HotSpot, I?m guessing due to C++ template expansion. I saw build times on a four-core, eight-thread, 64GB Xeon box go from about 10m up to 15m. (Yes, I?m aware that one could avoid this slowdown by not building Shenandoah, but who would want to do that?) Bisecting the repo to identify this slowdown was tricky since the changeset for 8231085 (b1a394e15ae9) leaves the repo in a state that doesn?t build. Please avoid that going forward -- every changeset should leave the repo in a buildable state. - Mark From kim.barrett at oracle.com Wed Sep 25 01:44:00 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 24 Sep 2019 21:44:00 -0400 Subject: RFR(XL): 8220310: Implementation: NUMA-Aware Memory Allocation for G1, Mutator (1/3) In-Reply-To: <2b37edd6-3e0f-013d-1616-9d003f8ac1ed@oracle.com> References: <06ACBF87-ADBE-499F-B668-0274E4925B26@oracle.com> <2b37edd6-3e0f-013d-1616-9d003f8ac1ed@oracle.com> Message-ID: <74ACAF31-8233-482A-892E-0D2E7CA72F4F@oracle.com> > On Sep 21, 2019, at 1:19 AM, sangheon.kim at oracle.com wrote: > > webrev: > http://cr.openjdk.java.net/~sangheki/8220310/webrev.1 > http://cr.openjdk.java.net/~sangheki/8220310/webrev.1.inc (this may not help much! :) ) > Testing: hs-tier 1 ~ 5 (with/without UseNUMA) ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1AllocRegion.hpp 96 uint _node_index; Protected; should be private. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1Allocator.cpp 42 _mutator_alloc_region(NULL), Should be _mutator_alloc_regions (plural), since it's now an array. Similarly, these should be pluralized: 67 void G1Allocator::init_mutator_alloc_region() { 74 void G1Allocator::release_mutator_alloc_region() { And this 48 // The number of MutatorAllocRegions used, one per memory node. 49 size_t _num_alloc_region; ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1Allocator.cpp 53 G1Allocator::~G1Allocator() { 54 for (uint i = 0; i < _num_alloc_region; i++) { 55 _mutator_alloc_region[i].~MutatorAllocRegion(); 56 } 57 FREE_C_HEAP_ARRAY(MutatorAllocRegion, _mutator_alloc_region); 58 } --- should also be calling _mutator_alloc_region[i].release() ?? --- or does destructor do that? ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1Arguments.cpp 161 if (UseNUMA) { 162 if (FLAG_IS_DEFAULT(AlwaysPreTouch)) { 163 FLAG_SET_DEFAULT(AlwaysPreTouch, true); 164 } 165 if (!AlwaysPreTouch && FLAG_IS_CMDLINE(AlwaysPreTouch)) { 166 warning("Disabling AlwaysPreTouch is incompatible with UseNUMA. Disabling UseNUMA."); 167 FLAG_SET_ERGO(UseNUMA, false); 168 } 169 } Stefan asked about why AlwaysPreTouch is required when UseNUMA. I have a different question. Assuming UseNUMA does require AlwaysPreTouch, why is !AlwaysPreTouch winning here? Why not have UseNUMA win if they are conflicting? But see discussion below about G1RegionsSmallerThanCommitSizeMapper::commit_regions(), which suggested AlwaysPreTouch is required. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1PageBasedVirtualSpace.cpp 83 G1PageBasedVirtualSpace::~G1PageBasedVirtualSpace() { ... 92 _numa = NULL; 93 } [pre-existing] Destructors are for resource management. Nulling out / zeroing out members in a destructor generally isn't useful. This is really a comment on the existing code rather than a request to change anything. The addition of line 92 is okay in context, just the context is not good. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RegionToSpaceMapper.cpp 108 _next_node_index(G1MemoryNodeManager::mgr()->num_active_nodes() - 1), 109 _max_node_index(G1MemoryNodeManager::mgr()->num_active_nodes()) { Consider reversing the order of these members and their initializers, so the _next_node_index can use _max_node_index rather than another call to num_active_nodes(). ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RegionToSpaceMapper.cpp 113 uint next_node_index() const { 114 return _next_node_index; 115 } I think this is mis-named. It's the current index for the distributor. I think it should just be called "node_index". ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1RegionToSpaceMapper.cpp I'm confused by G1RegionsSmallerThanCommitSizeMapper::commit_regions(). For the LargerThan mapper, we have a sequence of regions that completely covers a sequence of pages, and we commit all of the associated pages using the requested node_index. For the SmallerThan mapper, we have a sequence of regions split up into subsequences that are each contained in a single page. The first such subsequence might be on an already committed page. Similarly for the last subsequence. Nothing is done to those pages. In between there may be a series of region sequences, with each region sequence on a single page. If there are more than one of these region sequences then more than one page will need to be committed. As we step through the seuqnce of pages and commit them, we also step the numa index to use for each page. Stefan asked a question in this area about the mechanism by which the node stepping is provided, and you responded with what sounds like an improvement. But I have a different question. Why are we committing different pages on different numa nodes? The caller requested these regions be on the requested node. Why are we not honoring that request (as much as possible within the constraints of possible leading and trailing regions being on already committed pages.) The comment for G1NodeDistributor discusses (at a high level) what it's doing (e.g. a short summary of the above description), but there is no discussion of why that distribution is needed or desirable. There might be a good reason for this behavior, in which case your response with an improvement sounds good. But if so, I'm guessing I won't be the only one who doesn't know what that reason might be, and it would be good to provide an explanatory comment. And of course, if there isn't a good reason... I think there is also a problem here if AlwaysPreTouch is false. (As discussed earlier, maybe it isn't required to be true.) The node index for the committed regions gets set (in make_regions_available) via the result of the syscall, so we really need pretouch to have been done. The alternative would be to assume commit_regions used the requested numa node. But with the request stepping that wouldn't hold. Of course, it also doesn't hold for any leading or trailing regions that were covered by already committed pages. I think this is the basis of your argument that AlwaysPreTouch is required for UseNUMA, and I think I'm now agreeing. Otherwise we may think the leading and trailing regions in the sequence are on a different node than they actually are, since the associated pages may have already been committed on a different node than requested, but not yet touched. But I still don't know why we would want to cycle through nodes for the "middle" pages. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegionManager.cpp 127 if (G1VerifyNUMAIdOfHeapRegions) { 128 // Read actual node index via system call. 129 uint actual_node_index = mgr->index_of_address(hr->bottom()); 130 if (hr->node_index() != actual_node_index) { Can we actually do this here? I thought the system call only gave a useful answer if the addressed location has been paged in. I'm not sure that's necessarily happened at this point. I think Stefan suggested the logging of mismatches between requested and actual numa node for a region should occur at region retirement. We could log mismatches there and correct the region's information. But see discussion above about G1RegionsSmallerThanCommitSizeMapper::commit_regions(). If AlwaysPreTouch is indeed required, then this code is okay. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/heapRegionSet.inline.hpp 156 HeapRegion * cur; s/HeapRegion */HeapRegion*/ ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp 38 static const uint InvalidNodeIndex = (uint)os::InvalidId; 39 static const uint AnyNodeIndex = (uint)os::AnyId; I complained about these in my review of webrev.0. These are making very strong assumptions about the values of the os Id values, for no good reason that I can see. You responded "But the intend is to make same value after casting for same meaning constants instead of randomly chosen ones." I don't buy that. There aren't any casts that I can see between NUMA ids and indexes. Nor should there be any such casts. If there were, I'd strongly question them, as being units mismatches. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp 54 virtual const int* node_ids() const { static int dummy_id = 0; return &dummy_id; } dummy_id should be const. I would probably put that definition in the .cpp file. I've run into multiple compiler bugs with function scoped static variables in inline functions. Not recently, but I'm paranoid. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp 34 class G1MemoryNodeManager : public CHeapObj { ... 43 static G1MemoryNodeManager* create(); Given that we have a factory function that should be used for creation, the constructor ought to be non-public. It needs to be protected so the derived G1MemoryNodeManager can refer to it. A different approach would have G1MemoryNodeManager be abstract (with all virtuals but the destructor being pure), with hidden (possibly private nested) classes for the single-node and multi-node cases. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1MemoryNodeManager.cpp 62 if (UseNUMA SOLARIS_ONLY(&& false)) { I thought we were only providing Linux support. This seems like it would attempt to work (and probably fail somewhere later) on other platforms (anything not Linux or Solaris). ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1MemoryNodeManager.cpp 64 G1NUMA* numa = new G1NUMA(); 65 66 if (numa != NULL) { numa cannot be NULL here. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1MemoryNodeManager.cpp 86 G1MemoryMultiNodeManager::~G1MemoryMultiNodeManager() { 87 delete _numa; 88 } This is leaving a stale pointer to the G1NUMA object in wherever G1NUMA::set_numa stashed it. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1CollectedHeap.cpp 1500 _mem_node_mgr(G1MemoryNodeManager::create()), Maybe this manager should be created in G1CH::initialize() (and initialized here to NULL). Then the page_size could be passed to create, and there wouldn't be a need to later set the page size of the manager and pass that along to the G1NUMA, instead both getting it as a constructor argument. Then the associated setters go away too. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1NUMA.hpp 95 // Gets a next valid numa id. 96 inline int next_numa_id(); Appears to be unused. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1MemoryNodeManager.cpp 102 uint G1MemoryMultiNodeManager::index_of_current_thread() const { 103 int node_id = os::numa_get_group_id(); 104 return _numa->index_of_numa_id(node_id); 105 } Other than here, os::numa_xxx usage is encapsulated in G1NUMA, with the manager forwarding to the G1NUMA object as needed. I suggest doing that here too. (Note that this file doesn't #include os.hpp.) I think doing so eliminates the need for G1NUMA::index_of_numa_id(), which also seems like a good thing. ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1NUMA.cpp 92 void G1NUMA::touch_memory(address aligned_address, size_t size_in_bytes, uint numa_index) { Assert aligned_address is page aligned? Assert size_in_bytes is a page aligned? ------------------------------------------------------------------------------ src/hotspot/share/gc/g1/g1NUMA.cpp 42 memset(_numa_id_to_index_map, 43 G1MemoryNodeManager::InvalidNodeIndex, 44 sizeof(uint) * _len_numa_id_to_index_map); memset only works here because all bytes of InvalidNodeIndex happen to have the same value. I would prefer an explicit fill loop rather than memset here. Or a static assert on the value, but that's probably more code. ------------------------------------------------------------------------------ From kim.barrett at oracle.com Wed Sep 25 01:50:06 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 24 Sep 2019 21:50:06 -0400 Subject: RFR(XL): 8220310: Implementation: NUMA-Aware Memory Allocation for G1, Mutator (1/3) In-Reply-To: <74ACAF31-8233-482A-892E-0D2E7CA72F4F@oracle.com> References: <06ACBF87-ADBE-499F-B668-0274E4925B26@oracle.com> <2b37edd6-3e0f-013d-1616-9d003f8ac1ed@oracle.com> <74ACAF31-8233-482A-892E-0D2E7CA72F4F@oracle.com> Message-ID: <80DABD88-FA7C-4614-8698-CCD00E5F6F42@oracle.com> > On Sep 24, 2019, at 9:44 PM, Kim Barrett wrote: > >> On Sep 21, 2019, at 1:19 AM, sangheon.kim at oracle.com wrote: >> >> webrev: >> http://cr.openjdk.java.net/~sangheki/8220310/webrev.1 >> http://cr.openjdk.java.net/~sangheki/8220310/webrev.1.inc (this may not help much! :) ) >> Testing: hs-tier 1 ~ 5 (with/without UseNUMA) I forgot to say that, while I had a lot of comments, I think webrev.1 is a big improvement. My new comments are detailed issues and not suggestions for major refactorings or the like. From rkennke at redhat.com Wed Sep 25 07:13:59 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 25 Sep 2019 09:13:59 +0200 Subject: RFR: JDK-8231086: Shenandoah: Stronger invariant for object-arraycopy In-Reply-To: <20190924163459.120313957@eggemoggin.niobe.net> References: <5891bb2e-52ba-b3af-c89a-dfb60ac4e4ac@redhat.com> <20190924163459.120313957@eggemoggin.niobe.net> Message-ID: Hi Mark, > Perhaps you?re aware of this already, but after this change and its > predecessor (8231085) it takes significantly longer to compile HotSpot, > I?m guessing due to C++ template expansion. I saw build times on a > four-core, eight-thread, 64GB Xeon box go from about 10m up to 15m. Ugh. I just tried on my TR1950X box with 64G mem, before it was 3:30m, now it takes 5:06. So yeah, that's bad. It is also a bit surprising, because the change *removed* a whole lot of template crap. It also added some (smaller) templated loop. I'll investigate what causes the slowdown and if we can reasonably fix this. > (Yes, I?m aware that one could avoid this slowdown by not building > Shenandoah, but who would want to do that?) :-) > Bisecting the repo to identify this slowdown was tricky since the > changeset for 8231085 (b1a394e15ae9) leaves the repo in a state that > doesn?t build. Please avoid that going forward -- every changeset > should leave the repo in a buildable state. I'm sorry for that. I should probably have folded the two changes together into one. I'll avoid it in the future. Roman From per.liden at oracle.com Wed Sep 25 08:57:28 2019 From: per.liden at oracle.com (Per Liden) Date: Wed, 25 Sep 2019 10:57:28 +0200 Subject: RFR: 8231294: ZGC: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted002 fails Message-ID: The resexhausted002 test uses an 8M heap, which is a bit too small for ZGC. Bumping it to 128M to allow the test to pass on ZGC too. Bug: https://bugs.openjdk.java.net/browse/JDK-8231294 Webrev: http://cr.openjdk.java.net/~pliden/8231294/webrev.0 Testing: Manual testing with G1 and ZGC /Per From thomas.schatzl at oracle.com Wed Sep 25 09:40:21 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 25 Sep 2019 11:40:21 +0200 Subject: RFR: 8231153: Improve concurrent refinement statistics In-Reply-To: <1DADC595-3106-4CE7-BA5D-7B6C7EE0E81E@oracle.com> References: <1DADC595-3106-4CE7-BA5D-7B6C7EE0E81E@oracle.com> Message-ID: <4a851a19-0979-c696-0c80-1165bd755834@oracle.com> Hi Kim, On 24.09.19 04:58, Kim Barrett wrote: > Please review this change to collection additional statistics about > the generation and processing of dirty cards. These are needed for > future improvements to the control of the concurrent refinement > threads (JDK-8137022). > > This change adds tracking and prediction of the rate at which > concurrent refinement threads process cards. It also adds tracking > and prediction of the rate at which mutator threads generate cards. > > As part of that, there is now tracking of total cards processed by the > concurrent refinement threads and the mutator threads. This is now > used by G1RemsetSummary instead of the buffer counts previously reported. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8231153 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8231153/open.00/ > > Testing: > mach5 tier1-5 > some local by-hand testing to look at the rate tracking. > I think the change in functionality is good. I can't really judge the appropriateness of the input values to the predictors since I do not know their actual use and haven't gotten a feeling about whether there is a problem with that :) My experience is that sometimes it is better to be slightly inaccurate in the values passed to reduce impact of outliers; in this case this may e.g. be more appropriate to use concurrently scanned cards instead of the actual concurrently refined cards as the difference are cards filtered out during refinement, and actual work is only done on *scanned* cards anyway. That may not be the case here, just an example - I can't tell. E.g. there may be a huge change in refined cards, but scanned cards do not change, and you are actually really interested in the latter. When reading the change I had the following thoughts to improve readability: - maybe some comment somewhere what "scanned" really means compared to "refined". Initially I was surprised with the change at G1RemSet::_num_conc_scanned_cards, but some thinking made me aware of the difference. - the change reuses the "processed" term for counted cards in a few places, and it is unclear to me what the difference to just "refined" cards would be in some cases. I.e. I think the additional term for apparently the same thing seems unnecessary, and I would recommend changing it to just "refined". E.g. G1DirtyCardQueueSet::refine_buffer(, size_t* cards_processed) -> ...(, size_t* num_cards_refined). There is already a distinction between concurrently and mutator processed cards by adding the appropriate term to the names, so "processed" seems superfluous. Also seen in G1ConcurrentRefineThread::run_service(). - I would also suggest to add a "num_" prefix to numbers/counts of values. E.g. in the above method declaration it is unclear whether "cards_processed" is a pointer to an array of cards or simply the number/count of cards given only the name. Other occurrences: G1ConcurrentRefineThread::refined_cards -> num_refined_cards G1DirtyCardQueueSet::mutator_refined_cards -> num_mutator_refined_cards I saw that the change actually removes a few of these prefixes in G1RemSetSummary, which I am not really happy about; this extra redundancy does help with readability imho, particularly when you pass them around (also when you start dealing with both at the same time :)) - in G1Policy::_pending_cards should be renamed to "_pending_cards_at_start_of_gc" since we also now have a "_pending_cards_after_last_gc" to distinguish their use a little better? - pre-existing: probably rename G1RemSet::_num_conc_scanned_cards and G1RemSetSummary::_conc_scanned_cards to "_concurrent_scanned_cards" to match the "_concurrent_refined_cards". - I would remove the two "Bleh" in g1ConcurrentRefine.cpp - while I can see their point I would probably rather file an enhancement request :) - not sure, but I think exposing size() and start() and in G1FreeIdSet seems unnecessary: the only user is G1DirtyCardQueueSet anyway, and it is already owner of G1FreeIdSet. I.e. it knows these values already (and passes it to the initializer of the G1FreeIdSet instance, and already has a getter for the size() value), so getting it back from G1FreeIdSet seems a bit strange to me, but I am okay with current code. Thanks, Thomas From rkennke at redhat.com Wed Sep 25 10:11:13 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 25 Sep 2019 12:11:13 +0200 Subject: RFR: JDK-8231447: Shenandoah: Compilation-time regression after JDK-8231086 Message-ID: <2c2c9adc-cb89-5ac6-a787-24e118e857bf@redhat.com> With JDK-8231086, compilation time of fastdebug build regressed from ~3:30m to ~5:00m here. See also: https://mail.openjdk.java.net/pipermail/shenandoah-dev/2019-September/010622.html The problem appears to be the presence of clone-barrier code in shenandoahBarrierSet.inline.hpp which is then subsequently processed by anything that includes that file (incl. and in particular the Access API). I propose to fix it by breaking out the clone_barrier() code into its own file, and only include that where we need it shenandoahRuntime.cpp, and let the access-API clone() call a non-inlined entry point that resides in shenandoahBarrierSet.cpp. This brings compile-time back to what it was before. http://cr.openjdk.java.net/~rkennke/JDK-8231447/webrev.00/ Testing: hotspot_gc_shenandoah Ok? Roman From shade at redhat.com Wed Sep 25 10:17:00 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 25 Sep 2019 12:17:00 +0200 Subject: RFR: JDK-8231447: Shenandoah: Compilation-time regression after JDK-8231086 In-Reply-To: <2c2c9adc-cb89-5ac6-a787-24e118e857bf@redhat.com> References: <2c2c9adc-cb89-5ac6-a787-24e118e857bf@redhat.com> Message-ID: On 9/25/19 12:11 PM, Roman Kennke wrote: > http://cr.openjdk.java.net/~rkennke/JDK-8231447/webrev.00/ Looks okay. *) The include guard is not correct in shenandoahBarrierSetClone.inline.hpp: 24 #ifndef SHARE_GC_SHENANDOAH_SHENANDOAHBARRIERSETCLONE_HPP 25 #define SHARE_GC_SHENANDOAH_SHENANDOAHBARRIERSETCLONE_HPP ... 98 #endif // SHARE_GC_SHENANDOAH_SHENANDOAHBARRIERSETCLONE_HPP Should be SHARE_GC_SHENANDOAH_SHENANDOAHBARRIERSETCLONE_INLINE_HPP? *) Does this compile with --disable-precompiled-headers? -- Thanks, -Aleksey From rkennke at redhat.com Wed Sep 25 10:29:00 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 25 Sep 2019 12:29:00 +0200 Subject: RFR: JDK-8231447: Shenandoah: Compilation-time regression after JDK-8231086 In-Reply-To: References: <2c2c9adc-cb89-5ac6-a787-24e118e857bf@redhat.com> Message-ID: <7f86798b-a659-2e8c-bcd7-7024d87e5dd6@redhat.com> > On 9/25/19 12:11 PM, Roman Kennke wrote: >> http://cr.openjdk.java.net/~rkennke/JDK-8231447/webrev.00/ > > Looks okay. > > *) The include guard is not correct in shenandoahBarrierSetClone.inline.hpp: > > 24 #ifndef SHARE_GC_SHENANDOAH_SHENANDOAHBARRIERSETCLONE_HPP > 25 #define SHARE_GC_SHENANDOAH_SHENANDOAHBARRIERSETCLONE_HPP > ... > 98 #endif // SHARE_GC_SHENANDOAH_SHENANDOAHBARRIERSETCLONE_HPP > > Should be SHARE_GC_SHENANDOAH_SHENANDOAHBARRIERSETCLONE_INLINE_HPP? Right: http://cr.openjdk.java.net/~rkennke/JDK-8231447/webrev.01/ > *) Does this compile with --disable-precompiled-headers? yes. Good? Roman From shade at redhat.com Wed Sep 25 10:29:45 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 25 Sep 2019 12:29:45 +0200 Subject: RFR: JDK-8231447: Shenandoah: Compilation-time regression after JDK-8231086 In-Reply-To: <7f86798b-a659-2e8c-bcd7-7024d87e5dd6@redhat.com> References: <2c2c9adc-cb89-5ac6-a787-24e118e857bf@redhat.com> <7f86798b-a659-2e8c-bcd7-7024d87e5dd6@redhat.com> Message-ID: On 9/25/19 12:29 PM, Roman Kennke wrote: > Right: > http://cr.openjdk.java.net/~rkennke/JDK-8231447/webrev.01/ > >> *) Does this compile with --disable-precompiled-headers? > > yes. > > Good? Good. -- Thanks, -Aleksey From rkennke at redhat.com Wed Sep 25 10:36:55 2019 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 25 Sep 2019 12:36:55 +0200 Subject: RFR: JDK-8231086: Shenandoah: Stronger invariant for object-arraycopy In-Reply-To: References: <5891bb2e-52ba-b3af-c89a-dfb60ac4e4ac@redhat.com> <20190924163459.120313957@eggemoggin.niobe.net> Message-ID: Hi Mark, We identified and fixed the problem. See: https://mail.openjdk.java.net/pipermail/shenandoah-dev/2019-September/010626.html and: https://bugs.openjdk.java.net/browse/JDK-8231447 Thanks for raising attention to this issue! Roman > Hi Mark, > >> Perhaps you?re aware of this already, but after this change and its >> predecessor (8231085) it takes significantly longer to compile HotSpot, >> I?m guessing due to C++ template expansion.? I saw build times on a >> four-core, eight-thread, 64GB Xeon box go from about 10m up to 15m. > > Ugh. > I just tried on my TR1950X box with 64G mem, before it was 3:30m, now it > takes 5:06. So yeah, that's bad. > > It is also a bit surprising, because the change *removed* a whole lot of > template crap. It also added some (smaller) templated loop. > > I'll investigate what causes the slowdown and if we can reasonably fix > this. > >> (Yes, I?m aware that one could avoid this slowdown by not building >> ? Shenandoah, but who would want to do that?) > > :-) > >> Bisecting the repo to identify this slowdown was tricky since the >> changeset for 8231085 (b1a394e15ae9) leaves the repo in a state that >> doesn?t build.? Please avoid that going forward -- every changeset >> should leave the repo in a buildable state. > > I'm sorry for that. I should probably have folded the two changes > together into one. I'll avoid it in the future. > > Roman > From thomas.schatzl at oracle.com Wed Sep 25 10:42:51 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Wed, 25 Sep 2019 12:42:51 +0200 Subject: RFR (L): 8230706: Waiting on completion of strong nmethod processing causes long pause times with G1 Message-ID: Hi all, can I have reviews for this change that fixes a regression introduced in jdk11? So, currently in g1, in the concurrent start pause we have to mark through all live oops referenced from thread stacks to keep those alive across the concurrent marking in presence of class unloading. These are "strong" nmethods. However G1 also scans oops of nmethods as part of the per-region nmethod remembered sets, but should not mark their oops as their classes should remain unloadable. These are "weak" nmethods. Regardless of either way of processing, nmethods are claimed, to prevent processing them multiple times as happens often. Which means that if an nmethod is claimed first via the "weak" path, its oops will not be marked properly, and *boom*. In order to prevent this, currently there is a hard synchronization, i.e. wait barrrier between strong nmethod processing and the actual evacuation (which ultimately does the weak nmethod processing). This is a problem, particularly since jdk11, because stacks are claimed on a per thread basis. I.e. an extremely deep stack could make all 100's or 1000's of your cpu cores wait idly. The "particular since jdk11" part is that since then the amount of work done during nmethod iteration is only bounded by the amount of live objects to copy (i.e. JDK-6672778 added task queue trimming which is generally a good thing), which can take a lot of time. In this case, in some internal Cassandra setup we have seen ridiculously long waiting times as a single thread apparently evacuates the whole young gen... :( This change moves the wait to the end of the evacuation, remembering any nmethods that were claimed by the weak nmethod processing before the strong nmethod processing got to it. Since the amount of nmethods that need to be marked through has always been very low (single digit), that phase took <0.1ms typically. There is some attempt to parallelize this phase based on the number of nmethods (this pass only needs to mark the oops <= TAMS in nmethods, no more) anyway though. I will look into merging parallel phases into a single one in the post-evacuation phase soon to get rid of this additional spin-up of threads (only during concurrent mark) again. During this work I tried several alternatives that were rejected: - disabling task queue trimming; that works, but still has the problem with deep thread stacks - moving the actual wait deep into the evacuation code: made a mess with the code, and still does not really solve the problem - instead of remembering nmethods, concurrently process them. Does not work because x86 embedded oops are not word-aligned, so this is not a good idea. There is always the option of doing the stack scanning in the concurrent phase: that seemingly requires much more work, e.g. by using method return barriers and has been left as a future enhancement. CR: https://bugs.openjdk.java.net/browse/JDK-8230706 Webrev: http://cr.openjdk.java.net/~tschatzl/8230706/webrev/ Testing: hs-tier1-5, many cassandra runs - no more exceptionally long concurrent start pauses :) Thanks, Thomas From mark.reinhold at oracle.com Wed Sep 25 18:48:36 2019 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Wed, 25 Sep 2019 11:48:36 -0700 Subject: RFR: JDK-8231086: Shenandoah: Stronger invariant for object-arraycopy In-Reply-To: References: <5891bb2e-52ba-b3af-c89a-dfb60ac4e4ac@redhat.com> <20190924163459.120313957@eggemoggin.niobe.net> Message-ID: <20190925114836.36816618@eggemoggin.niobe.net> 2019/9/25 3:36:55 -0700, Roman Kennke : > We identified and fixed the problem. See: > > https://mail.openjdk.java.net/pipermail/shenandoah-dev/2019-September/010626.html > > and: > https://bugs.openjdk.java.net/browse/JDK-8231447 > > Thanks for raising attention to this issue! Sure thing! - Mark From kim.barrett at oracle.com Thu Sep 26 01:50:52 2019 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 25 Sep 2019 21:50:52 -0400 Subject: RFR: 8231153: Improve concurrent refinement statistics In-Reply-To: <4a851a19-0979-c696-0c80-1165bd755834@oracle.com> References: <1DADC595-3106-4CE7-BA5D-7B6C7EE0E81E@oracle.com> <4a851a19-0979-c696-0c80-1165bd755834@oracle.com> Message-ID: > On Sep 25, 2019, at 5:40 AM, Thomas Schatzl wrote: > > Hi Kim, > > On 24.09.19 04:58, Kim Barrett wrote: >> [?]CR: >> https://bugs.openjdk.java.net/browse/JDK-8231153 >> Webrev: >> https://cr.openjdk.java.net/~kbarrett/8231153/open.00/ >> Testing: >> mach5 tier1-5 >> some local by-hand testing to look at the rate tracking. > > I think the change in functionality is good. I can't really judge the appropriateness of the input values to the predictors since I do not know their actual use and haven't gotten a feeling about whether there is a problem with that :) Thanks for looking at it. The current algorithm for controlling refinement thread activation has problems because its inputs have little to do with what it should be trying to achieve. These new predictors provide information that I currently think will be useful to making improvements there, but since I haven?t fully worked out the details there might be further adjustments coming along later. I?m only responding to some of your comments here; I?ll be responding to others later, along with a new webrev. But in some cases I have some questions needing answers before proceeding with some of the rework. > My experience is that sometimes it is better to be slightly inaccurate in the values passed to reduce impact of outliers; in this case this may e.g. be more appropriate to use concurrently scanned cards instead of the actual concurrently refined cards as the difference are cards filtered out during refinement, and actual work is only done on *scanned* cards anyway. That may not be the case here, just an example - I can't tell. > E.g. there may be a huge change in refined cards, but scanned cards do not change, and you are actually really interested in the latter. I considered changing RemSet::refine_card_concurrently() to return a bool value indicating whether the card was scanned or not. But I'm not sure that's really worthwhile, since most of the the reasons for not scanning are (I think) relatively rare. (Stale cards and failure to be filtered earlier for timing reasons.) The exception is the HCC. However, doing that would really just introduce another rate, e.g. the expected fraction of enqueued cards that need scanning. I have seen large variability in that fraction. It's been a while since I was looking at that, but I think it had to do with the difference between young-only and mixed GCs. It might be that separate statistics need to be maintained for those cases. But I also think the recent changes to the redirtying process may ameliorate that difference. I'll be looking into that as part of future changes to make use of this new data. > - I would also suggest to add a "num_" prefix to numbers/counts of values. I guess our tastes differ, because I strongly disliked those num_ prefixes. They smack of Hungarian Notation and the like (of which I'm not a fan). > - pre-existing: probably rename G1RemSet::_num_conc_scanned_cards and G1RemSetSummary::_conc_scanned_cards to "_concurrent_scanned_cards" to match the "_concurrent_refined_cards?. RemSet::_num_conc_scanned_cards is only used for logging, and has a poor implementation (racy increments). Given its usage and being in a somewhat performance critical code path, I'm reluctant to fix the implementation. I considered just removing it completely, since the value isn't critical and could be misleading. What do you think? > - I would remove the two "Bleh" in g1ConcurrentRefine.cpp - while I can see their point I would probably rather file an enhancement request :) Oops, forgot to clean up those comments. I don?t think there?s a useful RFR here. The problem is fundamental to how we do higher order functions in this code base. I?ve tried being different in some places and mostly got push-back to conform. > - not sure, but I think exposing size() and start() and in G1FreeIdSet seems unnecessary: the only user is G1DirtyCardQueueSet anyway, and it is already owner of G1FreeIdSet. I.e. it knows these values already (and passes it to the initializer of the G1FreeIdSet instance, and already has a getter for the size() value), so getting it back from G1FreeIdSet seems a bit strange to me, but I am okay with current code. The start() function is used in DCQS::mut_process_buffer(). However, the start offset now appears to be a premature generalization. In the RFR for JDK-8216258 I said Also generalized to allow the id set to start with a non-zero id, because that looks like it might simplify other changes I'm working on. Nothing seems to have happened with that, and I no longer even recall what changes I was thinking about there. Maybe I should back out the new start() and size() and eliminate the support for a start offset entirely. I could do that as part of this change, or save it for another RFE. From maoliang.ml at alibaba-inc.com Thu Sep 26 06:49:05 2019 From: maoliang.ml at alibaba-inc.com (Liang Mao) Date: Thu, 26 Sep 2019 14:49:05 +0800 Subject: =?UTF-8?B?UmU6IEcxIHBhdGNoIG9mIGVsYXN0aWMgSmF2YSBoZWFw?= In-Reply-To: <3140197d-8cab-4a86-af92-58431c74cb6b.maoliang.ml@alibaba-inc.com> References: <6270ce59-4a8e-431e-9ccf-f6d2c0f927eb.maoliang.ml@alibaba-inc.com> , , <1267a5dd2cf6cc1d03df64d07a06ba0f45195951.camel@oracle.com>, <3140197d-8cab-4a86-af92-58431c74cb6b.maoliang.ml@alibaba-inc.com> Message-ID: Hi All, Here is the user guide of G1ElasticHeap patch. Hope it will help to understand. G1ElasticHeap G1ElasticHeap is a GC feature to return memory of Java heap to OS to reduce the memory footprint of Java process. To enable this feature, you need to use G1 GC by options: -XX:+UseG1GC -XX:+G1ElasticHeap. ## Usage There are 3 modes which can be enabled in G1ElasticHeap. ### 1. Periodic uncommit Memory will be uncommitted by periodic GC. To enable periodic uncommit, use option -XX:+ElasticHeapPeriodicUncommit or dynamically enable the option via jinfo: `jinfo -flag +ElasticHeapPeriodicUncommit PID` Related options: > ElasticHeapPeriodicYGCIntervalMillis, 15000 \ (target young GC interval 15 seconds in default) \ (eg, if Java runs with MaxNewSize=4g, young GC every 30 seconds, G1ElasticHeap will keep 15s GC interval and make a max 2g young generation to uncommit 2g memory) > ElasticHeapPeriodicInitialMarkIntervalMillis, 3600000 \ (Target initial mark interval, 1 hour in default. Unused memory of old generation will be uncommitted after last mixed GC.) > ElasticHeapPeriodicUncommitStartupDelay, 300 \ (Delay after startup to do memory uncommit, 300 seconds in default) > ElasticHeapPeriodicMinYoungCommitPercent, 50 \ (Percentage of young generation to keep, default 50% of the young generation will not be uncommitted) ### 2. Generation limit To limit the young/old generation separately. Use jcmd or MXBean to enable. Example: `jcmd PID ElasticHeap young_commit_percent=40 uncommit_ihop=40` > young_commit_percent: percentage of young generation(MaxNewSize) to keep, rest of young generation will be uncommitted > uncommit_ihop: Percentage to trigger G1 concurrent cycle and mixed GC (like InitiatingHeapOccupancyPercent). Unused memory of old generation will be uncommitted after last mixed GC. Use MXBean: ``` MBeanServer server = ManagementFactory.getPlatformMBeanServer(); ElasticHeapMXBean elasticHeapMXBean = ManagementFactory.newPlatformMXBeanProxy(server, "com.alibaba.management:type=ElasticHeap", ElasticHeapMXBean.class); elasticHeapMXBean.setYoungGenCommitPercent(40); elasticHeapMXBean.setUncommitIHOP(40); ``` ### 3. Softmx mode Dynamically to limit the heap as a percentage of origin Xmx. Use jcmd: `jcmd PID ElasticHeap softmx_percent=60` Use MXBean: `elasticHeapMXBean.setSoftmxPercent(70);` ### Other G1ElasticHeap advanced options: > ElasticHeapMinYoungCommitPercent, 10 \ (Mininum percentage of young generation) > ElasticHeapYGCIntervalMinMillis, 5000 \ (Mininum young GC interval) > ElasticHeapInitialMarkIntervalMinMillis, 60000 \ (Mininum initial mark interval) > ElasticHeapEagerMixedGCIntervalMillis, 15000 \ (Guaranteed mixed GC interval, to make sure the mixed will happen in time to uncommit memory after last mixed GC) > ElasticHeapOldGenReservePercent, 5 \ (To keep a mininum percentage of Xmx for old generation in the uncommitment after last mixed GC) > ElasticHeapPeriodicYGCIntervalCeilingPercent, 25 \ ElasticHeapPeriodicYGCIntervalFloorPercent, 25 \ (The actual young GC interval will fluctuate between \ ElasticHeapPeriodicYGCIntervalMillis * (100 - ElasticHeapPeriodicYGCIntervalFloorPercent) / 100 and \ ElasticHeapPeriodicYGCIntervalMillis * (100 + ElasticHeapPeriodicYGCIntervalCeilingPercent) / 100 ) Thanks, Liang ------------------------------------------------------------------ From:MAO, Liang Send Time:2019 Sep. 12 (Thu.) 17:01 To:Thomas Schatzl ; hotspot-gc-dev Subject:Re: G1 patch of elastic Java heap Hi Thomas, Sorry for some late to prepare the code. Now you can see the code in following webrev: http://cr.openjdk.java.net/~luchsh/elasticHeap/ It contains 2 patchs of hotspot and jdk against OpenJDK 8u222-ga. Most of the core source code is in src/share/vm/gc_implementation/g1/elasticHeap.cpp I want to explain some brief ideas of the patch: 1. G1ElasticHeap provides no addtional STW time to existent GC pauses. The memory commit/uncommit(mmap/munmap) will be executed in a concurrent thread because mmap/munmap costs significant time(more than 100ms easily on GB memory). 2. We provides 3 different evaluation modes to do the memory saving. a. Generation limit The young generation and old generation can be limited respectively by jcmd/MXBean. A typical scenario is that we have a lot of applications which have large young generation(default G1 is 60% as well) which is prepared for peak traffic. We don't need the large young generation memory all the time if not in a peak traffic. b. Periodic uncommit The young/old generation will be uncommit respectively based on young/old GC interval. The idea is similar to the original JEP 346 but treats young/old generation separately. For example, an application doesn't need to have large young generation all the time as described above. Periodic uncommit will save young generation memory to keep an acceptable young GC interval. c. Softmx It adjust the capacity of heap in runtime by jcmd/MXBean. 3.We still have some limitations that we need to make sure Xms=Xmx and save the memory in different policy. I made it a single patch so it may not be friendly for reading because of a lot of code and funcionalities. We can discuss a single feature like young generation uncommit or concurrent uncommit first. Please let me know if you want me to explain more details on some specific points. Thanks, Liang ------------------------------------------------------------------ From:Thomas Schatzl Send Time:2019 Jun. 19 (Wed.) 18:04 To:"MAO, Liang" ; hotspot-gc-dev Subject:Re: G1 patch of elastic Java heap Hi Liang, On Wed, 2019-06-19 at 17:38 +0800, ??(??) wrote: > Hi Thomas, > > Thank you for your quick reply. > As we implemented this feature in AlibabaJDK8, do you think if it is > feasible to just provide a CR based on JDK8? > So that you guys can have a look and then we can continue discussing > on the possible JEP and implementations in upstream. > > Otherwise we need spend time on preparing patches based on tip for > further discussion... > > BTW, I'm surely covered by the Alibaba OCA entry. for discussion a jdk8 patch/webrev is fine I guess. I would prefer a webrev because it can be referenced more easily though. Thanks, Thomas From per.liden at oracle.com Thu Sep 26 07:14:33 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 26 Sep 2019 09:14:33 +0200 Subject: RFR: 8231294: ZGC: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted002 fails In-Reply-To: References: Message-ID: <7af66362-44da-a192-5688-ae1fc1987061@oracle.com> (CC:ing hotspot-runtime-dev, as the test belongs to runtime) On 9/25/19 10:57 AM, Per Liden wrote: > The resexhausted002 test uses an 8M heap, which is a bit too small for > ZGC. Bumping it to 128M to allow the test to pass on ZGC too. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231294 > Webrev: http://cr.openjdk.java.net/~pliden/8231294/webrev.0 > > Testing: Manual testing with G1 and ZGC > > /Per From thomas.schatzl at oracle.com Thu Sep 26 08:24:53 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Thu, 26 Sep 2019 10:24:53 +0200 Subject: RFR: 8231153: Improve concurrent refinement statistics In-Reply-To: References: <1DADC595-3106-4CE7-BA5D-7B6C7EE0E81E@oracle.com> <4a851a19-0979-c696-0c80-1165bd755834@oracle.com> Message-ID: Hi Kim, On 26.09.19 03:50, Kim Barrett wrote: >> On Sep 25, 2019, at 5:40 AM, Thomas Schatzl wrote: >> >> Hi Kim, >> >> On 24.09.19 04:58, Kim Barrett wrote: >>> [?]CR: >>> https://bugs.openjdk.java.net/browse/JDK-8231153 >>> Webrev: >>> https://cr.openjdk.java.net/~kbarrett/8231153/open.00/ >>> Testing: >>> mach5 tier1-5 >>> some local by-hand testing to look at the rate tracking. >> >> I think the change in functionality is good. I can't really judge the appropriateness of the input values to the predictors since I do not know their actual use and haven't gotten a feeling about whether there is a problem with that :) > > Thanks for looking at it. > > The current algorithm for controlling refinement thread activation has > problems because its inputs have little to do with what it should be > trying to achieve. These new predictors provide information that I > currently think will be useful to making improvements there, but since > I haven?t fully worked out the details there might be further adjustments > coming along later. > [...]> > I considered changing RemSet::refine_card_concurrently() to return a > bool value indicating whether the card was scanned or not. But I'm not > sure that's really worthwhile, since most of the the reasons for not > scanning are (I think) relatively rare. (Stale cards and failure to > be filtered earlier for timing reasons.) The exception is the HCC. > However, doing that would really just introduce another rate, e.g. the > expected fraction of enqueued cards that need scanning. > > I have seen large variability in that fraction. It's been a while > since I was looking at that, but I think it had to do with the > difference between young-only and mixed GCs. It might be that separate > statistics need to be maintained for those cases. But I also think the > recent changes to the redirtying process may ameliorate that > difference. I'll be looking into that as part of future changes to > make use of this new data. > Looking forward to these changes :) >> - I would also suggest to add a "num_" prefix to numbers/counts of values. > > I guess our tastes differ, because I strongly disliked those num_ prefixes. > They smack of Hungarian Notation and the like (of which I'm not a fan). :) I like the little better distinction particularly in code where there is reason for confusion: e.g. if need to handle both counts and actual data and they are similarly named (like "card" and "cards"), which makes it very hard to parse longer blocks of code where both are used (even worse in the same statement). In this case, with a "size_t* processed_cards" argument (something like that) I can't tell easily by looking at the signature whether this is a single count value that is passed in via a pointer as inout-parameter, or this is an array of card counts or actually cards. There is still the possibility that this is an array of counts then, but from the context that does not make sense. Ymmv :) >> - pre-existing: probably rename G1RemSet::_num_conc_scanned_cards and G1RemSetSummary::_conc_scanned_cards to "_concurrent_scanned_cards" to match the "_concurrent_refined_cards?. > > RemSet::_num_conc_scanned_cards is only used for logging, and has a > poor implementation (racy increments). Given its usage and being in a > somewhat performance critical code path, I'm reluctant to fix the > implementation. I considered just removing it completely, since the > value isn't critical and could be misleading. What do you think? I am fine with that; please make this change also fix https://bugs.openjdk.java.net/browse/JDK-8043505 then :) >> - not sure, but I think exposing size() and start() and in G1FreeIdSet seems unnecessary: the only user is G1DirtyCardQueueSet anyway, and it is already owner of G1FreeIdSet. I.e. it knows these values already (and passes it to the initializer of the G1FreeIdSet instance, and already has a getter for the size() value), so getting it back from G1FreeIdSet seems a bit strange to me, but I am okay with current code. > > The start() function is used in DCQS::mut_process_buffer(). However, > the start offset now appears to be a premature generalization. In the > RFR for JDK-8216258 I said This is still in the DCQS class. > > Also generalized to allow the id set to start with a non-zero id, > because that looks like it might simplify other changes I'm working on. > > Nothing seems to have happened with that, and I no longer even recall what > changes I was thinking about there. Maybe I should back out the new start() > and size() and eliminate the support for a start offset entirely. > > I could do that as part of this change, or save it for another RFE. > As you prefer. It does not seem an extremely big change, so either is fine with me. Thanks, Thomas From shade at redhat.com Thu Sep 26 11:30:56 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 26 Sep 2019 13:30:56 +0200 Subject: RFR: 8231294: ZGC: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted002 fails In-Reply-To: <7af66362-44da-a192-5688-ae1fc1987061@oracle.com> References: <7af66362-44da-a192-5688-ae1fc1987061@oracle.com> Message-ID: <0a3bfd28-2583-1aef-4c9a-f64b35f8fbcb@redhat.com> On 9/26/19 9:14 AM, Per Liden wrote: > (CC:ing hotspot-runtime-dev, as the test belongs to runtime) > > On 9/25/19 10:57 AM, Per Liden wrote: >> The resexhausted002 test uses an 8M heap, which is a bit too small for ZGC. Bumping it to 128M to >> allow the test to pass on ZGC too. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231294 >> Webrev: http://cr.openjdk.java.net/~pliden/8231294/webrev.0 This looks good to me. I tested it locally with Serial, Parallel, CMS, G1, Shenandoah, ZGC -- and all work fine. (Epsilon does not work, but that is expected). -- Thanks, -Aleksey From david.holmes at oracle.com Thu Sep 26 11:37:29 2019 From: david.holmes at oracle.com (David Holmes) Date: Thu, 26 Sep 2019 21:37:29 +1000 Subject: RFR: 8231294: ZGC: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted002 fails In-Reply-To: <7af66362-44da-a192-5688-ae1fc1987061@oracle.com> References: <7af66362-44da-a192-5688-ae1fc1987061@oracle.com> Message-ID: <6af1228b-993e-2f2f-40e0-b144606307ba@oracle.com> Hi Per, On 26/09/2019 5:14 pm, Per Liden wrote: > (CC:ing hotspot-runtime-dev, as the test belongs to runtime) > > On 9/25/19 10:57 AM, Per Liden wrote: >> The resexhausted002 test uses an 8M heap, which is a bit too small for >> ZGC. Bumping it to 128M to allow the test to pass on ZGC too. It's unfortunate that we need to do make it so big, but as long as it still functions okay and in a short time then it should be okay. If we see a problem we can split into two tests runs: one for ZGC and one for not ZGC. Thanks, David >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231294 >> Webrev: http://cr.openjdk.java.net/~pliden/8231294/webrev.0 >> >> Testing: Manual testing with G1 and ZGC >> >> /Per From per.liden at oracle.com Thu Sep 26 11:49:15 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 26 Sep 2019 13:49:15 +0200 Subject: RFR: 8231294: ZGC: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted002 fails In-Reply-To: <0a3bfd28-2583-1aef-4c9a-f64b35f8fbcb@redhat.com> References: <7af66362-44da-a192-5688-ae1fc1987061@oracle.com> <0a3bfd28-2583-1aef-4c9a-f64b35f8fbcb@redhat.com> Message-ID: <39d302e9-4b3e-2ac1-2da5-07ec45ca9ee3@oracle.com> Thanks for reviewing, Aleksey! /Per On 9/26/19 1:30 PM, Aleksey Shipilev wrote: > On 9/26/19 9:14 AM, Per Liden wrote: >> (CC:ing hotspot-runtime-dev, as the test belongs to runtime) >> >> On 9/25/19 10:57 AM, Per Liden wrote: >>> The resexhausted002 test uses an 8M heap, which is a bit too small for ZGC. Bumping it to 128M to >>> allow the test to pass on ZGC too. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231294 >>> Webrev: http://cr.openjdk.java.net/~pliden/8231294/webrev.0 > > This looks good to me. > > I tested it locally with Serial, Parallel, CMS, G1, Shenandoah, ZGC -- and all work fine. (Epsilon > does not work, but that is expected). > From per.liden at oracle.com Thu Sep 26 11:49:30 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 26 Sep 2019 13:49:30 +0200 Subject: RFR: 8231294: ZGC: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted002 fails In-Reply-To: <6af1228b-993e-2f2f-40e0-b144606307ba@oracle.com> References: <7af66362-44da-a192-5688-ae1fc1987061@oracle.com> <6af1228b-993e-2f2f-40e0-b144606307ba@oracle.com> Message-ID: Thanks for reviewing, David! /Per On 9/26/19 1:37 PM, David Holmes wrote: > Hi Per, > > On 26/09/2019 5:14 pm, Per Liden wrote: >> (CC:ing hotspot-runtime-dev, as the test belongs to runtime) >> >> On 9/25/19 10:57 AM, Per Liden wrote: >>> The resexhausted002 test uses an 8M heap, which is a bit too small >>> for ZGC. Bumping it to 128M to allow the test to pass on ZGC too. > > It's unfortunate that we need to do make it so big, but as long as it > still functions okay and in a short time then it should be okay. If we > see a problem we can split into two tests runs: one for ZGC and one for > not ZGC. > > Thanks, > David > >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231294 >>> Webrev: http://cr.openjdk.java.net/~pliden/8231294/webrev.0 >>> >>> Testing: Manual testing with G1 and ZGC >>> >>> /Per From daniel.daugherty at oracle.com Thu Sep 26 14:28:28 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 26 Sep 2019 10:28:28 -0400 Subject: RFR: 8231294: ZGC: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted002 fails In-Reply-To: <7af66362-44da-a192-5688-ae1fc1987061@oracle.com> References: <7af66362-44da-a192-5688-ae1fc1987061@oracle.com> Message-ID: <6c76ecbe-75dc-2b50-64ab-5a75167c49cd@oracle.com> Just for the record... JVM/TI belongs to the Serviceability Team so I've added serviceability-dev at ... However, the Runtime team and Serviceability team quite often tag team on JVM/TI issues and lurk on each other's aliases... Dan On 9/26/19 3:14 AM, Per Liden wrote: > (CC:ing hotspot-runtime-dev, as the test belongs to runtime) > > On 9/25/19 10:57 AM, Per Liden wrote: >> The resexhausted002 test uses an 8M heap, which is a bit too small >> for ZGC. Bumping it to 128M to allow the test to pass on ZGC too. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231294 >> Webrev: http://cr.openjdk.java.net/~pliden/8231294/webrev.0 >> >> Testing: Manual testing with G1 and ZGC >> >> /Per From serguei.spitsyn at oracle.com Thu Sep 26 18:06:38 2019 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 26 Sep 2019 11:06:38 -0700 Subject: RFR: 8231294: ZGC: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted002 fails In-Reply-To: <6c76ecbe-75dc-2b50-64ab-5a75167c49cd@oracle.com> References: <7af66362-44da-a192-5688-ae1fc1987061@oracle.com> <6c76ecbe-75dc-2b50-64ab-5a75167c49cd@oracle.com> Message-ID: <1452ee17-815c-396b-c121-c709c631836e@oracle.com> Dan, thank you for adding the serviceability-dev mailing list. It looks good but has been already pushed. :) Thanks, Serguei On 9/26/19 07:28, Daniel D. Daugherty wrote: > Just for the record... JVM/TI belongs to the Serviceability Team > so I've added serviceability-dev at ... > > However, the Runtime team and Serviceability team quite often tag > team on JVM/TI issues and lurk on each other's aliases... > > Dan > > > On 9/26/19 3:14 AM, Per Liden wrote: >> (CC:ing hotspot-runtime-dev, as the test belongs to runtime) >> >> On 9/25/19 10:57 AM, Per Liden wrote: >>> The resexhausted002 test uses an 8M heap, which is a bit too small >>> for ZGC. Bumping it to 128M to allow the test to pass on ZGC too. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231294 >>> Webrev: http://cr.openjdk.java.net/~pliden/8231294/webrev.0 >>> >>> Testing: Manual testing with G1 and ZGC >>> >>> /Per > From sangheon.kim at oracle.com Thu Sep 26 18:14:37 2019 From: sangheon.kim at oracle.com (sangheon.kim at oracle.com) Date: Thu, 26 Sep 2019 11:14:37 -0700 Subject: RFR(XL): 8220310: Implementation: NUMA-Aware Memory Allocation for G1, Mutator (1/3) In-Reply-To: References: <06ACBF87-ADBE-499F-B668-0274E4925B26@oracle.com> <2b37edd6-3e0f-013d-1616-9d003f8ac1ed@oracle.com> <6ad56365-4507-8909-c13d-8ed15ab7cba9@oracle.com> Message-ID: <4f949763-9533-40d9-6569-3dff93b19920@oracle.com> Hi Stefan, On 9/24/19 2:02 AM, Stefan Johansson wrote: > Hi Sangheon, > > On 2019-09-24 00:33, sangheon.kim at oracle.com wrote: >> Hi Stefan, >> >> Many thanks for reviewing and discussing this! >> And please allow me to post next webrev with more comments. > > Yes, no rush. :) > >> >> On 9/23/19 8:14 AM, Stefan Johansson wrote: >>> Hi Sangheon, >>> >>> Thanks for the updated webrev, I like where we are heading with >>> this. I still have some comments but most of them are fairly small. >>> The only big thing is that I still think we could simplify things a >>> lot by not passing down node_index to the lower level commit >>> functions and instead just focus on keeping a strict manual >>> interleaving of regions/pages (whichever is largest). I know this >>> wouldn't allow us to expand by a single region on a specific node, >>> but since this is a fairly uncommon case I think we could live with >>> that. >>> >>> src/hotspot/share/gc/g1/g1Allocator.* >>> ------------------------------------- >>> There are a few calls like this: >>> uint node_index = _g1h->mem_node_mgr()->index_of_current_thread(); >>> I would like a private helper to wrap this, something like >>> current_node_index(). >> Will do. >> >>> --- >>> >>> hotspot/share/gc/g1/g1Arguments.cpp >>> ----------------------------------- >>> ?161?? if (UseNUMA) { >>> ?162???? if (FLAG_IS_DEFAULT(AlwaysPreTouch)) { >>> ?163?????? FLAG_SET_DEFAULT(AlwaysPreTouch, true); >>> ?164???? } >>> ?165???? if (!AlwaysPreTouch && FLAG_IS_CMDLINE(AlwaysPreTouch)) { >>> ?166?????? warning("Disabling AlwaysPreTouch is incompatible with >>> UseNUMA. Disabling UseNUMA."); >>> ?167?????? FLAG_SET_ERGO(UseNUMA, false); >>> ?168???? } >>> ?169?? } >>> >>> I'm not sure I understand why we have to enforce AlwaysPreTouch when >>> NUMA is enabled. This will improve performance, but I think the NUMA >>> support should work even without it. Or am I missing something? >> When we allocate a new region, we have to know the node index of a >> region came from free list. >> However, without pretouch, we can't know whether the region is >> properly located or not. i.e. when checking node id via >> os::numa_get_address_id(), we will get invalid id even though we >> called numa_make_local() because the page (of the bottom of heap >> region) is not yet faulted. >> >> We may set HeapRegion::_node_index with preferred index after calling >> numa_make_local() ( without actually checking the node id). But not >> sure whether it is a way to go. >> If you are suggesting this, maybe we need more discussions and tests. >> So unless others have strong opinion on this, I would like to address >> in a separate CR. >> > I think we should go with trusting the OS and set the preferred index > on the HeapRegion and then we can have our verification/check on what > actual node we ended up on during a safepoint. But we can discuss this > more. I'm not sure how much more testing this requires, in the end we > cannot know what the OS will do underneath. Done. This is the only major change at webrev.2. But as I said, unfortunately I had to introduce a new structure to save 'numa id' when we commit. The problem is that when we commit, HeapRegions are not yet initialized. So the newly introduced structure stores numa id of given region and then those are used to set HeapRegion::_node_index later. > >>> --- >>> >>> src/hotspot/share/gc/g1/g1CollectedHeap.* >>> ----------------------------------------- >>> As we discussed offline I'm not sure we need pass the node_index >>> down in the call to expand. Since we're not strict in giving out >>> memory from the correct node during the mutator phase, I think we >>> could do the same during the GC in the case we need to expand. >> Yeah, the only difference would be: unlike the mutator case, we even >> don't try to get the identical node and expect to get random one. I >> do understand this case would be rare, but I think doing our best to >> match is better even though we have to sacrifice a little bit of code >> complexity. >> > I see you point, but I don't fully agree. If the case is rare, or we > can't show that handing out a region with the correct index is giving > a benefit I think we should leave this as a separate enhancement later > on. Doing so we might even be able to design it in a smarter way. As we dicussed internally, expanding with node index is okay for the scope of NUMA JEP so left as is. > >>> --- >>> >>> src/hotspot/share/gc/g1/g1RegionToSpaceMapper.cpp >>> ------------------------------------------------- >>> ?162???? G1NodeDistributor itr(node_index); >>> ?163 >>> ?164???? for (uint i = start_idx; i < start_idx + num_regions; i++) { >>> ?165?????? // If there are many pages to touch, different node ids >>> will be used. >>> ?166?????? uint processed_node_index = itr.next_node_index(); >>> ?... >>> ?179???????? zero_filled = _storage.commit(idx, 1, >>> processed_node_index); >>> ?180???????? itr.next(); >>> ?181?????? } >>> >>> Do we really need to do this? Can't we just rely on the code in >>> G1NUMA::touch_memory_roundrobin() to do the right thing? Or is this >>> just to allow committing a single region on the correct17134 node? >> Yes, this is to commit a single region and touch_memory_roundrobin() >> is not called. >> i.e. HeapRegion is smaller than page size, so we commit one page at >> once and skip for next region until the page is filled with region. >> e.g. 2MB page size, 1MB heap region. every 2 commit for heap regions, >> 1 page will be committed. >> However, your comment reminded me to enhance this area. Instead of >> starting from the first node, we can just call >> G1NUMA::next_numa_index() at G1NodeDistributor::next(). >> > Ok, I see, as long as we need to be able to commit to a specific node > this is needed. > >>> --- >>> >>> src/hotspot/share/gc/g1/g1_globals.hpp >>> -------------------------------------- >>> Instead of adding those flags I think we should use log-tags and >>> guard the logic by using log_is_enabled(), or LogTarget::is_enable() >>> as you do in heapRegionManager.cpp. To allow these to be excluded >>> with other logging turned on I suggest using specific log-tag-sets >>> for each of them something like: >>> G1PrintNUMAIdOfHeapRegions -> gc, numa, region (debug or trace) >>> G1VerifyNUMAIdOfHeapRegions -> gc, numa, region, verification (debug >>> or trace) >> Will do as you suggested. >> Kim also suggested same one but I wanted to hear opinions. :) >> >> G1PrintNUMAIdOfHeapRegions -> gc, numa, region (debug) >> G1VerifyNUMAIdOfHeapRegions -> gc, numa, region, verification (debug) Changed to belong to 'trace' level for both as during my test, it is too annoying to see at debug level. Will post webrev.2 after addressing Kim's comment. Thanks, Sangheon >> >>> --- >>> >>> src/hotspot/share/gc/g1/heapRegionManager.cpp >>> --------------------------------------------- >>> ?112?? if (mgr->num_active_nodes() > 1) { >>> ?113???? uint valid_node_index = >>> mgr->valid_node_index(requested_node_index); >>> ?114???? // Try to allocate with requested node index. >>> ?115???? hr = _free_list.remove_region_with_node_index(from_head, >>> valid_node_index, NULL); >>> ?116?? } >>> >>> I've seen some problems in my testing with this code, >>> valid_node_index() now call _numa->next_numa_index() when the index >>> isn't valid. This means that both the lower level commit code will >>> call this function as well as the allocation path in some cases (for >>> example single region humongous). I'm not sure if this will lead to >>> any real problems but it makes it harder to understand what is going >>> on underneath and I've seen it lead to having imbalance over the >>> nodes. To avoid this I think we could instead do: >>> >>> if (mgr->num_active_nodes() > 1 && >>> ??? mgr->is_valid_node_index(requested_node_index) { >>> ? hr = _free_list.remove_region_with_node_index(from_head, >>> requested_node_index, NULL); >>> } >>> >>> This way we only remove with index if we have a valid index, >>> otherwise we get a HeapRegion from the code below, which should be >>> more or less the same as taking whatever index and removing with that: >>> ?118?? if (hr == NULL) { >>> ?... >>> ?121???? hr = _free_list.remove_region(from_head); >>> ?122?? } >>> >>> If this is an ok approach I think valid_node_index() is not longer >>> used and can be removed. >> Looks nice suggestion. >> So instead of relying on G1NUMA::next_numa_index(), just using the >> first region would relief the node imbalance. I like this but let me >> back with some testing results. >> >>> --- >>> >>> src/hotspot/share/gc/g1/g1NUMA.inline.hpp >>> ----------------------------------------- >>> G1NUMA::next_numa_id() seems to be unused and could then be removed. >> Will remove. >> >>> --- >>> >>> I'll continue looking at the patch and taking it for some spins. I >>> might come with additional feedback later on, but these are my >>> initial findings. >> Okay! >> >> Thanks, >> Sangheon >> >> >>> >>> Thanks, >>> Stefan >>> >>> >>> On 2019-09-21 07:19, sangheon.kim at oracle.com wrote: >>>> Hi Kim, >>>> >>>> Many thanks for this thorough review! >>>> >>>> On 9/16/19 4:12 PM, Kim Barrett wrote: >>>>>> On Sep 4, 2019, at 3:16 AM, sangheon.kim at oracle.com wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> Please review this patch, making G1 NUMA aware. >>>>>> This is the first part of G1 NUMA implementation. >>>>>> >>>>>> - At G1 CollectedHeap initialization time, the given heap regions >>>>>> will be split and touched. >>>>>> - Internally node index(uint) is used instead of node id(int >>>>>> type) for easier access to arrays. >>>>>> - Only Linux is supported. >>>>>> - New logs are under gc+heap+numa >>>>>> - Current memory touch implementation will be replaced with >>>>>> WorkGang by JDK-8230411: NUMA aware work gang. >>>>>> >>>>>> CR: https://bugs.openjdk.java.net/browse/JDK-8220310 >>>>>> Webrev: http://cr.openjdk.java.net/~sangheki/8220310/webrev.0 >>>>>> Testing: hs-tier 1 ~ 5 with +- UseNUMA. >>>>>> >>>>>> Thanks, >>>>>> Sangheon >>>> I had long off-line discussion with Kim, Thomas and Stefan J. >>>> And 2 major changes are: >>>> >>>> 1) Memory touch thread is removed. This was a general way to >>>> implement for both Linux and Solaris. However, Solaris is not >>>> supported anymore, so memory touch thread is just removed. >>>> And this made the code much simpler. >>>> >>>> 2) G1NUMA doesn't manage numa id of each page and we don't care >>>> about each page instead HeapRegion is only cared. i.e. we assume >>>> numa id of all pages between bottom and end of HeapRegion to be >>>> same. This also made related codes much simpler. >>>> >>>> There are some other changes as well. >>>> >>>> >>>>> Here is an initial set of comments. Some of these might be >>>>> nonsense; I'm >>>>> not all that familiar with NUMA support. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> Just in general, it's kind of disappointing that NUMA support for G1 >>>>> apparently involves this much code, and is completely different from >>>>> ParallelGC NUMA support, and both are completely different from ZGC >>>>> NUMA support.? (And Shenandoah is probably yet another still.) >>>>> Oh well... >>>> Yes, I agree on that. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> According to off-line discussion, the intent is that a HeapRegion >>>>> only >>>>> contains memory from one NUMA node.? That isn't obvious though, >>>>> and there >>>>> are API and data structure choices that obscure that.? It seems >>>>> like some >>>>> things could be simplified with that constraint in mind. I think >>>>> some of >>>>> that is a result of trying to leave open support for NUMA aware >>>>> off-heap >>>>> data structures, but that isn't part of this change, and might be >>>>> better >>>>> kept separate from NUMA aware Java heap support. >>>>> >>>>> I'll try to point out places where I think there are opportunities >>>>> for >>>>> simplification. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> According to off-line discussion, the touching mechanism might not be >>>>> necessary.? If that's true, that would be a significant >>>>> simplification.? For >>>>> now I won't comment on the code for the touch threads, though I think >>>>> I found some problems there before the discussion suggesting we might >>>>> not need them happened. >>>> Thanks. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1NUMA.hpp >>>>> ?? 64 // Manages NUMA node related information and memory touch >>>>> threads. >>>>> ... >>>>> >>>>> I found the comment describing G1NUMA pretty hard to parse. Even >>>>> after looking at the code I'm having trouble figuring out what's >>>>> being >>>>> said. >>>>> >>>>> This part I was able to figure out, but think it could be worded >>>>> better. >>>>> ?? 70 // Also provides a way to convert NUMA id and NUMA >>>>> index(starting from zero and sequatial). >>>>> ?? 71 // Actual NUMA ids would not be sequential nor not start >>>>> from zero, so a conversion is necessary >>>>> ?? 72 // between id and index. >>>>> >>>>> Something like >>>>> >>>>> ?? Also provides a mapping between NUMA indices (sequential, starting >>>>> ?? from zero) and NUMA ids (which may not start at zero and may >>>>> not be >>>>> ?? sequential).? Indices are generally used by clients, and mapped to >>>>> ?? ids when dealing with the OS/hardware layer. >>>> DONE >>>> >>>>> >>>>> (I *think* that's true.) >>>>> >>>>> But then, why expose node_ids at all in G1NUMA? It seems like >>>>> node_ids >>>>> could be completely relegated to an implementation detail that >>>>> clients >>>>> never need to deal with. >>>> ... >>>>> As evidence for not needing to expose ids to clients, there are no >>>>> callers of numa_id_of_index in any of the series of 3 patches.? And >>>>> G1NUMA::numa_ids() and G1MemoryMultNodeManager::node_ids() are only >>>>> used in a couple of places for printing/logging and by >>>>> WB_G1MemoryNodeIds.? I'm not sure what the last is for, but the >>>>> printing and logging code could be isolated into G1NUMA. >>>> Basically I totally agree with you for not exposing numa id. >>>> Current patches are only using node id for logging purpose at some >>>> locations because users are only interested on node id. >>>> >>>> Also Thread::lgrp_id() (renaming lgrp id, numa id etc.. is >>>> different topic that we had before) is also public, so my feeling >>>> is current try is enough. :) >>>> >>>> WB_G1MemoryNodeIds() is for jtreg testing purpose which checks >>>> whether the given memory has node id as we expected. >>>> >>>>> >>>>> Also, it looks like the index to node_id mapping provided here is >>>>> duplicating the information provided by os::Linux::nindex_to_node. >>>>> Not that this necessarily helps with anything in generic code... >>>> Yes, I'm aware of _nindex_to_node but I think maintaining such >>>> conversion at G1NUMA is okay. Other NUMA implementation such as >>>> parallel gc relies on node id so completely removing node id at >>>> general code seems different scope. >>>> Unless you have strong opinion on addressing it in this patch, I >>>> would like proceed as is. >>>> If it is the case, we can consider your suggestion as a separate >>>> enhancement. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>>>> ?? 57?? virtual int* node_ids() const { static int dummy_id = 0; >>>>> return &dummy_id; } >>>>> >>>>> Can the return type here be "const int*" ?? That would be better >>>>> if possible. >>>>> >>>>> [This is probably moot if clients only deal with numa indexes and not >>>>> node_ids, as suggested earlier.] >>>> DONE >>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>>>> ?? 63?? virtual uint guarantee_valid_node_index(uint node_index) >>>>> const { return 0; } >>>>> >>>>> Given naming conventions elsewhere, the name of this function >>>>> makes me >>>>> expect it to be a verification function. >>>> Agree. >>>> valid_node_index() is better? webrev.1 changed name. >>>> Or are you suggesting to remove this and then the caller do such >>>> thing? e.g. >>>> void A(uint node_index) { >>>> uint valid_node_index = node_index; >>>> if (!is_valid_node_index(valid_node_index)) { >>>> ?? valid_node_index = random_numa_index(); >>>> ... >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1NUMA.inline.hpp >>>>> ?? 29 #include "gc/shared/taskqueue.inline.hpp" >>>>> >>>>> This #include seems to be just to obtain randomParkAndMiller. It >>>>> would be >>>>> better to refactor that function into its own file(s). Or make it >>>>> part of >>>>> some component providing various random number generators. >>>> Not need because of below change. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1NUMA.inline.hpp >>>>> ?? 45?? int seed = randomParkAndMiller(_seed); >>>>> ?? 46?? Atomic::store(seed, &_seed); >>>>> >>>>> This sequence may lose updates.? I don't know if that's important. >>>>> And why isn't this just using os::random(), or maybe refactoring that >>>>> slightly to gain access to the external seed version.? I don't >>>>> want this >>>>> change to be made: >>>>> http://cr.openjdk.java.net/~sangheki/8220310/webrev.0/src/hotspot/share/gc/shared/taskqueue.inline.hpp.udiff.html >>>>> >>>> Thomas and Stefan J. also complained on this. >>>> Changed to get next numa index with round-robin manner. i.e. >>>> doesn't use random() at all. >>>> >>>>> And do we *really* need randomness here at all?? Is there a reason >>>>> not >>>>> to simply cycle through the nodes? >>>> Not really. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/os/linux/os_linux.cpp >>>>> 3016?? size_t num_pages_left = size_in_bytes / page_size; >>>>> >>>>> Is size_in_bytes required to be a multiple of page_size? I believe >>>>> it is, >>>>> and an assert of that would be useful. >>>> This code is removed. And yes to answer to your question. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/os/linux/os_linux.cpp >>>>> 3016?? size_t num_pages_left = size_in_bytes / page_size; >>>>> 3017?? const size_t num_pages_total = num_pages_left; >>>>> 3018?? size_t num_buffer_entries = MIN2(num_pages_left, max_pages); >>>>> >>>>> I think clearer would be >>>>> >>>>> 3017?? const size_t num_pages_total = size_in_bytes / page_size; >>>>> 3018?? size_t num_buffer_entries = MIN2(num_pages_total, max_pages); >>>>> >>>>> and move this down to the loop setup: >>>>> 3016?? size_t num_pages_left = num_pages_total; >>>> This code is removed. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/os/linux/os_linux.cpp >>>>> 3020?? void** pages = (void**)alloca(sizeof(void*) * >>>>> num_buffer_entries); >>>>> 3021?? int* status = (int*)alloca(sizeof(int) * num_buffer_entries); >>>>> >>>>> I think a ResourceMark and NEW_RESOURCE_ARRAYs would be preferable to >>>>> using alloca() here.? alloca() has some dangers and I think is >>>>> currently >>>>> only used in some very specialized places where it really is needed. >>>> This code is removed. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/os/linux/os_linux.cpp >>>>> 3061 } >>>>> 3062 // Moves current thread on a specific node and it will not >>>>> migrate to >>>>> >>>>> Missing blank line between functions. >>>> This code is removed. >>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/os/linux/os_linux.cpp >>>>> 3031?????? pages[i] = (void*)base; >>>>> >>>>> Unnecessary cast. >>>> This code is removed. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/os/linux/os_linux.cpp >>>>> 3072???? case AFFINITY_NONE: >>>>> 3073?????? // Do nothing. >>>>> 3074?????? return true; >>>>> 3075?????? break; >>>>> >>>>> Shouldn't this be calling numa_run_on_node with -1 node id? The >>>>> documentation for numa_run_on_node() says "Passing -1 permits the >>>>> kernel to schedule on all nodes again." >>>> My understand of that description is let kernel decide the numa >>>> node which is not actually want here. We want to move current >>>> thread to the preferred numa node. >>>> >>>> Currently we don't need this API (numa_set_affinity() ) so removed >>>> on webrev.1. >>>> However following patch which is not scope of this JEP will use it. >>>> I will add your suggestion below? when we need this. >>>> >>>>> >>>>> But see also next comment about numa_affinity_set. >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/runtime/os.hpp >>>>> ? 401?? // Available affinities. >>>>> ? 402?? enum NumaAffinity { >>>>> ? 403???? AFFINITY_STRONG, >>>>> ? 404???? AFFINITY_NONE >>>>> ? 405?? }; >>>>> ? 406?? static bool numa_affinity_set(Thread* thread, int numa_id, >>>>> NumaAffinity affinity); >>>>> >>>>> Could there ever be other affinities?? This API doesn't seem right. >>>>> It seems to me it should be something like >>>>> >>>>> ?? static bool numa_set_affinity(Thread* thread, int numa_id); >>>>> ?? static bool numa_clear_affinity(Thread* thread); >>>>> >>>>> No need for NumaAffinity enum. >>>> The enum is also kind of remainder after removing Solaris >>>> implementation which has more types. >>>> But I agree with you as there are only 2 types now, it is simpler >>>> to separate API without enum. >>>> >>>>> >>>>> (I think numa_set/clear_affinity is better than >>>>> numa_affinity_set/clear; >>>>> numa is a "namespace" and we usually set/clear/get_something rather >>>>> than something_set/clear/get.) >>>>> >>>>> Or maybe this should just be >>>>> >>>>> ?? static bool numa_set_affinity(Thread* thread, int numa_id); >>>>> >>>>> with "clearing" provided by passing AnyId as the numa_id? Still don't >>>>> need the NumaAffinity enum for this. >>>> Agree on this naming stuff. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1NUMA.cpp >>>>> ? 124?? guarantee(rs.size() % page_size == 0, "Reserved space size >>>>> (" SIZE_FORMAT ") should be aligned " >>>>> ? 141?? guarantee(size_in_bytes % page_size == 0, "The given range >>>>> (" SIZE_FORMAT ") should be aligned " >>>>> ? 209???? guarantee((uintptr_t)G1HeapRegionSize % page_size == 0, >>>>> "G1HeapRegionSize (" SIZE_FORMAT ") and page size (" >>>>> ? 234???? guarantee((uintptr_t)_page_size % _heap_region_size == >>>>> 0, "G1HeapRegionSize (" SIZE_FORMAT ") and page size (" >>>>> >>>>> Use is_aligned.? I think casts shouldn't be needed. >>>> These lines are removed after major refactoring. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1NUMA.cpp >>>>> ?? 43?? _numa_id_to_index_map = NEW_C_HEAP_ARRAY_RETURN_NULL(uint, >>>>> _len_numa_id_to_index_map, mtGC); >>>>> ?? 44?? memset(_numa_id_to_index_map, 0, sizeof(uint) * >>>>> _len_numa_id_to_index_map); >>>>> >>>>> Using NEW_xxx_RETURN_NULL and then ignoring the possibility of a NULL >>>>> result.? It would be better to use the non-_RETURN_NULL variant and >>>>> get a decent error message rather than the segfault from >>>>> attemptint to >>>>> write to NULL. >>>> Changed to use non-_RETURN_NULL version. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1NUMA.cpp >>>>> ?? 34 void G1NUMA::init_numa_id_to_index_map(int* numa_ids, uint >>>>> num_numa_ids) { >>>>> >>>>> int* numa_ids => const int* numa_ids. >>>> DONE >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1NUMA.cpp >>>>> ?? 34 void G1NUMA::init_numa_id_to_index_map(int* numa_ids, uint >>>>> num_numa_ids) { >>>>> ... >>>>> ?? 44?? memset(_numa_id_to_index_map, 0, sizeof(uint) * >>>>> _len_numa_id_to_index_map); >>>>> Rather than this memset and then writing to all of them later, >>>>> instead >>>>> initialize all entries to InvalidNodeIndex and then assign from >>>>> numa_ids. >>>> DONE >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1MemoryTouchThread.cpp >>>>> ? 261?? _wakeup_monitor = new Monitor(Mutex::nonleaf, "Wakeup >>>>> Monitor for Touch Control", true, Monitor::_safepoint_check_never); >>>>> ? 262?? _sync_monitor = new Monitor(Mutex::nonleaf, "Sync Monitor >>>>> for Touch Control", true, Monitor::_safepoint_check_never); >>>>> >>>>> For singleton mutex/monitor, we generally prefer to put them in >>>>> mutexLocker. >>>>> Probably passed as constructor arguments to G1NUMAThreadsControl.? >>>>> And >>>>> I know there are counter-examples. >>>> Not necessary as memory touch thread is gone. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>>>> ?? 30 extern const uint AnyNodeIndex; >>>>> ?? 31 extern const uint InvalidNodeIndex; >>>>> >>>>> Why are these at namespace scope.? And they seem misplaced here. >>>>> Seems like they should belong to the G1NUMA class, which is >>>>> responsible for indexing nodes and mapping between indexes and ids. >>>> Changed to G1MemoryNodeManager::AnyNodeIndex. >>>> And I wanted to hide or minimize exposing G1NUMA in general. >>>> Memory node manager also has index related APIs, so let manager >>>> have such constants seems okay. >>>> >>>>> >>>>> ?? 30 const uint AnyNodeIndex = (uint)os::AnyId; >>>>> ?? 31 const uint InvalidNodeIndex = (uint)os::InvalidId; >>>>> And why these values?? That seems like a units mismatch. Just use >>>>> UINT_MAX and UINT_MAX - 1. >>>> I don't have strong opinion on this. >>>> But the intend is to make same value after casting for same meaning >>>> constants instead of randomly chosen ones. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1NUMA.hpp >>>>> ?? 80?? // For invalid numa id, return InvalidNodeIndex. So the >>>>> caller need exception handling. >>>>> >>>>> "need exception handling"? >>>> Thomas also commented on it so reworded. But finally the class is >>>> removed. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/heapRegionManager.cpp >>>>> ? 114???? uint allocated_node_index; >>>>> >>>>> This variable gets assigned via an out parameter here: >>>>> ? 116???? hr = _free_list.remove_region_with_node_index(from_head, >>>>> valid_node_index, &allocated_node_index); >>>>> >>>>> and by assignment here: >>>>> ? 122???????? allocated_node_index = >>>>> mgr->node_index_of_address(hr->bottom()); >>>>> >>>>> but is never used. If the lack of use is not itself a mistake, >>>>> then the out >>>>> parameter of remove_region_with_node_index appears to be unnecessary. >>>> The variable will be used at patch3 (JDK-8220312) but slightly >>>> changed to allow NULL at allocated_node_index parameter at >>>> remove_region_with_node_index(). >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/heapRegionSet.inline.hpp >>>>> ? 151 inline HeapRegion* >>>>> FreeRegionList::remove_region_with_node_index(bool from_head, >>>>> >>>>> As written, I think this might always return NULL if >>>>> requested_node_index is >>>>> not a valid index. I think callers ensure that it's valid, but an >>>>> assert of >>>>> the argument would be help. >>>> Done >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/heapRegionSet.inline.hpp >>>>> ? 175???? if (cur == NULL || cur_depth >= max_search_depth) { // >>>>> not found >>>>> >>>>> There's an off-by-one issue here. The max_search_depth time >>>>> through the >>>>> while-control-expression may have found a matching region, checked >>>>> via the >>>>> numa_index_of_address lookup. >>>>> >>>>> Either the test should be "current_numa_index != >>>>> requested_numa_index" or >>>>> the loop should be changed. >>>> This code is removed. >>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/heapRegionSet.inline.hpp >>>>> ? 151 inline HeapRegion* >>>>> FreeRegionList::remove_region_with_node_index(bool from_head, >>>>> >>>>> I think the list manipulation could be simplified quite a bit.? I >>>>> think we >>>>> don't need the "last" variable.? And the splicing out of cur can >>>>> be unified >>>>> between the from_head and !from_head cases, e.g. something like this: >>>>> >>>>> ?? // Find the region to use, searching from _head or _tail as >>>>> requested. >>>>> ?? if (from_head) { >>>>> ???? for (cur = _head; >>>>> ????????? cur != NULL && cur_depth < max_search_depth; >>>>> ????????? cur = cur->next(), ++cur_depth) { >>>>> ?????? if (current_numa_index == >>>>> numa->numa_index_of_address(cur->bottom)) { >>>>> ???????? break; >>>>> ?????? } >>>>> ???? } >>>>> ?? } else { >>>>> ???? for (cur = _tail; >>>>> ????????? cur != NULL && cur_depth < max_search_depth; >>>>> ????????? cur = cur->prev(), ++cur_depth) { >>>>> ?????? if (current_numa_index == >>>>> numa->numa_index_of_address(cur->bottom)) { >>>>> ???????? break; >>>>> ?????? } >>>>> ???? } >>>>> ?? } >>>>> ?? if (cur == NULL || cur_depth >= max_search_depth) { >>>>> ???? return NULL;????????????????????? // Didn't find a region to use >>>>> ?? } >>>>> ?? // Splice the region out of the list. >>>>> ?? HeapRegion* prev = cur->prev(); >>>>> ?? HeapRegion* next = cur->next(); >>>>> ?? if (prev == NULL) { >>>>> ???? _head = next; >>>>> ?? } else { >>>>> ???? prev->set_next(next); >>>>> ?? } >>>>> ?? if (next == NULL) { >>>>> ???? _tail = prev; >>>>> ?? } else { >>>>> ???? next->set_prev(prev); >>>>> ?? } >>>>> ?? cur->set_prev(NULL); >>>>> ?? cur->set_next(NULL); >>>>> ?? ... >>>> Nice suggestion! >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1NUMA.cpp >>>>> ? 166 class DumpIterator : public StackObj { >>>>> >>>>> Polluting global namespace.? Suggest making DumpIterator and the >>>>> classes >>>>> derived from it (G1HeapRegion{Larger,Smaller}Iterator) private nested >>>>> classes of G1NUMA, making the names unimportant outside of that >>>>> context. >>>> Not need any more. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/heterogeneousHeapRegionManager.hpp >>>>> ? 122?? virtual HeapRegion* allocate_free_region(HeapRegionType >>>>> type); >>>>> ? 123?? virtual HeapRegion* allocate_free_region(HeapRegionType >>>>> type, uint node_index); >>>>> >>>>> The base class (HeapRegionManager) no longer has the one-arg >>>>> function.? So >>>>> this one-arg function is no longer overriding the base class >>>>> definition. >>>>> (Too bad it wasn't declared as C++11 "override".) >>>>> >>>>> I don't see any one-arg callers, so seems like it can just be >>>>> removed. >>>> DONE >>>> Nice finding! :) >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1PageBasedNUMAVirtualSpace.cpp >>>>> >>>>> I dislike the new structure here, with G1PageBasedVirtualSpace now >>>>> being >>>>> both a base class and a concrete class (which is often a problematic >>>>> design).? And the new derived class doesn't really do all that much. >>>>> >>>>> It seems like it would be much simpler to just have >>>>> G1PageBasedVirtualSpace >>>>> check UseNUMA to conditionalize it's commit and uncommit functions. >>>>> Especially since it's commit already needs to have some knowlege >>>>> of NUMA >>>>> because of the new numa_index argument. >>>> DONE >>>> Removed NUMA version and reverted related changes to allow inherit >>>> are reverted. >>>> Now, G1PageBasedVirtualSpace does additional work after checking NUMA. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1PageBasedNUMAVirtualSpace.cpp >>>>> ?? 52 void G1PageBasedNUMAVirtualSpace::uncommit(size_t >>>>> start_page, size_t size_in_pages) { >>>>> >>>>> Do we actually need to do anything here?? We're resetting the >>>>> G1NUMA::_numa_id_of_pages_table (to AnyId), but it's not clear we >>>>> really >>>>> need to do that.? When we later recommit any of those pages, that >>>>> table will >>>>> be updated appropriately. >>>>> >>>>> Maybe rather than having G1NUMA expose quite so much detail it should >>>>> instead have post_commit and a pre_uncommit functions. >>>> I think clearing the node index may help for debugging. Unlike >>>> webrev.0, numa id/index information is located at HeapRegion which >>>> is much simpler. i.e. HeapRegionManager::uncommit_regions() does >>>> resetting node index to InvalidNodeIndex. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1NUMA.hpp >>>>> ?? 75?? G1NUMAPageIdTable _numa_id_of_pages_table; >>>>> >>>>> I was surprised by the need for this, but I can't find anything in >>>>> the NUMA >>>>> API that provides this information.? That seems strange, because >>>>> one would >>>>> think that information must exist somewhere in the NUMA >>>>> implementation. >>>> Removed. >>>> And this made the patch much simpler! >>>> >>>> And here are your comments from other email. >>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>>>> ?? 64?? // Returns node index of the given address. >>>>> ?? 65?? // If the node id of the address is invalid return >>>>> randomly selected node index. >>>>> ?? 66?? virtual uint valid_index_of_address(HeapWord* o) const { >>>>> return 0; } >>>>> >>>>> This function is unused. >>>> Removed. >>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> src/hotspot/share/gc/g1/g1MemoryNodeManager.hpp >>>>> ?? 70?? virtual uint node_index_of_address(HeapWord* o) const { >>>>> return 0; } >>>>> >>>>> The only caller of this function is >>>>> HeapRegionManager::allocate_free_region. >>>>> As noted in an earlier comment, the result of that call is unused. >>>> Removed as you commented but I had to revert as a part of >>>> _numa_id_of_pages_table removal. >>>> Previously we retrieved the page info from _numa_id_of_pages_table, >>>> but new version reads directly via system call. >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>>> As pointed out to me offline, there*are*? ways to query the >>>>> address -> >>>>> associated numa node mapping.? So I'm going back and questioning >>>>> the need >>>>> for _numa_id_of_pages_table and the code for managing it. >>>> As we discussed offline, I removed this table. So webrev.1 checks >>>> the bottom of HeapRegion and then saves it at >>>> HeapRegion::_node_index. However we do need a way to check node >>>> index of the given address and so I added back. >>>> >>>> ?From my testing, checking a numa id of one page seems okay to do >>>> on runtime. Previously I did checking for all pages of given memory >>>> so it was a bit slow. >>>> >>>> One thing that didn't include at webrev.1 is combining with >>>> existing one. >>>> The newly introduced os::numa_get_address_id() is a copy from >>>> zNUMA_linux.cpp. It may be unified but didn't try yet. >>>> >>>> webrev: >>>> http://cr.openjdk.java.net/~sangheki/8220310/webrev.1 >>>> http://cr.openjdk.java.net/~sangheki/8220310/webrev.1.inc (this may >>>> not help much! :) ) >>>> Testing: hs-tier 1 ~ 5 (with/without UseNUMA) >>>> >>>> Thanks, >>>> Sangheon >>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> >>>> >> From daniel.daugherty at oracle.com Thu Sep 26 18:20:23 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 26 Sep 2019 14:20:23 -0400 Subject: RFR: 8231294: ZGC: vmTestbase/nsk/jvmti/ResourceExhausted/resexhausted002 fails In-Reply-To: <1452ee17-815c-396b-c121-c709c631836e@oracle.com> References: <7af66362-44da-a192-5688-ae1fc1987061@oracle.com> <6c76ecbe-75dc-2b50-64ab-5a75167c49cd@oracle.com> <1452ee17-815c-396b-c121-c709c631836e@oracle.com> Message-ID: <585efd40-6870-743b-d1b3-ed176c3a73c4@oracle.com> Just figured you might want to know that the heap size change was happening... :-) Dan On 9/26/19 2:06 PM, serguei.spitsyn at oracle.com wrote: > Dan, thank you for adding the serviceability-dev mailing list. > It looks good but has been already pushed. :) > > Thanks, > Serguei > > > On 9/26/19 07:28, Daniel D. Daugherty wrote: >> Just for the record... JVM/TI belongs to the Serviceability Team >> so I've added serviceability-dev at ... >> >> However, the Runtime team and Serviceability team quite often tag >> team on JVM/TI issues and lurk on each other's aliases... >> >> Dan >> >> >> On 9/26/19 3:14 AM, Per Liden wrote: >>> (CC:ing hotspot-runtime-dev, as the test belongs to runtime) >>> >>> On 9/25/19 10:57 AM, Per Liden wrote: >>>> The resexhausted002 test uses an 8M heap, which is a bit too small >>>> for ZGC. Bumping it to 128M to allow the test to pass on ZGC too. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231294 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8231294/webrev.0 >>>> >>>> Testing: Manual testing with G1 and ZGC >>>> >>>> /Per >> > From per.liden at oracle.com Thu Sep 26 20:39:01 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 26 Sep 2019 22:39:01 +0200 Subject: RFR: 8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails Message-ID: <0564cbfb-7a67-d0b8-2b51-824c4b93166b@oracle.com> Please review this one-liner to disable vmTestbase/nsk/jvmti/Allocate/alloc001 when using ZGC. The root problem is that ZGC doesn't properly respect address space limitations (RLIMIT_AS). This test will be re-enabled again as part of fixing https://bugs.openjdk.java.net/browse/JDK-8231552 Bug: https://bugs.openjdk.java.net/browse/JDK-8231296 Webrev: http://cr.openjdk.java.net/~pliden/8231296/webrev.0 /Per From igor.ignatyev at oracle.com Thu Sep 26 20:58:28 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 26 Sep 2019 13:58:28 -0700 Subject: RFR: 8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails In-Reply-To: <0564cbfb-7a67-d0b8-2b51-824c4b93166b@oracle.com> References: <0564cbfb-7a67-d0b8-2b51-824c4b93166b@oracle.com> Message-ID: <553F158A-E36E-4F9D-8D21-EE8FC63D0057@oracle.com> Hi Per, wouldn't it be better to put this test into zgc-specific problem list (test/hotspot/jtreg/ProblemList-zgc.txt)? Thanks, -- Igor > On Sep 26, 2019, at 1:39 PM, Per Liden wrote: > > Please review this one-liner to disable vmTestbase/nsk/jvmti/Allocate/alloc001 when using ZGC. The root problem is that ZGC doesn't properly respect address space limitations (RLIMIT_AS). This test will be re-enabled again as part of fixing https://bugs.openjdk.java.net/browse/JDK-8231552 > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231296 > Webrev: http://cr.openjdk.java.net/~pliden/8231296/webrev.0 > > /Per From per.liden at oracle.com Thu Sep 26 21:32:46 2019 From: per.liden at oracle.com (Per Liden) Date: Thu, 26 Sep 2019 23:32:46 +0200 Subject: RFR: 8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails In-Reply-To: <553F158A-E36E-4F9D-8D21-EE8FC63D0057@oracle.com> References: <0564cbfb-7a67-d0b8-2b51-824c4b93166b@oracle.com> <553F158A-E36E-4F9D-8D21-EE8FC63D0057@oracle.com> Message-ID: Hi Igor, I don't think it belongs in the problem list, for two reasons: 1) The test doesn't fail because of a bug. It fails because ZGC doesn't currently support that use case. In other words, the test shouldn't be testing something that isn't supposed to work. 2) As far as I know, the test in question is part of rt-tiers, where it's executed with the default GC (G1). So, the problem is typically only visible when doing non-CI testing with ZGC enabled. cheers, Per On 9/26/19 10:58 PM, Igor Ignatyev wrote: > Hi Per, > > wouldn't it be better to put this test into zgc-specific problem list (test/hotspot/jtreg/ProblemList-zgc.txt)? > > Thanks, > -- Igor > >> On Sep 26, 2019, at 1:39 PM, Per Liden wrote: >> >> Please review this one-liner to disable vmTestbase/nsk/jvmti/Allocate/alloc001 when using ZGC. The root problem is that ZGC doesn't properly respect address space limitations (RLIMIT_AS). This test will be re-enabled again as part of fixing https://bugs.openjdk.java.net/browse/JDK-8231552 >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231296 >> Webrev: http://cr.openjdk.java.net/~pliden/8231296/webrev.0 >> >> /Per > From serguei.spitsyn at oracle.com Thu Sep 26 22:20:36 2019 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 26 Sep 2019 15:20:36 -0700 Subject: RFR: 8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails In-Reply-To: <0564cbfb-7a67-d0b8-2b51-824c4b93166b@oracle.com> References: <0564cbfb-7a67-d0b8-2b51-824c4b93166b@oracle.com> Message-ID: <9d8db88e-f7dc-89ca-a827-c3fa57ce0d16@oracle.com> Hi Per, It looks good and trivial. Thanks, Serguei On 9/26/19 1:39 PM, Per Liden wrote: > Please review this one-liner to disable > vmTestbase/nsk/jvmti/Allocate/alloc001 when using ZGC. The root > problem is that ZGC doesn't properly respect address space limitations > (RLIMIT_AS). This test will be re-enabled again as part of fixing > https://bugs.openjdk.java.net/browse/JDK-8231552 > > Bug: https://bugs.openjdk.java.net/browse/JDK-8231296 > Webrev: http://cr.openjdk.java.net/~pliden/8231296/webrev.0 > > /Per From igor.ignatyev at oracle.com Fri Sep 27 01:49:49 2019 From: igor.ignatyev at oracle.com (Igor Ignatev) Date: Thu, 26 Sep 2019 18:49:49 -0700 Subject: RFR: 8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails In-Reply-To: References: Message-ID: <3B6004E4-D5EC-411E-845E-1D8712B3CC4D@oracle.com> Hi Per, > On Sep 26, 2019, at 2:32 PM, Per Liden wrote: > > ?Hi Igor, > > I don't think it belongs in the problem list, for two reasons: > > 1) The test doesn't fail because of a bug. It fails because ZGC doesn't currently support that use case. In other words, the test shouldn't be testing something that isn't supposed to work. Problem lists aren?t only to exclude tests which fail due to bugs, they can be and are used for use cases you described as well. > > 2) As far as I know, the test in question is part of rt-tiers, where it's executed with the default GC (G1). So, the problem is typically only visible when doing non-CI testing with ZGC enabled. I agree that necessity to pass extra make arts whenever you are doing adhoc testing, esp. on a localhost, is cumbersome, but you are expected to do that anyway as some of tests are known to fail only w/ zGC and are in ProblemList-zgc.txt. The main advantage of problem list over @requiers is semi-automatic reminding to re-enable tests when a defect is fixed/feature is implemented, it also sends a clear message that we plan to make a test pass. In compiler, we 1st use @requiers to temporary exclude tests from graal testing, but soon we realized that there are two different meanings:1) test isn't able to pass w/ graal and will never be able to and 2) test can?t pass w/ graal now (b/c of test / product bugs and/or some features aren?t there), but we want to fix product/test and reenable test. we now try to use @requiers for 1 and problem list for 2, so the reasons and expectations are clearer, but there are still some tests which have @requiers which actually should be in problem list, and it?s expensive to go thru all of them to dig out which it should be. So I just don?t want you guys to get the same problems we got. > > cheers, > Per > >> On 9/26/19 10:58 PM, Igor Ignatyev wrote: >> Hi Per, >> wouldn't it be better to put this test into zgc-specific problem list (test/hotspot/jtreg/ProblemList-zgc.txt)? >> Thanks, >> -- Igor >>>> On Sep 26, 2019, at 1:39 PM, Per Liden wrote: >>> >>> Please review this one-liner to disable vmTestbase/nsk/jvmti/Allocate/alloc001 when using ZGC. The root problem is that ZGC doesn't properly respect address space limitations (RLIMIT_AS). This test will be re-enabled again as part of fixing https://bugs.openjdk.java.net/browse/JDK-8231552 >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231296 >>> Webrev: http://cr.openjdk.java.net/~pliden/8231296/webrev.0 >>> >>> /Per From per.liden at oracle.com Fri Sep 27 06:16:19 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 27 Sep 2019 08:16:19 +0200 Subject: RFR: 8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails In-Reply-To: <3B6004E4-D5EC-411E-845E-1D8712B3CC4D@oracle.com> References: <3B6004E4-D5EC-411E-845E-1D8712B3CC4D@oracle.com> Message-ID: <9d34ee27-e581-6255-5796-85082a7a5a4b@oracle.com> Hi Igor, On 9/27/19 3:49 AM, Igor Ignatev wrote: > Hi Per, > >> On Sep 26, 2019, at 2:32 PM, Per Liden wrote: >> >> ?Hi Igor, >> >> I don't think it belongs in the problem list, for two reasons: >> >> 1) The test doesn't fail because of a bug. It fails because ZGC doesn't currently support that use case. In other words, the test shouldn't be testing something that isn't supposed to work. > Problem lists aren?t only to exclude tests which fail due to bugs, they can be and are used for use cases you described as well. >> >> 2) As far as I know, the test in question is part of rt-tiers, where it's executed with the default GC (G1). So, the problem is typically only visible when doing non-CI testing with ZGC enabled. > I agree that necessity to pass extra make arts whenever you are doing adhoc testing, esp. on a localhost, is cumbersome, but you are expected to do that anyway as some of tests are known to fail only w/ zGC and are in ProblemList-zgc.txt. > > The main advantage of problem list over @requiers is semi-automatic reminding to re-enable tests when a defect is fixed/feature is implemented, it also sends a clear message that we plan to make a test pass. > > In compiler, we 1st use @requiers to temporary exclude tests from graal testing, but soon we realized that there are two different meanings:1) test isn't able to pass w/ graal and will never be able to and 2) test can?t pass w/ graal now (b/c of test / product bugs and/or some features aren?t there), but we want to fix product/test and reenable test. we now try to use @requiers for 1 and problem list for 2, so the reasons and expectations are clearer, but there are still some tests which have @requiers which actually should be in problem list, and it?s expensive to go thru all of them to dig out which it should be. So I just don?t want you guys to get the same problems we got. Ok, thanks. I think I see the problem list more of a temporary measure to avoid noise in the CI pipeline, i.e. when the test should work but doesn't. In cases where a test isn't expected to work, I think @requires is better. However, I do see your point, it's not always a black or white call. cheers, Per >> >> cheers, >> Per >> >>> On 9/26/19 10:58 PM, Igor Ignatyev wrote: >>> Hi Per, >>> wouldn't it be better to put this test into zgc-specific problem list (test/hotspot/jtreg/ProblemList-zgc.txt)? >>> Thanks, >>> -- Igor >>>>> On Sep 26, 2019, at 1:39 PM, Per Liden wrote: >>>> >>>> Please review this one-liner to disable vmTestbase/nsk/jvmti/Allocate/alloc001 when using ZGC. The root problem is that ZGC doesn't properly respect address space limitations (RLIMIT_AS). This test will be re-enabled again as part of fixing https://bugs.openjdk.java.net/browse/JDK-8231552 >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231296 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8231296/webrev.0 >>>> >>>> /Per > From stefan.karlsson at oracle.com Fri Sep 27 07:16:24 2019 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 27 Sep 2019 09:16:24 +0200 Subject: RFR: 8231563: ZGC: Fails to warn when user sets the max heap size to larger than 16TB Message-ID: Hi all, Please review this small patch to fix the max heap size check in ZGC. https://cr.openjdk.java.net/~stefank/8231563/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8231563 After this fix the JVM refuses to start if a too high -Xmx is set: $ java -XX:+UnlockExperimentalVMOptions -XX:+UseZGC -Xmx17t -version: Error occurred during initialization of VM Java heap too large Thanks, StefanK From thomas.schatzl at oracle.com Fri Sep 27 08:46:50 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 27 Sep 2019 10:46:50 +0200 Subject: RFR: 8231563: ZGC: Fails to warn when user sets the max heap size to larger than 16TB In-Reply-To: References: Message-ID: Hi, On 27.09.19 09:16, Stefan Karlsson wrote: > Hi all, > > Please review this small patch to fix the max heap size check in ZGC. > > https://cr.openjdk.java.net/~stefank/8231563/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8231563 > > After this fix the JVM refuses to start if a too high -Xmx is set: > > $ java -XX:+UnlockExperimentalVMOptions -XX:+UseZGC -Xmx17t -version: > Error occurred during initialization of VM > Java heap too large looks good to me. Thomas From per.liden at oracle.com Fri Sep 27 08:58:55 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 27 Sep 2019 10:58:55 +0200 Subject: RFR: 8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails In-Reply-To: <9d8db88e-f7dc-89ca-a827-c3fa57ce0d16@oracle.com> References: <0564cbfb-7a67-d0b8-2b51-824c4b93166b@oracle.com> <9d8db88e-f7dc-89ca-a827-c3fa57ce0d16@oracle.com> Message-ID: Thanks Serguei! /Per On 9/27/19 12:20 AM, serguei.spitsyn at oracle.com wrote: > Hi Per, > > It looks good and trivial. > > Thanks, > Serguei > > On 9/26/19 1:39 PM, Per Liden wrote: >> Please review this one-liner to disable >> vmTestbase/nsk/jvmti/Allocate/alloc001 when using ZGC. The root >> problem is that ZGC doesn't properly respect address space limitations >> (RLIMIT_AS). This test will be re-enabled again as part of fixing >> https://bugs.openjdk.java.net/browse/JDK-8231552 >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8231296 >> Webrev: http://cr.openjdk.java.net/~pliden/8231296/webrev.0 >> >> /Per > From thomas.schatzl at oracle.com Fri Sep 27 10:26:04 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 27 Sep 2019 12:26:04 +0200 Subject: RFR (XXS): 8231553: Deprecate unused G1RSetScanBlockSize command line option Message-ID: <4f836c3a-8422-a929-a270-b0e7e47efc55@oracle.com> Hi all, can I have reviews for the changeset to start the removal process for the G1RSetScanBlockSize option? In JDK-8213108 we removed its (only) use, so we should get rid of the option. There is also a CSR for that (JDK-8231568) for which I need reviewers. CR: https://bugs.openjdk.java.net/browse/JDK-8231553 Webrev: http://cr.openjdk.java.net/~tschatzl/8231553/webrev Testing: local compilation, make sure that the deprecation message shows up Thanks, Thomaas From stefan.johansson at oracle.com Fri Sep 27 12:04:31 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 27 Sep 2019 14:04:31 +0200 Subject: RFR (XXS): 8231553: Deprecate unused G1RSetScanBlockSize command line option In-Reply-To: <4f836c3a-8422-a929-a270-b0e7e47efc55@oracle.com> References: <4f836c3a-8422-a929-a270-b0e7e47efc55@oracle.com> Message-ID: <2d03f36d-d424-2bf7-3598-eaf1a80456e2@oracle.com> Hi Thomas, On 2019-09-27 12:26, Thomas Schatzl wrote: > Hi all, > > ? can I have reviews for the changeset to start the removal process for > the G1RSetScanBlockSize option? In JDK-8213108 we removed its (only) > use, so we should get rid of the option. > > There is also a CSR for that (JDK-8231568) for which I need reviewers. Reviewed. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8231553 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8231553/webrev Looks good, StefanJ > Testing: > local compilation, make sure that the deprecation message shows up > > Thanks, > ? Thomaas From thomas.schatzl at oracle.com Fri Sep 27 12:17:08 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 27 Sep 2019 14:17:08 +0200 Subject: RFR (XXS): 8231553: Deprecate unused G1RSetScanBlockSize command line option In-Reply-To: <2d03f36d-d424-2bf7-3598-eaf1a80456e2@oracle.com> References: <4f836c3a-8422-a929-a270-b0e7e47efc55@oracle.com> <2d03f36d-d424-2bf7-3598-eaf1a80456e2@oracle.com> Message-ID: Hi, On 27.09.19 14:04, Stefan Johansson wrote: > Hi Thomas, > > On 2019-09-27 12:26, Thomas Schatzl wrote: >> Hi all, >> >> ?? can I have reviews for the changeset to start the removal process >> for the G1RSetScanBlockSize option? In JDK-8213108 we removed its >> (only) use, so we should get rid of the option. >> >> There is also a CSR for that (JDK-8231568) for which I need reviewers. > Reviewed. > >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8231553 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8231553/webrev > Looks good, > StefanJ > thanks and thanks. Thomas From leo.korinth at oracle.com Fri Sep 27 12:23:22 2019 From: leo.korinth at oracle.com (Leo Korinth) Date: Fri, 27 Sep 2019 14:23:22 +0200 Subject: RFR (XXS): 8231553: Deprecate unused G1RSetScanBlockSize command line option In-Reply-To: References: <4f836c3a-8422-a929-a270-b0e7e47efc55@oracle.com> <2d03f36d-d424-2bf7-3598-eaf1a80456e2@oracle.com> Message-ID: Looks good! Thanks, Leo On 27/09/2019 14:17, Thomas Schatzl wrote: > Hi, > > On 27.09.19 14:04, Stefan Johansson wrote: >> Hi Thomas, >> >> On 2019-09-27 12:26, Thomas Schatzl wrote: >>> Hi all, >>> >>> ?? can I have reviews for the changeset to start the removal process >>> for the G1RSetScanBlockSize option? In JDK-8213108 we removed its >>> (only) use, so we should get rid of the option. >>> >>> There is also a CSR for that (JDK-8231568) for which I need reviewers. >> Reviewed. >> >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8231553 >>> Webrev: >>> http://cr.openjdk.java.net/~tschatzl/8231553/webrev >> Looks good, >> StefanJ >> > > ? thanks and thanks. > > Thomas From thomas.schatzl at oracle.com Fri Sep 27 12:32:14 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 27 Sep 2019 14:32:14 +0200 Subject: RFR (XXS): 8231553: Deprecate unused G1RSetScanBlockSize command line option In-Reply-To: References: <4f836c3a-8422-a929-a270-b0e7e47efc55@oracle.com> <2d03f36d-d424-2bf7-3598-eaf1a80456e2@oracle.com> Message-ID: Hi Leo, On 27.09.19 14:23, Leo Korinth wrote: > Looks good! > > Thanks, > Leo > thanks for your review! Thomas From daniel.daugherty at oracle.com Fri Sep 27 12:48:48 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 27 Sep 2019 08:48:48 -0400 Subject: RFR: 8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails In-Reply-To: <9d34ee27-e581-6255-5796-85082a7a5a4b@oracle.com> References: <3B6004E4-D5EC-411E-845E-1D8712B3CC4D@oracle.com> <9d34ee27-e581-6255-5796-85082a7a5a4b@oracle.com> Message-ID: <383ff735-b7de-2bdb-9add-cfb1320ccd9c@oracle.com> On 9/27/19 2:16 AM, Per Liden wrote: > Hi Igor, > > On 9/27/19 3:49 AM, Igor Ignatev wrote: >> Hi Per, >> >>> On Sep 26, 2019, at 2:32 PM, Per Liden wrote: >>> >>> ?Hi Igor, >>> >>> I don't think it belongs in the problem list, for two reasons: >>> >>> 1) The test doesn't fail because of a bug. It fails because ZGC >>> doesn't currently support that use case. In other words, the test >>> shouldn't be testing something that isn't supposed to work. >> Problem lists aren?t only to exclude tests which fail due to bugs, >> they can be and are used for use cases you described as well. >>> >>> 2) As far as I know, the test in question is part of rt-tiers, where >>> it's executed with the default GC (G1). So, the problem is typically >>> only visible when doing non-CI testing with ZGC enabled. >> I agree that necessity to pass extra make arts whenever you are doing >> adhoc testing, esp. on a localhost, is cumbersome, but you are >> expected to do that anyway as some of tests are known to fail only w/ >> zGC and are in ProblemList-zgc.txt. >> >> The main advantage of problem list over @requiers is semi-automatic >> reminding to re-enable tests when a defect is fixed/feature is >> implemented, it also sends a clear message that we plan to make a >> test pass. >> >> In compiler, we 1st use @requiers to temporary exclude tests from >> graal testing, but soon we realized that there are two different >> meanings:1) test isn't able to pass w/ graal and will never be able >> to and 2) test can?t pass w/ graal now (b/c of test / product bugs >> and/or some features aren?t there), but we want to fix product/test >> and reenable test. we now try to use @requiers for 1 and problem list >> for 2, so the reasons and expectations are clearer, but there are >> still some tests which have @requiers which actually should be in >> problem list, and it?s expensive to go thru all of them to dig out >> which it should be. So I just don?t want you guys to get the same >> problems we got. > > Ok, thanks. I think I see the problem list more of a temporary measure > to avoid noise in the CI pipeline, i.e. when the test should work but > doesn't. In cases where a test isn't expected to work, I think > @requires is better. However, I do see your point, it's not always a > black or white call. In your code review invite you say this: > The root problem is that ZGC doesn't properly respect address space > limitations (RLIMIT_AS). To me that says the current behavior is temporary and then you say this: > This test will be re-enabled again as part of fixing > https://bugs.openjdk.java.net/browse/JDK-8231552 and that confirms that the issue is planned to be resolved. I'm not sure that I agree that 8231552 is an RFE rather than a bug, but that's not the point here... To me, both of those sentences would lead to a ProblemListing and not an @requires. Your call... Dan > > cheers, > Per > >>> >>> cheers, >>> Per >>> >>>> On 9/26/19 10:58 PM, Igor Ignatyev wrote: >>>> Hi Per, >>>> wouldn't it be better to put this test into zgc-specific problem >>>> list (test/hotspot/jtreg/ProblemList-zgc.txt)? >>>> Thanks, >>>> -- Igor >>>>>> On Sep 26, 2019, at 1:39 PM, Per Liden wrote: >>>>> >>>>> Please review this one-liner to disable >>>>> vmTestbase/nsk/jvmti/Allocate/alloc001 when using ZGC. The root >>>>> problem is that ZGC doesn't properly respect address space >>>>> limitations (RLIMIT_AS). This test will be re-enabled again as >>>>> part of fixing https://bugs.openjdk.java.net/browse/JDK-8231552 >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231296 >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8231296/webrev.0 >>>>> >>>>> /Per >> From per.liden at oracle.com Fri Sep 27 13:18:31 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 27 Sep 2019 15:18:31 +0200 Subject: RFR: 8231563: ZGC: Fails to warn when user sets the max heap size to larger than 16TB In-Reply-To: References: Message-ID: <18a67b8d-79b1-ccfd-daa5-9f8552ee2f9a@oracle.com> Looks good! /Per On 9/27/19 9:16 AM, Stefan Karlsson wrote: > Hi all, > > Please review this small patch to fix the max heap size check in ZGC. > > https://cr.openjdk.java.net/~stefank/8231563/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8231563 > > After this fix the JVM refuses to start if a too high -Xmx is set: > > $ java -XX:+UnlockExperimentalVMOptions -XX:+UseZGC -Xmx17t -version: > Error occurred during initialization of VM > Java heap too large > > Thanks, > StefanK From per.liden at oracle.com Fri Sep 27 13:39:26 2019 From: per.liden at oracle.com (Per Liden) Date: Fri, 27 Sep 2019 15:39:26 +0200 Subject: RFR: 8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails In-Reply-To: <383ff735-b7de-2bdb-9add-cfb1320ccd9c@oracle.com> References: <3B6004E4-D5EC-411E-845E-1D8712B3CC4D@oracle.com> <9d34ee27-e581-6255-5796-85082a7a5a4b@oracle.com> <383ff735-b7de-2bdb-9add-cfb1320ccd9c@oracle.com> Message-ID: <096bb4aa-f4fb-c369-b5b9-461a60ad9c55@oracle.com> Hi Dan, The background here is that ZGC previously (by design) required a certain amount of address space to run. So it was a deliberate decision to not support RLIMIT_AS. However, as ZGC's design have progressed, we're now in a position where we could reconsider this (which is why the RFE was filed). In _theory_, we could come to the conclusion that we still don't want to support this and just close the RFE as Won't fix. That's (very) unlikely, but I think that line of thinking shows why sticking this in the problem list is the wrong thing to do here, because that problem list entry would then potentially stay there forever. Until that RFE is pushed, that test should not even attempt to test it with ZGC, because it's not a supported use case. Anyways... ;) cheers, Per On 9/27/19 2:48 PM, Daniel D. Daugherty wrote: [...] > > In your code review invite you say this: > >> The root problem is that ZGC doesn't properly respect address space >> limitations (RLIMIT_AS). > > To me that says the current behavior is temporary and then you say this: > >> This test will be re-enabled again as part of fixing >> https://bugs.openjdk.java.net/browse/JDK-8231552 > > and that confirms that the issue is planned to be resolved. I'm not > sure that I agree that 8231552 is an RFE rather than a bug, but that's > not the point here... > > To me, both of those sentences would lead to a ProblemListing > and not an @requires. Your call... > > Dan > > >> >> cheers, >> Per >> >>>> >>>> cheers, >>>> Per >>>> >>>>> On 9/26/19 10:58 PM, Igor Ignatyev wrote: >>>>> Hi Per, >>>>> wouldn't it be better to put this test into zgc-specific problem >>>>> list (test/hotspot/jtreg/ProblemList-zgc.txt)? >>>>> Thanks, >>>>> -- Igor >>>>>>> On Sep 26, 2019, at 1:39 PM, Per Liden wrote: >>>>>> >>>>>> Please review this one-liner to disable >>>>>> vmTestbase/nsk/jvmti/Allocate/alloc001 when using ZGC. The root >>>>>> problem is that ZGC doesn't properly respect address space >>>>>> limitations (RLIMIT_AS). This test will be re-enabled again as >>>>>> part of fixing https://bugs.openjdk.java.net/browse/JDK-8231552 >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231296 >>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8231296/webrev.0 >>>>>> >>>>>> /Per >>> > From daniel.daugherty at oracle.com Fri Sep 27 13:43:56 2019 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 27 Sep 2019 09:43:56 -0400 Subject: RFR: 8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails In-Reply-To: <096bb4aa-f4fb-c369-b5b9-461a60ad9c55@oracle.com> References: <3B6004E4-D5EC-411E-845E-1D8712B3CC4D@oracle.com> <9d34ee27-e581-6255-5796-85082a7a5a4b@oracle.com> <383ff735-b7de-2bdb-9add-cfb1320ccd9c@oracle.com> <096bb4aa-f4fb-c369-b5b9-461a60ad9c55@oracle.com> Message-ID: <7c0124c3-ead6-05d1-e15a-5023752753cd@oracle.com> Thanks for the background info. I think that's a more convincing explanation and I'm okay with the change as is. Please add a comment to JDK-8231552 making it clear that vmTestbase/nsk/jvmti/Allocate/alloc001 should be reenabled if/when the bug^H^H^H RFE is fixed. :-) Dan On 9/27/19 9:39 AM, Per Liden wrote: > Hi Dan, > > The background here is that ZGC previously (by design) required a > certain amount of address space to run. So it was a deliberate > decision to not support RLIMIT_AS. However, as ZGC's design have > progressed, we're now in a position where we could reconsider this > (which is why the RFE was filed). In _theory_, we could come to the > conclusion that we still don't want to support this and just close the > RFE as Won't fix. That's (very) unlikely, but I think that line of > thinking shows why sticking this in the problem list is the wrong > thing to do here, because that problem list entry would then > potentially stay there forever. Until that RFE is pushed, that test > should not even attempt to test it with ZGC, because it's not a > supported use case. > > Anyways... ;) > > cheers, > Per > > On 9/27/19 2:48 PM, Daniel D. Daugherty wrote: > [...] >> >> In your code review invite you say this: >> >>> The root problem is that ZGC doesn't properly respect address space >>> limitations (RLIMIT_AS). >> >> To me that says the current behavior is temporary and then you say this: >> >>> This test will be re-enabled again as part of fixing >>> https://bugs.openjdk.java.net/browse/JDK-8231552 >> >> and that confirms that the issue is planned to be resolved. I'm not >> sure that I agree that 8231552 is an RFE rather than a bug, but that's >> not the point here... >> >> To me, both of those sentences would lead to a ProblemListing >> and not an @requires. Your call... >> >> Dan >> >> >>> >>> cheers, >>> Per >>> >>>>> >>>>> cheers, >>>>> Per >>>>> >>>>>> On 9/26/19 10:58 PM, Igor Ignatyev wrote: >>>>>> Hi Per, >>>>>> wouldn't it be better to put this test into zgc-specific problem >>>>>> list (test/hotspot/jtreg/ProblemList-zgc.txt)? >>>>>> Thanks, >>>>>> -- Igor >>>>>>>> On Sep 26, 2019, at 1:39 PM, Per Liden >>>>>>>> wrote: >>>>>>> >>>>>>> Please review this one-liner to disable >>>>>>> vmTestbase/nsk/jvmti/Allocate/alloc001 when using ZGC. The root >>>>>>> problem is that ZGC doesn't properly respect address space >>>>>>> limitations (RLIMIT_AS). This test will be re-enabled again as >>>>>>> part of fixing https://bugs.openjdk.java.net/browse/JDK-8231552 >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231296 >>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8231296/webrev.0 >>>>>>> >>>>>>> /Per >>>> >> From stefan.johansson at oracle.com Fri Sep 27 21:16:30 2019 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Fri, 27 Sep 2019 23:16:30 +0200 Subject: RFR (L): 8230706: Waiting on completion of strong nmethod processing causes long pause times with G1 In-Reply-To: References: Message-ID: <0F637570-EC97-47C5-B493-B33681133149@oracle.com> Hi Thomas, > 25 sep. 2019 kl. 12:42 skrev Thomas Schatzl : > > Hi all, > > can I have reviews for this change that fixes a regression introduced in jdk11? > > So, currently in g1, in the concurrent start pause we have to mark through all live oops referenced from thread stacks to keep those alive across the concurrent marking in presence of class unloading. These are "strong" nmethods. > > However G1 also scans oops of nmethods as part of the per-region nmethod remembered sets, but should not mark their oops as their classes should remain unloadable. These are "weak" nmethods. > > Regardless of either way of processing, nmethods are claimed, to prevent processing them multiple times as happens often. Which means that if an nmethod is claimed first via the "weak" path, its oops will not be marked properly, and *boom*. > > In order to prevent this, currently there is a hard synchronization, i.e. wait barrrier between strong nmethod processing and the actual evacuation (which ultimately does the weak nmethod processing). > > This is a problem, particularly since jdk11, because stacks are claimed on a per thread basis. I.e. an extremely deep stack could make all 100's or 1000's of your cpu cores wait idly. The "particular since jdk11" part is that since then the amount of work done during nmethod iteration is only bounded by the amount of live objects to copy (i.e. JDK-6672778 added task queue trimming which is generally a good thing), which can take a lot of time. > > In this case, in some internal Cassandra setup we have seen ridiculously long waiting times as a single thread apparently evacuates the whole young gen... :( > > This change moves the wait to the end of the evacuation, remembering any nmethods that were claimed by the weak nmethod processing before the strong nmethod processing got to it. > > Since the amount of nmethods that need to be marked through has always been very low (single digit), that phase took <0.1ms typically. There is some attempt to parallelize this phase based on the number of nmethods (this pass only needs to mark the oops <= TAMS in nmethods, no more) anyway though. > > I will look into merging parallel phases into a single one in the post-evacuation phase soon to get rid of this additional spin-up of threads (only during concurrent mark) again. > > During this work I tried several alternatives that were rejected: > - disabling task queue trimming; that works, but still has the problem with deep thread stacks > - moving the actual wait deep into the evacuation code: made a mess with the code, and still does not really solve the problem > - instead of remembering nmethods, concurrently process them. Does not work because x86 embedded oops are not word-aligned, so this is not a good idea. > > There is always the option of doing the stack scanning in the concurrent phase: that seemingly requires much more work, e.g. by using method return barriers and has been left as a future enhancement. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8230706 > Webrev: > http://cr.openjdk.java.net/~tschatzl/8230706/webrev/ Thanks for hunting down and fixing this problem. The change looks good in general, but I have some comments: src/hotspot/share/code/nmethod.cpp ? I would like the bit-operations to be hidden in functions, something like: is_claimed(nmethod* nm) { return ((uintptr_t)nm & 0x1) != 0;} claim(nmethod* nm) { return (nmethod*)((uintptr_t)nm | 0x1); } mask_claimed(nmethod* nm) { return (nmethod*)((uintptr_t)nm & ~0x1); } The names might be improved, but you get my point. ? src/hotspot/share/gc/g1/g1CodeBlobClosure.hpp ? 55 // Do we need to collect nmethods if they have already been claimed. 56 bool is_strong_iteration() const { return _pss != NULL; } I?m not sure about this name, especially since it won?t be true for all ?strong? closures, just the ones during initial mark right? Naming is hard, so I?m not sure I have a better suggestion overall, but what do you think about calling this method should_collect_strong_nmethods()? --- src/hotspot/share/gc/g1/g1CollectedHeap.cpp ? 3845 class G1MarkStrongNMethodsTask : public AbstractGangTask { 3846 G1ParScanThreadStateSet* _per_thread_states; 3847 volatile uint _claim; 3848 public: 3849 G1MarkStrongNMethodsTask(G1ParScanThreadStateSet* per_thread_states) 3850 : AbstractGangTask("Remark strong nmethods"), _per_thread_states(per_thread_states), _claim(0) { } 3851 3852 void work(uint worker_id) { 3853 G1RemarkStrongNMethodsClosure cl(worker_id); 3854 _per_thread_states->iterate(&cl, &_claim); 3855 } 3856 } cl(per_thread_states); Move the G1MarkStrongNMethodsTask definition outside the function, it think that is more along the lines what we usually do. ? Thanks, Stefan > Testing: > hs-tier1-5, many cassandra runs - no more exceptionally long concurrent start pauses :) > > Thanks, > Thomas From igor.ignatyev at oracle.com Sat Sep 28 16:47:52 2019 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Sat, 28 Sep 2019 09:47:52 -0700 Subject: RFR: 8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails In-Reply-To: <7c0124c3-ead6-05d1-e15a-5023752753cd@oracle.com> References: <3B6004E4-D5EC-411E-845E-1D8712B3CC4D@oracle.com> <9d34ee27-e581-6255-5796-85082a7a5a4b@oracle.com> <383ff735-b7de-2bdb-9add-cfb1320ccd9c@oracle.com> <096bb4aa-f4fb-c369-b5b9-461a60ad9c55@oracle.com> <7c0124c3-ead6-05d1-e15a-5023752753cd@oracle.com> Message-ID: <58EB6707-22C7-44BB-9F0A-D73DD4E764EF@oracle.com> Hi Per, if you decide to go w/ @requires, I'd encourage you to also put an explaining comment in the test descriptor using jtreg's tag @comment, so at least it will be easier for future readers to understand why the test can't / shouldn't be run w/ ZGC. Thanks, -- Igor > On Sep 27, 2019, at 6:43 AM, Daniel D. Daugherty wrote: > > Thanks for the background info. I think that's a more convincing > explanation and I'm okay with the change as is. > > Please add a comment to JDK-8231552 making it clear that > vmTestbase/nsk/jvmti/Allocate/alloc001 should be reenabled > if/when the bug^H^H^H RFE is fixed. :-) > > Dan > > > On 9/27/19 9:39 AM, Per Liden wrote: >> Hi Dan, >> >> The background here is that ZGC previously (by design) required a certain amount of address space to run. So it was a deliberate decision to not support RLIMIT_AS. However, as ZGC's design have progressed, we're now in a position where we could reconsider this (which is why the RFE was filed). In _theory_, we could come to the conclusion that we still don't want to support this and just close the RFE as Won't fix. That's (very) unlikely, but I think that line of thinking shows why sticking this in the problem list is the wrong thing to do here, because that problem list entry would then potentially stay there forever. Until that RFE is pushed, that test should not even attempt to test it with ZGC, because it's not a supported use case. >> >> Anyways... ;) >> >> cheers, >> Per >> >> On 9/27/19 2:48 PM, Daniel D. Daugherty wrote: >> [...] >>> >>> In your code review invite you say this: >>> >>>> The root problem is that ZGC doesn't properly respect address space limitations (RLIMIT_AS). >>> >>> To me that says the current behavior is temporary and then you say this: >>> >>>> This test will be re-enabled again as part of fixing https://bugs.openjdk.java.net/browse/JDK-8231552 >>> >>> and that confirms that the issue is planned to be resolved. I'm not >>> sure that I agree that 8231552 is an RFE rather than a bug, but that's >>> not the point here... >>> >>> To me, both of those sentences would lead to a ProblemListing >>> and not an @requires. Your call... >>> >>> Dan >>> >>> >>>> >>>> cheers, >>>> Per >>>> >>>>>> >>>>>> cheers, >>>>>> Per >>>>>> >>>>>>> On 9/26/19 10:58 PM, Igor Ignatyev wrote: >>>>>>> Hi Per, >>>>>>> wouldn't it be better to put this test into zgc-specific problem list (test/hotspot/jtreg/ProblemList-zgc.txt)? >>>>>>> Thanks, >>>>>>> -- Igor >>>>>>>>> On Sep 26, 2019, at 1:39 PM, Per Liden wrote: >>>>>>>> >>>>>>>> Please review this one-liner to disable vmTestbase/nsk/jvmti/Allocate/alloc001 when using ZGC. The root problem is that ZGC doesn't properly respect address space limitations (RLIMIT_AS). This test will be re-enabled again as part of fixing https://bugs.openjdk.java.net/browse/JDK-8231552 >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231296 >>>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8231296/webrev.0 >>>>>>>> >>>>>>>> /Per >>>>> >>> > From thomas.schatzl at oracle.com Mon Sep 30 11:14:00 2019 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 30 Sep 2019 13:14:00 +0200 Subject: RFR (L): 8230706: Waiting on completion of strong nmethod processing causes long pause times with G1 In-Reply-To: <0F637570-EC97-47C5-B493-B33681133149@oracle.com> References: <0F637570-EC97-47C5-B493-B33681133149@oracle.com> Message-ID: <5c6b06b1-de44-3cb7-7fc8-0b641df5f353@oracle.com> Hi Stefan, On 27.09.19 23:16, Stefan Johansson wrote: > Hi Thomas, > >> 25 sep. 2019 kl. 12:42 skrev Thomas Schatzl : >> >> Hi all, >> >> can I have reviews for this change that fixes a regression introduced in jdk11? [..] >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8230706 >> Webrev: >> http://cr.openjdk.java.net/~tschatzl/8230706/webrev/ > > Thanks for hunting down and fixing this problem. The change looks good in general, but I have some comments: > src/hotspot/share/code/nmethod.cpp > ? > I would like the bit-operations to be hidden in functions, something like: > is_claimed(nmethod* nm) { return ((uintptr_t)nm & 0x1) != 0;} > claim(nmethod* nm) { return (nmethod*)((uintptr_t)nm | 0x1); } > mask_claimed(nmethod* nm) { return (nmethod*)((uintptr_t)nm & ~0x1); } > The names might be improved, but you get my point. > ? > > src/hotspot/share/gc/g1/g1CodeBlobClosure.hpp > ? > 55 // Do we need to collect nmethods if they have already been claimed. > 56 bool is_strong_iteration() const { return _pss != NULL; } > > I?m not sure about this name, especially since it won?t be true for all ?strong? closures, just the ones during initial mark right? Naming is hard, so I?m not sure I have a better suggestion overall, but what do you think about calling this method should_collect_strong_nmethods()? > --- > > src/hotspot/share/gc/g1/g1CollectedHeap.cpp > ? > 3845 class G1MarkStrongNMethodsTask : public AbstractGangTask { > 3846 G1ParScanThreadStateSet* _per_thread_states; > 3847 volatile uint _claim; > 3848 public: > 3849 G1MarkStrongNMethodsTask(G1ParScanThreadStateSet* per_thread_states) > 3850 : AbstractGangTask("Remark strong nmethods"), _per_thread_states(per_thread_states), _claim(0) { } > 3851 > 3852 void work(uint worker_id) { > 3853 G1RemarkStrongNMethodsClosure cl(worker_id); > 3854 _per_thread_states->iterate(&cl, &_claim); > 3855 } > 3856 } cl(per_thread_states); > > Move the G1MarkStrongNMethodsTask definition outside the function, it think that is more along the lines what we usually do. > ? All fixed in new webrev: http://cr.openjdk.java.net/~tschatzl/8230706/webrev.0_to_1 (diff) http://cr.openjdk.java.net/~tschatzl/8230706/webrev.1 (full) Rerunning hs-tier1-5, almost done Thanks, Thomas From rkennke at redhat.com Mon Sep 30 13:12:19 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 30 Sep 2019 15:12:19 +0200 Subject: RFR: JDK-8231583: Shenandoah fails with finagle-http Message-ID: <996d7d26-4c18-62fd-ff57-76f1dddfd5c5@redhat.com> We're seeing failure with the finagle-http workload in renaissance benchmarks. The problem is caused by ShBSA::resolve_forward_pointer_not_null() trying to borrow a register. In order to do so, it pushes rscratch1, and later pops it again. This clashes when dst also happens to be rscratch1, in this case the pop would restore the original value. We need the register allocator to allocate an extra tmp reg for this. C1 should be good because rscratch1 is never used by register allocation there. The whole cmpxchg-oop sequence can be simplified a lot. Maybe we can reduce tmp register usage there? Bug: https://bugs.openjdk.java.net/browse/JDK-8231583 Webrev: http://cr.openjdk.java.net/~rkennke/JDK-8231583/webrev.00/ Testing: the failing test now passes. hotspot_gc_shenandoah is good too. Ok? Roman From per.liden at oracle.com Mon Sep 30 13:17:56 2019 From: per.liden at oracle.com (Per Liden) Date: Mon, 30 Sep 2019 15:17:56 +0200 Subject: RFR: 8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails In-Reply-To: <7c0124c3-ead6-05d1-e15a-5023752753cd@oracle.com> References: <3B6004E4-D5EC-411E-845E-1D8712B3CC4D@oracle.com> <9d34ee27-e581-6255-5796-85082a7a5a4b@oracle.com> <383ff735-b7de-2bdb-9add-cfb1320ccd9c@oracle.com> <096bb4aa-f4fb-c369-b5b9-461a60ad9c55@oracle.com> <7c0124c3-ead6-05d1-e15a-5023752753cd@oracle.com> Message-ID: <91ae07aa-9004-fd38-e891-b1aafebc6db5@oracle.com> Hi Dan, On 9/27/19 3:43 PM, Daniel D. Daugherty wrote: > Thanks for the background info. I think that's a more convincing > explanation and I'm okay with the change as is. Thanks! > > Please add a comment to JDK-8231552 making it clear that > vmTestbase/nsk/jvmti/Allocate/alloc001 should be reenabled > if/when the bug^H^H^H RFE is fixed. :-) Done ;) > > Dan > > > On 9/27/19 9:39 AM, Per Liden wrote: >> Hi Dan, >> >> The background here is that ZGC previously (by design) required a >> certain amount of address space to run. So it was a deliberate >> decision to not support RLIMIT_AS. However, as ZGC's design have >> progressed, we're now in a position where we could reconsider this >> (which is why the RFE was filed). In _theory_, we could come to the >> conclusion that we still don't want to support this and just close the >> RFE as Won't fix. That's (very) unlikely, but I think that line of >> thinking shows why sticking this in the problem list is the wrong >> thing to do here, because that problem list entry would then >> potentially stay there forever. Until that RFE is pushed, that test >> should not even attempt to test it with ZGC, because it's not a >> supported use case. >> >> Anyways... ;) >> >> cheers, >> Per >> >> On 9/27/19 2:48 PM, Daniel D. Daugherty wrote: >> [...] >>> >>> In your code review invite you say this: >>> >>>> The root problem is that ZGC doesn't properly respect address space >>>> limitations (RLIMIT_AS). >>> >>> To me that says the current behavior is temporary and then you say this: >>> >>>> This test will be re-enabled again as part of fixing >>>> https://bugs.openjdk.java.net/browse/JDK-8231552 >>> >>> and that confirms that the issue is planned to be resolved. I'm not >>> sure that I agree that 8231552 is an RFE rather than a bug, but that's >>> not the point here... >>> >>> To me, both of those sentences would lead to a ProblemListing >>> and not an @requires. Your call... >>> >>> Dan >>> >>> >>>> >>>> cheers, >>>> Per >>>> >>>>>> >>>>>> cheers, >>>>>> Per >>>>>> >>>>>>> On 9/26/19 10:58 PM, Igor Ignatyev wrote: >>>>>>> Hi Per, >>>>>>> wouldn't it be better to put this test into zgc-specific problem >>>>>>> list (test/hotspot/jtreg/ProblemList-zgc.txt)? >>>>>>> Thanks, >>>>>>> -- Igor >>>>>>>>> On Sep 26, 2019, at 1:39 PM, Per Liden >>>>>>>>> wrote: >>>>>>>> >>>>>>>> Please review this one-liner to disable >>>>>>>> vmTestbase/nsk/jvmti/Allocate/alloc001 when using ZGC. The root >>>>>>>> problem is that ZGC doesn't properly respect address space >>>>>>>> limitations (RLIMIT_AS). This test will be re-enabled again as >>>>>>>> part of fixing https://bugs.openjdk.java.net/browse/JDK-8231552 >>>>>>>> >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231296 >>>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8231296/webrev.0 >>>>>>>> >>>>>>>> /Per >>>>> >>> > From per.liden at oracle.com Mon Sep 30 13:18:39 2019 From: per.liden at oracle.com (Per Liden) Date: Mon, 30 Sep 2019 15:18:39 +0200 Subject: RFR: 8231296: ZGC: vmTestbase/nsk/jvmti/Allocate/alloc001/ fails In-Reply-To: <58EB6707-22C7-44BB-9F0A-D73DD4E764EF@oracle.com> References: <3B6004E4-D5EC-411E-845E-1D8712B3CC4D@oracle.com> <9d34ee27-e581-6255-5796-85082a7a5a4b@oracle.com> <383ff735-b7de-2bdb-9add-cfb1320ccd9c@oracle.com> <096bb4aa-f4fb-c369-b5b9-461a60ad9c55@oracle.com> <7c0124c3-ead6-05d1-e15a-5023752753cd@oracle.com> <58EB6707-22C7-44BB-9F0A-D73DD4E764EF@oracle.com> Message-ID: <13de14f2-a6c0-1eaf-56bd-ffff3d5f2da7@oracle.com> Hi Igor, On 9/28/19 6:47 PM, Igor Ignatyev wrote: > Hi Per, > > if you decide to go w/ @requires, I'd encourage you to also put an explaining comment in the test descriptor using jtreg's tag @comment, so at least it will be easier for future readers to understand why the test can't / shouldn't be run w/ ZGC. Will do! cheers, Per > > Thanks, > -- Igor > >> On Sep 27, 2019, at 6:43 AM, Daniel D. Daugherty wrote: >> >> Thanks for the background info. I think that's a more convincing >> explanation and I'm okay with the change as is. >> >> Please add a comment to JDK-8231552 making it clear that >> vmTestbase/nsk/jvmti/Allocate/alloc001 should be reenabled >> if/when the bug^H^H^H RFE is fixed. :-) >> >> Dan >> >> >> On 9/27/19 9:39 AM, Per Liden wrote: >>> Hi Dan, >>> >>> The background here is that ZGC previously (by design) required a certain amount of address space to run. So it was a deliberate decision to not support RLIMIT_AS. However, as ZGC's design have progressed, we're now in a position where we could reconsider this (which is why the RFE was filed). In _theory_, we could come to the conclusion that we still don't want to support this and just close the RFE as Won't fix. That's (very) unlikely, but I think that line of thinking shows why sticking this in the problem list is the wrong thing to do here, because that problem list entry would then potentially stay there forever. Until that RFE is pushed, that test should not even attempt to test it with ZGC, because it's not a supported use case. >>> >>> Anyways... ;) >>> >>> cheers, >>> Per >>> >>> On 9/27/19 2:48 PM, Daniel D. Daugherty wrote: >>> [...] >>>> >>>> In your code review invite you say this: >>>> >>>>> The root problem is that ZGC doesn't properly respect address space limitations (RLIMIT_AS). >>>> >>>> To me that says the current behavior is temporary and then you say this: >>>> >>>>> This test will be re-enabled again as part of fixing https://bugs.openjdk.java.net/browse/JDK-8231552 >>>> >>>> and that confirms that the issue is planned to be resolved. I'm not >>>> sure that I agree that 8231552 is an RFE rather than a bug, but that's >>>> not the point here... >>>> >>>> To me, both of those sentences would lead to a ProblemListing >>>> and not an @requires. Your call... >>>> >>>> Dan >>>> >>>> >>>>> >>>>> cheers, >>>>> Per >>>>> >>>>>>> >>>>>>> cheers, >>>>>>> Per >>>>>>> >>>>>>>> On 9/26/19 10:58 PM, Igor Ignatyev wrote: >>>>>>>> Hi Per, >>>>>>>> wouldn't it be better to put this test into zgc-specific problem list (test/hotspot/jtreg/ProblemList-zgc.txt)? >>>>>>>> Thanks, >>>>>>>> -- Igor >>>>>>>>>> On Sep 26, 2019, at 1:39 PM, Per Liden wrote: >>>>>>>>> >>>>>>>>> Please review this one-liner to disable vmTestbase/nsk/jvmti/Allocate/alloc001 when using ZGC. The root problem is that ZGC doesn't properly respect address space limitations (RLIMIT_AS). This test will be re-enabled again as part of fixing https://bugs.openjdk.java.net/browse/JDK-8231552 >>>>>>>>> >>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8231296 >>>>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8231296/webrev.0 >>>>>>>>> >>>>>>>>> /Per >>>>>> >>>> >> > From shade at redhat.com Mon Sep 30 13:32:38 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 30 Sep 2019 15:32:38 +0200 Subject: RFR: JDK-8231583: Shenandoah fails with finagle-http In-Reply-To: <996d7d26-4c18-62fd-ff57-76f1dddfd5c5@redhat.com> References: <996d7d26-4c18-62fd-ff57-76f1dddfd5c5@redhat.com> Message-ID: <229ca151-528b-8f4e-031b-fa1bd84e3053@redhat.com> On 9/30/19 3:12 PM, Roman Kennke wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8231583 > Webrev: > http://cr.openjdk.java.net/~rkennke/JDK-8231583/webrev.00/ Awww. Please change the synopsis to reflect what is being done, not the symptom of it. Suggestion: Shenandoah: register clash in SBSA::resolve_forwarding_pointer borrowing I believe we are better off just fixing the borrowing scheme, like (not heavily tested): diff -r 6902d5acd5e8 src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp --- a/src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp Mon Sep 30 10:19:03 2019 +0200 +++ b/src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp Mon Sep 30 15:30:52 2019 +0200 @@ -212,7 +212,12 @@ // No free registers available. Make one useful. tmp = rscratch1; + if (tmp == dst) { + tmp = rscratch2; + } __ push(RegSet::of(tmp), sp); } + assert_different_registers(tmp, dst); + Label done; __ ldr(tmp, Address(dst, oopDesc::mark_offset_in_bytes())); diff -r 6902d5acd5e8 src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp --- a/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp Mon Sep 30 10:19:03 2019 +0200 +++ b/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp Mon Sep 30 15:30:52 2019 +0200 @@ -272,7 +272,12 @@ // No free registers available. Make one useful. tmp = LP64_ONLY(rscratch1) NOT_LP64(rdx); + if (tmp == dst) { + tmp = LP64_ONLY(rscratch2) NOT_LP64(rcx); + } __ push(tmp); } + assert_different_registers(dst, tmp); + Label done; __ movptr(tmp, Address(dst, oopDesc::mark_offset_in_bytes())); -- Thanks, -Aleksey From rkennke at redhat.com Mon Sep 30 14:57:25 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 30 Sep 2019 16:57:25 +0200 Subject: RFR: JDK-8231583: Shenandoah: Fix register clash in SBSA::resolve_forwarding_pointer() borrowing In-Reply-To: <229ca151-528b-8f4e-031b-fa1bd84e3053@redhat.com> References: <996d7d26-4c18-62fd-ff57-76f1dddfd5c5@redhat.com> <229ca151-528b-8f4e-031b-fa1bd84e3053@redhat.com> Message-ID: <2f995643-c3dd-a1ec-09b5-8ab0c1d4c6fb@redhat.com> Hi Aleksey, >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8231583 >> Webrev: >> http://cr.openjdk.java.net/~rkennke/JDK-8231583/webrev.00/ > > Awww. > > Please change the synopsis to reflect what is being done, not the symptom of it. Suggestion: > Shenandoah: register clash in SBSA::resolve_forwarding_pointer borrowing Right. Changed to: Shenandoah: Fix register clash in SBSA::resolve_forwarding_pointer() borrowing > I believe we are better off just fixing the borrowing scheme, like (not heavily tested): > > diff -r 6902d5acd5e8 src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp > --- a/src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp Mon Sep 30 > 10:19:03 2019 +0200 > +++ b/src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp Mon Sep 30 > 15:30:52 2019 +0200 > @@ -212,7 +212,12 @@ > // No free registers available. Make one useful. > tmp = rscratch1; > + if (tmp == dst) { > + tmp = rscratch2; > + } > __ push(RegSet::of(tmp), sp); > } > > + assert_different_registers(tmp, dst); > + > Label done; > __ ldr(tmp, Address(dst, oopDesc::mark_offset_in_bytes())); > diff -r 6902d5acd5e8 src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp > --- a/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp Mon Sep 30 10:19:03 > 2019 +0200 > +++ b/src/hotspot/cpu/x86/gc/shenandoah/shenandoahBarrierSetAssembler_x86.cpp Mon Sep 30 15:30:52 > 2019 +0200 > @@ -272,7 +272,12 @@ > // No free registers available. Make one useful. > tmp = LP64_ONLY(rscratch1) NOT_LP64(rdx); > + if (tmp == dst) { > + tmp = LP64_ONLY(rscratch2) NOT_LP64(rcx); > + } > __ push(tmp); > } > > + assert_different_registers(dst, tmp); > + > Label done; > __ movptr(tmp, Address(dst, oopDesc::mark_offset_in_bytes())); > > Yeah. I took it a little further and removed the tmp argument to resolve_forwarding_pointer() altogether because we have no caller that passes anything interesting to it. http://cr.openjdk.java.net/~rkennke/JDK-8231583/webrev.01/ I believe aarch64 should be good (and should actually not require borrowing) because rscratch1 is not used by the register allocators there. Ok now? Roman From shade at redhat.com Mon Sep 30 15:33:30 2019 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 30 Sep 2019 17:33:30 +0200 Subject: RFR: JDK-8231583: Shenandoah: Fix register clash in SBSA::resolve_forwarding_pointer() borrowing In-Reply-To: <2f995643-c3dd-a1ec-09b5-8ab0c1d4c6fb@redhat.com> References: <996d7d26-4c18-62fd-ff57-76f1dddfd5c5@redhat.com> <229ca151-528b-8f4e-031b-fa1bd84e3053@redhat.com> <2f995643-c3dd-a1ec-09b5-8ab0c1d4c6fb@redhat.com> Message-ID: <43626b96-7c3d-5a1b-df9f-e3900ce2742a@redhat.com> On 9/30/19 4:57 PM, Roman Kennke wrote: > http://cr.openjdk.java.net/~rkennke/JDK-8231583/webrev.01/ I believe at this point we should be doing a very safe change that does not include collateral cleanups: http://cr.openjdk.java.net/~shade/8231583/webrev.01/ Testing: hotspot_gc_shenandoah {x86_32, x86_64, aarch64} -- Thanks, -Aleksey From rkennke at redhat.com Mon Sep 30 15:52:22 2019 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 30 Sep 2019 17:52:22 +0200 Subject: RFR: JDK-8231583: Shenandoah: Fix register clash in SBSA::resolve_forwarding_pointer() borrowing In-Reply-To: <43626b96-7c3d-5a1b-df9f-e3900ce2742a@redhat.com> References: <996d7d26-4c18-62fd-ff57-76f1dddfd5c5@redhat.com> <229ca151-528b-8f4e-031b-fa1bd84e3053@redhat.com> <2f995643-c3dd-a1ec-09b5-8ab0c1d4c6fb@redhat.com> <43626b96-7c3d-5a1b-df9f-e3900ce2742a@redhat.com> Message-ID: <07cdb113-bd3b-60c4-86a2-c008cd6b6fec@redhat.com> Yeah, looks good. I also tested this with the original failing testcase, and it was good. Thanks, Roman > On 9/30/19 4:57 PM, Roman Kennke wrote: >> http://cr.openjdk.java.net/~rkennke/JDK-8231583/webrev.01/ > > I believe at this point we should be doing a very safe change that does not include collateral cleanups: > http://cr.openjdk.java.net/~shade/8231583/webrev.01/ > > Testing: hotspot_gc_shenandoah {x86_32, x86_64, aarch64} >