From zgu at redhat.com Tue May 1 11:26:39 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 1 May 2018 07:26:39 -0400 Subject: RFR(XXS) 8201542: Remove unused _gc_timer field in GCMemoryManager In-Reply-To: <6126c73b-9166-d57c-b238-913da7240a1e@oracle.com> References: <186a0988-2eab-6734-24af-f1c8c2a8590f@redhat.com> <6126c73b-9166-d57c-b238-913da7240a1e@oracle.com> Message-ID: <4adcd2c7-c123-f658-5f3c-dda075620de0@redhat.com> Hi David, On 04/30/2018 05:35 PM, David Holmes wrote: > Hi Zhengyu, > > Your patch is based off the wrong repo - jdk/hs. Please update to jdk/jdk. > I rebased to jdk/jdk, the patch has no conflict. Thanks, -Zhengyu > Thanks, > David > > On 1/05/2018 1:32 AM, Zhengyu Gu wrote: >> Please review this trivial cleanup to remove unused _gc_timer in >> GCMemoryManager. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8201542 >> Webrev: http://cr.openjdk.java.net/~zgu/8201542/webrev.00/ >> >> Test: >> ?? hotspot_servicebility on Linux x64 (fastdebug and release) >> >> Thanks, >> >> -Zhengyu From zgu at redhat.com Tue May 1 11:27:07 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 1 May 2018 07:27:07 -0400 Subject: RFR(XXS) 8201542: Remove unused _gc_timer field in GCMemoryManager In-Reply-To: References: <186a0988-2eab-6734-24af-f1c8c2a8590f@redhat.com> Message-ID: Thanks, Yumin. -Zhengyu On 04/30/2018 04:30 PM, yumin qi wrote: > Looks good. > > Thanks > Yumin > > On Mon, Apr 30, 2018 at 8:32 AM, Zhengyu Gu > wrote: > > Please review this trivial cleanup to remove unused _gc_timer in > GCMemoryManager. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8201542 > > Webrev: http://cr.openjdk.java.net/~zgu/8201542/webrev.00/ > > > Test: > ? hotspot_servicebility on Linux x64 (fastdebug and release) > > Thanks, > > -Zhengyu > > From david.holmes at oracle.com Tue May 1 11:29:39 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 1 May 2018 21:29:39 +1000 Subject: RFR(XXS) 8201542: Remove unused _gc_timer field in GCMemoryManager In-Reply-To: <4adcd2c7-c123-f658-5f3c-dda075620de0@redhat.com> References: <186a0988-2eab-6734-24af-f1c8c2a8590f@redhat.com> <6126c73b-9166-d57c-b238-913da7240a1e@oracle.com> <4adcd2c7-c123-f658-5f3c-dda075620de0@redhat.com> Message-ID: <943a9bf7-200f-5b0e-7156-b413562113e9@oracle.com> Thanks Zhengyu. Looks fine. David On 1/05/2018 9:26 PM, Zhengyu Gu wrote: > Hi David, > > > > On 04/30/2018 05:35 PM, David Holmes wrote: >> Hi Zhengyu, >> >> Your patch is based off the wrong repo - jdk/hs. Please update to >> jdk/jdk. >> > > I rebased to jdk/jdk, the patch has no conflict. > > Thanks, > > -Zhengyu > >> Thanks, >> David >> >> On 1/05/2018 1:32 AM, Zhengyu Gu wrote: >>> Please review this trivial cleanup to remove unused _gc_timer in >>> GCMemoryManager. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8201542 >>> Webrev: http://cr.openjdk.java.net/~zgu/8201542/webrev.00/ >>> >>> Test: >>> ?? hotspot_servicebility on Linux x64 (fastdebug and release) >>> >>> Thanks, >>> >>> -Zhengyu From lois.foltan at oracle.com Tue May 1 11:33:27 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 1 May 2018 07:33:27 -0400 Subject: RFR(XXS) 8201542: Remove unused _gc_timer field in GCMemoryManager In-Reply-To: <4adcd2c7-c123-f658-5f3c-dda075620de0@redhat.com> References: <186a0988-2eab-6734-24af-f1c8c2a8590f@redhat.com> <6126c73b-9166-d57c-b238-913da7240a1e@oracle.com> <4adcd2c7-c123-f658-5f3c-dda075620de0@redhat.com> Message-ID: <8a03e471-18fb-c51b-d367-0500c7053c6b@oracle.com> Looks good Zhengyu. Lois On 5/1/2018 7:26 AM, Zhengyu Gu wrote: > Hi David, > > > > On 04/30/2018 05:35 PM, David Holmes wrote: >> Hi Zhengyu, >> >> Your patch is based off the wrong repo - jdk/hs. Please update to >> jdk/jdk. >> > > I rebased to jdk/jdk, the patch has no conflict. > > Thanks, > > -Zhengyu > >> Thanks, >> David >> >> On 1/05/2018 1:32 AM, Zhengyu Gu wrote: >>> Please review this trivial cleanup to remove unused _gc_timer in >>> GCMemoryManager. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8201542 >>> Webrev: http://cr.openjdk.java.net/~zgu/8201542/webrev.00/ >>> >>> Test: >>> ?? hotspot_servicebility on Linux x64 (fastdebug and release) >>> >>> Thanks, >>> >>> -Zhengyu From zgu at redhat.com Tue May 1 12:34:28 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 1 May 2018 08:34:28 -0400 Subject: RFR(XXS) 8201542: Remove unused _gc_timer field in GCMemoryManager In-Reply-To: <943a9bf7-200f-5b0e-7156-b413562113e9@oracle.com> References: <186a0988-2eab-6734-24af-f1c8c2a8590f@redhat.com> <6126c73b-9166-d57c-b238-913da7240a1e@oracle.com> <4adcd2c7-c123-f658-5f3c-dda075620de0@redhat.com> <943a9bf7-200f-5b0e-7156-b413562113e9@oracle.com> Message-ID: Hi David, Thanks. I pushed. -Zhengyu On 05/01/2018 07:29 AM, David Holmes wrote: > Thanks Zhengyu. > > Looks fine. > > David > > On 1/05/2018 9:26 PM, Zhengyu Gu wrote: >> Hi David, >> >> >> >> On 04/30/2018 05:35 PM, David Holmes wrote: >>> Hi Zhengyu, >>> >>> Your patch is based off the wrong repo - jdk/hs. Please update to >>> jdk/jdk. >>> >> >> I rebased to jdk/jdk, the patch has no conflict. >> >> Thanks, >> >> -Zhengyu >> >>> Thanks, >>> David >>> >>> On 1/05/2018 1:32 AM, Zhengyu Gu wrote: >>>> Please review this trivial cleanup to remove unused _gc_timer in >>>> GCMemoryManager. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8201542 >>>> Webrev: http://cr.openjdk.java.net/~zgu/8201542/webrev.00/ >>>> >>>> Test: >>>> ?? hotspot_servicebility on Linux x64 (fastdebug and release) >>>> >>>> Thanks, >>>> >>>> -Zhengyu From zgu at redhat.com Tue May 1 12:35:27 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 1 May 2018 08:35:27 -0400 Subject: RFR(XXS) 8201542: Remove unused _gc_timer field in GCMemoryManager In-Reply-To: <8a03e471-18fb-c51b-d367-0500c7053c6b@oracle.com> References: <186a0988-2eab-6734-24af-f1c8c2a8590f@redhat.com> <6126c73b-9166-d57c-b238-913da7240a1e@oracle.com> <4adcd2c7-c123-f658-5f3c-dda075620de0@redhat.com> <8a03e471-18fb-c51b-d367-0500c7053c6b@oracle.com> Message-ID: Hi Lois, Thanks for the review. I pushed before seeing this, sorry. -Zhengyu On 05/01/2018 07:33 AM, Lois Foltan wrote: > Looks good Zhengyu. > Lois > > On 5/1/2018 7:26 AM, Zhengyu Gu wrote: >> Hi David, >> >> >> >> On 04/30/2018 05:35 PM, David Holmes wrote: >>> Hi Zhengyu, >>> >>> Your patch is based off the wrong repo - jdk/hs. Please update to >>> jdk/jdk. >>> >> >> I rebased to jdk/jdk, the patch has no conflict. >> >> Thanks, >> >> -Zhengyu >> >>> Thanks, >>> David >>> >>> On 1/05/2018 1:32 AM, Zhengyu Gu wrote: >>>> Please review this trivial cleanup to remove unused _gc_timer in >>>> GCMemoryManager. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8201542 >>>> Webrev: http://cr.openjdk.java.net/~zgu/8201542/webrev.00/ >>>> >>>> Test: >>>> ?? hotspot_servicebility on Linux x64 (fastdebug and release) >>>> >>>> Thanks, >>>> >>>> -Zhengyu > From erik.osterlund at oracle.com Tue May 1 13:32:03 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 1 May 2018 15:32:03 +0200 Subject: 8202377: Modularize C2 GC barriers Message-ID: <5AE86C53.4070708@oracle.com> Hi, The GC barriers for C2 are not as modular as they could be. It currently uses switch statements to check which GC barrier set is being used, and call one or another barrier based on that, in a way that it can only be used for write barriers. My proposed solution is to follow the same pattern that has been used by C1 (and the rest of HotSpot), which is to provide a GC barrier set code generation helper for C2. Its name is BarrierSetC2. Each barrier set class has its own BarrierSetC2, following a mirrored inheritance hierarchy to the BarrierSet hierarchy. You generate the accesses using some access_* member functions on GraphKit, which calls into BarrierSetC2. A lot of the design looks very similar to BarrierSetC1. In C1, there was a wrapper object called LIRAccess that wrapped a bunch of context parameters that were passed around in the barrier set hierarchy. There is a similar wrapper for C2 that I call C2Access. Users of the API do not see it. They call, e.g. access_load_at, in GraphKit during parsing. The access functions wrap the access in a C2Access object with a bunch of context parameters, and calls the currently selected BarrierSetC2 backend accessor with this context. For the atomic accesses, there is a C2AtomicAccess, inheriting from C2Access. It contains more context, as required by the atomic accesses (e.g. explicit alias_idx, whether the node needs pinning with an SCM projection, and a memory node). Apart from the normal shared decorators, C2 does use its own additional decorators for its own use: * C2_MISMATCHED and C2_UNALIGNED (describing properties of unsafe accesses) * C2_WEAK_CMPXCHG: describing if a cmpxchg may have false negatives * C2_CONTROL_DEPENDENT_LOAD: use when a load should have control dependency * C2_PINNED_LOAD: use for loads that must be pinned * C2_UNSAFE_ACCESS: Used to recognize this is an unsafe access. This decorator implies that loads have control dependency and need pinning, unless it can be proven that the access will be inside the bounds of an object. * C2_READ_ACCESS and C2_WRITE_ACCESS: This denotes whether the access reads or writes to memory. Or both for atomics. It is useful for for figuring out what fencing is required for a given access and ordering semantics, as well as being useful for Shenandoah to figure out what type of barrier to use to ensure memory consistency. The accesses go through a similar process as they do in C1. Let's take BarrierSetC2::store_at for example. It uses the the C2AccessFence scoped object helper to figure out what membars are required to surround the access, resolve the address (no-op for all GCs with a to-space invariant, which is all GCs except Shenandoah in HotSpot at the moment), and then calls store_at_resolved. The store_at_resolved member function generates the access and the barriers around it. The abstract ModRefBarrierSetC2 barrier set introduces the notion of pre/post write barriers, and lets concrete barrier sets do sprinkle their GC barriers in there. It calls BarrierSetC2::store_at_resolved to generate the actual access. For example CardTableBarrierSet only needs to override its post barrier for this to work as expected. The other accesses follow a similar pattern. The Compile class now has a type erase (void*) per compilation unit state that is created for each compilation unit (with BarrierSetC2::create_barrier_state). For the GCs in HotSpot today, this is always NULL. But for GCs that have their own macro nodes, the compilation unit can be used for, e.g. lists of barrier-specific macro nodes, that should not pollute the Compile object. Such macro nodes can be expanded during macro expansion using the BarrierSetC2::expand_macro_nodes member function. There are a few other helpers that may be good for a GC to have, like figuring out if a node is a GC barrier (for escape analysis), whether a GC barrier can be eliminated (for example using ReduceInitialCardMarks), whether array_copy requires GC barriers, how to step over a GC barrier. There is also a helper for loop optimizing GC barrier nodes. This work will help to pave way for a new class of collectors utilizing load barriers (ZGC and Shenandoah) for concurrent compaction. Webrev: http://cr.openjdk.java.net/~eosterlund/8202377/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8202377 Thanks, /Erik From erik.osterlund at oracle.com Tue May 1 15:00:40 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 1 May 2018 17:00:40 +0200 Subject: RFR: 8202479: Add missing try_resolve_jobject_in_native calls Message-ID: <5AE88118.7080203@oracle.com> Hi, There are some missing calls to try_resolve_jobject_in_native for the jni fast get field optimization. On x86_64, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT and T_LONG, but is missing for T_FLOAT and T_DOUBLE. On SPARC, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT, but is missing for T_LONG, T_FLOAT and T_DOUBLE. On AArch64, it is used for all types. Here is a patch to add the missing calls to try_resolve_jobject_in_native. Webrev: http://cr.openjdk.java.net/~eosterlund/8202479/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8202479 Thanks, /Erik From shade at redhat.com Tue May 1 16:39:38 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 1 May 2018 18:39:38 +0200 Subject: RFR 8202379: ARM32 is broken after JDK-8201543 (Modularize C1 GC barriers) In-Reply-To: <1914516d-ec4b-1c45-194a-3107371fb1f9@redhat.com> References: <2f5d4f30-dfb5-4959-c032-e27d747ee07e@redhat.com> <1914516d-ec4b-1c45-194a-3107371fb1f9@redhat.com> Message-ID: <4b31ffb0-b5ee-4820-23f8-6826a68240b2@redhat.com> On 04/30/2018 11:09 AM, Andrew Haley wrote: > On 04/30/2018 10:05 AM, Aleksey Shipilev wrote: >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8202379 >> >> Fix: >> http://cr.openjdk.java.net/~shade/8202379/webrev.01/ >> >> Testing: arm32 build, Raspi 3 runs (some selected jcstress tests) > > Looks OK, thanks. Thanks for review! Anyone else? -Aleksey From erik.osterlund at oracle.com Tue May 1 16:45:14 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Tue, 1 May 2018 18:45:14 +0200 Subject: RFR 8202379: ARM32 is broken after JDK-8201543 (Modularize C1 GC barriers) In-Reply-To: <4b31ffb0-b5ee-4820-23f8-6826a68240b2@redhat.com> References: <2f5d4f30-dfb5-4959-c032-e27d747ee07e@redhat.com> <1914516d-ec4b-1c45-194a-3107371fb1f9@redhat.com> <4b31ffb0-b5ee-4820-23f8-6826a68240b2@redhat.com> Message-ID: <478D9BFD-2F34-46EF-980D-1EC281F91A0E@oracle.com> Hi Aleksey, Looks good. Thanks, /Erik > On 1 May 2018, at 18:39, Aleksey Shipilev wrote: > >> On 04/30/2018 11:09 AM, Andrew Haley wrote: >>> On 04/30/2018 10:05 AM, Aleksey Shipilev wrote: >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8202379 >>> >>> Fix: >>> http://cr.openjdk.java.net/~shade/8202379/webrev.01/ >>> >>> Testing: arm32 build, Raspi 3 runs (some selected jcstress tests) >> >> Looks OK, thanks. > > Thanks for review! Anyone else? > > -Aleksey > > > From shade at redhat.com Tue May 1 16:48:48 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 1 May 2018 18:48:48 +0200 Subject: RFR 8202379: ARM32 is broken after JDK-8201543 (Modularize C1 GC barriers) In-Reply-To: <478D9BFD-2F34-46EF-980D-1EC281F91A0E@oracle.com> References: <2f5d4f30-dfb5-4959-c032-e27d747ee07e@redhat.com> <1914516d-ec4b-1c45-194a-3107371fb1f9@redhat.com> <4b31ffb0-b5ee-4820-23f8-6826a68240b2@redhat.com> <478D9BFD-2F34-46EF-980D-1EC281F91A0E@oracle.com> Message-ID: <3f7d6a68-c079-aa3d-3938-bf18cce76014@redhat.com> Thanks Erik! -Aleksey P.S. I think you also wanted to review 8201786. On 05/01/2018 06:45 PM, Erik Osterlund wrote: > Hi Aleksey, > > Looks good. > > Thanks, > /Erik > >> On 1 May 2018, at 18:39, Aleksey Shipilev wrote: >> >>> On 04/30/2018 11:09 AM, Andrew Haley wrote: >>>> On 04/30/2018 10:05 AM, Aleksey Shipilev wrote: >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8202379 >>>> >>>> Fix: >>>> http://cr.openjdk.java.net/~shade/8202379/webrev.01/ >>>> >>>> Testing: arm32 build, Raspi 3 runs (some selected jcstress tests) >>> >>> Looks OK, thanks. >> >> Thanks for review! Anyone else? >> >> -Aleksey >> >> >> From erik.osterlund at oracle.com Tue May 1 17:10:38 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Tue, 1 May 2018 19:10:38 +0200 Subject: RFR 8201786: Modularize interpreter GC barriers: leftovers for ARM32 In-Reply-To: <828a25d9-d4b1-ec05-71e3-ceddef4b7a24@redhat.com> References: <4d22aacb-aa0d-4423-1a22-775e2be7eeaf@redhat.com> <1524847169.9114.6.camel@gmail.com> <668e8a59-cba1-a2ce-48a0-cf9cfcde032b@redhat.com> <1524866512.9114.10.camel@gmail.com> <828a25d9-d4b1-ec05-71e3-ceddef4b7a24@redhat.com> Message-ID: <89150A82-E74C-4046-ABF8-80AAA0C86744@oracle.com> Hi Aleksey, 1) The jobject resolve grabs the BarrierSetAssembler and calls straight into it, instead of making an access_load_at call. I would prefer using the access call. 2) The macro assembler has a bunch of G1 includes left. I hope they can be removed now. 3) The CardTableBarrierSet checks for CMS flags instead of checking whether the card table is scanned concurrently or not. 4) The template interpreter generator Reference.get intrinsic is missing an ON_WEAK_OOP_REF decorator, which is required to get the right G1 barriers for example. 5) In the template table, all do_oop_load/store accesses should be IN_HEAP. They are missing that decorator. That will cause those accesses to skip pre/post write barriers. 6) I don?t know if you want to venture into the shady domain of jni fast get field. If you do, the implicit jweak resolve in there on ARM should call BarrierSetAssembler::try_resolve_jobject_in_native. It currently clears the jweak tag in r1, which is used by the calling concention. Therefore, if the speculative load fails, then the runtime will be called as a fallback, except now thinking it is a jobject and not jweak, causing the runtime to use inappropriate barriers. I filed https://bugs.openjdk.java.net/browse/JDK-8202480 for that. So might be a different RFR I guess. Otherwise this looks good. Thanks a lot for contributing this. /Erik > On 30 Apr 2018, at 11:34, Aleksey Shipilev wrote: > > On 04/28/2018 12:01 AM, Edward Nevill wrote: >>>>> http://cr.openjdk.java.net/~shade/8201786/webrev.01/ > >> So, all we can say is that it is no worse after your patch. > > Good! > >> On 04/27/2018 08:49 PM, Boris Ulasevich wrote: >> I have checked your change with [JCK vm+lang subset] X [G1|SerialGC] on Raspi 2 - Passed Ok. > > Thanks for testing, Edward and Boris! > > Rebased the patch for current jdk/jdk (trivial header definition clash resolved): > http://cr.openjdk.java.net/~shade/8201786/webrev.02/ > > I need JDK-8202379 to go in first, because current arm32 is broken: > http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031909.html > > Thanks, > -Aleksey > From kim.barrett at oracle.com Tue May 1 17:59:01 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 1 May 2018 13:59:01 -0400 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> Message-ID: > On Apr 27, 2018, at 4:26 PM, Michal Vala wrote: >>> For now, proposed patch looks like this: >>> >>> --- old/src/hotspot/os/linux/os_linux.inline.hpp 2018-04-20 09:16:34.498343185 +0200 >>> +++ new/src/hotspot/os/linux/os_linux.inline.hpp 2018-04-20 09:16:34.214342670 +0200 >>> @@ -98,26 +98,8 @@ >>> >>> inline struct dirent* os::readdir(DIR* dirp, dirent *dbuf) >>> { >>> -// readdir_r has been deprecated since glibc 2.24. >>> -// See https://sourceware.org/bugzilla/show_bug.cgi?id=19056 for more details. >>> -#pragma GCC diagnostic push >>> -#pragma GCC diagnostic ignored "-Wdeprecated-declarations" >>> - >>> - dirent* p; >>> - int status; >>> assert(dirp != NULL, "just checking"); >>> - >>> - // NOTE: Linux readdir_r (on RH 6.2 and 7.2 at least) is NOT like the POSIX >>> - // version. Here is the doc for this function: >>> - // http://www.gnu.org/manual/glibc-2.2.3/html_node/libc_262.html >>> - >>> - if((status = ::readdir_r(dirp, dbuf, &p)) != 0) { >>> - errno = status; >>> - return NULL; >>> - } else >>> - return p; >>> - >>> -#pragma GCC diagnostic pop >>> + return ::readdir(dirp); >>> } >>> >>> inline int os::closedir(DIR *dirp) { >> This looks good. > > Thanks! > Someone to sponsor this please? Do you have a sponsor yet? If not, I?ll do it. From coleen.phillimore at oracle.com Tue May 1 19:24:49 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 1 May 2018 15:24:49 -0400 Subject: RFR (S) 8202447: Fix unloading_occurred to mean unloading_occurred Message-ID: <58b3d0ec-fc8c-cf01-06ce-f652a7f7fc3f@oracle.com> Summary: nmethod unloading does not need to test for jvmti to set unloading_occurred, nor do we need to clean weak Klasses in metadata if unloading does not occur. open webrev at http://cr.openjdk.java.net/~coleenp/8202447.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8202447 Tested with mach5 hs-tier1-4, rerunning hs-tier3.? Kitchensink 24 hours, and runThese with all 4 GCs. Thanks, Coleen From kim.barrett at oracle.com Tue May 1 21:13:35 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 1 May 2018 17:13:35 -0400 Subject: RFR: 8200557: OopStorage parallel iteration scales poorly In-Reply-To: References: <7CBB6FF2-AC36-4723-BBA1-B4FA277C707B@oracle.com> Message-ID: <2C30CDA1-7070-49A3-9D40-8625EDBA582E@oracle.com> > On Apr 25, 2018, at 5:30 PM, coleen.phillimore at oracle.com wrote: > > > http://cr.openjdk.java.net/~kbarrett/8200557/open.00/src/hotspot/share/gc/shared/oopStorage.cpp.udiff.html > > +OopStorage::BasicParState::IterationData::IterationData() : > + _segment_start(0), > + _segment_end(0), > + _processed(0) > +{} > > > Can this constructor be inlined in the declaration? This seems out of place here (and I had to hunt for it). I'm changing IterationData to be a simple POD-struct. It's very limited usage scope makes more than that overkill. > Can some other simple accessors be inlined in the declarations as well? That would decrease jumping around trying to understand things. > > I think the OopStorage::BasicParState functions should be in a new .cpp file since there's now *src/hotspot/share/gc/shared/oopStorageParState.hpp > *Can you do this as a follow up/cleanup RFE? I would rather not. The headers were split up to reduce include dependencies. That's not so much of an issue for oopStorage.cpp. And the implementation of OopStorage::BasicParState is pretty intimately tied to the rest of OopStorage. > http://cr.openjdk.java.net/~kbarrett/8200557/open.00/src/hotspot/share/gc/shared/oopStorage.hpp.udiff.html > > class Block; // Forward decl; defined in .inline.hpp file. > + class BlockArray; // Forward decl; defined in .inline.hpp file. > class BlockList; // Forward decl for BlockEntry friend decl. > > Can you give a short description of these, ie: > > class Block; // Block of oop* stored in OopStorage > + class BlockArray; // Storage for all active oop Blocks > class BlockList; // List of oop Blocks from which are available for allocation > > Or maybe a short description over the class declarations? The trouble with these generic names is you forget why they exist. I've added some/more descriptions, and moved some to hopefully be better placed to be found. > I'd call BlockList a AllocationBlockList and BlockArray an ActiveBlockArray For naming, I think BlockArray => ActiveArray, BlockList => AllocateList, BlockEntry => AllocateListEntry. Recall that before this change there wasn't an active *array*. rather there were two BlockLists, active and allocate, so the generic name made more sense then. I'll do this as a followup RFE; there's also a simplification to be made to BlockList, now that there's only one, which I was planning to do as a followup. > http://cr.openjdk.java.net/~kbarrett/8200557/open.00/src/hotspot/share/gc/shared/oopStorage.inline.hpp.frames.html > > 38 class OopStorage::BlockArray { > > > This would be a good place to comment the purpose of this and a line about refcounting. > > http://cr.openjdk.java.net/~kbarrett/8200557/open.00/src/hotspot/share/gc/shared/oopStorage.hpp.frames.html > > Why does BlockList need to be included in the .hpp file? BlockList needs to be in the .hpp file because there is a member in OopStorage, requiring it to be complete. Block and Block can be forward declared. But I now realize that BlockEntry could also be forward-declared and moved to the .inline.hpp file. > http://cr.openjdk.java.net/~kbarrett/8200557/open.00/src/hotspot/share/gc/shared/oopStorage.cpp.frames.html > > All these places have block_count() which are indistinguishable. It looks like _block_count for the BlockArray should always be acquired outside of BlockArray class or rather give that as the only public option to prevent mistakes. It seems like there wouldn't be much performance cost to calling load_acquire once at the safepoint iteration. block_count() (and _block_count) are used in places where we know we're in a safepoint or that we hold the _allocate_mutex. block_count_acquire() is used where neither of those is true. Responding to Erik's questions about block_count() vs block_count_acquire(), OopStorage::block_count() and OopStorage::total_memory_usage() are in the latter case. I've waffled over whether it is better to use an unnecessary acquire or possibly need to explain why it isn't necessary. Of course, then one may need to explain an apparently unnecessary acquire. After looking over things in this area yet again, I think there are some places where it's better to directly use the _block_count member rather than an accessor function anyway, so I'm doing that. And an unnecessary acquire seems like it needs as much explaination as not having it, so removing those too. > Comment about _why_ refcounts and what do they do here would be nice. > > 151 void OopStorage::BlockArray::increment_refcount() const { > > > 567 void OopStorage::relinquish_block_array(BlockArray* array) const { > 568 if (array->decrement_refcount()) { > 569 assert(array != _active_array, "invariant"); > 570 BlockArray::destroy(array); > 571 } > 572 } > > > Here array is a pointer to the active array so iterators could destroy the array if they are the last ones to use it. Is this right? Yes. > I've read through most of the rest of the code and it looks good apart from how complicated it is, which I don't know how to suggest to improve it other than some reminder comments and maybe more descriptive names. New webrevs: full: http://cr.openjdk.java.net/~kbarrett/8200557/open.00/ incr: http://cr.openjdk.java.net/~kbarrett/8200557/open.01.inc/ From kim.barrett at oracle.com Tue May 1 21:15:13 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 1 May 2018 17:15:13 -0400 Subject: RFR: 8200557: OopStorage parallel iteration scales poorly In-Reply-To: <5AE1D5DF.90909@oracle.com> References: <7CBB6FF2-AC36-4723-BBA1-B4FA277C707B@oracle.com> <5AE1D5DF.90909@oracle.com> Message-ID: > On Apr 26, 2018, at 9:36 AM, Erik ?sterlund wrote: > > Hi Kim, > > FIrst off, some high level comments: > > 1) I would have simply grabbed the allocation mutex when incrementing the reference counter when starting concurrent iteration, instead of using RCU. The path where we grab this active list for concurrent iteration is stone cold, so I don't expect there to be any observable benefit in using RCU here. But I am going to let that pass and just note we think differently about things here, and I think that is okay. A very early version of OopStorage had locking between allocation and concurrent iteration; that was deemed problematic, since an application (via allocate calls) could delay a concurrent GC phase. > 2) It is a bit unfortunate that going down the RCU path led to yet another implementation of RCU because GlobalCounter can not yet satisfy your scenario. But I think that you have done a good job making the RCU implementation pluggable in OopStorage, and we can easily remove it once the ThreadsListHandle required by GlobalCounter starts supporting lock-free nesting soon. I am speculating that your approach could be folded in to the GlobalCounter, to accomodate non-JavaThreads and non-VM threads, which that solution currently does not support. But perhaps that is a discussion for later. I expect GlobalCounter to be able to meet the needs of this scenario eventually. It doesn't at present, but that limitation is being worked on. Once that's done, it should be easy to switch and drop the private mechanism. (While there are some differences that might make this alternative to GlobalCounter interesting to pursue further, the don't matter for the present OopStorage use-case.) > Low level comments: > * Noting that the new RCU mechanism will like the last one not work on PPC until there is an atomic counter increment with more conservative memory ordering. But I won't blame you for this. > > In oopStorage.cpp: > > 879 size_t OopStorage::block_count() const { > 880 WithActiveArray wab(this); > 881 return wab.active_array().block_count_acquire(); > 882 } > > Why do you need acquire here? I don't see subsequent loads into the elements of the array, which seems to be what the paired release_store when writing the counter protects against. > > 884 size_t OopStorage::total_memory_usage() const { > 885 size_t total_size = sizeof(OopStorage); > 886 total_size += strlen(name()) + 1; > 887 total_size += sizeof(BlockArray); > 888 WithActiveArray wab(this); > 889 const BlockArray& blocks = wab.active_array(); > 890 total_size += blocks.block_count_acquire() * Block::allocation_size(); > 891 total_size += blocks.size() * sizeof(Block*); > 892 return total_size; > 893 } > > Same as above: what reordering is the block_count_acquire protecting against? No element in the array is read after the acquire in program order, that I can see. See response to Coleen. > 760 _name(dup_name(name)), > 761 _active_array(BlockArray::create(initial_active_array_size)), > 762 _allocate_list(&Block::get_allocate_entry), > 763 _deferred_updates(NULL), > 764 _allocate_mutex(allocate_mutex), > 765 _active_mutex(active_mutex), > 766 _allocation_count(0), > 767 _concurrent_iteration_active(false) > 768 { > 769 _active_array->increment_refcount(); > > I wonder if you could make BlockArray::create() always return an already retained block array, instead of updating it after. > > 496 BlockArray* new_array = BlockArray::create(new_size); > 497 if (new_array == NULL) return false; > 498 new_array->copy_from(old_array); > 499 replace_active_array(new_array); > > And same here where a block array is created without retain reference counter, which is then incremented manually inside of replace_active_array. Seems to me like it would be easier to make BlockArray::create return already retained block arrays. My experience with using refcounts has been that starting with an implicit reference at construction time (e.g. initialize the refcount to 1 at construction) is more often than not really not helpful. However, this comment led me to notice the constructor for OopStorage should be checking the result of BlockArray::create. From karen.kinnear at oracle.com Tue May 1 22:08:04 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Tue, 1 May 2018 18:08:04 -0400 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: References: <5AE0612F.8060200@oracle.com> <9a5f2a18-f7d4-d1ad-6618-b92bb7d71dd9@samersoff.net> Message-ID: <3B7915CA-50D7-4303-8C7B-C17D5911BC29@oracle.com> Lots of great work here. Review part 1 below. I will not be reviewing build files or test files - I believe you have other coverage for those. I did this quickly, so if any of my comments don?t make sense after detailed study - just let me know. I owe you a review of the share/jfr files still. A couple of issues that I think should be addressed before checking in the code: 1. New logTag.hpp log tags One is called ?setting? -> could you make this less general please? like jfrsetting? Or this a subcategory so we already know we are under ?jfr?? Note: I would expect that log tags would be added to the CSR since that is a supported interface 2. Code removed two command line flags: EnableTracing and UseLockedTracing with product flags, the normal approach is to deprecate given they now do nothing, I would strongly recommend that you Obsolete before removing i.e. accept the flags, but do not do anything for 1 release, to allow script migration. Expire the flags in the next release. So test that you get a warning if you use them for JDK 11. Note: CSR says EnableTracing flag is removed -> please update CSR for both to be made obsolete 3. CSR question - did jcmd change at all? If it did I would expect that to also be covered in the CSR, but I think it did not. A couple of issues that should be addressed before JDK11 ships 1. FastTime naming - we need to not present an attractive nuisance here. Please rename to clarify at least: is_platform_trusted_for_fast_time() -> ?platform_provides_fast_unordered_time? or something equally clear about risk in the name - there is a local with a similar need to change name FastTime* APIs: e.g. FastTimeElapsedCounterSource -> please include Unordered in the name For folks new to this issue - the rdtsc (read time stamp counter) instruction can return different times across different sockets, so time can go backwards, so we do not use this for any vm or java times, just for diagnostic timestamps and we highly recommend that folks modifying the jvm do not start using it accidentally. 2. Docker container compatibility Can you please check with Bob Vandette e.g. for cgroup handling - the new os_perf_linux.cpp may have some overlap/need modifications simliar to what he did with os_linux.cpp - e.g. for number of cpus, sockets, etc. And please test with a container - Misha can give guidance here. Other comments: 1. #INCLUDE_JFR Did you build with #INCLUDE_JFR not defined? Is the theory that we still may want a minimalVM without JFR? The model is not clear to me in terms of when you include files or generate events e.g. c1_GraphBuilder.cpp, systemDictionary.cpp, synchronizer.cpp, ? Could this please be on your future clean-up list? p.s. I do like the extraction of the post_xxx_event logic 2. Duplication Is there any duplication in the VM_Version and VM_Version_ext handling? And are you covering older CPUs we no longer need to cover? Vladimir K would know the precise list. 3. objectCountEventSender.hpp: comment to clean up: + // The following two functions have the exact same signature as + // hotspot/src/share/vm/gc_implementation/shared/objectCountEventSender.hpp + // + // This file will replace the open file if a closed build is performed. + // These function signatures can therefore not be changed if the open + // signatures aren't changed as well. 4. ObjectMonitor::enter Why was this event modified differently from others - i.e. it stores the stack trace in the _owner field? And is there a reason we flush here? 5. jfrMacros.hpp I don't see any users of JFR_FLUSH? 6. vmStructs.cpp removed all the VM_XXX_TRACE which was conditional. Did SA never use it? Or will that need updating? 7. Assuming jdk.management.jfr and jdk.jfr changes were just name/include locations/copyrights but nothing substantial - I did a quick skim thanks, Karen > On Apr 28, 2018, at 4:17 AM, Erik Gahlin wrote: > > Yes, If we don?t include JFR, we should get the stubs > > Will fix. > > Erik > >> On 28 Apr 2018, at 09:59, Dmitry Samersoff wrote: >> >> Erik, >> >> globals.cpp:1051 these changes seems to be unnecessary. >> >> -Dmitry >> >> On 25.04.2018 14:06, Erik Gahlin wrote: >>> Greetings, >>> >>> Could I have a review of 8199712: Flight Recorder >>> >>> As mentioned in the preview [1] the tracing backend has been removed. >>> Event metadata has been consolidated into a single XML file and event >>> classes are now generated by GenerateJfrFiles.java. >>> >>> Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. >>> >>> For details about the feature, see the JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8193393 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~egahlin/8199712.0/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8199712 >>> >>> [1] >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359.html >>> >>> Thanks >>> Erik and Markus >> >> >> -- >> Dmitry Samersoff >> http://devnull.samersoff.net >> * There will come soft rains ... >> > From serguei.spitsyn at oracle.com Tue May 1 22:10:49 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 1 May 2018 15:10:49 -0700 Subject: RFR (S) 8202447: Fix unloading_occurred to mean unloading_occurred In-Reply-To: <58b3d0ec-fc8c-cf01-06ce-f652a7f7fc3f@oracle.com> References: <58b3d0ec-fc8c-cf01-06ce-f652a7f7fc3f@oracle.com> Message-ID: Hi Coleen, The fix looks good to me. Thanks, Serguei On 5/1/18 12:24, coleen.phillimore at oracle.com wrote: > Summary: nmethod unloading does not need to test for jvmti to set > unloading_occurred, nor do we need to clean weak Klasses in metadata > if unloading does not occur. > > open webrev at http://cr.openjdk.java.net/~coleenp/8202447.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8202447 > > Tested with mach5 hs-tier1-4, rerunning hs-tier3.? Kitchensink 24 > hours, and runThese with all 4 GCs. > > Thanks, > Coleen From coleen.phillimore at oracle.com Tue May 1 22:35:51 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 1 May 2018 18:35:51 -0400 Subject: RFR (S) 8202447: Fix unloading_occurred to mean unloading_occurred In-Reply-To: References: <58b3d0ec-fc8c-cf01-06ce-f652a7f7fc3f@oracle.com> Message-ID: <38605790-f755-dbcc-7c5e-35ee24346e3f@oracle.com> Thank you, Serguei.? I'm glad you reviewed this! Coleen On 5/1/18 6:10 PM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > The fix looks good to me. > > Thanks, > Serguei > > > On 5/1/18 12:24, coleen.phillimore at oracle.com wrote: >> Summary: nmethod unloading does not need to test for jvmti to set >> unloading_occurred, nor do we need to clean weak Klasses in metadata >> if unloading does not occur. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8202447.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8202447 >> >> Tested with mach5 hs-tier1-4, rerunning hs-tier3.? Kitchensink 24 >> hours, and runThese with all 4 GCs. >> >> Thanks, >> Coleen > From serguei.spitsyn at oracle.com Tue May 1 22:41:36 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Tue, 1 May 2018 15:41:36 -0700 Subject: RFR (S) 8202447: Fix unloading_occurred to mean unloading_occurred In-Reply-To: <38605790-f755-dbcc-7c5e-35ee24346e3f@oracle.com> References: <58b3d0ec-fc8c-cf01-06ce-f652a7f7fc3f@oracle.com> <38605790-f755-dbcc-7c5e-35ee24346e3f@oracle.com> Message-ID: <8ca7bb2d-b471-b13a-5e92-fa8e62647452@oracle.com> I'm glad too. :) Have not reviewed you fixes for a while. Thanks, Serguei On 5/1/18 15:35, coleen.phillimore at oracle.com wrote: > > Thank you, Serguei.? I'm glad you reviewed this! > Coleen > > On 5/1/18 6:10 PM, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> The fix looks good to me. >> >> Thanks, >> Serguei >> >> >> On 5/1/18 12:24, coleen.phillimore at oracle.com wrote: >>> Summary: nmethod unloading does not need to test for jvmti to set >>> unloading_occurred, nor do we need to clean weak Klasses in metadata >>> if unloading does not occur. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8202447.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8202447 >>> >>> Tested with mach5 hs-tier1-4, rerunning hs-tier3.? Kitchensink 24 >>> hours, and runThese with all 4 GCs. >>> >>> Thanks, >>> Coleen >> > From vladimir.kozlov at oracle.com Tue May 1 23:37:19 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 1 May 2018 16:37:19 -0700 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: <5AE0612F.8060200@oracle.com> References: <5AE0612F.8060200@oracle.com> Message-ID: Hi Erik, I am working on 8184349 and adding & !vm.graal.enabled to @requires for tests which use CMS. Mostly they are JFR tests which are in this review list. And I found some test have incorrect commands (merged 2 lines together): * @test @requires vm.gc == "null" | vm.gc == "Serial" I see that you fixed some. But there few left: test/jdk/jdk/jfr/event/gc/detailed/TestStressAllocationGCEventsWithDefNew.java test/jdk/jdk/jfr/event/gc/detailed/TestStressAllocationGCEventsWithG1.java test/jdk/jdk/jfr/event/gc/detailed/TestStressBigAllocationGCEventsWithParallel.java Thanks, Vladimir On 4/25/18 4:06 AM, Erik Gahlin wrote: > Greetings, > > Could I have a review of 8199712: Flight Recorder > > As mentioned in the preview [1] the tracing backend has been removed. > Event metadata has been consolidated into a single XML file and event > classes are now generated by GenerateJfrFiles.java. > > Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. > > For details about the feature, see the JEP: > https://bugs.openjdk.java.net/browse/JDK-8193393 > > Webrev: > http://cr.openjdk.java.net/~egahlin/8199712.0/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8199712 > > [1] > http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359.html > > Thanks > Erik and Markus From vladimir.kozlov at oracle.com Wed May 2 00:29:02 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 1 May 2018 17:29:02 -0700 Subject: [11] RFR(XS) 8202505) ctw2 tasks are timing out in hs-tier3 Message-ID: <12b18b61-bedc-1b1c-79a7-d0b336c8418d@oracle.com> http://java.se.oracle.com:10065/mdash/jobs/vkozlov-8202505_00-20180501-2323-20687 When CompileTheWorld does compilation classes are loaded and may be initialized. Most are side effect free. Except AWT classes which try to interact with system and may hang if classes initializers are not executed in expected order. We had this problem CTW + AWT long before. Suggested fix is not to run CTW java_desktop.java tests on Windows: diff -r 2ace90aec488 test/hotspot/jtreg/applications/ctw/modules/java_desktop.java --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java @@ -25,6 +25,9 @@ * @test * @summary run CTW for some classes from java.desktop module * + * @comment On Windows test hangs when it try to initialize AWT classes. + * @requires os.family != "windows" + * * @library /test/lib / /testlibrary/ctw/src * @modules java.base/jdk.internal.jimage * java.base/jdk.internal.misc diff -r 2ace90aec488 test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java @@ -25,6 +25,9 @@ * @test * @summary run CTW for some classes from java.desktop module * + * @comment On Windows test hangs when it try to initialize AWT classes. + * @requires os.family != "windows" + * * @library /test/lib / /testlibrary/ctw/src * @modules java.base/jdk.internal.jimage * java.base/jdk.internal.misc -- Thanks, Vladimir From igor.ignatyev at oracle.com Wed May 2 01:11:56 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 1 May 2018 18:11:56 -0700 Subject: [11] RFR(XS) 8202505) ctw2 tasks are timing out in hs-tier3 In-Reply-To: <12b18b61-bedc-1b1c-79a7-d0b336c8418d@oracle.com> References: <12b18b61-bedc-1b1c-79a7-d0b336c8418d@oracle.com> Message-ID: <19E479CD-B427-47BF-A533-01EF3E5841BE@oracle.com> I'd prefer to problem-list java_desktop tests on windows instead of using '@requires'. Thanks, -- Igor > On May 1, 2018, at 5:29 PM, Vladimir Kozlov wrote: > > http://java.se.oracle.com:10065/mdash/jobs/vkozlov-8202505_00-20180501-2323-20687 > > When CompileTheWorld does compilation classes are loaded and may be initialized. Most are side effect free. Except AWT classes which try to interact with system and may hang if classes initializers are not executed in expected order. We had this problem CTW + AWT long before. > > Suggested fix is not to run CTW java_desktop.java tests on Windows: > > diff -r 2ace90aec488 test/hotspot/jtreg/applications/ctw/modules/java_desktop.java > --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java > +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java > @@ -25,6 +25,9 @@ > * @test > * @summary run CTW for some classes from java.desktop module > * > + * @comment On Windows test hangs when it try to initialize AWT classes. > + * @requires os.family != "windows" > + * > * @library /test/lib / /testlibrary/ctw/src > * @modules java.base/jdk.internal.jimage > * java.base/jdk.internal.misc > diff -r 2ace90aec488 test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java > --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java > +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java > @@ -25,6 +25,9 @@ > * @test > * @summary run CTW for some classes from java.desktop module > * > + * @comment On Windows test hangs when it try to initialize AWT classes. > + * @requires os.family != "windows" > + * > * @library /test/lib / /testlibrary/ctw/src > * @modules java.base/jdk.internal.jimage > * java.base/jdk.internal.misc > > > -- > Thanks, > Vladimir From igor.ignatyev at oracle.com Wed May 2 01:14:47 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 1 May 2018 18:14:47 -0700 Subject: [11] RFR(XS) 8202505) ctw2 tasks are timing out in hs-tier3 In-Reply-To: <19E479CD-B427-47BF-A533-01EF3E5841BE@oracle.com> References: <12b18b61-bedc-1b1c-79a7-d0b336c8418d@oracle.com> <19E479CD-B427-47BF-A533-01EF3E5841BE@oracle.com> Message-ID: <72B3A246-949D-44C0-9D3B-9848B76D6EE5@oracle.com> actually, java_desktop is already problem-listed due to 8189604 "possible hang in sun.awt.shell.Win32ShellFolder2$KnownFolderDefinition::": > applications/ctw/modules/java_desktop.java 8189604 windows-all > applications/ctw/modules/jdk_jconsole.java 8189604 windows-all we just need to add a line for java_desktop_2.java. Thanks, -- Igor > On May 1, 2018, at 6:11 PM, Igor Ignatyev wrote: > > I'd prefer to problem-list java_desktop tests on windows instead of using '@requires'. > > Thanks, > -- Igor > >> On May 1, 2018, at 5:29 PM, Vladimir Kozlov wrote: >> >> http://java.se.oracle.com:10065/mdash/jobs/vkozlov-8202505_00-20180501-2323-20687 >> >> When CompileTheWorld does compilation classes are loaded and may be initialized. Most are side effect free. Except AWT classes which try to interact with system and may hang if classes initializers are not executed in expected order. We had this problem CTW + AWT long before. >> >> Suggested fix is not to run CTW java_desktop.java tests on Windows: >> >> diff -r 2ace90aec488 test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >> --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >> +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >> @@ -25,6 +25,9 @@ >> * @test >> * @summary run CTW for some classes from java.desktop module >> * >> + * @comment On Windows test hangs when it try to initialize AWT classes. >> + * @requires os.family != "windows" >> + * >> * @library /test/lib / /testlibrary/ctw/src >> * @modules java.base/jdk.internal.jimage >> * java.base/jdk.internal.misc >> diff -r 2ace90aec488 test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >> --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >> +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >> @@ -25,6 +25,9 @@ >> * @test >> * @summary run CTW for some classes from java.desktop module >> * >> + * @comment On Windows test hangs when it try to initialize AWT classes. >> + * @requires os.family != "windows" >> + * >> * @library /test/lib / /testlibrary/ctw/src >> * @modules java.base/jdk.internal.jimage >> * java.base/jdk.internal.misc >> >> >> -- >> Thanks, >> Vladimir > From vladimir.kozlov at oracle.com Wed May 2 01:22:25 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 1 May 2018 18:22:25 -0700 Subject: [11] RFR(XS) 8202505) ctw2 tasks are timing out in hs-tier3 In-Reply-To: <72B3A246-949D-44C0-9D3B-9848B76D6EE5@oracle.com> References: <12b18b61-bedc-1b1c-79a7-d0b336c8418d@oracle.com> <19E479CD-B427-47BF-A533-01EF3E5841BE@oracle.com> <72B3A246-949D-44C0-9D3B-9848B76D6EE5@oracle.com> Message-ID: <3f65a8c2-0f18-7780-0d91-2220b8de1d5f@oracle.com> I am okay with blacklisting them. But it is not enough to prevent running them as part of :ctw_2 group. Any idea how to fix that without @requires? Thanks, Vladimir On 5/1/18 6:14 PM, Igor Ignatyev wrote: > actually, java_desktop is already problem-listed due to 8189604 "possible hang in sun.awt.shell.Win32ShellFolder2$KnownFolderDefinition::": >> applications/ctw/modules/java_desktop.java 8189604 windows-all >> applications/ctw/modules/jdk_jconsole.java 8189604 windows-all > > we just need to add a line for java_desktop_2.java. > > Thanks, > -- Igor > >> On May 1, 2018, at 6:11 PM, Igor Ignatyev wrote: >> >> I'd prefer to problem-list java_desktop tests on windows instead of using '@requires'. >> >> Thanks, >> -- Igor >> >>> On May 1, 2018, at 5:29 PM, Vladimir Kozlov wrote: >>> >>> http://java.se.oracle.com:10065/mdash/jobs/vkozlov-8202505_00-20180501-2323-20687 >>> >>> When CompileTheWorld does compilation classes are loaded and may be initialized. Most are side effect free. Except AWT classes which try to interact with system and may hang if classes initializers are not executed in expected order. We had this problem CTW + AWT long before. >>> >>> Suggested fix is not to run CTW java_desktop.java tests on Windows: >>> >>> diff -r 2ace90aec488 test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>> --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>> +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>> @@ -25,6 +25,9 @@ >>> * @test >>> * @summary run CTW for some classes from java.desktop module >>> * >>> + * @comment On Windows test hangs when it try to initialize AWT classes. >>> + * @requires os.family != "windows" >>> + * >>> * @library /test/lib / /testlibrary/ctw/src >>> * @modules java.base/jdk.internal.jimage >>> * java.base/jdk.internal.misc >>> diff -r 2ace90aec488 test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>> --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>> +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>> @@ -25,6 +25,9 @@ >>> * @test >>> * @summary run CTW for some classes from java.desktop module >>> * >>> + * @comment On Windows test hangs when it try to initialize AWT classes. >>> + * @requires os.family != "windows" >>> + * >>> * @library /test/lib / /testlibrary/ctw/src >>> * @modules java.base/jdk.internal.jimage >>> * java.base/jdk.internal.misc >>> >>> >>> -- >>> Thanks, >>> Vladimir >> > From vladimir.kozlov at oracle.com Wed May 2 01:24:37 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 1 May 2018 18:24:37 -0700 Subject: [11] RFR(XS) 8202505) ctw2 tasks are timing out in hs-tier3 In-Reply-To: <3f65a8c2-0f18-7780-0d91-2220b8de1d5f@oracle.com> References: <12b18b61-bedc-1b1c-79a7-d0b336c8418d@oracle.com> <19E479CD-B427-47BF-A533-01EF3E5841BE@oracle.com> <72B3A246-949D-44C0-9D3B-9848B76D6EE5@oracle.com> <3f65a8c2-0f18-7780-0d91-2220b8de1d5f@oracle.com> Message-ID: Or it hangs only on java_desktop_2.java when java_desktop.java is not really run? How we can verify that? Thanks, Vladimir On 5/1/18 6:22 PM, Vladimir Kozlov wrote: > I am okay with blacklisting them. But it is not enough to prevent > running them as part of :ctw_2 group. Any idea how to fix that without > @requires? > > Thanks, > Vladimir > > On 5/1/18 6:14 PM, Igor Ignatyev wrote: >> actually,? java_desktop is already problem-listed due to 8189604 >> "possible hang in >> sun.awt.shell.Win32ShellFolder2$KnownFolderDefinition::": >>> applications/ctw/modules/java_desktop.java 8189604 windows-all >>> applications/ctw/modules/jdk_jconsole.java 8189604 windows-all >> >> we just need to add a line for java_desktop_2.java. >> >> Thanks, >> -- Igor >> >>> On May 1, 2018, at 6:11 PM, Igor Ignatyev >>> wrote: >>> >>> I'd prefer to problem-list java_desktop tests on windows instead of >>> using '@requires'. >>> >>> Thanks, >>> -- Igor >>> >>>> On May 1, 2018, at 5:29 PM, Vladimir Kozlov >>>> wrote: >>>> >>>> http://java.se.oracle.com:10065/mdash/jobs/vkozlov-8202505_00-20180501-2323-20687 >>>> >>>> >>>> When CompileTheWorld does compilation classes are loaded and may be >>>> initialized. Most are side effect free. Except AWT classes >>>> which try to interact with system and may hang if classes >>>> initializers are not executed in expected order. We had this problem >>>> CTW + AWT long before. >>>> >>>> Suggested fix is not to run CTW java_desktop.java tests on Windows: >>>> >>>> diff -r 2ace90aec488 >>>> test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>> --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>> +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>> @@ -25,6 +25,9 @@ >>>> * @test >>>> * @summary run CTW for some classes from java.desktop module >>>> * >>>> + * @comment On Windows test hangs when it try to initialize AWT >>>> classes. >>>> + * @requires os.family != "windows" >>>> + * >>>> * @library /test/lib / /testlibrary/ctw/src >>>> * @modules java.base/jdk.internal.jimage >>>> *????????? java.base/jdk.internal.misc >>>> diff -r 2ace90aec488 >>>> test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>> --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>> +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>> @@ -25,6 +25,9 @@ >>>> * @test >>>> * @summary run CTW for some classes from java.desktop module >>>> * >>>> + * @comment On Windows test hangs when it try to initialize AWT >>>> classes. >>>> + * @requires os.family != "windows" >>>> + * >>>> * @library /test/lib / /testlibrary/ctw/src >>>> * @modules java.base/jdk.internal.jimage >>>> *????????? java.base/jdk.internal.misc >>>> >>>> >>>> -- >>>> Thanks, >>>> Vladimir >>> >> From igor.ignatyev at oracle.com Wed May 2 01:27:02 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 1 May 2018 18:27:02 -0700 Subject: [11] RFR(XS) 8202505) ctw2 tasks are timing out in hs-tier3 In-Reply-To: <3f65a8c2-0f18-7780-0d91-2220b8de1d5f@oracle.com> References: <12b18b61-bedc-1b1c-79a7-d0b336c8418d@oracle.com> <19E479CD-B427-47BF-A533-01EF3E5841BE@oracle.com> <72B3A246-949D-44C0-9D3B-9848B76D6EE5@oracle.com> <3f65a8c2-0f18-7780-0d91-2220b8de1d5f@oracle.com> Message-ID: <5C5CF545-2BE3-4226-A74D-8F5B7B4D3924@oracle.com> adding tests problem list excludes them from running in all test executions thru make, which includes all regular test executions. Thanks, -- Igor > On May 1, 2018, at 6:22 PM, Vladimir Kozlov wrote: > > I am okay with blacklisting them. But it is not enough to prevent running them as part of :ctw_2 group. Any idea how to fix that without @requires? > > Thanks, > Vladimir > > On 5/1/18 6:14 PM, Igor Ignatyev wrote: >> actually, java_desktop is already problem-listed due to 8189604 "possible hang in sun.awt.shell.Win32ShellFolder2$KnownFolderDefinition::": >>> applications/ctw/modules/java_desktop.java 8189604 windows-all >>> applications/ctw/modules/jdk_jconsole.java 8189604 windows-all >> we just need to add a line for java_desktop_2.java. >> Thanks, >> -- Igor >>> On May 1, 2018, at 6:11 PM, Igor Ignatyev wrote: >>> >>> I'd prefer to problem-list java_desktop tests on windows instead of using '@requires'. >>> >>> Thanks, >>> -- Igor >>> >>>> On May 1, 2018, at 5:29 PM, Vladimir Kozlov wrote: >>>> >>>> http://java.se.oracle.com:10065/mdash/jobs/vkozlov-8202505_00-20180501-2323-20687 >>>> >>>> When CompileTheWorld does compilation classes are loaded and may be initialized. Most are side effect free. Except AWT classes which try to interact with system and may hang if classes initializers are not executed in expected order. We had this problem CTW + AWT long before. >>>> >>>> Suggested fix is not to run CTW java_desktop.java tests on Windows: >>>> >>>> diff -r 2ace90aec488 test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>> --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>> +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>> @@ -25,6 +25,9 @@ >>>> * @test >>>> * @summary run CTW for some classes from java.desktop module >>>> * >>>> + * @comment On Windows test hangs when it try to initialize AWT classes. >>>> + * @requires os.family != "windows" >>>> + * >>>> * @library /test/lib / /testlibrary/ctw/src >>>> * @modules java.base/jdk.internal.jimage >>>> * java.base/jdk.internal.misc >>>> diff -r 2ace90aec488 test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>> --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>> +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>> @@ -25,6 +25,9 @@ >>>> * @test >>>> * @summary run CTW for some classes from java.desktop module >>>> * >>>> + * @comment On Windows test hangs when it try to initialize AWT classes. >>>> + * @requires os.family != "windows" >>>> + * >>>> * @library /test/lib / /testlibrary/ctw/src >>>> * @modules java.base/jdk.internal.jimage >>>> * java.base/jdk.internal.misc >>>> >>>> >>>> -- >>>> Thanks, >>>> Vladimir >>> From vladimir.kozlov at oracle.com Wed May 2 01:31:44 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 1 May 2018 18:31:44 -0700 Subject: [11] RFR(XS) 8202505) ctw2 tasks are timing out in hs-tier3 In-Reply-To: <5C5CF545-2BE3-4226-A74D-8F5B7B4D3924@oracle.com> References: <12b18b61-bedc-1b1c-79a7-d0b336c8418d@oracle.com> <19E479CD-B427-47BF-A533-01EF3E5841BE@oracle.com> <72B3A246-949D-44C0-9D3B-9848B76D6EE5@oracle.com> <3f65a8c2-0f18-7780-0d91-2220b8de1d5f@oracle.com> <5C5CF545-2BE3-4226-A74D-8F5B7B4D3924@oracle.com> Message-ID: <1f0e9264-7ddc-405e-ddd5-2311b0cc22c2@oracle.com> Got it. Thank you! I will do as you suggested. Will resend RFR after testing. Thanks, Vladimir On 5/1/18 6:27 PM, Igor Ignatyev wrote: > adding tests problem list excludes them from running in all test executions thru make, which includes all regular test executions. > > Thanks, > -- Igor > >> On May 1, 2018, at 6:22 PM, Vladimir Kozlov wrote: >> >> I am okay with blacklisting them. But it is not enough to prevent running them as part of :ctw_2 group. Any idea how to fix that without @requires? >> >> Thanks, >> Vladimir >> >> On 5/1/18 6:14 PM, Igor Ignatyev wrote: >>> actually, java_desktop is already problem-listed due to 8189604 "possible hang in sun.awt.shell.Win32ShellFolder2$KnownFolderDefinition::": >>>> applications/ctw/modules/java_desktop.java 8189604 windows-all >>>> applications/ctw/modules/jdk_jconsole.java 8189604 windows-all >>> we just need to add a line for java_desktop_2.java. >>> Thanks, >>> -- Igor >>>> On May 1, 2018, at 6:11 PM, Igor Ignatyev wrote: >>>> >>>> I'd prefer to problem-list java_desktop tests on windows instead of using '@requires'. >>>> >>>> Thanks, >>>> -- Igor >>>> >>>>> On May 1, 2018, at 5:29 PM, Vladimir Kozlov wrote: >>>>> >>>>> http://java.se.oracle.com:10065/mdash/jobs/vkozlov-8202505_00-20180501-2323-20687 >>>>> >>>>> When CompileTheWorld does compilation classes are loaded and may be initialized. Most are side effect free. Except AWT classes which try to interact with system and may hang if classes initializers are not executed in expected order. We had this problem CTW + AWT long before. >>>>> >>>>> Suggested fix is not to run CTW java_desktop.java tests on Windows: >>>>> >>>>> diff -r 2ace90aec488 test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>>> --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>>> +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>>> @@ -25,6 +25,9 @@ >>>>> * @test >>>>> * @summary run CTW for some classes from java.desktop module >>>>> * >>>>> + * @comment On Windows test hangs when it try to initialize AWT classes. >>>>> + * @requires os.family != "windows" >>>>> + * >>>>> * @library /test/lib / /testlibrary/ctw/src >>>>> * @modules java.base/jdk.internal.jimage >>>>> * java.base/jdk.internal.misc >>>>> diff -r 2ace90aec488 test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>>> --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>>> +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>>> @@ -25,6 +25,9 @@ >>>>> * @test >>>>> * @summary run CTW for some classes from java.desktop module >>>>> * >>>>> + * @comment On Windows test hangs when it try to initialize AWT classes. >>>>> + * @requires os.family != "windows" >>>>> + * >>>>> * @library /test/lib / /testlibrary/ctw/src >>>>> * @modules java.base/jdk.internal.jimage >>>>> * java.base/jdk.internal.misc >>>>> >>>>> >>>>> -- >>>>> Thanks, >>>>> Vladimir >>>> > From igor.ignatyev at oracle.com Wed May 2 02:10:34 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 1 May 2018 19:10:34 -0700 Subject: RFR(L) : 8199375 : [TESTBUG] Open source vm testbase monitoring tests Message-ID: <0BDC93A6-7D0E-4B4B-A797-936A58A296B9@oracle.com> http://cr.openjdk.java.net/~iignatyev//8199375/webrev.00/index.html > 41276 lines changed: 41274 ins; 1 del; 1 mod; Hi all, could you please review the patch which open sources monitoring tests from vm testbase? The tests were developed to test hotspot related JMX functionality. as w/ common VM testbase code, these tests are old, they have been run from hotspot testing for a long period of time. originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. in a long term, we are planning to rework them. JBS: https://bugs.openjdk.java.net/browse/JDK-8199375 webrev: http://cr.openjdk.java.net/~iignatyev/8199375/webrev.00/index.html testing: vmTestbase_nsk_monitoring test group Thanks, -- Igor From vladimir.kozlov at oracle.com Wed May 2 02:52:15 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 1 May 2018 19:52:15 -0700 Subject: [11] RFR(XS) 8202505) ctw2 tasks are timing out in hs-tier3 In-Reply-To: <1f0e9264-7ddc-405e-ddd5-2311b0cc22c2@oracle.com> References: <12b18b61-bedc-1b1c-79a7-d0b336c8418d@oracle.com> <19E479CD-B427-47BF-A533-01EF3E5841BE@oracle.com> <72B3A246-949D-44C0-9D3B-9848B76D6EE5@oracle.com> <3f65a8c2-0f18-7780-0d91-2220b8de1d5f@oracle.com> <5C5CF545-2BE3-4226-A74D-8F5B7B4D3924@oracle.com> <1f0e9264-7ddc-405e-ddd5-2311b0cc22c2@oracle.com> Message-ID: Testing passed. Here are changes: diff -r 2ace90aec488 test/hotspot/jtreg/ProblemList.txt --- a/test/hotspot/jtreg/ProblemList.txt +++ b/test/hotspot/jtreg/ProblemList.txt @@ -52,6 +52,7 @@ compiler/c2/Test8007294.java 8192992 generic-all applications/ctw/modules/java_desktop.java 8189604 windows-all +applications/ctw/modules/java_desktop_2.java 8189604 windows-all applications/ctw/modules/jdk_jconsole.java 8189604 windows-all ############################################################################# On 5/1/18 6:31 PM, Vladimir Kozlov wrote: > Got it. Thank you! I will do as you suggested. Will resend RFR after > testing. > > Thanks, > Vladimir > > On 5/1/18 6:27 PM, Igor Ignatyev wrote: >> adding tests problem list excludes them from running in all test >> executions thru make, which includes all regular test executions. >> >> Thanks, >> -- Igor >> >>> On May 1, 2018, at 6:22 PM, Vladimir Kozlov >>> wrote: >>> >>> I am okay with blacklisting them. But it is not enough to prevent >>> running them as part of :ctw_2 group. Any idea how to fix that >>> without @requires? >>> >>> Thanks, >>> Vladimir >>> >>> On 5/1/18 6:14 PM, Igor Ignatyev wrote: >>>> actually,? java_desktop is already problem-listed due to 8189604 >>>> "possible hang in >>>> sun.awt.shell.Win32ShellFolder2$KnownFolderDefinition::": >>>>> applications/ctw/modules/java_desktop.java 8189604 windows-all >>>>> applications/ctw/modules/jdk_jconsole.java 8189604 windows-all >>>> we just need to add a line for java_desktop_2.java. >>>> Thanks, >>>> -- Igor >>>>> On May 1, 2018, at 6:11 PM, Igor Ignatyev >>>>> wrote: >>>>> >>>>> I'd prefer to problem-list java_desktop tests on windows instead of >>>>> using '@requires'. >>>>> >>>>> Thanks, >>>>> -- Igor >>>>> >>>>>> On May 1, 2018, at 5:29 PM, Vladimir Kozlov >>>>>> wrote: >>>>>> >>>>>> http://java.se.oracle.com:10065/mdash/jobs/vkozlov-8202505_00-20180501-2323-20687 >>>>>> >>>>>> >>>>>> When CompileTheWorld does compilation classes are loaded and may >>>>>> be initialized. Most are side effect free. Except AWT >>>>>> classes which try to interact with system and may hang if classes >>>>>> initializers are not executed in expected order. We had this >>>>>> problem CTW + AWT long before. >>>>>> >>>>>> Suggested fix is not to run CTW java_desktop.java tests on Windows: >>>>>> >>>>>> diff -r 2ace90aec488 >>>>>> test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>>>> --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>>>> +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>>>> @@ -25,6 +25,9 @@ >>>>>> * @test >>>>>> * @summary run CTW for some classes from java.desktop module >>>>>> * >>>>>> + * @comment On Windows test hangs when it try to initialize AWT >>>>>> classes. >>>>>> + * @requires os.family != "windows" >>>>>> + * >>>>>> * @library /test/lib / /testlibrary/ctw/src >>>>>> * @modules java.base/jdk.internal.jimage >>>>>> *????????? java.base/jdk.internal.misc >>>>>> diff -r 2ace90aec488 >>>>>> test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>>>> --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>>>> +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>>>> @@ -25,6 +25,9 @@ >>>>>> * @test >>>>>> * @summary run CTW for some classes from java.desktop module >>>>>> * >>>>>> + * @comment On Windows test hangs when it try to initialize AWT >>>>>> classes. >>>>>> + * @requires os.family != "windows" >>>>>> + * >>>>>> * @library /test/lib / /testlibrary/ctw/src >>>>>> * @modules java.base/jdk.internal.jimage >>>>>> *????????? java.base/jdk.internal.misc >>>>>> >>>>>> >>>>>> -- >>>>>> Thanks, >>>>>> Vladimir >>>>> >> From vladimir.kozlov at oracle.com Wed May 2 02:54:43 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 1 May 2018 19:54:43 -0700 Subject: RFR(L) : 8199375 : [TESTBUG] Open source vm testbase monitoring tests In-Reply-To: <0BDC93A6-7D0E-4B4B-A797-936A58A296B9@oracle.com> References: <0BDC93A6-7D0E-4B4B-A797-936A58A296B9@oracle.com> Message-ID: Igor, Why you need to list each test in TEST.groups and not just directory as we do in other cases? Thanks, Vladimir On 5/1/18 7:10 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8199375/webrev.00/index.html >> 41276 lines changed: 41274 ins; 1 del; 1 mod; > > Hi all, > > could you please review the patch which open sources monitoring tests from vm testbase? > > The tests were developed to test hotspot related JMX functionality. as w/ common VM testbase code, these tests are old, they have been run from hotspot testing for a long period of time. originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. in a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199375 > webrev: http://cr.openjdk.java.net/~iignatyev/8199375/webrev.00/index.html > testing: vmTestbase_nsk_monitoring test group > > Thanks, > -- Igor > From gnu.andrew at redhat.com Wed May 2 04:25:02 2018 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Wed, 2 May 2018 05:25:02 +0100 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> Message-ID: On 26 April 2018 at 23:55, Kim Barrett wrote: snip... > > I disagree, and still think the perfMemory_linux.cpp change should be > removed. > > (1) The change to perfMemory_linux.cpp is entirely unnecessary to > address the problem this bug is about. > > (2) It violates the (implied) protocol for os::readdir. If > Linux-specific code wants to invoke Linux-specific behavior, it should > do so by calling a Linux-specific API, not abuse an intended to be > portable API. > > (3) It minorly interferes with some desirable future work. If there > were some significant benefit to doing so, I wouldn't give this much > weight, but I don't see a significant benefit. > > (4) The only benefit is saving some rare short-term memory > allocations. I don't think that's worth the above costs. > > Note that the Windows version of os::readdir also ignores the second > argument, but all callers provide it anyway. > > I've opened a new CR for general os::readdir cleanup. > https://bugs.openjdk.java.net/browse/JDK-8202353 > Ok, I see your points and don't feel particularly strongly either way. The original version of this patch I wrote didn't include those changes either. -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Web Site: http://fuseyism.com Twitter: https://twitter.com/gnu_andrew_java PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From igor.ignatyev at oracle.com Wed May 2 04:29:50 2018 From: igor.ignatyev at oracle.com (Igor Ignatev) Date: Tue, 1 May 2018 21:29:50 -0700 Subject: [11] RFR(XS) 8202505) ctw2 tasks are timing out in hs-tier3 In-Reply-To: References: <12b18b61-bedc-1b1c-79a7-d0b336c8418d@oracle.com> <19E479CD-B427-47BF-A533-01EF3E5841BE@oracle.com> <72B3A246-949D-44C0-9D3B-9848B76D6EE5@oracle.com> <3f65a8c2-0f18-7780-0d91-2220b8de1d5f@oracle.com> <5C5CF545-2BE3-4226-A74D-8F5B7B4D3924@oracle.com> <1f0e9264-7ddc-405e-ddd5-2311b0cc22c2@oracle.com> Message-ID: <0743278D-FC01-4E4C-AC0E-DB59FDA09C46@oracle.com> Looks good to me. ? Igor > On May 1, 2018, at 7:52 PM, Vladimir Kozlov wrote: > > Testing passed. Here are changes: > > diff -r 2ace90aec488 test/hotspot/jtreg/ProblemList.txt > --- a/test/hotspot/jtreg/ProblemList.txt > +++ b/test/hotspot/jtreg/ProblemList.txt > @@ -52,6 +52,7 @@ > compiler/c2/Test8007294.java 8192992 generic-all > > applications/ctw/modules/java_desktop.java 8189604 windows-all > +applications/ctw/modules/java_desktop_2.java 8189604 windows-all > applications/ctw/modules/jdk_jconsole.java 8189604 windows-all > > ############################################################################# > > >> On 5/1/18 6:31 PM, Vladimir Kozlov wrote: >> Got it. Thank you! I will do as you suggested. Will resend RFR after testing. >> Thanks, >> Vladimir >>> On 5/1/18 6:27 PM, Igor Ignatyev wrote: >>> adding tests problem list excludes them from running in all test executions thru make, which includes all regular test executions. >>> >>> Thanks, >>> -- Igor >>> >>>> On May 1, 2018, at 6:22 PM, Vladimir Kozlov wrote: >>>> >>>> I am okay with blacklisting them. But it is not enough to prevent running them as part of :ctw_2 group. Any idea how to fix that without @requires? >>>> >>>> Thanks, >>>> Vladimir >>>> >>>>> On 5/1/18 6:14 PM, Igor Ignatyev wrote: >>>>> actually, java_desktop is already problem-listed due to 8189604 "possible hang in sun.awt.shell.Win32ShellFolder2$KnownFolderDefinition::": >>>>>> applications/ctw/modules/java_desktop.java 8189604 windows-all >>>>>> applications/ctw/modules/jdk_jconsole.java 8189604 windows-all >>>>> we just need to add a line for java_desktop_2.java. >>>>> Thanks, >>>>> -- Igor >>>>>> On May 1, 2018, at 6:11 PM, Igor Ignatyev wrote: >>>>>> >>>>>> I'd prefer to problem-list java_desktop tests on windows instead of using '@requires'. >>>>>> >>>>>> Thanks, >>>>>> -- Igor >>>>>> >>>>>>> On May 1, 2018, at 5:29 PM, Vladimir Kozlov wrote: >>>>>>> >>>>>>> http://java.se.oracle.com:10065/mdash/jobs/vkozlov-8202505_00-20180501-2323-20687 >>>>>>> >>>>>>> When CompileTheWorld does compilation classes are loaded and may be initialized. Most are side effect free. Except AWT classes which try to interact with system and may hang if classes initializers are not executed in expected order. We had this problem CTW + AWT long before. >>>>>>> >>>>>>> Suggested fix is not to run CTW java_desktop.java tests on Windows: >>>>>>> >>>>>>> diff -r 2ace90aec488 test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>>>>> --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>>>>> +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop.java >>>>>>> @@ -25,6 +25,9 @@ >>>>>>> * @test >>>>>>> * @summary run CTW for some classes from java.desktop module >>>>>>> * >>>>>>> + * @comment On Windows test hangs when it try to initialize AWT classes. >>>>>>> + * @requires os.family != "windows" >>>>>>> + * >>>>>>> * @library /test/lib / /testlibrary/ctw/src >>>>>>> * @modules java.base/jdk.internal.jimage >>>>>>> * java.base/jdk.internal.misc >>>>>>> diff -r 2ace90aec488 test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>>>>> --- a/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>>>>> +++ b/test/hotspot/jtreg/applications/ctw/modules/java_desktop_2.java >>>>>>> @@ -25,6 +25,9 @@ >>>>>>> * @test >>>>>>> * @summary run CTW for some classes from java.desktop module >>>>>>> * >>>>>>> + * @comment On Windows test hangs when it try to initialize AWT classes. >>>>>>> + * @requires os.family != "windows" >>>>>>> + * >>>>>>> * @library /test/lib / /testlibrary/ctw/src >>>>>>> * @modules java.base/jdk.internal.jimage >>>>>>> * java.base/jdk.internal.misc >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Thanks, >>>>>>> Vladimir >>>>>> >>> From igor.ignatyev at oracle.com Wed May 2 04:39:09 2018 From: igor.ignatyev at oracle.com (Igor Ignatev) Date: Tue, 1 May 2018 21:39:09 -0700 Subject: RFR(L) : 8199375 : [TESTBUG] Open source vm testbase monitoring tests In-Reply-To: References: <0BDC93A6-7D0E-4B4B-A797-936A58A296B9@oracle.com> Message-ID: <395AD9F0-DBBF-4FCC-A01F-9C0D42B83BBD@oracle.com> Vladimir, Tests are listed only in _quick test group b/c it doesn?t include all tests from the directory. We use this group in some of our test configurations, and :vmTestbase_nsk_monitoring in others. vmTestbase_nsk_monitoring is defined by the directory as other groups. Thanks, ? Igor > On May 1, 2018, at 7:54 PM, Vladimir Kozlov wrote: > > Igor, > > Why you need to list each test in TEST.groups and not just directory as we do in other cases? > > Thanks, > Vladimir > >> On 5/1/18 7:10 PM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8199375/webrev.00/index.html >>> 41276 lines changed: 41274 ins; 1 del; 1 mod; >> Hi all, >> could you please review the patch which open sources monitoring tests from vm testbase? >> The tests were developed to test hotspot related JMX functionality. as w/ common VM testbase code, these tests are old, they have been run from hotspot testing for a long period of time. originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. in a long term, we are planning to rework them. >> JBS: https://bugs.openjdk.java.net/browse/JDK-8199375 >> webrev: http://cr.openjdk.java.net/~iignatyev/8199375/webrev.00/index.html >> testing: vmTestbase_nsk_monitoring test group >> Thanks, >> -- Igor From shade at redhat.com Wed May 2 07:47:55 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 2 May 2018 09:47:55 +0200 Subject: RFR [8u] 8165489 (S): Missing G1 barrier in Unsafe_GetObjectVolatile In-Reply-To: <19ca5b19-d458-b37f-1a7c-7404a027306a@redhat.com> References: <19ca5b19-d458-b37f-1a7c-7404a027306a@redhat.com> Message-ID: <7d5f6f33-4d13-a2f8-49e8-ce6dc588d470@redhat.com> No takers? :) -Aleksey On 04/25/2018 12:34 PM, Aleksey Shipilev wrote: > (trying again on hotspot-dev@) > > Hi, > > Please review this this backport/fix for 8u. This improves G1 stability in 8u, and provides the base > for Shenandoah JDK 8 backport that piggybacks on G1 pre-barriers. The fix does not apply cleanly, > because there is also Unsafe_GetObject140 that needs fixing too. > > JDK 9 bug: > https://bugs.openjdk.java.net/browse/JDK-8165489 > > JDK 9 fix: > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/a696583f5ddb > > JDK 8u fix: > http://cr.openjdk.java.net/~shade/8165489-8u/webrev.01/ > > Testing: x86_64 build, Shenandoah tests > > Thanks, > -Aleksey > From magnus.ihse.bursie at oracle.com Wed May 2 06:58:52 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 2 May 2018 08:58:52 +0200 Subject: RFR(L) : 8199375 : [TESTBUG] Open source vm testbase monitoring tests In-Reply-To: <0BDC93A6-7D0E-4B4B-A797-936A58A296B9@oracle.com> References: <0BDC93A6-7D0E-4B4B-A797-936A58A296B9@oracle.com> Message-ID: Build changes look fine. /Magnus > 2 maj 2018 kl. 04:10 skrev Igor Ignatyev : > > http://cr.openjdk.java.net/~iignatyev//8199375/webrev.00/index.html >> 41276 lines changed: 41274 ins; 1 del; 1 mod; > > Hi all, > > could you please review the patch which open sources monitoring tests from vm testbase? > > The tests were developed to test hotspot related JMX functionality. as w/ common VM testbase code, these tests are old, they have been run from hotspot testing for a long period of time. originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. in a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199375 > webrev: http://cr.openjdk.java.net/~iignatyev/8199375/webrev.00/index.html > testing: vmTestbase_nsk_monitoring test group > > Thanks, > -- Igor From robbin.ehn at oracle.com Wed May 2 08:31:22 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 2 May 2018 10:31:22 +0200 Subject: RFR(L) 8195099: Low latency hashtable for read-mostly scenarios In-Reply-To: <0CCE37A8-A9AA-414E-9898-5B875B647BE6@oracle.com> References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <0CCE37A8-A9AA-414E-9898-5B875B647BE6@oracle.com> Message-ID: <3e8cc7c3-f7fd-e117-8ef2-1e016125cf66@oracle.com> Hi Gerard, On 2018-04-30 21:23, Gerard Ziemski wrote: > hi Robbin, > > Still reviewing, but I have a couple of questions and some feedback I wanted to ask/share with you so far: > > > #1 Where does the choice of "12" for _task_size_log2 come from in: > > BucketsOperation(ConcurrentHashTable* cht) > : _cht(cht), _next_to_claim(0), _task_size_log2(12), > _stop_task(0), _size_log2(0) {} > 2^12 comes from benchmarking the stringstable using this hashtable. Where we do not want to delay a safepoint but still have enough work to get some throughput. > Shouldn't "12" be a constant? Fixed. > > > #2 Where does "8192" come from in: It comes from spinYield.hpp > > // SpinYield would be unfair here > while (!this->trylock()) { > if ((++i) == 8192) { > > Shouldn't "8192" be a constant? > Fixed. > > #3 How come "_first" doesn't need to be volatile to be used in Atomic::cmpxchg ? > > Node* _first; > ... > if (Atomic::cmpxchg(node, &_first, expect) == expect) { An implicit cast to the same compatible type which is volatile-qualified is okey. > > > #4 Inconsistent parameter names - in: > > template > bool get_insert_lazy(Thread* thread, LOOKUP_FUNC& lookup, VALUE_FUNC& val_f, > bool* grow_hint = NULL) { > return get_insert_lazy(thread, lookup, val_f, noOp, grow_hint); > } > > We use "val_f" with "_f? suffix, so we should have "lookup_f", not "lookup" (many other instances) and "found_f", not "foundf" in > > template > bool get(Thread* thread, LOOKUP_FUNC& lookup, FOUND_FUNC& foundf, > bool* grow_hint = NULL); > Fixed. > > #5 I wasn't familiar with CRTP, so I read up on it, but I still don't see the CRTP in BaseConfig - which is the base class and which is derived? In particular I don't see "class Derived : public Base" pattern here? It's the supplied config that extends the BaseConfig which is the derived. In gtest (test_concurrentHashtable.cpp) you will see this: // Simplest working CRPT implementation for the hash-table. struct Pointer : public SimpleTestTable::BaseConfig { > > More to come? > > > btw. I edited the subject of the email slightly by adding the bug number to it, so it?s easy to search for. > Thanks, I'll send the update to the original mail. /Robbin > > cheers > > >> On Apr 26, 2018, at 2:38 AM, Robbin Ehn wrote: >> >> >> Hi all, please review. >> >> The lower latency of the new gcs have higher demands on runtime data-structure >> in terms of latency. This is a concurrent hashtable using the global-counter >> (8195099). >> >> Webrev: >> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8195098 >> >> * Readers never blocks or spins. >> * Inserts do CAS from a read-side, need to re-CAS during re-size/deletion in targeted bucket or insert collision. (inserts are conceptually a reader) >> * Deletes locks the targeted bucket, all other buckets are free for operations. >> * Re-size locks one bucket at the time, all other buckets are free for operations. >> >> It does concurrent re-size by doubling size and use one more bit from hash. >> That means a bucket in the old table contains nodes to either one of two buckets >> in the new table. Each node in the chain is sorted into one of the two buckets >> with concurrent readers. Shrinking just take two node chains and put them >> together in the corresponding bucket. To keep track of what is visible to the >> concurrent readers it's uses a global-counter, needed during re-size and for >> deletes. >> >> A gtest is provided which passes on our platforms, we also have a prototype of the stringtable using this which passes tier 1-5 on our platforms. >> >> Various people have pre-reviewed various parts, thanks! And a special thanks to >> Coleen for a lot of reviewing! >> >> Thanks, Robbin > From robbin.ehn at oracle.com Wed May 2 08:34:30 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 2 May 2018 10:34:30 +0200 Subject: RFR(L): Low latency hashtable for read-mostly scenarios In-Reply-To: <00344c0b-6122-be48-53e8-4d9a99bb2b6b@oracle.com> References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <00344c0b-6122-be48-53e8-4d9a99bb2b6b@oracle.com> Message-ID: <3fcef8f2-c4b6-314a-13be-c1cd0d361385@oracle.com> Hi Coleen, On 2018-05-01 00:18, coleen.phillimore at oracle.com wrote: > > Robbin, > > What is gi_bd in these names in the test?? Can you give them more descriptive names? > > cht_gi_bd_insert_new > cht_gi_bd_get_old > > Are they get/insert bulk/delete? Yes, fixed. Update to original mail. > > I'm fine with the rest of the code.?? It's a huge project and it enables > concurrent string and symbol table cleanup, so this is a big enhancement. Thanks Robbin. > > thanks, > Coleen > > On 4/26/18 3:38 AM, Robbin Ehn wrote: >> >> Hi all, please review. >> >> The lower latency of the new gcs have higher demands on runtime data-structure >> in terms of latency. This is a concurrent hashtable using the global-counter >> (8195099). >> >> Webrev: >> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8195098 >> >> * Readers never blocks or spins. >> * Inserts do CAS from a read-side, need to re-CAS during re-size/deletion in >> targeted bucket or insert collision. (inserts are conceptually a reader) >> * Deletes locks the targeted bucket, all other buckets are free for operations. >> * Re-size locks one bucket at the time, all other buckets are free for >> operations. >> >> It does concurrent re-size by doubling size and use one more bit from hash. >> That means a bucket in the old table contains nodes to either one of two buckets >> in the new table. Each node in the chain is sorted into one of the two buckets >> with concurrent readers. Shrinking just take two node chains and put them >> together in the corresponding bucket. To keep track of what is visible to the >> concurrent readers it's uses a global-counter, needed during re-size and for >> deletes. >> >> A gtest is provided which passes on our platforms, we also have a prototype of >> the stringtable using this which passes tier 1-5 on our platforms. >> >> Various people have pre-reviewed various parts, thanks! And a special thanks to >> Coleen for a lot of reviewing! >> >> Thanks, Robbin > From bsrbnd at gmail.com Wed May 2 08:38:36 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 2 May 2018 10:38:36 +0200 Subject: Large page use crashes the JVM on some Linux systems In-Reply-To: References: <3bfbdf0a-7c9c-6b95-8c6a-a92c169dbf9b@oracle.com> <9d291862-b0ae-09b6-ebad-8c070bfd1446@oracle.com> Message-ID: Hi Claes, On 24 April 2018 at 21:39, Claes Redestad wrote: > The root issue here could very well be that the SHM sanity test is > insufficient. Adding the same test as we already do for TLBFS seems like the > wrong approach. > > I'm not the most knowledgeable about SHM, though, in fact not knowledgeable > at all, so let's try and get you subscribed to hotspot-dev and spark a > discussion on the list. > > /Claes As suggested, I've added a SHM sanity test (derived from the TLBFS one) which ensure that the allocated memory is using huge pages in '/proc/self/maps'. Does the following patch look better (tier1 seems OK)? Thanks, Bernard diff --git a/src/hotspot/os/linux/os_linux.cpp b/src/hotspot/os/linux/os_linux.cpp --- a/src/hotspot/os/linux/os_linux.cpp +++ b/src/hotspot/os/linux/os_linux.cpp @@ -3317,6 +3317,28 @@ return result; } +static bool check_hugepage(void* p) { + bool result = false; + FILE *fp = fopen("/proc/self/maps", "r"); + if (fp) { + while (!feof(fp)) { + char chars[257]; + long x = 0; + if (fgets(chars, sizeof(chars), fp)) { + if (sscanf(chars, "%lx-%*x", &x) == 1 + && x == (long)p) { + if (strstr (chars, "hugepage")) { + result = true; + break; + } + } + } + } + fclose(fp); + } + return result; +} + bool os::Linux::hugetlbfs_sanity_check(bool warn, size_t page_size) { bool result = false; void *p = mmap(NULL, page_size, PROT_READ|PROT_WRITE, @@ -3325,23 +3347,7 @@ if (p != MAP_FAILED) { // We don't know if this really is a huge page or not. - FILE *fp = fopen("/proc/self/maps", "r"); - if (fp) { - while (!feof(fp)) { - char chars[257]; - long x = 0; - if (fgets(chars, sizeof(chars), fp)) { - if (sscanf(chars, "%lx-%*x", &x) == 1 - && x == (long)p) { - if (strstr (chars, "hugepage")) { - result = true; - break; - } - } - } - } - fclose(fp); - } + result = check_hugepage(p); munmap(p, page_size); } @@ -3352,6 +3358,27 @@ return result; } +bool os::Linux::shm_sanity_check(bool warn, size_t page_size) { + bool result = false; + int shmid = shmget(IPC_PRIVATE, page_size, SHM_HUGETLB|IPC_CREAT|SHM_R|SHM_W); + if (shmid != -1) { + void* p = shmat(shmid, NULL, 0); + shmctl(shmid, IPC_RMID, NULL); + + if (p != (void*) -1) { + // We don't know if this really is a huge page or not. + result = check_hugepage(p); + shmdt(p); + } + } + + if (warn && !result) { + warning("SHM is not supported by the operating system."); + } + + return result; +} + // Set the coredump_filter bits to include largepages in core dump (bit 6) // // From the coredump_filter documentation: @@ -3506,6 +3533,14 @@ UseHugeTLBFS = false; } + if (UseSHM) { + bool warn_on_failure = !FLAG_IS_DEFAULT(UseSHM); + if (shm_sanity_check(warn_on_failure, page_size)) { + return true; + } + UseSHM = false; + } + return UseSHM; } diff --git a/src/hotspot/os/linux/os_linux.hpp b/src/hotspot/os/linux/os_linux.hpp --- a/src/hotspot/os/linux/os_linux.hpp +++ b/src/hotspot/os/linux/os_linux.hpp @@ -99,6 +99,7 @@ static bool setup_large_page_type(size_t page_size); static bool transparent_huge_pages_sanity_check(bool warn, size_t pages_size); static bool hugetlbfs_sanity_check(bool warn, size_t page_size); + static bool shm_sanity_check(bool warn, size_t page_size); static char* reserve_memory_special_shm(size_t bytes, size_t alignment, char* req_addr, bool exec); static char* reserve_memory_special_huge_tlbfs(size_t bytes, size_t alignment, char* req_addr, bool exec); diff --git a/test/hotspot/jtreg/gc/g1/TestLargePageUseForAuxMemory.java b/test/hotspot/jtreg/gc/g1/TestLargePageUseForAuxMemory.java --- a/test/hotspot/jtreg/gc/g1/TestLargePageUseForAuxMemory.java +++ b/test/hotspot/jtreg/gc/g1/TestLargePageUseForAuxMemory.java @@ -60,7 +60,8 @@ } long size = parseMemoryString(pageSizeStr); - if (size != expectedSize) { + // Small pages may be used if no large pages are available. + if (size != expectedSize && size != smallPageSize) { output.reportDiagnosticSummary(); throw new RuntimeException("Match from '" + pattern + "' got " + size + " expected: " + expectedSize); } From robbin.ehn at oracle.com Wed May 2 09:03:50 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 2 May 2018 11:03:50 +0200 Subject: RFR(L): 8195098: Low latency hashtable for read-mostly scenarios In-Reply-To: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> Message-ID: <8defc9dd-e908-1bd5-8fdb-929905773ab0@oracle.com> Hi all, Here is an update with Gerard's and Coleen's input. Inc: http://cr.openjdk.java.net/~rehn/8195098/v1/inc/webrev/ Full: http://cr.openjdk.java.net/~rehn/8195098/v1/full/webrev/ Thanks, Robbin On 2018-04-26 09:38, Robbin Ehn wrote: > > Hi all, please review. > > The lower latency of the new gcs have higher demands on runtime data-structure > in terms of latency. This is a concurrent hashtable using the global-counter > (8195099). > > Webrev: > http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8195098 > > * Readers never blocks or spins. > * Inserts do CAS from a read-side, need to re-CAS during re-size/deletion in > targeted bucket or insert collision. (inserts are conceptually a reader) > * Deletes locks the targeted bucket, all other buckets are free for operations. > * Re-size locks one bucket at the time, all other buckets are free for operations. > > It does concurrent re-size by doubling size and use one more bit from hash. > That means a bucket in the old table contains nodes to either one of two buckets > in the new table. Each node in the chain is sorted into one of the two buckets > with concurrent readers. Shrinking just take two node chains and put them > together in the corresponding bucket. To keep track of what is visible to the > concurrent readers it's uses a global-counter, needed during re-size and for > deletes. > > A gtest is provided which passes on our platforms, we also have a prototype of > the stringtable using this which passes tier 1-5 on our platforms. > > Various people have pre-reviewed various parts, thanks! And a special thanks to > Coleen for a lot of reviewing! > > Thanks, Robbin From mvala at redhat.com Wed May 2 09:10:20 2018 From: mvala at redhat.com (Michal Vala) Date: Wed, 2 May 2018 11:10:20 +0200 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> Message-ID: <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> On 05/01/2018 07:59 PM, Kim Barrett wrote: >> On Apr 27, 2018, at 4:26 PM, Michal Vala wrote: >>>> For now, proposed patch looks like this: >>>> >>>> --- old/src/hotspot/os/linux/os_linux.inline.hpp 2018-04-20 09:16:34.498343185 +0200 >>>> +++ new/src/hotspot/os/linux/os_linux.inline.hpp 2018-04-20 09:16:34.214342670 +0200 >>>> @@ -98,26 +98,8 @@ >>>> >>>> inline struct dirent* os::readdir(DIR* dirp, dirent *dbuf) >>>> { >>>> -// readdir_r has been deprecated since glibc 2.24. >>>> -// See https://sourceware.org/bugzilla/show_bug.cgi?id=19056 for more details. >>>> -#pragma GCC diagnostic push >>>> -#pragma GCC diagnostic ignored "-Wdeprecated-declarations" >>>> - >>>> - dirent* p; >>>> - int status; >>>> assert(dirp != NULL, "just checking"); >>>> - >>>> - // NOTE: Linux readdir_r (on RH 6.2 and 7.2 at least) is NOT like the POSIX >>>> - // version. Here is the doc for this function: >>>> - // http://www.gnu.org/manual/glibc-2.2.3/html_node/libc_262.html >>>> - >>>> - if((status = ::readdir_r(dirp, dbuf, &p)) != 0) { >>>> - errno = status; >>>> - return NULL; >>>> - } else >>>> - return p; >>>> - >>>> -#pragma GCC diagnostic pop >>>> + return ::readdir(dirp); >>>> } >>>> >>>> inline int os::closedir(DIR *dirp) { >>> This looks good. >> >> Thanks! >> Someone to sponsor this please? > > Do you have a sponsor yet? If not, I?ll do it. > No, I don't. I'd really appreciate if you sponsor it. -- Michal Vala OpenJDK QE Red Hat Czech From shade at redhat.com Wed May 2 09:21:43 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 2 May 2018 11:21:43 +0200 Subject: RFR 8201786: Modularize interpreter GC barriers: leftovers for ARM32 In-Reply-To: <89150A82-E74C-4046-ABF8-80AAA0C86744@oracle.com> References: <4d22aacb-aa0d-4423-1a22-775e2be7eeaf@redhat.com> <1524847169.9114.6.camel@gmail.com> <668e8a59-cba1-a2ce-48a0-cf9cfcde032b@redhat.com> <1524866512.9114.10.camel@gmail.com> <828a25d9-d4b1-ec05-71e3-ceddef4b7a24@redhat.com> <89150A82-E74C-4046-ABF8-80AAA0C86744@oracle.com> Message-ID: <6ea56b54-cfa3-0222-e220-768c8cc2804a@redhat.com> Thanks for review! Full: http://cr.openjdk.java.net/~shade/8201786/webrev.03/ Changes from this review: http://cr.openjdk.java.net/~shade/8201786/webrev.03.diff Testing: arm32-hflt builds, adhoc runs with Serial/G1 -Xint on Raspi 3 On 05/01/2018 07:10 PM, Erik Osterlund wrote: > 1) The jobject resolve grabs the BarrierSetAssembler and calls straight into it, instead of making > an access_load_at call. I would prefer using the access call. Right. Fixed. > 2) The macro assembler has a bunch of G1 includes left. I hope they can be removed now. Right. Removed. > 3) The CardTableBarrierSet checks for CMS flags instead of checking whether the card table is > scanned concurrently or not. This is fixed separately, and you have reviewed this separately: https://bugs.openjdk.java.net/browse/JDK-8202418 > 4) The template interpreter generator Reference.get intrinsic is missing an ON_WEAK_OOP_REF > decorator, which is required to get the right G1 barriers for example. Good spot, fixed. After this change, assert_different_register started to fail with G1, so I had to use the proper batch of temporary registers. > 5) In the template table, all do_oop_load/store accesses should be IN_HEAP. They are missing that > decorator. That will cause those accesses to skip pre/post write barriers. This is where I am confused. New ARM32 code in templateTable_arm.cpp repeats the shape spotted in templateTable_x86.cpp. We call TemplateTable::do_oop_{load|store} with default decorators, sometimes mixing in IN_HEAP_ARRAY. It then delegates into MacroAssembler::{load|store}_heap_oop, which mixes in "IN_HEAP |". > 6) I don?t know if you want to venture into the shady domain of jni fast get field. If you do, > the implicit jweak resolve in there on ARM should call > BarrierSetAssembler::try_resolve_jobject_in_native. ... I filed > https://bugs.openjdk.java.net/browse/JDK-8202480 for that. So might be a different RFR I guess. Yes, let it be separate RFR. -Aleksey From swatibits14 at gmail.com Wed May 2 10:24:17 2018 From: swatibits14 at gmail.com (Swati Sharma) Date: Wed, 2 May 2018 15:54:17 +0530 Subject: UseNUMA membind Issue in openJDK In-Reply-To: <9a0310b7-2880-db69-cfbc-7abba844ecbf@oracle.com> References: <9a0310b7-2880-db69-cfbc-7abba844ecbf@oracle.com> Message-ID: Hi David, I have localized the struct bitmask declaration in os_linux.cpp. Here is the updated patch ===================================PATCH=================================================== diff --git a/src/hotspot/os/linux/os_linux.cpp b/src/hotspot/os/linux/os_linux.cpp --- a/src/hotspot/os/linux/os_linux.cpp +++ b/src/hotspot/os/linux/os_linux.cpp @@ -2832,14 +2832,42 @@ // Map all node ids in which is possible to allocate memory. Also nodes are // not always consecutively available, i.e. available from 0 to the highest // node number. + // If the nodes have been bound explicitly using numactl membind, then + // allocate memory from those nodes only. for (size_t node = 0; node <= highest_node_number; node++) { - if (Linux::isnode_in_configured_nodes(node)) { + if (Linux::isnode_in_bounded_nodes(node)) { ids[i++] = node; } } return i; } +extern "C" struct bitmask { + unsigned long size; /* number of bits in the map */ + unsigned long *maskp; +}; +// Check if single memory node bound. +// Returns true if single memory node bound. +bool os::Linux::issingle_node_bound() { + struct bitmask* bmp = _numa_get_membind != NULL ? _numa_get_membind() : NULL; + if(bmp == NULL) return false; + int issingle = 0; + // System can have more than 64 nodes so check in all the elements of + // unsigned long array + for (unsigned long i = 0; i < (bmp->size / (8 * sizeof(unsigned long))); i++) { + if (bmp->maskp != NULL && (((bmp->maskp[i]) & (((bmp->maskp[i])) - 1)) == 0)) { + issingle++; + } else if (bmp->maskp[i] == 0) { + continue; + } else { + return false; + } + } + if (issingle == 1) + return true; + return false; +} + bool os::get_page_info(char *start, page_info* info) { return false; } @@ -2930,6 +2958,10 @@ libnuma_dlsym(handle, "numa_bitmask_isbitset"))); set_numa_distance(CAST_TO_FN_PTR(numa_distance_func_t, libnuma_dlsym(handle, "numa_distance"))); + set_numa_set_membind(CAST_TO_FN_PTR(numa_set_membind_func_t, + libnuma_dlsym(handle, "numa_set_membind"))); + set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, + libnuma_v2_dlsym(handle, "numa_get_membind"))); if (numa_available() != -1) { set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, "numa_all_nodes")); @@ -3054,6 +3086,8 @@ os::Linux::numa_set_bind_policy_func_t os::Linux::_numa_set_bind_policy; os::Linux::numa_bitmask_isbitset_func_t os::Linux::_numa_bitmask_isbitset; os::Linux::numa_distance_func_t os::Linux::_numa_distance; +os::Linux::numa_set_membind_func_t os::Linux::_numa_set_membind; +os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; unsigned long* os::Linux::_numa_all_nodes; struct bitmask* os::Linux::_numa_all_nodes_ptr; struct bitmask* os::Linux::_numa_nodes_ptr; @@ -4962,8 +4996,9 @@ if (!Linux::libnuma_init()) { UseNUMA = false; } else { - if ((Linux::numa_max_node() < 1)) { - // There's only one node(they start from 0), disable NUMA. + if ((Linux::numa_max_node() < 1) || Linux::issingle_node_bound()) { + // If there's only one node(they start from 0) or if the process + // is bound explicitly to a single node using membind, disable NUMA. UseNUMA = false; } } diff --git a/src/hotspot/os/linux/os_linux.hpp b/src/hotspot/os/linux/os_linux.hpp --- a/src/hotspot/os/linux/os_linux.hpp +++ b/src/hotspot/os/linux/os_linux.hpp @@ -228,6 +228,8 @@ typedef int (*numa_tonode_memory_func_t)(void *start, size_t size, int node); typedef void (*numa_interleave_memory_func_t)(void *start, size_t size, unsigned long *nodemask); typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t size, struct bitmask* mask); + typedef void (*numa_set_membind_func_t)(struct bitmask *mask); + typedef struct bitmask* (*numa_get_membind_func_t)(void); typedef void (*numa_set_bind_policy_func_t)(int policy); typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, unsigned int n); @@ -244,6 +246,8 @@ static numa_set_bind_policy_func_t _numa_set_bind_policy; static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; static numa_distance_func_t _numa_distance; + static numa_set_membind_func_t _numa_set_membind; + static numa_get_membind_func_t _numa_get_membind; static unsigned long* _numa_all_nodes; static struct bitmask* _numa_all_nodes_ptr; static struct bitmask* _numa_nodes_ptr; @@ -259,6 +263,8 @@ static void set_numa_set_bind_policy(numa_set_bind_policy_func_t func) { _numa_set_bind_policy = func; } static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t func) { _numa_bitmask_isbitset = func; } static void set_numa_distance(numa_distance_func_t func) { _numa_distance = func; } + static void set_numa_set_membind(numa_set_membind_func_t func) { _numa_set_membind = func; } + static void set_numa_get_membind(numa_get_membind_func_t func) { _numa_get_membind = func; } static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes = ptr; } static void set_numa_all_nodes_ptr(struct bitmask **ptr) { _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } static void set_numa_nodes_ptr(struct bitmask **ptr) { _numa_nodes_ptr = (ptr == NULL ? NULL : *ptr); } @@ -320,6 +326,15 @@ } else return 0; } + // Check if node in bounded nodes + static bool isnode_in_bounded_nodes(int node) { + struct bitmask* bmp = _numa_get_membind != NULL ? _numa_get_membind() : NULL; + if (bmp != NULL && _numa_bitmask_isbitset != NULL && _numa_bitmask_isbitset(bmp, node)) { + return true; + } else + return false; + } + static bool issingle_node_bound(); }; #endif // OS_LINUX_VM_OS_LINUX_HPP ============================================================================================ Thanks, Swati On Thu, Apr 26, 2018 at 6:10 PM, David Holmes wrote: > > Hi Swati, > > On 26/04/2018 10:20 PM, Swati Sharma wrote: >> >> Hi Everyone, >> >> I work at AMD and this is my first patch as a new member of openJDK >> community. > > > Welcome! > > I can't comment on the actual NUMA details of the patch (though I can see what you're doing), but the struct bitmask declaration in os.hpp should be localized in os_linux.hpp as far as I can see, as it's only needed internally in the Linux code. > > Thanks, > David > ----- > > >> I have found some issue while running specjbb2015 composite workload with >> the flag -XX:+UseNUMA. It seems that JVM does not allocate memory according >> to the explicit node binding done using "numactl --membind". >> >> E.g. If bound to a single memory node, JVM divides the whole heap based on >> the total number of numa nodes available on the system which creates more >> logical groups(lgrps) than required which cannot be used except the one. >> >> The following examples will explain clearly : >> (Note : Collected GC logs with >> -Xlog:gc*=debug:file=gc.log:time,uptimemillis) >> 1) Allocating a heap of 22GB for single node divides the whole heap in 8 >> lgrp(Actual no of Nodes are 8) >> $numactl --cpunodebind=0 --membind=0 java -Xmx24g -Xms24g -Xmn22g >> -XX:+UseNUMA >> >> eden space 22511616K(22GB), 12% used >> lgrp 0 space 2813952K, 100% used lgrp 1 space >> 2813952K, 0% used lgrp 2 space 2813952K, 0% used >> lgrp 3 space 2813952K, 0% used lgrp 4 space >> 2813952K, 0% used lgrp 5 space 2813952K, 0% used >> lgrp 6 space 2813952K, 0% used lgrp 7 space >> 2813952K, 0% used >> >> Observation : Instead of disabling UseNUMA for single node binding JVM >> divides the memory in 8 lgrps and allocates memory always on the bounded >> node hence in eden space allocation never happens more than 12%. >> >> 2) Another case of binding to node 0 and 7 results in dividing the heap in >> 8lgrp >> $numactl --cpunodebind=0,7 ?membind=0,7 java -Xms50g -Xmx50g -Xmn45g >> -XX:+UseNUMA >> >> eden space 46718976K, 6% used >> lgrp 0 space 5838848K, 14% used lgrp 1 space 5838848K, >> 0% used lgrp 2 space 5838848K, 0% used >> lgrp 3 space 5838848K, 0% used lgrp 4 space >> 5838848K, 0% used lgrp 5 space 5838848K, 0% >> used >> lgrp 6 space 5838848K, 0% used lgrp 7 space >> 5847040K, 35% used >> >> Observation : Similar to first case allocation happens only on 0th and 7th >> node and rest of the lgrps never gets used. >> >> After applying the patch, JVM divides the given heap size according to the >> bounded memory nodes only. >> >> 1) Binding to single node disables UseNUMA >> eden space 46718976K(45GB), 99% used >> >> Observation : UseNUMA gets disabled hence no lgrp creation and the whole >> heap allocation happens on the bounded node. >> >> 2) Binding to node 0 and 7 >> $ numactl --cpunodebind=0,7 ?membind=0,7 java -Xms50g -Xmx50g -Xmn45g >> -XX:+UseNUMA >> eden space 46718976K(45GB), 99% used >> lgrp 0 space 23359488K(23.5GB), 100% used lgrp 7 space >> 23359488K(23.5GB), 99% used >> >> Observation : Only two lgrps gets created and heap size gets divided >> equally in both nodes. >> >> If there is no binding, then JVM will divide the whole heap based on the >> number of NUMA nodes available on the system. >> >> The following patch fixes the issue(attached also). >> Please review and let me know your comments. >> >> Regression testing using jtreg (make -J=1 run-test-tier1 run-test-tier2) >> didn't show any new failures. >> >> ===============================PATCH======================================== >> diff --git a/src/hotspot/os/linux/os_linux.cpp >> b/src/hotspot/os/linux/os_linux.cpp >> --- a/src/hotspot/os/linux/os_linux.cpp >> +++ b/src/hotspot/os/linux/os_linux.cpp >> @@ -2832,8 +2832,10 @@ >> // Map all node ids in which is possible to allocate memory. Also nodes >> are >> // not always consecutively available, i.e. available from 0 to the >> highest >> // node number. >> + // If the nodes have been bound explicitly using numactl membind, then >> + // allocate memory from those nodes only. >> for (size_t node = 0; node <= highest_node_number; node++) { >> - if (Linux::isnode_in_configured_nodes(node)) { >> + if (Linux::isnode_in_bounded_nodes(node)) { >> ids[i++] = node; >> } >> } >> @@ -2930,6 +2932,10 @@ >> libnuma_dlsym(handle, >> "numa_bitmask_isbitset"))); >> set_numa_distance(CAST_TO_FN_PTR(numa_distance_func_t, >> libnuma_dlsym(handle, >> "numa_distance"))); >> + set_numa_set_membind(CAST_TO_FN_PTR(numa_set_membind_func_t, >> + libnuma_dlsym(handle, >> "numa_set_membind"))); >> + set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, >> + libnuma_v2_dlsym(handle, >> "numa_get_membind"))); >> >> if (numa_available() != -1) { >> set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, >> "numa_all_nodes")); >> @@ -3054,6 +3060,8 @@ >> os::Linux::numa_set_bind_policy_func_t os::Linux::_numa_set_bind_policy; >> os::Linux::numa_bitmask_isbitset_func_t os::Linux::_numa_bitmask_isbitset; >> os::Linux::numa_distance_func_t os::Linux::_numa_distance; >> +os::Linux::numa_set_membind_func_t os::Linux::_numa_set_membind; >> +os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; >> unsigned long* os::Linux::_numa_all_nodes; >> struct bitmask* os::Linux::_numa_all_nodes_ptr; >> struct bitmask* os::Linux::_numa_nodes_ptr; >> @@ -4962,8 +4970,9 @@ >> if (!Linux::libnuma_init()) { >> UseNUMA = false; >> } else { >> - if ((Linux::numa_max_node() < 1)) { >> - // There's only one node(they start from 0), disable NUMA. >> + if ((Linux::numa_max_node() < 1) || Linux::issingle_node_bound()) { >> + // If there's only one node(they start from 0) or if the process >> + // is bound explicitly to a single node using membind, disable >> NUMA. >> UseNUMA = false; >> } >> } >> diff --git a/src/hotspot/os/linux/os_linux.hpp >> b/src/hotspot/os/linux/os_linux.hpp >> --- a/src/hotspot/os/linux/os_linux.hpp >> +++ b/src/hotspot/os/linux/os_linux.hpp >> @@ -228,6 +228,8 @@ >> typedef int (*numa_tonode_memory_func_t)(void *start, size_t size, int >> node); >> typedef void (*numa_interleave_memory_func_t)(void *start, size_t size, >> unsigned long *nodemask); >> typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t >> size, struct bitmask* mask); >> + typedef void (*numa_set_membind_func_t)(struct bitmask *mask); >> + typedef struct bitmask* (*numa_get_membind_func_t)(void); >> >> typedef void (*numa_set_bind_policy_func_t)(int policy); >> typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, >> unsigned int n); >> @@ -244,6 +246,8 @@ >> static numa_set_bind_policy_func_t _numa_set_bind_policy; >> static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; >> static numa_distance_func_t _numa_distance; >> + static numa_set_membind_func_t _numa_set_membind; >> + static numa_get_membind_func_t _numa_get_membind; >> static unsigned long* _numa_all_nodes; >> static struct bitmask* _numa_all_nodes_ptr; >> static struct bitmask* _numa_nodes_ptr; >> @@ -259,6 +263,8 @@ >> static void set_numa_set_bind_policy(numa_set_bind_policy_func_t func) { >> _numa_set_bind_policy = func; } >> static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t func) >> { _numa_bitmask_isbitset = func; } >> static void set_numa_distance(numa_distance_func_t func) { >> _numa_distance = func; } >> + static void set_numa_set_membind(numa_set_membind_func_t func) { >> _numa_set_membind = func; } >> + static void set_numa_get_membind(numa_get_membind_func_t func) { >> _numa_get_membind = func; } >> static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes = >> ptr; } >> static void set_numa_all_nodes_ptr(struct bitmask **ptr) { >> _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } >> static void set_numa_nodes_ptr(struct bitmask **ptr) { _numa_nodes_ptr = >> (ptr == NULL ? NULL : *ptr); } >> @@ -320,6 +326,34 @@ >> } else >> return 0; >> } >> + // Check if node in bounded nodes >> + static bool isnode_in_bounded_nodes(int node) { >> + struct bitmask* bmp = _numa_get_membind != NULL ? _numa_get_membind() >> : NULL; >> + if (bmp != NULL && _numa_bitmask_isbitset != NULL && >> _numa_bitmask_isbitset(bmp, node)) { >> + return true; >> + } else >> + return false; >> + } >> + // Check if a single node is bound >> + static bool issingle_node_bound() { >> + struct bitmask* bmp = _numa_get_membind != NULL ? _numa_get_membind() >> : NULL; >> + if(bmp == NULL) return false; >> + int issingle = 0; >> + // System can have more than 64 nodes so check in all the elements of >> + // unsigned long array >> + for (unsigned long i = 0; i < (bmp->size / (8 * sizeof(unsigned >> long))); i++) { >> + if (bmp->maskp != NULL && (((bmp->maskp[i]) & (((bmp->maskp[i])) - >> 1)) == 0)) { >> + issingle++; >> + } else if (bmp->maskp[i] == 0) { >> + continue; >> + } else { >> + return false; >> + } >> + } >> + if (issingle == 1) >> + return true; >> + return false; >> + } >> }; >> >> #endif // OS_LINUX_VM_OS_LINUX_HPP >> diff --git a/src/hotspot/share/runtime/os.hpp >> b/src/hotspot/share/runtime/os.hpp >> --- a/src/hotspot/share/runtime/os.hpp >> +++ b/src/hotspot/share/runtime/os.hpp >> @@ -81,6 +81,10 @@ >> CriticalPriority = 11 // Critical thread priority >> }; >> >> +extern "C" struct bitmask { >> + unsigned long size; /* number of bits in the map */ >> + unsigned long *maskp; >> +}; >> // Executable parameter flag for os::commit_memory() and >> // os::commit_memory_or_exit(). >> const bool ExecMem = true; >> >> ============================================================================= >> >> Thanks, >> Swati >> From erik.osterlund at oracle.com Wed May 2 10:43:10 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 2 May 2018 12:43:10 +0200 Subject: RFR 8201786: Modularize interpreter GC barriers: leftovers for ARM32 In-Reply-To: <6ea56b54-cfa3-0222-e220-768c8cc2804a@redhat.com> References: <4d22aacb-aa0d-4423-1a22-775e2be7eeaf@redhat.com> <1524847169.9114.6.camel@gmail.com> <668e8a59-cba1-a2ce-48a0-cf9cfcde032b@redhat.com> <1524866512.9114.10.camel@gmail.com> <828a25d9-d4b1-ec05-71e3-ceddef4b7a24@redhat.com> <89150A82-E74C-4046-ABF8-80AAA0C86744@oracle.com> <6ea56b54-cfa3-0222-e220-768c8cc2804a@redhat.com> Message-ID: <5AE9963E.8010000@oracle.com> Hi Aleksey, On 2018-05-02 11:21, Aleksey Shipilev wrote: > Thanks for review! > > Full: > http://cr.openjdk.java.net/~shade/8201786/webrev.03/ > > Changes from this review: > http://cr.openjdk.java.net/~shade/8201786/webrev.03.diff > > Testing: arm32-hflt builds, adhoc runs with Serial/G1 -Xint on Raspi 3 > > On 05/01/2018 07:10 PM, Erik Osterlund wrote: >> 1) The jobject resolve grabs the BarrierSetAssembler and calls straight into it, instead of making >> an access_load_at call. I would prefer using the access call. > Right. Fixed. > >> 2) The macro assembler has a bunch of G1 includes left. I hope they can be removed now. > Right. Removed. > >> 3) The CardTableBarrierSet checks for CMS flags instead of checking whether the card table is >> scanned concurrently or not. > This is fixed separately, and you have reviewed this separately: > https://bugs.openjdk.java.net/browse/JDK-8202418 > >> 4) The template interpreter generator Reference.get intrinsic is missing an ON_WEAK_OOP_REF >> decorator, which is required to get the right G1 barriers for example. > Good spot, fixed. After this change, assert_different_register started to fail with G1, so I had to > use the proper batch of temporary registers. > >> 5) In the template table, all do_oop_load/store accesses should be IN_HEAP. They are missing that >> decorator. That will cause those accesses to skip pre/post write barriers. > This is where I am confused. New ARM32 code in templateTable_arm.cpp repeats the shape spotted in > templateTable_x86.cpp. We call TemplateTable::do_oop_{load|store} with default decorators, sometimes > mixing in IN_HEAP_ARRAY. It then delegates into MacroAssembler::{load|store}_heap_oop, which mixes > in "IN_HEAP |". Right, I see now that you use load/store_heap_oop. For some reason I read it as access_load/store_at. Your changes look good now. Thanks for sorting this out. /Erik > >> 6) I don?t know if you want to venture into the shady domain of jni fast get field. If you do, >> the implicit jweak resolve in there on ARM should call >> BarrierSetAssembler::try_resolve_jobject_in_native. ... I filed >> https://bugs.openjdk.java.net/browse/JDK-8202480 for that. So might be a different RFR I guess. > Yes, let it be separate RFR. > > -Aleksey > > From martin.doerr at sap.com Wed May 2 10:52:52 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 2 May 2018 10:52:52 +0000 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: Message-ID: Hi Matthias, looks like a nice enhancement. We can get substantially more information. I wonder if we really need the sizes in kB in addition to MB. Maybe other reviewers would like to comment on this, too. We should have line breaks. Best regards, Martin -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Baesken, Matthias Sent: Montag, 30. April 2018 16:53 To: 'hotspot-dev at openjdk.java.net' Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows On Windows, the os::print_memory_info misses a few memory-related infos that might be interesting : - current and peak WorkingSet size (= physical memory assigned to the process) - current and peak commit charge (also known as "private bytes" in Windows tools) - on 32bit Windows : user-mode portion/free user mode portion of virtual address-space (Total/AvailVirtual) because it shows how close do we get to the 2-4 GB per process border. - the current naming of "swap/free-swap" memory is a bit misleading; in the Windows world swap is a special page file used for UWP apps. (see Windows Internals : "The swap file" (chapter 5 Memory management) Windows 8 added another page file called a swap file. It is ... exclusively used for UWP (Universal Windows Platform) apps. Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the UWP related values, it is about page file sizes (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just virtual memory might be more appropriate). https://msdn.microsoft.com/de-de/library/windows/desktop/aa366770(v=vs.85).aspx documents it in the following way: ullTotalPageFile: The current committed memory limit for the system or the current process, whichever is smaller,... ullAvailPageFile : The maximum amount of memory the current process can commit, in bytes. This value is equal to or smaller than the system-wide available commit value Aditionally I suggest having output of the various memory-values in M (megabyte) as well , the k (kilobyte) output sometimes gives very high and unreadable numbers). Could you please review my change ? Bug : https://bugs.openjdk.java.net/browse/JDK-8202427 Change : http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ Thanks, Matthias From matthias.baesken at sap.com Wed May 2 10:59:49 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 2 May 2018 10:59:49 +0000 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: Message-ID: Hi Martin, thanks for your input . I add hotspot-runtime-dev . > > I wonder if we really need the sizes in kB in addition to MB. Maybe other > reviewers would like to comment on this, too. > We could also just print the MB value , let's see what other think. Another option might be to have a flexible output (kB for smaller memory values , MB (or GB) for larger ) ? Best regards, Matthias > -----Original Message----- > From: Doerr, Martin > Sent: Mittwoch, 2. Mai 2018 12:53 > To: Baesken, Matthias ; 'hotspot- > dev at openjdk.java.net' > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > > Hi Matthias, > > looks like a nice enhancement. We can get substantially more information. > > I wonder if we really need the sizes in kB in addition to MB. Maybe other > reviewers would like to comment on this, too. > > We should have line breaks. > > Best regards, > Martin > > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > Behalf Of Baesken, Matthias > Sent: Montag, 30. April 2018 16:53 > To: 'hotspot-dev at openjdk.java.net' > Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows > > On Windows, > the os::print_memory_info misses a few memory-related infos that might > be interesting : > - current and peak WorkingSet size (= physical memory assigned to the > process) > - current and peak commit charge (also known as "private bytes" in Windows > tools) > - on 32bit Windows : > user-mode portion/free user mode portion of virtual address-space > (Total/AvailVirtual) because it shows how close do we get to the 2-4 GB per > process border. > > > - the current naming of "swap/free-swap" memory is a bit misleading; > in the Windows world swap is a special page file used for UWP apps. > (see Windows Internals : "The swap file" (chapter 5 Memory management) > Windows 8 added another page file called a swap file. It is ... exclusively used > for UWP (Universal Windows Platform) apps. > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the UWP > related values, it is about page file sizes > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just virtual > memory might be more appropriate). > > > https://msdn.microsoft.com/de- > de/library/windows/desktop/aa366770(v=vs.85).aspx > > documents it in the following way: > ullTotalPageFile: > The current committed memory limit for the system or the current process, > whichever is smaller,... > > ullAvailPageFile : > The maximum amount of memory the current process can commit, in bytes. > This value is equal to or smaller than the system-wide available commit value > > > > Aditionally I suggest having output of the various memory-values in M > (megabyte) as well , the k (kilobyte) output sometimes gives very high and > unreadable numbers). > > > Could you please review my change ? > > > Bug : > > https://bugs.openjdk.java.net/browse/JDK-8202427 > > > Change : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > > > Thanks, Matthias > From erik.osterlund at oracle.com Wed May 2 11:14:48 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 2 May 2018 13:14:48 +0200 Subject: RFR 8201786: Modularize interpreter GC barriers: leftovers for ARM32 In-Reply-To: <5AE9963E.8010000@oracle.com> References: <4d22aacb-aa0d-4423-1a22-775e2be7eeaf@redhat.com> <1524847169.9114.6.camel@gmail.com> <668e8a59-cba1-a2ce-48a0-cf9cfcde032b@redhat.com> <1524866512.9114.10.camel@gmail.com> <828a25d9-d4b1-ec05-71e3-ceddef4b7a24@redhat.com> <89150A82-E74C-4046-ABF8-80AAA0C86744@oracle.com> <6ea56b54-cfa3-0222-e220-768c8cc2804a@redhat.com> <5AE9963E.8010000@oracle.com> Message-ID: <5AE99DA8.7010502@oracle.com> Hi Aleksey, I spoke too soon. I realize now that in do_oop_store/load, you have a decorator argument, but you never pass it along to load/store_heap_oop. That seems wrong. Now it seems like it works because you never pass in any decorators to do_oop_store/load. For example the array accesses have IN_HEAP_ARRAY, which causes card marking barriers to use precise marking instead. But it looks like they will never learn that this was indeed an array as the decorators are not passed along. Thanks, /Erik On 2018-05-02 12:43, Erik ?sterlund wrote: > Hi Aleksey, > > On 2018-05-02 11:21, Aleksey Shipilev wrote: >> Thanks for review! >> >> Full: >> http://cr.openjdk.java.net/~shade/8201786/webrev.03/ >> >> Changes from this review: >> http://cr.openjdk.java.net/~shade/8201786/webrev.03.diff >> >> Testing: arm32-hflt builds, adhoc runs with Serial/G1 -Xint on Raspi 3 >> >> On 05/01/2018 07:10 PM, Erik Osterlund wrote: >>> 1) The jobject resolve grabs the BarrierSetAssembler and calls >>> straight into it, instead of making >>> an access_load_at call. I would prefer using the access call. >> Right. Fixed. >> >>> 2) The macro assembler has a bunch of G1 includes left. I hope they >>> can be removed now. >> Right. Removed. >> >>> 3) The CardTableBarrierSet checks for CMS flags instead of checking >>> whether the card table is >>> scanned concurrently or not. >> This is fixed separately, and you have reviewed this separately: >> https://bugs.openjdk.java.net/browse/JDK-8202418 >> >>> 4) The template interpreter generator Reference.get intrinsic is >>> missing an ON_WEAK_OOP_REF >>> decorator, which is required to get the right G1 barriers for example. >> Good spot, fixed. After this change, assert_different_register >> started to fail with G1, so I had to >> use the proper batch of temporary registers. >> >>> 5) In the template table, all do_oop_load/store accesses should be >>> IN_HEAP. They are missing that >>> decorator. That will cause those accesses to skip pre/post write >>> barriers. >> This is where I am confused. New ARM32 code in templateTable_arm.cpp >> repeats the shape spotted in >> templateTable_x86.cpp. We call TemplateTable::do_oop_{load|store} >> with default decorators, sometimes >> mixing in IN_HEAP_ARRAY. It then delegates into >> MacroAssembler::{load|store}_heap_oop, which mixes >> in "IN_HEAP |". > > Right, I see now that you use load/store_heap_oop. For some reason I > read it as access_load/store_at. > > Your changes look good now. Thanks for sorting this out. > > /Erik > >> >>> 6) I don?t know if you want to venture into the shady domain of jni >>> fast get field. If you do, >>> the implicit jweak resolve in there on ARM should call >>> BarrierSetAssembler::try_resolve_jobject_in_native. ... I filed >>> https://bugs.openjdk.java.net/browse/JDK-8202480 for that. So might >>> be a different RFR I guess. >> Yes, let it be separate RFR. >> >> -Aleksey >> >> > From magnus.ihse.bursie at oracle.com Wed May 2 12:02:22 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 2 May 2018 14:02:22 +0200 Subject: [11] RFR for JDK-8202544: Hide unused exports in libzip In-Reply-To: <4b0e27bb-8167-4ae5-c5e5-6ec054015d30@oracle.com> References: <4b0e27bb-8167-4ae5-c5e5-6ec054015d30@oracle.com> Message-ID: Looks good to me, FWIW. /Magnus > 2 maj 2018 kl. 13:52 skrev Alexey Ivanov : > > Hi, > > Could you please review the following fix for jdk11? > > bug: https://bugs.openjdk.java.net/browse/JDK-8202544 > webrev: http://cr.openjdk.java.net/~aivanov/8202544/jdk11/webrev.0/ > > The following exported functions in libzip are not used: > ZIP_GetEntry, ZIP_FreeEntry, ZIP_Lock, ZIP_Unlock, ZIP_Read > > I removed JNIEXPORT modifiers from these functions. With the fix, they're not exported on Windows; on Linux they're listed as Local rather than Global. > > I ran tests, no failures. > > > Thank you in advance. > > Regards, > Alexey From coleen.phillimore at oracle.com Wed May 2 12:05:16 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 2 May 2018 08:05:16 -0400 Subject: RFR(L): 8195098: Low latency hashtable for read-mostly scenarios In-Reply-To: <8defc9dd-e908-1bd5-8fdb-929905773ab0@oracle.com> References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <8defc9dd-e908-1bd5-8fdb-929905773ab0@oracle.com> Message-ID: On 5/2/18 5:03 AM, Robbin Ehn wrote: > Hi all, > > Here is an update with Gerard's and Coleen's input. > > Inc: > http://cr.openjdk.java.net/~rehn/8195098/v1/inc/webrev/ > Full: > http://cr.openjdk.java.net/~rehn/8195098/v1/full/webrev/ > > Thanks, Robbin This keeps getting better.? Nice long test names! Thanks, Coleen > > On 2018-04-26 09:38, Robbin Ehn wrote: >> >> Hi all, please review. >> >> The lower latency of the new gcs have higher demands on runtime >> data-structure >> in terms of latency. This is a concurrent hashtable using the >> global-counter >> (8195099). >> >> Webrev: >> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8195098 >> >> * Readers never blocks or spins. >> * Inserts do CAS from a read-side, need to re-CAS during >> re-size/deletion in targeted bucket or insert collision. (inserts are >> conceptually a reader) >> * Deletes locks the targeted bucket, all other buckets are free for >> operations. >> * Re-size locks one bucket at the time, all other buckets are free >> for operations. >> >> It does concurrent re-size by doubling size and use one more bit from >> hash. >> That means a bucket in the old table contains nodes to either one of >> two buckets >> in the new table. Each node in the chain is sorted into one of the >> two buckets >> with concurrent readers. Shrinking just take two node chains and put >> them >> together in the corresponding bucket. To keep track of what is >> visible to the >> concurrent readers it's uses a global-counter, needed during re-size >> and for >> deletes. >> >> A gtest is provided which passes on our platforms, we also have a >> prototype of the stringtable using this which passes tier 1-5 on our >> platforms. >> >> Various people have pre-reviewed various parts, thanks! And a special >> thanks to >> Coleen for a lot of reviewing! >> >> Thanks, Robbin From robbin.ehn at oracle.com Wed May 2 12:39:10 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 2 May 2018 14:39:10 +0200 Subject: RFR: 8176717: GC log file handle leaked to child processes In-Reply-To: References: Message-ID: <19fd7c9a-0224-f363-08f8-ce9daaeb4d02@oracle.com> Hi Leo, I looked at http://cr.openjdk.java.net/~lkorinth/8176717/02/. I think it would be good with some sort of error checking if fcntl failed, maybe just an assert. 1282 if (fd != -1) { 1283 fcntl(fd, F_SETFD, fcntl(fd, F_GETFD) | FD_CLOEXEC); 1284 } Otherwise looks good, thanks, Robbin. On 2018-03-12 14:20, Leo Korinth wrote: > Hi, > > This fix is for all operating systems though the problem only seams to appear on > windows. > > I am creating a proxy function for fopen (os::fopen_retain) that appends the > non-standard "e" mode for linux and bsds. For windows the "N" mode is used. For > other operating systems, I assume that I can use fcntl F_SETFD FD_CLOEXEC. I > think this will work for AIX, Solaris and other operating systems that do not > support the "e" flag. Feedback otherwise please! > > The reason that I use the mode "e" and not only fcntl for linux and bsds is > threefold. First, I still need to use mode flags on windows as it does not > support fcntl. Second, I probably save a system call. Third, the change will be > applied directly, and there will be no point in time (between system calls) when > the process can leak the file descriptor, so it is safer. > > The test case forks three VMs in a row. By doing so we know that the second VM > is opened with a specific log file. The third VM should have less open file > descriptors (as it is does not use logging) which is checked using a > UnixOperatingSystemMXBean. This is not possible on windows, so on windows I try > to rename the file, which will not work if the file is opened (the actual reason > the bug was opened). > > The added test case shows that the bug fix closes the log file on windows. The > VM on other operating systems closed the log file even before the fix. > > Maybe the test case should be moved to a different path? > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8176717 > https://bugs.openjdk.java.net/browse/JDK-8176809 > > Webrev: > http://cr.openjdk.java.net/~lkorinth/8176717/00/ > > Testing: > hs-tier1, hs-tier2 and TestInheritFD.java > (on 64-bit linux, solaris, windows and mac) > > Thanks, > Leo From david.holmes at oracle.com Wed May 2 12:40:52 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 2 May 2018 22:40:52 +1000 Subject: RFR(L) 8195099: Low latency hashtable for read-mostly scenarios In-Reply-To: <3e8cc7c3-f7fd-e117-8ef2-1e016125cf66@oracle.com> References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <0CCE37A8-A9AA-414E-9898-5B875B647BE6@oracle.com> <3e8cc7c3-f7fd-e117-8ef2-1e016125cf66@oracle.com> Message-ID: Not a review, one comment ... On 2/05/2018 6:31 PM, Robbin Ehn wrote: > On 2018-04-30 21:23, Gerard Ziemski wrote: >> #3 How come "_first" doesn't need to be volatile to be used in >> Atomic::cmpxchg ? >> >> Node* _first; >> ... >> ? if (Atomic::cmpxchg(node, &_first, expect) == expect) { > > An implicit cast to the same compatible type which is volatile-qualified > is okey. Any variable used in a lock-free manner should be declared volatile as general good practice IMHO. David ----- >> >> >> #4 Inconsistent parameter names - in: >> >> ? template >> ? bool get_insert_lazy(Thread* thread, LOOKUP_FUNC& lookup, >> VALUE_FUNC& val_f, >> ?????????????????????? bool* grow_hint = NULL) { >> ??? return get_insert_lazy(thread, lookup, val_f, noOp, grow_hint); >> ? } >> >> We use "val_f" with "_f? suffix, so we should have "lookup_f", not >> "lookup" (many other instances) and "found_f", not "foundf" in >> >> ? template >> ? bool get(Thread* thread, LOOKUP_FUNC& lookup, FOUND_FUNC& foundf, >> ?????????? bool* grow_hint = NULL); >> > > Fixed. > >> >> #5 I wasn't familiar with CRTP, so I read up on it, but I still don't >> see the CRTP in BaseConfig - which is the base class and which is >> derived? In particular I don't see "class Derived : public >> Base" pattern here? > > It's the supplied config that extends the BaseConfig which is the derived. > > In gtest (test_concurrentHashtable.cpp) you will see this: > > // Simplest working CRPT implementation for the hash-table. > struct Pointer : public SimpleTestTable::BaseConfig { > >> >> More to come? >> >> >> btw. I edited the subject of the email slightly by adding the bug >> number to it, so it?s easy to search for. >> > > Thanks, I'll send the update to the original mail. > > /Robbin > > > >> >> cheers >> >> >>> On Apr 26, 2018, at 2:38 AM, Robbin Ehn wrote: >>> >>> >>> Hi all, please review. >>> >>> The lower latency of the new gcs have higher demands on runtime >>> data-structure >>> in terms of latency. This is a concurrent hashtable using the >>> global-counter >>> (8195099). >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8195098 >>> >>> * Readers never blocks or spins. >>> * Inserts do CAS from a read-side, need to re-CAS during >>> re-size/deletion in targeted bucket or insert collision. (inserts are >>> conceptually a reader) >>> * Deletes locks the targeted bucket, all other buckets are free for >>> operations. >>> * Re-size locks one bucket at the time, all other buckets are free >>> for operations. >>> >>> It does concurrent re-size by doubling size and use one more bit from >>> hash. >>> That means a bucket in the old table contains nodes to either one of >>> two buckets >>> in the new table. Each node in the chain is sorted into one of the >>> two buckets >>> with concurrent readers. Shrinking just take two node chains and put >>> them >>> together in the corresponding bucket. To keep track of what is >>> visible to the >>> concurrent readers it's uses a global-counter, needed during re-size >>> and for >>> deletes. >>> >>> A gtest is provided which passes on our platforms, we also have a >>> prototype of the stringtable using this which passes tier 1-5 on our >>> platforms. >>> >>> Various people have pre-reviewed various parts, thanks! And a special >>> thanks to >>> Coleen for a lot of reviewing! >>> >>> Thanks, Robbin >> From shade at redhat.com Wed May 2 13:51:00 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 2 May 2018 15:51:00 +0200 Subject: RFR 8201786: Modularize interpreter GC barriers: leftovers for ARM32 In-Reply-To: <5AE99DA8.7010502@oracle.com> References: <4d22aacb-aa0d-4423-1a22-775e2be7eeaf@redhat.com> <1524847169.9114.6.camel@gmail.com> <668e8a59-cba1-a2ce-48a0-cf9cfcde032b@redhat.com> <1524866512.9114.10.camel@gmail.com> <828a25d9-d4b1-ec05-71e3-ceddef4b7a24@redhat.com> <89150A82-E74C-4046-ABF8-80AAA0C86744@oracle.com> <6ea56b54-cfa3-0222-e220-768c8cc2804a@redhat.com> <5AE9963E.8010000@oracle.com> <5AE99DA8.7010502@oracle.com> Message-ID: On 05/02/2018 01:14 PM, Erik ?sterlund wrote: > I spoke too soon. I realize now that in do_oop_store/load, you have a decorator argument, but you > never pass it along to load/store_heap_oop. That seems wrong. Now it seems like it works because you > never pass in any decorators to do_oop_store/load. For example the array accesses have > IN_HEAP_ARRAY, which causes card marking barriers to use precise marking instead. But it looks like > they will never learn that this was indeed an array as the decorators are not passed along. Argh. It is worse than that: there seems to be the implicit conversion of "bool is_null" to "DecoratorSet set", which is wildly wrong. Fixed like this: http://cr.openjdk.java.net/~shade/8201786/webrev.04.diff Full webrev: http://cr.openjdk.java.net/~shade/8201786/webrev.04/ arm32-hflt seems to still build/work :) Thanks, -Aleksey From robbin.ehn at oracle.com Wed May 2 13:56:56 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 2 May 2018 15:56:56 +0200 Subject: RFR(L) 8195099: Low latency hashtable for read-mostly scenarios In-Reply-To: References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <0CCE37A8-A9AA-414E-9898-5B875B647BE6@oracle.com> <3e8cc7c3-f7fd-e117-8ef2-1e016125cf66@oracle.com> Message-ID: <4cf61dce-3f8e-9baf-6278-68002f615e84@oracle.com> On 2018-05-02 14:40, David Holmes wrote: > Not a review, one comment ... > > On 2/05/2018 6:31 PM, Robbin Ehn wrote: >> On 2018-04-30 21:23, Gerard Ziemski wrote: >>> #3 How come "_first" doesn't need to be volatile to be used in Atomic::cmpxchg ? >>> >>> Node* _first; >>> ... >>> ? if (Atomic::cmpxchg(node, &_first, expect) == expect) { >> >> An implicit cast to the same compatible type which is volatile-qualified is okey. > > Any variable used in a lock-free manner should be declared volatile as general > good practice IMHO. Generally I agree with you. But in this case I didn't find the volatile qualifier helpful, it just clobbered the code. I didn't see any readability improvements from having essential all lines with Node* also having volatile in them. What I would like is a type that forces the use of the correct ordering, e.g. Atomic _next; /Robbin > > David > ----- > >>> >>> >>> #4 Inconsistent parameter names - in: >>> >>> ? template >>> ? bool get_insert_lazy(Thread* thread, LOOKUP_FUNC& lookup, VALUE_FUNC& val_f, >>> ?????????????????????? bool* grow_hint = NULL) { >>> ??? return get_insert_lazy(thread, lookup, val_f, noOp, grow_hint); >>> ? } >>> >>> We use "val_f" with "_f? suffix, so we should have "lookup_f", not "lookup" >>> (many other instances) and "found_f", not "foundf" in >>> >>> ? template >>> ? bool get(Thread* thread, LOOKUP_FUNC& lookup, FOUND_FUNC& foundf, >>> ?????????? bool* grow_hint = NULL); >>> >> >> Fixed. >> >>> >>> #5 I wasn't familiar with CRTP, so I read up on it, but I still don't see the >>> CRTP in BaseConfig - which is the base class and which is derived? In >>> particular I don't see "class Derived : public Base" pattern here? >> >> It's the supplied config that extends the BaseConfig which is the derived. >> >> In gtest (test_concurrentHashtable.cpp) you will see this: >> >> // Simplest working CRPT implementation for the hash-table. >> struct Pointer : public SimpleTestTable::BaseConfig { >> >>> >>> More to come? >>> >>> >>> btw. I edited the subject of the email slightly by adding the bug number to >>> it, so it?s easy to search for. >>> >> >> Thanks, I'll send the update to the original mail. >> >> /Robbin >> >> >> >>> >>> cheers >>> >>> >>>> On Apr 26, 2018, at 2:38 AM, Robbin Ehn wrote: >>>> >>>> >>>> Hi all, please review. >>>> >>>> The lower latency of the new gcs have higher demands on runtime data-structure >>>> in terms of latency. This is a concurrent hashtable using the global-counter >>>> (8195099). >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8195098 >>>> >>>> * Readers never blocks or spins. >>>> * Inserts do CAS from a read-side, need to re-CAS during re-size/deletion in >>>> targeted bucket or insert collision. (inserts are conceptually a reader) >>>> * Deletes locks the targeted bucket, all other buckets are free for operations. >>>> * Re-size locks one bucket at the time, all other buckets are free for >>>> operations. >>>> >>>> It does concurrent re-size by doubling size and use one more bit from hash. >>>> That means a bucket in the old table contains nodes to either one of two >>>> buckets >>>> in the new table. Each node in the chain is sorted into one of the two buckets >>>> with concurrent readers. Shrinking just take two node chains and put them >>>> together in the corresponding bucket. To keep track of what is visible to the >>>> concurrent readers it's uses a global-counter, needed during re-size and for >>>> deletes. >>>> >>>> A gtest is provided which passes on our platforms, we also have a prototype >>>> of the stringtable using this which passes tier 1-5 on our platforms. >>>> >>>> Various people have pre-reviewed various parts, thanks! And a special thanks to >>>> Coleen for a lot of reviewing! >>>> >>>> Thanks, Robbin >>> From stefan.karlsson at oracle.com Wed May 2 14:37:38 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 2 May 2018 16:37:38 +0200 Subject: RFR: 8200729: Conditional compilation of GCs Message-ID: Hi all, Please review these patches to allow for conditional compilation of the GCs in HotSpot. Full patch: http://cr.openjdk.java.net/~stefank/8200729/webrev.04/all/ (See below for a more fine-grained division into smaller patches) Today Parallel, G1, and CMS, are all guarded by INCLUDE_ALL_GCS. INCLUDE_ALL_GCS becomes defined to 1 for our server (--with-jvm-variants=server) builds, and is defined to 0 for the minimal (--with-jvm-variants=minimal) builds. There are also ways to forcefully remove these GCs from the compilation by configuring with, for example, --with-jvm-features=all-gcs. The proposed patch removes INCLUDE_ALL_GCS (and all-gcs) and replaces it with INCLUDE_CMSGC, INCLUDE_G1GC, and INCLUDE_PARALLELGC. In addition to that, INCLUDE_SERIALGC has been added to guard the Serial GC code. Future GCs should adopt this scheme before they get incorporated into the code base. Note, that there will be some files in gc/shared that are going to have to know about all GCs, but the goal is to have very few of these INCLUDE_ checks in the non-GC parts of HotSpot. With this patch, it's also going to be easier to stop compiling CMS when the time as come for it to move from deprecated to removed. Note, that even though this adds great flexibility, and allows for easy inclusion / exclusion of GCs, there's no guarantee that a specific combination of GCs is going to be maintained in the future. Just like not all combinations of the different Runtime features (CDS, NMT, JFR) and Compiler features (AOT, COMPILER1, COMPILER2) are guaranteed to compile and be supported. I've sanity checked the different GC configurations build locally, but I've only fully tested the "server" variant, and "minimal" has only been built locally. There's a more high-level discussion around the flexibility the --with-jvm-features flag adds, and if we really should allow it. Since this patch only builds upon that existing flexibility, and I don't change the two defined jvm variants, I'd appreciate if that discussion could be kept out of this review thread. For further discussion around this, please see: http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021663.html This is the patch queue: The first patch simply cleans up some INCLUDE_ALL_GCS usages in platform-specific files. Some of these changes are already being cleaned up by other RFEs: http://cr.openjdk.java.net/~stefank/8200729/webrev.04/00.removeUnneededIncludeAllGCs/ The second patch pre-cleans some include files: http://cr.openjdk.java.net/~stefank/8200729/webrev.04/01.fixIncludes/ The following is the main patch, which include all relevant HotSpot changes. For a while now, we have been actively working on abstracting away GC specific code from non-GC directories, but as can be seen in this patch, there are still a few places left. Hopefully, we will get rid of most of these in the near future. http://cr.openjdk.java.net/~stefank/8200729/webrev.04/02.mainPatch/ The last patch adds the make file support to turn on and off the different GCs. The content of this patch has evolved from versions written by myself, Per, and Magnus Ihse Bursie. Magnus last proposed version used the names gc-cms, gc-g1, gc-parallel, and gc-serial, but I've changed them to cmsgc, g1gc, parallelgc, and serialgc, so that they match the INCLUDE_ defines. http://cr.openjdk.java.net/~stefank/8200729/webrev.04/03.selectIndivudualGCsMakePatch/ Thanks, StefanK From alexey.ivanov at oracle.com Wed May 2 11:52:12 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Wed, 2 May 2018 12:52:12 +0100 Subject: [11] RFR for JDK-8202544: Hide unused exports in libzip Message-ID: <4b0e27bb-8167-4ae5-c5e5-6ec054015d30@oracle.com> Hi, Could you please review the following fix for jdk11? bug: https://bugs.openjdk.java.net/browse/JDK-8202544 webrev: http://cr.openjdk.java.net/~aivanov/8202544/jdk11/webrev.0/ The following exported functions in libzip are not used: ZIP_GetEntry, ZIP_FreeEntry, ZIP_Lock, ZIP_Unlock, ZIP_Read I removed JNIEXPORT modifiers from these functions. With the fix, they're not exported on Windows; on Linux they're listed as Local rather than Global. I ran tests, no failures. Thank you in advance. Regards, Alexey From erik.osterlund at oracle.com Wed May 2 17:20:38 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Wed, 2 May 2018 19:20:38 +0200 Subject: RFR 8201786: Modularize interpreter GC barriers: leftovers for ARM32 In-Reply-To: References: <4d22aacb-aa0d-4423-1a22-775e2be7eeaf@redhat.com> <1524847169.9114.6.camel@gmail.com> <668e8a59-cba1-a2ce-48a0-cf9cfcde032b@redhat.com> <1524866512.9114.10.camel@gmail.com> <828a25d9-d4b1-ec05-71e3-ceddef4b7a24@redhat.com> <89150A82-E74C-4046-ABF8-80AAA0C86744@oracle.com> <6ea56b54-cfa3-0222-e220-768c8cc2804a@redhat.com> <5AE9963E.8010000@oracle.com> <5AE99DA8.7010502@oracle.com> Message-ID: Hi Aleksey, Looks good. Thanks, /Erik > On 2 May 2018, at 15:51, Aleksey Shipilev wrote: > >> On 05/02/2018 01:14 PM, Erik ?sterlund wrote: >> I spoke too soon. I realize now that in do_oop_store/load, you have a decorator argument, but you >> never pass it along to load/store_heap_oop. That seems wrong. Now it seems like it works because you >> never pass in any decorators to do_oop_store/load. For example the array accesses have >> IN_HEAP_ARRAY, which causes card marking barriers to use precise marking instead. But it looks like >> they will never learn that this was indeed an array as the decorators are not passed along. > > Argh. It is worse than that: there seems to be the implicit conversion of "bool is_null" to > "DecoratorSet set", which is wildly wrong. Fixed like this: > http://cr.openjdk.java.net/~shade/8201786/webrev.04.diff > > Full webrev: > http://cr.openjdk.java.net/~shade/8201786/webrev.04/ > > arm32-hflt seems to still build/work :) > > Thanks, > -Aleksey > From shade at redhat.com Wed May 2 17:46:49 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 2 May 2018 19:46:49 +0200 Subject: RFR 8201786: Modularize interpreter GC barriers: leftovers for ARM32 In-Reply-To: References: <4d22aacb-aa0d-4423-1a22-775e2be7eeaf@redhat.com> <1524847169.9114.6.camel@gmail.com> <668e8a59-cba1-a2ce-48a0-cf9cfcde032b@redhat.com> <1524866512.9114.10.camel@gmail.com> <828a25d9-d4b1-ec05-71e3-ceddef4b7a24@redhat.com> <89150A82-E74C-4046-ABF8-80AAA0C86744@oracle.com> <6ea56b54-cfa3-0222-e220-768c8cc2804a@redhat.com> <5AE9963E.8010000@oracle.com> <5AE99DA8.7010502@oracle.com> Message-ID: <324f666d-6f7c-77aa-c9d8-6d5c0aa72971@redhat.com> On 05/02/2018 07:20 PM, Erik Osterlund wrote: > Hi Aleksey, > > Looks good. Excellent, thanks! Pushed this and the CMS cleanup change. -Aleksey From martin.doerr at sap.com Wed May 2 17:53:34 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 2 May 2018 17:53:34 +0000 Subject: RFR(L) 8195099: Low latency hashtable for read-mostly scenarios In-Reply-To: <4cf61dce-3f8e-9baf-6278-68002f615e84@oracle.com> References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <0CCE37A8-A9AA-414E-9898-5B875B647BE6@oracle.com> <3e8cc7c3-f7fd-e117-8ef2-1e016125cf66@oracle.com> <4cf61dce-3f8e-9baf-6278-68002f615e84@oracle.com> Message-ID: Hi Robbin, thanks for contributing and especially for having quite a few memory barriers already in place. Yes, C++11 Atomics would be nice. The current version accesses _next without any compiler barriers so I'm kind of concerned about compilers which tend to reload and reorder accesses. We should double-check if nothing can go wrong when not using volatile. I guess this code hasn't been tested on weak memory platforms? So we should better take a close look at the barriers. It'd be great if we could use precise ordering semantics for the Atomic::cmpxchg and Atomic::add (which is currently being reviewed as JDK-8202080). Best regards, Martin -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Robbin Ehn Sent: Mittwoch, 2. Mai 2018 15:57 To: David Holmes ; Gerard Ziemski Cc: hotspot-dev developers Subject: Re: RFR(L) 8195099: Low latency hashtable for read-mostly scenarios On 2018-05-02 14:40, David Holmes wrote: > Not a review, one comment ... > > On 2/05/2018 6:31 PM, Robbin Ehn wrote: >> On 2018-04-30 21:23, Gerard Ziemski wrote: >>> #3 How come "_first" doesn't need to be volatile to be used in Atomic::cmpxchg ? >>> >>> Node* _first; >>> ... >>> ? if (Atomic::cmpxchg(node, &_first, expect) == expect) { >> >> An implicit cast to the same compatible type which is volatile-qualified is okey. > > Any variable used in a lock-free manner should be declared volatile as general > good practice IMHO. Generally I agree with you. But in this case I didn't find the volatile qualifier helpful, it just clobbered the code. I didn't see any readability improvements from having essential all lines with Node* also having volatile in them. What I would like is a type that forces the use of the correct ordering, e.g. Atomic _next; /Robbin > > David > ----- > >>> >>> >>> #4 Inconsistent parameter names - in: >>> >>> ? template >>> ? bool get_insert_lazy(Thread* thread, LOOKUP_FUNC& lookup, VALUE_FUNC& val_f, >>> ?????????????????????? bool* grow_hint = NULL) { >>> ??? return get_insert_lazy(thread, lookup, val_f, noOp, grow_hint); >>> ? } >>> >>> We use "val_f" with "_f? suffix, so we should have "lookup_f", not "lookup" >>> (many other instances) and "found_f", not "foundf" in >>> >>> ? template >>> ? bool get(Thread* thread, LOOKUP_FUNC& lookup, FOUND_FUNC& foundf, >>> ?????????? bool* grow_hint = NULL); >>> >> >> Fixed. >> >>> >>> #5 I wasn't familiar with CRTP, so I read up on it, but I still don't see the >>> CRTP in BaseConfig - which is the base class and which is derived? In >>> particular I don't see "class Derived : public Base" pattern here? >> >> It's the supplied config that extends the BaseConfig which is the derived. >> >> In gtest (test_concurrentHashtable.cpp) you will see this: >> >> // Simplest working CRPT implementation for the hash-table. >> struct Pointer : public SimpleTestTable::BaseConfig { >> >>> >>> More to come? >>> >>> >>> btw. I edited the subject of the email slightly by adding the bug number to >>> it, so it?s easy to search for. >>> >> >> Thanks, I'll send the update to the original mail. >> >> /Robbin >> >> >> >>> >>> cheers >>> >>> >>>> On Apr 26, 2018, at 2:38 AM, Robbin Ehn wrote: >>>> >>>> >>>> Hi all, please review. >>>> >>>> The lower latency of the new gcs have higher demands on runtime data-structure >>>> in terms of latency. This is a concurrent hashtable using the global-counter >>>> (8195099). >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8195098 >>>> >>>> * Readers never blocks or spins. >>>> * Inserts do CAS from a read-side, need to re-CAS during re-size/deletion in >>>> targeted bucket or insert collision. (inserts are conceptually a reader) >>>> * Deletes locks the targeted bucket, all other buckets are free for operations. >>>> * Re-size locks one bucket at the time, all other buckets are free for >>>> operations. >>>> >>>> It does concurrent re-size by doubling size and use one more bit from hash. >>>> That means a bucket in the old table contains nodes to either one of two >>>> buckets >>>> in the new table. Each node in the chain is sorted into one of the two buckets >>>> with concurrent readers. Shrinking just take two node chains and put them >>>> together in the corresponding bucket. To keep track of what is visible to the >>>> concurrent readers it's uses a global-counter, needed during re-size and for >>>> deletes. >>>> >>>> A gtest is provided which passes on our platforms, we also have a prototype >>>> of the stringtable using this which passes tier 1-5 on our platforms. >>>> >>>> Various people have pre-reviewed various parts, thanks! And a special thanks to >>>> Coleen for a lot of reviewing! >>>> >>>> Thanks, Robbin >>> From vladimir.kozlov at oracle.com Wed May 2 17:59:50 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 2 May 2018 10:59:50 -0700 Subject: RFR(L) : 8199375 : [TESTBUG] Open source vm testbase monitoring tests In-Reply-To: <395AD9F0-DBBF-4FCC-A01F-9C0D42B83BBD@oracle.com> References: <0BDC93A6-7D0E-4B4B-A797-936A58A296B9@oracle.com> <395AD9F0-DBBF-4FCC-A01F-9C0D42B83BBD@oracle.com> Message-ID: I wish we have ability to include other files with definitions into TEST.group file. It is very ugly to double size of TEST.group file just for that purpose. Thanks, Vladimir On 5/1/18 9:39 PM, Igor Ignatev wrote: > Vladimir, > > Tests are listed only in _quick test group b/c it doesn?t include all tests from the directory. We use this group in some of our test configurations, and :vmTestbase_nsk_monitoring in others. vmTestbase_nsk_monitoring is defined by the directory as other groups. > > Thanks, > ? Igor > >> On May 1, 2018, at 7:54 PM, Vladimir Kozlov wrote: >> >> Igor, >> >> Why you need to list each test in TEST.groups and not just directory as we do in other cases? >> >> Thanks, >> Vladimir >> >>> On 5/1/18 7:10 PM, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev//8199375/webrev.00/index.html >>>> 41276 lines changed: 41274 ins; 1 del; 1 mod; >>> Hi all, >>> could you please review the patch which open sources monitoring tests from vm testbase? >>> The tests were developed to test hotspot related JMX functionality. as w/ common VM testbase code, these tests are old, they have been run from hotspot testing for a long period of time. originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. in a long term, we are planning to rework them. >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8199375 >>> webrev: http://cr.openjdk.java.net/~iignatyev/8199375/webrev.00/index.html >>> testing: vmTestbase_nsk_monitoring test group >>> Thanks, >>> -- Igor > From igor.ignatyev at oracle.com Wed May 2 18:57:48 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 2 May 2018 11:57:48 -0700 Subject: RFR(L) : 8199375 : [TESTBUG] Open source vm testbase monitoring tests In-Reply-To: References: <0BDC93A6-7D0E-4B4B-A797-936A58A296B9@oracle.com> <395AD9F0-DBBF-4FCC-A01F-9C0D42B83BBD@oracle.com> Message-ID: <97CBC6A4-0E3C-4D92-BE29-4FC67FAE574F@oracle.com> Vladimir, we can introduce 'quick' keyword, mark these tests w/ them and use this keyword in test selection. I personally don't like this way either, as it uses a loosely defined property. it also might be possible to create a separate test group file and use it to define only _quick groups. I'll look into these and other possible options. in any case, although it will create a temporary mess, I'd prefer not to change how we define these *_quick test groups or how we do test selection, until all vm testbase tests are open sourced as it might create unneeded complications w/ test execution. I'll file an RFE to not forget about it. Thanks, -- Igor > On May 2, 2018, at 10:59 AM, Vladimir Kozlov wrote: > > I wish we have ability to include other files with definitions into TEST.group file. It is very ugly to double size of TEST.group file just for that purpose. > > Thanks, > Vladimir > > On 5/1/18 9:39 PM, Igor Ignatev wrote: >> Vladimir, >> Tests are listed only in _quick test group b/c it doesn?t include all tests from the directory. We use this group in some of our test configurations, and :vmTestbase_nsk_monitoring in others. vmTestbase_nsk_monitoring is defined by the directory as other groups. >> Thanks, >> ? Igor >>> On May 1, 2018, at 7:54 PM, Vladimir Kozlov wrote: >>> >>> Igor, >>> >>> Why you need to list each test in TEST.groups and not just directory as we do in other cases? >>> >>> Thanks, >>> Vladimir >>> >>>> On 5/1/18 7:10 PM, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev//8199375/webrev.00/index.html >>>>> 41276 lines changed: 41274 ins; 1 del; 1 mod; >>>> Hi all, >>>> could you please review the patch which open sources monitoring tests from vm testbase? >>>> The tests were developed to test hotspot related JMX functionality. as w/ common VM testbase code, these tests are old, they have been run from hotspot testing for a long period of time. originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. in a long term, we are planning to rework them. >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8199375 >>>> webrev: http://cr.openjdk.java.net/~iignatyev/8199375/webrev.00/index.html >>>> testing: vmTestbase_nsk_monitoring test group >>>> Thanks, >>>> -- Igor From vladimir.kozlov at oracle.com Wed May 2 19:13:06 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 2 May 2018 12:13:06 -0700 Subject: RFR(L) : 8199375 : [TESTBUG] Open source vm testbase monitoring tests In-Reply-To: <97CBC6A4-0E3C-4D92-BE29-4FC67FAE574F@oracle.com> References: <0BDC93A6-7D0E-4B4B-A797-936A58A296B9@oracle.com> <395AD9F0-DBBF-4FCC-A01F-9C0D42B83BBD@oracle.com> <97CBC6A4-0E3C-4D92-BE29-4FC67FAE574F@oracle.com> Message-ID: <9ebc00f5-d0fb-df28-f9b8-0f6c945f17e3@oracle.com> On 5/2/18 11:57 AM, Igor Ignatyev wrote: > Vladimir, > > we can introduce 'quick' keyword, mark these tests w/ them and use this keyword in test selection. I personally don't like this way either, as it uses a loosely defined property. it also might be possible to create a separate test group file and use it to define only _quick groups. I'll look into these and other possible options. > > in any case, although it will create a temporary mess, I'd prefer not to change how we define these *_quick test groups or how we do test selection, until all vm testbase tests are open sourced as it might create unneeded complications w/ test execution. I'll file an RFE to not forget about it. Yes, this sounds good. Reviewed. Thanks, Vladimir > > Thanks, > -- Igor > >> On May 2, 2018, at 10:59 AM, Vladimir Kozlov wrote: >> >> I wish we have ability to include other files with definitions into TEST.group file. It is very ugly to double size of TEST.group file just for that purpose. >> >> Thanks, >> Vladimir >> >> On 5/1/18 9:39 PM, Igor Ignatev wrote: >>> Vladimir, >>> Tests are listed only in _quick test group b/c it doesn?t include all tests from the directory. We use this group in some of our test configurations, and :vmTestbase_nsk_monitoring in others. vmTestbase_nsk_monitoring is defined by the directory as other groups. >>> Thanks, >>> ? Igor >>>> On May 1, 2018, at 7:54 PM, Vladimir Kozlov wrote: >>>> >>>> Igor, >>>> >>>> Why you need to list each test in TEST.groups and not just directory as we do in other cases? >>>> >>>> Thanks, >>>> Vladimir >>>> >>>>> On 5/1/18 7:10 PM, Igor Ignatyev wrote: >>>>> http://cr.openjdk.java.net/~iignatyev//8199375/webrev.00/index.html >>>>>> 41276 lines changed: 41274 ins; 1 del; 1 mod; >>>>> Hi all, >>>>> could you please review the patch which open sources monitoring tests from vm testbase? >>>>> The tests were developed to test hotspot related JMX functionality. as w/ common VM testbase code, these tests are old, they have been run from hotspot testing for a long period of time. originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. in a long term, we are planning to rework them. >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8199375 >>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8199375/webrev.00/index.html >>>>> testing: vmTestbase_nsk_monitoring test group >>>>> Thanks, >>>>> -- Igor > From zgu at redhat.com Wed May 2 19:41:10 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Wed, 2 May 2018 15:41:10 -0400 Subject: RFR 8199868: Support JNI critical functions in object pinning API In-Reply-To: <19c791e7-1bf8-69ef-7090-b7da800f2021@redhat.com> References: <931060af-b44d-f348-92ba-e98d623d4c84@redhat.com> <19c791e7-1bf8-69ef-7090-b7da800f2021@redhat.com> Message-ID: Hi, Can I have reviews for this RFR? This patch completes object pinning for JNI critical section, provides critical native support. The approach is quite straightforward: During generating native wrapper for critical native method, it generates runtime call to pin every array argument, before unpacks them. For pinned objects, it also needs to save them for unpinning after JNI function call completes. If argument is passed on stack, it saves pinned object at the original slot (as pin_object() may move the object). For register based arguments, it reuses oop handle area (where GCLocker based implementation saves register based arguments for safepoints). Currently, only Shenandoah uses object pinning for JNI critical section, this patch has been baked quite some time there. However, I am new to Runtime Stub code, I would appreciate your comments and suggestions. I rebased patch to jdk/jdk repo. Webrev: http://cr.openjdk.java.net/~zgu/8199868/webrev.02/ Thanks, -Zhengyu On 04/06/2018 10:35 PM, Zhengyu Gu wrote: > Offline discussion with Aleksey, he suggested that > pin/unpin_critical_native_array methods can be made more generic as > pin/unpin_object. > > Updated webrev: http://cr.openjdk.java.net/~zgu/8199868/webrev.01/ > > Test: > ? Reran all tests, submit-hs tests still clean. > > Thanks, > > -Zhengyu > > On 04/06/2018 08:55 AM, Aleksey Shipilev wrote: >> On 04/04/2018 07:47 PM, Zhengyu Gu wrote: >>> Please review this patch that adds JNI critical native support to >>> object pinning. >>> >>> Shenandoah does not block GC while JNI critical session is in >>> progress. This patch allows it to pin >>> all incoming array objects before critical native call, and unpin >>> them after call completes. >>> >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199868 >>> Webrev: http://cr.openjdk.java.net/~zgu/8199868/webrev.00/ >> >> Looks good to me, but somebody more savvy with runtime stub generation >> should take a closer look. >> >> *) Should probably be "Why we are here?" >> >> 2867?? assert(Universe::heap()->supports_object_pinning(), "Why we >> here?"); >> >> 2876?? assert(Universe::heap()->supports_object_pinning(), "Why we >> here?"); >> >> >> Thanks, >> -Aleksey >> From kim.barrett at oracle.com Wed May 2 20:40:22 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 2 May 2018 16:40:22 -0400 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> Message-ID: > On May 2, 2018, at 5:10 AM, Michal Vala wrote: > > > > On 05/01/2018 07:59 PM, Kim Barrett wrote: >>> On Apr 27, 2018, at 4:26 PM, Michal Vala wrote: >>> Someone to sponsor this please? >> Do you have a sponsor yet? If not, I?ll do it. > > No, I don't. I'd really appreciate if you sponsor it. Will do. I should be able to take care of it in the next day or so. From mikhailo.seledtsov at oracle.com Wed May 2 21:05:12 2018 From: mikhailo.seledtsov at oracle.com (mikhailo) Date: Wed, 2 May 2018 14:05:12 -0700 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: References: <5AE0612F.8060200@oracle.com> Message-ID: <8815301e-9aec-080e-9960-e52718ddd2b3@oracle.com> Hi Erik, My review is based on: http://cr.openjdk.java.net/~egahlin/8199712.0/ I looked at the test portion only. Overall looks good, however I have some comments: ? 1. A number of JFC files still have "internal copyright header" ???? E.g.: open/test/jdk/jdk/jfr/event/gc/collection/gc-testsettings.jfc ???? ???? More .jfc files under open/test/jdk/jdk/jfr/ have same issue, or no copyright header at all: ???????? ./event/gc/collection/gc-testsettings.jfc ???????? ./event/gc/detailed/promotionfailed-testsettings.jfc ???????? ./event/gc/detailed/evacuationfailed-testsettings.jfc ./event/gc/detailed/concurrentmodefailure-testsettings.jfc ???????? ./api/recording/settings/settings.jfc ???????? ./jcmd/jcmd-testsettings.2.jfc ???????? ./jcmd/jcmd-testsettings.jfc ???????? ./jcmd/jcmd-testsettings3.jfc ? 2. The following files are missing copyright statement: ????? /open/test/jdk/jdk/jfr/event/io/MakeJAR.sh ? 3. Please update the copyright year: ???? test/lib/jdk/test/lib/thread/TestThread.java ???? test/lib/jdk/test/lib/thread/XRun.java The rest of test related files look good to me, Misha On 05/01/2018 04:37 PM, Vladimir Kozlov wrote: > Hi Erik, > > I am working on 8184349 and adding? & !vm.graal.enabled to @requires > for tests which use CMS. Mostly they are JFR tests which are in this > review list. And I found some test have incorrect commands (merged 2 > lines together): > > ?* @test @requires vm.gc == "null" | vm.gc == "Serial" > > I see that you fixed some. But there few left: > > test/jdk/jdk/jfr/event/gc/detailed/TestStressAllocationGCEventsWithDefNew.java > > test/jdk/jdk/jfr/event/gc/detailed/TestStressAllocationGCEventsWithG1.java > > test/jdk/jdk/jfr/event/gc/detailed/TestStressBigAllocationGCEventsWithParallel.java > > > Thanks, > Vladimir > > On 4/25/18 4:06 AM, Erik Gahlin wrote: >> Greetings, >> >> Could I have a review of 8199712: Flight Recorder >> >> As mentioned in the preview [1] the tracing backend has been removed. >> Event metadata has been consolidated into a single XML file and event >> classes are now generated by GenerateJfrFiles.java. >> >> Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. >> >> For details about the feature, see the JEP: >> https://bugs.openjdk.java.net/browse/JDK-8193393 >> >> Webrev: >> http://cr.openjdk.java.net/~egahlin/8199712.0/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8199712 >> >> [1] >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359.html >> >> Thanks >> Erik and Markus From per.liden at oracle.com Wed May 2 21:12:15 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 2 May 2018 23:12:15 +0200 Subject: RFR 8199868: Support JNI critical functions in object pinning API In-Reply-To: References: <931060af-b44d-f348-92ba-e98d623d4c84@redhat.com> <19c791e7-1bf8-69ef-7090-b7da800f2021@redhat.com> Message-ID: <16c60f53-d835-29f1-981f-4f70537ceb62@oracle.com> Hi, On 05/02/2018 09:41 PM, Zhengyu Gu wrote: > Hi, > > Can I have reviews for this RFR? > > This patch completes object pinning for JNI critical section, provides > critical native support. > > The approach is quite straightforward: > > During generating native wrapper for critical native method, it > generates runtime call to pin every array argument, before unpacks them. > > For pinned objects, it also needs to save them for unpinning after JNI > function call completes. > > If argument is passed on stack, it saves pinned object at the original > slot (as pin_object() may move the object). For register based > arguments, it reuses oop handle area (where GCLocker based > implementation saves register based arguments for safepoints). > > Currently, only Shenandoah uses object pinning for JNI critical section, > this patch has been baked quite some time there. However, I am new to > Runtime Stub code, I would appreciate your comments and suggestions. > > I rebased patch to jdk/jdk repo. > > Webrev: http://cr.openjdk.java.net/~zgu/8199868/webrev.02/ Just want to say that I would really like to see this patch go in. As mentioned, it completes the object pinning story and it's useful for other GCs too (at least ZGC and possibly G1). However, I also agree with Aleksey that some one who really knows this code needs to review this. Unfortunately that's not me. Anyone? cheers, Per > > Thanks, > > -Zhengyu > > > On 04/06/2018 10:35 PM, Zhengyu Gu wrote: >> Offline discussion with Aleksey, he suggested that >> pin/unpin_critical_native_array methods can be made more generic as >> pin/unpin_object. >> >> Updated webrev: http://cr.openjdk.java.net/~zgu/8199868/webrev.01/ >> >> Test: >> Reran all tests, submit-hs tests still clean. >> >> Thanks, >> >> -Zhengyu >> >> On 04/06/2018 08:55 AM, Aleksey Shipilev wrote: >>> On 04/04/2018 07:47 PM, Zhengyu Gu wrote: >>>> Please review this patch that adds JNI critical native support to >>>> object pinning. >>>> >>>> Shenandoah does not block GC while JNI critical session is in >>>> progress. This patch allows it to pin >>>> all incoming array objects before critical native call, and unpin >>>> them after call completes. >>>> >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199868 >>>> Webrev: http://cr.openjdk.java.net/~zgu/8199868/webrev.00/ >>> >>> Looks good to me, but somebody more savvy with runtime stub >>> generation should take a closer look. >>> >>> *) Should probably be "Why we are here?" >>> >>> 2867 assert(Universe::heap()->supports_object_pinning(), "Why we >>> here?"); >>> >>> 2876 assert(Universe::heap()->supports_object_pinning(), "Why we >>> here?"); >>> >>> >>> Thanks, >>> -Aleksey >>> From serguei.spitsyn at oracle.com Wed May 2 21:18:56 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 2 May 2018 14:18:56 -0700 Subject: RFR(L) : 8199375 : [TESTBUG] Open source vm testbase monitoring tests In-Reply-To: <0BDC93A6-7D0E-4B4B-A797-936A58A296B9@oracle.com> References: <0BDC93A6-7D0E-4B4B-A797-936A58A296B9@oracle.com> Message-ID: <8fd2e124-47c9-98ff-90e8-104f1a78ed28@oracle.com> Hi Igor, It looks good. Thanks, Serguei On 5/1/18 19:10, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8199375/webrev.00/index.html >> 41276 lines changed: 41274 ins; 1 del; 1 mod; > Hi all, > > could you please review the patch which open sources monitoring tests from vm testbase? > > The tests were developed to test hotspot related JMX functionality. as w/ common VM testbase code, these tests are old, they have been run from hotspot testing for a long period of time. originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. in a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199375 > webrev: http://cr.openjdk.java.net/~iignatyev/8199375/webrev.00/index.html > testing: vmTestbase_nsk_monitoring test group > > Thanks, > -- Igor From coleen.phillimore at oracle.com Wed May 2 21:42:52 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 2 May 2018 17:42:52 -0400 Subject: RFR: 8200557: OopStorage parallel iteration scales poorly In-Reply-To: <2C30CDA1-7070-49A3-9D40-8625EDBA582E@oracle.com> References: <7CBB6FF2-AC36-4723-BBA1-B4FA277C707B@oracle.com> <2C30CDA1-7070-49A3-9D40-8625EDBA582E@oracle.com> Message-ID: <63eef203-796d-a6cb-6c40-baee5e7a299c@oracle.com> Changes look good. On 5/1/18 5:13 PM, Kim Barrett wrote: >> On Apr 25, 2018, at 5:30 PM, coleen.phillimore at oracle.com wrote: >> >> >> http://cr.openjdk.java.net/~kbarrett/8200557/open.00/src/hotspot/share/gc/shared/oopStorage.cpp.udiff.html >> >> +OopStorage::BasicParState::IterationData::IterationData() : >> + _segment_start(0), >> + _segment_end(0), >> + _processed(0) >> +{} >> >> >> Can this constructor be inlined in the declaration? This seems out of place here (and I had to hunt for it). > I'm changing IterationData to be a simple POD-struct. It's very > limited usage scope makes more than that overkill. http://cr.openjdk.java.net/~kbarrett/8200557/open.01.inc/src/hotspot/share/gc/shared/oopStorageParState.inline.hpp.udiff.html Just because it's a POD, you could still have the interesting function? - void note_processed_segment() { _processed += _segment_end - _segment_start; } > >> Can some other simple accessors be inlined in the declarations as well? That would decrease jumping around trying to understand things. >> >> I think the OopStorage::BasicParState functions should be in a new .cpp file since there's now *src/hotspot/share/gc/shared/oopStorageParState.hpp >> *Can you do this as a follow up/cleanup RFE? > I would rather not. The headers were split up to reduce include > dependencies. That's not so much of an issue for oopStorage.cpp. And > the implementation of OopStorage::BasicParState is pretty intimately > tied to the rest of OopStorage. All right. > >> http://cr.openjdk.java.net/~kbarrett/8200557/open.00/src/hotspot/share/gc/shared/oopStorage.hpp.udiff.html >> >> class Block; // Forward decl; defined in .inline.hpp file. >> + class BlockArray; // Forward decl; defined in .inline.hpp file. >> class BlockList; // Forward decl for BlockEntry friend decl. >> >> Can you give a short description of these, ie: >> >> class Block; // Block of oop* stored in OopStorage >> + class BlockArray; // Storage for all active oop Blocks >> class BlockList; // List of oop Blocks from which are available for allocation >> >> Or maybe a short description over the class declarations? The trouble with these generic names is you forget why they exist. > I've added some/more descriptions, and moved some to hopefully be > better placed to be found. I like the new comments.? I still think pointing out that these are forward declarations and not what they are used for here is not very interesting, though.? A short 1/2 liner as suggested would be better in case one doesn't need to read the rest of the code. > >> I'd call BlockList a AllocationBlockList and BlockArray an ActiveBlockArray > For naming, I think BlockArray => ActiveArray, BlockList => > AllocateList, BlockEntry => AllocateListEntry. Recall that before > this change there wasn't an active *array*. rather there were two > BlockLists, active and allocate, so the generic name made more sense > then. I'll do this as a followup RFE; there's also a simplification > to be made to BlockList, now that there's only one, which I was > planning to do as a followup. That would be fine.? Hopefully an easier code review :) > >> http://cr.openjdk.java.net/~kbarrett/8200557/open.00/src/hotspot/share/gc/shared/oopStorage.inline.hpp.frames.html >> >> 38 class OopStorage::BlockArray { >> >> >> This would be a good place to comment the purpose of this and a line about refcounting. >> >> http://cr.openjdk.java.net/~kbarrett/8200557/open.00/src/hotspot/share/gc/shared/oopStorage.hpp.frames.html >> >> Why does BlockList need to be included in the .hpp file? > BlockList needs to be in the .hpp file because there is a member in > OopStorage, requiring it to be complete. Block and Block can be > forward declared. But I now realize that BlockEntry could also be > forward-declared and moved to the .inline.hpp file. > >> http://cr.openjdk.java.net/~kbarrett/8200557/open.00/src/hotspot/share/gc/shared/oopStorage.cpp.frames.html >> >> All these places have block_count() which are indistinguishable. It looks like _block_count for the BlockArray should always be acquired outside of BlockArray class or rather give that as the only public option to prevent mistakes. It seems like there wouldn't be much performance cost to calling load_acquire once at the safepoint iteration. > block_count() (and _block_count) are used in places where we know > we're in a safepoint or that we hold the _allocate_mutex. > block_count_acquire() is used where neither of those is true. > > Responding to Erik's questions about block_count() vs > block_count_acquire(), OopStorage::block_count() and > OopStorage::total_memory_usage() are in the latter case. I've waffled > over whether it is better to use an unnecessary acquire or possibly > need to explain why it isn't necessary. Of course, then one may need > to explain an apparently unnecessary acquire. > > After looking over things in this area yet again, I think there are > some places where it's better to directly use the _block_count member > rather than an accessor function anyway, so I'm doing that. And an > unnecessary acquire seems like it needs as much explaination as not > having it, so removing those too. Okay this is less confusing about when you need acquire vs not. It's volatile so that's good. >> Comment about _why_ refcounts and what do they do here would be nice. >> >> 151 void OopStorage::BlockArray::increment_refcount() const { >> >> >> 567 void OopStorage::relinquish_block_array(BlockArray* array) const { >> 568 if (array->decrement_refcount()) { >> 569 assert(array != _active_array, "invariant"); >> 570 BlockArray::destroy(array); >> 571 } >> 572 } >> >> >> Here array is a pointer to the active array so iterators could destroy the array if they are the last ones to use it. Is this right? > Yes. > >> I've read through most of the rest of the code and it looks good apart from how complicated it is, which I don't know how to suggest to improve it other than some reminder comments and maybe more descriptive names. > New webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8200557/open.00/ > incr: http://cr.openjdk.java.net/~kbarrett/8200557/open.01.inc/ > > Thanks for moving the comments into the implementation.? I like where they are now. If you decide to update any more comments, I don't need to see another webrev. thanks, Coleen From vladimir.kozlov at oracle.com Wed May 2 22:13:51 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 2 May 2018 15:13:51 -0700 Subject: [11] RFR(S) 8202552: [AOT][JVMCI] Incorrect usage of INCLUDE_JVMCI and INCLUDE_AOT Message-ID: http://cr.openjdk.java.net/~kvn/8202552/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8202552 Stefan K. found several places where #ifdef instead of #if is used for INCLUDE_JVMCI. I also found places where we can use COMPILER2_OR_JVMCI. An other problem surprised me that we don't set INCLUDE_AOT to 1. Makefile defines that variable without value. I changed code to match other variables setting - in makefile set it to 0 if it is not part of build and set it to 1 in macros.hpp if it is not defined. Tested with tier1, tier2 and tier2 with Graal as JIT compiler. -- Thanks, Vladimir From david.holmes at oracle.com Wed May 2 22:17:41 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 3 May 2018 08:17:41 +1000 Subject: RFR(L) 8195099: Low latency hashtable for read-mostly scenarios In-Reply-To: <4cf61dce-3f8e-9baf-6278-68002f615e84@oracle.com> References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <0CCE37A8-A9AA-414E-9898-5B875B647BE6@oracle.com> <3e8cc7c3-f7fd-e117-8ef2-1e016125cf66@oracle.com> <4cf61dce-3f8e-9baf-6278-68002f615e84@oracle.com> Message-ID: <44b88b98-081c-5fb2-d57d-fa17bfa57160@oracle.com> On 2/05/2018 11:56 PM, Robbin Ehn wrote: > On 2018-05-02 14:40, David Holmes wrote: >> Not a review, one comment ... >> >> On 2/05/2018 6:31 PM, Robbin Ehn wrote: >>> On 2018-04-30 21:23, Gerard Ziemski wrote: >>>> #3 How come "_first" doesn't need to be volatile to be used in >>>> Atomic::cmpxchg ? >>>> >>>> Node* _first; >>>> ... >>>> ? if (Atomic::cmpxchg(node, &_first, expect) == expect) { >>> >>> An implicit cast to the same compatible type which is >>> volatile-qualified is okey. >> >> Any variable used in a lock-free manner should be declared volatile as >> general good practice IMHO. > > Generally I agree with you. But in this case I didn't find the volatile > qualifier helpful, it just clobbered the code. I didn't see any readability > improvements from having essential all lines with Node* also having > volatile in them. I'm not concerned with readability I'm concerned with doing the minimum to try to ensure that a variable that can be concurrently mutated and accessed in a lock-free manner is at least flagged to the compiler as something it should be careful about dealing with! > What I would like is a type that forces the use of the correct ordering, > e.g. > Atomic _next; Meanwhile Node * volatile _next; is the closest thing we have. David ----- > /Robbin > >> >> David >> ----- >> >>>> >>>> >>>> #4 Inconsistent parameter names - in: >>>> >>>> ? template >>>> ? bool get_insert_lazy(Thread* thread, LOOKUP_FUNC& lookup, >>>> VALUE_FUNC& val_f, >>>> ?????????????????????? bool* grow_hint = NULL) { >>>> ??? return get_insert_lazy(thread, lookup, val_f, noOp, grow_hint); >>>> ? } >>>> >>>> We use "val_f" with "_f? suffix, so we should have "lookup_f", not >>>> "lookup" (many other instances) and "found_f", not "foundf" in >>>> >>>> ? template >>>> ? bool get(Thread* thread, LOOKUP_FUNC& lookup, FOUND_FUNC& foundf, >>>> ?????????? bool* grow_hint = NULL); >>>> >>> >>> Fixed. >>> >>>> >>>> #5 I wasn't familiar with CRTP, so I read up on it, but I still >>>> don't see the CRTP in BaseConfig - which is the base class and which >>>> is derived? In particular I don't see "class Derived : public >>>> Base" pattern here? >>> >>> It's the supplied config that extends the BaseConfig which is the >>> derived. >>> >>> In gtest (test_concurrentHashtable.cpp) you will see this: >>> >>> // Simplest working CRPT implementation for the hash-table. >>> struct Pointer : public SimpleTestTable::BaseConfig { >>> >>>> >>>> More to come? >>>> >>>> >>>> btw. I edited the subject of the email slightly by adding the bug >>>> number to it, so it?s easy to search for. >>>> >>> >>> Thanks, I'll send the update to the original mail. >>> >>> /Robbin >>> >>> >>> >>>> >>>> cheers >>>> >>>> >>>>> On Apr 26, 2018, at 2:38 AM, Robbin Ehn wrote: >>>>> >>>>> >>>>> Hi all, please review. >>>>> >>>>> The lower latency of the new gcs have higher demands on runtime >>>>> data-structure >>>>> in terms of latency. This is a concurrent hashtable using the >>>>> global-counter >>>>> (8195099). >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8195098 >>>>> >>>>> * Readers never blocks or spins. >>>>> * Inserts do CAS from a read-side, need to re-CAS during >>>>> re-size/deletion in targeted bucket or insert collision. (inserts >>>>> are conceptually a reader) >>>>> * Deletes locks the targeted bucket, all other buckets are free for >>>>> operations. >>>>> * Re-size locks one bucket at the time, all other buckets are free >>>>> for operations. >>>>> >>>>> It does concurrent re-size by doubling size and use one more bit >>>>> from hash. >>>>> That means a bucket in the old table contains nodes to either one >>>>> of two buckets >>>>> in the new table. Each node in the chain is sorted into one of the >>>>> two buckets >>>>> with concurrent readers. Shrinking just take two node chains and >>>>> put them >>>>> together in the corresponding bucket. To keep track of what is >>>>> visible to the >>>>> concurrent readers it's uses a global-counter, needed during >>>>> re-size and for >>>>> deletes. >>>>> >>>>> A gtest is provided which passes on our platforms, we also have a >>>>> prototype of the stringtable using this which passes tier 1-5 on >>>>> our platforms. >>>>> >>>>> Various people have pre-reviewed various parts, thanks! And a >>>>> special thanks to >>>>> Coleen for a lot of reviewing! >>>>> >>>>> Thanks, Robbin >>>> From vladimir.kozlov at oracle.com Wed May 2 22:19:42 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 2 May 2018 15:19:42 -0700 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: <8815301e-9aec-080e-9960-e52718ddd2b3@oracle.com> References: <5AE0612F.8060200@oracle.com> <8815301e-9aec-080e-9960-e52718ddd2b3@oracle.com> Message-ID: Test serviceability/jfr/TestGCOldWithJFR.java should be also ported. Vladimir K On 5/2/18 2:05 PM, mikhailo wrote: > Hi Erik, > > My review is based on: http://cr.openjdk.java.net/~egahlin/8199712.0/ > I looked at the test portion only. > Overall looks good, however I have some comments: > > ? 1. A number of JFC files still have "internal copyright header" > ???? E.g.: open/test/jdk/jdk/jfr/event/gc/collection/gc-testsettings.jfc > ???? > ???? More .jfc files under open/test/jdk/jdk/jfr/ have same issue, or > no copyright header at all: > ???????? ./event/gc/collection/gc-testsettings.jfc > ???????? ./event/gc/detailed/promotionfailed-testsettings.jfc > ???????? ./event/gc/detailed/evacuationfailed-testsettings.jfc > ./event/gc/detailed/concurrentmodefailure-testsettings.jfc > ???????? ./api/recording/settings/settings.jfc > ???????? ./jcmd/jcmd-testsettings.2.jfc > ???????? ./jcmd/jcmd-testsettings.jfc > ???????? ./jcmd/jcmd-testsettings3.jfc > > > ? 2. The following files are missing copyright statement: > ????? /open/test/jdk/jdk/jfr/event/io/MakeJAR.sh > > ? 3. Please update the copyright year: > ???? test/lib/jdk/test/lib/thread/TestThread.java > ???? test/lib/jdk/test/lib/thread/XRun.java > > > The rest of test related files look good to me, > Misha > > > On 05/01/2018 04:37 PM, Vladimir Kozlov wrote: >> Hi Erik, >> >> I am working on 8184349 and adding? & !vm.graal.enabled to @requires >> for tests which use CMS. Mostly they are JFR tests which are in this >> review list. And I found some test have incorrect commands (merged 2 >> lines together): >> >> ?* @test @requires vm.gc == "null" | vm.gc == "Serial" >> >> I see that you fixed some. But there few left: >> >> test/jdk/jdk/jfr/event/gc/detailed/TestStressAllocationGCEventsWithDefNew.java >> >> test/jdk/jdk/jfr/event/gc/detailed/TestStressAllocationGCEventsWithG1.java >> >> test/jdk/jdk/jfr/event/gc/detailed/TestStressBigAllocationGCEventsWithParallel.java >> >> >> Thanks, >> Vladimir >> >> On 4/25/18 4:06 AM, Erik Gahlin wrote: >>> Greetings, >>> >>> Could I have a review of 8199712: Flight Recorder >>> >>> As mentioned in the preview [1] the tracing backend has been removed. >>> Event metadata has been consolidated into a single XML file and event >>> classes are now generated by GenerateJfrFiles.java. >>> >>> Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. >>> >>> For details about the feature, see the JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8193393 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~egahlin/8199712.0/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8199712 >>> >>> [1] >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359.html >>> >>> >>> Thanks >>> Erik and Markus > From stefan.karlsson at oracle.com Thu May 3 07:20:16 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 3 May 2018 09:20:16 +0200 Subject: [11] RFR(S) 8202552: [AOT][JVMCI] Incorrect usage of INCLUDE_JVMCI and INCLUDE_AOT In-Reply-To: References: Message-ID: Looks good to me. Thanks for fixing. StefanK On 2018-05-03 00:13, Vladimir Kozlov wrote: > http://cr.openjdk.java.net/~kvn/8202552/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8202552 > > Stefan K. found several places where #ifdef instead of #if is used for > INCLUDE_JVMCI. > I also found places where we can use COMPILER2_OR_JVMCI. > > An other problem surprised me that we don't set INCLUDE_AOT to 1. > Makefile defines that variable without value. I changed code to match > other variables setting - in makefile set it to 0 if it is not part of > build and set it to 1 in macros.hpp if it is not defined. > > Tested with tier1, tier2 and tier2 with Graal as JIT compiler. > From robbin.ehn at oracle.com Thu May 3 08:17:22 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 3 May 2018 10:17:22 +0200 Subject: RFR(L) 8195099: Low latency hashtable for read-mostly scenarios In-Reply-To: References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <0CCE37A8-A9AA-414E-9898-5B875B647BE6@oracle.com> <3e8cc7c3-f7fd-e117-8ef2-1e016125cf66@oracle.com> <4cf61dce-3f8e-9baf-6278-68002f615e84@oracle.com> Message-ID: <190cde76-63d2-fa0f-79ab-b9089e8c2cfe@oracle.com> Hi Martin, On 05/02/2018 07:53 PM, Doerr, Martin wrote: > Hi Robbin, > > thanks for contributing and especially for having quite a few memory barriers already in place. > > Yes, C++11 Atomics would be nice. > > The current version accesses _next without any compiler barriers so I'm kind of concerned about compilers which tend to reload and reorder accesses. We should double-check if nothing can go wrong when not using volatile. No, we read it via: return OrderAccess::load_acquire(&_next); In store case we trust the implied barriers from CAS: new_node->set_next(first_at_start); } if (bucket->cas_first(new_node, first_at_start)) { It now actually have a stronger than needed ordering. We do not always need acquire on load, but volatile semantics is always wrong, we either have stronger or lesser. E.g. in internal_remove loading the _next pointer should not need acquire. After CAS:ing in the lock bit, the chain is stable and we unlock with a release_store. Loads between the CAS and the release_store my be re-ordered since the chain is stable. > > I guess this code hasn't been tested on weak memory platforms? > So we should better take a close look at the barriers. An earlier version have been successfully tested on aarch64. But it did not have the CAS inserts. > > It'd be great if we could use precise ordering semantics for the Atomic::cmpxchg and Atomic::add (which is currently being reviewed as JDK-8202080). I did on purpose not do any special handling for platforms not following the documented semantics. My thought was either you would do 8202080 or just add needed barrier in e.g. Bucket::cas_first. Btw, I think 8202080 looks good, specially like conservative as default. With conservative as default on cmpxchg and add for global counter, this should work on weaker platforms, but not alpha. /Robbin > > Best regards, > Martin > > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Robbin Ehn > Sent: Mittwoch, 2. Mai 2018 15:57 > To: David Holmes ; Gerard Ziemski > Cc: hotspot-dev developers > Subject: Re: RFR(L) 8195099: Low latency hashtable for read-mostly scenarios > > On 2018-05-02 14:40, David Holmes wrote: >> Not a review, one comment ... >> >> On 2/05/2018 6:31 PM, Robbin Ehn wrote: >>> On 2018-04-30 21:23, Gerard Ziemski wrote: >>>> #3 How come "_first" doesn't need to be volatile to be used in Atomic::cmpxchg ? >>>> >>>> Node* _first; >>>> ... >>>> ? if (Atomic::cmpxchg(node, &_first, expect) == expect) { >>> >>> An implicit cast to the same compatible type which is volatile-qualified is okey. >> >> Any variable used in a lock-free manner should be declared volatile as general >> good practice IMHO. > > Generally I agree with you. But in this case I didn't find the volatile > qualifier helpful, it just clobbered the code. I didn't see any readability > improvements from having essential all lines with Node* also having volatile in > them. > > What I would like is a type that forces the use of the correct ordering, e.g. > Atomic _next; > > /Robbin > >> >> David >> ----- >> >>>> >>>> >>>> #4 Inconsistent parameter names - in: >>>> >>>> ? template >>>> ? bool get_insert_lazy(Thread* thread, LOOKUP_FUNC& lookup, VALUE_FUNC& val_f, >>>> ?????????????????????? bool* grow_hint = NULL) { >>>> ??? return get_insert_lazy(thread, lookup, val_f, noOp, grow_hint); >>>> ? } >>>> >>>> We use "val_f" with "_f? suffix, so we should have "lookup_f", not "lookup" >>>> (many other instances) and "found_f", not "foundf" in >>>> >>>> ? template >>>> ? bool get(Thread* thread, LOOKUP_FUNC& lookup, FOUND_FUNC& foundf, >>>> ?????????? bool* grow_hint = NULL); >>>> >>> >>> Fixed. >>> >>>> >>>> #5 I wasn't familiar with CRTP, so I read up on it, but I still don't see the >>>> CRTP in BaseConfig - which is the base class and which is derived? In >>>> particular I don't see "class Derived : public Base" pattern here? >>> >>> It's the supplied config that extends the BaseConfig which is the derived. >>> >>> In gtest (test_concurrentHashtable.cpp) you will see this: >>> >>> // Simplest working CRPT implementation for the hash-table. >>> struct Pointer : public SimpleTestTable::BaseConfig { >>> >>>> >>>> More to come? >>>> >>>> >>>> btw. I edited the subject of the email slightly by adding the bug number to >>>> it, so it?s easy to search for. >>>> >>> >>> Thanks, I'll send the update to the original mail. >>> >>> /Robbin >>> >>> >>> >>>> >>>> cheers >>>> >>>> >>>>> On Apr 26, 2018, at 2:38 AM, Robbin Ehn wrote: >>>>> >>>>> >>>>> Hi all, please review. >>>>> >>>>> The lower latency of the new gcs have higher demands on runtime data-structure >>>>> in terms of latency. This is a concurrent hashtable using the global-counter >>>>> (8195099). >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8195098 >>>>> >>>>> * Readers never blocks or spins. >>>>> * Inserts do CAS from a read-side, need to re-CAS during re-size/deletion in >>>>> targeted bucket or insert collision. (inserts are conceptually a reader) >>>>> * Deletes locks the targeted bucket, all other buckets are free for operations. >>>>> * Re-size locks one bucket at the time, all other buckets are free for >>>>> operations. >>>>> >>>>> It does concurrent re-size by doubling size and use one more bit from hash. >>>>> That means a bucket in the old table contains nodes to either one of two >>>>> buckets >>>>> in the new table. Each node in the chain is sorted into one of the two buckets >>>>> with concurrent readers. Shrinking just take two node chains and put them >>>>> together in the corresponding bucket. To keep track of what is visible to the >>>>> concurrent readers it's uses a global-counter, needed during re-size and for >>>>> deletes. >>>>> >>>>> A gtest is provided which passes on our platforms, we also have a prototype >>>>> of the stringtable using this which passes tier 1-5 on our platforms. >>>>> >>>>> Various people have pre-reviewed various parts, thanks! And a special thanks to >>>>> Coleen for a lot of reviewing! >>>>> >>>>> Thanks, Robbin >>>> From robbin.ehn at oracle.com Thu May 3 08:35:12 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 3 May 2018 10:35:12 +0200 Subject: RFR(L) 8195099: Low latency hashtable for read-mostly scenarios In-Reply-To: <44b88b98-081c-5fb2-d57d-fa17bfa57160@oracle.com> References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <0CCE37A8-A9AA-414E-9898-5B875B647BE6@oracle.com> <3e8cc7c3-f7fd-e117-8ef2-1e016125cf66@oracle.com> <4cf61dce-3f8e-9baf-6278-68002f615e84@oracle.com> <44b88b98-081c-5fb2-d57d-fa17bfa57160@oracle.com> Message-ID: On 05/03/2018 12:17 AM, David Holmes wrote: > On 2/05/2018 11:56 PM, Robbin Ehn wrote: >> On 2018-05-02 14:40, David Holmes wrote: >>> Not a review, one comment ... >>> >>> On 2/05/2018 6:31 PM, Robbin Ehn wrote: >>>> On 2018-04-30 21:23, Gerard Ziemski wrote: >>>>> #3 How come "_first" doesn't need to be volatile to be used in >>>>> Atomic::cmpxchg ? >>>>> >>>>> Node* _first; >>>>> ... >>>>> ? if (Atomic::cmpxchg(node, &_first, expect) == expect) { >>>> >>>> An implicit cast to the same compatible type which is volatile-qualified is >>>> okey. >>> >>> Any variable used in a lock-free manner should be declared volatile as >>> general good practice IMHO. >> >> Generally I agree with you. But in this case I didn't find the volatile >> qualifier helpful, it just clobbered the code. I didn't see any readability >> improvements from having essential all lines with Node* also having volatile >> in them. > > I'm not concerned with readability I'm concerned with doing the minimum to try > to ensure that a variable that can be concurrently mutated and accessed in a > lock-free manner is at least flagged to the compiler as something it should be > careful about dealing with! The minimum is la/sr. Mutating it as a volatile is a bug. Hiding bugs with volatile is not something I like to do. E.g. if code crashes and volatile makes the crash go away, you have not fixed the bug. So yes this is _only_ readability and consistency. It seem most people favor the consistency of hotspot-style, therefore I will follow the hotspot-style and make those volatile (_first/_next). I'll send new version on RFR thread. Thanks, Robbin > >> What I would like is a type that forces the use of the correct ordering, e.g. >> Atomic _next; > > Meanwhile > > Node * volatile _next; > > is the closest thing we have. > > David > ----- > >> /Robbin >> >>> >>> David >>> ----- >>> >>>>> >>>>> >>>>> #4 Inconsistent parameter names - in: >>>>> >>>>> ? template >>>>> ? bool get_insert_lazy(Thread* thread, LOOKUP_FUNC& lookup, VALUE_FUNC& val_f, >>>>> ?????????????????????? bool* grow_hint = NULL) { >>>>> ??? return get_insert_lazy(thread, lookup, val_f, noOp, grow_hint); >>>>> ? } >>>>> >>>>> We use "val_f" with "_f? suffix, so we should have "lookup_f", not "lookup" >>>>> (many other instances) and "found_f", not "foundf" in >>>>> >>>>> ? template >>>>> ? bool get(Thread* thread, LOOKUP_FUNC& lookup, FOUND_FUNC& foundf, >>>>> ?????????? bool* grow_hint = NULL); >>>>> >>>> >>>> Fixed. >>>> >>>>> >>>>> #5 I wasn't familiar with CRTP, so I read up on it, but I still don't see >>>>> the CRTP in BaseConfig - which is the base class and which is derived? In >>>>> particular I don't see "class Derived : public Base" pattern here? >>>> >>>> It's the supplied config that extends the BaseConfig which is the derived. >>>> >>>> In gtest (test_concurrentHashtable.cpp) you will see this: >>>> >>>> // Simplest working CRPT implementation for the hash-table. >>>> struct Pointer : public SimpleTestTable::BaseConfig { >>>> >>>>> >>>>> More to come? >>>>> >>>>> >>>>> btw. I edited the subject of the email slightly by adding the bug number to >>>>> it, so it?s easy to search for. >>>>> >>>> >>>> Thanks, I'll send the update to the original mail. >>>> >>>> /Robbin >>>> >>>> >>>> >>>>> >>>>> cheers >>>>> >>>>> >>>>>> On Apr 26, 2018, at 2:38 AM, Robbin Ehn wrote: >>>>>> >>>>>> >>>>>> Hi all, please review. >>>>>> >>>>>> The lower latency of the new gcs have higher demands on runtime >>>>>> data-structure >>>>>> in terms of latency. This is a concurrent hashtable using the global-counter >>>>>> (8195099). >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8195098 >>>>>> >>>>>> * Readers never blocks or spins. >>>>>> * Inserts do CAS from a read-side, need to re-CAS during re-size/deletion >>>>>> in targeted bucket or insert collision. (inserts are conceptually a reader) >>>>>> * Deletes locks the targeted bucket, all other buckets are free for >>>>>> operations. >>>>>> * Re-size locks one bucket at the time, all other buckets are free for >>>>>> operations. >>>>>> >>>>>> It does concurrent re-size by doubling size and use one more bit from hash. >>>>>> That means a bucket in the old table contains nodes to either one of two >>>>>> buckets >>>>>> in the new table. Each node in the chain is sorted into one of the two >>>>>> buckets >>>>>> with concurrent readers. Shrinking just take two node chains and put them >>>>>> together in the corresponding bucket. To keep track of what is visible to the >>>>>> concurrent readers it's uses a global-counter, needed during re-size and for >>>>>> deletes. >>>>>> >>>>>> A gtest is provided which passes on our platforms, we also have a >>>>>> prototype of the stringtable using this which passes tier 1-5 on our >>>>>> platforms. >>>>>> >>>>>> Various people have pre-reviewed various parts, thanks! And a special >>>>>> thanks to >>>>>> Coleen for a lot of reviewing! >>>>>> >>>>>> Thanks, Robbin >>>>> From magnus.ihse.bursie at oracle.com Thu May 3 08:40:07 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 3 May 2018 10:40:07 +0200 Subject: [11] RFR(S) 8202552: [AOT][JVMCI] Incorrect usage of INCLUDE_JVMCI and INCLUDE_AOT In-Reply-To: References: Message-ID: <728b7bc3-a6ae-c913-4256-c7bc58434f70@oracle.com> On 2018-05-03 00:13, Vladimir Kozlov wrote: > http://cr.openjdk.java.net/~kvn/8202552/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8202552 Looks good to me. Just some thinking out loud... The way we handle the responsibility split between the makefiles and the hotspot source files is actually a bit corny. :-( It would make more sense for the makefiles to set -DINCLUDE_features=1 when the feature is enabled, and -DINCLUDE_feature=0 when it is not (or just leave it out), and skip all the #ifndef INCLUDE_ #define INCLUDE_ 1 ...in macros.hpp. But that is the work of a future cleanup. This patch makes sure we use the same pattern everywhere, which is good. /Magnus > > Stefan K. found several places where #ifdef instead of #if is used for > INCLUDE_JVMCI. > I also found places where we can use COMPILER2_OR_JVMCI. > > An other problem surprised me that we don't set INCLUDE_AOT to 1. > Makefile defines that variable without value. I changed code to match > other variables setting - in makefile set it to 0 if it is not part of > build and set it to 1 in macros.hpp if it is not defined. > > Tested with tier1, tier2 and tier2 with Graal as JIT compiler. From magnus.ihse.bursie at oracle.com Thu May 3 09:06:45 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 3 May 2018 11:06:45 +0200 Subject: RFR: 8200729: Conditional compilation of GCs In-Reply-To: References: Message-ID: <493e0b27-7200-c361-661b-05573645f279@oracle.com> On 2018-05-02 16:37, Stefan Karlsson wrote: > Hi all, > > Please review these patches to allow for conditional compilation of > the GCs in HotSpot. > > Full patch: > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/all/ It's nice to see this cleanup in build logic! I spotted one issue with the build logic, otherwise it looks good. In JvmFeatures.gmk, remove the? "JVM_EXCLUDE_FILES += none" lines. They are not needed and are technically incorrect (makes the build tries to exclude files named "none"). /Magnus > > (See below for a more fine-grained division into smaller patches) > > Today Parallel, G1, and CMS, are all guarded by INCLUDE_ALL_GCS. > INCLUDE_ALL_GCS becomes defined to 1 for our server > (--with-jvm-variants=server) builds, and is defined to 0 for the > minimal (--with-jvm-variants=minimal) builds. There are also ways to > forcefully remove these GCs from the compilation by configuring with, > for example, --with-jvm-features=all-gcs. > > The proposed patch removes INCLUDE_ALL_GCS (and all-gcs) and replaces > it with INCLUDE_CMSGC, INCLUDE_G1GC, and INCLUDE_PARALLELGC. In > addition to that, INCLUDE_SERIALGC has been added to guard the Serial > GC code. > > Future GCs should adopt this scheme before they get incorporated into > the code base. Note, that there will be some files in gc/shared that > are going to have to know about all GCs, but the goal is to have very > few of these INCLUDE_ checks in the non-GC parts of HotSpot. > > With this patch, it's also going to be easier to stop compiling CMS > when the time as come for it to move from deprecated to removed. > > Note, that even though this adds great flexibility, and allows for > easy inclusion / exclusion of GCs, there's no guarantee that a > specific combination of GCs is going to be maintained in the future. > Just like not all combinations of the different Runtime features (CDS, > NMT, JFR) and Compiler features (AOT, COMPILER1, COMPILER2) are > guaranteed to compile and be supported. I've sanity checked the > different GC configurations build locally, but I've only fully tested > the "server" variant, and "minimal" has only been built locally. > > There's a more high-level discussion around the flexibility the > --with-jvm-features flag adds, and if we really should allow it. Since > this patch only builds upon that existing flexibility, and I don't > change the two defined jvm variants, I'd appreciate if that discussion > could be kept out of this review thread. For further discussion around > this, please see: > > http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021663.html > > This is the patch queue: > > The first patch simply cleans up some INCLUDE_ALL_GCS usages in > platform-specific files. Some of these changes are already being > cleaned up by other RFEs: > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/00.removeUnneededIncludeAllGCs/ > > > > The second patch pre-cleans some include files: > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/01.fixIncludes/ > > > The following is the main patch, which include all relevant HotSpot > changes. For a while now, we have been actively working on abstracting > away GC specific code from non-GC directories, but as can be seen in > this patch, there are still a few places left. Hopefully, we will get > rid of most of these in the near future. > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/02.mainPatch/ > > > The last patch adds the make file support to turn on and off the > different GCs. The content of this patch has evolved from versions > written by myself, Per, and Magnus Ihse Bursie. Magnus last proposed > version used the names gc-cms, gc-g1, gc-parallel, and gc-serial, but > I've changed them to cmsgc, g1gc, parallelgc, and serialgc, so that > they match the INCLUDE_ defines. > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/03.selectIndivudualGCsMakePatch/ > > > Thanks, > StefanK From david.holmes at oracle.com Thu May 3 09:08:34 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 3 May 2018 19:08:34 +1000 Subject: RFR(L) 8195099: Low latency hashtable for read-mostly scenarios In-Reply-To: References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <0CCE37A8-A9AA-414E-9898-5B875B647BE6@oracle.com> <3e8cc7c3-f7fd-e117-8ef2-1e016125cf66@oracle.com> <4cf61dce-3f8e-9baf-6278-68002f615e84@oracle.com> <44b88b98-081c-5fb2-d57d-fa17bfa57160@oracle.com> Message-ID: On 3/05/2018 6:35 PM, Robbin Ehn wrote: > On 05/03/2018 12:17 AM, David Holmes wrote: >> On 2/05/2018 11:56 PM, Robbin Ehn wrote: >>> On 2018-05-02 14:40, David Holmes wrote: >>>> Not a review, one comment ... >>>> >>>> On 2/05/2018 6:31 PM, Robbin Ehn wrote: >>>>> On 2018-04-30 21:23, Gerard Ziemski wrote: >>>>>> #3 How come "_first" doesn't need to be volatile to be used in >>>>>> Atomic::cmpxchg ? >>>>>> >>>>>> Node* _first; >>>>>> ... >>>>>> ? if (Atomic::cmpxchg(node, &_first, expect) == expect) { >>>>> >>>>> An implicit cast to the same compatible type which is >>>>> volatile-qualified is okey. >>>> >>>> Any variable used in a lock-free manner should be declared volatile >>>> as general good practice IMHO. >>> >>> Generally I agree with you. But in this case I didn't find the volatile >>> qualifier helpful, it just clobbered the code. I didn't see any >>> readability >>> improvements from having essential all lines with Node* also having >>> volatile in them. >> >> I'm not concerned with readability I'm concerned with doing the >> minimum to try to ensure that a variable that can be concurrently >> mutated and accessed in a lock-free manner is at least flagged to the >> compiler as something it should be careful about dealing with! > > The minimum is la/sr. Mutating it as a volatile is a bug. Hiding bugs > with volatile is not something I like to do. E.g. if code crashes and > volatile makes the crash go away, you have not fixed the bug. > So yes this is _only_ readability and consistency. So there are never any raw accesses to these fields only load_acquire, store_release, and atomic operations? In that case okay. And my apologies as I did not realize that. David > It seem most people favor the consistency of hotspot-style, therefore I > will follow the hotspot-style and make those volatile (_first/_next). > > I'll send new version on RFR thread. > > Thanks, Robbin > >> >>> What I would like is a type that forces the use of the correct >>> ordering, e.g. >>> Atomic _next; >> >> Meanwhile >> >> Node * volatile _next; >> >> is the closest thing we have. >> >> David >> ----- >> >>> /Robbin >>> >>>> >>>> David >>>> ----- >>>> >>>>>> >>>>>> >>>>>> #4 Inconsistent parameter names - in: >>>>>> >>>>>> ? template >>>>>> ? bool get_insert_lazy(Thread* thread, LOOKUP_FUNC& lookup, >>>>>> VALUE_FUNC& val_f, >>>>>> ?????????????????????? bool* grow_hint = NULL) { >>>>>> ??? return get_insert_lazy(thread, lookup, val_f, noOp, grow_hint); >>>>>> ? } >>>>>> >>>>>> We use "val_f" with "_f? suffix, so we should have "lookup_f", not >>>>>> "lookup" (many other instances) and "found_f", not "foundf" in >>>>>> >>>>>> ? template >>>>>> ? bool get(Thread* thread, LOOKUP_FUNC& lookup, FOUND_FUNC& foundf, >>>>>> ?????????? bool* grow_hint = NULL); >>>>>> >>>>> >>>>> Fixed. >>>>> >>>>>> >>>>>> #5 I wasn't familiar with CRTP, so I read up on it, but I still >>>>>> don't see the CRTP in BaseConfig - which is the base class and >>>>>> which is derived? In particular I don't see "class Derived : >>>>>> public Base" pattern here? >>>>> >>>>> It's the supplied config that extends the BaseConfig which is the >>>>> derived. >>>>> >>>>> In gtest (test_concurrentHashtable.cpp) you will see this: >>>>> >>>>> // Simplest working CRPT implementation for the hash-table. >>>>> struct Pointer : public SimpleTestTable::BaseConfig { >>>>> >>>>>> >>>>>> More to come? >>>>>> >>>>>> >>>>>> btw. I edited the subject of the email slightly by adding the bug >>>>>> number to it, so it?s easy to search for. >>>>>> >>>>> >>>>> Thanks, I'll send the update to the original mail. >>>>> >>>>> /Robbin >>>>> >>>>> >>>>> >>>>>> >>>>>> cheers >>>>>> >>>>>> >>>>>>> On Apr 26, 2018, at 2:38 AM, Robbin Ehn >>>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> Hi all, please review. >>>>>>> >>>>>>> The lower latency of the new gcs have higher demands on runtime >>>>>>> data-structure >>>>>>> in terms of latency. This is a concurrent hashtable using the >>>>>>> global-counter >>>>>>> (8195099). >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >>>>>>> >>>>>>> Bug: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8195098 >>>>>>> >>>>>>> * Readers never blocks or spins. >>>>>>> * Inserts do CAS from a read-side, need to re-CAS during >>>>>>> re-size/deletion in targeted bucket or insert collision. (inserts >>>>>>> are conceptually a reader) >>>>>>> * Deletes locks the targeted bucket, all other buckets are free >>>>>>> for operations. >>>>>>> * Re-size locks one bucket at the time, all other buckets are >>>>>>> free for operations. >>>>>>> >>>>>>> It does concurrent re-size by doubling size and use one more bit >>>>>>> from hash. >>>>>>> That means a bucket in the old table contains nodes to either one >>>>>>> of two buckets >>>>>>> in the new table. Each node in the chain is sorted into one of >>>>>>> the two buckets >>>>>>> with concurrent readers. Shrinking just take two node chains and >>>>>>> put them >>>>>>> together in the corresponding bucket. To keep track of what is >>>>>>> visible to the >>>>>>> concurrent readers it's uses a global-counter, needed during >>>>>>> re-size and for >>>>>>> deletes. >>>>>>> >>>>>>> A gtest is provided which passes on our platforms, we also have a >>>>>>> prototype of the stringtable using this which passes tier 1-5 on >>>>>>> our platforms. >>>>>>> >>>>>>> Various people have pre-reviewed various parts, thanks! And a >>>>>>> special thanks to >>>>>>> Coleen for a lot of reviewing! >>>>>>> >>>>>>> Thanks, Robbin >>>>>> From stefan.karlsson at oracle.com Thu May 3 09:09:57 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 3 May 2018 11:09:57 +0200 Subject: RFR: 8200729: Conditional compilation of GCs In-Reply-To: <493e0b27-7200-c361-661b-05573645f279@oracle.com> References: <493e0b27-7200-c361-661b-05573645f279@oracle.com> Message-ID: On 2018-05-03 11:06, Magnus Ihse Bursie wrote: > On 2018-05-02 16:37, Stefan Karlsson wrote: >> Hi all, >> >> Please review these patches to allow for conditional compilation of >> the GCs in HotSpot. >> >> Full patch: >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/all/ > > It's nice to see this cleanup in build logic! > > I spotted one issue with the build logic, otherwise it looks good. In > JvmFeatures.gmk, remove the? "JVM_EXCLUDE_FILES += none" lines. They are > not needed and are technically incorrect (makes the build tries to > exclude files named "none"). Thanks for reviewing this, and helping out writing the make files changes! I'll remove the += none lines. StefanK > > /Magnus > >> >> (See below for a more fine-grained division into smaller patches) >> >> Today Parallel, G1, and CMS, are all guarded by INCLUDE_ALL_GCS. >> INCLUDE_ALL_GCS becomes defined to 1 for our server >> (--with-jvm-variants=server) builds, and is defined to 0 for the >> minimal (--with-jvm-variants=minimal) builds. There are also ways to >> forcefully remove these GCs from the compilation by configuring with, >> for example, --with-jvm-features=all-gcs. >> >> The proposed patch removes INCLUDE_ALL_GCS (and all-gcs) and replaces >> it with INCLUDE_CMSGC, INCLUDE_G1GC, and INCLUDE_PARALLELGC. In >> addition to that, INCLUDE_SERIALGC has been added to guard the Serial >> GC code. >> >> Future GCs should adopt this scheme before they get incorporated into >> the code base. Note, that there will be some files in gc/shared that >> are going to have to know about all GCs, but the goal is to have very >> few of these INCLUDE_ checks in the non-GC parts of HotSpot. >> >> With this patch, it's also going to be easier to stop compiling CMS >> when the time as come for it to move from deprecated to removed. >> >> Note, that even though this adds great flexibility, and allows for >> easy inclusion / exclusion of GCs, there's no guarantee that a >> specific combination of GCs is going to be maintained in the future. >> Just like not all combinations of the different Runtime features (CDS, >> NMT, JFR) and Compiler features (AOT, COMPILER1, COMPILER2) are >> guaranteed to compile and be supported. I've sanity checked the >> different GC configurations build locally, but I've only fully tested >> the "server" variant, and "minimal" has only been built locally. >> >> There's a more high-level discussion around the flexibility the >> --with-jvm-features flag adds, and if we really should allow it. Since >> this patch only builds upon that existing flexibility, and I don't >> change the two defined jvm variants, I'd appreciate if that discussion >> could be kept out of this review thread. For further discussion around >> this, please see: >> >> http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021663.html >> >> This is the patch queue: >> >> The first patch simply cleans up some INCLUDE_ALL_GCS usages in >> platform-specific files. Some of these changes are already being >> cleaned up by other RFEs: >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/00.removeUnneededIncludeAllGCs/ >> >> >> >> The second patch pre-cleans some include files: >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/01.fixIncludes/ >> >> >> The following is the main patch, which include all relevant HotSpot >> changes. For a while now, we have been actively working on abstracting >> away GC specific code from non-GC directories, but as can be seen in >> this patch, there are still a few places left. Hopefully, we will get >> rid of most of these in the near future. >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/02.mainPatch/ >> >> >> The last patch adds the make file support to turn on and off the >> different GCs. The content of this patch has evolved from versions >> written by myself, Per, and Magnus Ihse Bursie. Magnus last proposed >> version used the names gc-cms, gc-g1, gc-parallel, and gc-serial, but >> I've changed them to cmsgc, g1gc, parallelgc, and serialgc, so that >> they match the INCLUDE_ defines. >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/03.selectIndivudualGCsMakePatch/ >> >> >> Thanks, >> StefanK > From robbin.ehn at oracle.com Thu May 3 09:21:47 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 3 May 2018 11:21:47 +0200 Subject: RFR(L) 8195099: Low latency hashtable for read-mostly scenarios In-Reply-To: References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <0CCE37A8-A9AA-414E-9898-5B875B647BE6@oracle.com> <3e8cc7c3-f7fd-e117-8ef2-1e016125cf66@oracle.com> <4cf61dce-3f8e-9baf-6278-68002f615e84@oracle.com> <44b88b98-081c-5fb2-d57d-fa17bfa57160@oracle.com> Message-ID: On 05/03/2018 11:08 AM, David Holmes wrote: > On 3/05/2018 6:35 PM, Robbin Ehn wrote: >> On 05/03/2018 12:17 AM, David Holmes wrote: >>> On 2/05/2018 11:56 PM, Robbin Ehn wrote: >>>> On 2018-05-02 14:40, David Holmes wrote: >>>>> Not a review, one comment ... >>>>> >>>>> On 2/05/2018 6:31 PM, Robbin Ehn wrote: >>>>>> On 2018-04-30 21:23, Gerard Ziemski wrote: >>>>>>> #3 How come "_first" doesn't need to be volatile to be used in >>>>>>> Atomic::cmpxchg ? >>>>>>> >>>>>>> Node* _first; >>>>>>> ... >>>>>>> ? if (Atomic::cmpxchg(node, &_first, expect) == expect) { >>>>>> >>>>>> An implicit cast to the same compatible type which is volatile-qualified >>>>>> is okey. >>>>> >>>>> Any variable used in a lock-free manner should be declared volatile as >>>>> general good practice IMHO. >>>> >>>> Generally I agree with you. But in this case I didn't find the volatile >>>> qualifier helpful, it just clobbered the code. I didn't see any readability >>>> improvements from having essential all lines with Node* also having volatile >>>> in them. >>> >>> I'm not concerned with readability I'm concerned with doing the minimum to >>> try to ensure that a variable that can be concurrently mutated and accessed >>> in a lock-free manner is at least flagged to the compiler as something it >>> should be careful about dealing with! >> >> The minimum is la/sr. Mutating it as a volatile is a bug. Hiding bugs with >> volatile is not something I like to do. E.g. if code crashes and volatile >> makes the crash go away, you have not fixed the bug. >> So yes this is _only_ readability and consistency. > > So there are never any raw accesses to these fields only load_acquire, > store_release, and atomic operations? In that case okay. And my apologies as I > did not realize that. Hi, David Loads always go via load_acquire, but the store is a plain store using the barrier provided from the CAS. The reason for this is when the store happens the Node is not yet visible to any other thread. Thus both setting the _value and _next trust the barrier from publishing the Node via the CAS. As I said, people seem to be very keen on the consistency of hotspot-style so sending out a new version with _first and _next volatile as you suggested. It still looks okey to me. Thanks, Robbin > > David > > > >> It seem most people favor the consistency of hotspot-style, therefore I will >> follow the hotspot-style and make those volatile (_first/_next). >> >> I'll send new version on RFR thread. >> >> Thanks, Robbin >> >>> >>>> What I would like is a type that forces the use of the correct ordering, e.g. >>>> Atomic _next; >>> >>> Meanwhile >>> >>> Node * volatile _next; >>> >>> is the closest thing we have. >>> >>> David >>> ----- >>> >>>> /Robbin >>>> >>>>> >>>>> David >>>>> ----- >>>>> >>>>>>> >>>>>>> >>>>>>> #4 Inconsistent parameter names - in: >>>>>>> >>>>>>> ? template >>>>>>> ? bool get_insert_lazy(Thread* thread, LOOKUP_FUNC& lookup, VALUE_FUNC& >>>>>>> val_f, >>>>>>> ?????????????????????? bool* grow_hint = NULL) { >>>>>>> ??? return get_insert_lazy(thread, lookup, val_f, noOp, grow_hint); >>>>>>> ? } >>>>>>> >>>>>>> We use "val_f" with "_f? suffix, so we should have "lookup_f", not >>>>>>> "lookup" (many other instances) and "found_f", not "foundf" in >>>>>>> >>>>>>> ? template >>>>>>> ? bool get(Thread* thread, LOOKUP_FUNC& lookup, FOUND_FUNC& foundf, >>>>>>> ?????????? bool* grow_hint = NULL); >>>>>>> >>>>>> >>>>>> Fixed. >>>>>> >>>>>>> >>>>>>> #5 I wasn't familiar with CRTP, so I read up on it, but I still don't see >>>>>>> the CRTP in BaseConfig - which is the base class and which is derived? In >>>>>>> particular I don't see "class Derived : public Base" pattern here? >>>>>> >>>>>> It's the supplied config that extends the BaseConfig which is the derived. >>>>>> >>>>>> In gtest (test_concurrentHashtable.cpp) you will see this: >>>>>> >>>>>> // Simplest working CRPT implementation for the hash-table. >>>>>> struct Pointer : public SimpleTestTable::BaseConfig { >>>>>> >>>>>>> >>>>>>> More to come? >>>>>>> >>>>>>> >>>>>>> btw. I edited the subject of the email slightly by adding the bug number >>>>>>> to it, so it?s easy to search for. >>>>>>> >>>>>> >>>>>> Thanks, I'll send the update to the original mail. >>>>>> >>>>>> /Robbin >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> cheers >>>>>>> >>>>>>> >>>>>>>> On Apr 26, 2018, at 2:38 AM, Robbin Ehn wrote: >>>>>>>> >>>>>>>> >>>>>>>> Hi all, please review. >>>>>>>> >>>>>>>> The lower latency of the new gcs have higher demands on runtime >>>>>>>> data-structure >>>>>>>> in terms of latency. This is a concurrent hashtable using the >>>>>>>> global-counter >>>>>>>> (8195099). >>>>>>>> >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >>>>>>>> >>>>>>>> Bug: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8195098 >>>>>>>> >>>>>>>> * Readers never blocks or spins. >>>>>>>> * Inserts do CAS from a read-side, need to re-CAS during >>>>>>>> re-size/deletion in targeted bucket or insert collision. (inserts are >>>>>>>> conceptually a reader) >>>>>>>> * Deletes locks the targeted bucket, all other buckets are free for >>>>>>>> operations. >>>>>>>> * Re-size locks one bucket at the time, all other buckets are free for >>>>>>>> operations. >>>>>>>> >>>>>>>> It does concurrent re-size by doubling size and use one more bit from hash. >>>>>>>> That means a bucket in the old table contains nodes to either one of two >>>>>>>> buckets >>>>>>>> in the new table. Each node in the chain is sorted into one of the two >>>>>>>> buckets >>>>>>>> with concurrent readers. Shrinking just take two node chains and put them >>>>>>>> together in the corresponding bucket. To keep track of what is visible >>>>>>>> to the >>>>>>>> concurrent readers it's uses a global-counter, needed during re-size and >>>>>>>> for >>>>>>>> deletes. >>>>>>>> >>>>>>>> A gtest is provided which passes on our platforms, we also have a >>>>>>>> prototype of the stringtable using this which passes tier 1-5 on our >>>>>>>> platforms. >>>>>>>> >>>>>>>> Various people have pre-reviewed various parts, thanks! And a special >>>>>>>> thanks to >>>>>>>> Coleen for a lot of reviewing! >>>>>>>> >>>>>>>> Thanks, Robbin >>>>>>> From robbin.ehn at oracle.com Thu May 3 09:30:43 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 3 May 2018 11:30:43 +0200 Subject: RFR(L): 8195098: Low latency hashtable for read-mostly scenarios In-Reply-To: <8defc9dd-e908-1bd5-8fdb-929905773ab0@oracle.com> References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <8defc9dd-e908-1bd5-8fdb-929905773ab0@oracle.com> Message-ID: Hi all, After feedback from DavidH (volatile) here is an update: Inc: http://cr.openjdk.java.net/~rehn/8195098/v2/inc/webrev/ Full: http://cr.openjdk.java.net/~rehn/8195098/v2/full/webrev/ Thanks! /Robbin On 05/02/2018 11:03 AM, Robbin Ehn wrote: > Hi all, > > Here is an update with Gerard's and Coleen's input. > > Inc: > http://cr.openjdk.java.net/~rehn/8195098/v1/inc/webrev/ > Full: > http://cr.openjdk.java.net/~rehn/8195098/v1/full/webrev/ > > Thanks, Robbin > > On 2018-04-26 09:38, Robbin Ehn wrote: >> >> Hi all, please review. >> >> The lower latency of the new gcs have higher demands on runtime data-structure >> in terms of latency. This is a concurrent hashtable using the global-counter >> (8195099). >> >> Webrev: >> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8195098 >> >> * Readers never blocks or spins. >> * Inserts do CAS from a read-side, need to re-CAS during re-size/deletion in >> targeted bucket or insert collision. (inserts are conceptually a reader) >> * Deletes locks the targeted bucket, all other buckets are free for operations. >> * Re-size locks one bucket at the time, all other buckets are free for >> operations. >> >> It does concurrent re-size by doubling size and use one more bit from hash. >> That means a bucket in the old table contains nodes to either one of two buckets >> in the new table. Each node in the chain is sorted into one of the two buckets >> with concurrent readers. Shrinking just take two node chains and put them >> together in the corresponding bucket. To keep track of what is visible to the >> concurrent readers it's uses a global-counter, needed during re-size and for >> deletes. >> >> A gtest is provided which passes on our platforms, we also have a prototype of >> the stringtable using this which passes tier 1-5 on our platforms. >> >> Various people have pre-reviewed various parts, thanks! And a special thanks to >> Coleen for a lot of reviewing! >> >> Thanks, Robbin From david.holmes at oracle.com Thu May 3 09:42:25 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 3 May 2018 19:42:25 +1000 Subject: RFR(L): 8195098: Low latency hashtable for read-mostly scenarios In-Reply-To: References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <8defc9dd-e908-1bd5-8fdb-929905773ab0@oracle.com> Message-ID: On 3/05/2018 7:30 PM, Robbin Ehn wrote: > Hi all, > > After feedback from DavidH (volatile) here is an update: > Inc: > http://cr.openjdk.java.net/~rehn/8195098/v2/inc/webrev/ Looks fine. Thanks. David ----- > Full: > http://cr.openjdk.java.net/~rehn/8195098/v2/full/webrev/ > > Thanks! > > /Robbin > > On 05/02/2018 11:03 AM, Robbin Ehn wrote: >> Hi all, >> >> Here is an update with Gerard's and Coleen's input. >> >> Inc: >> http://cr.openjdk.java.net/~rehn/8195098/v1/inc/webrev/ >> Full: >> http://cr.openjdk.java.net/~rehn/8195098/v1/full/webrev/ >> >> Thanks, Robbin >> >> On 2018-04-26 09:38, Robbin Ehn wrote: >>> >>> Hi all, please review. >>> >>> The lower latency of the new gcs have higher demands on runtime >>> data-structure >>> in terms of latency. This is a concurrent hashtable using the >>> global-counter >>> (8195099). >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8195098 >>> >>> * Readers never blocks or spins. >>> * Inserts do CAS from a read-side, need to re-CAS during >>> re-size/deletion in targeted bucket or insert collision. (inserts are >>> conceptually a reader) >>> * Deletes locks the targeted bucket, all other buckets are free for >>> operations. >>> * Re-size locks one bucket at the time, all other buckets are free >>> for operations. >>> >>> It does concurrent re-size by doubling size and use one more bit from >>> hash. >>> That means a bucket in the old table contains nodes to either one of >>> two buckets >>> in the new table. Each node in the chain is sorted into one of the >>> two buckets >>> with concurrent readers. Shrinking just take two node chains and put >>> them >>> together in the corresponding bucket. To keep track of what is >>> visible to the >>> concurrent readers it's uses a global-counter, needed during re-size >>> and for >>> deletes. >>> >>> A gtest is provided which passes on our platforms, we also have a >>> prototype of the stringtable using this which passes tier 1-5 on our >>> platforms. >>> >>> Various people have pre-reviewed various parts, thanks! And a special >>> thanks to >>> Coleen for a lot of reviewing! >>> >>> Thanks, Robbin From robbin.ehn at oracle.com Thu May 3 09:46:48 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 3 May 2018 11:46:48 +0200 Subject: RFR(L): 8195098: Low latency hashtable for read-mostly scenarios In-Reply-To: References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <8defc9dd-e908-1bd5-8fdb-929905773ab0@oracle.com> Message-ID: <051b6728-0b21-3822-5d5c-82109d16607f@oracle.com> On 05/03/2018 11:42 AM, David Holmes wrote: > On 3/05/2018 7:30 PM, Robbin Ehn wrote: >> Hi all, >> >> After feedback from DavidH (volatile) here is an update: >> Inc: >> http://cr.openjdk.java.net/~rehn/8195098/v2/inc/webrev/ > > Looks fine. Thanks. Thanks David! /Robbin > > David > ----- > >> Full: >> http://cr.openjdk.java.net/~rehn/8195098/v2/full/webrev/ >> >> Thanks! >> >> /Robbin >> >> On 05/02/2018 11:03 AM, Robbin Ehn wrote: >>> Hi all, >>> >>> Here is an update with Gerard's and Coleen's input. >>> >>> Inc: >>> http://cr.openjdk.java.net/~rehn/8195098/v1/inc/webrev/ >>> Full: >>> http://cr.openjdk.java.net/~rehn/8195098/v1/full/webrev/ >>> >>> Thanks, Robbin >>> >>> On 2018-04-26 09:38, Robbin Ehn wrote: >>>> >>>> Hi all, please review. >>>> >>>> The lower latency of the new gcs have higher demands on runtime data-structure >>>> in terms of latency. This is a concurrent hashtable using the global-counter >>>> (8195099). >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8195098 >>>> >>>> * Readers never blocks or spins. >>>> * Inserts do CAS from a read-side, need to re-CAS during re-size/deletion in >>>> targeted bucket or insert collision. (inserts are conceptually a reader) >>>> * Deletes locks the targeted bucket, all other buckets are free for operations. >>>> * Re-size locks one bucket at the time, all other buckets are free for >>>> operations. >>>> >>>> It does concurrent re-size by doubling size and use one more bit from hash. >>>> That means a bucket in the old table contains nodes to either one of two >>>> buckets >>>> in the new table. Each node in the chain is sorted into one of the two buckets >>>> with concurrent readers. Shrinking just take two node chains and put them >>>> together in the corresponding bucket. To keep track of what is visible to the >>>> concurrent readers it's uses a global-counter, needed during re-size and for >>>> deletes. >>>> >>>> A gtest is provided which passes on our platforms, we also have a prototype >>>> of the stringtable using this which passes tier 1-5 on our platforms. >>>> >>>> Various people have pre-reviewed various parts, thanks! And a special thanks to >>>> Coleen for a lot of reviewing! >>>> >>>> Thanks, Robbin From robbin.ehn at oracle.com Thu May 3 09:55:10 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 3 May 2018 11:55:10 +0200 Subject: RFR (S) 8202447: Fix unloading_occurred to mean unloading_occurred In-Reply-To: References: <58b3d0ec-fc8c-cf01-06ce-f652a7f7fc3f@oracle.com> Message-ID: <70201348-8125-d916-409f-0173d682147a@oracle.com> Hi Coleen, On 05/02/2018 12:10 AM, serguei.spitsyn at oracle.com wrote: > Hi Coleen, > > The fix looks good to me. +1, thanks for fixing. /Robbin > > Thanks, > Serguei > > > On 5/1/18 12:24, coleen.phillimore at oracle.com wrote: >> Summary: nmethod unloading does not need to test for jvmti to set >> unloading_occurred, nor do we need to clean weak Klasses in metadata if >> unloading does not occur. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8202447.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8202447 >> >> Tested with mach5 hs-tier1-4, rerunning hs-tier3.? Kitchensink 24 hours, and >> runThese with all 4 GCs. >> >> Thanks, >> Coleen > From leo.korinth at oracle.com Thu May 3 10:01:16 2018 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 3 May 2018 12:01:16 +0200 Subject: RFR: 8176717: GC log file handle leaked to child processes In-Reply-To: <19fd7c9a-0224-f363-08f8-ce9daaeb4d02@oracle.com> References: <19fd7c9a-0224-f363-08f8-ce9daaeb4d02@oracle.com> Message-ID: <88555bb9-b8c7-88b7-d428-2711704d129b@oracle.com> Hi Robbin, On 02/05/18 14:39, Robbin Ehn wrote: > Hi Leo, > > I looked at http://cr.openjdk.java.net/~lkorinth/8176717/02/. > > I think it would be good with some sort of error checking if fcntl > failed, maybe just an assert. > 1282???? if (fd != -1) { > 1283?????? fcntl(fd, F_SETFD, fcntl(fd, F_GETFD) | FD_CLOEXEC); > 1284???? } Changed to: if (fd != -1) { int fd_flags = fcntl(fd, F_GETFD); if (fd_flags != -1) { fcntl(fd, F_SETFD, fd_flags | FD_CLOEXEC); } } I agree that this change is somewhat better and also looks more like the existing code that handles FD_CLOEXEC. New full webrev: http://cr.openjdk.java.net/~lkorinth/8176717/03 Testing: TestInheritFD.java passed on my computer. (running hs-tier{1,2} and TestInheritFD.java on mach5) Thanks for the review! /Leo > Otherwise looks good, thanks, Robbin. > > On 2018-03-12 14:20, Leo Korinth wrote: >> Hi, >> >> This fix is for all operating systems though the problem only seams to >> appear on windows. >> >> I am creating a proxy function for fopen (os::fopen_retain) that >> appends the non-standard "e" mode for linux and bsds. For windows the >> "N" mode is used. For other operating systems, I assume that I can use >> fcntl F_SETFD FD_CLOEXEC. I think this will work for AIX, Solaris and >> other operating systems that do not support the "e" flag. Feedback >> otherwise please! >> >> The reason that I use the mode "e" and not only fcntl for linux and >> bsds is threefold. First, I still need to use mode flags on windows as >> it does not support fcntl. Second, I probably save a system call. >> Third, the change will be applied directly, and there will be no point >> in time (between system calls) when the process can leak the file >> descriptor, so it is safer. >> >> The test case forks three VMs in a row. By doing so we know that the >> second VM is opened with a specific log file. The third VM should have >> less open file descriptors (as it is does not use logging) which is >> checked using a UnixOperatingSystemMXBean. This is not possible on >> windows, so on windows I try to rename the file, which will not work >> if the file is opened (the actual reason the bug was opened). >> >> The added test case shows that the bug fix closes the log file on >> windows. The VM on other operating systems closed the log file even >> before the fix. >> >> Maybe the test case should be moved to a different path? >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8176717 >> https://bugs.openjdk.java.net/browse/JDK-8176809 >> >> Webrev: >> http://cr.openjdk.java.net/~lkorinth/8176717/00/ >> >> Testing: >> hs-tier1, hs-tier2 and TestInheritFD.java >> (on 64-bit linux, solaris, windows and mac) >> >> Thanks, >> Leo From martin.doerr at sap.com Thu May 3 10:03:12 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 3 May 2018 10:03:12 +0000 Subject: RFR(L) 8195099: Low latency hashtable for read-mostly scenarios In-Reply-To: <190cde76-63d2-fa0f-79ab-b9089e8c2cfe@oracle.com> References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <0CCE37A8-A9AA-414E-9898-5B875B647BE6@oracle.com> <3e8cc7c3-f7fd-e117-8ef2-1e016125cf66@oracle.com> <4cf61dce-3f8e-9baf-6278-68002f615e84@oracle.com> <190cde76-63d2-fa0f-79ab-b9089e8c2cfe@oracle.com> Message-ID: <8b9a3fe46d7c431cad6f4e3c31dce48e@sap.com> Hi Robbin, thanks for making _next volatile. I think it doesn't disturb too much (except IA64 where volatile implies acquire+release, but we're not tuning openJDK for this platform) and I feel more comfortable with it. Also thanks for your explanations. Best regards, Martin -----Original Message----- From: Robbin Ehn [mailto:robbin.ehn at oracle.com] Sent: Donnerstag, 3. Mai 2018 10:17 To: Doerr, Martin ; David Holmes ; Gerard Ziemski Cc: hotspot-dev developers Subject: Re: RFR(L) 8195099: Low latency hashtable for read-mostly scenarios Hi Martin, On 05/02/2018 07:53 PM, Doerr, Martin wrote: > Hi Robbin, > > thanks for contributing and especially for having quite a few memory barriers already in place. > > Yes, C++11 Atomics would be nice. > > The current version accesses _next without any compiler barriers so I'm kind of concerned about compilers which tend to reload and reorder accesses. We should double-check if nothing can go wrong when not using volatile. No, we read it via: return OrderAccess::load_acquire(&_next); In store case we trust the implied barriers from CAS: new_node->set_next(first_at_start); } if (bucket->cas_first(new_node, first_at_start)) { It now actually have a stronger than needed ordering. We do not always need acquire on load, but volatile semantics is always wrong, we either have stronger or lesser. E.g. in internal_remove loading the _next pointer should not need acquire. After CAS:ing in the lock bit, the chain is stable and we unlock with a release_store. Loads between the CAS and the release_store my be re-ordered since the chain is stable. > > I guess this code hasn't been tested on weak memory platforms? > So we should better take a close look at the barriers. An earlier version have been successfully tested on aarch64. But it did not have the CAS inserts. > > It'd be great if we could use precise ordering semantics for the Atomic::cmpxchg and Atomic::add (which is currently being reviewed as JDK-8202080). I did on purpose not do any special handling for platforms not following the documented semantics. My thought was either you would do 8202080 or just add needed barrier in e.g. Bucket::cas_first. Btw, I think 8202080 looks good, specially like conservative as default. With conservative as default on cmpxchg and add for global counter, this should work on weaker platforms, but not alpha. /Robbin > > Best regards, > Martin > > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Robbin Ehn > Sent: Mittwoch, 2. Mai 2018 15:57 > To: David Holmes ; Gerard Ziemski > Cc: hotspot-dev developers > Subject: Re: RFR(L) 8195099: Low latency hashtable for read-mostly scenarios > > On 2018-05-02 14:40, David Holmes wrote: >> Not a review, one comment ... >> >> On 2/05/2018 6:31 PM, Robbin Ehn wrote: >>> On 2018-04-30 21:23, Gerard Ziemski wrote: >>>> #3 How come "_first" doesn't need to be volatile to be used in Atomic::cmpxchg ? >>>> >>>> Node* _first; >>>> ... >>>> ? if (Atomic::cmpxchg(node, &_first, expect) == expect) { >>> >>> An implicit cast to the same compatible type which is volatile-qualified is okey. >> >> Any variable used in a lock-free manner should be declared volatile as general >> good practice IMHO. > > Generally I agree with you. But in this case I didn't find the volatile > qualifier helpful, it just clobbered the code. I didn't see any readability > improvements from having essential all lines with Node* also having volatile in > them. > > What I would like is a type that forces the use of the correct ordering, e.g. > Atomic _next; > > /Robbin > >> >> David >> ----- >> >>>> >>>> >>>> #4 Inconsistent parameter names - in: >>>> >>>> ? template >>>> ? bool get_insert_lazy(Thread* thread, LOOKUP_FUNC& lookup, VALUE_FUNC& val_f, >>>> ?????????????????????? bool* grow_hint = NULL) { >>>> ??? return get_insert_lazy(thread, lookup, val_f, noOp, grow_hint); >>>> ? } >>>> >>>> We use "val_f" with "_f? suffix, so we should have "lookup_f", not "lookup" >>>> (many other instances) and "found_f", not "foundf" in >>>> >>>> ? template >>>> ? bool get(Thread* thread, LOOKUP_FUNC& lookup, FOUND_FUNC& foundf, >>>> ?????????? bool* grow_hint = NULL); >>>> >>> >>> Fixed. >>> >>>> >>>> #5 I wasn't familiar with CRTP, so I read up on it, but I still don't see the >>>> CRTP in BaseConfig - which is the base class and which is derived? In >>>> particular I don't see "class Derived : public Base" pattern here? >>> >>> It's the supplied config that extends the BaseConfig which is the derived. >>> >>> In gtest (test_concurrentHashtable.cpp) you will see this: >>> >>> // Simplest working CRPT implementation for the hash-table. >>> struct Pointer : public SimpleTestTable::BaseConfig { >>> >>>> >>>> More to come? >>>> >>>> >>>> btw. I edited the subject of the email slightly by adding the bug number to >>>> it, so it?s easy to search for. >>>> >>> >>> Thanks, I'll send the update to the original mail. >>> >>> /Robbin >>> >>> >>> >>>> >>>> cheers >>>> >>>> >>>>> On Apr 26, 2018, at 2:38 AM, Robbin Ehn wrote: >>>>> >>>>> >>>>> Hi all, please review. >>>>> >>>>> The lower latency of the new gcs have higher demands on runtime data-structure >>>>> in terms of latency. This is a concurrent hashtable using the global-counter >>>>> (8195099). >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rehn/8195098/v0/webrev/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8195098 >>>>> >>>>> * Readers never blocks or spins. >>>>> * Inserts do CAS from a read-side, need to re-CAS during re-size/deletion in >>>>> targeted bucket or insert collision. (inserts are conceptually a reader) >>>>> * Deletes locks the targeted bucket, all other buckets are free for operations. >>>>> * Re-size locks one bucket at the time, all other buckets are free for >>>>> operations. >>>>> >>>>> It does concurrent re-size by doubling size and use one more bit from hash. >>>>> That means a bucket in the old table contains nodes to either one of two >>>>> buckets >>>>> in the new table. Each node in the chain is sorted into one of the two buckets >>>>> with concurrent readers. Shrinking just take two node chains and put them >>>>> together in the corresponding bucket. To keep track of what is visible to the >>>>> concurrent readers it's uses a global-counter, needed during re-size and for >>>>> deletes. >>>>> >>>>> A gtest is provided which passes on our platforms, we also have a prototype >>>>> of the stringtable using this which passes tier 1-5 on our platforms. >>>>> >>>>> Various people have pre-reviewed various parts, thanks! And a special thanks to >>>>> Coleen for a lot of reviewing! >>>>> >>>>> Thanks, Robbin >>>> From robbin.ehn at oracle.com Thu May 3 12:04:31 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 3 May 2018 14:04:31 +0200 Subject: RFR: 8176717: GC log file handle leaked to child processes In-Reply-To: <88555bb9-b8c7-88b7-d428-2711704d129b@oracle.com> References: <19fd7c9a-0224-f363-08f8-ce9daaeb4d02@oracle.com> <88555bb9-b8c7-88b7-d428-2711704d129b@oracle.com> Message-ID: Thanks for fixing, looks good. /Robbin On 2018-05-03 12:01, Leo Korinth wrote: > Hi Robbin, > > On 02/05/18 14:39, Robbin Ehn wrote: >> Hi Leo, >> >> I looked at http://cr.openjdk.java.net/~lkorinth/8176717/02/. >> >> I think it would be good with some sort of error checking if fcntl failed, >> maybe just an assert. >> 1282???? if (fd != -1) { >> 1283?????? fcntl(fd, F_SETFD, fcntl(fd, F_GETFD) | FD_CLOEXEC); >> 1284???? } > Changed to: > ??? if (fd != -1) { > ????? int fd_flags = fcntl(fd, F_GETFD); > ????? if (fd_flags != -1) { > ??????? fcntl(fd, F_SETFD, fd_flags | FD_CLOEXEC); > ????? } > ??? } > > I agree that this change is somewhat better and also looks more like the > existing code that handles FD_CLOEXEC. > > New full webrev: > http://cr.openjdk.java.net/~lkorinth/8176717/03 > > Testing: > TestInheritFD.java passed on my computer. > (running hs-tier{1,2} and TestInheritFD.java on mach5) > > Thanks for the review! > /Leo > >> Otherwise looks good, thanks, Robbin. >> >> On 2018-03-12 14:20, Leo Korinth wrote: >>> Hi, >>> >>> This fix is for all operating systems though the problem only seams to appear >>> on windows. >>> >>> I am creating a proxy function for fopen (os::fopen_retain) that appends the >>> non-standard "e" mode for linux and bsds. For windows the "N" mode is used. >>> For other operating systems, I assume that I can use fcntl F_SETFD >>> FD_CLOEXEC. I think this will work for AIX, Solaris and other operating >>> systems that do not support the "e" flag. Feedback otherwise please! >>> >>> The reason that I use the mode "e" and not only fcntl for linux and bsds is >>> threefold. First, I still need to use mode flags on windows as it does not >>> support fcntl. Second, I probably save a system call. Third, the change will >>> be applied directly, and there will be no point in time (between system >>> calls) when the process can leak the file descriptor, so it is safer. >>> >>> The test case forks three VMs in a row. By doing so we know that the second >>> VM is opened with a specific log file. The third VM should have less open >>> file descriptors (as it is does not use logging) which is checked using a >>> UnixOperatingSystemMXBean. This is not possible on windows, so on windows I >>> try to rename the file, which will not work if the file is opened (the actual >>> reason the bug was opened). >>> >>> The added test case shows that the bug fix closes the log file on windows. >>> The VM on other operating systems closed the log file even before the fix. >>> >>> Maybe the test case should be moved to a different path? >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8176717 >>> https://bugs.openjdk.java.net/browse/JDK-8176809 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~lkorinth/8176717/00/ >>> >>> Testing: >>> hs-tier1, hs-tier2 and TestInheritFD.java >>> (on 64-bit linux, solaris, windows and mac) >>> >>> Thanks, >>> Leo From coleen.phillimore at oracle.com Thu May 3 13:21:22 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 3 May 2018 09:21:22 -0400 Subject: RFR (S) 8202447: Fix unloading_occurred to mean unloading_occurred In-Reply-To: <70201348-8125-d916-409f-0173d682147a@oracle.com> References: <58b3d0ec-fc8c-cf01-06ce-f652a7f7fc3f@oracle.com> <70201348-8125-d916-409f-0173d682147a@oracle.com> Message-ID: <97b63b1b-3620-4c85-38ec-455a92456857@oracle.com> Thank you for the review! Coleen On 5/3/18 5:55 AM, Robbin Ehn wrote: > Hi Coleen, > > On 05/02/2018 12:10 AM, serguei.spitsyn at oracle.com wrote: >> Hi Coleen, >> >> The fix looks good to me. > > +1, thanks for fixing. > > /Robbin > >> >> Thanks, >> Serguei >> >> >> On 5/1/18 12:24, coleen.phillimore at oracle.com wrote: >>> Summary: nmethod unloading does not need to test for jvmti to set >>> unloading_occurred, nor do we need to clean weak Klasses in metadata >>> if unloading does not occur. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8202447.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8202447 >>> >>> Tested with mach5 hs-tier1-4, rerunning hs-tier3.? Kitchensink 24 >>> hours, and runThese with all 4 GCs. >>> >>> Thanks, >>> Coleen >> From erik.helin at oracle.com Thu May 3 13:19:10 2018 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 3 May 2018 15:19:10 +0200 Subject: RFR: 8200729: Conditional compilation of GCs In-Reply-To: References: Message-ID: <1a5cc589-79cf-4139-d894-c4dc3e3ca6bd@oracle.com> On 05/02/2018 04:37 PM, Stefan Karlsson wrote: > Hi all, Hi Stefan, thanks for taking on this work, much appreciated! On 05/02/2018 04:37 PM, Stefan Karlsson wrote: > The first patch simply cleans up some INCLUDE_ALL_GCS usages in platform-specific files. Some of these changes are already being cleaned up by other RFEs: > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/00.removeUnneededIncludeAllGCs/ Looks good, Reviewed. On 05/02/2018 04:37 PM, Stefan Karlsson wrote: > The second patch pre-cleans some include files: > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/01.fixIncludes/ Also looks good, Reviewed. On 05/02/2018 04:37 PM, Stefan Karlsson wrote: > The following is the main patch, which include all relevant HotSpot changes. For a while now, we have been actively working on abstracting away GC specific code from non-GC directories, but as can be seen in this patch, there are still a few places left. Hopefully, we will get rid of most of these in the near future. > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/02.mainPatch/ Very nice, just one small nit: - barrierSetConfig.hpp: could you move "+ G1GC_ONLY(f(G1BarrierSet))" into FOR_EACH_CONCRETE_BARRIER_SET_DO ? As in // Do something for each concrete barrier set part of the build. #define FOR_EACH_CONCRETE_BARRIER_SET_DO(f) \ f(CardTableBarrierSet) \ G1GC_ONLY(f(G1BarrierSet)) As a note for people following along this thread (and to a future version of myself), the following work is ongoing to further clean up the GC specific bits and pieces sprinkled throughout the rest of the VM: - 8202377: Modularize C2 GC barriers - 8202547: Move G1 runtime calls used by generated code to G1BarrierSetRuntime - 8201436: Replace oop_ps_push_contents with oop_iterate and closure In addition to the above work, this patch highlights a few more places that needs to be taken care of: - get rid of #if INCLUDE_CMS in defNewGeneration.cpp - split collector specific parts of gcTrace.hpp into collector specific tracers - rework Threads::create_thread_roots_tasks into something more generic that parallel then can use to implement its own create_thread_roots_task - remove marksweep_init() and set up MarkSweep::_gc_timer and MarkSweep::_gc_tracer in SerialHeap and ParallelScavengeHeap - benchmark (and measure file sizes) to see if it is still worthwhile to keep the INCLUDE_OOP_OOP_ITERATE_BACKWARDS guard (i.e. can we always compile the oop_iterate_backwards methods?) - need to do some work to encapsulate the G1 and CDS code (see e.g. oop.cpp, metaspaceShared.cpp and file.cpp) - move VM_CollectForMetadataAllocation::initiate_concurrent_GC to something like CollectedHeap::initiate_concurrent_GC_for_metadata_allocation (preferably with a shorter name (the above list is _not_ complete, there will always be a need for cleanups :D) > The last patch adds the make file support to turn on and off the different GCs. The content of this patch has evolved from versions written by myself, Per, and Magnus Ihse Bursie. Magnus last proposed version used the names gc-cms, gc-g1, gc-parallel, and gc-serial, but I've changed them to cmsgc, g1gc, parallelgc, and serialgc, so that they match the INCLUDE_ defines. > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/03.selectIndivudualGCsMakePatch/ My Makefile knowledge is limited to smaller hacks, I will leave this patch for Magnus to review (which I see he already has done). Overall, very nice work Stefan, thanks! Erik From stefan.karlsson at oracle.com Thu May 3 13:29:21 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 3 May 2018 15:29:21 +0200 Subject: RFR: 8200729: Conditional compilation of GCs In-Reply-To: <1a5cc589-79cf-4139-d894-c4dc3e3ca6bd@oracle.com> References: <1a5cc589-79cf-4139-d894-c4dc3e3ca6bd@oracle.com> Message-ID: On 2018-05-03 15:19, Erik Helin wrote: > On 05/02/2018 04:37 PM, Stefan Karlsson wrote: > > Hi all, > > Hi Stefan, > > thanks for taking on this work, much appreciated! > > On 05/02/2018 04:37 PM, Stefan Karlsson wrote: > > The first patch simply cleans up some INCLUDE_ALL_GCS usages in > platform-specific files. Some of these changes are already being cleaned > up by other RFEs: > > > > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/00.removeUnneededIncludeAllGCs/ > > > Looks good, Reviewed. > > On 05/02/2018 04:37 PM, Stefan Karlsson wrote: > > The second patch pre-cleans some include files: > > > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/01.fixIncludes/ > > Also looks good, Reviewed. > > On 05/02/2018 04:37 PM, Stefan Karlsson wrote: > > The following is the main patch, which include all relevant HotSpot > changes. For a while now, we have been actively working on abstracting > away GC specific code from non-GC directories, but as can be seen in > this patch, there are still a few places left. Hopefully, we will get > rid of most of these in the near future. > > > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/02.mainPatch/ > > Very nice, just one small nit: > > - barrierSetConfig.hpp: > ? could you move "+? G1GC_ONLY(f(G1BarrierSet))" into > ? FOR_EACH_CONCRETE_BARRIER_SET_DO ? As in > > ?? // Do something for each concrete barrier set part of the build. > ?? #define FOR_EACH_CONCRETE_BARRIER_SET_DO(f)????????? \ > ???? f(CardTableBarrierSet)???????????????????????????? \ > ???? G1GC_ONLY(f(G1BarrierSet)) > Yes. That's better. > As a note for people following along this thread (and to a future > version of myself), the following work is ongoing to further clean up > the GC specific bits and pieces sprinkled throughout the rest of the VM: > - 8202377: Modularize C2 GC barriers > - 8202547: Move G1 runtime calls used by generated code to > G1BarrierSetRuntime > - 8201436: Replace oop_ps_push_contents with oop_iterate and closure > > In addition to the above work, this patch highlights a few more places > that needs to be taken care of: > - get rid of #if INCLUDE_CMS in defNewGeneration.cpp > - split collector specific parts of gcTrace.hpp into collector > ? specific tracers > - rework Threads::create_thread_roots_tasks into something more generic > ? that parallel then can use to implement its own > ? create_thread_roots_task > - remove marksweep_init() and set up MarkSweep::_gc_timer and > ? MarkSweep::_gc_tracer in SerialHeap and ParallelScavengeHeap > - benchmark (and measure file sizes) to see if it is still worthwhile to > ? keep the INCLUDE_OOP_OOP_ITERATE_BACKWARDS guard (i.e. can we always > ? compile the oop_iterate_backwards methods?) > - need to do some work to encapsulate the G1 and CDS code > ? (see e.g. oop.cpp, metaspaceShared.cpp and file.cpp) > - move VM_CollectForMetadataAllocation::initiate_concurrent_GC to > ? something like > ? CollectedHeap::initiate_concurrent_GC_for_metadata_allocation > ? (preferably with a shorter name > > (the above list is _not_ complete, there will always be a need for > cleanups :D) I agree. > > > The last patch adds the make file support to turn on and off the > different GCs. The content of this patch has evolved from versions > written by myself, Per, and Magnus Ihse Bursie. Magnus last proposed > version used the names gc-cms, gc-g1, gc-parallel, and gc-serial, but > I've changed them to cmsgc, g1gc, parallelgc, and serialgc, so that they > match the INCLUDE_ defines. > > > > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/03.selectIndivudualGCsMakePatch/ > > > My Makefile knowledge is limited to smaller hacks, I will leave this > patch for Magnus to review (which I see he already has done). > > Overall, very nice work Stefan, thanks! Thanks Erik! StefanK > Erik From erik.osterlund at oracle.com Thu May 3 13:40:37 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 3 May 2018 15:40:37 +0200 Subject: RFR: 8200557: OopStorage parallel iteration scales poorly In-Reply-To: References: <7CBB6FF2-AC36-4723-BBA1-B4FA277C707B@oracle.com> <5AE1D5DF.90909@oracle.com> Message-ID: <5AEB1155.9090608@oracle.com> Hi Kim, On 2018-05-01 23:15, Kim Barrett wrote: >> On Apr 26, 2018, at 9:36 AM, Erik ?sterlund wrote: >> >> Hi Kim, >> >> FIrst off, some high level comments: >> >> 1) I would have simply grabbed the allocation mutex when incrementing the reference counter when starting concurrent iteration, instead of using RCU. The path where we grab this active list for concurrent iteration is stone cold, so I don't expect there to be any observable benefit in using RCU here. But I am going to let that pass and just note we think differently about things here, and I think that is okay. > A very early version of OopStorage had locking between allocation and > concurrent iteration; that was deemed problematic, since an > application (via allocate calls) could delay a concurrent GC phase. Okay. Not sure I necessarily agree that allocations delaying a concurrent GC phase is a problem here. But I'm just gonna go with that. >> 2) It is a bit unfortunate that going down the RCU path led to yet another implementation of RCU because GlobalCounter can not yet satisfy your scenario. But I think that you have done a good job making the RCU implementation pluggable in OopStorage, and we can easily remove it once the ThreadsListHandle required by GlobalCounter starts supporting lock-free nesting soon. I am speculating that your approach could be folded in to the GlobalCounter, to accomodate non-JavaThreads and non-VM threads, which that solution currently does not support. But perhaps that is a discussion for later. > I expect GlobalCounter to be able to meet the needs of this scenario > eventually. It doesn't at present, but that limitation is being worked > on. Once that's done, it should be easy to switch and drop the private > mechanism. (While there are some differences that might make this > alternative to GlobalCounter interesting to pursue further, the don't > matter for the present OopStorage use-case.) Agreed. >> Low level comments: >> * Noting that the new RCU mechanism will like the last one not work on PPC until there is an atomic counter increment with more conservative memory ordering. But I won't blame you for this. >> >> In oopStorage.cpp: >> >> 879 size_t OopStorage::block_count() const { >> 880 WithActiveArray wab(this); >> 881 return wab.active_array().block_count_acquire(); >> 882 } >> >> Why do you need acquire here? I don't see subsequent loads into the elements of the array, which seems to be what the paired release_store when writing the counter protects against. >> >> 884 size_t OopStorage::total_memory_usage() const { >> 885 size_t total_size = sizeof(OopStorage); >> 886 total_size += strlen(name()) + 1; >> 887 total_size += sizeof(BlockArray); >> 888 WithActiveArray wab(this); >> 889 const BlockArray& blocks = wab.active_array(); >> 890 total_size += blocks.block_count_acquire() * Block::allocation_size(); >> 891 total_size += blocks.size() * sizeof(Block*); >> 892 return total_size; >> 893 } >> >> Same as above: what reordering is the block_count_acquire protecting against? No element in the array is read after the acquire in program order, that I can see. > See response to Coleen. I read it, and am satisfied with the explanation. >> 760 _name(dup_name(name)), >> 761 _active_array(BlockArray::create(initial_active_array_size)), >> 762 _allocate_list(&Block::get_allocate_entry), >> 763 _deferred_updates(NULL), >> 764 _allocate_mutex(allocate_mutex), >> 765 _active_mutex(active_mutex), >> 766 _allocation_count(0), >> 767 _concurrent_iteration_active(false) >> 768 { >> 769 _active_array->increment_refcount(); >> >> I wonder if you could make BlockArray::create() always return an already retained block array, instead of updating it after. >> >> 496 BlockArray* new_array = BlockArray::create(new_size); >> 497 if (new_array == NULL) return false; >> 498 new_array->copy_from(old_array); >> 499 replace_active_array(new_array); >> >> And same here where a block array is created without retain reference counter, which is then incremented manually inside of replace_active_array. Seems to me like it would be easier to make BlockArray::create return already retained block arrays. > My experience with using refcounts has been that starting with an > implicit reference at construction time (e.g. initialize the refcount > to 1 at construction) is more often than not really not helpful. > > However, this comment led me to notice the constructor for OopStorage > should be checking the result of BlockArray::create. Okay. Having used Objective-C, which was explicitly reference counted, and by convention returned new objects with a ref count of 1, I am biased to think that is a good convention. Then you know that if you ever see a ref count of 0, it's because it should be deleted, and never because it either needs to be deleted or was just created. But that is just my opinion I guess, so I'm okay if you don't agree about that. Looks good. Thanks, /Erik From coleen.phillimore at oracle.com Thu May 3 14:18:40 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 3 May 2018 10:18:40 -0400 Subject: RFR: 8200729: Conditional compilation of GCs In-Reply-To: References: Message-ID: <2872206c-2eee-b1f4-1b5c-5a2b7bd9a827@oracle.com> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/02.mainPatch/src/hotspot/share/oops/instanceKlass.hpp.frames.html 1237 #endif //INCLUDE_OOP_OOP_ITERATE_BACKWARDS Can you add // INCLUDE... here.??? My personal nit is an #endif that's far away from the #ifdef. http://cr.openjdk.java.net/~stefank/8200729/webrev.04/02.mainPatch/src/hotspot/share/oops/instanceMirrorKlass.inline.hpp.frames.html This one too although these are a bit closer. I reviewed oops, runtime and utilities directories.? Looks good. Thanks, Coleen On 5/2/18 10:37 AM, Stefan Karlsson wrote: > Hi all, > > Please review these patches to allow for conditional compilation of > the GCs in HotSpot. > > Full patch: > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/all/ > > (See below for a more fine-grained division into smaller patches) > > Today Parallel, G1, and CMS, are all guarded by INCLUDE_ALL_GCS. > INCLUDE_ALL_GCS becomes defined to 1 for our server > (--with-jvm-variants=server) builds, and is defined to 0 for the > minimal (--with-jvm-variants=minimal) builds. There are also ways to > forcefully remove these GCs from the compilation by configuring with, > for example, --with-jvm-features=all-gcs. > > The proposed patch removes INCLUDE_ALL_GCS (and all-gcs) and replaces > it with INCLUDE_CMSGC, INCLUDE_G1GC, and INCLUDE_PARALLELGC. In > addition to that, INCLUDE_SERIALGC has been added to guard the Serial > GC code. > > Future GCs should adopt this scheme before they get incorporated into > the code base. Note, that there will be some files in gc/shared that > are going to have to know about all GCs, but the goal is to have very > few of these INCLUDE_ checks in the non-GC parts of HotSpot. > > With this patch, it's also going to be easier to stop compiling CMS > when the time as come for it to move from deprecated to removed. > > Note, that even though this adds great flexibility, and allows for > easy inclusion / exclusion of GCs, there's no guarantee that a > specific combination of GCs is going to be maintained in the future. > Just like not all combinations of the different Runtime features (CDS, > NMT, JFR) and Compiler features (AOT, COMPILER1, COMPILER2) are > guaranteed to compile and be supported. I've sanity checked the > different GC configurations build locally, but I've only fully tested > the "server" variant, and "minimal" has only been built locally. > > There's a more high-level discussion around the flexibility the > --with-jvm-features flag adds, and if we really should allow it. Since > this patch only builds upon that existing flexibility, and I don't > change the two defined jvm variants, I'd appreciate if that discussion > could be kept out of this review thread. For further discussion around > this, please see: > > http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021663.html > > This is the patch queue: > > The first patch simply cleans up some INCLUDE_ALL_GCS usages in > platform-specific files. Some of these changes are already being > cleaned up by other RFEs: > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/00.removeUnneededIncludeAllGCs/ > > > > The second patch pre-cleans some include files: > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/01.fixIncludes/ > > > The following is the main patch, which include all relevant HotSpot > changes. For a while now, we have been actively working on abstracting > away GC specific code from non-GC directories, but as can be seen in > this patch, there are still a few places left. Hopefully, we will get > rid of most of these in the near future. > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/02.mainPatch/ > > > The last patch adds the make file support to turn on and off the > different GCs. The content of this patch has evolved from versions > written by myself, Per, and Magnus Ihse Bursie. Magnus last proposed > version used the names gc-cms, gc-g1, gc-parallel, and gc-serial, but > I've changed them to cmsgc, g1gc, parallelgc, and serialgc, so that > they match the INCLUDE_ defines. > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/03.selectIndivudualGCsMakePatch/ > > > Thanks, > StefanK From markus.gronlund at oracle.com Thu May 3 14:34:54 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Thu, 3 May 2018 07:34:54 -0700 (PDT) Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: <79cfed54-8b6f-97c6-7fbc-db4f5e5ed39b@oracle.com> References: <5AE0612F.8060200@oracle.com> <79cfed54-8b6f-97c6-7fbc-db4f5e5ed39b@oracle.com> Message-ID: <7310e56b-adf5-4666-8612-d6f254d37948@default> Hi Coleen, thanks for taking a look. We have filed https://bugs.openjdk.java.net/browse/JDK-8202578 to follow up on the placement of where to post the class unload events. About that, it might be possible to at least move the actual post_class_unload_event() function to InstanceKlass::notify_unload_class() as you suggest, with the proviso that we insert the "unloading hook", Jfr::on_unloading_classes(), just after the classes_do(InstanceKlass::notify_unload_class). Another problem, maybe minor, with moving the posting of class unload events to InstanceKlass::notify_unload_class() is how to maintain the same timestamp for each individual unload event. The posting for unloads is "two-stage" in that the first pass fires an unload event (instant, same timestamp) for each klass; the second stage enters the JFR framework via on_unloading_classes() to save all constants for these classes now about to disappear. I think we could do that though, something like this: ? // Tell serviceability tools these classes are unloading classes_do(InstanceKlass::notify_unload_class); JFR_ONLY(Jfr::on_unloading_classes();) ... About also getting post_class_load_event() and post_class_define_event() to InstanceKlass, this will be more problematic. The class load event is a duration event, in that it measures the actual time it took to load the class (it is stack allocated inside SystemDictionary::resolve_instance_class_or_null()). Therefore it might not be possible to move it. Maybe for define_event() as it is only an instant event. We can sort out the intricacies involved in the context of https://bugs.openjdk.java.net/browse/JDK-8202578 . About headers and include guards: There is some inconsistency here, as it is not obvious what needs to be guarded at the include site. We have reworked this a bit (will be included in updated webrev) according to the following: Inclusion of headers for event programming can be unconditionally included, and the INCLUDE_JFR definition evaluation will happen in the header itself. The main reason is to support event programming without explicit conditionals (around the event code itself). The machine generated event code will have a dual, an empty stub, for every event that is outside INCLUDE_JFR. This applies mainly to the following pattern: // unconditionally included for access to event class ... #include "jfr/jfrEvent(s).hpp ... Other headers (support headers) will be changed where possible to not have the INCLUDE_JFR guard inside them unless absolutely needed (only for additional code that need empty stubs). This will push the include condition to the include site, which at least is similar to what is done in other code, which is verbose albeit somewhat consistent. That means we will need to add sections such as: ... #if INCLUDE_JFR #include "someheader.hpp" #endif An unfortunate side-effect of this is that some sites (esp. in .hpp's) will now need dual macros like: ... #if INCLUDE_JFR #include "someheader.hpp" #endif // Since the above is now under an include guard, this will need to follow: JFR_ONLY(DEFINE_FIELD;) This is now macro-on-macro... Maybe the above case is a situation where nested INCLUDE_JFR with stubs is warranted. Moving forward maybe we will be able to move away from this macro:fied code for improved readability (depending on what we need for minimalVM). Thanks again Markus -----Original Message----- From: Coleen Phillimore Sent: den 26 april 2018 19:59 To: hotspot-dev at openjdk.java.net Subject: Re: RFR(XL): 8199712: Flight Recorder http://cr.openjdk.java.net/~egahlin/8199712.0/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html We can file another RFE for this but I think you could call post_class_unload_event() from in InstanceKlass and call from inside of InstanceKlass::notify_unload_class. void ClassLoaderData::unload() { ? _unloading = true; ? // Tell serviceability tools these classes are unloading ? classes_do(InstanceKlass::notify_unload_class); Rather than walking through _klasses again during unloading.? I think we should see if this is possible to improve this after this checkin. http://cr.openjdk.java.net/~egahlin/8199712.0/src/hotspot/share/classfile/systemDictionary.cpp.udiff.html Then move post_class_load and post_class_define events to instanceKlass.cpp too. http://cr.openjdk.java.net/~egahlin/8199712.0/src/hotspot/share/utilities/vmError.cpp.udiff.html Sometimes files are #included inside of #if INCLUDE_JFR and sometimes they aren't.?? Should jfr/jfr.hpp have the #if INCLUDE_JFR inside of it? I reviewed all the shared code in directories classfile, runtime, oops, utilities, except for utilities/ticks.hpp changes, which require an expert for that. It looks like you have my last change that was in jfrArtifacts.cpp now in new/src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeManager.hpp Great! Thanks, Coleen On 4/25/18 7:06 AM, Erik Gahlin wrote: > Greetings, > > Could I have a review of 8199712: Flight Recorder > > As mentioned in the preview [1] the tracing backend has been removed. > Event metadata has been consolidated into a single XML file and event > classes are now generated by GenerateJfrFiles.java. > > Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. > > For details about the feature, see the JEP: > https://bugs.openjdk.java.net/browse/JDK-8193393 > > Webrev: > http://cr.openjdk.java.net/~egahlin/8199712.0/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8199712 > > [1] > http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359.h > tml > > Thanks > Erik and Markus From erik.gahlin at oracle.com Thu May 3 15:36:38 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Thu, 3 May 2018 17:36:38 +0200 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: <3B7915CA-50D7-4303-8C7B-C17D5911BC29@oracle.com> References: <5AE0612F.8060200@oracle.com> <9a5f2a18-f7d4-d1ad-6618-b92bb7d71dd9@samersoff.net> <3B7915CA-50D7-4303-8C7B-C17D5911BC29@oracle.com> Message-ID: <5AEB2C86.1050701@oracle.com> On 2018-05-02 00:08, Karen Kinnear wrote: > Lots of great work here. Review part 1 below. > > I will not be reviewing build files or test files - I believe you have > other coverage for those. I did this quickly, > so if any of my comments don?t make sense after detailed study - just > let me know. > > I owe you a review of the share/jfr files still. > > A couple of issues that I think should be addressed before checking in > the code: > > 1. New logTag.hpp log tags > One is called ?setting? -> could you make this less general please? > like jfrsetting? > Or this a subcategory so we already know we are under ?jfr?? > I think the idea is that tags should be kept simple, so they can be reused by different subsystems, and then you combine them when you log things, i.e. jfr+setting > Note: I would expect that log tags would be added to the CSR since > that is a supported interface > > 2. Code removed two command line flags: EnableTracing and UseLockedTracing > with product flags, the normal approach is to deprecate > given they now do nothing, I would strongly recommend that you > Obsolete before removing > i.e. accept the flags, but do not do anything for 1 release, to allow > script migration. Expire the flags > in the next release. So test that you get a warning if you use them > for JDK 11. > > Note: CSR says EnableTracing flag is removed -> please update CSR for > both to be made obsolete > I will update the CSR and the code, but shouldn't we use Expired, since the feature is removed. (If somebody actually built something on top of -XX:EnableTracing you want it to fail fast. Starting the JVM even though the feature is removed is hiding problems, which may make things worse at a later stage.) > 3. CSR question - did jcmd change at all? Not really. We did a small code change that impacts -XX:StartFlightRecording and JFR.start. If you specify -XX:StartFlightRecording=filename=dump.jfr a recording will be dumped on exit. This is the only reasonable behavior. You wouldn't specify a filename unless you wanted to dump it. In previous CSRs we have not specified behavior/heuristics of flags that are unspecified. If a user specifies -XX:StartFlightRecording=filename=dump.jfr,dumponexit=true/false we still honor that. > If it did I would expect that to also be covered in the CSR, > but I think it did not. > > A couple of issues that should be addressed before JDK11 ships > > 1. FastTime naming - we need to not present an attractive nuisance here. > Please rename to clarify at least: > is_platform_trusted_for_fast_time() -> > ?platform_provides_fast_unordered_time? or something equally clear > about risk in the name > - there is a local with a similar need to change name > FastTime* APIs: e.g. FastTimeElapsedCounterSource -> please include > Unordered in the name > For folks new to this issue - the rdtsc (read time stamp counter) > instruction can return different times across different sockets, > so time can go backwards, so we do not use this for any vm or java > times, just for diagnostic timestamps > and we highly recommend that folks modifying the jvm do not start > using it accidentally. > Markus will comment fast time support. > 2. Docker container compatibility > Can you please check with Bob Vandette e.g. for cgroup handling - the > new os_perf_linux.cpp may have some overlap/need > modifications simliar to what he did with os_linux.cpp - e.g. for > number of cpus, sockets, etc. > And please test with a container - Misha can give guidance here. > See separate e-mail > Other comments: > > 1. #INCLUDE_JFR > Did you build with #INCLUDE_JFR not defined? > Is the theory that we still may want a minimalVM without JFR? > The model is not clear to me in terms of when you include files or > generate events > e.g. c1_GraphBuilder.cpp, systemDictionary.cpp, synchronizer.cpp, ? > Could this please be on your future clean-up list? > p.s. I do like the extraction of the post_xxx_event logic > > 2. Duplication > Is there any duplication in the VM_Version and VM_Version_ext handling? > And are you covering older CPUs we no longer need to cover? Vladimir K > would know the precise list. > We have added a RFE for this: https://bugs.openjdk.java.net/browse/JDK-8202579 > 3. objectCountEventSender.hpp: comment to clean up: > + // The following two functions have the exact same signature as > + // > hotspot/src/share/vm/gc_implementation/shared/objectCountEventSender.hpp > + // > + // This file will replace the open file if a closed build is performed. > + // These function signatures can therefore not be changed if the open > + // signatures aren't changed as well. > Will fix. > 4. ObjectMonitor::enter > Why was this event modified differently from others - i.e. it stores > the stack trace in the _owner field? And is there a reason we flush here? > > 5. jfrMacros.hpp > I don't see any users of JFR_FLUSH? > Markus will comment 4 and 5. > 6. vmStructs.cpp removed all the VM_XXX_TRACE which was conditional. > Did SA never use it? Or will that need updating? > The SA support has not worked since JDK 9. Bugs have been filed, but they have not been fixed. The support was added by sustaining so it was possible to extract a .jfr file from a core dump. With JDK 9, a recording is written if a crash occurs, so the functionality is not needed anymore. Even if somebody would like the functionality back, for whatever reason, most of declarations are wrong anyway. I don't think we should bring in code that is broken into OpenJDK. SA is also a nightmare to maintain. Let's keep it out. > 7. Assuming jdk.management.jfr and jdk.jfr changes were just > name/include locations/copyrights > but nothing substantial - I did a quick skim We have only changed copyright headers and spelling errors in the jdk modules. Thanks Erik > > thanks, > Karen > >> On Apr 28, 2018, at 4:17 AM, Erik Gahlin > > wrote: >> >> Yes, If we don?t include JFR, we should get the stubs >> >> Will fix. >> >> Erik >> >>> On 28 Apr 2018, at 09:59, Dmitry Samersoff >> > wrote: >>> >>> Erik, >>> >>> globals.cpp:1051 these changes seems to be unnecessary. >>> >>> -Dmitry >>> >>> On 25.04.2018 14:06, Erik Gahlin wrote: >>>> Greetings, >>>> >>>> Could I have a review of 8199712: Flight Recorder >>>> >>>> As mentioned in the preview [1] the tracing backend has been removed. >>>> Event metadata has been consolidated into a single XML file and event >>>> classes are now generated by GenerateJfrFiles.java. >>>> >>>> Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. >>>> >>>> For details about the feature, see the JEP: >>>> https://bugs.openjdk.java.net/browse/JDK-8193393 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~egahlin/8199712.0/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8199712 >>>> >>>> [1] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359.html >>>> >>>> Thanks >>>> Erik and Markus >>> >>> >>> -- >>> Dmitry Samersoff >>> http://devnull.samersoff.net >>> * There will come soft rains ... >>> >> > From erik.gahlin at oracle.com Thu May 3 15:45:14 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Thu, 3 May 2018 17:45:14 +0200 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: <8815301e-9aec-080e-9960-e52718ddd2b3@oracle.com> References: <5AE0612F.8060200@oracle.com> <8815301e-9aec-080e-9960-e52718ddd2b3@oracle.com> Message-ID: <5AEB2E8A.1090906@oracle.com> Hi Misha, Thanks for the review. I will fix 1, 2, and 3. Erik > Hi Erik, > > My review is based on: http://cr.openjdk.java.net/~egahlin/8199712.0/ > I looked at the test portion only. > Overall looks good, however I have some comments: > > 1. A number of JFC files still have "internal copyright header" > E.g.: open/test/jdk/jdk/jfr/event/gc/collection/gc-testsettings.jfc > > More .jfc files under open/test/jdk/jdk/jfr/ have same issue, or > no copyright header at all: > ./event/gc/collection/gc-testsettings.jfc > ./event/gc/detailed/promotionfailed-testsettings.jfc > ./event/gc/detailed/evacuationfailed-testsettings.jfc > ./event/gc/detailed/concurrentmodefailure-testsettings.jfc > ./api/recording/settings/settings.jfc > ./jcmd/jcmd-testsettings.2.jfc > ./jcmd/jcmd-testsettings.jfc > ./jcmd/jcmd-testsettings3.jfc > > > 2. The following files are missing copyright statement: > /open/test/jdk/jdk/jfr/event/io/MakeJAR.sh > > 3. Please update the copyright year: > test/lib/jdk/test/lib/thread/TestThread.java > test/lib/jdk/test/lib/thread/XRun.java > > > The rest of test related files look good to me, > Misha > > > On 05/01/2018 04:37 PM, Vladimir Kozlov wrote: >> Hi Erik, >> >> I am working on 8184349 and adding & !vm.graal.enabled to @requires >> for tests which use CMS. Mostly they are JFR tests which are in this >> review list. And I found some test have incorrect commands (merged 2 >> lines together): >> >> * @test @requires vm.gc == "null" | vm.gc == "Serial" >> >> I see that you fixed some. But there few left: >> >> test/jdk/jdk/jfr/event/gc/detailed/TestStressAllocationGCEventsWithDefNew.java >> >> test/jdk/jdk/jfr/event/gc/detailed/TestStressAllocationGCEventsWithG1.java >> >> test/jdk/jdk/jfr/event/gc/detailed/TestStressBigAllocationGCEventsWithParallel.java >> >> >> Thanks, >> Vladimir >> >> On 4/25/18 4:06 AM, Erik Gahlin wrote: >>> Greetings, >>> >>> Could I have a review of 8199712: Flight Recorder >>> >>> As mentioned in the preview [1] the tracing backend has been >>> removed. Event metadata has been consolidated into a single XML file >>> and event classes are now generated by GenerateJfrFiles.java. >>> >>> Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. >>> >>> For details about the feature, see the JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8193393 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~egahlin/8199712.0/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8199712 >>> >>> [1] >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359.html >>> >>> Thanks >>> Erik and Markus > From vladimir.kozlov at oracle.com Thu May 3 15:57:27 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 3 May 2018 08:57:27 -0700 Subject: [11] RFR(S) 8202552: [AOT][JVMCI] Incorrect usage of INCLUDE_JVMCI and INCLUDE_AOT In-Reply-To: References: Message-ID: <0a4567fc-935f-2769-0c39-b6df85d3b6a6@oracle.com> Thank you, Stefan Vladimir K On 5/3/18 12:20 AM, Stefan Karlsson wrote: > Looks good to me. Thanks for fixing. > > StefanK > > On 2018-05-03 00:13, Vladimir Kozlov wrote: >> http://cr.openjdk.java.net/~kvn/8202552/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8202552 >> >> Stefan K. found several places where #ifdef instead of #if is used for >> INCLUDE_JVMCI. >> I also found places where we can use COMPILER2_OR_JVMCI. >> >> An other problem surprised me that we don't set INCLUDE_AOT to 1. >> Makefile defines that variable without value. I changed code to match >> other variables setting - in makefile set it to 0 if it is not part of >> build and set it to 1 in macros.hpp if it is not defined. >> >> Tested with tier1, tier2 and tier2 with Graal as JIT compiler. >> From vladimir.kozlov at oracle.com Thu May 3 15:59:51 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 3 May 2018 08:59:51 -0700 Subject: [11] RFR(S) 8202552: [AOT][JVMCI] Incorrect usage of INCLUDE_JVMCI and INCLUDE_AOT In-Reply-To: <728b7bc3-a6ae-c913-4256-c7bc58434f70@oracle.com> References: <728b7bc3-a6ae-c913-4256-c7bc58434f70@oracle.com> Message-ID: <2b271a2b-bdb3-6d5b-dd8f-91d2e2efdff9@oracle.com> Thank you, Magnus On 5/3/18 1:40 AM, Magnus Ihse Bursie wrote: > On 2018-05-03 00:13, Vladimir Kozlov wrote: >> http://cr.openjdk.java.net/~kvn/8202552/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8202552 > > Looks good to me. > > Just some thinking out loud... The way we handle the responsibility > split between the makefiles and the hotspot source files is actually a > bit corny. :-( It would make more sense for the makefiles to set > -DINCLUDE_features=1 when the feature is enabled, and > -DINCLUDE_feature=0 when it is not (or just leave it out), and skip all > the #ifndef INCLUDE_ #define INCLUDE_ 1 ...in macros.hpp. Yes, I thought the same thing but it was too much change for this fix. > > But that is the work of a future cleanup. This patch makes sure we use > the same pattern everywhere, which is good. Thanks, Vladimir K > > /Magnus > > >> >> Stefan K. found several places where #ifdef instead of #if is used for >> INCLUDE_JVMCI. >> I also found places where we can use COMPILER2_OR_JVMCI. >> >> An other problem surprised me that we don't set INCLUDE_AOT to 1. >> Makefile defines that variable without value. I changed code to match >> other variables setting - in makefile set it to 0 if it is not part of >> build and set it to 1 in macros.hpp if it is not defined. >> >> Tested with tier1, tier2 and tier2 with Graal as JIT compiler. > From coleen.phillimore at oracle.com Thu May 3 16:12:04 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 3 May 2018 12:12:04 -0400 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: <7310e56b-adf5-4666-8612-d6f254d37948@default> References: <5AE0612F.8060200@oracle.com> <79cfed54-8b6f-97c6-7fbc-db4f5e5ed39b@oracle.com> <7310e56b-adf5-4666-8612-d6f254d37948@default> Message-ID: On 5/3/18 10:34 AM, Markus Gronlund wrote: > Hi Coleen, > > thanks for taking a look. > > We have filed https://bugs.openjdk.java.net/browse/JDK-8202578 to follow up on the placement of where to post the class unload events. > > About that, it might be possible to at least move the actual post_class_unload_event() function to InstanceKlass::notify_unload_class() as you suggest, with the proviso that we insert the "unloading hook", Jfr::on_unloading_classes(), just after the classes_do(InstanceKlass::notify_unload_class). Another problem, maybe minor, with moving the posting of class unload events to InstanceKlass::notify_unload_class() is how to maintain the same timestamp for each individual unload event. > > The posting for unloads is "two-stage" in that the first pass fires an unload event (instant, same timestamp) for each klass; the second stage enters the JFR framework via on_unloading_classes() to save all constants for these classes now about to disappear. I think we could do that though, something like this: > > ? > // Tell serviceability tools these classes are unloading > classes_do(InstanceKlass::notify_unload_class); > > JFR_ONLY(Jfr::on_unloading_classes();) > ... > > About also getting post_class_load_event() and post_class_define_event() to InstanceKlass, this will be more problematic. The class load event is a duration event, in that it measures the actual time it took to load the class (it is stack allocated inside SystemDictionary::resolve_instance_class_or_null()). Therefore it might not be possible to move it. Maybe for define_event() as it is only an instant event.\ Yes, thank you for filing the RFE to follow up about this.? Can you cut/paste this into the description? > > We can sort out the intricacies involved in the context of https://bugs.openjdk.java.net/browse/JDK-8202578 . > > About headers and include guards: > > There is some inconsistency here, as it is not obvious what needs to be guarded at the include site. > > We have reworked this a bit (will be included in updated webrev) according to the following: > > Inclusion of headers for event programming can be unconditionally included, and the INCLUDE_JFR definition evaluation will happen in the header itself. The main reason is to support event programming without explicit conditionals (around the event code itself). The machine generated event code will have a dual, an empty stub, for every event that is outside INCLUDE_JFR. > > This applies mainly to the following pattern: > > // unconditionally included for access to event class > ... > #include "jfr/jfrEvent(s).hpp > ... > > Other headers (support headers) will be changed where possible to not have the INCLUDE_JFR guard inside them unless absolutely needed (only for additional code that need empty stubs). > This will push the include condition to the include site, which at least is similar to what is done in other code, which is verbose albeit somewhat consistent. > > That means we will need to add sections such as: > ... > #if INCLUDE_JFR > #include "someheader.hpp" > #endif > > An unfortunate side-effect of this is that some sites (esp. in .hpp's) will now need dual macros like: > > ... > #if INCLUDE_JFR > #include "someheader.hpp" > #endif > > // Since the above is now under an include guard, this will need to follow: > JFR_ONLY(DEFINE_FIELD;) > > This is now macro-on-macro... > > Maybe the above case is a situation where nested INCLUDE_JFR with stubs is warranted. Okay, thanks for the explanation. > > Moving forward maybe we will be able to move away from this macro:fied code for improved readability (depending on what we need for minimalVM). Yes, this would be nice to clean up over time. Thanks! Coleen > > Thanks again > Markus > > -----Original Message----- > From: Coleen Phillimore > Sent: den 26 april 2018 19:59 > To: hotspot-dev at openjdk.java.net > Subject: Re: RFR(XL): 8199712: Flight Recorder > > http://cr.openjdk.java.net/~egahlin/8199712.0/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html > > We can file another RFE for this but I think you could call > post_class_unload_event() from in InstanceKlass and call from inside of InstanceKlass::notify_unload_class. > > void ClassLoaderData::unload() { > ? _unloading = true; > > ? // Tell serviceability tools these classes are unloading > ? classes_do(InstanceKlass::notify_unload_class); > > Rather than walking through _klasses again during unloading.? I think we should see if this is possible to improve this after this checkin. > > http://cr.openjdk.java.net/~egahlin/8199712.0/src/hotspot/share/classfile/systemDictionary.cpp.udiff.html > > Then move post_class_load and post_class_define events to instanceKlass.cpp too. > > http://cr.openjdk.java.net/~egahlin/8199712.0/src/hotspot/share/utilities/vmError.cpp.udiff.html > > Sometimes files are #included inside of #if INCLUDE_JFR and sometimes they aren't.?? Should jfr/jfr.hpp have the #if INCLUDE_JFR inside of it? > > I reviewed all the shared code in directories classfile, runtime, oops, utilities, except for utilities/ticks.hpp changes, which require an expert for that. > > It looks like you have my last change that was in jfrArtifacts.cpp now in > > new/src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeManager.hpp > > Great! > > Thanks, > Coleen > > > On 4/25/18 7:06 AM, Erik Gahlin wrote: >> Greetings, >> >> Could I have a review of 8199712: Flight Recorder >> >> As mentioned in the preview [1] the tracing backend has been removed. >> Event metadata has been consolidated into a single XML file and event >> classes are now generated by GenerateJfrFiles.java. >> >> Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. >> >> For details about the feature, see the JEP: >> https://bugs.openjdk.java.net/browse/JDK-8193393 >> >> Webrev: >> http://cr.openjdk.java.net/~egahlin/8199712.0/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8199712 >> >> [1] >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359.h >> tml >> >> Thanks >> Erik and Markus From stefan.karlsson at oracle.com Thu May 3 16:42:35 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 3 May 2018 18:42:35 +0200 Subject: RFR: 8200729: Conditional compilation of GCs In-Reply-To: <2872206c-2eee-b1f4-1b5c-5a2b7bd9a827@oracle.com> References: <2872206c-2eee-b1f4-1b5c-5a2b7bd9a827@oracle.com> Message-ID: <79ee6ce9-eb5d-248b-61f4-87638038cc97@oracle.com> Hi Coleen, On 2018-05-03 16:18, coleen.phillimore at oracle.com wrote: > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/02.mainPatch/src/hotspot/share/oops/instanceKlass.hpp.frames.html > > > 1237 #endif //INCLUDE_OOP_OOP_ITERATE_BACKWARDS > > > Can you add // INCLUDE... here.??? My personal nit is an #endif that's > far away from the #ifdef. Sure. > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/02.mainPatch/src/hotspot/share/oops/instanceMirrorKlass.inline.hpp.frames.html > > > This one too although these are a bit closer. OK. > > I reviewed oops, runtime and utilities directories.? Looks good. Thanks for reviewing this! StefanK > Thanks, > Coleen > > On 5/2/18 10:37 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review these patches to allow for conditional compilation of >> the GCs in HotSpot. >> >> Full patch: >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/all/ >> >> (See below for a more fine-grained division into smaller patches) >> >> Today Parallel, G1, and CMS, are all guarded by INCLUDE_ALL_GCS. >> INCLUDE_ALL_GCS becomes defined to 1 for our server >> (--with-jvm-variants=server) builds, and is defined to 0 for the >> minimal (--with-jvm-variants=minimal) builds. There are also ways to >> forcefully remove these GCs from the compilation by configuring with, >> for example, --with-jvm-features=all-gcs. >> >> The proposed patch removes INCLUDE_ALL_GCS (and all-gcs) and replaces >> it with INCLUDE_CMSGC, INCLUDE_G1GC, and INCLUDE_PARALLELGC. In >> addition to that, INCLUDE_SERIALGC has been added to guard the Serial >> GC code. >> >> Future GCs should adopt this scheme before they get incorporated into >> the code base. Note, that there will be some files in gc/shared that >> are going to have to know about all GCs, but the goal is to have very >> few of these INCLUDE_ checks in the non-GC parts of HotSpot. >> >> With this patch, it's also going to be easier to stop compiling CMS >> when the time as come for it to move from deprecated to removed. >> >> Note, that even though this adds great flexibility, and allows for >> easy inclusion / exclusion of GCs, there's no guarantee that a >> specific combination of GCs is going to be maintained in the future. >> Just like not all combinations of the different Runtime features (CDS, >> NMT, JFR) and Compiler features (AOT, COMPILER1, COMPILER2) are >> guaranteed to compile and be supported. I've sanity checked the >> different GC configurations build locally, but I've only fully tested >> the "server" variant, and "minimal" has only been built locally. >> >> There's a more high-level discussion around the flexibility the >> --with-jvm-features flag adds, and if we really should allow it. Since >> this patch only builds upon that existing flexibility, and I don't >> change the two defined jvm variants, I'd appreciate if that discussion >> could be kept out of this review thread. For further discussion around >> this, please see: >> >> http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021663.html >> >> This is the patch queue: >> >> The first patch simply cleans up some INCLUDE_ALL_GCS usages in >> platform-specific files. Some of these changes are already being >> cleaned up by other RFEs: >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/00.removeUnneededIncludeAllGCs/ >> >> >> >> The second patch pre-cleans some include files: >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/01.fixIncludes/ >> >> >> The following is the main patch, which include all relevant HotSpot >> changes. For a while now, we have been actively working on abstracting >> away GC specific code from non-GC directories, but as can be seen in >> this patch, there are still a few places left. Hopefully, we will get >> rid of most of these in the near future. >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/02.mainPatch/ >> >> >> The last patch adds the make file support to turn on and off the >> different GCs. The content of this patch has evolved from versions >> written by myself, Per, and Magnus Ihse Bursie. Magnus last proposed >> version used the names gc-cms, gc-g1, gc-parallel, and gc-serial, but >> I've changed them to cmsgc, g1gc, parallelgc, and serialgc, so that >> they match the INCLUDE_ defines. >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/03.selectIndivudualGCsMakePatch/ >> >> >> Thanks, >> StefanK > From vladimir.kozlov at oracle.com Thu May 3 16:56:20 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 3 May 2018 09:56:20 -0700 Subject: RFR: 8200729: Conditional compilation of GCs In-Reply-To: References: Message-ID: I see that you don't guard Use*GC flags with _ONLY macros. It is hard to figure out from gcConfig.cpp what happened if UseG1GC is specified on command line for VM which does not include it. What happens? I don't see C1 changes but it looks like it was changed with 8201543. C2 changes seems fine but they will be also affected by 8202377. What is left is AOT and JVMCI/Graal. We thought to implement "cross" compilation when we can specify for which configuration we should generate AOT code. It includes GC barriers code. Currently I see that you keep all GCs in build as before so it is not a issue. And we record VM version so we will not generate and use G1 barriers if it is not in VM used by jaotc. So it seems to me compiler and AOT, Graal changes are fine. I would suggest in addition to hs-tier2 testing run hs-tier2-graal to make sure Graal still works. Thanks, Vladimir On 5/2/18 7:37 AM, Stefan Karlsson wrote: > Hi all, > > Please review these patches to allow for conditional compilation of the > GCs in HotSpot. > > Full patch: > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/all/ > > (See below for a more fine-grained division into smaller patches) > > Today Parallel, G1, and CMS, are all guarded by INCLUDE_ALL_GCS. > INCLUDE_ALL_GCS becomes defined to 1 for our server > (--with-jvm-variants=server) builds, and is defined to 0 for the minimal > (--with-jvm-variants=minimal) builds. There are also ways to forcefully > remove these GCs from the compilation by configuring with, for example, > --with-jvm-features=all-gcs. > > The proposed patch removes INCLUDE_ALL_GCS (and all-gcs) and replaces it > with INCLUDE_CMSGC, INCLUDE_G1GC, and INCLUDE_PARALLELGC. In addition to > that, INCLUDE_SERIALGC has been added to guard the Serial GC code. > > Future GCs should adopt this scheme before they get incorporated into > the code base. Note, that there will be some files in gc/shared that are > going to have to know about all GCs, but the goal is to have very few of > these INCLUDE_ checks in the non-GC parts of HotSpot. > > With this patch, it's also going to be easier to stop compiling CMS when > the time as come for it to move from deprecated to removed. > > Note, that even though this adds great flexibility, and allows for easy > inclusion / exclusion of GCs, there's no guarantee that a specific > combination of GCs is going to be maintained in the future. Just like > not all combinations of the different Runtime features (CDS, NMT, JFR) > and Compiler features (AOT, COMPILER1, COMPILER2) are guaranteed to > compile and be supported. I've sanity checked the different GC > configurations build locally, but I've only fully tested the "server" > variant, and "minimal" has only been built locally. > > There's a more high-level discussion around the flexibility the > --with-jvm-features flag adds, and if we really should allow it. Since > this patch only builds upon that existing flexibility, and I don't > change the two defined jvm variants, I'd appreciate if that discussion > could be kept out of this review thread. For further discussion around > this, please see: > > http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021663.html > > This is the patch queue: > > The first patch simply cleans up some INCLUDE_ALL_GCS usages in > platform-specific files. Some of these changes are already being cleaned > up by other RFEs: > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/00.removeUnneededIncludeAllGCs/ > > > > The second patch pre-cleans some include files: > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/01.fixIncludes/ > > > The following is the main patch, which include all relevant HotSpot > changes. For a while now, we have been actively working on abstracting > away GC specific code from non-GC directories, but as can be seen in > this patch, there are still a few places left. Hopefully, we will get > rid of most of these in the near future. > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/02.mainPatch/ > > > The last patch adds the make file support to turn on and off the > different GCs. The content of this patch has evolved from versions > written by myself, Per, and Magnus Ihse Bursie. Magnus last proposed > version used the names gc-cms, gc-g1, gc-parallel, and gc-serial, but > I've changed them to cmsgc, g1gc, parallelgc, and serialgc, so that they > match the INCLUDE_ defines. > > http://cr.openjdk.java.net/~stefank/8200729/webrev.04/03.selectIndivudualGCsMakePatch/ > > > Thanks, > StefanK From irogers at google.com Thu May 3 17:05:17 2018 From: irogers at google.com (Ian Rogers) Date: Thu, 03 May 2018 17:05:17 +0000 Subject: RFR 8199868: Support JNI critical functions in object pinning API In-Reply-To: <16c60f53-d835-29f1-981f-4f70537ceb62@oracle.com> References: <931060af-b44d-f348-92ba-e98d623d4c84@redhat.com> <19c791e7-1bf8-69ef-7090-b7da800f2021@redhat.com> <16c60f53-d835-29f1-981f-4f70537ceb62@oracle.com> Message-ID: Re JDK-8199868 - should pinning be limited to just critical APIs? The non-critical variants could also pin to avoid copies. I see this as related to JDK-8199919. It'd be nice to just move the critical APIs to use the same logic as the non-critical APIs, then pinning just becomes a performance optimization to avoid copying. Re JDK-8199919's comment that there is no deprecation mechanism, compilers like GCC allow function attributes that warn when a deprecated function is used. Thanks, Ian On Wed, May 2, 2018 at 2:12 PM Per Liden wrote: > Hi, > > On 05/02/2018 09:41 PM, Zhengyu Gu wrote: > > Hi, > > > > Can I have reviews for this RFR? > > > > This patch completes object pinning for JNI critical section, provides > > critical native support. > > > > The approach is quite straightforward: > > > > During generating native wrapper for critical native method, it > > generates runtime call to pin every array argument, before unpacks them. > > > > For pinned objects, it also needs to save them for unpinning after JNI > > function call completes. > > > > If argument is passed on stack, it saves pinned object at the original > > slot (as pin_object() may move the object). For register based > > arguments, it reuses oop handle area (where GCLocker based > > implementation saves register based arguments for safepoints). > > > > Currently, only Shenandoah uses object pinning for JNI critical section, > > this patch has been baked quite some time there. However, I am new to > > Runtime Stub code, I would appreciate your comments and suggestions. > > > > I rebased patch to jdk/jdk repo. > > > > Webrev: http://cr.openjdk.java.net/~zgu/8199868/webrev.02/ > > Just want to say that I would really like to see this patch go in. As > mentioned, it completes the object pinning story and it's useful for > other GCs too (at least ZGC and possibly G1). However, I also agree with > Aleksey that some one who really knows this code needs to review this. > Unfortunately that's not me. Anyone? > > cheers, > Per > > > > > Thanks, > > > > -Zhengyu > > > > > > On 04/06/2018 10:35 PM, Zhengyu Gu wrote: > >> Offline discussion with Aleksey, he suggested that > >> pin/unpin_critical_native_array methods can be made more generic as > >> pin/unpin_object. > >> > >> Updated webrev: http://cr.openjdk.java.net/~zgu/8199868/webrev.01/ > >> > >> Test: > >> Reran all tests, submit-hs tests still clean. > >> > >> Thanks, > >> > >> -Zhengyu > >> > >> On 04/06/2018 08:55 AM, Aleksey Shipilev wrote: > >>> On 04/04/2018 07:47 PM, Zhengyu Gu wrote: > >>>> Please review this patch that adds JNI critical native support to > >>>> object pinning. > >>>> > >>>> Shenandoah does not block GC while JNI critical session is in > >>>> progress. This patch allows it to pin > >>>> all incoming array objects before critical native call, and unpin > >>>> them after call completes. > >>>> > >>>> > >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199868 > >>>> Webrev: http://cr.openjdk.java.net/~zgu/8199868/webrev.00/ > >>> > >>> Looks good to me, but somebody more savvy with runtime stub > >>> generation should take a closer look. > >>> > >>> *) Should probably be "Why we are here?" > >>> > >>> 2867 assert(Universe::heap()->supports_object_pinning(), "Why we > >>> here?"); > >>> > >>> 2876 assert(Universe::heap()->supports_object_pinning(), "Why we > >>> here?"); > >>> > >>> > >>> Thanks, > >>> -Aleksey > >>> > From sgehwolf at redhat.com Thu May 3 17:15:03 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 03 May 2018 19:15:03 +0200 Subject: [8u] RFR(XS): 8202600: [Zero] Undefined behaviour in src/os_cpu/linux_zero/vm/os_linux_zero.cpp Message-ID: <1525367703.4079.29.camel@redhat.com> Hi, Please review this trivial change which fixes a potential crasher bug in JDK 8 Zero. JDK 10+ don't have this problem since they got fixed with JDK-8143245. I figured a smaller fix like this would be more likely to get accepted as a backport. If you'd rather have JDK-8143245 backported to JDK 8, that would be fine with me too. Bug: https://bugs.openjdk.java.net/browse/JDK-8202600 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8202600/webrev.jdk8.01/ Testing: We've been using this patch downstream for months now without issues. Thanks, Severin From stefan.karlsson at oracle.com Thu May 3 17:22:23 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 3 May 2018 19:22:23 +0200 Subject: RFR: 8200729: Conditional compilation of GCs In-Reply-To: References: Message-ID: Hi Vladimir, On 2018-05-03 18:56, Vladimir Kozlov wrote: > I see that you don't guard Use*GC flags with _ONLY macros. It is hard to > figure out from gcConfig.cpp what happened if UseG1GC is specified on > command line for VM which does not include it. What happens? Right. The current patch leaves the Use*GC flags available in all builds, so that we can accept the flag but give a warning: $ build/slowdebug-no-g1/jdk/bin/java -XX:+UseG1GC Java HotSpot(TM) 64-Bit Server VM warning: -XX:+UseG1GC not supported in this VM This is handled in GCConfig::select_gc_ergonomically with this line: NOT_G1GC( UNSUPPORTED_OPTION(UseG1GC);) That expands into: // Disable options not supported in this release, with a warning if they // were explicitly requested on the command-line #define UNSUPPORTED_OPTION(opt) \ do { \ if (opt) { \ if (FLAG_IS_CMDLINE(opt)) { \ warning("-XX:+" #opt " not supported in this VM"); \ } \ FLAG_SET_DEFAULT(opt, false); \ } \ } while(0) > > I don't see C1 changes but it looks like it was changed with 8201543. Right. There's no INCLUDE_ guards left in C1! > C2 changes seems fine but they will be also affected by 8202377. Exactly. > > What is left is AOT and JVMCI/Graal. We thought to implement "cross" > compilation when we can specify for which configuration we should > generate AOT code. It includes GC barriers code. Currently I see that > you keep all GCs in build as before so it is not a issue. And we record > VM version so we will not generate and use G1 barriers if it is not in > VM used by jaotc. > > So it seems to me compiler and AOT, Graal changes are fine. OK. Thanks for taking a close look at this. > > I would suggest in addition to hs-tier2 testing run hs-tier2-graal to > make sure Graal still works. I'll make sure to test with hs-tier2-graal. Thanks for reviewing! StefanK > > Thanks, > Vladimir > > On 5/2/18 7:37 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review these patches to allow for conditional compilation of >> the GCs in HotSpot. >> >> Full patch: >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/all/ >> >> (See below for a more fine-grained division into smaller patches) >> >> Today Parallel, G1, and CMS, are all guarded by INCLUDE_ALL_GCS. >> INCLUDE_ALL_GCS becomes defined to 1 for our server >> (--with-jvm-variants=server) builds, and is defined to 0 for the >> minimal (--with-jvm-variants=minimal) builds. There are also ways to >> forcefully remove these GCs from the compilation by configuring with, >> for example, --with-jvm-features=all-gcs. >> >> The proposed patch removes INCLUDE_ALL_GCS (and all-gcs) and replaces >> it with INCLUDE_CMSGC, INCLUDE_G1GC, and INCLUDE_PARALLELGC. In >> addition to that, INCLUDE_SERIALGC has been added to guard the Serial >> GC code. >> >> Future GCs should adopt this scheme before they get incorporated into >> the code base. Note, that there will be some files in gc/shared that >> are going to have to know about all GCs, but the goal is to have very >> few of these INCLUDE_ checks in the non-GC parts of HotSpot. >> >> With this patch, it's also going to be easier to stop compiling CMS >> when the time as come for it to move from deprecated to removed. >> >> Note, that even though this adds great flexibility, and allows for >> easy inclusion / exclusion of GCs, there's no guarantee that a >> specific combination of GCs is going to be maintained in the future. >> Just like not all combinations of the different Runtime features (CDS, >> NMT, JFR) and Compiler features (AOT, COMPILER1, COMPILER2) are >> guaranteed to compile and be supported. I've sanity checked the >> different GC configurations build locally, but I've only fully tested >> the "server" variant, and "minimal" has only been built locally. >> >> There's a more high-level discussion around the flexibility the >> --with-jvm-features flag adds, and if we really should allow it. Since >> this patch only builds upon that existing flexibility, and I don't >> change the two defined jvm variants, I'd appreciate if that discussion >> could be kept out of this review thread. For further discussion around >> this, please see: >> >> http://mail.openjdk.java.net/pipermail/build-dev/2018-April/021663.html >> >> This is the patch queue: >> >> The first patch simply cleans up some INCLUDE_ALL_GCS usages in >> platform-specific files. Some of these changes are already being >> cleaned up by other RFEs: >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/00.removeUnneededIncludeAllGCs/ >> >> >> >> The second patch pre-cleans some include files: >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/01.fixIncludes/ >> >> >> The following is the main patch, which include all relevant HotSpot >> changes. For a while now, we have been actively working on abstracting >> away GC specific code from non-GC directories, but as can be seen in >> this patch, there are still a few places left. Hopefully, we will get >> rid of most of these in the near future. >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/02.mainPatch/ >> >> >> The last patch adds the make file support to turn on and off the >> different GCs. The content of this patch has evolved from versions >> written by myself, Per, and Magnus Ihse Bursie. Magnus last proposed >> version used the names gc-cms, gc-g1, gc-parallel, and gc-serial, but >> I've changed them to cmsgc, g1gc, parallelgc, and serialgc, so that >> they match the INCLUDE_ defines. >> >> http://cr.openjdk.java.net/~stefank/8200729/webrev.04/03.selectIndivudualGCsMakePatch/ >> >> >> Thanks, >> StefanK From bsrbnd at gmail.com Thu May 3 17:52:27 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Thu, 3 May 2018 19:52:27 +0200 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> Message-ID: Hi, On 2 May 2018 at 22:40, Kim Barrett wrote: >> On May 2, 2018, at 5:10 AM, Michal Vala wrote: >> >> >> >> On 05/01/2018 07:59 PM, Kim Barrett wrote: >>>> On Apr 27, 2018, at 4:26 PM, Michal Vala wrote: >>>> Someone to sponsor this please? >>> Do you have a sponsor yet? If not, I?ll do it. >> >> No, I don't. I'd really appreciate if you sponsor it. > > Will do. I should be able to take care of it in the next day or so. > I've noted some similar issues at several other places which makes the JDK build fail on Linux with glibc = 2.26, see comments here: https://bugs.openjdk.java.net/browse/JDK-8202353 Here is a patch for that, does this look reasonable (tier1 seems OK)? Thanks, Bernard diff -r 1871c5d07caf src/java.base/unix/native/libjava/TimeZone_md.c --- a/src/java.base/unix/native/libjava/TimeZone_md.c Fri Apr 27 15:55:29 2018 -0700 +++ b/src/java.base/unix/native/libjava/TimeZone_md.c Thu May 03 17:59:31 2018 +0200 @@ -122,7 +122,9 @@ DIR *dirp = NULL; struct stat statbuf; struct dirent64 *dp = NULL; +#ifndef __linux__ struct dirent64 *entry = NULL; +#endif char *pathname = NULL; int fd = -1; char *dbuf = NULL; @@ -140,7 +142,7 @@ if (name_max < 1024) { name_max = 1024; } - +#ifndef __linux__ entry = (struct dirent64 *)malloc(offsetof(struct dirent64, d_name) + name_max + 1); if (entry == NULL) { (void) closedir(dirp); @@ -148,6 +150,9 @@ } while (readdir64_r(dirp, entry, &dp) == 0 && dp != NULL) { +#else + while ((dp = readdir64(dirp)) != NULL) { +#endif /* * Skip '.' and '..' (and possibly other .* files) */ @@ -213,10 +218,11 @@ free((void *) pathname); pathname = NULL; } - +#ifndef __linux__ if (entry != NULL) { free((void *) entry); } +#endif if (dirp != NULL) { (void) closedir(dirp); } diff -r 1871c5d07caf src/java.base/unix/native/libjava/UnixFileSystem_md.c --- a/src/java.base/unix/native/libjava/UnixFileSystem_md.c Fri Apr 27 15:55:29 2018 -0700 +++ b/src/java.base/unix/native/libjava/UnixFileSystem_md.c Thu May 03 17:59:31 2018 +0200 @@ -312,7 +312,9 @@ { DIR *dir = NULL; struct dirent64 *ptr; +#ifndef __linux__ struct dirent64 *result; +#endif int len, maxlen; jobjectArray rv, old; jclass str_class; @@ -324,14 +326,14 @@ dir = opendir(path); } END_PLATFORM_STRING(env, path); if (dir == NULL) return NULL; - +#ifndef __linux__ ptr = malloc(sizeof(struct dirent64) + (PATH_MAX + 1)); if (ptr == NULL) { JNU_ThrowOutOfMemoryError(env, "heap allocation failed"); closedir(dir); return NULL; } - +#endif /* Allocate an initial String array */ len = 0; maxlen = 16; @@ -339,7 +341,11 @@ if (rv == NULL) goto error; /* Scan the directory */ +#ifndef __linux__ while ((readdir64_r(dir, ptr, &result) == 0) && (result != NULL)) { +#else + while ((ptr = readdir64(dir)) != NULL) { +#endif jstring name; if (!strcmp(ptr->d_name, ".") || !strcmp(ptr->d_name, "..")) continue; @@ -360,7 +366,9 @@ (*env)->DeleteLocalRef(env, name); } closedir(dir); +#ifndef __linux__ free(ptr); +#endif /* Copy the final results into an appropriately-sized array */ old = rv; @@ -375,7 +383,9 @@ error: closedir(dir); +#ifndef __linux__ free(ptr); +#endif return NULL; } diff -r 1871c5d07caf src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c --- a/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Fri Apr 27 15:55:29 2018 -0700 +++ b/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Thu May 03 17:59:31 2018 +0200 @@ -726,12 +726,18 @@ char name_extra[PATH_MAX + 1 - sizeof result->d_name]; } entry; struct dirent64* ptr = &entry.buf; +#ifndef __linux__ int res; +#endif DIR* dirp = jlong_to_ptr(value); /* EINTR not listed as a possible error */ /* TDB: reentrant version probably not required here */ +#ifndef __linux__ res = readdir64_r(dirp, ptr, &result); +#else + ptr = result = readdir64(dirp); +#endif #ifdef _AIX /* On AIX, readdir_r() returns EBADF (i.e. '9') and sets 'result' to NULL for the */ @@ -741,10 +747,12 @@ } #endif +#ifndef __linux__ if (res != 0) { throwUnixException(env, res); return NULL; } else { +#endif if (result == NULL) { return NULL; } else { @@ -755,7 +763,9 @@ } return bytes; } +#ifndef __linux__ } +#endif } JNIEXPORT void JNICALL diff -r 1871c5d07caf src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c --- a/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c Fri Apr 27 15:55:29 2018 -0700 +++ b/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c Thu May 03 17:59:31 2018 +0200 @@ -75,10 +75,10 @@ #endif /* _ALLBSD_SOURCE */ static struct dirent* read_dir(DIR* dirp, struct dirent* entry) { -#ifdef __solaris__ +#if defined(__solaris__) || defined(__linux__) struct dirent* dbuf = readdir(dirp); return dbuf; -#else /* __linux__ || _ALLBSD_SOURCE */ +#else /* _ALLBSD_SOURCE */ struct dirent* p; if (readdir_r(dirp, entry, &p) == 0) { return p; From zgu at redhat.com Thu May 3 17:53:47 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 3 May 2018 13:53:47 -0400 Subject: RFR 8199868: Support JNI critical functions in object pinning API In-Reply-To: References: <931060af-b44d-f348-92ba-e98d623d4c84@redhat.com> <19c791e7-1bf8-69ef-7090-b7da800f2021@redhat.com> <16c60f53-d835-29f1-981f-4f70537ceb62@oracle.com> Message-ID: Hi Ian, On 05/03/2018 01:05 PM, Ian Rogers wrote: > Re JDK-8199868 - should pinning be limited to just critical APIs? The > non-critical variants could also pin to avoid copies. > I see this as related to?JDK-8199919. It'd be nice to just move the > critical APIs to use the same logic as the non-critical APIs, then > pinning just becomes a performance optimization to avoid copying. Re > JDK-8199919's comment that there is no deprecation mechanism, compilers > like GCC allow function attributes that warn when a deprecated function > is used. This is an interesting idea. I filed JDK-8202604 (https://bugs.openjdk.java.net/browse/JDK-8202604) to track this. Thanks, -Zhengyu > > Thanks, > Ian > > > > On Wed, May 2, 2018 at 2:12 PM Per Liden > wrote: > > Hi, > > On 05/02/2018 09:41 PM, Zhengyu Gu wrote: > > Hi, > > > > Can I have reviews for this RFR? > > > > This patch completes object pinning for JNI critical section, > provides > > critical native support. > > > > The approach is quite straightforward: > > > > During generating native wrapper for critical native method, it > > generates runtime call to pin every array argument, before > unpacks them. > > > > For pinned objects, it also needs to save them for unpinning > after JNI > > function call completes. > > > > If argument is passed on stack, it saves pinned object at the > original > > slot (as pin_object() may move the object). For register based > > arguments, it reuses oop handle area (where GCLocker based > > implementation saves register based arguments for safepoints). > > > > Currently, only Shenandoah uses object pinning for JNI critical > section, > > this patch has been baked quite some time there. However, I am > new to > > Runtime Stub code, I would appreciate your comments and suggestions. > > > > I rebased patch to jdk/jdk repo. > > > > Webrev: http://cr.openjdk.java.net/~zgu/8199868/webrev.02/ > > Just want to say that I would really like to see this patch go in. As > mentioned, it completes the object pinning story and it's useful for > other GCs too (at least ZGC and possibly G1). However, I also agree > with > Aleksey that some one who really knows this code needs to review this. > Unfortunately that's not me. Anyone? > > cheers, > Per > > > > > Thanks, > > > > -Zhengyu > > > > > > On 04/06/2018 10:35 PM, Zhengyu Gu wrote: > >> Offline discussion with Aleksey, he suggested that > >> pin/unpin_critical_native_array methods can be made more generic as > >> pin/unpin_object. > >> > >> Updated webrev: http://cr.openjdk.java.net/~zgu/8199868/webrev.01/ > >> > >> Test: > >>? ? Reran all tests, submit-hs tests still clean. > >> > >> Thanks, > >> > >> -Zhengyu > >> > >> On 04/06/2018 08:55 AM, Aleksey Shipilev wrote: > >>> On 04/04/2018 07:47 PM, Zhengyu Gu wrote: > >>>> Please review this patch that adds JNI critical native support to > >>>> object pinning. > >>>> > >>>> Shenandoah does not block GC while JNI critical session is in > >>>> progress. This patch allows it to pin > >>>> all incoming array objects before critical native call, and unpin > >>>> them after call completes. > >>>> > >>>> > >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8199868 > >>>> Webrev: http://cr.openjdk.java.net/~zgu/8199868/webrev.00/ > >>> > >>> Looks good to me, but somebody more savvy with runtime stub > >>> generation should take a closer look. > >>> > >>> *) Should probably be "Why we are here?" > >>> > >>> 2867? ?assert(Universe::heap()->supports_object_pinning(), "Why we > >>> here?"); > >>> > >>> 2876? ?assert(Universe::heap()->supports_object_pinning(), "Why we > >>> here?"); > >>> > >>> > >>> Thanks, > >>> -Aleksey > >>> > From kim.barrett at oracle.com Thu May 3 18:19:30 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 3 May 2018 14:19:30 -0400 Subject: RFR: 8200557: OopStorage parallel iteration scales poorly In-Reply-To: <63eef203-796d-a6cb-6c40-baee5e7a299c@oracle.com> References: <7CBB6FF2-AC36-4723-BBA1-B4FA277C707B@oracle.com> <2C30CDA1-7070-49A3-9D40-8625EDBA582E@oracle.com> <63eef203-796d-a6cb-6c40-baee5e7a299c@oracle.com> Message-ID: <60DC2F59-AB9D-4EBD-9B35-74FAF21E1B4F@oracle.com> > On May 2, 2018, at 5:42 PM, coleen.phillimore at oracle.com wrote: > > > Changes look good. Thanks. > > I like the new comments. I still think pointing out that these are forward declarations and not what they are used for here is not very interesting, though. A short 1/2 liner as suggested would be better in case one doesn't need to read the rest of the code. >> [?] >> > Thanks for moving the comments into the implementation. I like where they are now. > > If you decide to update any more comments, I don't need to see another webrev. For the record, I'm making this additional change: - class Block; // Forward decl; defined in .inline.hpp file. - class BlockArray; // Forward decl; defined in .inline.hpp file. - class BlockEntry; // Forward decl; defined in .inline.hpp file. + class Block; // Fixed-size array of oops, plus bookkeeping. + class BlockArray; // Array of Blocks, plus bookkeeping. + class BlockEntry; // Provides BlockList links in a Block. I really don't remember why the old comments were as they were; that's not my normal commenting style. Maybe someone asked for something like that in pre-review? But I can't think who would have, or find any record of such. Oh well. From markus.gronlund at oracle.com Thu May 3 18:33:05 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Thu, 3 May 2018 11:33:05 -0700 (PDT) Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: <5AEB2C86.1050701@oracle.com> References: <5AE0612F.8060200@oracle.com> <9a5f2a18-f7d4-d1ad-6618-b92bb7d71dd9@samersoff.net> <3B7915CA-50D7-4303-8C7B-C17D5911BC29@oracle.com> <5AEB2C86.1050701@oracle.com> Message-ID: <5b81e4aa-d8f1-4da8-8592-598049502c54@default> Hi Karen, many thanks for taking a look. I have added a few, short and complementary answers inline to Erik's on your first set of questions. Again, much appreciate you looking into this. Thanks Markus -----Original Message----- From: Erik Gahlin Sent: den 3 maj 2018 17:37 To: Karen Kinnear Cc: hotspot-jfr-dev at openjdk.java.net; hotspot-dev Source Developers Subject: Re: RFR(XL): 8199712: Flight Recorder On 2018-05-02 00:08, Karen Kinnear wrote: > Lots of great work here. Review part 1 below. > > I will not be reviewing build files or test files - I believe you have > other coverage for those. I did this quickly, so if any of my comments > don?t make sense after detailed study - just let me know. > > I owe you a review of the share/jfr files still. > > A couple of issues that I think should be addressed before checking in > the code: > > 1. New logTag.hpp log tags > One is called ?setting? -> could you make this less general please? > like jfrsetting? > Or this a subcategory so we already know we are under ?jfr?? > I think the idea is that tags should be kept simple, so they can be reused by different subsystems, and then you combine them when you log things, i.e. jfr+setting > Note: I would expect that log tags would be added to the CSR since > that is a supported interface > > 2. Code removed two command line flags: EnableTracing and > UseLockedTracing with product flags, the normal approach is to > deprecate given they now do nothing, I would strongly recommend that > you Obsolete before removing i.e. accept the flags, but do not do > anything for 1 release, to allow script migration. Expire the flags in > the next release. So test that you get a warning if you use them for > JDK 11. > > Note: CSR says EnableTracing flag is removed -> please update CSR for > both to be made obsolete > I will update the CSR and the code, but shouldn't we use Expired, since the feature is removed. (If somebody actually built something on top of -XX:EnableTracing you want it to fail fast. Starting the JVM even though the feature is removed is hiding problems, which may make things worse at a later stage.) > 3. CSR question - did jcmd change at all? Not really. We did a small code change that impacts -XX:StartFlightRecording and JFR.start. If you specify -XX:StartFlightRecording=filename=dump.jfr a recording will be dumped on exit. This is the only reasonable behavior. You wouldn't specify a filename unless you wanted to dump it. In previous CSRs we have not specified behavior/heuristics of flags that are unspecified. If a user specifies -XX:StartFlightRecording=filename=dump.jfr,dumponexit=true/false we still honor that. > If it did I would expect that to also be covered in the CSR, but I > think it did not. > > A couple of issues that should be addressed before JDK11 ships > > 1. FastTime naming - we need to not present an attractive nuisance here. > Please rename to clarify at least: > is_platform_trusted_for_fast_time() -> > ?platform_provides_fast_unordered_time? or something equally clear > about risk in the name > - there is a local with a similar need to change name > FastTime* APIs: e.g. FastTimeElapsedCounterSource -> please include > Unordered in the name > For folks new to this issue - the rdtsc (read time stamp counter) > instruction can return different times across different sockets, > so time can go backwards, so we do not use this for any vm or java > times, just for diagnostic timestamps > and we highly recommend that folks modifying the jvm do not start > using it accidentally. > Markus will comment fast time support. [MG] Answer: Yes I agree, "is_platform_trusted" is already removed; I will change the name of the time source and add a disclaimer (think you wrote some of the original wording): // Not guaranteed to be synchronized across hardware threads and // therefore software threads, and can be updated asynchronously // by software. now() can jump backwards as well as jump forward // when threads query different cores/sockets. // Very much not recommended for general use. Caveat emptor. class FastUnorderedElapsedCounterSource { public: typedef jlong Type; static uint64_t frequency(); static Type now(); static double seconds(Type value); static uint64_t milliseconds(Type value); static uint64_t microseconds(Type value); static uint64_t nanoseconds(Type value); }; > 2. Docker container compatibility > Can you please check with Bob Vandette e.g. for cgroup handling - the > new os_perf_linux.cpp may have some overlap/need modifications simliar > to what he did with os_linux.cpp - e.g. for number of cpus, sockets, > etc. > And please test with a container - Misha can give guidance here. > See separate e-mail > Other comments: > > 1. #INCLUDE_JFR > Did you build with #INCLUDE_JFR not defined? > Is the theory that we still may want a minimalVM without JFR? > The model is not clear to me in terms of when you include files or > generate events e.g. c1_GraphBuilder.cpp, systemDictionary.cpp, > synchronizer.cpp, ? Could this please be on your future clean-up list? > p.s. I do like the extraction of the post_xxx_event logic [MG] Answer: Yes, it is possible to build the VM without JFR. The current patch has this configure capability: --enable-jfr=[no/yes] Although that will change (input from build team), and there will instead be this (updated in next webrev): --with-jvm-features=-jfr Listing ??jfr? (note the minus sign) in here will build a VM with no JFR support. I think we have lost track of what the actual requirements are currently for MinimalVM and if this is (still) the main reason we do this. The image size reduction for excluding JFR at compile time is minimal indeed, but since we already had the conditional model from the open/closed split, we brought that over pretty much as-is. Please see my earlier mail to Coleen about the headers and include guards, http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-May/032027.htm > 2. Duplication > Is there any duplication in the VM_Version and VM_Version_ext handling? > And are you covering older CPUs we no longer need to cover? Vladimir K > would know the precise list. > We have added a RFE for this: https://bugs.openjdk.java.net/browse/JDK-8202579 > 3. objectCountEventSender.hpp: comment to clean up: > + // The following two functions have the exact same signature as // > hotspot/src/share/vm/gc_implementation/shared/objectCountEventSender.h > pp > + // > + // This file will replace the open file if a closed build is performed. > + // These function signatures can therefore not be changed if the > + open // signatures aren't changed as well. > Will fix. > 4. ObjectMonitor::enter > Why was this event modified differently from others - i.e. it stores > the stack trace in the _owner field? And is there a reason we flush here? [MG] Answer: This is a new capability that will allow for better handing inside and around the vicinity of critical sections (such as ObjectMonitor::Enter()): This code does not store the stacktrace in the _owner field however, instead: The additional functionality will ensure that the thread has enough space (thread local) to accomplish a write of a JavaMonitorEvent and will also capture and save the java stacktrace _before_ entering wait. This is to ensure that we need not do these operations at the point of returning from the wait, which implies we are now the holder of the monitor. This relative simple solution allows us to do any "slower path" work pre-emptively, instead of when the thread is the exclusive owner of some precious resource. Note that the flush is conditional, so it is only done iff the event will not fit. This solution is generic and might be put to good effect at other sites as well. There is some small follow-up work here though in that we currently don't have good size estimation for events that use strings. > 5. jfrMacros.hpp > I don't see any users of JFR_FLUSH? > Markus will comment 4 and 5. [MG] Answer: See previous answer, this primitive is part of the extended capability. Might not yet be used at sites. > 6. vmStructs.cpp removed all the VM_XXX_TRACE which was conditional. > Did SA never use it? Or will that need updating? > The SA support has not worked since JDK 9. Bugs have been filed, but they have not been fixed. The support was added by sustaining so it was possible to extract a .jfr file from a core dump. With JDK 9, a recording is written if a crash occurs, so the functionality is not needed anymore. Even if somebody would like the functionality back, for whatever reason, most of declarations are wrong anyway. I don't think we should bring in code that is broken into OpenJDK. SA is also a nightmare to maintain. Let's keep it out. > 7. Assuming jdk.management.jfr and jdk.jfr changes were just > name/include locations/copyrights but nothing substantial - I did a > quick skim We have only changed copyright headers and spelling errors in the jdk modules. Thanks Erik > > thanks, > Karen > >> On Apr 28, 2018, at 4:17 AM, Erik Gahlin > > wrote: >> >> Yes, If we don?t include JFR, we should get the stubs >> >> Will fix. >> >> Erik >> >>> On 28 Apr 2018, at 09:59, Dmitry Samersoff >> > wrote: >>> >>> Erik, >>> >>> globals.cpp:1051 these changes seems to be unnecessary. >>> >>> -Dmitry >>> >>> On 25.04.2018 14:06, Erik Gahlin wrote: >>>> Greetings, >>>> >>>> Could I have a review of 8199712: Flight Recorder >>>> >>>> As mentioned in the preview [1] the tracing backend has been removed. >>>> Event metadata has been consolidated into a single XML file and >>>> event classes are now generated by GenerateJfrFiles.java. >>>> >>>> Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. >>>> >>>> For details about the feature, see the JEP: >>>> https://bugs.openjdk.java.net/browse/JDK-8193393 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~egahlin/8199712.0/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8199712 >>>> >>>> [1] >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/03135 >>>> 9.html >>>> >>>> Thanks >>>> Erik and Markus >>> >>> >>> -- >>> Dmitry Samersoff >>> http://devnull.samersoff.net >>> * There will come soft rains ... >>> >> > From kim.barrett at oracle.com Thu May 3 19:28:00 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 3 May 2018 15:28:00 -0400 Subject: RFR: 8200557: OopStorage parallel iteration scales poorly In-Reply-To: <5AEB1155.9090608@oracle.com> References: <7CBB6FF2-AC36-4723-BBA1-B4FA277C707B@oracle.com> <5AE1D5DF.90909@oracle.com> <5AEB1155.9090608@oracle.com> Message-ID: <17C86656-6FE6-4070-BCD7-4B34380D7B2F@oracle.com> > On May 3, 2018, at 9:40 AM, Erik ?sterlund wrote: >>> And same here where a block array is created without retain reference counter, which is then incremented manually inside of replace_active_array. Seems to me like it would be easier to make BlockArray::create return already retained block arrays. >> My experience with using refcounts has been that starting with an >> implicit reference at construction time (e.g. initialize the refcount >> to 1 at construction) is more often than not really not helpful. >> >> However, this comment led me to notice the constructor for OopStorage >> should be checking the result of BlockArray::create. > > Okay. Having used Objective-C, which was explicitly reference counted, and by convention returned new objects with a ref count of 1, I am biased to think that is a good convention. Then you know that if you ever see a ref count of 0, it's because it should be deleted, and never because it either needs to be deleted or was just created. But that is just my opinion I guess, so I'm okay if you don't agree about that. Dredging through old memories, I think the bad experiences I've had with an initial refcount of 1 were related to working in an environment with smart pointers and with exceptions. So I could be convinced to reconsider this in a followup, if you want. > Looks good. Thanks. From kim.barrett at oracle.com Thu May 3 19:43:57 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 3 May 2018 15:43:57 -0400 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> Message-ID: > On May 3, 2018, at 1:52 PM, B. Blaser wrote: > > Hi, > > On 2 May 2018 at 22:40, Kim Barrett wrote: >>> On May 2, 2018, at 5:10 AM, Michal Vala wrote: >>> >>> >>> >>> On 05/01/2018 07:59 PM, Kim Barrett wrote: >>>>> On Apr 27, 2018, at 4:26 PM, Michal Vala wrote: >>>>> Someone to sponsor this please? >>>> Do you have a sponsor yet? If not, I?ll do it. >>> >>> No, I don't. I'd really appreciate if you sponsor it. >> >> Will do. I should be able to take care of it in the next day or so. >> > > I've noted some similar issues at several other places which makes the > JDK build fail on Linux with glibc = 2.26, see comments here: > https://bugs.openjdk.java.net/browse/JDK-8202353 > > Here is a patch for that, does this look reasonable (tier1 seems OK)? I think we should move in the direction of eliminating the use of readdir_r entirely, either under JDK-8202353, or leave that as only the HotSpot part and have another RFE for the uses in java.base/unix/native. From thomas.stuefe at gmail.com Thu May 3 19:56:35 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 03 May 2018 19:56:35 +0000 Subject: [8u] RFR(XS): 8202600: [Zero] Undefined behaviour in src/os_cpu/linux_zero/vm/os_linux_zero.cpp In-Reply-To: <1525367703.4079.29.camel@redhat.com> References: <1525367703.4079.29.camel@redhat.com> Message-ID: Hi Severin, Looks fine and trivial. Best regards, Thomas On Thu 3. May 2018 at 19:15, Severin Gehwolf wrote: > Hi, > > Please review this trivial change which fixes a potential crasher bug > in JDK 8 Zero. JDK 10+ don't have this problem since they got fixed > with JDK-8143245. I figured a smaller fix like this would be more > likely to get accepted as a backport. If you'd rather have JDK-8143245 > backported to JDK 8, that would be fine with me too. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202600 > webrev: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8202600/webrev.jdk8.01/ > > Testing: We've been using this patch downstream for months now without > issues. > > Thanks, > Severin > From shade at redhat.com Thu May 3 20:00:56 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 3 May 2018 22:00:56 +0200 Subject: [8u] RFR(XS): 8202600: [Zero] Undefined behaviour in src/os_cpu/linux_zero/vm/os_linux_zero.cpp In-Reply-To: <1525367703.4079.29.camel@redhat.com> References: <1525367703.4079.29.camel@redhat.com> Message-ID: <7d2c62d1-c306-6def-9dfb-16a238e18662@redhat.com> On 05/03/2018 07:15 PM, Severin Gehwolf wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8202600 > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8202600/webrev.jdk8.01/ Looks good. Seems like no other arch has this problem? -Aleksey From Derek.White at cavium.com Thu May 3 21:26:59 2018 From: Derek.White at cavium.com (White, Derek) Date: Thu, 3 May 2018 21:26:59 +0000 Subject: UseNUMA membind Issue in openJDK In-Reply-To: References: <9a0310b7-2880-db69-cfbc-7abba844ecbf@oracle.com> Message-ID: Hi Swati, I think this was reported as JDK-8189922 "UseNUMA memory interleaving allocates heap for unavailable nodes". https://bugs.openjdk.java.net/browse/JDK-8189922 Thanks for working on a fix for this! - Derek > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > Behalf Of Swati Sharma > Sent: Wednesday, May 02, 2018 6:24 AM > To: David Holmes > Cc: hotspot-dev at openjdk.java.net; Prakash.Raghavendra at amd.com > Subject: Re: UseNUMA membind Issue in openJDK > > Hi David, > > I have localized the struct bitmask declaration in os_linux.cpp. > > Here is the updated patch > ===================================PATCH====================== > ============================= > diff --git a/src/hotspot/os/linux/os_linux.cpp > b/src/hotspot/os/linux/os_linux.cpp > --- a/src/hotspot/os/linux/os_linux.cpp > +++ b/src/hotspot/os/linux/os_linux.cpp > @@ -2832,14 +2832,42 @@ > // Map all node ids in which is possible to allocate memory. Also nodes are > // not always consecutively available, i.e. available from 0 to the highest > // node number. > + // If the nodes have been bound explicitly using numactl membind, > + then // allocate memory from those nodes only. > for (size_t node = 0; node <= highest_node_number; node++) { > - if (Linux::isnode_in_configured_nodes(node)) { > + if (Linux::isnode_in_bounded_nodes(node)) { > ids[i++] = node; > } > } > return i; > } > > +extern "C" struct bitmask { > + unsigned long size; /* number of bits in the map */ > + unsigned long *maskp; > +}; > +// Check if single memory node bound. > +// Returns true if single memory node bound. > +bool os::Linux::issingle_node_bound() { > + struct bitmask* bmp = _numa_get_membind != NULL ? > _numa_get_membind() : > NULL; > + if(bmp == NULL) return false; > + int issingle = 0; > + // System can have more than 64 nodes so check in all the elements of > + // unsigned long array for (unsigned long i = 0; i < (bmp->size / (8 > + * sizeof(unsigned long))); > i++) { > + if (bmp->maskp != NULL && (((bmp->maskp[i]) & (((bmp->maskp[i])) - > + 1)) > == 0)) { > + issingle++; > + } else if (bmp->maskp[i] == 0) { > + continue; > + } else { > + return false; > + } > + } > + if (issingle == 1) > + return true; > + return false; > +} > + > bool os::get_page_info(char *start, page_info* info) { > return false; > } > @@ -2930,6 +2958,10 @@ > libnuma_dlsym(handle, > "numa_bitmask_isbitset"))); > set_numa_distance(CAST_TO_FN_PTR(numa_distance_func_t, > libnuma_dlsym(handle, "numa_distance"))); > + > set_numa_set_membind(CAST_TO_FN_PTR(numa_set_membind_func_t, > + libnuma_dlsym(handle, > "numa_set_membind"))); > + > set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > + libnuma_v2_dlsym(handle, > "numa_get_membind"))); > > if (numa_available() != -1) { > set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, > "numa_all_nodes")); @@ -3054,6 +3086,8 @@ > os::Linux::numa_set_bind_policy_func_t os::Linux::_numa_set_bind_policy; > os::Linux::numa_bitmask_isbitset_func_t os::Linux::_numa_bitmask_isbitset; > os::Linux::numa_distance_func_t os::Linux::_numa_distance; > +os::Linux::numa_set_membind_func_t os::Linux::_numa_set_membind; > +os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; > unsigned long* os::Linux::_numa_all_nodes; struct bitmask* > os::Linux::_numa_all_nodes_ptr; struct bitmask* > os::Linux::_numa_nodes_ptr; @@ -4962,8 +4996,9 @@ > if (!Linux::libnuma_init()) { > UseNUMA = false; > } else { > - if ((Linux::numa_max_node() < 1)) { > - // There's only one node(they start from 0), disable NUMA. > + if ((Linux::numa_max_node() < 1) || Linux::issingle_node_bound()) { > + // If there's only one node(they start from 0) or if the process > + // is bound explicitly to a single node using membind, disable > NUMA. > UseNUMA = false; > } > } > diff --git a/src/hotspot/os/linux/os_linux.hpp > b/src/hotspot/os/linux/os_linux.hpp > --- a/src/hotspot/os/linux/os_linux.hpp > +++ b/src/hotspot/os/linux/os_linux.hpp > @@ -228,6 +228,8 @@ > typedef int (*numa_tonode_memory_func_t)(void *start, size_t size, int > node); > typedef void (*numa_interleave_memory_func_t)(void *start, size_t size, > unsigned long *nodemask); > typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t > size, struct bitmask* mask); > + typedef void (*numa_set_membind_func_t)(struct bitmask *mask); > + typedef struct bitmask* (*numa_get_membind_func_t)(void); > > typedef void (*numa_set_bind_policy_func_t)(int policy); > typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, > unsigned int n); @@ -244,6 +246,8 @@ > static numa_set_bind_policy_func_t _numa_set_bind_policy; > static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; > static numa_distance_func_t _numa_distance; > + static numa_set_membind_func_t _numa_set_membind; static > + numa_get_membind_func_t _numa_get_membind; > static unsigned long* _numa_all_nodes; > static struct bitmask* _numa_all_nodes_ptr; > static struct bitmask* _numa_nodes_ptr; @@ -259,6 +263,8 @@ > static void set_numa_set_bind_policy(numa_set_bind_policy_func_t func) > { _numa_set_bind_policy = func; } > static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t > func) { _numa_bitmask_isbitset = func; } > static void set_numa_distance(numa_distance_func_t func) { > _numa_distance = func; } > + static void set_numa_set_membind(numa_set_membind_func_t func) { > _numa_set_membind = func; } > + static void set_numa_get_membind(numa_get_membind_func_t func) { > _numa_get_membind = func; } > static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes = > ptr; } > static void set_numa_all_nodes_ptr(struct bitmask **ptr) { > _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > static void set_numa_nodes_ptr(struct bitmask **ptr) { _numa_nodes_ptr > = (ptr == NULL ? NULL : *ptr); } @@ -320,6 +326,15 @@ > } else > return 0; > } > + // Check if node in bounded nodes > + static bool isnode_in_bounded_nodes(int node) { > + struct bitmask* bmp = _numa_get_membind != NULL ? > + _numa_get_membind() > : NULL; > + if (bmp != NULL && _numa_bitmask_isbitset != NULL && > _numa_bitmask_isbitset(bmp, node)) { > + return true; > + } else > + return false; > + } > + static bool issingle_node_bound(); > }; > > #endif // OS_LINUX_VM_OS_LINUX_HPP > > =============================================================== > ============================= > > Thanks, > Swati > > On Thu, Apr 26, 2018 at 6:10 PM, David Holmes > wrote: > > > > Hi Swati, > > > > On 26/04/2018 10:20 PM, Swati Sharma wrote: > >> > >> Hi Everyone, > >> > >> I work at AMD and this is my first patch as a new member of openJDK > >> community. > > > > > > Welcome! > > > > I can't comment on the actual NUMA details of the patch (though I can > > see > what you're doing), but the struct bitmask declaration in os.hpp should be > localized in os_linux.hpp as far as I can see, as it's only needed internally in > the Linux code. > > > > Thanks, > > David > > ----- > > > > > >> I have found some issue while running specjbb2015 composite workload > with > >> the flag -XX:+UseNUMA. It seems that JVM does not allocate memory > according > >> to the explicit node binding done using "numactl --membind". > >> > >> E.g. If bound to a single memory node, JVM divides the whole heap > >> based > on > >> the total number of numa nodes available on the system which creates > >> more logical groups(lgrps) than required which cannot be used except the > one. > >> > >> The following examples will explain clearly : > >> (Note : Collected GC logs with > >> -Xlog:gc*=debug:file=gc.log:time,uptimemillis) > >> 1) Allocating a heap of 22GB for single node divides the whole heap > >> in 8 lgrp(Actual no of Nodes are 8) > >> $numactl --cpunodebind=0 --membind=0 java -Xmx24g -Xms24g > >> -Xmn22g -XX:+UseNUMA > >> > >> eden space 22511616K(22GB), 12% used > >> lgrp 0 space 2813952K, 100% used lgrp 1 space > >> 2813952K, 0% used lgrp 2 space 2813952K, 0% used > >> lgrp 3 space 2813952K, 0% used lgrp 4 > space > >> 2813952K, 0% used lgrp 5 space 2813952K, 0% used > >> lgrp 6 space 2813952K, 0% used lgrp 7 > space > >> 2813952K, 0% used > >> > >> Observation : Instead of disabling UseNUMA for single node binding > >> JVM divides the memory in 8 lgrps and allocates memory always on the > >> bounded node hence in eden space allocation never happens more than > 12%. > >> > >> 2) Another case of binding to node 0 and 7 results in dividing the > >> heap > in > >> 8lgrp > >> $numactl --cpunodebind=0,7 ?membind=0,7 java -Xms50g -Xmx50g - > Xmn45g > >> -XX:+UseNUMA > >> > >> eden space 46718976K, 6% used > >> lgrp 0 space 5838848K, 14% used lgrp 1 space > 5838848K, > >> 0% used lgrp 2 space 5838848K, 0% used > >> lgrp 3 space 5838848K, 0% used lgrp 4 space > >> 5838848K, 0% used lgrp 5 space 5838848K, 0% > >> used > >> lgrp 6 space 5838848K, 0% used lgrp 7 space > >> 5847040K, 35% used > >> > >> Observation : Similar to first case allocation happens only on 0th > >> and > 7th > >> node and rest of the lgrps never gets used. > >> > >> After applying the patch, JVM divides the given heap size according > >> to > the > >> bounded memory nodes only. > >> > >> 1) Binding to single node disables UseNUMA > >> eden space 46718976K(45GB), 99% used > >> > >> Observation : UseNUMA gets disabled hence no lgrp creation and the > >> whole heap allocation happens on the bounded node. > >> > >> 2) Binding to node 0 and 7 > >> $ numactl --cpunodebind=0,7 ?membind=0,7 java -Xms50g -Xmx50g > -Xmn45g > >> -XX:+UseNUMA > >> eden space 46718976K(45GB), 99% used > >> lgrp 0 space 23359488K(23.5GB), 100% used lgrp 7 space > >> 23359488K(23.5GB), 99% used > >> > >> Observation : Only two lgrps gets created and heap size gets divided > >> equally in both nodes. > >> > >> If there is no binding, then JVM will divide the whole heap based on > >> the number of NUMA nodes available on the system. > >> > >> The following patch fixes the issue(attached also). > >> Please review and let me know your comments. > >> > >> Regression testing using jtreg (make -J=1 run-test-tier1 > >> run-test-tier2) didn't show any new failures. > >> > >> > ===============================PATCH========================== > ============== > >> diff --git a/src/hotspot/os/linux/os_linux.cpp > >> b/src/hotspot/os/linux/os_linux.cpp > >> --- a/src/hotspot/os/linux/os_linux.cpp > >> +++ b/src/hotspot/os/linux/os_linux.cpp > >> @@ -2832,8 +2832,10 @@ > >> // Map all node ids in which is possible to allocate memory. Also > nodes > >> are > >> // not always consecutively available, i.e. available from 0 to > >> the highest > >> // node number. > >> + // If the nodes have been bound explicitly using numactl membind, > >> + then // allocate memory from those nodes only. > >> for (size_t node = 0; node <= highest_node_number; node++) { > >> - if (Linux::isnode_in_configured_nodes(node)) { > >> + if (Linux::isnode_in_bounded_nodes(node)) { > >> ids[i++] = node; > >> } > >> } > >> @@ -2930,6 +2932,10 @@ > >> > >> libnuma_dlsym(handle, "numa_bitmask_isbitset"))); > >> set_numa_distance(CAST_TO_FN_PTR(numa_distance_func_t, > >> libnuma_dlsym(handle, > >> "numa_distance"))); > >> + > set_numa_set_membind(CAST_TO_FN_PTR(numa_set_membind_func_t, > >> + libnuma_dlsym(handle, > >> "numa_set_membind"))); > >> + > set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > >> + libnuma_v2_dlsym(handle, > >> "numa_get_membind"))); > >> > >> if (numa_available() != -1) { > >> set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, > >> "numa_all_nodes")); @@ -3054,6 +3060,8 @@ > >> os::Linux::numa_set_bind_policy_func_t > os::Linux::_numa_set_bind_policy; > >> os::Linux::numa_bitmask_isbitset_func_t > os::Linux::_numa_bitmask_isbitset; > >> os::Linux::numa_distance_func_t os::Linux::_numa_distance; > >> +os::Linux::numa_set_membind_func_t os::Linux::_numa_set_membind; > >> +os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; > >> unsigned long* os::Linux::_numa_all_nodes; > >> struct bitmask* os::Linux::_numa_all_nodes_ptr; > >> struct bitmask* os::Linux::_numa_nodes_ptr; @@ -4962,8 +4970,9 @@ > >> if (!Linux::libnuma_init()) { > >> UseNUMA = false; > >> } else { > >> - if ((Linux::numa_max_node() < 1)) { > >> - // There's only one node(they start from 0), disable NUMA. > >> + if ((Linux::numa_max_node() < 1) || > >> + Linux::issingle_node_bound()) > { > >> + // If there's only one node(they start from 0) or if the process > >> + // is bound explicitly to a single node using membind, > >> + disable > >> NUMA. > >> UseNUMA = false; > >> } > >> } > >> diff --git a/src/hotspot/os/linux/os_linux.hpp > >> b/src/hotspot/os/linux/os_linux.hpp > >> --- a/src/hotspot/os/linux/os_linux.hpp > >> +++ b/src/hotspot/os/linux/os_linux.hpp > >> @@ -228,6 +228,8 @@ > >> typedef int (*numa_tonode_memory_func_t)(void *start, size_t > >> size, > int > >> node); > >> typedef void (*numa_interleave_memory_func_t)(void *start, size_t > size, > >> unsigned long *nodemask); > >> typedef void (*numa_interleave_memory_v2_func_t)(void *start, > >> size_t size, struct bitmask* mask); > >> + typedef void (*numa_set_membind_func_t)(struct bitmask *mask); > >> + typedef struct bitmask* (*numa_get_membind_func_t)(void); > >> > >> typedef void (*numa_set_bind_policy_func_t)(int policy); > >> typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, > >> unsigned int n); @@ -244,6 +246,8 @@ > >> static numa_set_bind_policy_func_t _numa_set_bind_policy; > >> static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; > >> static numa_distance_func_t _numa_distance; > >> + static numa_set_membind_func_t _numa_set_membind; static > >> + numa_get_membind_func_t _numa_get_membind; > >> static unsigned long* _numa_all_nodes; > >> static struct bitmask* _numa_all_nodes_ptr; > >> static struct bitmask* _numa_nodes_ptr; @@ -259,6 +263,8 @@ > >> static void set_numa_set_bind_policy(numa_set_bind_policy_func_t > func) { > >> _numa_set_bind_policy = func; } > >> static void > >> set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t > func) > >> { _numa_bitmask_isbitset = func; } > >> static void set_numa_distance(numa_distance_func_t func) { > >> _numa_distance = func; } > >> + static void set_numa_set_membind(numa_set_membind_func_t func) > { > >> _numa_set_membind = func; } > >> + static void set_numa_get_membind(numa_get_membind_func_t func) > { > >> _numa_get_membind = func; } > >> static void set_numa_all_nodes(unsigned long* ptr) { > >> _numa_all_nodes > = > >> ptr; } > >> static void set_numa_all_nodes_ptr(struct bitmask **ptr) { > >> _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > >> static void set_numa_nodes_ptr(struct bitmask **ptr) { > _numa_nodes_ptr = > >> (ptr == NULL ? NULL : *ptr); } > >> @@ -320,6 +326,34 @@ > >> } else > >> return 0; > >> } > >> + // Check if node in bounded nodes > >> + static bool isnode_in_bounded_nodes(int node) { > >> + struct bitmask* bmp = _numa_get_membind != NULL ? > _numa_get_membind() > >> : NULL; > >> + if (bmp != NULL && _numa_bitmask_isbitset != NULL && > >> _numa_bitmask_isbitset(bmp, node)) { > >> + return true; > >> + } else > >> + return false; > >> + } > >> + // Check if a single node is bound static bool > >> + issingle_node_bound() { > >> + struct bitmask* bmp = _numa_get_membind != NULL ? > _numa_get_membind() > >> : NULL; > >> + if(bmp == NULL) return false; > >> + int issingle = 0; > >> + // System can have more than 64 nodes so check in all the > >> + elements > of > >> + // unsigned long array > >> + for (unsigned long i = 0; i < (bmp->size / (8 * sizeof(unsigned > >> long))); i++) { > >> + if (bmp->maskp != NULL && (((bmp->maskp[i]) & > >> + (((bmp->maskp[i])) > - > >> 1)) == 0)) { > >> + issingle++; > >> + } else if (bmp->maskp[i] == 0) { > >> + continue; > >> + } else { > >> + return false; > >> + } > >> + } > >> + if (issingle == 1) > >> + return true; > >> + return false; > >> + } > >> }; > >> > >> #endif // OS_LINUX_VM_OS_LINUX_HPP > >> diff --git a/src/hotspot/share/runtime/os.hpp > >> b/src/hotspot/share/runtime/os.hpp > >> --- a/src/hotspot/share/runtime/os.hpp > >> +++ b/src/hotspot/share/runtime/os.hpp > >> @@ -81,6 +81,10 @@ > >> CriticalPriority = 11 // Critical thread priority > >> }; > >> > >> +extern "C" struct bitmask { > >> + unsigned long size; /* number of bits in the map */ > >> + unsigned long *maskp; > >> +}; > >> // Executable parameter flag for os::commit_memory() and > >> // os::commit_memory_or_exit(). > >> const bool ExecMem = true; > >> > >> > =============================================================== > ============== > >> > >> Thanks, > >> Swati > >> From vladimir.kozlov at oracle.com Thu May 3 22:12:35 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 3 May 2018 15:12:35 -0700 Subject: [11] RFR(L) 8184349: There should be some verification that EnableJVMCI is disabled if a GC not supporting JVMCI is selected Message-ID: <8b33f96f-48c5-6df3-5efe-f77dd19961c5@oracle.com> http://cr.openjdk.java.net/~kvn/8184349/webrev.02/ https://bugs.openjdk.java.net/browse/JDK-8184349 Recent testing problem after Graal dropped CMS (throw exception) made this RFE more urgent. I decided to fix it. The main fix is for JVMCI to check if GC is supported and exit VM with error if not [1]. It is called from Arguments::apply_ergo() after GC is selected in GCConfig::initialize(). Main changes are refactoring. I used this opportunity (inspired by GCConfig) to move compiler related code from arguments.cpp file into compilerDefinitions.* files. And renamed it to compilerConfig.*. Two new CompilerConfig methods check_comp_args_consistency() and CompilerConfig::ergo_initialize() are called from arguments.cpp. The rest are test fixing. Mostly to not run CMS GC with Graal JIT. One test CheckCompileThresholdScaling.java was modified because I skipped scaling compiler threshold in Interpreter mode (CompileThreshold = 0). For tests which use CMS I added @requires !vm.graal.enabled. Unfortunately I did not fix all tests which use CMS. Some tests have several @run commands for each GC. And some tests fork new process to test different GCs. Changes for those tests are more complicated and I filed follow up bug 8202611 [2] I will fix after this. Tested tier1,tier2,tier2-graal -- Thanks, Vladimir [1] http://cr.openjdk.java.net/~kvn/8184349/webrev.02/src/hotspot/share/jvmci/jvmci_globals.cpp.udiff.html [2] https://bugs.openjdk.java.net/browse/JDK-8202611 From poweruserm at live.com.au Thu May 3 22:54:12 2018 From: poweruserm at live.com.au (A Z) Date: Thu, 3 May 2018 22:54:12 +0000 Subject: Inquiry of JEP status. Message-ID: To whom it concerns, I have had the suggestion that I can enquire on this email list as to the status of JEP 306 on this email list. It appears to have been raised to Candidate, while remaining marked unresolved. Is there any consideration for change on this significant area inside Java SE? poweruserm. From igor.ignatyev at oracle.com Fri May 4 04:14:21 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 3 May 2018 21:14:21 -0700 Subject: RFR(L) : 8199382 : [TESTBUG] Open source VM testbase JDI tests Message-ID: <302F3A6E-6485-42E7-BB6C-B21A483B18F2@oracle.com> http://cr.openjdk.java.net/~iignatyev/8199382/webrev.00/index.html > 577169 lines changed: 577169 ins; 0 del; 0 mod; Hi all, could you please review the patch which open sources JDI tests from vm testbase? These tests cover different aspects of JDI implementation. As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. JBS: https://bugs.openjdk.java.net/browse/JDK-8199382 webrev: http://cr.openjdk.java.net/~iignatyev/8199382/webrev.00/index.html testing: vmTestbase_nsk_jdi test group Thanks, -- Igor From david.holmes at oracle.com Fri May 4 06:31:00 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 4 May 2018 16:31:00 +1000 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: <5AEB2C86.1050701@oracle.com> References: <5AE0612F.8060200@oracle.com> <9a5f2a18-f7d4-d1ad-6618-b92bb7d71dd9@samersoff.net> <3B7915CA-50D7-4303-8C7B-C17D5911BC29@oracle.com> <5AEB2C86.1050701@oracle.com> Message-ID: Hi Erik, Wearing my CSR hat at the moment ... On 4/05/2018 1:36 AM, Erik Gahlin wrote: > On 2018-05-02 00:08, Karen Kinnear wrote: >> 2. Code removed two command line flags: EnableTracing and >> UseLockedTracing >> ?with product flags, the normal approach is to deprecate >> ?given they now do nothing, I would strongly recommend that you >> Obsolete before removing >> i.e. accept the flags, but do not do anything for 1 release, to allow >> script migration. Expire the flags >> in the next release. So test that you get a warning if you use them >> for JDK 11. >> >> Note: CSR says EnableTracing flag is removed -> please update CSR for >> both to be made obsolete >> > I will update the CSR and the code, but shouldn't we use Expired, since > the feature is removed. > > (If somebody actually built something on top of -XX:EnableTracing you > want it to fail fast. Starting the JVM even though the feature is > removed is hiding problems, which may make things worse at a later stage.) To me that argues more for the full deprecation process. If someone has built something on top of EnableTracing then we should not be ripping it out from under them. We need to separate the existence of the flag from its affects on mainline hotspot code. To that end the flag can be marked deprecated so that anyone using it is made aware of the intent to remove it - at which point they can make their case for keeping it, or choose to maintain it downstream. At the same time the JFR implementation is free to do nothing in response to the flag being set - if that is a change in JFR's behaviour then that would be a topic for the JFR release note entry. Granted this does seem an unusual circumstance. Cheers, David From tobias.hartmann at oracle.com Fri May 4 07:16:09 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 4 May 2018 09:16:09 +0200 Subject: [11] RFR(S): 8202565: C1 compilation crashes with "assert(is_double_stack() && !is_virtual()) failed: type check" Message-ID: <78095c21-bcb8-7eeb-7bcd-aeb92bfecc61@oracle.com> Hi, please review the following patch: https://bugs.openjdk.java.net/browse/JDK-8202565 http://cr.openjdk.java.net/~thartmann/8202565/webrev.00/ C1 crashes because the LIR contains an instruction moving a T_OBJECT from the stack to a T_LONG double-register ("move [stack:19|L] [rsirsi|J]"). This code is part of a G1 post barrier for an object field store (stack:19 is the base oop). This problem was introduced by the modularization of the C1 GC barriers [1]. ModRefBarrierSetC1::resolve_address() may eagerly resolve the store address into a register, assuming that the post write barrier needs the address in a register anyway. However, in the failing case of a field store, the post barrier does not use precise marking and therefore only uses the base address. The unnecessary leal increases register pressure around the barrier code and causes a spill of the base address which then needs to be (re-)loaded from the stack. The patch restores pre-JDK-8201543 behavior by only eagerly resolving the address into a register if the store is precise. More details (including relevant parts of the LIR) are in the bug comments. Thanks, Tobias [1] https://bugs.openjdk.java.net/browse/JDK-8201543 http://cr.openjdk.java.net/~eosterlund/8201543/webrev.02/ From rwestrel at redhat.com Fri May 4 07:27:37 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Fri, 04 May 2018 09:27:37 +0200 Subject: [11] RFR(S): 8202565: C1 compilation crashes with "assert(is_double_stack() && !is_virtual()) failed: type check" In-Reply-To: <78095c21-bcb8-7eeb-7bcd-aeb92bfecc61@oracle.com> References: <78095c21-bcb8-7eeb-7bcd-aeb92bfecc61@oracle.com> Message-ID: > http://cr.openjdk.java.net/~thartmann/8202565/webrev.00/ That looks good to me. Roland. From tobias.hartmann at oracle.com Fri May 4 07:29:15 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 4 May 2018 09:29:15 +0200 Subject: [11] RFR(S): 8202565: C1 compilation crashes with "assert(is_double_stack() && !is_virtual()) failed: type check" In-Reply-To: References: <78095c21-bcb8-7eeb-7bcd-aeb92bfecc61@oracle.com> Message-ID: Hi Roland, thanks for the review! Best regards, Tobias On 04.05.2018 09:27, Roland Westrelin wrote: > >> http://cr.openjdk.java.net/~thartmann/8202565/webrev.00/ > > That looks good to me. > > Roland. > From erik.osterlund at oracle.com Fri May 4 08:07:04 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 4 May 2018 10:07:04 +0200 Subject: [11] RFR(S): 8202565: C1 compilation crashes with "assert(is_double_stack() && !is_virtual()) failed: type check" In-Reply-To: <78095c21-bcb8-7eeb-7bcd-aeb92bfecc61@oracle.com> References: <78095c21-bcb8-7eeb-7bcd-aeb92bfecc61@oracle.com> Message-ID: <5AEC14A8.1000903@oracle.com> Hi Tobias, Looks good. Thanks, /Erik On 2018-05-04 09:16, Tobias Hartmann wrote: > Hi, > > please review the following patch: > https://bugs.openjdk.java.net/browse/JDK-8202565 > http://cr.openjdk.java.net/~thartmann/8202565/webrev.00/ > > C1 crashes because the LIR contains an instruction moving a T_OBJECT from the stack to a T_LONG > double-register ("move [stack:19|L] [rsirsi|J]"). This code is part of a G1 post barrier for an > object field store (stack:19 is the base oop). > > This problem was introduced by the modularization of the C1 GC barriers [1]. > ModRefBarrierSetC1::resolve_address() may eagerly resolve the store address into a register, > assuming that the post write barrier needs the address in a register anyway. However, in the failing > case of a field store, the post barrier does not use precise marking and therefore only uses the > base address. The unnecessary leal increases register pressure around the barrier code and causes a > spill of the base address which then needs to be (re-)loaded from the stack. > > The patch restores pre-JDK-8201543 behavior by only eagerly resolving the address into a register if > the store is precise. More details (including relevant parts of the LIR) are in the bug comments. > > Thanks, > Tobias > > [1] https://bugs.openjdk.java.net/browse/JDK-8201543 > http://cr.openjdk.java.net/~eosterlund/8201543/webrev.02/ From tobias.hartmann at oracle.com Fri May 4 08:10:32 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 4 May 2018 10:10:32 +0200 Subject: [11] RFR(S): 8202565: C1 compilation crashes with "assert(is_double_stack() && !is_virtual()) failed: type check" In-Reply-To: <5AEC14A8.1000903@oracle.com> References: <78095c21-bcb8-7eeb-7bcd-aeb92bfecc61@oracle.com> <5AEC14A8.1000903@oracle.com> Message-ID: <5a1371d1-c002-657d-6b63-1add6eed598e@oracle.com> Hi Erik, thanks for the review! Best regards, Tobias On 04.05.2018 10:07, Erik ?sterlund wrote: > Hi Tobias, > > Looks good. > > Thanks, > /Erik > > On 2018-05-04 09:16, Tobias Hartmann wrote: >> Hi, >> >> please review the following patch: >> https://bugs.openjdk.java.net/browse/JDK-8202565 >> http://cr.openjdk.java.net/~thartmann/8202565/webrev.00/ >> >> C1 crashes because the LIR contains an instruction moving a T_OBJECT from the stack to a T_LONG >> double-register ("move [stack:19|L] [rsirsi|J]"). This code is part of a G1 post barrier for an >> object field store (stack:19 is the base oop). >> >> This problem was introduced by the modularization of the C1 GC barriers [1]. >> ModRefBarrierSetC1::resolve_address() may eagerly resolve the store address into a register, >> assuming that the post write barrier needs the address in a register anyway. However, in the failing >> case of a field store, the post barrier does not use precise marking and therefore only uses the >> base address. The unnecessary leal increases register pressure around the barrier code and causes a >> spill of the base address which then needs to be (re-)loaded from the stack. >> >> The patch restores pre-JDK-8201543 behavior by only eagerly resolving the address into a register if >> the store is precise. More details (including relevant parts of the LIR) are in the bug comments. >> >> Thanks, >> Tobias >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8201543 >> http://cr.openjdk.java.net/~eosterlund/8201543/webrev.02/ > From sgehwolf at redhat.com Fri May 4 08:23:35 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 04 May 2018 10:23:35 +0200 Subject: [8u] RFR(XS): 8202600: [Zero] Undefined behaviour in src/os_cpu/linux_zero/vm/os_linux_zero.cpp In-Reply-To: <7d2c62d1-c306-6def-9dfb-16a238e18662@redhat.com> References: <1525367703.4079.29.camel@redhat.com> <7d2c62d1-c306-6def-9dfb-16a238e18662@redhat.com> Message-ID: <1525422215.4179.9.camel@redhat.com> On Thu, 2018-05-03 at 22:00 +0200, Aleksey Shipilev wrote: > On 05/03/2018 07:15 PM, Severin Gehwolf wrote: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202600 > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8202600/webrev.jdk8.01/ > > Looks good. Thanks for the review! > Seems like no other arch has this problem? I didn't see any other affected arches in JDK 8. Thanks, Severin From sgehwolf at redhat.com Fri May 4 08:26:29 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 04 May 2018 10:26:29 +0200 Subject: [8u] RFR(XS): 8202600: [Zero] Undefined behaviour in src/os_cpu/linux_zero/vm/os_linux_zero.cpp In-Reply-To: References: <1525367703.4079.29.camel@redhat.com> Message-ID: <1525422389.4179.12.camel@redhat.com> Hi, On Thu, 2018-05-03 at 19:56 +0000, Thomas St?fe wrote: > Hi Severin, > > Looks fine and trivial. Thanks for the review! Could I get another review from an JDK 8 Updates Project Reviewer, please? Thanks, Severin > Best regards, Thomas > > On Thu 3. May 2018 at 19:15, Severin Gehwolf wrote: > > Hi, > > > > Please review this trivial change which fixes a potential crasher bug > > in JDK 8 Zero. JDK 10+ don't have this problem since they got fixed > > with JDK-8143245. I figured a smaller fix like this would be more > > likely to get accepted as a backport. If you'd rather have JDK-8143245 > > backported to JDK 8, that would be fine with me too. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202600 > > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8202600/webrev.jdk8.01/ > > > > Testing: We've been using this patch downstream for months now without > > issues. > > > > Thanks, > > Severin From david.holmes at oracle.com Fri May 4 08:29:42 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 4 May 2018 18:29:42 +1000 Subject: [8u] RFR(XS): 8202600: [Zero] Undefined behaviour in src/os_cpu/linux_zero/vm/os_linux_zero.cpp In-Reply-To: <1525422389.4179.12.camel@redhat.com> References: <1525367703.4079.29.camel@redhat.com> <1525422389.4179.12.camel@redhat.com> Message-ID: <694ab183-1504-cf44-65d1-fe41f1afd038@oracle.com> Reviewed. Cheers, David On 4/05/2018 6:26 PM, Severin Gehwolf wrote: > Hi, > > On Thu, 2018-05-03 at 19:56 +0000, Thomas St?fe wrote: >> Hi Severin, >> >> Looks fine and trivial. > > Thanks for the review! > > Could I get another review from an JDK 8 Updates Project Reviewer, > please? > > Thanks, > Severin > >> Best regards, Thomas >> >> On Thu 3. May 2018 at 19:15, Severin Gehwolf wrote: >>> Hi, >>> >>> Please review this trivial change which fixes a potential crasher bug >>> in JDK 8 Zero. JDK 10+ don't have this problem since they got fixed >>> with JDK-8143245. I figured a smaller fix like this would be more >>> likely to get accepted as a backport. If you'd rather have JDK-8143245 >>> backported to JDK 8, that would be fine with me too. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8202600 >>> webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8202600/webrev.jdk8.01/ >>> >>> Testing: We've been using this patch downstream for months now without >>> issues. >>> >>> Thanks, >>> Severin From karen.kinnear at oracle.com Fri May 4 14:10:10 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Fri, 4 May 2018 10:10:10 -0400 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: <5AEB2C86.1050701@oracle.com> References: <5AE0612F.8060200@oracle.com> <9a5f2a18-f7d4-d1ad-6618-b92bb7d71dd9@samersoff.net> <3B7915CA-50D7-4303-8C7B-C17D5911BC29@oracle.com> <5AEB2C86.1050701@oracle.com> Message-ID: Erik, Thanks for the detailed explanations and responding to all points raised :-) > On May 3, 2018, at 11:36 AM, Erik Gahlin wrote: > > On 2018-05-02 00:08, Karen Kinnear wrote: >> Lots of great work here. Review part 1 below. >> >> I will not be reviewing build files or test files - I believe you have other coverage for those. I did this quickly, >> so if any of my comments don?t make sense after detailed study - just let me know. >> >> I owe you a review of the share/jfr files still. >> >> A couple of issues that I think should be addressed before checking in the code: >> >> 1. New logTag.hpp log tags >> One is called ?setting? -> could you make this less general please? like jfrsetting? >> Or this a subcategory so we already know we are under ?jfr?? >> > I think the idea is that tags should be kept simple, so they can be reused by different subsystems, and then you combine them when you log things, i.e. jfr+setting Thanks for the explanation - I had missed that. > >> Note: I would expect that log tags would be added to the CSR since that is a supported interface >> >> 2. Code removed two command line flags: EnableTracing and UseLockedTracing >> with product flags, the normal approach is to deprecate >> given they now do nothing, I would strongly recommend that you Obsolete before removing >> i.e. accept the flags, but do not do anything for 1 release, to allow script migration. Expire the flags >> in the next release. So test that you get a warning if you use them for JDK 11. >> >> Note: CSR says EnableTracing flag is removed -> please update CSR for both to be made obsolete >> > I will update the CSR and the code, but shouldn't we use Expired, since the feature is removed. > > (If somebody actually built something on top of -XX:EnableTracing you want it to fail fast. Starting the JVM even though the feature is removed is hiding problems, which may make things worse at a later stage.) Let me clarify - the approach we take is to remove the functionality but leave the flags for one release in order to give people time to update their scripts. If you read arguments.cpp it describes the deprecation process. You did a 1-step - complete removal - i.e Expired. What I am asking is that you change this to a 2-step - first go to Obsolete - i.e. leave the flags in so their scripts won?t break, but will print a warning, and do nothing when the flags are set. We have evolved this to be more customer friendly. > >> 3. CSR question - did jcmd change at all? > Not really. We did a small code change that impacts -XX:StartFlightRecording and JFR.start. If you specify -XX:StartFlightRecording=filename=dump.jfr a recording will be dumped on exit. This is the only reasonable behavior. You wouldn't specify a filename unless you wanted to dump it. In previous CSRs we have not specified behavior/heuristics of flags that are unspecified. If a user specifies -XX:StartFlightRecording=filename=dump.jfr,dumponexit=true/false we still honor that. Thank you for the explanation. All set. > >> If it did I would expect that to also be covered in the CSR, >> but I think it did not. >> thanks, Karen ... > > Thanks > Erik > >> >> thanks, >> Karen >> >>> On Apr 28, 2018, at 4:17 AM, Erik Gahlin > wrote: >>> >>> Yes, If we don?t include JFR, we should get the stubs >>> >>> Will fix. >>> >>> Erik >>> >>>> On 28 Apr 2018, at 09:59, Dmitry Samersoff < dms at samersoff.net > wrote: >>>> >>>> Erik, >>>> >>>> globals.cpp:1051 these changes seems to be unnecessary. >>>> >>>> -Dmitry >>>> >>>> On 25.04.2018 14:06, Erik Gahlin wrote: >>>>> Greetings, >>>>> >>>>> Could I have a review of 8199712: Flight Recorder >>>>> >>>>> As mentioned in the preview [1] the tracing backend has been removed. >>>>> Event metadata has been consolidated into a single XML file and event >>>>> classes are now generated by GenerateJfrFiles.java. >>>>> >>>>> Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. >>>>> >>>>> For details about the feature, see the JEP: >>>>> https://bugs.openjdk.java.net/browse/JDK-8193393 >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~egahlin/8199712.0/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8199712 >>>>> >>>>> [1] >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359.html >>>>> >>>>> Thanks >>>>> Erik and Markus >>>> >>>> >>>> -- >>>> Dmitry Samersoff >>>> http://devnull.samersoff.net >>>> * There will come soft rains ... >>>> >>> >> > From karen.kinnear at oracle.com Fri May 4 14:17:07 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Fri, 4 May 2018 10:17:07 -0400 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: <5b81e4aa-d8f1-4da8-8592-598049502c54@default> References: <5AE0612F.8060200@oracle.com> <9a5f2a18-f7d4-d1ad-6618-b92bb7d71dd9@samersoff.net> <3B7915CA-50D7-4303-8C7B-C17D5911BC29@oracle.com> <5AEB2C86.1050701@oracle.com> <5b81e4aa-d8f1-4da8-8592-598049502c54@default> Message-ID: Markus, Thanks for the changes and taking time to explain things. > On May 3, 2018, at 2:33 PM, Markus Gronlund wrote: > > Hi Karen, > > many thanks for taking a look. I have added a few, short and complementary answers inline to Erik's on your first set of questions. > > Again, much appreciate you looking into this. > > Thanks > Markus > > -----Original Message----- > From: Erik Gahlin > Sent: den 3 maj 2018 17:37 > To: Karen Kinnear > > Cc: hotspot-jfr-dev at openjdk.java.net ; hotspot-dev Source Developers > > Subject: Re: RFR(XL): 8199712: Flight Recorder > ... > >> If it did I would expect that to also be covered in the CSR, but I >> think it did not. >> >> A couple of issues that should be addressed before JDK11 ships >> >> 1. FastTime naming - we need to not present an attractive nuisance here. >> Please rename to clarify at least: >> is_platform_trusted_for_fast_time() -> >> ?platform_provides_fast_unordered_time? or something equally clear >> about risk in the name >> - there is a local with a similar need to change name >> FastTime* APIs: e.g. FastTimeElapsedCounterSource -> please include >> Unordered in the name >> For folks new to this issue - the rdtsc (read time stamp counter) >> instruction can return different times across different sockets, >> so time can go backwards, so we do not use this for any vm or java >> times, just for diagnostic timestamps >> and we highly recommend that folks modifying the jvm do not start >> using it accidentally. >> > Markus will comment fast time support. > > [MG] > Answer: > Yes I agree, "is_platform_trusted" is already removed; I will change the name of the time source and add a disclaimer (think you wrote some of the original wording): Thank you very much for the changes :-) > > // Not guaranteed to be synchronized across hardware threads and > // therefore software threads, and can be updated asynchronously > // by software. now() can jump backwards as well as jump forward > // when threads query different cores/sockets. > // Very much not recommended for general use. Caveat emptor. > > class FastUnorderedElapsedCounterSource { > public: > typedef jlong Type; > static uint64_t frequency(); > static Type now(); > static double seconds(Type value); > static uint64_t milliseconds(Type value); > static uint64_t microseconds(Type value); > static uint64_t nanoseconds(Type value); > }; > >> Other comments: >> >> 1. #INCLUDE_JFR >> Did you build with #INCLUDE_JFR not defined? >> Is the theory that we still may want a minimalVM without JFR? >> The model is not clear to me in terms of when you include files or >> generate events e.g. c1_GraphBuilder.cpp, systemDictionary.cpp, >> synchronizer.cpp, ? Could this please be on your future clean-up list? >> p.s. I do like the extraction of the post_xxx_event logic > > [MG] > Answer: > Yes, it is possible to build the VM without JFR. The current patch has this configure capability: > --enable-jfr=[no/yes] > > Although that will change (input from build team), and there will instead be this (updated in next webrev): > --with-jvm-features=-jfr > > Listing ??jfr? (note the minus sign) in here will build a VM with no JFR support. > > I think we have lost track of what the actual requirements are currently for MinimalVM and if this is (still) the main reason we do this. The image size reduction for excluding JFR at compile time is minimal indeed, but since we already had the conditional model from the open/closed split, we brought that over pretty much as-is. I believe we still have reasons to build a MinimalVM. I don?t know the JFR size impact, so it may be moot. Thank you for bringing this over and yes this is something we will want to explore more down the road. > > Please see my earlier mail to Coleen about the headers and include guards, http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-May/032027.htm I saw that and liked the approach. > >> 2. Duplication >> Is there any duplication in the VM_Version and VM_Version_ext handling? >> And are you covering older CPUs we no longer need to cover? Vladimir K >> would know the precise list. >> > > We have added a RFE for this: https://bugs.openjdk.java.net/browse/JDK-8202579 Many thanks. > >> 3. objectCountEventSender.hpp: comment to clean up: >> + // The following two functions have the exact same signature as // >> hotspot/src/share/vm/gc_implementation/shared/objectCountEventSender.h >> pp >> + // >> + // This file will replace the open file if a closed build is performed. >> + // These function signatures can therefore not be changed if the >> + open // signatures aren't changed as well. >> > Will fix. > >> 4. ObjectMonitor::enter >> Why was this event modified differently from others - i.e. it stores >> the stack trace in the _owner field? And is there a reason we flush here? > > [MG] > Answer: > This is a new capability that will allow for better handing inside and around the vicinity of critical sections (such as ObjectMonitor::Enter()): > > This code does not store the stacktrace in the _owner field however, instead: > > The additional functionality will ensure that the thread has enough space (thread local) to accomplish a write of a JavaMonitorEvent and will also capture and save the java stacktrace _before_ entering wait. This is to ensure that we need not do these operations at the point of returning from the wait, which implies we are now the holder of the monitor. This relative simple solution allows us to do any "slower path" work pre-emptively, instead of when the thread is the exclusive owner of some precious resource. Note that the flush is conditional, so it is only done iff the event will not fit. > This solution is generic and might be put to good effect at other sites as well. There is some small follow-up work here though in that we currently don't have good size estimation for events that use strings. Thank you for the explanation. Totally appreciate the goal and the approach. > >> 5. jfrMacros.hpp >> I don't see any users of JFR_FLUSH? >> > Markus will comment 4 and 5. > > [MG] > Answer: > See previous answer, this primitive is part of the extended capability. Might not yet be used at sites. > thanks, Karen >> >>> On Apr 28, 2018, at 4:17 AM, Erik Gahlin >> >> wrote: >>> >>> Yes, If we don?t include JFR, we should get the stubs >>> >>> Will fix. >>> >>> Erik >>> >>>> On 28 Apr 2018, at 09:59, Dmitry Samersoff >>>> >> wrote: >>>> >>>> Erik, >>>> >>>> globals.cpp:1051 these changes seems to be unnecessary. >>>> >>>> -Dmitry >>>> >>>> On 25.04.2018 14:06, Erik Gahlin wrote: >>>>> Greetings, >>>>> >>>>> Could I have a review of 8199712: Flight Recorder >>>>> >>>>> As mentioned in the preview [1] the tracing backend has been removed. >>>>> Event metadata has been consolidated into a single XML file and >>>>> event classes are now generated by GenerateJfrFiles.java. >>>>> >>>>> Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. >>>>> >>>>> For details about the feature, see the JEP: >>>>> https://bugs.openjdk.java.net/browse/JDK-8193393 >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~egahlin/8199712.0/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8199712 >>>>> >>>>> [1] >>>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/03135 >>>>> 9.html >>>>> >>>>> Thanks >>>>> Erik and Markus >>>> >>>> >>>> -- >>>> Dmitry Samersoff >>>> http://devnull.samersoff.net >>>> * There will come soft rains ... From karen.kinnear at oracle.com Fri May 4 14:21:11 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Fri, 4 May 2018 10:21:11 -0400 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: References: <5AE0612F.8060200@oracle.com> <9a5f2a18-f7d4-d1ad-6618-b92bb7d71dd9@samersoff.net> <3B7915CA-50D7-4303-8C7B-C17D5911BC29@oracle.com> <5AEB2C86.1050701@oracle.com> Message-ID: <7D77EEF5-36AC-40F4-8B7A-5E7489C83D53@oracle.com> David, I am arguing for 2-step - going straight to Obsolete - since 1) they have already checked with the community about willingness to move from the existing tracing to free use of JFR. 2) there is not a clean way to have both the old tracing and the new jfr in the same repository So I would recommend Obsolete - you have removed the functionality - but give a grace period to remove the flags, so put the flags back in in obsolete status so they warn but do not prevent starting the application. In my mind the Obsolete stage marks the separation of the existence of the flag from the effects. thanks, Karen > On May 4, 2018, at 2:31 AM, David Holmes wrote: > > Hi Erik, > > Wearing my CSR hat at the moment ... > > On 4/05/2018 1:36 AM, Erik Gahlin wrote: >> On 2018-05-02 00:08, Karen Kinnear wrote: >>> 2. Code removed two command line flags: EnableTracing and UseLockedTracing >>> with product flags, the normal approach is to deprecate >>> given they now do nothing, I would strongly recommend that you Obsolete before removing >>> i.e. accept the flags, but do not do anything for 1 release, to allow script migration. Expire the flags >>> in the next release. So test that you get a warning if you use them for JDK 11. >>> >>> Note: CSR says EnableTracing flag is removed -> please update CSR for both to be made obsolete >>> >> I will update the CSR and the code, but shouldn't we use Expired, since the feature is removed. >> (If somebody actually built something on top of -XX:EnableTracing you want it to fail fast. Starting the JVM even though the feature is removed is hiding problems, which may make things worse at a later stage.) > > To me that argues more for the full deprecation process. If someone has built something on top of EnableTracing then we should not be ripping it out from under them. We need to separate the existence of the flag from its affects on mainline hotspot code. To that end the flag can be marked deprecated so that anyone using it is made aware of the intent to remove it - at which point they can make their case for keeping it, or choose to maintain it downstream. At the same time the JFR implementation is free to do nothing in response to the flag being set - if that is a change in JFR's behaviour then that would be a topic for the JFR release note entry. > > Granted this does seem an unusual circumstance. > > Cheers, > David From bsrbnd at gmail.com Fri May 4 15:31:26 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Fri, 4 May 2018 17:31:26 +0200 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> Message-ID: On 3 May 2018 at 21:43, Kim Barrett wrote: >> On May 3, 2018, at 1:52 PM, B. Blaser wrote: >> >> Hi, >> >> On 2 May 2018 at 22:40, Kim Barrett wrote: >>>> On May 2, 2018, at 5:10 AM, Michal Vala wrote: >>>> >>>> >>>> >>>> On 05/01/2018 07:59 PM, Kim Barrett wrote: >>>>>> On Apr 27, 2018, at 4:26 PM, Michal Vala wrote: >>>>>> Someone to sponsor this please? >>>>> Do you have a sponsor yet? If not, I?ll do it. >>>> >>>> No, I don't. I'd really appreciate if you sponsor it. >>> >>> Will do. I should be able to take care of it in the next day or so. >>> >> >> I've noted some similar issues at several other places which makes the >> JDK build fail on Linux with glibc = 2.26, see comments here: >> https://bugs.openjdk.java.net/browse/JDK-8202353 >> >> Here is a patch for that, does this look reasonable (tier1 seems OK)? > > I think we should move in the direction of eliminating the use of readdir_r entirely, > either under JDK-8202353, or leave that as only the HotSpot part and have another > RFE for the uses in java.base/unix/native. I'll create a new JBS issue (unless you want to) since the fix is mostly located in core-libs and only intended to make the build complete. But I'll still need someone to review and push the fix, would you be interested in doing this? Thanks, Bernard From erik.joelsson at oracle.com Fri May 4 15:33:12 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 4 May 2018 08:33:12 -0700 Subject: RFR(L) : 8199382 : [TESTBUG] Open source VM testbase JDI tests In-Reply-To: <302F3A6E-6485-42E7-BB6C-B21A483B18F2@oracle.com> References: <302F3A6E-6485-42E7-BB6C-B21A483B18F2@oracle.com> Message-ID: Build change looks good. /Erik On 2018-05-03 21:14, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8199382/webrev.00/index.html >> 577169 lines changed: 577169 ins; 0 del; 0 mod; > Hi all, > > could you please review the patch which open sources JDI tests from vm testbase? These tests cover different aspects of JDI implementation. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199382 > webrev: http://cr.openjdk.java.net/~iignatyev/8199382/webrev.00/index.html > testing: vmTestbase_nsk_jdi test group > > Thanks, > -- Igor From Alan.Bateman at oracle.com Fri May 4 15:42:59 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 4 May 2018 16:42:59 +0100 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> Message-ID: <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> On 04/05/2018 16:31, B. Blaser wrote: > : > > I'll create a new JBS issue (unless you want to) since the fix is > mostly located in core-libs and only intended to make the build > complete. > But I'll still need someone to review and push the fix, would you be > interested in doing this? > I think someone needs to explore the behavior on other platforms too as the #ifndef __linux__ patches are ugly. Is there discussion on the original thread about this? -Alan From kim.barrett at oracle.com Fri May 4 16:23:48 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 4 May 2018 12:23:48 -0400 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> Message-ID: <815413A7-5809-4820-A939-F9EC4D8AD61C@oracle.com> > On May 4, 2018, at 11:42 AM, Alan Bateman wrote: > > On 04/05/2018 16:31, B. Blaser wrote: >> : >> >> I'll create a new JBS issue (unless you want to) since the fix is >> mostly located in core-libs and only intended to make the build >> complete. >> But I'll still need someone to review and push the fix, would you be >> interested in doing this? >> > I think someone needs to explore the behavior on other platforms too as the #ifndef __linux__ patches are ugly. Is there discussion on the original thread about this? > > -Alan Yes, see https://bugs.openjdk.java.net/browse/JDK-8202353 From vladimir.kozlov at oracle.com Fri May 4 17:06:07 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 4 May 2018 10:06:07 -0700 Subject: RFR(L) : 8199382 : [TESTBUG] Open source VM testbase JDI tests In-Reply-To: <302F3A6E-6485-42E7-BB6C-B21A483B18F2@oracle.com> References: <302F3A6E-6485-42E7-BB6C-B21A483B18F2@oracle.com> Message-ID: <6afa32f6-8dcf-be66-a499-20882e059769@oracle.com> Other then my usual concern about listing all tests in TEST.groups changes looks good. Thanks, Vladimir On 5/3/18 9:14 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8199382/webrev.00/index.html >> 577169 lines changed: 577169 ins; 0 del; 0 mod; > > Hi all, > > could you please review the patch which open sources JDI tests from vm testbase? These tests cover different aspects of JDI implementation. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199382 > webrev: http://cr.openjdk.java.net/~iignatyev/8199382/webrev.00/index.html > testing: vmTestbase_nsk_jdi test group > > Thanks, > -- Igor > From boris.ulasevich at bell-sw.com Fri May 4 18:13:51 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Fri, 4 May 2018 21:13:51 +0300 Subject: RFR 8202379: ARM32 is broken after JDK-8201543 (Modularize C1 GC barriers) In-Reply-To: <3f7d6a68-c079-aa3d-3938-bf18cce76014@redhat.com> References: <2f5d4f30-dfb5-4959-c032-e27d747ee07e@redhat.com> <1914516d-ec4b-1c45-194a-3107371fb1f9@redhat.com> <4b31ffb0-b5ee-4820-23f8-6826a68240b2@redhat.com> <478D9BFD-2F34-46EF-980D-1EC281F91A0E@oracle.com> <3f7d6a68-c079-aa3d-3938-bf18cce76014@redhat.com> Message-ID: <09c11874-483c-1d42-23e4-ed2d374d4b15@bell-sw.com> Hi, I believe the fix is not complete. I see SIGILL on ARM32 RPi (cross ARM32 build on Raspberry Pi) with simple command line: ./jdk-b49938/bin/java -Xcomp -XX:+TieredCompilation -version I reproduced the issue starting from r49906 (8201543: Modularize C1 GC barriers) with r49938 fix (8202379: ARM32 is broken after JDK-8201543 (Modularize C1 GC barriers)). # hg update -r49906 # hg diff -r49937:49938 > build_fix # patch -p1 < build_fix # make CONF=linux-arm-normal-server-release I did not dig deeper yet. regards, Boris On 01.05.2018 19:48, Aleksey Shipilev wrote: > Thanks Erik! > > -Aleksey > > P.S. I think you also wanted to review 8201786. > > On 05/01/2018 06:45 PM, Erik Osterlund wrote: >> Hi Aleksey, >> >> Looks good. >> >> Thanks, >> /Erik >> >>> On 1 May 2018, at 18:39, Aleksey Shipilev wrote: >>> >>>> On 04/30/2018 11:09 AM, Andrew Haley wrote: >>>>> On 04/30/2018 10:05 AM, Aleksey Shipilev wrote: >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8202379 >>>>> >>>>> Fix: >>>>> http://cr.openjdk.java.net/~shade/8202379/webrev.01/ >>>>> >>>>> Testing: arm32 build, Raspi 3 runs (some selected jcstress tests) >>>> >>>> Looks OK, thanks. >>> >>> Thanks for review! Anyone else? >>> >>> -Aleksey >>> >>> >>> > > From rkennke at redhat.com Fri May 4 21:29:26 2018 From: rkennke at redhat.com (Roman Kennke) Date: Fri, 4 May 2018 23:29:26 +0200 Subject: RFR: JDK-8202676: AArch64: Missing enter/leave around barrier leads to infinite loop Message-ID: <5c42f2b7-ea18-2d7c-2884-91a4efe40590@redhat.com> In aarch64's TemplateInterpreterGenerator::generate_Reference_get_entry(void), there used to be enter()/leave() calls around the g1 pre-barrier. This is necessary in case the barrier calls into the runtime, to setup/remove stack frames for the call. With the interpreter BarrierSetAssembler work, this seems to have been dropped. It does lead to stack corruption, sometimes endless loops, etc. This patch re-instates the enter() and leave() calls around the barrier where they used to be. http://cr.openjdk.java.net/~rkennke/JDK-8202676/webrev.00/ Can I please get a review? Thanks, Roman From david.holmes at oracle.com Fri May 4 23:00:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Sat, 5 May 2018 09:00:05 +1000 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: <7D77EEF5-36AC-40F4-8B7A-5E7489C83D53@oracle.com> References: <5AE0612F.8060200@oracle.com> <9a5f2a18-f7d4-d1ad-6618-b92bb7d71dd9@samersoff.net> <3B7915CA-50D7-4303-8C7B-C17D5911BC29@oracle.com> <5AEB2C86.1050701@oracle.com> <7D77EEF5-36AC-40F4-8B7A-5E7489C83D53@oracle.com> Message-ID: <157b03b9-91a7-64aa-d3df-8d0cf7640e99@oracle.com> Hi Karen, On 5/05/2018 12:21 AM, Karen Kinnear wrote: > David, > > I am arguing for 2-step - going straight to Obsolete - since > 1) they have already checked with the community about willingness to move from the existing tracing to free use of JFR. > 2) there is not a clean way to have both the old tracing and the new jfr in the same repository So basically we're ripping out the only code that uses this flag, and in the process will completely break any third-party tracing frameworks that may in theory have built on this. So those frameworks will no longer even build. > So I would recommend Obsolete - you have removed the functionality - but give a grace period to remove the flags, > so put the flags back in in obsolete status so they warn but do not prevent starting the application. Ok this is a concession to the potential users of said hypothetical tracing systems. That's fine. Thanks, David > In my mind the Obsolete stage marks the separation of the existence of the flag from the effects. > > thanks, > Karen > >> On May 4, 2018, at 2:31 AM, David Holmes wrote: >> >> Hi Erik, >> >> Wearing my CSR hat at the moment ... >> >> On 4/05/2018 1:36 AM, Erik Gahlin wrote: >>> On 2018-05-02 00:08, Karen Kinnear wrote: >>>> 2. Code removed two command line flags: EnableTracing and UseLockedTracing >>>> with product flags, the normal approach is to deprecate >>>> given they now do nothing, I would strongly recommend that you Obsolete before removing >>>> i.e. accept the flags, but do not do anything for 1 release, to allow script migration. Expire the flags >>>> in the next release. So test that you get a warning if you use them for JDK 11. >>>> >>>> Note: CSR says EnableTracing flag is removed -> please update CSR for both to be made obsolete >>>> >>> I will update the CSR and the code, but shouldn't we use Expired, since the feature is removed. >>> (If somebody actually built something on top of -XX:EnableTracing you want it to fail fast. Starting the JVM even though the feature is removed is hiding problems, which may make things worse at a later stage.) >> >> To me that argues more for the full deprecation process. If someone has built something on top of EnableTracing then we should not be ripping it out from under them. We need to separate the existence of the flag from its affects on mainline hotspot code. To that end the flag can be marked deprecated so that anyone using it is made aware of the intent to remove it - at which point they can make their case for keeping it, or choose to maintain it downstream. At the same time the JFR implementation is free to do nothing in response to the flag being set - if that is a change in JFR's behaviour then that would be a topic for the JFR release note entry. >> >> Granted this does seem an unusual circumstance. >> >> Cheers, >> David > From poweruserm at live.com.au Sat May 5 01:53:36 2018 From: poweruserm at live.com.au (A Z) Date: Sat, 5 May 2018 01:53:36 +0000 Subject: JEP 306 Message-ID: Is it possible for someone to reply explainting what the Candidate status, or really the thinking and present state, of JEP 306 is? I have been recommended to this list for the sake of this question. ? http://openjdk.java.net/jeps/306 https://bugs.openjdk.java.net/browse/JDK-8175916 From aph at redhat.com Sat May 5 08:10:49 2018 From: aph at redhat.com (Andrew Haley) Date: Sat, 5 May 2018 09:10:49 +0100 Subject: RFR: JDK-8202676: AArch64: Missing enter/leave around barrier leads to infinite loop In-Reply-To: <5c42f2b7-ea18-2d7c-2884-91a4efe40590@redhat.com> References: <5c42f2b7-ea18-2d7c-2884-91a4efe40590@redhat.com> Message-ID: <035652c5-8389-29d3-9fe9-750ed8ebc398@redhat.com> On 04/05/18 22:29, Roman Kennke wrote: > In aarch64's > TemplateInterpreterGenerator::generate_Reference_get_entry(void), there > used to be enter()/leave() calls around the g1 pre-barrier. This is > necessary in case the barrier calls into the runtime, to setup/remove > stack frames for the call. With the interpreter BarrierSetAssembler > work, this seems to have been dropped. It does lead to stack corruption, > sometimes endless loops, etc. > > This patch re-instates the enter() and leave() calls around the barrier > where they used to be. > > http://cr.openjdk.java.net/~rkennke/JDK-8202676/webrev.00/ > > Can I please get a review? This is the second time in the last year or so that enter/leave pair has been deleted by GC engineers! Please stop doing this! :-) Anyway, I think it now makes more sense for the enter/leave pair to be in G1BarrierSetAssembler::g1_write_barrier_pre before the push(saved, sp). It makes more logical sense there. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From rkennke at redhat.com Sat May 5 09:51:47 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sat, 5 May 2018 11:51:47 +0200 Subject: RFR: JDK-8202676: AArch64: Missing enter/leave around barrier leads to infinite loop In-Reply-To: <035652c5-8389-29d3-9fe9-750ed8ebc398@redhat.com> References: <5c42f2b7-ea18-2d7c-2884-91a4efe40590@redhat.com> <035652c5-8389-29d3-9fe9-750ed8ebc398@redhat.com> Message-ID: <83bbb39f-3589-9bb8-283c-1fddf2bd2d40@redhat.com> Am 05.05.2018 um 10:10 schrieb Andrew Haley: > On 04/05/18 22:29, Roman Kennke wrote: >> In aarch64's >> TemplateInterpreterGenerator::generate_Reference_get_entry(void), there >> used to be enter()/leave() calls around the g1 pre-barrier. This is >> necessary in case the barrier calls into the runtime, to setup/remove >> stack frames for the call. With the interpreter BarrierSetAssembler >> work, this seems to have been dropped. It does lead to stack corruption, >> sometimes endless loops, etc. >> >> This patch re-instates the enter() and leave() calls around the barrier >> where they used to be. >> >> http://cr.openjdk.java.net/~rkennke/JDK-8202676/webrev.00/ >> >> Can I please get a review? > > This is the second time in the last year or so that enter/leave pair has > been deleted by GC engineers!? Please stop doing this!? :-) > > Anyway, I think it now makes more sense for the enter/leave pair to be > in G1BarrierSetAssembler::g1_write_barrier_pre before the > push(saved, sp).? It makes more logical sense there. > Right. http://cr.openjdk.java.net/~rkennke/JDK-8202676/webrev.01/ Does it hurt if it's ever called from a place where enter() / leave() is not strictly required? I.e. if we already have an interpreter frame? Good to push? Roman From shade at redhat.com Sat May 5 10:52:37 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Sat, 5 May 2018 12:52:37 +0200 Subject: RFR (S) 8202684: Minimal VM build is broken after JDK-8199067, JDK-8202638 Message-ID: <670eac07-2d76-8bf4-cd31-0b48302d0fd3@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8202684 Fix: http://cr.openjdk.java.net/~shade/8202684/webrev.01/ Does the same thing as we do for other tests. Plus introduces NMT-specific inline macros. Testing: x86_64 server build, x86 minimal build, x86 server build Thanks, -Aleksey From bsrbnd at gmail.com Sat May 5 12:03:42 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sat, 5 May 2018 14:03:42 +0200 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> Message-ID: On 4 May 2018 at 17:42, Alan Bateman wrote: > On 04/05/2018 16:31, B. Blaser wrote: >> >> : >> >> I'll create a new JBS issue (unless you want to) since the fix is >> mostly located in core-libs and only intended to make the build >> complete. >> But I'll still need someone to review and push the fix, would you be >> interested in doing this? >> > I think someone needs to explore the behavior on other platforms too as the > #ifndef __linux__ patches are ugly. Yes sure but I cannot test it elsewhere myself... and we can easily remove them once POSIX officially deprecates 'readdir_r' or if someone validates the fix on other systems. > Is there discussion on the original > thread about this? As Kim said, JDK-8202353 summarize the situation. > -Alan I've just made a small improvement (below) using 'errno' to preserve the exact behavior when necessary, what do yo think? Bernard diff -r 1871c5d07caf src/java.base/unix/native/libjava/TimeZone_md.c --- a/src/java.base/unix/native/libjava/TimeZone_md.c Fri Apr 27 15:55:29 2018 -0700 +++ b/src/java.base/unix/native/libjava/TimeZone_md.c Sat May 05 12:54:56 2018 +0200 @@ -122,7 +122,9 @@ DIR *dirp = NULL; struct stat statbuf; struct dirent64 *dp = NULL; +#ifndef __linux__ struct dirent64 *entry = NULL; +#endif char *pathname = NULL; int fd = -1; char *dbuf = NULL; @@ -140,7 +142,7 @@ if (name_max < 1024) { name_max = 1024; } - +#ifndef __linux__ entry = (struct dirent64 *)malloc(offsetof(struct dirent64, d_name) + name_max + 1); if (entry == NULL) { (void) closedir(dirp); @@ -148,6 +150,9 @@ } while (readdir64_r(dirp, entry, &dp) == 0 && dp != NULL) { +#else + while ((dp = readdir64(dirp)) != NULL) { +#endif /* * Skip '.' and '..' (and possibly other .* files) */ @@ -213,10 +218,11 @@ free((void *) pathname); pathname = NULL; } - +#ifndef __linux__ if (entry != NULL) { free((void *) entry); } +#endif if (dirp != NULL) { (void) closedir(dirp); } diff -r 1871c5d07caf src/java.base/unix/native/libjava/UnixFileSystem_md.c --- a/src/java.base/unix/native/libjava/UnixFileSystem_md.c Fri Apr 27 15:55:29 2018 -0700 +++ b/src/java.base/unix/native/libjava/UnixFileSystem_md.c Sat May 05 12:54:56 2018 +0200 @@ -312,7 +312,9 @@ { DIR *dir = NULL; struct dirent64 *ptr; +#ifndef __linux__ struct dirent64 *result; +#endif int len, maxlen; jobjectArray rv, old; jclass str_class; @@ -324,14 +326,14 @@ dir = opendir(path); } END_PLATFORM_STRING(env, path); if (dir == NULL) return NULL; - +#ifndef __linux__ ptr = malloc(sizeof(struct dirent64) + (PATH_MAX + 1)); if (ptr == NULL) { JNU_ThrowOutOfMemoryError(env, "heap allocation failed"); closedir(dir); return NULL; } - +#endif /* Allocate an initial String array */ len = 0; maxlen = 16; @@ -339,7 +341,11 @@ if (rv == NULL) goto error; /* Scan the directory */ +#ifndef __linux__ while ((readdir64_r(dir, ptr, &result) == 0) && (result != NULL)) { +#else + while ((ptr = readdir64(dir)) != NULL) { +#endif jstring name; if (!strcmp(ptr->d_name, ".") || !strcmp(ptr->d_name, "..")) continue; @@ -360,7 +366,9 @@ (*env)->DeleteLocalRef(env, name); } closedir(dir); +#ifndef __linux__ free(ptr); +#endif /* Copy the final results into an appropriately-sized array */ old = rv; @@ -375,7 +383,9 @@ error: closedir(dir); +#ifndef __linux__ free(ptr); +#endif return NULL; } diff -r 1871c5d07caf src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c --- a/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Fri Apr 27 15:55:29 2018 -0700 +++ b/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Sat May 05 12:54:56 2018 +0200 @@ -726,12 +726,19 @@ char name_extra[PATH_MAX + 1 - sizeof result->d_name]; } entry; struct dirent64* ptr = &entry.buf; +#ifndef __linux__ int res; +#endif DIR* dirp = jlong_to_ptr(value); /* EINTR not listed as a possible error */ /* TDB: reentrant version probably not required here */ +#ifndef __linux__ res = readdir64_r(dirp, ptr, &result); +#else + errno = 0; + ptr = result = readdir64(dirp); +#endif #ifdef _AIX /* On AIX, readdir_r() returns EBADF (i.e. '9') and sets 'result' to NULL for the */ @@ -741,8 +748,13 @@ } #endif +#ifndef __linux__ if (res != 0) { throwUnixException(env, res); +#else + if (errno != 0) { + throwUnixException(env, errno); +#endif return NULL; } else { if (result == NULL) { diff -r 1871c5d07caf src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c --- a/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c Fri Apr 27 15:55:29 2018 -0700 +++ b/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c Sat May 05 12:54:56 2018 +0200 @@ -75,10 +75,10 @@ #endif /* _ALLBSD_SOURCE */ static struct dirent* read_dir(DIR* dirp, struct dirent* entry) { -#ifdef __solaris__ +#if defined(__solaris__) || defined(__linux__) struct dirent* dbuf = readdir(dirp); return dbuf; -#else /* __linux__ || _ALLBSD_SOURCE */ +#else /* _ALLBSD_SOURCE */ struct dirent* p; if (readdir_r(dirp, entry, &p) == 0) { return p; From erik.osterlund at oracle.com Sat May 5 12:12:53 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Sat, 5 May 2018 14:12:53 +0200 Subject: RFR: JDK-8202676: AArch64: Missing enter/leave around barrier leads to infinite loop In-Reply-To: <5c42f2b7-ea18-2d7c-2884-91a4efe40590@redhat.com> References: <5c42f2b7-ea18-2d7c-2884-91a4efe40590@redhat.com> Message-ID: Hi Roman, Looks good. Thanks, /Erik > On 4 May 2018, at 23:29, Roman Kennke wrote: > > In aarch64's > TemplateInterpreterGenerator::generate_Reference_get_entry(void), there > used to be enter()/leave() calls around the g1 pre-barrier. This is > necessary in case the barrier calls into the runtime, to setup/remove > stack frames for the call. With the interpreter BarrierSetAssembler > work, this seems to have been dropped. It does lead to stack corruption, > sometimes endless loops, etc. > > This patch re-instates the enter() and leave() calls around the barrier > where they used to be. > > http://cr.openjdk.java.net/~rkennke/JDK-8202676/webrev.00/ > > Can I please get a review? > > Thanks, Roman > From erik.osterlund at oracle.com Sat May 5 12:16:00 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Sat, 5 May 2018 14:16:00 +0200 Subject: RFR (S) 8202684: Minimal VM build is broken after JDK-8199067, JDK-8202638 In-Reply-To: <670eac07-2d76-8bf4-cd31-0b48302d0fd3@redhat.com> References: <670eac07-2d76-8bf4-cd31-0b48302d0fd3@redhat.com> Message-ID: <92F3490C-A95F-41FC-8EF1-2318A6D6865D@oracle.com> Hi Aleksey, When there are whole cpp files I want to conditionally compile only if a feature is enabled, I prefer adding rules in the make files to not compile those files rather than wrapping the whole file in ifdefs. Is that doable for the test file? Thanks, /Erik > On 5 May 2018, at 12:52, Aleksey Shipilev wrote: > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8202684 > > Fix: > http://cr.openjdk.java.net/~shade/8202684/webrev.01/ > > Does the same thing as we do for other tests. Plus introduces NMT-specific inline macros. > > Testing: x86_64 server build, x86 minimal build, x86 server build > > Thanks, > -Aleksey > From shade at redhat.com Sat May 5 12:29:24 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Sat, 5 May 2018 14:29:24 +0200 Subject: RFR (S) 8202684: Minimal VM build is broken after JDK-8199067, JDK-8202638 In-Reply-To: <92F3490C-A95F-41FC-8EF1-2318A6D6865D@oracle.com> References: <670eac07-2d76-8bf4-cd31-0b48302d0fd3@redhat.com> <92F3490C-A95F-41FC-8EF1-2318A6D6865D@oracle.com> Message-ID: <45400c73-0e97-5dc8-a1e8-22ba19c64f8a@redhat.com> On 05/05/2018 02:16 PM, Erik Osterlund wrote: > When there are whole cpp files I want to conditionally compile only if a feature is enabled, I > prefer adding rules in the make files to not compile those files rather than wrapping the whole > file in ifdefs. Is that doable for the test file? Maybe! Debatable :) I did what's already done for similar test, test_virtualMemoryTracker.cpp [1], and that seemed to be the best option for consistency reasons. -Aleksey [1] http://hg.openjdk.java.net/jdk/jdk/file/88cc95780b6e/test/hotspot/gtest/runtime/test_virtualMemoryTracker.cpp#l29 From erik.osterlund at oracle.com Sat May 5 12:38:29 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Sat, 5 May 2018 14:38:29 +0200 Subject: RFR (S) 8202684: Minimal VM build is broken after JDK-8199067, JDK-8202638 In-Reply-To: <45400c73-0e97-5dc8-a1e8-22ba19c64f8a@redhat.com> References: <670eac07-2d76-8bf4-cd31-0b48302d0fd3@redhat.com> <92F3490C-A95F-41FC-8EF1-2318A6D6865D@oracle.com> <45400c73-0e97-5dc8-a1e8-22ba19c64f8a@redhat.com> Message-ID: <261D12BD-6AAC-401D-881A-726B3710A625@oracle.com> The fact that there are more suggests to me there is bigger gain in having a directory that is excluded. But I?m okay with leaving that for another day. Looks good. Thanks, /Erik > On 5 May 2018, at 14:29, Aleksey Shipilev wrote: > >> On 05/05/2018 02:16 PM, Erik Osterlund wrote: >> When there are whole cpp files I want to conditionally compile only if a feature is enabled, I >> prefer adding rules in the make files to not compile those files rather than wrapping the whole >> file in ifdefs. Is that doable for the test file? > > Maybe! Debatable :) I did what's already done for similar test, test_virtualMemoryTracker.cpp [1], > and that seemed to be the best option for consistency reasons. > > -Aleksey > > [1] > http://hg.openjdk.java.net/jdk/jdk/file/88cc95780b6e/test/hotspot/gtest/runtime/test_virtualMemoryTracker.cpp#l29 > From thomas.stuefe at gmail.com Sat May 5 14:19:56 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 5 May 2018 16:19:56 +0200 Subject: RFR (S) 8202684: Minimal VM build is broken after JDK-8199067, JDK-8202638 In-Reply-To: <670eac07-2d76-8bf4-cd31-0b48302d0fd3@redhat.com> References: <670eac07-2d76-8bf4-cd31-0b48302d0fd3@redhat.com> Message-ID: Hi Alexey, the fix is okay for now, I think. Save that the scale sub option for the VM.metaspace jcmd will not work for the minimal build. I think I will rewrite this to not use NMT functionality. So, minimal build is something to test too? I already test-build nonpch, zero and 32bit before pushing. I really think if these are important enough to cause immediate follow up fixes, they all should be included into the tier1 tests of oracle. I know, I am barking up the wrong tree here :) ..Thomas On Sat, May 5, 2018 at 12:52 PM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8202684 > > Fix: > http://cr.openjdk.java.net/~shade/8202684/webrev.01/ > > Does the same thing as we do for other tests. Plus introduces NMT-specific inline macros. > > Testing: x86_64 server build, x86 minimal build, x86 server build > > Thanks, > -Aleksey > From aph at redhat.com Sat May 5 14:54:30 2018 From: aph at redhat.com (Andrew Haley) Date: Sat, 5 May 2018 15:54:30 +0100 Subject: RFR: JDK-8202676: AArch64: Missing enter/leave around barrier leads to infinite loop In-Reply-To: <83bbb39f-3589-9bb8-283c-1fddf2bd2d40@redhat.com> References: <5c42f2b7-ea18-2d7c-2884-91a4efe40590@redhat.com> <035652c5-8389-29d3-9fe9-750ed8ebc398@redhat.com> <83bbb39f-3589-9bb8-283c-1fddf2bd2d40@redhat.com> Message-ID: <60e74ae8-fd54-4b6d-3034-4180f1d38d57@redhat.com> On 05/05/18 10:51, Roman Kennke wrote: > http://cr.openjdk.java.net/~rkennke/JDK-8202676/webrev.01/ > > Does it hurt if it's ever called from a place where enter() / leave() is > not strictly required? I.e. if we already have an interpreter frame? It's fine. > Good to push? On more thing: please add a comment at the start of TemplateInterpreterGenerator::generate_Reference_get_entry and BarrierSetAssembler::load_at and G1BarrierSetAssembler::g1_write_barrier_pre like this: // LR is live. It must be saved around calls. Feel free to add as many capitals as you like. This comment might help avoid it breaking again. You never know, it might work. You don't need another approval for that. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From kim.barrett at oracle.com Sat May 5 16:42:38 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 5 May 2018 12:42:38 -0400 Subject: RFR: 8202672: Build failed in metaspace.cpp with VS2017 Message-ID: <45D1CA98-136D-4726-97F2-C7F5D2853733@oracle.com> Note: Change already reviewed by erikj and dcubed. Thanks. JDK-8201572 broke build with VS2017. This change adds the missing whitespace between a string constant and an identifier so it doesn't look like an unknown user-defined string literal. diff -r e2dc18484400 -r 9e82ca74f086 src/hotspot/share/memory/metaspace.cpp --- a/src/hotspot/share/memory/metaspace.cpp Sat May 05 09:34:01 2018 -0700 +++ b/src/hotspot/share/memory/metaspace.cpp Sat May 05 12:38:15 2018 -0400 @@ -4131,7 +4131,7 @@ cl._stats_total.class_sm_stats().free_blocks_cap_words(); out->print("Deallocated from chunks in use: "); print_scaled_words_and_percentage(out, free_blocks_cap_words, committed_words, scale, 6); - out->print(" ("UINTX_FORMAT " blocks)", free_blocks_num); + out->print(" (" UINTX_FORMAT " blocks)", free_blocks_num); out->cr(); // Print total waste. From thomas.stuefe at gmail.com Sat May 5 16:50:29 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 05 May 2018 16:50:29 +0000 Subject: RFR: 8202672: Build failed in metaspace.cpp with VS2017 In-Reply-To: <45D1CA98-136D-4726-97F2-C7F5D2853733@oracle.com> References: <45D1CA98-136D-4726-97F2-C7F5D2853733@oracle.com> Message-ID: Hi Kim, Thanks for fixing! Best Regards, Thomas On Sat 5. May 2018 at 18:43, Kim Barrett wrote: > Note: Change already reviewed by erikj and dcubed. Thanks. > > JDK-8201572 broke build with VS2017. This change adds the missing > whitespace between a string constant and an identifier so it doesn't > look like an unknown user-defined string literal. > > diff -r e2dc18484400 -r 9e82ca74f086 src/hotspot/share/memory/metaspace.cpp > --- a/src/hotspot/share/memory/metaspace.cpp Sat May 05 09:34:01 2018 > -0700 > +++ b/src/hotspot/share/memory/metaspace.cpp Sat May 05 12:38:15 2018 > -0400 > @@ -4131,7 +4131,7 @@ > cl._stats_total.class_sm_stats().free_blocks_cap_words(); > out->print("Deallocated from chunks in use: "); > print_scaled_words_and_percentage(out, free_blocks_cap_words, > committed_words, scale, 6); > - out->print(" ("UINTX_FORMAT " blocks)", free_blocks_num); > + out->print(" (" UINTX_FORMAT " blocks)", free_blocks_num); > out->cr(); > > // Print total waste. > > From shade at redhat.com Sat May 5 17:02:59 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Sat, 5 May 2018 19:02:59 +0200 Subject: RFR (S) 8202684: Minimal VM build is broken after JDK-8199067, JDK-8202638 In-Reply-To: References: <670eac07-2d76-8bf4-cd31-0b48302d0fd3@redhat.com> Message-ID: <062c91c7-62f1-9eea-d82f-2130ce8f27a0@redhat.com> On 05/05/2018 04:19 PM, Thomas St?fe wrote: > the fix is okay for now, I think. Save that the scale sub option for > the VM.metaspace jcmd will not work for the minimal build. I think I > will rewrite this to not use NMT functionality. Thanks for review! Pushed. > So, minimal build is something to test too? I already test-build > nonpch, zero and 32bit before pushing. I really think if these are > important enough to cause immediate follow up fixes, they all should > be included into the tier1 tests of oracle. It would be nice to have at least build tests for most platforms. I think minimal build falls into "it would be nice to have it non-broken" category. My personal build server builds many configurations in the matter of course. Build fixes for these configs are not high priority, as we can always fix up still after the commit. My own interest in minimal was to check that jdk/jdk after GC interfaces reshuffling, even in these exotic configurations. Thanks, -Aleksey From kim.barrett at oracle.com Sat May 5 20:26:36 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 5 May 2018 16:26:36 -0400 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> Message-ID: <6AE7B98E-FF1B-43E0-8108-7128707BD5F5@oracle.com> > On May 5, 2018, at 8:03 AM, B. Blaser wrote: > > On 4 May 2018 at 17:42, Alan Bateman wrote: >> On 04/05/2018 16:31, B. Blaser wrote: >>> >>> : >>> >>> I'll create a new JBS issue (unless you want to) since the fix is >>> mostly located in core-libs and only intended to make the build >>> complete. >>> But I'll still need someone to review and push the fix, would you be >>> interested in doing this? >>> >> I think someone needs to explore the behavior on other platforms too as the >> #ifndef __linux__ patches are ugly. > > Yes sure but I cannot test it elsewhere myself... and we can easily > remove them once POSIX officially deprecates 'readdir_r' or if someone > validates the fix on other systems. I'm not sure anyone has the ability to test on all supported. That doesn't (and really can't) prevent making changes across platforms to platform-specific code. It just means one needs to get help with testing. Make the change across platforms, test on those platforms one has access to, and ask here for help testing on others. Waiting for a POSIX change is even more like watching paint dry than waiting for Java releases used to be. From rkennke at redhat.com Sat May 5 22:53:21 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sun, 6 May 2018 00:53:21 +0200 Subject: RFR: JDK-8202676: AArch64: Missing enter/leave around barrier leads to infinite loop In-Reply-To: <60e74ae8-fd54-4b6d-3034-4180f1d38d57@redhat.com> References: <5c42f2b7-ea18-2d7c-2884-91a4efe40590@redhat.com> <035652c5-8389-29d3-9fe9-750ed8ebc398@redhat.com> <83bbb39f-3589-9bb8-283c-1fddf2bd2d40@redhat.com> <60e74ae8-fd54-4b6d-3034-4180f1d38d57@redhat.com> Message-ID: <190b2abc-2751-2a58-7779-532e1ff1c37a@redhat.com> Am 05.05.2018 um 16:54 schrieb Andrew Haley: > On 05/05/18 10:51, Roman Kennke wrote: >> http://cr.openjdk.java.net/~rkennke/JDK-8202676/webrev.01/ >> >> Does it hurt if it's ever called from a place where enter() / leave() is >> not strictly required? I.e. if we already have an interpreter frame? > > It's fine. > >> Good to push? > > On more thing: please add a comment at the start of > TemplateInterpreterGenerator::generate_Reference_get_entry and > BarrierSetAssembler::load_at and > G1BarrierSetAssembler::g1_write_barrier_pre like this: > > // LR is live.? It must be saved around calls. > > Feel free to add as many capitals as you like. > > This comment might help avoid it breaking again.? You never know, it > might work. > > You don't need another approval for that. > Thanks for the review! Pushed as: http://hg.openjdk.java.net/jdk/jdk/rev/7238cb613dc5 Cheers, Roman From poweruserm at live.com.au Sun May 6 02:25:59 2018 From: poweruserm at live.com.au (A Z) Date: Sun, 6 May 2018 02:25:59 +0000 Subject: Question about Runtime relevant Java JEP. Message-ID: Is it possible for someone to reply explainting what the Candidate status, or really the thinking and present state, of JEP 306 is? I have been recommended to this list for the sake of this question. ? http://openjdk.java.net/jeps/306 https://bugs.openjdk.java.net/browse/JDK-8175916 From david.holmes at oracle.com Sun May 6 07:18:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Sun, 6 May 2018 17:18:30 +1000 Subject: RFR (S) 8202684: Minimal VM build is broken after JDK-8199067, JDK-8202638 In-Reply-To: <670eac07-2d76-8bf4-cd31-0b48302d0fd3@redhat.com> References: <670eac07-2d76-8bf4-cd31-0b48302d0fd3@redhat.com> Message-ID: <847ccb5b-b3a6-3860-bbcb-cdfffb2f32b8@oracle.com> Hi Aleksey, On 5/05/2018 8:52 PM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8202684 > > Fix: > http://cr.openjdk.java.net/~shade/8202684/webrev.01/ While this will fix your issue I can't help but think that the dependency on NMT should be far more explicit and potentially handled in a different place. It is very unclear to me why share/memory/metaspace/metaspaceDCmd.cpp is intimately tied to NMT, or what it means to use it without NMT. ?? David ----- > Does the same thing as we do for other tests. Plus introduces NMT-specific inline macros. > > Testing: x86_64 server build, x86 minimal build, x86 server build > > Thanks, > -Aleksey > From thomas.stuefe at gmail.com Sun May 6 11:39:34 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sun, 06 May 2018 11:39:34 +0000 Subject: RFR (S) 8202684: Minimal VM build is broken after JDK-8199067, JDK-8202638 In-Reply-To: <847ccb5b-b3a6-3860-bbcb-cdfffb2f32b8@oracle.com> References: <670eac07-2d76-8bf4-cd31-0b48302d0fd3@redhat.com> <847ccb5b-b3a6-3860-bbcb-cdfffb2f32b8@oracle.com> Message-ID: Hi David, You are right. See my previous answer to Alexey. I wIll fix it. Best Regards, Thomas On Sun 6. May 2018 at 09:19, David Holmes wrote: > Hi Aleksey, > > On 5/05/2018 8:52 PM, Aleksey Shipilev wrote: > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8202684 > > > > Fix: > > http://cr.openjdk.java.net/~shade/8202684/webrev.01/ > > While this will fix your issue I can't help but think that the > dependency on NMT should be far more explicit and potentially handled in > a different place. It is very unclear to me why > share/memory/metaspace/metaspaceDCmd.cpp is intimately tied to NMT, or > what it means to use it without NMT. ?? > > David > ----- > > > Does the same thing as we do for other tests. Plus introduces > NMT-specific inline macros. > > > > Testing: x86_64 server build, x86 minimal build, x86 server build > > > > Thanks, > > -Aleksey > > > From rkennke at redhat.com Sun May 6 13:47:40 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sun, 6 May 2018 15:47:40 +0200 Subject: Question about Runtime relevant Java JEP. In-Reply-To: References: Message-ID: <320250c2-186a-0a4e-4850-a97c0dec4a67@redhat.com> Am 06.05.2018 um 04:25 schrieb A Z: > Is it possible for someone to reply explainting > what the Candidate status, or really the thinking > and present state, of JEP 306 is? > > I have been recommended to this list for the sake > of this question. > > ? > > http://openjdk.java.net/jeps/306 > > https://bugs.openjdk.java.net/browse/JDK-8175916 I am not sure what you are asking. 'Candidate' means it's considered for inclusion soon. It's not yet 'targeted' which means it is not yet clear/decided if / which JDK version it will land in. I am not familiar with the present state of the floating point semantics. Roman From bsrbnd at gmail.com Sun May 6 16:35:32 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sun, 6 May 2018 18:35:32 +0200 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: <6AE7B98E-FF1B-43E0-8108-7128707BD5F5@oracle.com> References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> <6AE7B98E-FF1B-43E0-8108-7128707BD5F5@oracle.com> Message-ID: On 5 May 2018 at 22:26, Kim Barrett wrote: >> On May 5, 2018, at 8:03 AM, B. Blaser wrote: >> >> On 4 May 2018 at 17:42, Alan Bateman wrote: >>> On 04/05/2018 16:31, B. Blaser wrote: >>>> >>>> : >>>> >>>> I'll create a new JBS issue (unless you want to) since the fix is >>>> mostly located in core-libs and only intended to make the build >>>> complete. >>>> But I'll still need someone to review and push the fix, would you be >>>> interested in doing this? >>>> >>> I think someone needs to explore the behavior on other platforms too as the >>> #ifndef __linux__ patches are ugly. >> >> Yes sure but I cannot test it elsewhere myself... and we can easily >> remove them once POSIX officially deprecates 'readdir_r' or if someone >> validates the fix on other systems. > > I'm not sure anyone has the ability to test on all supported. That > doesn't (and really can't) prevent making changes across platforms to > platform-specific code. It just means one needs to get help with > testing. Make the change across platforms, test on those platforms > one has access to, and ask here for help testing on others. OK, I'll provide a cross-platform fix to be evaluated on other systems too. Thanks, Bernard > Waiting for a POSIX change is even more like watching paint dry than > waiting for Java releases used to be. From aph at redhat.com Sun May 6 20:49:29 2018 From: aph at redhat.com (Andrew Haley) Date: Sun, 6 May 2018 21:49:29 +0100 Subject: Inquiry of JEP status. In-Reply-To: References: Message-ID: On 03/05/18 23:54, A Z wrote: > I have had the suggestion that I can enquire on this email list as > to the status of > > JEP 306 > > on this email list. It appears to have been raised to Candidate, > while remaining marked unresolved. Is there any consideration > for change on this significant area inside Java SE? The only platform this affects is 32-bit x86, and interest in that cpu is diminishing every day. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From poweruserm at live.com.au Mon May 7 00:55:57 2018 From: poweruserm at live.com.au (A Z) Date: Mon, 7 May 2018 00:55:57 +0000 Subject: Question about status of JEP 306 Message-ID: I simply wanted to know what the "official" state of thought and point of view is about JEP 306. https://bugs.openjdk.java.net/browse/JDK-8175916 What are the relevant people thinking about this JEP? When would it shift to 'targeted'? From erik.osterlund at oracle.com Mon May 7 08:53:04 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 7 May 2018 10:53:04 +0200 Subject: RFR: 8202479: Add missing try_resolve_jobject_in_native calls In-Reply-To: <5AE88118.7080203@oracle.com> References: <5AE88118.7080203@oracle.com> Message-ID: <5AF013F0.3060209@oracle.com> Ping! On 2018-05-01 17:00, Erik ?sterlund wrote: > Hi, > > There are some missing calls to try_resolve_jobject_in_native for the > jni fast get field optimization. > > On x86_64, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT > and T_LONG, but is missing for T_FLOAT and T_DOUBLE. > On SPARC, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT, > but is missing for T_LONG, T_FLOAT and T_DOUBLE. > On AArch64, it is used for all types. > > Here is a patch to add the missing calls to > try_resolve_jobject_in_native. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8202479/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8202479 > > Thanks, > /Erik From thomas.schatzl at oracle.com Mon May 7 09:54:47 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 07 May 2018 11:54:47 +0200 Subject: RFR [8u] 8165489 (S): Missing G1 barrier in Unsafe_GetObjectVolatile In-Reply-To: <19ca5b19-d458-b37f-1a7c-7404a027306a@redhat.com> References: <19ca5b19-d458-b37f-1a7c-7404a027306a@redhat.com> Message-ID: <1525686887.7127.7.camel@oracle.com> Hi, On Wed, 2018-04-25 at 12:34 +0200, Aleksey Shipilev wrote: > (trying again on hotspot-dev@) > > Hi, > > Please review this this backport/fix for 8u. This improves G1 > stability in 8u, and provides the base > for Shenandoah JDK 8 backport that piggybacks on G1 pre-barriers. The > fix does not apply cleanly, > because there is also Unsafe_GetObject140 that needs fixing too. > > JDK 9 bug: > https://bugs.openjdk.java.net/browse/JDK-8165489 > > JDK 9 fix: > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/a696583f5ddb > > JDK 8u fix: > http://cr.openjdk.java.net/~shade/8165489-8u/webrev.01/ > > Testing: x86_64 build, Shenandoah tests I think this is good. Let me run it through jprt, I will report back later. Thomas From stefan.karlsson at oracle.com Mon May 7 11:37:33 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 7 May 2018 13:37:33 +0200 Subject: RFR: 8202709: Move oopDesc::is_archive_object to MetaspaceShared::is_archive_object Message-ID: <51a65446-5d14-5877-dc12-0b425ca8afe6@oracle.com> Hi all, Please review this patch to move oopDesc::is_archive_object to MetaspaceShared::is_archive_object. http://cr.openjdk.java.net/~stefank/8202709/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8202709 The motivation for this patch is to move dependencies to G1 out of oopDesc. I've placed the functions in MetaspaceShared, within INCLUDE_CDS_JAVA_HEAP guards, where related functions G1 Archive functions can be found. Thanks, StefanK From bsrbnd at gmail.com Mon May 7 12:19:14 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 7 May 2018 14:19:14 +0200 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> <6AE7B98E-FF1B-43E0-8108-7128707BD5F5@oracle.com> Message-ID: On 6 May 2018 at 18:35, B. Blaser wrote: > On 5 May 2018 at 22:26, Kim Barrett wrote: >>> On May 5, 2018, at 8:03 AM, B. Blaser wrote: >>> >>> On 4 May 2018 at 17:42, Alan Bateman wrote: >>>> On 04/05/2018 16:31, B. Blaser wrote: >>>>> >>>>> : >>>>> >>>>> I'll create a new JBS issue (unless you want to) since the fix is >>>>> mostly located in core-libs and only intended to make the build >>>>> complete. >>>>> But I'll still need someone to review and push the fix, would you be >>>>> interested in doing this? >>>>> >>>> I think someone needs to explore the behavior on other platforms too as the >>>> #ifndef __linux__ patches are ugly. >>> >>> Yes sure but I cannot test it elsewhere myself... and we can easily >>> remove them once POSIX officially deprecates 'readdir_r' or if someone >>> validates the fix on other systems. >> >> I'm not sure anyone has the ability to test on all supported. That >> doesn't (and really can't) prevent making changes across platforms to >> platform-specific code. It just means one needs to get help with >> testing. Make the change across platforms, test on those platforms >> one has access to, and ask here for help testing on others. > > OK, I'll provide a cross-platform fix to be evaluated on other systems too. Here it is, tier1 is successful on Linux with glibc=2.26. Any feedback on other systems would be very helpful. Does this look better? Thanks, Bernard diff -r e81481fea884 src/java.base/unix/native/libjava/TimeZone_md.c --- a/src/java.base/unix/native/libjava/TimeZone_md.c Mon May 07 08:56:35 2018 +0200 +++ b/src/java.base/unix/native/libjava/TimeZone_md.c Mon May 07 12:38:42 2018 +0200 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1999, 2016, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1999, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -122,32 +122,18 @@ DIR *dirp = NULL; struct stat statbuf; struct dirent64 *dp = NULL; - struct dirent64 *entry = NULL; char *pathname = NULL; int fd = -1; char *dbuf = NULL; char *tz = NULL; int res; - long name_max = 0; dirp = opendir(dir); if (dirp == NULL) { return NULL; } - name_max = pathconf(dir, _PC_NAME_MAX); - // If pathconf did not work, fall back to a mimimum buffer size. - if (name_max < 1024) { - name_max = 1024; - } - - entry = (struct dirent64 *)malloc(offsetof(struct dirent64, d_name) + name_max + 1); - if (entry == NULL) { - (void) closedir(dirp); - return NULL; - } - - while (readdir64_r(dirp, entry, &dp) == 0 && dp != NULL) { + while ((dp = readdir64(dirp)) != NULL) { /* * Skip '.' and '..' (and possibly other .* files) */ @@ -214,9 +200,6 @@ pathname = NULL; } - if (entry != NULL) { - free((void *) entry); - } if (dirp != NULL) { (void) closedir(dirp); } diff -r e81481fea884 src/java.base/unix/native/libjava/UnixFileSystem_md.c --- a/src/java.base/unix/native/libjava/UnixFileSystem_md.c Mon May 07 08:56:35 2018 +0200 +++ b/src/java.base/unix/native/libjava/UnixFileSystem_md.c Mon May 07 12:38:42 2018 +0200 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1998, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1998, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -312,7 +312,6 @@ { DIR *dir = NULL; struct dirent64 *ptr; - struct dirent64 *result; int len, maxlen; jobjectArray rv, old; jclass str_class; @@ -325,13 +324,6 @@ } END_PLATFORM_STRING(env, path); if (dir == NULL) return NULL; - ptr = malloc(sizeof(struct dirent64) + (PATH_MAX + 1)); - if (ptr == NULL) { - JNU_ThrowOutOfMemoryError(env, "heap allocation failed"); - closedir(dir); - return NULL; - } - /* Allocate an initial String array */ len = 0; maxlen = 16; @@ -339,7 +331,7 @@ if (rv == NULL) goto error; /* Scan the directory */ - while ((readdir64_r(dir, ptr, &result) == 0) && (result != NULL)) { + while ((ptr = readdir64(dir)) != NULL) { jstring name; if (!strcmp(ptr->d_name, ".") || !strcmp(ptr->d_name, "..")) continue; @@ -360,7 +352,6 @@ (*env)->DeleteLocalRef(env, name); } closedir(dir); - free(ptr); /* Copy the final results into an appropriately-sized array */ old = rv; @@ -375,7 +366,6 @@ error: closedir(dir); - free(ptr); return NULL; } diff -r e81481fea884 src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c --- a/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Mon May 07 08:56:35 2018 +0200 +++ b/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Mon May 07 12:38:42 2018 +0200 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2008, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -720,24 +720,20 @@ JNIEXPORT jbyteArray JNICALL Java_sun_nio_fs_UnixNativeDispatcher_readdir(JNIEnv* env, jclass this, jlong value) { - struct dirent64* result; - struct { - struct dirent64 buf; - char name_extra[PATH_MAX + 1 - sizeof result->d_name]; - } entry; - struct dirent64* ptr = &entry.buf; + struct dirent64* ptr = NULL; int res; DIR* dirp = jlong_to_ptr(value); /* EINTR not listed as a possible error */ - /* TDB: reentrant version probably not required here */ - res = readdir64_r(dirp, ptr, &result); + errno = 0; + ptr = readdir64(dirp); + res = errno; #ifdef _AIX - /* On AIX, readdir_r() returns EBADF (i.e. '9') and sets 'result' to NULL for the */ + /* On AIX, readdir() returns EBADF (i.e. '9') and sets 'result' to NULL for the */ /* directory stream end. Otherwise, 'errno' will contain the error code. */ if (res != 0) { - res = (result == NULL && res == EBADF) ? 0 : errno; + res = (ptr == NULL && res == EBADF) ? 0 : errno; } #endif @@ -745,7 +741,7 @@ throwUnixException(env, res); return NULL; } else { - if (result == NULL) { + if (ptr == NULL) { return NULL; } else { jsize len = strlen(ptr->d_name); diff -r e81481fea884 src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c --- a/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c Mon May 07 08:56:35 2018 +0200 +++ b/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c Mon May 07 12:38:42 2018 +0200 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2003, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -75,17 +75,8 @@ #endif /* _ALLBSD_SOURCE */ static struct dirent* read_dir(DIR* dirp, struct dirent* entry) { -#ifdef __solaris__ struct dirent* dbuf = readdir(dirp); return dbuf; -#else /* __linux__ || _ALLBSD_SOURCE */ - struct dirent* p; - if (readdir_r(dirp, entry, &p) == 0) { - return p; - } else { - return NULL; - } -#endif } // true = get available swap in bytes From thomas.schatzl at oracle.com Mon May 7 12:37:33 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 07 May 2018 14:37:33 +0200 Subject: RFR [8u] 8165489 (S): Missing G1 barrier in Unsafe_GetObjectVolatile In-Reply-To: <1525686887.7127.7.camel@oracle.com> References: <19ca5b19-d458-b37f-1a7c-7404a027306a@redhat.com> <1525686887.7127.7.camel@oracle.com> Message-ID: <1525696653.7127.15.camel@oracle.com> Hi again, On Mon, 2018-05-07 at 11:54 +0200, Thomas Schatzl wrote: > Hi, > > On Wed, 2018-04-25 at 12:34 +0200, Aleksey Shipilev wrote: > > (trying again on hotspot-dev@) > > > > Hi, > > > > Please review this this backport/fix for 8u. This improves G1 > > [...] > > > > JDK 9 bug: > > https://bugs.openjdk.java.net/browse/JDK-8165489 > > > > JDK 9 fix: > > http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/a696583f5ddb > > > > JDK 8u fix: > > http://cr.openjdk.java.net/~shade/8165489-8u/webrev.01/ > > > > Testing: x86_64 build, Shenandoah tests > > I think this is good. Let me run it through jprt, I will report > back later. jprt run passed. Thomas From stefan.karlsson at oracle.com Mon May 7 13:08:51 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 7 May 2018 15:08:51 +0200 Subject: RFR: 8202647: Add deduplicate_string function to CollectedHeap Message-ID: <65817be0-d41f-a591-820b-42bb7dca56ca@oracle.com> Hi all, Please review this patch to add a CollectedHeap::deduplicate_string virtual function. https://bugs.openjdk.java.net/browse/JDK-8202647 Today we have this G1 specific code inside the StringTable: #if INCLUDE_G1GC if (G1StringDedup::is_enabled()) { // Deduplicate the string before it is interned. Note that we should never // deduplicate a string after it has been interned. Doing so will counteract // compiler optimizations done on e.g. interned string literals. G1StringDedup::deduplicate(string()); } #endif This patch adds a new virtual call to CollectedHeap and hides the G1 specific code inside G1. I've verified that this doesn't cause any noticeable performance regressions with one of Robbin's intern string JMH micro benchmarks developed for JDK-8195097. Thanks, StefanK From stefan.karlsson at oracle.com Mon May 7 13:12:02 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 7 May 2018 15:12:02 +0200 Subject: RFR: 8202647: Add deduplicate_string function to CollectedHeap In-Reply-To: <65817be0-d41f-a591-820b-42bb7dca56ca@oracle.com> References: <65817be0-d41f-a591-820b-42bb7dca56ca@oracle.com> Message-ID: <4a9e8368-c55f-0edb-2c8b-e835b70ad172@oracle.com> And here is the webrev: http://cr.openjdk.java.net/~stefank/8202647/webrev.01/ StefanK On 2018-05-07 15:08, Stefan Karlsson wrote: > Hi all, > > Please review this patch to add a CollectedHeap::deduplicate_string > virtual function. > > https://bugs.openjdk.java.net/browse/JDK-8202647 > > Today we have this G1 specific code inside the StringTable: > > #if INCLUDE_G1GC > ?if (G1StringDedup::is_enabled()) { > ?// Deduplicate the string before it is interned. Note that we should > never > ?// deduplicate a string after it has been interned. Doing so will > counteract > ?// compiler optimizations done on e.g. interned string literals. > ?G1StringDedup::deduplicate(string()); > ?} > #endif > > This patch adds a new virtual call to CollectedHeap and hides the G1 > specific code inside G1. > > I've verified that this doesn't cause any noticeable performance > regressions with one of Robbin's intern string JMH micro benchmarks > developed for JDK-8195097. > > Thanks, > StefanK From robbin.ehn at oracle.com Mon May 7 13:25:28 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 7 May 2018 15:25:28 +0200 Subject: RFR: 8202647: Add deduplicate_string function to CollectedHeap In-Reply-To: <4a9e8368-c55f-0edb-2c8b-e835b70ad172@oracle.com> References: <65817be0-d41f-a591-820b-42bb7dca56ca@oracle.com> <4a9e8368-c55f-0edb-2c8b-e835b70ad172@oracle.com> Message-ID: <6c1e9000-14e3-90b1-7d4d-377fafaa37fa@oracle.com> On 05/07/2018 03:12 PM, Stefan Karlsson wrote: > And here is the webrev: > http://cr.openjdk.java.net/~stefank/8202647/webrev.01/ Looks good, thanks for fixing. /Robbin > > StefanK > > On 2018-05-07 15:08, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to add a CollectedHeap::deduplicate_string virtual >> function. >> >> https://bugs.openjdk.java.net/browse/JDK-8202647 >> >> Today we have this G1 specific code inside the StringTable: >> >> #if INCLUDE_G1GC >> ??if (G1StringDedup::is_enabled()) { >> ??// Deduplicate the string before it is interned. Note that we should never >> ??// deduplicate a string after it has been interned. Doing so will counteract >> ??// compiler optimizations done on e.g. interned string literals. >> ??G1StringDedup::deduplicate(string()); >> ??} >> #endif >> >> This patch adds a new virtual call to CollectedHeap and hides the G1 specific >> code inside G1. >> >> I've verified that this doesn't cause any noticeable performance regressions >> with one of Robbin's intern string JMH micro benchmarks developed for >> JDK-8195097. >> >> Thanks, >> StefanK From stefan.johansson at oracle.com Mon May 7 13:27:39 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Mon, 7 May 2018 15:27:39 +0200 Subject: RFR: 8202647: Add deduplicate_string function to CollectedHeap In-Reply-To: <4a9e8368-c55f-0edb-2c8b-e835b70ad172@oracle.com> References: <65817be0-d41f-a591-820b-42bb7dca56ca@oracle.com> <4a9e8368-c55f-0edb-2c8b-e835b70ad172@oracle.com> Message-ID: Hi Stefan, On 2018-05-07 15:12, Stefan Karlsson wrote: > And here is the webrev: > http://cr.openjdk.java.net/~stefank/8202647/webrev.01/ Looks good, thanks for cleaning this up, StefanJ > > StefanK > > On 2018-05-07 15:08, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to add a CollectedHeap::deduplicate_string >> virtual function. >> >> https://bugs.openjdk.java.net/browse/JDK-8202647 >> >> Today we have this G1 specific code inside the StringTable: >> >> #if INCLUDE_G1GC >> ??if (G1StringDedup::is_enabled()) { >> ??// Deduplicate the string before it is interned. Note that we should >> never >> ??// deduplicate a string after it has been interned. Doing so will >> counteract >> ??// compiler optimizations done on e.g. interned string literals. >> ??G1StringDedup::deduplicate(string()); >> ??} >> #endif >> >> This patch adds a new virtual call to CollectedHeap and hides the G1 >> specific code inside G1. >> >> I've verified that this doesn't cause any noticeable performance >> regressions with one of Robbin's intern string JMH micro benchmarks >> developed for JDK-8195097. >> >> Thanks, >> StefanK From stefan.karlsson at oracle.com Mon May 7 13:27:01 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 7 May 2018 15:27:01 +0200 Subject: RFR: 8202647: Add deduplicate_string function to CollectedHeap In-Reply-To: <6c1e9000-14e3-90b1-7d4d-377fafaa37fa@oracle.com> References: <65817be0-d41f-a591-820b-42bb7dca56ca@oracle.com> <4a9e8368-c55f-0edb-2c8b-e835b70ad172@oracle.com> <6c1e9000-14e3-90b1-7d4d-377fafaa37fa@oracle.com> Message-ID: <210dd82d-da94-24df-b29b-588b2ed8aa9c@oracle.com> On 2018-05-07 15:25, Robbin Ehn wrote: > On 05/07/2018 03:12 PM, Stefan Karlsson wrote: >> And here is the webrev: >> http://cr.openjdk.java.net/~stefank/8202647/webrev.01/ > > Looks good, thanks for fixing. Thanks, Robbin. StefanK > > /Robbin > >> >> StefanK >> >> On 2018-05-07 15:08, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to add a CollectedHeap::deduplicate_string >>> virtual function. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8202647 >>> >>> Today we have this G1 specific code inside the StringTable: >>> >>> #if INCLUDE_G1GC >>> ??if (G1StringDedup::is_enabled()) { >>> ??// Deduplicate the string before it is interned. Note that we >>> should never >>> ??// deduplicate a string after it has been interned. Doing so will >>> counteract >>> ??// compiler optimizations done on e.g. interned string literals. >>> ??G1StringDedup::deduplicate(string()); >>> ??} >>> #endif >>> >>> This patch adds a new virtual call to CollectedHeap and hides the G1 >>> specific code inside G1. >>> >>> I've verified that this doesn't cause any noticeable performance >>> regressions with one of Robbin's intern string JMH micro benchmarks >>> developed for JDK-8195097. >>> >>> Thanks, >>> StefanK From stefan.karlsson at oracle.com Mon May 7 13:27:38 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 7 May 2018 15:27:38 +0200 Subject: RFR: 8202647: Add deduplicate_string function to CollectedHeap In-Reply-To: References: <65817be0-d41f-a591-820b-42bb7dca56ca@oracle.com> <4a9e8368-c55f-0edb-2c8b-e835b70ad172@oracle.com> Message-ID: On 2018-05-07 15:27, Stefan Johansson wrote: > Hi Stefan, > > On 2018-05-07 15:12, Stefan Karlsson wrote: >> And here is the webrev: >> http://cr.openjdk.java.net/~stefank/8202647/webrev.01/ > Looks good, thanks for cleaning this up, Thanks, StefanJ. StefanK > StefanJ > >> >> StefanK >> >> On 2018-05-07 15:08, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to add a CollectedHeap::deduplicate_string >>> virtual function. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8202647 >>> >>> Today we have this G1 specific code inside the StringTable: >>> >>> #if INCLUDE_G1GC >>> ??if (G1StringDedup::is_enabled()) { >>> ??// Deduplicate the string before it is interned. Note that we >>> should never >>> ??// deduplicate a string after it has been interned. Doing so will >>> counteract >>> ??// compiler optimizations done on e.g. interned string literals. >>> ??G1StringDedup::deduplicate(string()); >>> ??} >>> #endif >>> >>> This patch adds a new virtual call to CollectedHeap and hides the G1 >>> specific code inside G1. >>> >>> I've verified that this doesn't cause any noticeable performance >>> regressions with one of Robbin's intern string JMH micro benchmarks >>> developed for JDK-8195097. >>> >>> Thanks, >>> StefanK From coleen.phillimore at oracle.com Mon May 7 13:51:50 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 7 May 2018 09:51:50 -0400 Subject: RFR: 8202479: Add missing try_resolve_jobject_in_native calls In-Reply-To: <5AF013F0.3060209@oracle.com> References: <5AE88118.7080203@oracle.com> <5AF013F0.3060209@oracle.com> Message-ID: <8a825b7f-a372-8979-12a4-626235de6f02@oracle.com> http://cr.openjdk.java.net/~eosterlund/8202479/webrev.00/src/hotspot/cpu/sparc/jniFastGetField_sparc.cpp.udiff.html Generally G3 is seen as G3_scratch in sparc code. Why don't both paths have G3 as tmp? thanks, Coleen On 5/7/18 4:53 AM, Erik ?sterlund wrote: > Ping! > > On 2018-05-01 17:00, Erik ?sterlund wrote: >> Hi, >> >> There are some missing calls to try_resolve_jobject_in_native for the >> jni fast get field optimization. >> >> On x86_64, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT >> and T_LONG, but is missing for T_FLOAT and T_DOUBLE. >> On SPARC, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT, >> but is missing for T_LONG, T_FLOAT and T_DOUBLE. >> On AArch64, it is used for all types. >> >> Here is a patch to add the missing calls to >> try_resolve_jobject_in_native. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8202479/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8202479 >> >> Thanks, >> /Erik > From stefan.karlsson at oracle.com Mon May 7 14:02:51 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 7 May 2018 16:02:51 +0200 Subject: RFR: 8202649: Move the Parallel GC specific task creation functions out of Threads Message-ID: Hi all, Please review this patch to move the Parallel GC specific task creation functions out of Threads. http://cr.openjdk.java.net/~stefank/8202649/webrev.01/ There exist (at least) two available functions to iterate over a subset of threads: static void non_java_threads_do(ThreadClosure* tc); static void threads_do(ThreadClosure* tc); This patch extends this set and adds: static void java_threads_do(ThreadClosure* tc); static void java_threads_and_vm_thread_do(ThreadClosure* tc); The latter is used to implement the create_thread_roots_tasks and create_thread_roots_marking_tasks inside the Parallel GC code. Thanks, StefanK From thomas.stuefe at gmail.com Mon May 7 14:23:57 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 7 May 2018 16:23:57 +0200 Subject: JBS, confused about JDK-8201572, JDK-8202638 Message-ID: Hi, I created JDK-8201572 to improve metaspace coding. Implemented fix, reviewed, tested, pushed. However, that item has been marked in the meantime as "target 12". So the push created a new item, JDK-8202638, marked as "backport of JDK-8201572", targeted for 11. JDK-8202638 now is marked as resolved, the original JDK-8201572 is still open. Can you help? JDK-8201572 should be closed and should have target release 11. Thank you! Thomas From erik.osterlund at oracle.com Mon May 7 14:39:35 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Mon, 7 May 2018 16:39:35 +0200 Subject: RFR: 8202479: Add missing try_resolve_jobject_in_native calls In-Reply-To: <8a825b7f-a372-8979-12a4-626235de6f02@oracle.com> References: <5AE88118.7080203@oracle.com> <5AF013F0.3060209@oracle.com> <8a825b7f-a372-8979-12a4-626235de6f02@oracle.com> Message-ID: <5AF06527.3060109@oracle.com> Hi Coleen, Thank you for having a look at this! On 2018-05-07 15:51, coleen.phillimore at oracle.com wrote: > > http://cr.openjdk.java.net/~eosterlund/8202479/webrev.00/src/hotspot/cpu/sparc/jniFastGetField_sparc.cpp.udiff.html > > > Generally G3 is seen as G3_scratch in sparc code. Yes, but I noticed that this is not the case in this file, and tried to follow the conventions for this file. > Why don't both paths have G3 as tmp? Because one of them retains the old safepoint counter in G3. So if I scratch G3, then a subsequent check that the safepoint counter has not changed from before the access was made, will fail. So in that peculiar case, I had to find a different register. I chose G1, because it is caller saved in both the Java and C ABIs - like G3. Thanks, /Erik > thanks, > Coleen > > On 5/7/18 4:53 AM, Erik ?sterlund wrote: >> Ping! >> >> On 2018-05-01 17:00, Erik ?sterlund wrote: >>> Hi, >>> >>> There are some missing calls to try_resolve_jobject_in_native for >>> the jni fast get field optimization. >>> >>> On x86_64, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT >>> and T_LONG, but is missing for T_FLOAT and T_DOUBLE. >>> On SPARC, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT, >>> but is missing for T_LONG, T_FLOAT and T_DOUBLE. >>> On AArch64, it is used for all types. >>> >>> Here is a patch to add the missing calls to >>> try_resolve_jobject_in_native. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8202479/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8202479 >>> >>> Thanks, >>> /Erik >> > From erik.helin at oracle.com Mon May 7 14:44:47 2018 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 7 May 2018 16:44:47 +0200 Subject: RFR: 8202649: Move the Parallel GC specific task creation functions out of Threads In-Reply-To: References: Message-ID: On 05/07/2018 04:02 PM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to move the Parallel GC specific task creation > functions out of Threads. > > http://cr.openjdk.java.net/~stefank/8202649/webrev.01/ Looks good to me, thanks for taking care of this, Reviewed! Erik > There exist (at least) two available functions to iterate over a subset > of threads: > ?static void non_java_threads_do(ThreadClosure* tc); > ?static void threads_do(ThreadClosure* tc); > > This patch extends this set and adds: > ?static void java_threads_do(ThreadClosure* tc); > ?static void java_threads_and_vm_thread_do(ThreadClosure* tc); > > The latter is used to implement the create_thread_roots_tasks and > create_thread_roots_marking_tasks inside the Parallel GC code. > > Thanks, > StefanK From coleen.phillimore at oracle.com Mon May 7 14:58:03 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 7 May 2018 10:58:03 -0400 Subject: RFR: 8202479: Add missing try_resolve_jobject_in_native calls In-Reply-To: <5AF06527.3060109@oracle.com> References: <5AE88118.7080203@oracle.com> <5AF013F0.3060209@oracle.com> <8a825b7f-a372-8979-12a4-626235de6f02@oracle.com> <5AF06527.3060109@oracle.com> Message-ID: On 5/7/18 10:39 AM, Erik ?sterlund wrote: > Hi Coleen, > > Thank you for having a look at this! > > On 2018-05-07 15:51, coleen.phillimore at oracle.com wrote: >> >> http://cr.openjdk.java.net/~eosterlund/8202479/webrev.00/src/hotspot/cpu/sparc/jniFastGetField_sparc.cpp.udiff.html >> >> >> Generally G3 is seen as G3_scratch in sparc code. > > Yes, but I noticed that this is not the case in this file, and tried > to follow the conventions for this file. > >> Why don't both paths have G3 as tmp? > > Because one of them retains the old safepoint counter in G3. So if I > scratch G3, then a subsequent check that the safepoint counter has not > changed from before the access was made, will fail. So in that > peculiar case, I had to find a different register. I chose G1, because > it is caller saved in both the Java and C ABIs - like G3. Ok, looks good then.? I like that you've trashed the register if the branch isn't taken for checking that it's free. thanks, Coleen > > Thanks, > /Erik > >> thanks, >> Coleen >> >> On 5/7/18 4:53 AM, Erik ?sterlund wrote: >>> Ping! >>> >>> On 2018-05-01 17:00, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> There are some missing calls to try_resolve_jobject_in_native for >>>> the jni fast get field optimization. >>>> >>>> On x86_64, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT >>>> and T_LONG, but is missing for T_FLOAT and T_DOUBLE. >>>> On SPARC, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT, >>>> but is missing for T_LONG, T_FLOAT and T_DOUBLE. >>>> On AArch64, it is used for all types. >>>> >>>> Here is a patch to add the missing calls to >>>> try_resolve_jobject_in_native. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8202479/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8202479 >>>> >>>> Thanks, >>>> /Erik >>> >> > From stefan.karlsson at oracle.com Mon May 7 15:12:53 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 7 May 2018 17:12:53 +0200 Subject: RFR: 8202649: Move the Parallel GC specific task creation functions out of Threads In-Reply-To: References: Message-ID: On 2018-05-07 16:44, Erik Helin wrote: > On 05/07/2018 04:02 PM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to move the Parallel GC specific task >> creation functions out of Threads. >> >> http://cr.openjdk.java.net/~stefank/8202649/webrev.01/ > > Looks good to me, thanks for taking care of this, Reviewed! Thanks, Erik! StefanK > > Erik > >> There exist (at least) two available functions to iterate over a >> subset of threads: >> ??static void non_java_threads_do(ThreadClosure* tc); >> ??static void threads_do(ThreadClosure* tc); >> >> This patch extends this set and adds: >> ??static void java_threads_do(ThreadClosure* tc); >> ??static void java_threads_and_vm_thread_do(ThreadClosure* tc); >> >> The latter is used to implement the create_thread_roots_tasks and >> create_thread_roots_marking_tasks inside the Parallel GC code. >> >> Thanks, >> StefanK From erik.osterlund at oracle.com Mon May 7 15:13:03 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Mon, 7 May 2018 17:13:03 +0200 Subject: RFR: 8202479: Add missing try_resolve_jobject_in_native calls In-Reply-To: References: <5AE88118.7080203@oracle.com> <5AF013F0.3060209@oracle.com> <8a825b7f-a372-8979-12a4-626235de6f02@oracle.com> <5AF06527.3060109@oracle.com> Message-ID: <656704B3-B321-4B6D-ADA9-D8E20A5660EF@oracle.com> Hi Coleen, Thanks for the review. /Erik > On 7 May 2018, at 16:58, coleen.phillimore at oracle.com wrote: > > > >> On 5/7/18 10:39 AM, Erik ?sterlund wrote: >> Hi Coleen, >> >> Thank you for having a look at this! >> >>> On 2018-05-07 15:51, coleen.phillimore at oracle.com wrote: >>> >>> http://cr.openjdk.java.net/~eosterlund/8202479/webrev.00/src/hotspot/cpu/sparc/jniFastGetField_sparc.cpp.udiff.html >>> >>> Generally G3 is seen as G3_scratch in sparc code. >> >> Yes, but I noticed that this is not the case in this file, and tried to follow the conventions for this file. >> >>> Why don't both paths have G3 as tmp? >> >> Because one of them retains the old safepoint counter in G3. So if I scratch G3, then a subsequent check that the safepoint counter has not changed from before the access was made, will fail. So in that peculiar case, I had to find a different register. I chose G1, because it is caller saved in both the Java and C ABIs - like G3. > > Ok, looks good then. I like that you've trashed the register if the branch isn't taken for checking that it's free. > thanks, > Coleen > >> >> Thanks, >> /Erik >> >>> thanks, >>> Coleen >>> >>>> On 5/7/18 4:53 AM, Erik ?sterlund wrote: >>>> Ping! >>>> >>>>> On 2018-05-01 17:00, Erik ?sterlund wrote: >>>>> Hi, >>>>> >>>>> There are some missing calls to try_resolve_jobject_in_native for the jni fast get field optimization. >>>>> >>>>> On x86_64, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT and T_LONG, but is missing for T_FLOAT and T_DOUBLE. >>>>> On SPARC, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT, but is missing for T_LONG, T_FLOAT and T_DOUBLE. >>>>> On AArch64, it is used for all types. >>>>> >>>>> Here is a patch to add the missing calls to try_resolve_jobject_in_native. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8202479/webrev.00/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8202479 >>>>> >>>>> Thanks, >>>>> /Erik >>>> >>> >> > From bsrbnd at gmail.com Mon May 7 15:20:11 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 7 May 2018 17:20:11 +0200 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> <6AE7B98E-FF1B-43E0-8108-7128707BD5F5@oracle.com> Message-ID: On 7 May 2018 at 14:19, B. Blaser wrote: > On 6 May 2018 at 18:35, B. Blaser wrote: >> On 5 May 2018 at 22:26, Kim Barrett wrote: >>>> On May 5, 2018, at 8:03 AM, B. Blaser wrote: >>>> >>>> On 4 May 2018 at 17:42, Alan Bateman wrote: >>>>> On 04/05/2018 16:31, B. Blaser wrote: >>>>>> >>>>>> : >>>>>> >>>>>> I'll create a new JBS issue (unless you want to) since the fix is >>>>>> mostly located in core-libs and only intended to make the build >>>>>> complete. >>>>>> But I'll still need someone to review and push the fix, would you be >>>>>> interested in doing this? >>>>>> >>>>> I think someone needs to explore the behavior on other platforms too as the >>>>> #ifndef __linux__ patches are ugly. >>>> >>>> Yes sure but I cannot test it elsewhere myself... and we can easily >>>> remove them once POSIX officially deprecates 'readdir_r' or if someone >>>> validates the fix on other systems. >>> >>> I'm not sure anyone has the ability to test on all supported. That >>> doesn't (and really can't) prevent making changes across platforms to >>> platform-specific code. It just means one needs to get help with >>> testing. Make the change across platforms, test on those platforms >>> one has access to, and ask here for help testing on others. >> >> OK, I'll provide a cross-platform fix to be evaluated on other systems too. > > Here it is, tier1 is successful on Linux with glibc=2.26. > Any feedback on other systems would be very helpful. Some more cleanup... Does this look better? Thanks, Bernard diff -r e81481fea884 src/java.base/unix/native/libjava/TimeZone_md.c --- a/src/java.base/unix/native/libjava/TimeZone_md.c Mon May 07 08:56:35 2018 +0200 +++ b/src/java.base/unix/native/libjava/TimeZone_md.c Mon May 07 15:14:26 2018 +0200 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1999, 2016, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1999, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -122,32 +122,18 @@ DIR *dirp = NULL; struct stat statbuf; struct dirent64 *dp = NULL; - struct dirent64 *entry = NULL; char *pathname = NULL; int fd = -1; char *dbuf = NULL; char *tz = NULL; int res; - long name_max = 0; dirp = opendir(dir); if (dirp == NULL) { return NULL; } - name_max = pathconf(dir, _PC_NAME_MAX); - // If pathconf did not work, fall back to a mimimum buffer size. - if (name_max < 1024) { - name_max = 1024; - } - - entry = (struct dirent64 *)malloc(offsetof(struct dirent64, d_name) + name_max + 1); - if (entry == NULL) { - (void) closedir(dirp); - return NULL; - } - - while (readdir64_r(dirp, entry, &dp) == 0 && dp != NULL) { + while ((dp = readdir64(dirp)) != NULL) { /* * Skip '.' and '..' (and possibly other .* files) */ @@ -214,9 +200,6 @@ pathname = NULL; } - if (entry != NULL) { - free((void *) entry); - } if (dirp != NULL) { (void) closedir(dirp); } diff -r e81481fea884 src/java.base/unix/native/libjava/UnixFileSystem_md.c --- a/src/java.base/unix/native/libjava/UnixFileSystem_md.c Mon May 07 08:56:35 2018 +0200 +++ b/src/java.base/unix/native/libjava/UnixFileSystem_md.c Mon May 07 15:14:26 2018 +0200 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1998, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1998, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -312,7 +312,6 @@ { DIR *dir = NULL; struct dirent64 *ptr; - struct dirent64 *result; int len, maxlen; jobjectArray rv, old; jclass str_class; @@ -325,13 +324,6 @@ } END_PLATFORM_STRING(env, path); if (dir == NULL) return NULL; - ptr = malloc(sizeof(struct dirent64) + (PATH_MAX + 1)); - if (ptr == NULL) { - JNU_ThrowOutOfMemoryError(env, "heap allocation failed"); - closedir(dir); - return NULL; - } - /* Allocate an initial String array */ len = 0; maxlen = 16; @@ -339,7 +331,7 @@ if (rv == NULL) goto error; /* Scan the directory */ - while ((readdir64_r(dir, ptr, &result) == 0) && (result != NULL)) { + while ((ptr = readdir64(dir)) != NULL) { jstring name; if (!strcmp(ptr->d_name, ".") || !strcmp(ptr->d_name, "..")) continue; @@ -360,7 +352,6 @@ (*env)->DeleteLocalRef(env, name); } closedir(dir); - free(ptr); /* Copy the final results into an appropriately-sized array */ old = rv; @@ -375,7 +366,6 @@ error: closedir(dir); - free(ptr); return NULL; } diff -r e81481fea884 src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c --- a/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Mon May 07 08:56:35 2018 +0200 +++ b/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c Mon May 07 15:14:26 2018 +0200 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2008, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -720,24 +720,20 @@ JNIEXPORT jbyteArray JNICALL Java_sun_nio_fs_UnixNativeDispatcher_readdir(JNIEnv* env, jclass this, jlong value) { - struct dirent64* result; - struct { - struct dirent64 buf; - char name_extra[PATH_MAX + 1 - sizeof result->d_name]; - } entry; - struct dirent64* ptr = &entry.buf; + struct dirent64* ptr = NULL; int res; DIR* dirp = jlong_to_ptr(value); /* EINTR not listed as a possible error */ - /* TDB: reentrant version probably not required here */ - res = readdir64_r(dirp, ptr, &result); + errno = 0; + ptr = readdir64(dirp); + res = errno; #ifdef _AIX - /* On AIX, readdir_r() returns EBADF (i.e. '9') and sets 'result' to NULL for the */ + /* On AIX, readdir() returns EBADF (i.e. '9') and sets 'result' to NULL for the */ /* directory stream end. Otherwise, 'errno' will contain the error code. */ if (res != 0) { - res = (result == NULL && res == EBADF) ? 0 : errno; + res = (ptr == NULL && res == EBADF) ? 0 : errno; } #endif @@ -745,7 +741,7 @@ throwUnixException(env, res); return NULL; } else { - if (result == NULL) { + if (ptr == NULL) { return NULL; } else { jsize len = strlen(ptr->d_name); diff -r e81481fea884 src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c --- a/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c Mon May 07 08:56:35 2018 +0200 +++ b/src/jdk.management/unix/native/libmanagement_ext/OperatingSystemImpl.c Mon May 07 15:14:26 2018 +0200 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2003, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2003, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -74,20 +74,6 @@ #endif /* _ALLBSD_SOURCE */ -static struct dirent* read_dir(DIR* dirp, struct dirent* entry) { -#ifdef __solaris__ - struct dirent* dbuf = readdir(dirp); - return dbuf; -#else /* __linux__ || _ALLBSD_SOURCE */ - struct dirent* p; - if (readdir_r(dirp, entry, &p) == 0) { - return p; - } else { - return NULL; - } -#endif -} - // true = get available swap in bytes // false = get total swap in bytes static jlong get_total_or_available_swap_space_size(JNIEnv* env, jboolean available) { @@ -432,7 +418,6 @@ return (100); #else /* solaris/linux */ DIR *dirp; - struct dirent dbuf; struct dirent* dentp; jlong fds = 0; @@ -453,7 +438,7 @@ // iterate through directory entries, skipping '.' and '..' // each entry represents an open file descriptor. - while ((dentp = read_dir(dirp, &dbuf)) != NULL) { + while ((dentp = readdir(dirp)) != NULL) { if (isdigit(dentp->d_name[0])) { fds++; } From daniel.daugherty at oracle.com Mon May 7 15:24:27 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 7 May 2018 11:24:27 -0400 Subject: JBS, confused about JDK-8201572, JDK-8202638 In-Reply-To: References: Message-ID: <4b14387a-498e-761d-4069-09ce13cce6a9@oracle.com> Hi Thomas, I have fixed this for you. I've also attached my note for how to fix this kind of issue. Before pushing a fix to jdk/jdk, please make sure that the "Fix Version" is set to appropriate release. In your bug's case, the bug was targeted at "12" (as part of release planning) and you pushed a fix for "11". Dan On 5/7/18 10:23 AM, Thomas St?fe wrote: > Hi, > > I created JDK-8201572 to improve metaspace coding. Implemented fix, > reviewed, tested, pushed. > > However, that item has been marked in the meantime as "target 12". So > the push created a new item, JDK-8202638, marked as "backport of > JDK-8201572", targeted for 11. > > JDK-8202638 now is marked as resolved, the original JDK-8201572 is still open. > > Can you help? JDK-8201572 should be closed and should have target release 11. > > Thank you! > > Thomas -------------- next part -------------- If a main bug is targeted to a release and the fix is pushed to a different release, then a backport bug is automatically created. Usually this is a "good thing", e.g., when you are really backporting a fix to (cur_release - 1), but not always... If the main bug is targeted to next_release (due to schedule planning), but someone finds the time to fix that bug in cur_release, then the bug should be retargeted to cur_release before pushing the fix. However, sometimes we forget to do that... Here is how to fix that: 1) Reopen the _backport_ bug: - Use a comment like the following (in the reopen dialog): Fix was pushed while main bug was targeted to '12'. Reset the main bug to fixed in '11', reset this bug to fix in '12' and closed as 'Not An Issue' since JDK12 will automatically get this fix from JDK11. - Reset the 'Fix Version/s' from '11' to '12'. - Close the _backport_ bug as "Not an Issue". 2) Clean up the main bug: - Copy the _open_ push notification comment from the _backport_ bug to the _main_ bug, e.g.: HG Updates added a comment - 3 days ago URL: http://hg.openjdk.java.net/jdk/jdk/rev/57dd7b4ba338 User: stuefe Date: 2018-05-04 06:41:16 +0000 - Copy the _closed_ push notification comment (if there is one) from the _backport_ bug to _Confidential_ comment in the _main_ bug. - Add a comment like the following to the _main_ bug: Fix was pushed while main bug was targeted to '12'. Reset the main bug to fixed in '11' and copied the hgupdater entry here. - Reset the _main_ bug 'Fix Version/s' from '12' to '11'. - Resolve the _main_ bug as "Fixed" in build "master". From thomas.stuefe at gmail.com Mon May 7 15:27:14 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 7 May 2018 17:27:14 +0200 Subject: JBS, confused about JDK-8201572, JDK-8202638 In-Reply-To: <4b14387a-498e-761d-4069-09ce13cce6a9@oracle.com> References: <4b14387a-498e-761d-4069-09ce13cce6a9@oracle.com> Message-ID: Hi Daniel, thanks a lot! It would be nice if jcheck would catch such issues. Best, Thomas On Mon, May 7, 2018 at 5:24 PM, Daniel D. Daugherty wrote: > Hi Thomas, > > I have fixed this for you. I've also attached my note for how to fix this > kind of issue. > > Before pushing a fix to jdk/jdk, please make sure that the "Fix Version" > is set to appropriate release. In your bug's case, the bug was targeted > at "12" (as part of release planning) and you pushed a fix for "11". > > Dan > > > > On 5/7/18 10:23 AM, Thomas St?fe wrote: >> >> Hi, >> >> I created JDK-8201572 to improve metaspace coding. Implemented fix, >> reviewed, tested, pushed. >> >> However, that item has been marked in the meantime as "target 12". So >> the push created a new item, JDK-8202638, marked as "backport of >> JDK-8201572", targeted for 11. >> >> JDK-8202638 now is marked as resolved, the original JDK-8201572 is still >> open. >> >> Can you help? JDK-8201572 should be closed and should have target release >> 11. >> >> Thank you! >> >> Thomas > > From jiangli.zhou at oracle.com Mon May 7 16:01:31 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Mon, 7 May 2018 09:01:31 -0700 Subject: RFR: 8202709: Move oopDesc::is_archive_object to MetaspaceShared::is_archive_object In-Reply-To: <51a65446-5d14-5877-dc12-0b425ca8afe6@oracle.com> References: <51a65446-5d14-5877-dc12-0b425ca8afe6@oracle.com> Message-ID: <71B7A275-43DF-4B84-889E-9D63540BD62B@oracle.com> Hi Stefan, Looks good. Thanks, Jiangli > On May 7, 2018, at 4:37 AM, Stefan Karlsson wrote: > > Hi all, > > Please review this patch to move oopDesc::is_archive_object to MetaspaceShared::is_archive_object. > > http://cr.openjdk.java.net/~stefank/8202709/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8202709 > > The motivation for this patch is to move dependencies to G1 out of oopDesc. I've placed the functions in MetaspaceShared, within INCLUDE_CDS_JAVA_HEAP guards, where related functions G1 Archive functions can be found. > > Thanks, > StefanK From vladimir.kozlov at oracle.com Mon May 7 18:33:01 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 7 May 2018 11:33:01 -0700 Subject: [11] RFR (M) 8202611: [GRAAL] Exclude CMS GC testing from runs with Graal Message-ID: http://cr.openjdk.java.net/~kvn/8202611/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8202611 JDK-8184349 handles Hotspot changes to add check that Graal support GC and exit on error if it does not. Since tests changes are big I decided to do it separate and may be push first. These changes excluded tests which use CMS from Graal runs. @requires !vm.graal.enabled was added for tests which use only CMS GC explicitly. Undocumented jtreg functionality is use to split @run command with -XX:+UseConcMarkSweepGC into separate jtreg comment block if test has several @run commands. Jtreg treats such test as 2 tests. New test API sun/hotspot/code/Compiler.java (which uses WhiteBox API) is added to check if particular JIT compiler is enabled. The Graal related code was copied from VMProps.java. This was added for cases when a test forks new process and use CMS GC. New API is used to guard such code to not run with Graal. Tested with Graal enabled as JIT. -- Thanks, Vladimir From rkennke at redhat.com Mon May 7 19:47:46 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 7 May 2018 21:47:46 +0200 Subject: RFR: JDK-8202016: Use obj+offset in interpreter array access Message-ID: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> In the TemplateTable::aastore() and InterpreterMacroAssembler::load_resolved_reference_at_index(), the element address is flattened, and then passed to the BarrierSetAssembler for actual access. GCs like Shenandoah need the original obj though. The proposed change replaces the flattening with base+index+disp addressing, and removes the shift and add computations. In the case of aastore, we need to re-fetch the rcx/index from the stack because it gets trashed on the way. Testing: passed: hotspot/jtreg:tier1 Can I please get a review? Thanks, Roman From rkennke at redhat.com Mon May 7 19:57:04 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 7 May 2018 21:57:04 +0200 Subject: RFR: JDK-8202016: Use obj+offset in interpreter array access In-Reply-To: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> References: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> Message-ID: Am 07.05.2018 um 21:47 schrieb Roman Kennke: > In the TemplateTable::aastore() and > InterpreterMacroAssembler::load_resolved_reference_at_index(), the > element address is flattened, and then passed to the BarrierSetAssembler > for actual access. GCs like Shenandoah need the original obj though. > > The proposed change replaces the flattening with base+index+disp > addressing, and removes the shift and add computations. In the case of > aastore, we need to re-fetch the rcx/index from the stack because it > gets trashed on the way. > > Testing: passed: hotspot/jtreg:tier1 > > Can I please get a review? > Thanks, Roman > The webrev is missing :-) http://cr.openjdk.java.net/~rkennke/JDK-8202016/webrev.01/ Roman From rkennke at redhat.com Mon May 7 20:31:39 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 7 May 2018 22:31:39 +0200 Subject: RFR: JDK-8200623: Primitive heap access for interpreter BarrierSetAssembler/x86 Message-ID: JDK-8199417 added better modularization for interpreter barriers. Shenandoah and possibly future GCs also need barriers for primitive access. Some notes on implementation: - float/double/long access produced some headaches for the following reasons: - float and double would either take XMMRegister which is not compatible with Register - or load-from/store-to the floating point stack (see MacroAssembler::load/store_float/double) - long access on x86_32 would load-into/store-from 2 registers, or else use a trick via the floating point stack to do atomic access None of this seemed easy/nice to do with the API. I helped myself by accepting noreg as dst/src argument, which means the corresponding tos (i.e. ltos, ftos, dtos) and the BSA would then access from/to xmm0/float-stack in case of float/double or the double-reg/float-stack in case of long/32bit, which is all that we ever need. I'm passing MO_RELAXED to long access calls to hint that we want atomic access or not. I hope that is ok. Tested: hotspot/jtreg:tier1 http://cr.openjdk.java.net/~rkennke/JDK-8200623/webrev.00/ Can I please get a review? Thanks, Roman From david.holmes at oracle.com Mon May 7 20:32:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 8 May 2018 06:32:23 +1000 Subject: JBS, confused about JDK-8201572, JDK-8202638 In-Reply-To: <4b14387a-498e-761d-4069-09ce13cce6a9@oracle.com> References: <4b14387a-498e-761d-4069-09ce13cce6a9@oracle.com> Message-ID: <2603b799-d2c9-834f-3792-a5215ccf21c9@oracle.com> On 8/05/2018 1:24 AM, Daniel D. Daugherty wrote: > Hi Thomas, > > I have fixed this for you. I've also attached my note for how to fix this > kind of issue. > > Before pushing a fix to jdk/jdk, please make sure that the "Fix Version" > is set to appropriate release. In your bug's case, the bug was targeted > at "12" (as part of release planning) and you pushed a fix for "11". We need to improve processes here: 1. If you file a bug you plan on fixing immediately, set the Fix version to current release (and ensure it is assigned to yourself) 2. "release planning" should not update assigned, targeted bugs without consulting the assignee. That said IIUC we're supposed to moving to using tbd_* as "future" Fix Version rather than hard wiring 12, 13 etc. That should also avoid the backport problem. Cheers, David > Dan > > > On 5/7/18 10:23 AM, Thomas St?fe wrote: >> Hi, >> >> I created JDK-8201572 to improve metaspace coding. Implemented fix, >> reviewed, tested, pushed. >> >> However, that item has been marked in the meantime as "target 12". So >> the push created a new item, JDK-8202638, marked as "backport of >> JDK-8201572", targeted for 11. >> >> JDK-8202638 now is marked as resolved, the original JDK-8201572 is >> still open. >> >> Can you help? JDK-8201572 should be closed and should have target >> release 11. >> >> Thank you! >> >> Thomas > From igor.ignatyev at oracle.com Mon May 7 22:35:55 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 7 May 2018 15:35:55 -0700 Subject: RFR(L) : 8199370: [TESTBUG] Open source vm testbase GC tests Message-ID: http://cr.openjdk.java.net/~iignatyev/8199370/webrev.00/index.html > 45710 lines changed: 45710 ins; 0 del; 0 mod; Hi all, could you please review the patch which open sources GC tests from vm testbase? it introduces the following test groups: - vmTestbase_vm_g1classunloading - vmTestbase_vm_gc_compact - vmTestbase_vm_gc_concurrent - vmTestbase_vm_gc_container - vmTestbase_vm_gc_juggle - vmTestbase_vm_gc_locker - vmTestbase_vm_gc_ref - vmTestbase_vm_gc_misc besides these test groups, which split tests by functionality under test, the patch also adds test groups used only for test selection purposes: - vmTestbase_vm_gc which includes all vmTestbase_vm_gc_* test groups - vmTestbase_vm_gc_quick -- "quick" tests from vmTestbase_vm_gc test group - vmTestbase_largepages which consists of tests which are believed to be good candidate to test largepage. this group is used in our testing w/ external vm flags and/or on pre-configured hosts. As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. JBS: https://bugs.openjdk.java.net/browse/JDK-8199370 webrev: http://cr.openjdk.java.net/~iignatyev/8199370/webrev.00/index.html Thanks, -- Igor From erik.joelsson at oracle.com Mon May 7 22:46:14 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 7 May 2018 15:46:14 -0700 Subject: RFR(L) : 8199370: [TESTBUG] Open source vm testbase GC tests In-Reply-To: References: Message-ID: <53e5b90f-0b4f-3f71-5615-ee87c97f97e7@oracle.com> Build change looks good. /Erik On 2018-05-07 15:35, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8199370/webrev.00/index.html >> 45710 lines changed: 45710 ins; 0 del; 0 mod; > Hi all, > > could you please review the patch which open sources GC tests from vm testbase? it introduces the following test groups: > - vmTestbase_vm_g1classunloading > - vmTestbase_vm_gc_compact > - vmTestbase_vm_gc_concurrent > - vmTestbase_vm_gc_container > - vmTestbase_vm_gc_juggle > - vmTestbase_vm_gc_locker > - vmTestbase_vm_gc_ref > - vmTestbase_vm_gc_misc > > besides these test groups, which split tests by functionality under test, the patch also adds test groups used only for test selection purposes: > - vmTestbase_vm_gc which includes all vmTestbase_vm_gc_* test groups > - vmTestbase_vm_gc_quick -- "quick" tests from vmTestbase_vm_gc test group > - vmTestbase_largepages which consists of tests which are believed to be good candidate to test largepage. this group is used in our testing w/ external vm flags and/or on pre-configured hosts. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199370 > webrev: http://cr.openjdk.java.net/~iignatyev/8199370/webrev.00/index.html > > Thanks, > -- Igor From vladimir.x.ivanov at oracle.com Tue May 8 00:45:39 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 7 May 2018 17:45:39 -0700 Subject: ResolvedMethodName::vmholder unused? In-Reply-To: References: Message-ID: <14655796-b874-606a-add4-5a93d52ccae6@oracle.com> Ioi, Though the field is never accessed directly, ResolvedMethodName::vmholder is still tracked by GC and keeps the metadata it accompanies (RMN::vmtarget) alive until RMN is reachable. Best regards, Vladimir Ivanov On 5/7/18 17:29, Ioi Lam wrote: > I don't see anywhere in HotSpot that uses > java_lang_invoke_ResolvedMethodName::_vmholder_offset, which is declared > here: > > http://hg.openjdk.java.net/jdk/jdk/file/7444101401b2/src/hotspot/share/classfile/javaClasses.hpp#l1057 > > http://hg.openjdk.java.net/jdk/jdk/file/9608f7f41c4e/src/java.base/share/classes/java/lang/invoke/MemberName.java#l75 > > > I tried commenting out the initialization of this field and was able to > run a simple Lambda test. > > diff -r 9255cb73f048 src/hotspot/share/classfile/javaClasses.cpp > --- a/src/hotspot/share/classfile/javaClasses.cpp??? Mon May 07 15:29:31 > 2018 -0700 > +++ b/src/hotspot/share/classfile/javaClasses.cpp??? Mon May 07 17:27:27 > 2018 -0700 > @@ -3808,7 +3808,7 @@ > ???? // Add a reference to the loader (actually mirror because > anonymous classes will not have > ???? // distinct loaders) to ensure the metadata is kept alive. > ???? // This mirror may be different than the one in clazz field. > -??? new_resolved_method->obj_field_put(_vmholder_offset, > m->method_holder()->java_mirror()); > +??? //new_resolved_method->obj_field_put(_vmholder_offset, > m->method_holder()->java_mirror()); > ???? resolved_method = ResolvedMethodTable::add_method(Handle(THREAD, > new_resolved_method)); > ?? } > > > Any plans to use vmholder in the future? Or, is this used by any > non-HotSpot VM? > > If no one uses it, I'll file an RFE to remove it, so we can save a > pointer per MemberName. > > Thanks > - Ioi > > > > _______________________________________________ > mlvm-dev mailing list > mlvm-dev at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev From coleen.phillimore at oracle.com Tue May 8 00:52:46 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 7 May 2018 20:52:46 -0400 Subject: ResolvedMethodName::vmholder unused? In-Reply-To: <14655796-b874-606a-add4-5a93d52ccae6@oracle.com> References: <14655796-b874-606a-add4-5a93d52ccae6@oracle.com> Message-ID: <8bdb329a-ce5f-5bfc-7e58-44b8d973f401@oracle.com> The vmholder field is the owner (mirror) of the Method* that is stored in the ResolvedMethodName.? If all references to the mirror go away, then the Method* memory in metaspace will be reclaimed. Unfortunately for a special case, the clazz field of MemberName is not the holder of the method.? Unless this has changed. ? java_lang_invoke_MemberName::set_clazz? (mname_oop, m_klass->java_mirror()); m_klass for an interface for vtable_call is java.lang.Object so would not keep the Method* from being reclaimed. ??? // Add a reference to the loader (actually mirror because anonymous classes will not have ??? // distinct loaders) to ensure the metadata is kept alive. ??? // This mirror may be different than the one in clazz field. ??? new_resolved_method->obj_field_put(_vmholder_offset, m->method_holder()->java_mirror()); ??? resolved_method = ResolvedMethodTable::add_method(Handle(THREAD, new_resolved_method)); Thanks, Coleen On 5/7/18 8:45 PM, Vladimir Ivanov wrote: > Ioi, > > Though the field is never accessed directly, > ResolvedMethodName::vmholder is still tracked by GC and keeps the > metadata it accompanies (RMN::vmtarget) alive until RMN is reachable. > > Best regards, > Vladimir Ivanov > > On 5/7/18 17:29, Ioi Lam wrote: >> I don't see anywhere in HotSpot that uses >> java_lang_invoke_ResolvedMethodName::_vmholder_offset, which is >> declared here: >> >> http://hg.openjdk.java.net/jdk/jdk/file/7444101401b2/src/hotspot/share/classfile/javaClasses.hpp#l1057 >> >> http://hg.openjdk.java.net/jdk/jdk/file/9608f7f41c4e/src/java.base/share/classes/java/lang/invoke/MemberName.java#l75 >> >> >> I tried commenting out the initialization of this field and was able >> to run a simple Lambda test. >> >> diff -r 9255cb73f048 src/hotspot/share/classfile/javaClasses.cpp >> --- a/src/hotspot/share/classfile/javaClasses.cpp??? Mon May 07 >> 15:29:31 2018 -0700 >> +++ b/src/hotspot/share/classfile/javaClasses.cpp??? Mon May 07 >> 17:27:27 2018 -0700 >> @@ -3808,7 +3808,7 @@ >> ????? // Add a reference to the loader (actually mirror because >> anonymous classes will not have >> ????? // distinct loaders) to ensure the metadata is kept alive. >> ????? // This mirror may be different than the one in clazz field. >> -??? new_resolved_method->obj_field_put(_vmholder_offset, >> m->method_holder()->java_mirror()); >> +??? //new_resolved_method->obj_field_put(_vmholder_offset, >> m->method_holder()->java_mirror()); >> ????? resolved_method = >> ResolvedMethodTable::add_method(Handle(THREAD, new_resolved_method)); >> ??? } >> >> >> Any plans to use vmholder in the future? Or, is this used by any >> non-HotSpot VM? >> >> If no one uses it, I'll file an RFE to remove it, so we can save a >> pointer per MemberName. >> >> Thanks >> - Ioi >> >> >> >> _______________________________________________ >> mlvm-dev mailing list >> mlvm-dev at openjdk.java.net >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev From kim.barrett at oracle.com Tue May 8 01:02:01 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 7 May 2018 21:02:01 -0400 Subject: RFR: 8202631: JVM_Clone to throw CloneNotSupportException for Reference object Message-ID: <3509AB48-4DF3-4629-B98F-94CB2E239A54@oracle.com> Please review this change to JVM_Clone to throw CloneNotSupportedException for instances of java.lang.ref.Reference. Per JDK-8201793, Reference and classes derived from it should not support copying via Object.clone. This is a belt-and-suspenders augmentation of that change. This change also reverts most of the earlier attempt to support cloning references. The check for Reference in Klass::set_is_cloneable to prevent setting its JVM_ACC_IS_CLONEABLE_FAST flag has been retained, to further ensure a call can't be intrinsified. This change also removes sun_misc_Cleaner_signature; see JDK-8194804. CRs: https://bugs.openjdk.java.net/browse/JDK-8202631 https://bugs.openjdk.java.net/browse/JDK-8194804 Webrev: http://cr.openjdk.java.net/~kbarrett/8202631/open.00/ Testing: {jdk,hs}-tier{1,2,3} on all Oracle-supported platforms. Manually executed test from JDK-8201793, with and without that change. From jcbeyler at google.com Tue May 8 01:10:53 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 08 May 2018 01:10:53 +0000 Subject: RFR 8171119: Low-Overhead Heap Profiling Message-ID: Hi all, With the awesome help of Serguei Spitsyn, we have moved forward on the implementation for JEP-331 and have the following webrev for review: Webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.18/ Bug: https://bugs.openjdk.java.net/browse/JDK-8171119 It is based on jdk/jdk so should patch well with a recent tip. Could we please have some reviews for the webrev? It would be greatly appreciated! Thanks for all your help! Jc From vladimir.kozlov at oracle.com Tue May 8 01:31:44 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 7 May 2018 18:31:44 -0700 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: References: Message-ID: <2498bec9-9433-8ea1-36ee-a36105696418@oracle.com> I did not look on JVMTI part and tests. It looks good to me. Where _thread field is used? Thanks, Vladimir On 5/7/18 6:10 PM, JC Beyler wrote: > Hi all, > > With the awesome help of Serguei Spitsyn, we have moved forward on the > implementation for JEP-331 and have the following webrev for review: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.18/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8171119 > > It is based on jdk/jdk so should patch well with a recent tip. > > Could we please have some reviews for the webrev? It would be greatly > appreciated! > > Thanks for all your help! > Jc > From erik.osterlund at oracle.com Tue May 8 01:41:17 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Tue, 8 May 2018 03:41:17 +0200 Subject: RFR: 8202631: JVM_Clone to throw CloneNotSupportException for Reference object In-Reply-To: <3509AB48-4DF3-4629-B98F-94CB2E239A54@oracle.com> References: <3509AB48-4DF3-4629-B98F-94CB2E239A54@oracle.com> Message-ID: <83F9ABFE-221A-42D8-95BE-379FB2CCE895@oracle.com> Hi Kim, Looks good. Thanks, /Erik > On 8 May 2018, at 03:02, Kim Barrett wrote: > > Please review this change to JVM_Clone to throw > CloneNotSupportedException for instances of java.lang.ref.Reference. > > Per JDK-8201793, Reference and classes derived from it should not > support copying via Object.clone. This is a belt-and-suspenders > augmentation of that change. > > This change also reverts most of the earlier attempt to support > cloning references. The check for Reference in Klass::set_is_cloneable > to prevent setting its JVM_ACC_IS_CLONEABLE_FAST flag has been retained, > to further ensure a call can't be intrinsified. > > This change also removes sun_misc_Cleaner_signature; see JDK-8194804. > > CRs: > https://bugs.openjdk.java.net/browse/JDK-8202631 > https://bugs.openjdk.java.net/browse/JDK-8194804 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8202631/open.00/ > > Testing: > {jdk,hs}-tier{1,2,3} on all Oracle-supported platforms. > Manually executed test from JDK-8201793, with and without that change. > From poweruserm at live.com.au Tue May 8 01:42:25 2018 From: poweruserm at live.com.au (A Z) Date: Tue, 8 May 2018 01:42:25 +0000 Subject: Question about Runtime relevant Java JEP. Message-ID: I was asking to know when JEP 306 will shift to 'targeted', or what thought and movements in this direction there are. It was mentioned elsewhere that this is only relevant to x86 older processors, but this is not true. Implementation of JEP 306 will eliminate arithmetic overflow and underflow via arithmetic, within-range arithmetic upon double and float, a change which is relevant to everyone. It would also ameliorate "naive" implementation of Java libraries. An example is the remaining underflow and overflow still possible with javax.vecmath.Vector3d.cross given the nature of the sourcecode on this method, and anything else similar. ? From ioi.lam at oracle.com Tue May 8 01:55:11 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 7 May 2018 18:55:11 -0700 Subject: ResolvedMethodName::vmholder unused? In-Reply-To: <14655796-b874-606a-add4-5a93d52ccae6@oracle.com> References: <14655796-b874-606a-add4-5a93d52ccae6@oracle.com> Message-ID: <4b981eb0-ea78-8e60-76e1-a5160a312330@oracle.com> Hi Vladimir, Where does the GC tracks the vmholder? I did a grep of 'vmholder' in all the source code and I can't find anything. Thanks - Ioi On 5/7/18 5:45 PM, Vladimir Ivanov wrote: > Ioi, > > Though the field is never accessed directly, > ResolvedMethodName::vmholder is still tracked by GC and keeps the > metadata it accompanies (RMN::vmtarget) alive until RMN is reachable. > > Best regards, > Vladimir Ivanov > > On 5/7/18 17:29, Ioi Lam wrote: >> I don't see anywhere in HotSpot that uses >> java_lang_invoke_ResolvedMethodName::_vmholder_offset, which is >> declared here: >> >> http://hg.openjdk.java.net/jdk/jdk/file/7444101401b2/src/hotspot/share/classfile/javaClasses.hpp#l1057 >> >> http://hg.openjdk.java.net/jdk/jdk/file/9608f7f41c4e/src/java.base/share/classes/java/lang/invoke/MemberName.java#l75 >> >> >> I tried commenting out the initialization of this field and was able >> to run a simple Lambda test. >> >> diff -r 9255cb73f048 src/hotspot/share/classfile/javaClasses.cpp >> --- a/src/hotspot/share/classfile/javaClasses.cpp??? Mon May 07 >> 15:29:31 2018 -0700 >> +++ b/src/hotspot/share/classfile/javaClasses.cpp??? Mon May 07 >> 17:27:27 2018 -0700 >> @@ -3808,7 +3808,7 @@ >> ????? // Add a reference to the loader (actually mirror because >> anonymous classes will not have >> ????? // distinct loaders) to ensure the metadata is kept alive. >> ????? // This mirror may be different than the one in clazz field. >> -??? new_resolved_method->obj_field_put(_vmholder_offset, >> m->method_holder()->java_mirror()); >> +??? //new_resolved_method->obj_field_put(_vmholder_offset, >> m->method_holder()->java_mirror()); >> ????? resolved_method = >> ResolvedMethodTable::add_method(Handle(THREAD, new_resolved_method)); >> ??? } >> >> >> Any plans to use vmholder in the future? Or, is this used by any >> non-HotSpot VM? >> >> If no one uses it, I'll file an RFE to remove it, so we can save a >> pointer per MemberName. >> >> Thanks >> - Ioi >> >> >> >> _______________________________________________ >> mlvm-dev mailing list >> mlvm-dev at openjdk.java.net >> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev From vladimir.x.ivanov at oracle.com Tue May 8 02:13:40 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 7 May 2018 19:13:40 -0700 Subject: ResolvedMethodName::vmholder unused? In-Reply-To: <4b981eb0-ea78-8e60-76e1-a5160a312330@oracle.com> References: <14655796-b874-606a-add4-5a93d52ccae6@oracle.com> <4b981eb0-ea78-8e60-76e1-a5160a312330@oracle.com> Message-ID: > Where does the GC tracks the vmholder? I did a grep of 'vmholder' in all > the source code and I can't find anything. There's a notion of injected fields in HotSpot which aren't present in class declaration, but present in the class at runtime (for details see JavaClasses::get_injected() called from ClassFileParser::parse_fields()). vmholder is an object field injected into ResolvedMethodName class. It's an ordinary object field, so there's no additional GC support needed to support it. Best regards, Vladimir Ivanov > > Thanks > > - Ioi > > > On 5/7/18 5:45 PM, Vladimir Ivanov wrote: >> Ioi, >> >> Though the field is never accessed directly, >> ResolvedMethodName::vmholder is still tracked by GC and keeps the >> metadata it accompanies (RMN::vmtarget) alive until RMN is reachable. >> >> Best regards, >> Vladimir Ivanov >> >> On 5/7/18 17:29, Ioi Lam wrote: >>> I don't see anywhere in HotSpot that uses >>> java_lang_invoke_ResolvedMethodName::_vmholder_offset, which is >>> declared here: >>> >>> http://hg.openjdk.java.net/jdk/jdk/file/7444101401b2/src/hotspot/share/classfile/javaClasses.hpp#l1057 >>> >>> http://hg.openjdk.java.net/jdk/jdk/file/9608f7f41c4e/src/java.base/share/classes/java/lang/invoke/MemberName.java#l75 >>> >>> >>> I tried commenting out the initialization of this field and was able >>> to run a simple Lambda test. >>> >>> diff -r 9255cb73f048 src/hotspot/share/classfile/javaClasses.cpp >>> --- a/src/hotspot/share/classfile/javaClasses.cpp??? Mon May 07 >>> 15:29:31 2018 -0700 >>> +++ b/src/hotspot/share/classfile/javaClasses.cpp??? Mon May 07 >>> 17:27:27 2018 -0700 >>> @@ -3808,7 +3808,7 @@ >>> ????? // Add a reference to the loader (actually mirror because >>> anonymous classes will not have >>> ????? // distinct loaders) to ensure the metadata is kept alive. >>> ????? // This mirror may be different than the one in clazz field. >>> -??? new_resolved_method->obj_field_put(_vmholder_offset, >>> m->method_holder()->java_mirror()); >>> +??? //new_resolved_method->obj_field_put(_vmholder_offset, >>> m->method_holder()->java_mirror()); >>> ????? resolved_method = >>> ResolvedMethodTable::add_method(Handle(THREAD, new_resolved_method)); >>> ??? } >>> >>> >>> Any plans to use vmholder in the future? Or, is this used by any >>> non-HotSpot VM? >>> >>> If no one uses it, I'll file an RFE to remove it, so we can save a >>> pointer per MemberName. >>> >>> Thanks >>> - Ioi >>> >>> >>> >>> _______________________________________________ >>> mlvm-dev mailing list >>> mlvm-dev at openjdk.java.net >>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev > From mandy.chung at oracle.com Tue May 8 02:19:28 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 7 May 2018 19:19:28 -0700 Subject: RFR: 8202631: JVM_Clone to throw CloneNotSupportException for Reference object In-Reply-To: <3509AB48-4DF3-4629-B98F-94CB2E239A54@oracle.com> References: <3509AB48-4DF3-4629-B98F-94CB2E239A54@oracle.com> Message-ID: On 5/7/18 6:02 PM, Kim Barrett wrote: > Please review this change to JVM_Clone to throw > CloneNotSupportedException for instances of java.lang.ref.Reference. > > Per JDK-8201793, Reference and classes derived from it should not > support copying via Object.clone. This is a belt-and-suspenders > augmentation of that change. > > This change also reverts most of the earlier attempt to support > cloning references. The check for Reference in Klass::set_is_cloneable > to prevent setting its JVM_ACC_IS_CLONEABLE_FAST flag has been retained, > to further ensure a call can't be intrinsified. > > This change also removes sun_misc_Cleaner_signature; see JDK-8194804. > > CRs: > https://bugs.openjdk.java.net/browse/JDK-8202631 > https://bugs.openjdk.java.net/browse/JDK-8194804 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8202631/open.00/ > Looks good. Mandy From kim.barrett at oracle.com Tue May 8 02:25:24 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 7 May 2018 22:25:24 -0400 Subject: RFR: 8202631: JVM_Clone to throw CloneNotSupportException for Reference object In-Reply-To: <83F9ABFE-221A-42D8-95BE-379FB2CCE895@oracle.com> References: <3509AB48-4DF3-4629-B98F-94CB2E239A54@oracle.com> <83F9ABFE-221A-42D8-95BE-379FB2CCE895@oracle.com> Message-ID: <5C9376E9-A2F9-4021-8DDF-9F2C853C55AE@oracle.com> > On May 7, 2018, at 9:41 PM, Erik Osterlund wrote: > > Hi Kim, > > Looks good. Thanks. > > Thanks, > /Erik > >> On 8 May 2018, at 03:02, Kim Barrett wrote: >> >> Please review this change to JVM_Clone to throw >> CloneNotSupportedException for instances of java.lang.ref.Reference. >> >> Per JDK-8201793, Reference and classes derived from it should not >> support copying via Object.clone. This is a belt-and-suspenders >> augmentation of that change. >> >> This change also reverts most of the earlier attempt to support >> cloning references. The check for Reference in Klass::set_is_cloneable >> to prevent setting its JVM_ACC_IS_CLONEABLE_FAST flag has been retained, >> to further ensure a call can't be intrinsified. >> >> This change also removes sun_misc_Cleaner_signature; see JDK-8194804. >> >> CRs: >> https://bugs.openjdk.java.net/browse/JDK-8202631 >> https://bugs.openjdk.java.net/browse/JDK-8194804 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8202631/open.00/ >> >> Testing: >> {jdk,hs}-tier{1,2,3} on all Oracle-supported platforms. >> Manually executed test from JDK-8201793, with and without that change. From kim.barrett at oracle.com Tue May 8 02:25:34 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 7 May 2018 22:25:34 -0400 Subject: RFR: 8202631: JVM_Clone to throw CloneNotSupportException for Reference object In-Reply-To: References: <3509AB48-4DF3-4629-B98F-94CB2E239A54@oracle.com> Message-ID: <994C43D0-B487-42C0-8431-1EA6BBFDC291@oracle.com> > On May 7, 2018, at 10:19 PM, mandy chung wrote: > > > > On 5/7/18 6:02 PM, Kim Barrett wrote: >> Please review this change to JVM_Clone to throw >> CloneNotSupportedException for instances of java.lang.ref.Reference. >> >> Per JDK-8201793, Reference and classes derived from it should not >> support copying via Object.clone. This is a belt-and-suspenders >> augmentation of that change. >> >> This change also reverts most of the earlier attempt to support >> cloning references. The check for Reference in Klass::set_is_cloneable >> to prevent setting its JVM_ACC_IS_CLONEABLE_FAST flag has been retained, >> to further ensure a call can't be intrinsified. >> >> This change also removes sun_misc_Cleaner_signature; see JDK-8194804. >> >> CRs: >> >> https://bugs.openjdk.java.net/browse/JDK-8202631 >> https://bugs.openjdk.java.net/browse/JDK-8194804 >> >> >> Webrev: >> >> http://cr.openjdk.java.net/~kbarrett/8202631/open.00/ >> >> >> > > Looks good. > > Mandy Thanks. From david.holmes at oracle.com Tue May 8 02:27:17 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 8 May 2018 12:27:17 +1000 Subject: RFR: 8202631: JVM_Clone to throw CloneNotSupportException for Reference object In-Reply-To: <3509AB48-4DF3-4629-B98F-94CB2E239A54@oracle.com> References: <3509AB48-4DF3-4629-B98F-94CB2E239A54@oracle.com> Message-ID: Looks good! Hadn't realized we had so much supporting code for this! Thanks, David On 8/05/2018 11:02 AM, Kim Barrett wrote: > Please review this change to JVM_Clone to throw > CloneNotSupportedException for instances of java.lang.ref.Reference. > > Per JDK-8201793, Reference and classes derived from it should not > support copying via Object.clone. This is a belt-and-suspenders > augmentation of that change. > > This change also reverts most of the earlier attempt to support > cloning references. The check for Reference in Klass::set_is_cloneable > to prevent setting its JVM_ACC_IS_CLONEABLE_FAST flag has been retained, > to further ensure a call can't be intrinsified. > > This change also removes sun_misc_Cleaner_signature; see JDK-8194804. > > CRs: > https://bugs.openjdk.java.net/browse/JDK-8202631 > https://bugs.openjdk.java.net/browse/JDK-8194804 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8202631/open.00/ > > Testing: > {jdk,hs}-tier{1,2,3} on all Oracle-supported platforms. > Manually executed test from JDK-8201793, with and without that change. > From jcbeyler at google.com Tue May 8 02:28:46 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 08 May 2018 02:28:46 +0000 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: <2498bec9-9433-8ea1-36ee-a36105696418@oracle.com> References: <2498bec9-9433-8ea1-36ee-a36105696418@oracle.com> Message-ID: Hi Vladimir, Good catch, I believe it was used before but no longer since we put the heap sampler information directly in the thread structure. I removed it for the next webrev. Could anyone do a review on the JVMTI parts and tests? Thanks a lot for your help! Jc On Mon, May 7, 2018 at 6:31 PM Vladimir Kozlov wrote: > I did not look on JVMTI part and tests. It looks good to me. > > Where _thread field is used? > > Thanks, > Vladimir > > On 5/7/18 6:10 PM, JC Beyler wrote: > > Hi all, > > > > With the awesome help of Serguei Spitsyn, we have moved forward on the > > implementation for JEP-331 and have the following webrev for review: > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.18/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8171119 > > > > It is based on jdk/jdk so should patch well with a recent tip. > > > > Could we please have some reviews for the webrev? It would be greatly > > appreciated! > > > > Thanks for all your help! > > Jc > > > From serguei.spitsyn at oracle.com Tue May 8 03:49:00 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 7 May 2018 20:49:00 -0700 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: References: <2498bec9-9433-8ea1-36ee-a36105696418@oracle.com> Message-ID: <37fda46d-89ae-36d1-3a54-70c56ee3ca2f@oracle.com> Hi Jc, I'll make one more pass through the JVMTI and test fixes. However, it would be good if at least one more pair of eyes looked at the tests as they are a big part of the fix. Thanks, Serguei On 5/7/18 19:28, JC Beyler wrote: > Hi Vladimir, > > Good catch, I believe it was used before but no longer since we put the > heap sampler information directly in the thread structure. I removed it for > the next webrev. > > Could anyone do a review on the JVMTI parts and tests? > > Thanks a lot for your help! > Jc > > On Mon, May 7, 2018 at 6:31 PM Vladimir Kozlov > wrote: > >> I did not look on JVMTI part and tests. It looks good to me. >> >> Where _thread field is used? >> >> Thanks, >> Vladimir >> >> On 5/7/18 6:10 PM, JC Beyler wrote: >>> Hi all, >>> >>> With the awesome help of Serguei Spitsyn, we have moved forward on the >>> implementation for JEP-331 and have the following webrev for review: >>> >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.18/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8171119 >>> >>> It is based on jdk/jdk so should patch well with a recent tip. >>> >>> Could we please have some reviews for the webrev? It would be greatly >>> appreciated! >>> >>> Thanks for all your help! >>> Jc >>> From kim.barrett at oracle.com Tue May 8 05:46:00 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 8 May 2018 01:46:00 -0400 Subject: RFR: 8202631: JVM_Clone to throw CloneNotSupportException for Reference object In-Reply-To: References: <3509AB48-4DF3-4629-B98F-94CB2E239A54@oracle.com> Message-ID: <6F0FF529-8885-47E6-A2C0-B0ED8025892A@oracle.com> > On May 7, 2018, at 10:27 PM, David Holmes wrote: > > Looks good! Thanks. From erik.helin at oracle.com Tue May 8 06:08:28 2018 From: erik.helin at oracle.com (Erik Helin) Date: Tue, 8 May 2018 08:08:28 +0200 Subject: RFR: 8202709: Move oopDesc::is_archive_object to MetaspaceShared::is_archive_object In-Reply-To: <51a65446-5d14-5877-dc12-0b425ca8afe6@oracle.com> References: <51a65446-5d14-5877-dc12-0b425ca8afe6@oracle.com> Message-ID: <10fd2def-e474-a805-9334-0b4ebf6539f1@oracle.com> On 05/07/2018 01:37 PM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to move oopDesc::is_archive_object to > MetaspaceShared::is_archive_object. > > http://cr.openjdk.java.net/~stefank/8202709/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8202709 Looks good to me, Reviewed. Before you push, would you mind removing the two extra newlines you added prior to the #endif in metaspaceShared.hpp? I'm guessing that they weren't meant to be part of the patch :) Thanks, Erik > The motivation for this patch is to move dependencies to G1 out of > oopDesc. I've placed the functions in MetaspaceShared, within > INCLUDE_CDS_JAVA_HEAP guards, where related functions G1 Archive > functions can be found. > > Thanks, > StefanK From stefan.karlsson at oracle.com Tue May 8 07:50:37 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 8 May 2018 09:50:37 +0200 Subject: RFR: 8202709: Move oopDesc::is_archive_object to MetaspaceShared::is_archive_object In-Reply-To: <71B7A275-43DF-4B84-889E-9D63540BD62B@oracle.com> References: <51a65446-5d14-5877-dc12-0b425ca8afe6@oracle.com> <71B7A275-43DF-4B84-889E-9D63540BD62B@oracle.com> Message-ID: Thanks, Jiangli. StefanK On 2018-05-07 18:01, Jiangli Zhou wrote: > Hi Stefan, > > Looks good. > > Thanks, > > Jiangli > >> On May 7, 2018, at 4:37 AM, Stefan Karlsson wrote: >> >> Hi all, >> >> Please review this patch to move oopDesc::is_archive_object to MetaspaceShared::is_archive_object. >> >> http://cr.openjdk.java.net/~stefank/8202709/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8202709 >> >> The motivation for this patch is to move dependencies to G1 out of oopDesc. I've placed the functions in MetaspaceShared, within INCLUDE_CDS_JAVA_HEAP guards, where related functions G1 Archive functions can be found. >> >> Thanks, >> StefanK > From stefan.karlsson at oracle.com Tue May 8 07:51:20 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 8 May 2018 09:51:20 +0200 Subject: RFR: 8202709: Move oopDesc::is_archive_object to MetaspaceShared::is_archive_object In-Reply-To: <10fd2def-e474-a805-9334-0b4ebf6539f1@oracle.com> References: <51a65446-5d14-5877-dc12-0b425ca8afe6@oracle.com> <10fd2def-e474-a805-9334-0b4ebf6539f1@oracle.com> Message-ID: <88691b19-beb8-9a8d-8400-351e9030e15a@oracle.com> On 2018-05-08 08:08, Erik Helin wrote: > On 05/07/2018 01:37 PM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to move oopDesc::is_archive_object to >> MetaspaceShared::is_archive_object. >> >> http://cr.openjdk.java.net/~stefank/8202709/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8202709 > > Looks good to me, Reviewed. Before you push, would you mind removing the > two extra newlines you added prior to the #endif in metaspaceShared.hpp? > I'm guessing that they weren't meant to be part of the patch :) I'll remove them. Thanks for reviewing. StefanK > > Thanks, > Erik > >> The motivation for this patch is to move dependencies to G1 out of >> oopDesc. I've placed the functions in MetaspaceShared, within >> INCLUDE_CDS_JAVA_HEAP guards, where related functions G1 Archive >> functions can be found. >> >> Thanks, >> StefanK From per.liden at oracle.com Tue May 8 07:57:13 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 8 May 2018 09:57:13 +0200 Subject: RFR: 8202649: Move the Parallel GC specific task creation functions out of Threads In-Reply-To: References: Message-ID: <4cffc989-392e-c815-f0c5-8afb72c938f0@oracle.com> On 05/07/2018 04:02 PM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to move the Parallel GC specific task creation > functions out of Threads. > > http://cr.openjdk.java.net/~stefank/8202649/webrev.01/ Looks good! /Per > > There exist (at least) two available functions to iterate over a subset > of threads: > ?static void non_java_threads_do(ThreadClosure* tc); > ?static void threads_do(ThreadClosure* tc); > > This patch extends this set and adds: > ?static void java_threads_do(ThreadClosure* tc); > ?static void java_threads_and_vm_thread_do(ThreadClosure* tc); > > The latter is used to implement the create_thread_roots_tasks and > create_thread_roots_marking_tasks inside the Parallel GC code. > > Thanks, > StefanK From stefan.karlsson at oracle.com Tue May 8 07:56:33 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 8 May 2018 09:56:33 +0200 Subject: RFR: 8202649: Move the Parallel GC specific task creation functions out of Threads In-Reply-To: <4cffc989-392e-c815-f0c5-8afb72c938f0@oracle.com> References: <4cffc989-392e-c815-f0c5-8afb72c938f0@oracle.com> Message-ID: <6194d2c5-7279-6f6d-f782-c486e9454459@oracle.com> Thanks, Per. StefanK On 2018-05-08 09:57, Per Liden wrote: > > On 05/07/2018 04:02 PM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to move the Parallel GC specific task >> creation functions out of Threads. >> >> http://cr.openjdk.java.net/~stefank/8202649/webrev.01/ > > Looks good! > > /Per > >> >> There exist (at least) two available functions to iterate over a >> subset of threads: >> ??static void non_java_threads_do(ThreadClosure* tc); >> ??static void threads_do(ThreadClosure* tc); >> >> This patch extends this set and adds: >> ??static void java_threads_do(ThreadClosure* tc); >> ??static void java_threads_and_vm_thread_do(ThreadClosure* tc); >> >> The latter is used to implement the create_thread_roots_tasks and >> create_thread_roots_marking_tasks inside the Parallel GC code. >> >> Thanks, >> StefanK From shade at redhat.com Tue May 8 08:25:39 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 8 May 2018 10:25:39 +0200 Subject: RFR [8u] 8165489 (S): Missing G1 barrier in Unsafe_GetObjectVolatile In-Reply-To: <1525696653.7127.15.camel@oracle.com> References: <19ca5b19-d458-b37f-1a7c-7404a027306a@redhat.com> <1525686887.7127.7.camel@oracle.com> <1525696653.7127.15.camel@oracle.com> Message-ID: <206fdc15-b8b1-83fc-0063-f95bb0ca409f@redhat.com> On 05/07/2018 02:37 PM, Thomas Schatzl wrote: > On Mon, 2018-05-07 at 11:54 +0200, Thomas Schatzl wrote: >> Hi, >> >> On Wed, 2018-04-25 at 12:34 +0200, Aleksey Shipilev wrote: >>> (trying again on hotspot-dev@) >>> >>> Hi, >>> >>> Please review this this backport/fix for 8u. This improves G1 >>> [...] >>> >>> JDK 9 bug: >>> https://bugs.openjdk.java.net/browse/JDK-8165489 >>> >>> JDK 9 fix: >>> http://hg.openjdk.java.net/jdk9/jdk9/hotspot/rev/a696583f5ddb >>> >>> JDK 8u fix: >>> http://cr.openjdk.java.net/~shade/8165489-8u/webrev.01/ >>> >>> Testing: x86_64 build, Shenandoah tests >> >> I think this is good. Let me run it through jprt, I will report >> back later. > > jprt run passed. Thank you, Thomas! I'll reference this in RFA thread on 8u-dev. -Aleksey From magnus.ihse.bursie at oracle.com Tue May 8 09:06:15 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 8 May 2018 11:06:15 +0200 Subject: RFR(L) : 8199370: [TESTBUG] Open source vm testbase GC tests In-Reply-To: References: Message-ID: <0fecb84f-9d8b-8056-ee01-6e16c5e9467a@oracle.com> On 2018-05-08 00:35, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8199370/webrev.00/index.html >> 45710 lines changed: 45710 ins; 0 del; 0 mod; > Hi all, > > could you please review the patch which open sources GC tests from vm testbase? it introduces the following test groups: > - vmTestbase_vm_g1classunloading > - vmTestbase_vm_gc_compact > - vmTestbase_vm_gc_concurrent > - vmTestbase_vm_gc_container > - vmTestbase_vm_gc_juggle > - vmTestbase_vm_gc_locker > - vmTestbase_vm_gc_ref > - vmTestbase_vm_gc_misc > > besides these test groups, which split tests by functionality under test, the patch also adds test groups used only for test selection purposes: > - vmTestbase_vm_gc which includes all vmTestbase_vm_gc_* test groups > - vmTestbase_vm_gc_quick -- "quick" tests from vmTestbase_vm_gc test group > - vmTestbase_largepages which consists of tests which are believed to be good candidate to test largepage. this group is used in our testing w/ external vm flags and/or on pre-configured hosts. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199370 > webrev: http://cr.openjdk.java.net/~iignatyev/8199370/webrev.00/index.html Build changes look good. /Magnus > > Thanks, > -- Igor From sgehwolf at redhat.com Tue May 8 09:19:16 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Tue, 08 May 2018 11:19:16 +0200 Subject: RFA/RFR [10] 8202540: Zero build is broken after JDK-8189871 (Refactor GC barriers to use declarative semantics) In-Reply-To: <40d5c482-c95d-3fcc-9178-79e1cb4a7a8e@redhat.com> References: <54d151db-dec8-8407-db07-4c5d72788754@redhat.com> <40d5c482-c95d-3fcc-9178-79e1cb4a7a8e@redhat.com> Message-ID: <8bf9087a21083fa66af86b57ccbf52c44985e76b.camel@redhat.com> On Tue, 2018-05-08 at 11:09 +0200, Aleksey Shipilev wrote: > jdk10u-fix-yes was granted for the 10u bug. > > Do I have to have the formal ack on this review thread as well? If > so, please ack. I believe since this isn't a straight forward backport from a JDK 11 change it needs to get reviewed by the relevant group. CC'ing hotspot- dev. FWIW, I can confirm that this fixes the Zero fastdebug build on JDK 10u. Thanks, Severin > -Aleksey > > On 05/02/2018 12:10 PM, Aleksey Shipilev wrote: > > This is a bit unconventional process-wise, because there is a fix > > in JDK 11, but it comes with the > > bunch of other unrelated changes that are not backportable to 10u. > > So, I opted to submit a separate > > bug for this. I added the jdk10u-fix-request tag and appropriate > > comment. > > > > 10u bug: > > https://bugs.openjdk.java.net/browse/JDK-8202540 > > > > 11 bug: > > https://bugs.openjdk.java.net/browse/JDK-8199220 > > > > 10u fix: > > > > diff -r e4530ef14c08 > > src/hotspot/cpu/zero/globalDefinitions_zero.hpp > > --- a/src/hotspot/cpu/zero/globalDefinitions_zero.hpp Wed > > Mar 28 14:24:17 2018 +0100 > > +++ b/src/hotspot/cpu/zero/globalDefinitions_zero.hpp Wed > > May 02 12:03:18 2018 +0200 > > @@ -26,6 +26,10 @@ > > #ifndef CPU_ZERO_VM_GLOBALDEFINITIONS_ZERO_HPP > > #define CPU_ZERO_VM_GLOBALDEFINITIONS_ZERO_HPP > > > > +#ifdef _LP64 > > +#define SUPPORTS_NATIVE_CX8 > > +#endif > > + > > #include > > > > // Indicates whether the C calling conventions require that > > > > Testing: x86_64 Zero builds > > > > Thanks, > > -Aleksey > > > > From poweruserm at live.com.au Tue May 8 09:24:14 2018 From: poweruserm at live.com.au (A Z) Date: Tue, 8 May 2018 09:24:14 +0000 Subject: Question about JEP 306 Message-ID: 'I am not sure what you are asking. 'Candidate' means it's considered for inclusion soon. It's not yet 'targeted' which means it is not yet clear/decided if / which JDK version it will land in. I am not familiar with the present state of the floating point semantics.' I was asking to know when JEP 306 will shift to 'targeted', or what things are happening in that direction. It was mentioned elsewhere that this is only relevant to x86 older processors, but this is not true. Implementation of JEP 306 will eliminate arithmetic overflow and underflow via arithmetic, within-range arithmetic upon double and float, a change which is relevant to everyone. It would also ameliorate "naive" implementation of Java libraries. An example is the remaining underflow and overflow still possible with javax.vecmath.Vector3d.cross given the nature of the sourcecode on this method, and anything else similar. Will Jep 306 changes actually go forward? From david.holmes at oracle.com Tue May 8 09:37:14 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 8 May 2018 19:37:14 +1000 Subject: RFA/RFR [10] 8202540: Zero build is broken after JDK-8189871 (Refactor GC barriers to use declarative semantics) In-Reply-To: <8bf9087a21083fa66af86b57ccbf52c44985e76b.camel@redhat.com> References: <54d151db-dec8-8407-db07-4c5d72788754@redhat.com> <40d5c482-c95d-3fcc-9178-79e1cb4a7a8e@redhat.com> <8bf9087a21083fa66af86b57ccbf52c44985e76b.camel@redhat.com> Message-ID: Reviewed. David On 8/05/2018 7:19 PM, Severin Gehwolf wrote: > On Tue, 2018-05-08 at 11:09 +0200, Aleksey Shipilev wrote: >> jdk10u-fix-yes was granted for the 10u bug. >> >> Do I have to have the formal ack on this review thread as well? If >> so, please ack. > > I believe since this isn't a straight forward backport from a JDK 11 > change it needs to get reviewed by the relevant group. CC'ing hotspot- > dev. > > FWIW, I can confirm that this fixes the Zero fastdebug build on JDK > 10u. > > Thanks, > Severin > >> -Aleksey >> >> On 05/02/2018 12:10 PM, Aleksey Shipilev wrote: >>> This is a bit unconventional process-wise, because there is a fix >>> in JDK 11, but it comes with the >>> bunch of other unrelated changes that are not backportable to 10u. >>> So, I opted to submit a separate >>> bug for this. I added the jdk10u-fix-request tag and appropriate >>> comment. >>> >>> 10u bug: >>> https://bugs.openjdk.java.net/browse/JDK-8202540 >>> >>> 11 bug: >>> https://bugs.openjdk.java.net/browse/JDK-8199220 >>> >>> 10u fix: >>> >>> diff -r e4530ef14c08 >>> src/hotspot/cpu/zero/globalDefinitions_zero.hpp >>> --- a/src/hotspot/cpu/zero/globalDefinitions_zero.hpp Wed >>> Mar 28 14:24:17 2018 +0100 >>> +++ b/src/hotspot/cpu/zero/globalDefinitions_zero.hpp Wed >>> May 02 12:03:18 2018 +0200 >>> @@ -26,6 +26,10 @@ >>> #ifndef CPU_ZERO_VM_GLOBALDEFINITIONS_ZERO_HPP >>> #define CPU_ZERO_VM_GLOBALDEFINITIONS_ZERO_HPP >>> >>> +#ifdef _LP64 >>> +#define SUPPORTS_NATIVE_CX8 >>> +#endif >>> + >>> #include >>> >>> // Indicates whether the C calling conventions require that >>> >>> Testing: x86_64 Zero builds >>> >>> Thanks, >>> -Aleksey >>> >> >> From erik.osterlund at oracle.com Tue May 8 10:50:06 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 8 May 2018 12:50:06 +0200 Subject: RFA/RFR [10] 8202540: Zero build is broken after JDK-8189871 (Refactor GC barriers to use declarative semantics) In-Reply-To: <8bf9087a21083fa66af86b57ccbf52c44985e76b.camel@redhat.com> References: <54d151db-dec8-8407-db07-4c5d72788754@redhat.com> <40d5c482-c95d-3fcc-9178-79e1cb4a7a8e@redhat.com> <8bf9087a21083fa66af86b57ccbf52c44985e76b.camel@redhat.com> Message-ID: <5AF180DE.70704@oracle.com> Hi, Looks good. Thanks, /Erik On 2018-05-08 11:19, Severin Gehwolf wrote: > On Tue, 2018-05-08 at 11:09 +0200, Aleksey Shipilev wrote: >> jdk10u-fix-yes was granted for the 10u bug. >> >> Do I have to have the formal ack on this review thread as well? If >> so, please ack. > I believe since this isn't a straight forward backport from a JDK 11 > change it needs to get reviewed by the relevant group. CC'ing hotspot- > dev. > > FWIW, I can confirm that this fixes the Zero fastdebug build on JDK > 10u. > > Thanks, > Severin > >> -Aleksey >> >> On 05/02/2018 12:10 PM, Aleksey Shipilev wrote: >>> This is a bit unconventional process-wise, because there is a fix >>> in JDK 11, but it comes with the >>> bunch of other unrelated changes that are not backportable to 10u. >>> So, I opted to submit a separate >>> bug for this. I added the jdk10u-fix-request tag and appropriate >>> comment. >>> >>> 10u bug: >>> https://bugs.openjdk.java.net/browse/JDK-8202540 >>> >>> 11 bug: >>> https://bugs.openjdk.java.net/browse/JDK-8199220 >>> >>> 10u fix: >>> >>> diff -r e4530ef14c08 >>> src/hotspot/cpu/zero/globalDefinitions_zero.hpp >>> --- a/src/hotspot/cpu/zero/globalDefinitions_zero.hpp Wed >>> Mar 28 14:24:17 2018 +0100 >>> +++ b/src/hotspot/cpu/zero/globalDefinitions_zero.hpp Wed >>> May 02 12:03:18 2018 +0200 >>> @@ -26,6 +26,10 @@ >>> #ifndef CPU_ZERO_VM_GLOBALDEFINITIONS_ZERO_HPP >>> #define CPU_ZERO_VM_GLOBALDEFINITIONS_ZERO_HPP >>> >>> +#ifdef _LP64 >>> +#define SUPPORTS_NATIVE_CX8 >>> +#endif >>> + >>> #include >>> >>> // Indicates whether the C calling conventions require that >>> >>> Testing: x86_64 Zero builds >>> >>> Thanks, >>> -Aleksey >>> >> From jcbeyler at google.com Tue May 8 14:46:09 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 08 May 2018 14:46:09 +0000 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: <37fda46d-89ae-36d1-3a54-70c56ee3ca2f@oracle.com> References: <2498bec9-9433-8ea1-36ee-a36105696418@oracle.com> <37fda46d-89ae-36d1-3a54-70c56ee3ca2f@oracle.com> Message-ID: Hi Serguei, Awesome! Thanks for your help and I look forward to your review! Any takers on the second pair of eyes on the tests? :) Thanks again all, your help is much appreciated, Jc On Mon, May 7, 2018 at 8:49 PM serguei.spitsyn at oracle.com < serguei.spitsyn at oracle.com> wrote: > Hi Jc, > > I'll make one more pass through the JVMTI and test fixes. > However, it would be good if at least one more pair of eyes > looked at the tests as they are a big part of the fix. > > Thanks, > Serguei > > > On 5/7/18 19:28, JC Beyler wrote: > > Hi Vladimir, > > > > Good catch, I believe it was used before but no longer since we put the > > heap sampler information directly in the thread structure. I removed it > for > > the next webrev. > > > > Could anyone do a review on the JVMTI parts and tests? > > > > Thanks a lot for your help! > > Jc > > > > On Mon, May 7, 2018 at 6:31 PM Vladimir Kozlov < > vladimir.kozlov at oracle.com> > > wrote: > > > >> I did not look on JVMTI part and tests. It looks good to me. > >> > >> Where _thread field is used? > >> > >> Thanks, > >> Vladimir > >> > >> On 5/7/18 6:10 PM, JC Beyler wrote: > >>> Hi all, > >>> > >>> With the awesome help of Serguei Spitsyn, we have moved forward on the > >>> implementation for JEP-331 and have the following webrev for review: > >>> > >>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.18/ > >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8171119 > >>> > >>> It is based on jdk/jdk so should patch well with a recent tip. > >>> > >>> Could we please have some reviews for the webrev? It would be greatly > >>> appreciated! > >>> > >>> Thanks for all your help! > >>> Jc > >>> > > From vladimir.kozlov at oracle.com Tue May 8 16:03:39 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 8 May 2018 09:03:39 -0700 Subject: RFR(XS): 8202564: java/lang/management/ThreadMXBean/ThreadCounts.java fails In-Reply-To: <0ceca1748a6e43b79beebd41d633dd12@sap.com> References: <0ceca1748a6e43b79beebd41d633dd12@sap.com> Message-ID: <180175fc-009a-fcaa-5a85-939799c234dc@oracle.com> Hi Martin, Thank you for finding the cause and fixing it. Looks good. Changing to hotspot-dev alias since it is not compiler code. Let other people look on it. Thanks, Vladimir On 5/8/18 6:50 AM, Doerr, Martin wrote: > Hi Vladimir, > > I?ve fixed the thread count issue. > > ?_exiting_daemon_threads_count? just needs to get decremented at the > right place (like ?_exiting_threads_count?). > > Please review: > > http://cr.openjdk.java.net/~mdoerr/8202564_thread_cnt/webrev.00/ > > Best regards, > > Martin > From daniel.daugherty at oracle.com Tue May 8 16:23:58 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 8 May 2018 12:23:58 -0400 Subject: RFR(XS): 8202564: java/lang/management/ThreadMXBean/ThreadCounts.java fails In-Reply-To: <180175fc-009a-fcaa-5a85-939799c234dc@oracle.com> References: <0ceca1748a6e43b79beebd41d633dd12@sap.com> <180175fc-009a-fcaa-5a85-939799c234dc@oracle.com> Message-ID: <693d5169-4f0b-8bdd-1663-899d531ebf1d@oracle.com> > http://cr.openjdk.java.net/~mdoerr/8202564_thread_cnt/webrev.00/ src/hotspot/share/services/threadService.cpp ??? You have moved the decrement of _exiting_daemon_threads_count ??? L125: ? if (daemon) { ??? L126: ??? Atomic::dec(&_exiting_daemon_threads_count); ??? L127: ? } ??? above this check: ??? L129: ? if (thread->is_hidden_from_external_view() || ??? L130: ????? thread->is_jvmti_agent_thread()) { ??? and away from this other decrement: ??? L136: _daemon_threads_count->set_value(_daemon_threads_count->get_value() - 1); ??? That makes sense because the original increment of _exiting_daemon_threads_count ??? happens here: ??? L140 void ThreadService::current_thread_exiting(JavaThread* jt) { ??? L141?? assert(jt == JavaThread::current(), "Called by current thread"); ??? L142?? Atomic::inc(&_exiting_threads_count); ??? L143 ??? L144?? oop threadObj = jt->threadObj(); ??? L145?? if (threadObj != NULL && java_lang_Thread::is_daemon(threadObj)) { ??? L146???? Atomic::inc(&_exiting_daemon_threads_count); ??? L147?? } ??? L148 } ??? and is not guarded by the hidden thread or jvmti check. For other ??? folks that are reading closely, these two checks are equivalent: ????? if (daemon) { ??? and: ????? if (threadObj != NULL && java_lang_Thread::is_daemon(threadObj)) { ??? Very nice catch! I'm just guessing here, but I think the recent CompilerThread changes now allow CompilerThreads to go thru the ThreadService::remove_thread() code path which exposed this bug. So if there was a late query about the number of daemon threads and a CompilerThread had exited, then the count would be off. However, if the CompilerThread exited a little bit later, then the test would pass... Thumbs up! Dan On 5/8/18 12:03 PM, Vladimir Kozlov wrote: > Hi Martin, > > Thank you for finding the cause and fixing it. Looks good. > > Changing to hotspot-dev alias since it is not compiler code. Let other > people look on it. > > Thanks, > Vladimir > > On 5/8/18 6:50 AM, Doerr, Martin wrote: >> Hi Vladimir, >> >> I?ve fixed the thread count issue. >> >> ?_exiting_daemon_threads_count? just needs to get decremented at the >> right place (like ?_exiting_threads_count?). >> >> Please review: >> >> http://cr.openjdk.java.net/~mdoerr/8202564_thread_cnt/webrev.00/ >> >> Best regards, >> >> Martin >> From kim.barrett at oracle.com Tue May 8 18:02:02 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 8 May 2018 14:02:02 -0400 Subject: RFR: 8179887 - Build failure with glibc >= 2.24: error: 'int readdir_r(DIR*, dirent*, dirent**)' is deprecated In-Reply-To: References: <121e71e5-bcc7-d907-ce4c-9fdbd0bbddf0@redhat.com> <403C5242-7255-4BE4-BD3E-A5EEAD278283@oracle.com> <3197d649-2c9e-299c-db40-b6715aea8b92@redhat.com> <2a790c19-7331-0908-7c75-9704720a6a0d@redhat.com> <78a4e259-2ac6-7318-6208-82ed8b35292a@oracle.com> <6AE7B98E-FF1B-43E0-8108-7128707BD5F5@oracle.com> Message-ID: > On May 7, 2018, at 11:20 AM, B. Blaser wrote: > > On 7 May 2018 at 14:19, B. Blaser wrote: >> On 6 May 2018 at 18:35, B. Blaser wrote: >>> On 5 May 2018 at 22:26, Kim Barrett wrote: >>>>> On May 5, 2018, at 8:03 AM, B. Blaser wrote: >>>>> >>>>> On 4 May 2018 at 17:42, Alan Bateman wrote: >>>>>> On 04/05/2018 16:31, B. Blaser wrote: >>>>>>> >>>>>>> : >>>>>>> >>>>>>> I'll create a new JBS issue (unless you want to) since the fix is >>>>>>> mostly located in core-libs and only intended to make the build >>>>>>> complete. >>>>>>> But I'll still need someone to review and push the fix, would you be >>>>>>> interested in doing this? >>>>>>> >>>>>> I think someone needs to explore the behavior on other platforms too as the >>>>>> #ifndef __linux__ patches are ugly. >>>>> >>>>> Yes sure but I cannot test it elsewhere myself... and we can easily >>>>> remove them once POSIX officially deprecates 'readdir_r' or if someone >>>>> validates the fix on other systems. >>>> >>>> I'm not sure anyone has the ability to test on all supported. That >>>> doesn't (and really can't) prevent making changes across platforms to >>>> platform-specific code. It just means one needs to get help with >>>> testing. Make the change across platforms, test on those platforms >>>> one has access to, and ask here for help testing on others. >>> >>> OK, I'll provide a cross-platform fix to be evaluated on other systems too. >> >> Here it is, tier1 is successful on Linux with glibc=2.26. >> Any feedback on other systems would be very helpful. > > Some more cleanup? I?ve created https://bugs.openjdk.java.net/browse/JDK-8202794 to cover this. Suggest switching the subject line. From coleen.phillimore at oracle.com Tue May 8 18:41:42 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 8 May 2018 14:41:42 -0400 Subject: RFR: 8202631: JVM_Clone to throw CloneNotSupportException for Reference object In-Reply-To: <3509AB48-4DF3-4629-B98F-94CB2E239A54@oracle.com> References: <3509AB48-4DF3-4629-B98F-94CB2E239A54@oracle.com> Message-ID: <5ff5d874-425b-2c77-196d-15b3601a172d@oracle.com> Looks good to me also. Coleen On 5/7/18 9:02 PM, Kim Barrett wrote: > Please review this change to JVM_Clone to throw > CloneNotSupportedException for instances of java.lang.ref.Reference. > > Per JDK-8201793, Reference and classes derived from it should not > support copying via Object.clone. This is a belt-and-suspenders > augmentation of that change. > > This change also reverts most of the earlier attempt to support > cloning references. The check for Reference in Klass::set_is_cloneable > to prevent setting its JVM_ACC_IS_CLONEABLE_FAST flag has been retained, > to further ensure a call can't be intrinsified. > > This change also removes sun_misc_Cleaner_signature; see JDK-8194804. > > CRs: > https://bugs.openjdk.java.net/browse/JDK-8202631 > https://bugs.openjdk.java.net/browse/JDK-8194804 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8202631/open.00/ > > Testing: > {jdk,hs}-tier{1,2,3} on all Oracle-supported platforms. > Manually executed test from JDK-8201793, with and without that change. > From igor.ignatyev at oracle.com Tue May 8 19:03:56 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 8 May 2018 12:03:56 -0700 Subject: RFR(L) : 8199370: [TESTBUG] Open source vm testbase GC tests In-Reply-To: <0fecb84f-9d8b-8056-ee01-6e16c5e9467a@oracle.com> References: <0fecb84f-9d8b-8056-ee01-6e16c5e9467a@oracle.com> Message-ID: <019E972C-8B01-4CB4-A918-443916607DBE@oracle.com> Erik, Magnus, thank you for reviewing build change. can someone from GC people Review the rest? -- Igor > On May 8, 2018, at 2:06 AM, Magnus Ihse Bursie wrote: > > On 2018-05-08 00:35, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev/8199370/webrev.00/index.html >>> 45710 lines changed: 45710 ins; 0 del; 0 mod; >> Hi all, >> >> could you please review the patch which open sources GC tests from vm testbase? it introduces the following test groups: >> - vmTestbase_vm_g1classunloading >> - vmTestbase_vm_gc_compact >> - vmTestbase_vm_gc_concurrent >> - vmTestbase_vm_gc_container >> - vmTestbase_vm_gc_juggle >> - vmTestbase_vm_gc_locker >> - vmTestbase_vm_gc_ref >> - vmTestbase_vm_gc_misc >> >> besides these test groups, which split tests by functionality under test, the patch also adds test groups used only for test selection purposes: >> - vmTestbase_vm_gc which includes all vmTestbase_vm_gc_* test groups >> - vmTestbase_vm_gc_quick -- "quick" tests from vmTestbase_vm_gc test group >> - vmTestbase_largepages which consists of tests which are believed to be good candidate to test largepage. this group is used in our testing w/ external vm flags and/or on pre-configured hosts. >> >> As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8199370 >> webrev: http://cr.openjdk.java.net/~iignatyev/8199370/webrev.00/index.html > Build changes look good. > > /Magnus > >> >> Thanks, >> -- Igor From kim.barrett at oracle.com Tue May 8 19:34:14 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 8 May 2018 15:34:14 -0400 Subject: RFR: 8202813: Move vm_weak processing from SystemDictionary::do_unloading to WeakProcessor Message-ID: Please review this change to move the clearing of dead entries in vm_weak_oop_storage from SystemDictionary::do_unloading to preceeding calls to WeakProcessor::weak_oops_do. This involves conditionalizing WeakProcessor invocations, to allow different calls to process different subsets of the native weak references. For example, vm_weak_oop_storage is treated as strong rather than weak when ClassUnloading is disabled. CR: https://bugs.openjdk.java.net/browse/JDK-8202813 Webrev: http://cr.openjdk.java.net/~kbarrett/8202813/open.00/ Testing: Mach5 jdk-tier{1-2}, hs-tier{1-5} Locally runthese30m. From kim.barrett at oracle.com Tue May 8 19:34:49 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 8 May 2018 15:34:49 -0400 Subject: RFR: 8202631: JVM_Clone to throw CloneNotSupportException for Reference object In-Reply-To: <5ff5d874-425b-2c77-196d-15b3601a172d@oracle.com> References: <3509AB48-4DF3-4629-B98F-94CB2E239A54@oracle.com> <5ff5d874-425b-2c77-196d-15b3601a172d@oracle.com> Message-ID: > On May 8, 2018, at 2:41 PM, coleen.phillimore at oracle.com wrote: > > > Looks good to me also. > Coleen Thanks. From leonid.mesnik at oracle.com Tue May 8 20:33:16 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 8 May 2018 13:33:16 -0700 Subject: RFR(XS):8202748: jtreg :hotspot_misc group shouldn't include vmTestbase tests Message-ID: Hi Could you please review following tiny fix which just exclude vmTestbase/ directory from :hotspot_misc. It contains recently open sourced vm testbase tests. There were no plan to add them into :hotspot_misc. webrev: http://cr.openjdk.java.net/~lmesnik/8202748/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8202748 Leonid From igor.ignatyev at oracle.com Tue May 8 20:44:19 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 8 May 2018 13:44:19 -0700 Subject: RFR(XS):8202748: jtreg :hotspot_misc group shouldn't include vmTestbase tests In-Reply-To: References: Message-ID: <5FC0B486-D8BB-430D-8D59-29BD40A924BD@oracle.com> Hi Leonid, thanks for taking care of this. the fix looks good to me. -- Igor > On May 8, 2018, at 1:33 PM, Leonid Mesnik wrote: > > Hi > > Could you please review following tiny fix which just exclude vmTestbase/ directory from :hotspot_misc. > It contains recently open sourced vm testbase tests. There were no plan to add them into :hotspot_misc. > > webrev: http://cr.openjdk.java.net/~lmesnik/8202748/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8202748 > > Leonid From george.triantafillou at oracle.com Tue May 8 20:51:04 2018 From: george.triantafillou at oracle.com (George Triantafillou) Date: Tue, 8 May 2018 16:51:04 -0400 Subject: RFR(XS):8202748: jtreg :hotspot_misc group shouldn't include vmTestbase tests In-Reply-To: References: Message-ID: Leonid, Looks good. -George On 5/8/2018 4:33 PM, Leonid Mesnik wrote: > Hi > > Could you please review following tiny fix which just exclude vmTestbase/ directory from :hotspot_misc. > It contains recently open sourced vm testbase tests. There were no plan to add them into :hotspot_misc. > > webrev: http://cr.openjdk.java.net/~lmesnik/8202748/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8202748 > > Leonid From mikhailo.seledtsov at oracle.com Tue May 8 20:53:31 2018 From: mikhailo.seledtsov at oracle.com (mikhailo) Date: Tue, 8 May 2018 13:53:31 -0700 Subject: RFR(XS):8202748: jtreg :hotspot_misc group shouldn't include vmTestbase tests In-Reply-To: <5FC0B486-D8BB-430D-8D59-29BD40A924BD@oracle.com> References: <5FC0B486-D8BB-430D-8D59-29BD40A924BD@oracle.com> Message-ID: <3ceba92e-7f6d-d0b4-eccd-22f81275a96f@oracle.com> +1 On 05/08/2018 01:44 PM, Igor Ignatyev wrote: > Hi Leonid, > > thanks for taking care of this. the fix looks good to me. > > -- Igor > >> On May 8, 2018, at 1:33 PM, Leonid Mesnik wrote: >> >> Hi >> >> Could you please review following tiny fix which just exclude vmTestbase/ directory from :hotspot_misc. >> It contains recently open sourced vm testbase tests. There were no plan to add them into :hotspot_misc. >> >> webrev: http://cr.openjdk.java.net/~lmesnik/8202748/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8202748 >> >> Leonid From leonid.mesnik at oracle.com Tue May 8 21:23:34 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 8 May 2018 14:23:34 -0700 Subject: 8199271: [TESTBUG] open source VM testbase stress tests Message-ID: <1BC403EC-BDF0-4CA2-B87F-B38A92B6765F@oracle.com> Hi Please review this change open sourcing vm testbase stress tests. These tests have been developed a long time ago for internal test harness and don't looks very nice now. They are open sourced with minimal changes only. The long term plan is to review and improve them moving out of vmTestbase directory in corresponding components. The fix introduce vmTestbase_nsk_stress test group and have make files changes required to build native part of tests. I've verified that "make -- run-test TEST=:vmTestbase_nsk_stress" pass on my linux locally. webrev: http://cr.openjdk.java.net/~lmesnik/8199271/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8199271 From erik.joelsson at oracle.com Tue May 8 21:41:54 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 8 May 2018 14:41:54 -0700 Subject: 8199271: [TESTBUG] open source VM testbase stress tests In-Reply-To: <1BC403EC-BDF0-4CA2-B87F-B38A92B6765F@oracle.com> References: <1BC403EC-BDF0-4CA2-B87F-B38A92B6765F@oracle.com> Message-ID: <4f0ea090-94c9-ffd7-3852-a9a9d14e8af8@oracle.com> Build changes look good. /Erik On 2018-05-08 14:23, Leonid Mesnik wrote: > Hi > > Please review this change open sourcing vm testbase stress tests. These tests have been developed a long time ago for internal test harness and don't looks very nice now. > They are open sourced with minimal changes only. The long term plan is to review and improve them moving out of vmTestbase directory in corresponding components. > > The fix introduce vmTestbase_nsk_stress test group and have make files changes required to build native part of tests. > > I've verified that "make -- run-test TEST=:vmTestbase_nsk_stress" pass on my linux locally. > > webrev: http://cr.openjdk.java.net/~lmesnik/8199271/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8199271 From martin.doerr at sap.com Wed May 9 07:45:25 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 9 May 2018 07:45:25 +0000 Subject: RFR(XS): 8202564: java/lang/management/ThreadMXBean/ThreadCounts.java fails In-Reply-To: <693d5169-4f0b-8bdd-1663-899d531ebf1d@oracle.com> References: <0ceca1748a6e43b79beebd41d633dd12@sap.com> <180175fc-009a-fcaa-5a85-939799c234dc@oracle.com> <693d5169-4f0b-8bdd-1663-899d531ebf1d@oracle.com> Message-ID: Hi Vladimir and Daniel, thank you for reviewing and pushing. Best regards, Martin -----Original Message----- From: Daniel D. Daugherty [mailto:daniel.daugherty at oracle.com] Sent: Dienstag, 8. Mai 2018 18:24 To: Vladimir Kozlov ; Doerr, Martin ; hotspot-dev Source Developers Subject: Re: RFR(XS): 8202564: java/lang/management/ThreadMXBean/ThreadCounts.java fails > http://cr.openjdk.java.net/~mdoerr/8202564_thread_cnt/webrev.00/ src/hotspot/share/services/threadService.cpp ??? You have moved the decrement of _exiting_daemon_threads_count ??? L125: ? if (daemon) { ??? L126: ??? Atomic::dec(&_exiting_daemon_threads_count); ??? L127: ? } ??? above this check: ??? L129: ? if (thread->is_hidden_from_external_view() || ??? L130: ????? thread->is_jvmti_agent_thread()) { ??? and away from this other decrement: ??? L136: _daemon_threads_count->set_value(_daemon_threads_count->get_value() - 1); ??? That makes sense because the original increment of _exiting_daemon_threads_count ??? happens here: ??? L140 void ThreadService::current_thread_exiting(JavaThread* jt) { ??? L141?? assert(jt == JavaThread::current(), "Called by current thread"); ??? L142?? Atomic::inc(&_exiting_threads_count); ??? L143 ??? L144?? oop threadObj = jt->threadObj(); ??? L145?? if (threadObj != NULL && java_lang_Thread::is_daemon(threadObj)) { ??? L146???? Atomic::inc(&_exiting_daemon_threads_count); ??? L147?? } ??? L148 } ??? and is not guarded by the hidden thread or jvmti check. For other ??? folks that are reading closely, these two checks are equivalent: ????? if (daemon) { ??? and: ????? if (threadObj != NULL && java_lang_Thread::is_daemon(threadObj)) { ??? Very nice catch! I'm just guessing here, but I think the recent CompilerThread changes now allow CompilerThreads to go thru the ThreadService::remove_thread() code path which exposed this bug. So if there was a late query about the number of daemon threads and a CompilerThread had exited, then the count would be off. However, if the CompilerThread exited a little bit later, then the test would pass... Thumbs up! Dan On 5/8/18 12:03 PM, Vladimir Kozlov wrote: > Hi Martin, > > Thank you for finding the cause and fixing it. Looks good. > > Changing to hotspot-dev alias since it is not compiler code. Let other > people look on it. > > Thanks, > Vladimir > > On 5/8/18 6:50 AM, Doerr, Martin wrote: >> Hi Vladimir, >> >> I've fixed the thread count issue. >> >> "_exiting_daemon_threads_count" just needs to get decremented at the >> right place (like "_exiting_threads_count"). >> >> Please review: >> >> http://cr.openjdk.java.net/~mdoerr/8202564_thread_cnt/webrev.00/ >> >> Best regards, >> >> Martin >> From poweruserm at live.com.au Wed May 9 10:43:10 2018 From: poweruserm at live.com.au (A Z) Date: Wed, 9 May 2018 10:43:10 +0000 Subject: JEP 306 Message-ID: Does anyone know what is going to be decided around full target status of JEP 306? I have been recommended to this list for the sake of this question. ? http://openjdk.java.net/jeps/306 https://bugs.openjdk.java.net/browse/JDK-8175916 From erik.osterlund at oracle.com Wed May 9 14:32:22 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 9 May 2018 16:32:22 +0200 Subject: 8202377: Modularize C2 GC barriers In-Reply-To: <5AE86C53.4070708@oracle.com> References: <5AE86C53.4070708@oracle.com> Message-ID: <5AF30676.8050909@oracle.com> Hi, I have rebased ZGC on top of these changes. In the process, I found a few more hooks that will be needed for ZGC. I added them, so here is an incremental update: http://cr.openjdk.java.net/~eosterlund/8202377/webrev.00_01/ Full webrev: http://cr.openjdk.java.net/~eosterlund/8202377/webrev.01/ Each new hook is a no-op for existing GCs. I am merely adding them to pave way for ZGC. Also made a few members public that need to be accessible by the ZGC barrier set backend. Nils has helped me look at this all day. So big thanks to Nils for taking the time to look at these changes. Thanks, /Erik On 2018-05-01 15:32, Erik ?sterlund wrote: > Hi, > > The GC barriers for C2 are not as modular as they could be. It > currently uses switch statements to check which GC barrier set is > being used, and call one or another barrier based on that, in a way > that it can only be used for write barriers. > > My proposed solution is to follow the same pattern that has been used > by C1 (and the rest of HotSpot), which is to provide a GC barrier set > code generation helper for C2. Its name is BarrierSetC2. Each barrier > set class has its own BarrierSetC2, following a mirrored inheritance > hierarchy to the BarrierSet hierarchy. You generate the accesses using > some access_* member functions on GraphKit, which calls into > BarrierSetC2. > > A lot of the design looks very similar to BarrierSetC1. In C1, there > was a wrapper object called LIRAccess that wrapped a bunch of context > parameters that were passed around in the barrier set hierarchy. There > is a similar wrapper for C2 that I call C2Access. Users of the API do > not see it. They call, e.g. access_load_at, in GraphKit during > parsing. The access functions wrap the access in a C2Access object > with a bunch of context parameters, and calls the currently selected > BarrierSetC2 backend accessor with this context. For the atomic > accesses, there is a C2AtomicAccess, inheriting from C2Access. It > contains more context, as required by the atomic accesses (e.g. > explicit alias_idx, whether the node needs pinning with an SCM > projection, and a memory node). > > Apart from the normal shared decorators, C2 does use its own > additional decorators for its own use: > * C2_MISMATCHED and C2_UNALIGNED (describing properties of unsafe > accesses) > * C2_WEAK_CMPXCHG: describing if a cmpxchg may have false negatives > * C2_CONTROL_DEPENDENT_LOAD: use when a load should have control > dependency > * C2_PINNED_LOAD: use for loads that must be pinned > * C2_UNSAFE_ACCESS: Used to recognize this is an unsafe access. This > decorator implies that loads have control dependency and need pinning, > unless it can be proven that the access will be inside the bounds of > an object. > * C2_READ_ACCESS and C2_WRITE_ACCESS: This denotes whether the access > reads or writes to memory. Or both for atomics. It is useful for for > figuring out what fencing is required for a given access and ordering > semantics, as well as being useful for Shenandoah to figure out what > type of barrier to use to ensure memory consistency. > > The accesses go through a similar process as they do in C1. Let's take > BarrierSetC2::store_at for example. It uses the the C2AccessFence > scoped object helper to figure out what membars are required to > surround the access, resolve the address (no-op for all GCs with a > to-space invariant, which is all GCs except Shenandoah in HotSpot at > the moment), and then calls store_at_resolved. The store_at_resolved > member function generates the access and the barriers around it. The > abstract ModRefBarrierSetC2 barrier set introduces the notion of > pre/post write barriers, and lets concrete barrier sets do sprinkle > their GC barriers in there. It calls BarrierSetC2::store_at_resolved > to generate the actual access. For example CardTableBarrierSet only > needs to override its post barrier for this to work as expected. The > other accesses follow a similar pattern. > > The Compile class now has a type erase (void*) per compilation unit > state that is created for each compilation unit (with > BarrierSetC2::create_barrier_state). For the GCs in HotSpot today, > this is always NULL. But for GCs that have their own macro nodes, the > compilation unit can be used for, e.g. lists of barrier-specific macro > nodes, that should not pollute the Compile object. Such macro nodes > can be expanded during macro expansion using the > BarrierSetC2::expand_macro_nodes member function. > > There are a few other helpers that may be good for a GC to have, like > figuring out if a node is a GC barrier (for escape analysis), whether > a GC barrier can be eliminated (for example using > ReduceInitialCardMarks), whether array_copy requires GC barriers, how > to step over a GC barrier. There is also a helper for loop optimizing > GC barrier nodes. > > This work will help to pave way for a new class of collectors > utilizing load barriers (ZGC and Shenandoah) for concurrent compaction. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8202377/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8202377 > > Thanks, > /Erik From alexey.ivanov at oracle.com Wed May 9 14:34:45 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Wed, 9 May 2018 15:34:45 +0100 Subject: [11] RFR for JDK-8202544: Hide unused exports in libzip In-Reply-To: References: <4b0e27bb-8167-4ae5-c5e5-6ec054015d30@oracle.com> Message-ID: <426447b2-974e-bbf6-e1df-3ce235930252@oracle.com> Any volunteers from core-libs and/or hotspot? Thank you, Alexey On 02/05/2018 13:02, Magnus Ihse Bursie wrote: > Looks good to me, FWIW. > > /Magnus > >> 2 maj 2018 kl. 13:52 skrev Alexey Ivanov : >> >> Hi, >> >> Could you please review the following fix for jdk11? >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8202544 >> webrev: http://cr.openjdk.java.net/~aivanov/8202544/jdk11/webrev.0/ >> >> The following exported functions in libzip are not used: >> ZIP_GetEntry, ZIP_FreeEntry, ZIP_Lock, ZIP_Unlock, ZIP_Read >> >> I removed JNIEXPORT modifiers from these functions. With the fix, they're not exported on Windows; on Linux they're listed as Local rather than Global. >> >> I ran tests, no failures. >> >> >> Thank you in advance. >> >> Regards, >> Alexey From nils.eliasson at oracle.com Wed May 9 14:33:17 2018 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Wed, 9 May 2018 16:33:17 +0200 Subject: 8202377: Modularize C2 GC barriers In-Reply-To: <5AF30676.8050909@oracle.com> References: <5AE86C53.4070708@oracle.com> <5AF30676.8050909@oracle.com> Message-ID: Hi Erik, A job well done. This extensive change improves the readability and quality of the code a lot. Thanks! Reviewed. // Nils On 2018-05-09 16:32, Erik ?sterlund wrote: > Hi, > > I have rebased ZGC on top of these changes. In the process, I found a > few more hooks that will be needed for ZGC. I added them, so here is > an incremental update: > http://cr.openjdk.java.net/~eosterlund/8202377/webrev.00_01/ > > Full webrev: > http://cr.openjdk.java.net/~eosterlund/8202377/webrev.01/ > > Each new hook is a no-op for existing GCs. I am merely adding them to > pave way for ZGC. > Also made a few members public that need to be accessible by the ZGC > barrier set backend. > > Nils has helped me look at this all day. So big thanks to Nils for > taking the time to look at these changes. > > Thanks, > /Erik > > On 2018-05-01 15:32, Erik ?sterlund wrote: >> Hi, >> >> The GC barriers for C2 are not as modular as they could be. It >> currently uses switch statements to check which GC barrier set is >> being used, and call one or another barrier based on that, in a way >> that it can only be used for write barriers. >> >> My proposed solution is to follow the same pattern that has been used >> by C1 (and the rest of HotSpot), which is to provide a GC barrier set >> code generation helper for C2. Its name is BarrierSetC2. Each barrier >> set class has its own BarrierSetC2, following a mirrored inheritance >> hierarchy to the BarrierSet hierarchy. You generate the accesses >> using some access_* member functions on GraphKit, which calls into >> BarrierSetC2. >> >> A lot of the design looks very similar to BarrierSetC1. In C1, there >> was a wrapper object called LIRAccess that wrapped a bunch of context >> parameters that were passed around in the barrier set hierarchy. >> There is a similar wrapper for C2 that I call C2Access. Users of the >> API do not see it. They call, e.g. access_load_at, in GraphKit during >> parsing. The access functions wrap the access in a C2Access object >> with a bunch of context parameters, and calls the currently selected >> BarrierSetC2 backend accessor with this context. For the atomic >> accesses, there is a C2AtomicAccess, inheriting from C2Access. It >> contains more context, as required by the atomic accesses (e.g. >> explicit alias_idx, whether the node needs pinning with an SCM >> projection, and a memory node). >> >> Apart from the normal shared decorators, C2 does use its own >> additional decorators for its own use: >> * C2_MISMATCHED and C2_UNALIGNED (describing properties of unsafe >> accesses) >> * C2_WEAK_CMPXCHG: describing if a cmpxchg may have false negatives >> * C2_CONTROL_DEPENDENT_LOAD: use when a load should have control >> dependency >> * C2_PINNED_LOAD: use for loads that must be pinned >> * C2_UNSAFE_ACCESS: Used to recognize this is an unsafe access. This >> decorator implies that loads have control dependency and need >> pinning, unless it can be proven that the access will be inside the >> bounds of an object. >> * C2_READ_ACCESS and C2_WRITE_ACCESS: This denotes whether the access >> reads or writes to memory. Or both for atomics. It is useful for for >> figuring out what fencing is required for a given access and ordering >> semantics, as well as being useful for Shenandoah to figure out what >> type of barrier to use to ensure memory consistency. >> >> The accesses go through a similar process as they do in C1. Let's >> take BarrierSetC2::store_at for example. It uses the the >> C2AccessFence scoped object helper to figure out what membars are >> required to surround the access, resolve the address (no-op for all >> GCs with a to-space invariant, which is all GCs except Shenandoah in >> HotSpot at the moment), and then calls store_at_resolved. The >> store_at_resolved member function generates the access and the >> barriers around it. The abstract ModRefBarrierSetC2 barrier set >> introduces the notion of pre/post write barriers, and lets concrete >> barrier sets do sprinkle their GC barriers in there. It calls >> BarrierSetC2::store_at_resolved to generate the actual access. For >> example CardTableBarrierSet only needs to override its post barrier >> for this to work as expected. The other accesses follow a similar >> pattern. >> >> The Compile class now has a type erase (void*) per compilation unit >> state that is created for each compilation unit (with >> BarrierSetC2::create_barrier_state). For the GCs in HotSpot today, >> this is always NULL. But for GCs that have their own macro nodes, the >> compilation unit can be used for, e.g. lists of barrier-specific >> macro nodes, that should not pollute the Compile object. Such macro >> nodes can be expanded during macro expansion using the >> BarrierSetC2::expand_macro_nodes member function. >> >> There are a few other helpers that may be good for a GC to have, like >> figuring out if a node is a GC barrier (for escape analysis), whether >> a GC barrier can be eliminated (for example using >> ReduceInitialCardMarks), whether array_copy requires GC barriers, how >> to step over a GC barrier. There is also a helper for loop optimizing >> GC barrier nodes. >> >> This work will help to pave way for a new class of collectors >> utilizing load barriers (ZGC and Shenandoah) for concurrent compaction. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8202377/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8202377 >> >> Thanks, >> /Erik > From erik.osterlund at oracle.com Wed May 9 14:50:14 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Wed, 9 May 2018 16:50:14 +0200 Subject: 8202377: Modularize C2 GC barriers In-Reply-To: References: <5AE86C53.4070708@oracle.com> <5AF30676.8050909@oracle.com> Message-ID: <5AF30AA6.20306@oracle.com> Hi Nils, Thank you for the review. :) /Erik On 2018-05-09 16:33, Nils Eliasson wrote: > Hi Erik, > > A job well done. This extensive change improves the readability and > quality of the code a lot. > > Thanks! > > Reviewed. > > // Nils > > > On 2018-05-09 16:32, Erik ?sterlund wrote: >> Hi, >> >> I have rebased ZGC on top of these changes. In the process, I found a >> few more hooks that will be needed for ZGC. I added them, so here is >> an incremental update: >> http://cr.openjdk.java.net/~eosterlund/8202377/webrev.00_01/ >> >> Full webrev: >> http://cr.openjdk.java.net/~eosterlund/8202377/webrev.01/ >> >> Each new hook is a no-op for existing GCs. I am merely adding them to >> pave way for ZGC. >> Also made a few members public that need to be accessible by the ZGC >> barrier set backend. >> >> Nils has helped me look at this all day. So big thanks to Nils for >> taking the time to look at these changes. >> >> Thanks, >> /Erik >> >> On 2018-05-01 15:32, Erik ?sterlund wrote: >>> Hi, >>> >>> The GC barriers for C2 are not as modular as they could be. It >>> currently uses switch statements to check which GC barrier set is >>> being used, and call one or another barrier based on that, in a way >>> that it can only be used for write barriers. >>> >>> My proposed solution is to follow the same pattern that has been >>> used by C1 (and the rest of HotSpot), which is to provide a GC >>> barrier set code generation helper for C2. Its name is BarrierSetC2. >>> Each barrier set class has its own BarrierSetC2, following a >>> mirrored inheritance hierarchy to the BarrierSet hierarchy. You >>> generate the accesses using some access_* member functions on >>> GraphKit, which calls into BarrierSetC2. >>> >>> A lot of the design looks very similar to BarrierSetC1. In C1, there >>> was a wrapper object called LIRAccess that wrapped a bunch of >>> context parameters that were passed around in the barrier set >>> hierarchy. There is a similar wrapper for C2 that I call C2Access. >>> Users of the API do not see it. They call, e.g. access_load_at, in >>> GraphKit during parsing. The access functions wrap the access in a >>> C2Access object with a bunch of context parameters, and calls the >>> currently selected BarrierSetC2 backend accessor with this context. >>> For the atomic accesses, there is a C2AtomicAccess, inheriting from >>> C2Access. It contains more context, as required by the atomic >>> accesses (e.g. explicit alias_idx, whether the node needs pinning >>> with an SCM projection, and a memory node). >>> >>> Apart from the normal shared decorators, C2 does use its own >>> additional decorators for its own use: >>> * C2_MISMATCHED and C2_UNALIGNED (describing properties of unsafe >>> accesses) >>> * C2_WEAK_CMPXCHG: describing if a cmpxchg may have false negatives >>> * C2_CONTROL_DEPENDENT_LOAD: use when a load should have control >>> dependency >>> * C2_PINNED_LOAD: use for loads that must be pinned >>> * C2_UNSAFE_ACCESS: Used to recognize this is an unsafe access. This >>> decorator implies that loads have control dependency and need >>> pinning, unless it can be proven that the access will be inside the >>> bounds of an object. >>> * C2_READ_ACCESS and C2_WRITE_ACCESS: This denotes whether the >>> access reads or writes to memory. Or both for atomics. It is useful >>> for for figuring out what fencing is required for a given access and >>> ordering semantics, as well as being useful for Shenandoah to figure >>> out what type of barrier to use to ensure memory consistency. >>> >>> The accesses go through a similar process as they do in C1. Let's >>> take BarrierSetC2::store_at for example. It uses the the >>> C2AccessFence scoped object helper to figure out what membars are >>> required to surround the access, resolve the address (no-op for all >>> GCs with a to-space invariant, which is all GCs except Shenandoah in >>> HotSpot at the moment), and then calls store_at_resolved. The >>> store_at_resolved member function generates the access and the >>> barriers around it. The abstract ModRefBarrierSetC2 barrier set >>> introduces the notion of pre/post write barriers, and lets concrete >>> barrier sets do sprinkle their GC barriers in there. It calls >>> BarrierSetC2::store_at_resolved to generate the actual access. For >>> example CardTableBarrierSet only needs to override its post barrier >>> for this to work as expected. The other accesses follow a similar >>> pattern. >>> >>> The Compile class now has a type erase (void*) per compilation unit >>> state that is created for each compilation unit (with >>> BarrierSetC2::create_barrier_state). For the GCs in HotSpot today, >>> this is always NULL. But for GCs that have their own macro nodes, >>> the compilation unit can be used for, e.g. lists of barrier-specific >>> macro nodes, that should not pollute the Compile object. Such macro >>> nodes can be expanded during macro expansion using the >>> BarrierSetC2::expand_macro_nodes member function. >>> >>> There are a few other helpers that may be good for a GC to have, >>> like figuring out if a node is a GC barrier (for escape analysis), >>> whether a GC barrier can be eliminated (for example using >>> ReduceInitialCardMarks), whether array_copy requires GC barriers, >>> how to step over a GC barrier. There is also a helper for loop >>> optimizing GC barrier nodes. >>> >>> This work will help to pave way for a new class of collectors >>> utilizing load barriers (ZGC and Shenandoah) for concurrent compaction. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8202377/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8202377 >>> >>> Thanks, >>> /Erik >> > From nils.eliasson at oracle.com Wed May 9 14:39:51 2018 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Wed, 9 May 2018 16:39:51 +0200 Subject: 8202377: Modularize C2 GC barriers In-Reply-To: <5AF30AA6.20306@oracle.com> References: <5AE86C53.4070708@oracle.com> <5AF30676.8050909@oracle.com> <5AF30AA6.20306@oracle.com> Message-ID: You are welcome to refactor the rest of C2 =) // Nils On 2018-05-09 16:50, Erik ?sterlund wrote: > Hi Nils, > > Thank you for the review. :) > > /Erik > > On 2018-05-09 16:33, Nils Eliasson wrote: >> Hi Erik, >> >> A job well done. This extensive change improves the readability and >> quality of the code a lot. >> >> Thanks! >> >> Reviewed. >> >> // Nils >> >> >> On 2018-05-09 16:32, Erik ?sterlund wrote: >>> Hi, >>> >>> I have rebased ZGC on top of these changes. In the process, I found >>> a few more hooks that will be needed for ZGC. I added them, so here >>> is an incremental update: >>> http://cr.openjdk.java.net/~eosterlund/8202377/webrev.00_01/ >>> >>> Full webrev: >>> http://cr.openjdk.java.net/~eosterlund/8202377/webrev.01/ >>> >>> Each new hook is a no-op for existing GCs. I am merely adding them >>> to pave way for ZGC. >>> Also made a few members public that need to be accessible by the ZGC >>> barrier set backend. >>> >>> Nils has helped me look at this all day. So big thanks to Nils for >>> taking the time to look at these changes. >>> >>> Thanks, >>> /Erik >>> >>> On 2018-05-01 15:32, Erik ?sterlund wrote: >>>> Hi, >>>> >>>> The GC barriers for C2 are not as modular as they could be. It >>>> currently uses switch statements to check which GC barrier set is >>>> being used, and call one or another barrier based on that, in a way >>>> that it can only be used for write barriers. >>>> >>>> My proposed solution is to follow the same pattern that has been >>>> used by C1 (and the rest of HotSpot), which is to provide a GC >>>> barrier set code generation helper for C2. Its name is >>>> BarrierSetC2. Each barrier set class has its own BarrierSetC2, >>>> following a mirrored inheritance hierarchy to the BarrierSet >>>> hierarchy. You generate the accesses using some access_* member >>>> functions on GraphKit, which calls into BarrierSetC2. >>>> >>>> A lot of the design looks very similar to BarrierSetC1. In C1, >>>> there was a wrapper object called LIRAccess that wrapped a bunch of >>>> context parameters that were passed around in the barrier set >>>> hierarchy. There is a similar wrapper for C2 that I call C2Access. >>>> Users of the API do not see it. They call, e.g. access_load_at, in >>>> GraphKit during parsing. The access functions wrap the access in a >>>> C2Access object with a bunch of context parameters, and calls the >>>> currently selected BarrierSetC2 backend accessor with this context. >>>> For the atomic accesses, there is a C2AtomicAccess, inheriting from >>>> C2Access. It contains more context, as required by the atomic >>>> accesses (e.g. explicit alias_idx, whether the node needs pinning >>>> with an SCM projection, and a memory node). >>>> >>>> Apart from the normal shared decorators, C2 does use its own >>>> additional decorators for its own use: >>>> * C2_MISMATCHED and C2_UNALIGNED (describing properties of unsafe >>>> accesses) >>>> * C2_WEAK_CMPXCHG: describing if a cmpxchg may have false negatives >>>> * C2_CONTROL_DEPENDENT_LOAD: use when a load should have control >>>> dependency >>>> * C2_PINNED_LOAD: use for loads that must be pinned >>>> * C2_UNSAFE_ACCESS: Used to recognize this is an unsafe access. >>>> This decorator implies that loads have control dependency and need >>>> pinning, unless it can be proven that the access will be inside the >>>> bounds of an object. >>>> * C2_READ_ACCESS and C2_WRITE_ACCESS: This denotes whether the >>>> access reads or writes to memory. Or both for atomics. It is useful >>>> for for figuring out what fencing is required for a given access >>>> and ordering semantics, as well as being useful for Shenandoah to >>>> figure out what type of barrier to use to ensure memory consistency. >>>> >>>> The accesses go through a similar process as they do in C1. Let's >>>> take BarrierSetC2::store_at for example. It uses the the >>>> C2AccessFence scoped object helper to figure out what membars are >>>> required to surround the access, resolve the address (no-op for all >>>> GCs with a to-space invariant, which is all GCs except Shenandoah >>>> in HotSpot at the moment), and then calls store_at_resolved. The >>>> store_at_resolved member function generates the access and the >>>> barriers around it. The abstract ModRefBarrierSetC2 barrier set >>>> introduces the notion of pre/post write barriers, and lets concrete >>>> barrier sets do sprinkle their GC barriers in there. It calls >>>> BarrierSetC2::store_at_resolved to generate the actual access. For >>>> example CardTableBarrierSet only needs to override its post barrier >>>> for this to work as expected. The other accesses follow a similar >>>> pattern. >>>> >>>> The Compile class now has a type erase (void*) per compilation unit >>>> state that is created for each compilation unit (with >>>> BarrierSetC2::create_barrier_state). For the GCs in HotSpot today, >>>> this is always NULL. But for GCs that have their own macro nodes, >>>> the compilation unit can be used for, e.g. lists of >>>> barrier-specific macro nodes, that should not pollute the Compile >>>> object. Such macro nodes can be expanded during macro expansion >>>> using the BarrierSetC2::expand_macro_nodes member function. >>>> >>>> There are a few other helpers that may be good for a GC to have, >>>> like figuring out if a node is a GC barrier (for escape analysis), >>>> whether a GC barrier can be eliminated (for example using >>>> ReduceInitialCardMarks), whether array_copy requires GC barriers, >>>> how to step over a GC barrier. There is also a helper for loop >>>> optimizing GC barrier nodes. >>>> >>>> This work will help to pave way for a new class of collectors >>>> utilizing load barriers (ZGC and Shenandoah) for concurrent >>>> compaction. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8202377/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8202377 >>>> >>>> Thanks, >>>> /Erik >>> >> > From aph at redhat.com Wed May 9 16:37:09 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 9 May 2018 17:37:09 +0100 Subject: Question about Runtime relevant Java JEP. In-Reply-To: References: Message-ID: <158279e2-095e-c02e-2e47-c7f3d148c805@redhat.com> On 05/08/2018 02:42 AM, A Z wrote: > I was asking to know when JEP 306 will shift to 'targeted', > or what thought and movements in this direction there are. > > It was mentioned elsewhere that this is only relevant to x86 > older processors, but this is not true. Please reply to that post, then. > Implementation of JEP 306 will eliminate arithmetic overflow and > underflow via arithmetic, within-range arithmetic upon double and > float, Why do you believe this? Can you point me to the language in JEP 306 which justifies your belief? > a change which is relevant to everyone. > > It would also ameliorate "naive" implementation of Java libraries. > > An example is the remaining underflow and overflow still possible with > > javax.vecmath.Vector3d.cross > > given the nature of the sourcecode on this method, and anything > else similar. Can you explain why you believe JEP 306 will make any difference to this? JEP 306 doesn't say anything about infinite-precision floating-point arithmetic. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From bsrbnd at gmail.com Wed May 9 19:08:22 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 9 May 2018 21:08:22 +0200 Subject: Large page use crashes the JVM on some Linux systems In-Reply-To: References: <3bfbdf0a-7c9c-6b95-8c6a-a92c169dbf9b@oracle.com> <9d291862-b0ae-09b6-ebad-8c070bfd1446@oracle.com> Message-ID: On 2 May 2018 at 10:38, B. Blaser wrote: > Hi Claes, > > On 24 April 2018 at 21:39, Claes Redestad wrote: >> The root issue here could very well be that the SHM sanity test is >> insufficient. Adding the same test as we already do for TLBFS seems like the >> wrong approach. >> >> I'm not the most knowledgeable about SHM, though, in fact not knowledgeable >> at all, so let's try and get you subscribed to hotspot-dev and spark a >> discussion on the list. >> >> /Claes Experimenting on different systems, I note that the following test ( 'shmget(SHM_HUGETLB) == -1' ) isn't always reliable: http://hg.openjdk.java.net/jdk/jdk/file/35b22ca681d1/src/hotspot/os/linux/os_linux.cpp#l3709 It may happen that small pages are used when no huge pages are available resulting in a further JVM crash due to a non-huge-page-size-alignment of the allocated memory, as discussed previously. It seems that the test above could be enforced by verifying that 'size <= (free+surplus-reserved)*hugepagesize' according to [1] and [2]. Here is a suggested fix for that, tier1 is OK (with 'HugePages_Total: 0') but any further testing on different configurations/systems would be very valuable. What do you think? Thanks, Bernard [1] https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt [2] $ man 5 proc diff -r e81481fea884 src/hotspot/os/linux/os_linux.cpp --- a/src/hotspot/os/linux/os_linux.cpp Mon May 07 08:56:35 2018 +0200 +++ b/src/hotspot/os/linux/os_linux.cpp Wed May 09 19:10:32 2018 +0200 @@ -3617,6 +3617,37 @@ shm_warning_format(str " (error = %d)", err); \ } while (0) +static bool shm_available(size_t bytes) { + size_t free = SIZE_MAX, rsvd = SIZE_MAX, surp = SIZE_MAX; + char line[40]; + + FILE *fp = fopen("/proc/meminfo", "r"); + if (fp) { + while ((free == SIZE_MAX || rsvd == SIZE_MAX || surp == SIZE_MAX) && !feof(fp)) { + fgets(line, sizeof(line), fp); + + if (free == SIZE_MAX && sscanf(line, "HugePages_Free: %zu", &free) == 1) + continue; + + if (rsvd == SIZE_MAX && sscanf(line, "HugePages_Rsvd: %zu", &rsvd) == 1) + continue; + + if (surp == SIZE_MAX && sscanf(line, "HugePages_Surp: %zu", &surp) == 1) + continue; + } + fclose(fp); + } + + gid_t gid = -1; + if (fp = fopen("/proc/sys/vm/hugetlb_shm_group", "r")) { + fscanf(fp, "%u", &gid); + fclose(fp); + } + + return (free+surp-rsvd)*os::large_page_size() >= bytes + && (!geteuid() || group_member(gid)); +} + static char* shmat_with_alignment(int shmid, size_t bytes, size_t alignment) { assert(is_aligned(bytes, alignment), "Must be divisible by the alignment"); @@ -3703,6 +3734,11 @@ return NULL; // Fallback to small pages. } + if (!shm_available(bytes)) { + shm_warning("Shared memory not available."); + return NULL; + } + // Create a large shared memory region to attach to based on size. // Currently, size is the total size of the heap. int shmid = shmget(IPC_PRIVATE, bytes, SHM_HUGETLB|IPC_CREAT|SHM_R|SHM_W); From robbin.ehn at oracle.com Wed May 9 21:03:14 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 9 May 2018 23:03:14 +0200 Subject: RFR(XS): 8202564: java/lang/management/ThreadMXBean/ThreadCounts.java fails In-Reply-To: References: <0ceca1748a6e43b79beebd41d633dd12@sap.com> <180175fc-009a-fcaa-5a85-939799c234dc@oracle.com> <693d5169-4f0b-8bdd-1663-899d531ebf1d@oracle.com> Message-ID: <91be4b3b-bec6-1205-9d84-41492bd0133c@oracle.com> Hi, sorry, I saw this now, just FYI: There is still an issue regarding this: static jlong get_daemon_thread_count() { return _daemon_threads_count->get_value() - _exiting_daemon_threads_count; } Many threads can have exited between the loads. And thus this method can return a negative number if I'm not mistaken. https://bugs.openjdk.java.net/browse/JDK-8202839 /Robbin On 2018-05-09 09:45, Doerr, Martin wrote: > Hi Vladimir and Daniel, > > thank you for reviewing and pushing. > > Best regards, > Martin > > > -----Original Message----- > From: Daniel D. Daugherty [mailto:daniel.daugherty at oracle.com] > Sent: Dienstag, 8. Mai 2018 18:24 > To: Vladimir Kozlov ; Doerr, Martin ; hotspot-dev Source Developers > Subject: Re: RFR(XS): 8202564: java/lang/management/ThreadMXBean/ThreadCounts.java fails > > > http://cr.openjdk.java.net/~mdoerr/8202564_thread_cnt/webrev.00/ > > src/hotspot/share/services/threadService.cpp > ??? You have moved the decrement of _exiting_daemon_threads_count > > ??? L125: ? if (daemon) { > ??? L126: ??? Atomic::dec(&_exiting_daemon_threads_count); > ??? L127: ? } > > ??? above this check: > > ??? L129: ? if (thread->is_hidden_from_external_view() || > ??? L130: ????? thread->is_jvmti_agent_thread()) { > > ??? and away from this other decrement: > > ??? L136: > _daemon_threads_count->set_value(_daemon_threads_count->get_value() - 1); > > ??? That makes sense because the original increment of > _exiting_daemon_threads_count > ??? happens here: > > ??? L140 void ThreadService::current_thread_exiting(JavaThread* jt) { > ??? L141?? assert(jt == JavaThread::current(), "Called by current thread"); > ??? L142?? Atomic::inc(&_exiting_threads_count); > ??? L143 > ??? L144?? oop threadObj = jt->threadObj(); > ??? L145?? if (threadObj != NULL && > java_lang_Thread::is_daemon(threadObj)) { > ??? L146???? Atomic::inc(&_exiting_daemon_threads_count); > ??? L147?? } > ??? L148 } > > ??? and is not guarded by the hidden thread or jvmti check. For other > ??? folks that are reading closely, these two checks are equivalent: > > ????? if (daemon) { > > ??? and: > > ????? if (threadObj != NULL && java_lang_Thread::is_daemon(threadObj)) { > > ??? Very nice catch! > > > I'm just guessing here, but I think the recent CompilerThread changes > now allow CompilerThreads to go thru the ThreadService::remove_thread() > code path which exposed this bug. So if there was a late query about > the number of daemon threads and a CompilerThread had exited, then the > count would be off. However, if the CompilerThread exited a little bit > later, then the test would pass... > > Thumbs up! > > Dan > > > > On 5/8/18 12:03 PM, Vladimir Kozlov wrote: >> Hi Martin, >> >> Thank you for finding the cause and fixing it. Looks good. >> >> Changing to hotspot-dev alias since it is not compiler code. Let other >> people look on it. >> >> Thanks, >> Vladimir >> >> On 5/8/18 6:50 AM, Doerr, Martin wrote: >>> Hi Vladimir, >>> >>> I've fixed the thread count issue. >>> >>> "_exiting_daemon_threads_count" just needs to get decremented at the >>> right place (like "_exiting_threads_count"). >>> >>> Please review: >>> >>> http://cr.openjdk.java.net/~mdoerr/8202564_thread_cnt/webrev.00/ >>> >>> Best regards, >>> >>> Martin >>> > From igor.ignatyev at oracle.com Wed May 9 21:09:53 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 9 May 2018 14:09:53 -0700 Subject: RFR(L) : 8199384 : [TESTBUG] Open source VM testbase MLVM tests Message-ID: <53E4CA40-DE86-41A2-A1BE-239BE1D85069@oracle.com> http://cr.openjdk.java.net/~iignatyev//8199384/webrev.00/index.html > 61414 lines changed: 61414 ins; 0 del; 0 mod; Hi all, could you please review this patch which open sources MLVM tests from VM testbase? these tests were developed in early days of JSR292 to test different aspects of MethodHandles and invokedynamic. As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. JBS: https://bugs.openjdk.java.net/browse/JDK-8199384 webrev: http://cr.openjdk.java.net/~iignatyev//8199384/webrev.00/index.html testing: :vmTestbase_vm_mlvm test group Thanks, -- Igor From erik.joelsson at oracle.com Wed May 9 21:29:44 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 9 May 2018 14:29:44 -0700 Subject: RFR(L) : 8199384 : [TESTBUG] Open source VM testbase MLVM tests In-Reply-To: <53E4CA40-DE86-41A2-A1BE-239BE1D85069@oracle.com> References: <53E4CA40-DE86-41A2-A1BE-239BE1D85069@oracle.com> Message-ID: Build changes look good. /Erik On 2018-05-09 14:09, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8199384/webrev.00/index.html >> 61414 lines changed: 61414 ins; 0 del; 0 mod; > Hi all, > > could you please review this patch which open sources MLVM tests from VM testbase? > > these tests were developed in early days of JSR292 to test different aspects of MethodHandles and invokedynamic. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199384 > webrev: http://cr.openjdk.java.net/~iignatyev//8199384/webrev.00/index.html > testing: :vmTestbase_vm_mlvm test group > > Thanks, > -- Igor > From gerald.thornbrugh at oracle.com Wed May 9 22:20:33 2018 From: gerald.thornbrugh at oracle.com (Gerald Thornbrugh) Date: Wed, 9 May 2018 16:20:33 -0600 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: <37fda46d-89ae-36d1-3a54-70c56ee3ca2f@oracle.com> References: <2498bec9-9433-8ea1-36ee-a36105696418@oracle.com> <37fda46d-89ae-36d1-3a54-70c56ee3ca2f@oracle.com> Message-ID: <6FC5A842-EAA5-491D-ABC1-C1733F431335@oracle.com> Hi Jc and Serguei, I have looked at the test fixes and overall they look good. I am a little concerned with some of the comments I read that included ?Flakiness? and ?unlucky? like this comment in HeapMonitorArrayAllSampledTest.java: // 10% error ensures a sanity test without becoming flaky. // Flakiness is due to the fact that this test is dependent on the sampling rate, which is a // statistical geometric variable around the sampling rate. This means that the test could be // unlucky and not achieve the mean average fast enough for the test case. So it seems there is some dependencies or variabilities that could cause the test to fail if it got unlucky. I understand that it is difficult to test features like this in a reproducible and reliable way. My fear is that under certain system loads or configurations these tests may get ?flaky? or ?unlucky? and start causing intermittent failures during test runs. What kind of testing has been performed for these changes and have you seen and failures due to ?flakiness? or ?unluckiness?? Thanks, Jerry > On May 7, 2018, at 9:49 PM, serguei.spitsyn at oracle.com wrote: > > Hi Jc, > > I'll make one more pass through the JVMTI and test fixes. > However, it would be good if at least one more pair of eyes > looked at the tests as they are a big part of the fix. > > Thanks, > Serguei > > > On 5/7/18 19:28, JC Beyler wrote: >> Hi Vladimir, >> >> Good catch, I believe it was used before but no longer since we put the >> heap sampler information directly in the thread structure. I removed it for >> the next webrev. >> >> Could anyone do a review on the JVMTI parts and tests? >> >> Thanks a lot for your help! >> Jc >> >> On Mon, May 7, 2018 at 6:31 PM Vladimir Kozlov >> wrote: >> >>> I did not look on JVMTI part and tests. It looks good to me. >>> >>> Where _thread field is used? >>> >>> Thanks, >>> Vladimir >>> >>> On 5/7/18 6:10 PM, JC Beyler wrote: >>>> Hi all, >>>> >>>> With the awesome help of Serguei Spitsyn, we have moved forward on the >>>> implementation for JEP-331 and have the following webrev for review: >>>> >>>> Webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.18/ >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8171119 >>>> >>>> It is based on jdk/jdk so should patch well with a recent tip. >>>> >>>> Could we please have some reviews for the webrev? It would be greatly >>>> appreciated! >>>> >>>> Thanks for all your help! >>>> Jc >>>> > From jcbeyler at google.com Wed May 9 23:13:56 2018 From: jcbeyler at google.com (JC Beyler) Date: Wed, 09 May 2018 23:13:56 +0000 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: <6FC5A842-EAA5-491D-ABC1-C1733F431335@oracle.com> References: <2498bec9-9433-8ea1-36ee-a36105696418@oracle.com> <37fda46d-89ae-36d1-3a54-70c56ee3ca2f@oracle.com> <6FC5A842-EAA5-491D-ABC1-C1733F431335@oracle.com> Message-ID: Hi Jerry, First off: Thanks for looking the tests over! I agree that it is not easy to test easily but we have been working really hard on getting it very stable. As is explained in the test plan item: JDK-8202570 (sub task of JDK-8202566 ), I've run these tests now more than 1000 times in a row (I've actually done it multiple times) and Serguei has been running the mach5 tests. I'll let him give an update on it but basically, most tests have actually been stable. There definitely is variability due to the way the sampling rate is calculated and the two systems that I put up to mitigate it has been to accept a certain error rate and to, in certain cases, if a first run failed, run it a second time. Serguei and I have been iterating on the mach5 testing results to ensure stability for the tests. Some of the tests have had a few changes to solve warming up the TLABs, total memory limits, and test timing limits but nothing really provoked a massive change in the testing framework in general. I'll let him chime in if I have forgotten anything. Thanks, let me know if this answers your questions and if there is anything else I could do :), Jc On Wed, May 9, 2018 at 3:20 PM Gerald Thornbrugh < gerald.thornbrugh at oracle.com> wrote: > Hi Jc and Serguei, > > I have looked at the test fixes and overall they look good. I am a little > concerned with some of the comments > I read that included ?Flakiness? and ?unlucky? like this comment > in HeapMonitorArrayAllSampledTest.java: > > // 10% error ensures a sanity test without becoming flaky. > // Flakiness is due to the fact that this test is dependent on the sampling rate, which is a > // statistical geometric variable around the sampling rate. This means that the test could be > // unlucky and not achieve the mean average fast enough for the test case. > > > So it seems there is some dependencies or variabilities that could cause > the test to fail if it got unlucky. > I understand that it is difficult to test features like this in a > reproducible and reliable way. > My fear is that under certain system loads or configurations these tests > may get ?flaky? or ?unlucky? and start > causing intermittent failures during test runs. > > What kind of testing has been performed for these changes and have you > seen and failures due to ?flakiness? > or ?unluckiness?? > > Thanks, > > Jerry > > On May 7, 2018, at 9:49 PM, serguei.spitsyn at oracle.com wrote: > > Hi Jc, > > I'll make one more pass through the JVMTI and test fixes. > However, it would be good if at least one more pair of eyes > looked at the tests as they are a big part of the fix. > > Thanks, > Serguei > > > On 5/7/18 19:28, JC Beyler wrote: > > Hi Vladimir, > > Good catch, I believe it was used before but no longer since we put the > heap sampler information directly in the thread structure. I removed it for > the next webrev. > > Could anyone do a review on the JVMTI parts and tests? > > Thanks a lot for your help! > Jc > > On Mon, May 7, 2018 at 6:31 PM Vladimir Kozlov > > wrote: > > I did not look on JVMTI part and tests. It looks good to me. > > Where _thread field is used? > > Thanks, > Vladimir > > On 5/7/18 6:10 PM, JC Beyler wrote: > > Hi all, > > With the awesome help of Serguei Spitsyn, we have moved forward on the > implementation for JEP-331 and have the following webrev for review: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.18/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8171119 > > It is based on jdk/jdk so should patch well with a recent tip. > > Could we please have some reviews for the webrev? It would be greatly > appreciated! > > Thanks for all your help! > Jc > > > > From vladimir.kozlov at oracle.com Wed May 9 23:26:16 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 9 May 2018 16:26:16 -0700 Subject: RFR(XS):8202748: jtreg :hotspot_misc group shouldn't include vmTestbase tests In-Reply-To: References: Message-ID: <4fac0f1c-5c0d-6067-672a-590f230b1ad2@oracle.com> Yes, please. I got hotspot_misc tasks timeout running with -Xcomp on all platforms. Thanks, Vladimir K On 5/8/18 1:33 PM, Leonid Mesnik wrote: > Hi > > Could you please review following tiny fix which just exclude vmTestbase/ directory from :hotspot_misc. > It contains recently open sourced vm testbase tests. There were no plan to add them into :hotspot_misc. > > webrev: http://cr.openjdk.java.net/~lmesnik/8202748/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8202748 > > Leonid > From vladimir.kozlov at oracle.com Wed May 9 23:47:12 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 9 May 2018 16:47:12 -0700 Subject: [11] RFR(XS) 8202773: Unhandled oop in JavaThread::collect_counters Message-ID: <4376e8c6-2e77-1928-e159-ebd4ded45fe1@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8202773 Based on discussion in bug report MutexLocker should be removed from this code: diff -r ae0ebd3cf949 src/hotspot/share/runtime/thread.cpp --- a/src/hotspot/share/runtime/thread.cpp +++ b/src/hotspot/share/runtime/thread.cpp @@ -1467,7 +1467,6 @@ void JavaThread::collect_counters(typeArrayOop array) { if (JVMCICounterSize > 0) { - MutexLocker tl(Threads_lock); JavaThreadIteratorWithHandle jtiwh; for (int i = 0; i < array->length(); i++) { array->long_at_put(i, _jvmci_old_thread_counters[i]); Tested with tier1,tier2,tier2-graal,hs-precheckin-comp -- Thanks, Vladimir From igor.ignatyev at oracle.com Thu May 10 00:14:50 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 9 May 2018 17:14:50 -0700 Subject: RFR(M) : 8202392: [TESTBUG] open source vm testbase heapdump tests Message-ID: http://cr.openjdk.java.net/~iignatyev//8202392/webrev.00/index.html > 1396 lines changed: 1396 ins; 0 del; 0 mod; Hi all, could you please review his patch which open sources heapdump tests from so-called VM testbase? as it's obvious from test names, they test heap dumping using jmap and a number of JVM flags. As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. webrev: http://cr.openjdk.java.net/~iignatyev//8202392/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8202392 testing: :vmTestbase_vm_heapdump test group Thanks, -- Igor From daniel.daugherty at oracle.com Thu May 10 12:52:01 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 10 May 2018 08:52:01 -0400 Subject: [11] RFR(XS) 8202773: Unhandled oop in JavaThread::collect_counters In-Reply-To: <4376e8c6-2e77-1928-e159-ebd4ded45fe1@oracle.com> References: <4376e8c6-2e77-1928-e159-ebd4ded45fe1@oracle.com> Message-ID: <76ebc597-6e85-f487-b9cb-faea35f92ef2@oracle.com> On 5/9/18 7:47 PM, Vladimir Kozlov wrote: > https://bugs.openjdk.java.net/browse/JDK-8202773 > > Based on discussion in bug report MutexLocker should be removed from > this code: Thumbs up. Dan > > diff -r ae0ebd3cf949 src/hotspot/share/runtime/thread.cpp > --- a/src/hotspot/share/runtime/thread.cpp > +++ b/src/hotspot/share/runtime/thread.cpp > @@ -1467,7 +1467,6 @@ > > ?void JavaThread::collect_counters(typeArrayOop array) { > ?? if (JVMCICounterSize > 0) { > -??? MutexLocker tl(Threads_lock); > ???? JavaThreadIteratorWithHandle jtiwh; > ???? for (int i = 0; i < array->length(); i++) { > ?????? array->long_at_put(i, _jvmci_old_thread_counters[i]); > > Tested with tier1,tier2,tier2-graal,hs-precheckin-comp > From vladimir.kozlov at oracle.com Thu May 10 16:21:36 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 10 May 2018 09:21:36 -0700 Subject: [11] RFR(XS) 8202773: Unhandled oop in JavaThread::collect_counters In-Reply-To: <76ebc597-6e85-f487-b9cb-faea35f92ef2@oracle.com> References: <4376e8c6-2e77-1928-e159-ebd4ded45fe1@oracle.com> <76ebc597-6e85-f487-b9cb-faea35f92ef2@oracle.com> Message-ID: <941e52b4-c8e5-16a3-dba0-b1524fc7a515@oracle.com> Thank you, Dan Vladimir K On 5/10/18 5:52 AM, Daniel D. Daugherty wrote: > On 5/9/18 7:47 PM, Vladimir Kozlov wrote: >> https://bugs.openjdk.java.net/browse/JDK-8202773 >> >> Based on discussion in bug report MutexLocker should be removed from >> this code: > > Thumbs up. > > Dan > > >> >> diff -r ae0ebd3cf949 src/hotspot/share/runtime/thread.cpp >> --- a/src/hotspot/share/runtime/thread.cpp >> +++ b/src/hotspot/share/runtime/thread.cpp >> @@ -1467,7 +1467,6 @@ >> >> ?void JavaThread::collect_counters(typeArrayOop array) { >> ?? if (JVMCICounterSize > 0) { >> -??? MutexLocker tl(Threads_lock); >> ???? JavaThreadIteratorWithHandle jtiwh; >> ???? for (int i = 0; i < array->length(); i++) { >> ?????? array->long_at_put(i, _jvmci_old_thread_counters[i]); >> >> Tested with tier1,tier2,tier2-graal,hs-precheckin-comp >> > From gerard.ziemski at oracle.com Thu May 10 17:55:05 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Thu, 10 May 2018 12:55:05 -0500 Subject: RFR(L): 8195098: Low latency hashtable for read-mostly scenarios In-Reply-To: <8defc9dd-e908-1bd5-8fdb-929905773ab0@oracle.com> References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <8defc9dd-e908-1bd5-8fdb-929905773ab0@oracle.com> Message-ID: hi Robbin, There are 2 minor issues that caught my eye, anything else we can fix in followup issues: #1 // This method looks on the _invisible_epoch field and // does a write_synchronize if needed. void write_synchonize_on_visible_epoch(Thread* thread); a) I think in the word ?looks? in the comment should be ?locks? b) Why is the method name using word ?visible? when it locks on an ?invisible? epoch? #2 ScopedCS and MultiGetHandle look identical? // ScopedCS template inline ConcurrentHashTable:: ScopedCS::ScopedCS(Thread* thread, ConcurrentHashTable* cht) : _thread(thread), _cht(cht) { GlobalCounter::critical_section_begin(_thread); // This version is published now. if (OrderAccess::load_acquire(&_cht->_invisible_epoch) != NULL) { OrderAccess::release_store_fence(&_cht->_invisible_epoch, (Thread*)NULL); } } template inline ConcurrentHashTable:: ScopedCS::~ScopedCS() { GlobalCounter::critical_section_end(_thread); } // MultiGetHandle template inline ConcurrentHashTable:: MultiGetHandle::MultiGetHandle(Thread* thread, ConcurrentHashTable* cht) : _thread(thread), _cht(cht) { GlobalCounter::critical_section_begin(_thread); // This version is published now. if (OrderAccess::load_acquire(&_cht->_invisible_epoch) != NULL) { OrderAccess::release_store_fence(&_cht->_invisible_epoch, (Thread*)NULL); } } template inline ConcurrentHashTable:: MultiGetHandle::~MultiGetHandle() { GlobalCounter::critical_section_end(_thread); } cheers > On May 2, 2018, at 4:03 AM, Robbin Ehn wrote: > > Hi all, > > Here is an update with Gerard's and Coleen's input. > > Inc: > http://cr.openjdk.java.net/~rehn/8195098/v1/inc/webrev/ > Full: > http://cr.openjdk.java.net/~rehn/8195098/v1/full/webrev/ > > Thanks, Robbin From christoph.langer at sap.com Thu May 10 21:11:55 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 10 May 2018 21:11:55 +0000 Subject: [11] RFR for JDK-8202544: Hide unused exports in libzip In-Reply-To: <426447b2-974e-bbf6-e1df-3ce235930252@oracle.com> References: <4b0e27bb-8167-4ae5-c5e5-6ec054015d30@oracle.com> <426447b2-974e-bbf6-e1df-3ce235930252@oracle.com> Message-ID: <4cfd517954d1462e9c8e4fe5f34aada5@sap.com> Hi Alexey, looks good to me. Symbols don't seem to be needed outside libzip (java.base). Best regards Christoph > -----Original Message----- > From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On Behalf Of > Alexey Ivanov > Sent: Mittwoch, 9. Mai 2018 16:35 > To: core-libs ; hotspot-dev dev at openjdk.java.net> > Cc: build-dev > Subject: Re: [11] RFR for JDK-8202544: Hide unused exports in libzip > > Any volunteers from core-libs and/or hotspot? > > Thank you, > Alexey > > On 02/05/2018 13:02, Magnus Ihse Bursie wrote: > > Looks good to me, FWIW. > > > > /Magnus > > > >> 2 maj 2018 kl. 13:52 skrev Alexey Ivanov : > >> > >> Hi, > >> > >> Could you please review the following fix for jdk11? > >> > >> bug: https://bugs.openjdk.java.net/browse/JDK-8202544 > >> webrev: http://cr.openjdk.java.net/~aivanov/8202544/jdk11/webrev.0/ > >> > >> The following exported functions in libzip are not used: > >> ZIP_GetEntry, ZIP_FreeEntry, ZIP_Lock, ZIP_Unlock, ZIP_Read > >> > >> I removed JNIEXPORT modifiers from these functions. With the fix, > they're not exported on Windows; on Linux they're listed as Local rather than > Global. > >> > >> I ran tests, no failures. > >> > >> > >> Thank you in advance. > >> > >> Regards, > >> Alexey From per.liden at oracle.com Fri May 11 08:47:59 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 11 May 2018 10:47:59 +0200 Subject: RFR: 8202989: Add missing decorators in calls to to arraycopy_prologue/epilogue Message-ID: Calls to BarrierSetAssambler::arraycopy_prologue/epilogue() are missing the IN_HEAP and IN_HEAP_ARRAY access decorators. This is sort of ok, since the context (we're doing an arraycopy) imply these. However, by not explicitly specifying them we loose the ability to do straight forward checks/asserts on expected/allowed decorators later on in the GC specific BarrierSetAssermbler backends. To avoid having the backends know about this implicit relationship I propose that we explicitly specify them at the call-sites. Bug: https://bugs.openjdk.java.net/browse/JDK-8202989 Webrev: http://cr.openjdk.java.net/~pliden/8202989/webrev.0 Testing: hs-tier{1,2} /Per From aph at redhat.com Fri May 11 09:09:14 2018 From: aph at redhat.com (Andrew Haley) Date: Fri, 11 May 2018 10:09:14 +0100 Subject: Fwd: [Mach5] mach5-one-aph-JDK-8185505-3-20180510-1725-21972: Build tasks UNSTABLE. Test tasks UNSTABLE. Failed tests: 1 In-Reply-To: <1914840614.174.1525975079929.JavaMail.root@sca00lvx.us.oracle.com> References: <1914840614.174.1525975079929.JavaMail.root@sca00lvx.us.oracle.com> Message-ID: <7cc34dff-d73f-d65e-258d-2b6bf7cc4d9d@redhat.com> This is the AArch64 AOT patch. So, it looks like I broke something. I'd appreciate it if a kind Oracle insider would give me a clue about what I broke. Thank you. Andrew. From per.liden at oracle.com Fri May 11 11:16:52 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 11 May 2018 13:16:52 +0200 Subject: RFR: 8202993: Add support for x86 testptr/testq with register and address Message-ID: <3afb8fa8-ae8d-1e8c-b042-fb61d0b9e9f3@oracle.com> On x86, we currently only have MacroAssembler::testptr() overloads for testing registers and immediate values. In ZGC, we need a version taking a register and an address. Bug: https://bugs.openjdk.java.net/browse/JDK-8202993 Webrev: http://cr.openjdk.java.net/~pliden/8202993/webrev.0 /Per From alexey.ivanov at oracle.com Fri May 11 11:20:41 2018 From: alexey.ivanov at oracle.com (Alexey Ivanov) Date: Fri, 11 May 2018 12:20:41 +0100 Subject: [11] RFR for JDK-8202544: Hide unused exports in libzip In-Reply-To: <4cfd517954d1462e9c8e4fe5f34aada5@sap.com> References: <4b0e27bb-8167-4ae5-c5e5-6ec054015d30@oracle.com> <426447b2-974e-bbf6-e1df-3ce235930252@oracle.com> <4cfd517954d1462e9c8e4fe5f34aada5@sap.com> Message-ID: <168390df-4124-6a92-0d30-0f34f6a48815@oracle.com> Hi Christoph, Thank you for your review. I was checking the changeset before pushing and noticed I had forgotten to remove JNIEXPORT modifier from ZIP_GetEntry in zip_util.h. Here's the updated webrev: http://cr.openjdk.java.net/~aivanov/8202544/jdk11/webrev.1/ The only difference with previous one is in this snippet in zip_util.h: -JNIEXPORT jzentry * +jzentry * ?ZIP_GetEntry(jzfile *zip, char *name, jint ulen); Thank you! Regards, Alexey On 10/05/2018 22:11, Langer, Christoph wrote: > Hi Alexey, > > looks good to me. Symbols don't seem to be needed outside libzip (java.base). > > Best regards > Christoph > >> -----Original Message----- >> From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On Behalf Of >> Alexey Ivanov >> Sent: Mittwoch, 9. Mai 2018 16:35 >> To: core-libs ; hotspot-dev > dev at openjdk.java.net> >> Cc: build-dev >> Subject: Re: [11] RFR for JDK-8202544: Hide unused exports in libzip >> >> Any volunteers from core-libs and/or hotspot? >> >> Thank you, >> Alexey >> >> On 02/05/2018 13:02, Magnus Ihse Bursie wrote: >>> Looks good to me, FWIW. >>> >>> /Magnus >>> >>>> 2 maj 2018 kl. 13:52 skrev Alexey Ivanov : >>>> >>>> Hi, >>>> >>>> Could you please review the following fix for jdk11? >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8202544 >>>> webrev: http://cr.openjdk.java.net/~aivanov/8202544/jdk11/webrev.0/ >>>> >>>> The following exported functions in libzip are not used: >>>> ZIP_GetEntry, ZIP_FreeEntry, ZIP_Lock, ZIP_Unlock, ZIP_Read >>>> >>>> I removed JNIEXPORT modifiers from these functions. With the fix, >> they're not exported on Windows; on Linux they're listed as Local rather than >> Global. >>>> I ran tests, no failures. >>>> >>>> >>>> Thank you in advance. >>>> >>>> Regards, >>>> Alexey From david.holmes at oracle.com Fri May 11 11:27:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 11 May 2018 21:27:03 +1000 Subject: Fwd: [Mach5] mach5-one-aph-JDK-8185505-3-20180510-1725-21972: Build tasks UNSTABLE. Test tasks UNSTABLE. Failed tests: 1 In-Reply-To: <7cc34dff-d73f-d65e-258d-2b6bf7cc4d9d@redhat.com> References: <1914840614.174.1525975079929.JavaMail.root@sca00lvx.us.oracle.com> <7cc34dff-d73f-d65e-258d-2b6bf7cc4d9d@redhat.com> Message-ID: Hi Andrew, On 11/05/2018 7:09 PM, Andrew Haley wrote: > This is the AArch64 AOT patch. So, it looks like I broke something. I'd > appreciate it if a kind Oracle insider would give me a clue about what I > broke. Thank you. I'm not sure exactly how the submit repo is supposed to be used but it looks like what you submitted is not current and doesn't include the jvmFlags changes from changeset: 49857:31e07291ae29 user: gziemski date: Mon Apr 23 10:59:39 2018 -0500 summary: 8081519: Split globals.hpp to factor out the Flag class It seems your version of the open repo gets combined with the current version of our closed repo, and the build fails because the closed files have includes for runtime/flags/jvmFlag.hpp and it doesn't exist in your open repo. So your open builds all passed, but all the closed builds failed. And no tests were run because they all operate on the closed builds. Cheers, David > Andrew. > From aph at redhat.com Fri May 11 12:06:33 2018 From: aph at redhat.com (Andrew Haley) Date: Fri, 11 May 2018 13:06:33 +0100 Subject: Fwd: [Mach5] mach5-one-aph-JDK-8185505-3-20180510-1725-21972: Build tasks UNSTABLE. Test tasks UNSTABLE. Failed tests: 1 In-Reply-To: References: <1914840614.174.1525975079929.JavaMail.root@sca00lvx.us.oracle.com> <7cc34dff-d73f-d65e-258d-2b6bf7cc4d9d@redhat.com> Message-ID: On 05/11/2018 12:27 PM, David Holmes wrote: > On 11/05/2018 7:09 PM, Andrew Haley wrote: >> This is the AArch64 AOT patch. So, it looks like I broke something. I'd >> appreciate it if a kind Oracle insider would give me a clue about what I >> broke. Thank you. > > I'm not sure exactly how the submit repo is supposed to be used but it > looks like what you submitted is not current and doesn't include the > jvmFlags changes from > > changeset: 49857:31e07291ae29 > user: gziemski > date: Mon Apr 23 10:59:39 2018 -0500 > summary: 8081519: Split globals.hpp to factor out the Flag class > > It seems your version of the open repo gets combined with the current > version of our closed repo, and the build fails because the closed files > have includes for runtime/flags/jvmFlag.hpp and it doesn't exist in your > open repo. > > So your open builds all passed, but all the closed builds failed. And no > tests were run because they all operate on the closed builds. Oh, wow. I would surely not have guessed that. I'll have a look at the submit repo to try to find out why it is so far behind. Thank you very much for helping. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From robbin.ehn at oracle.com Fri May 11 12:15:09 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 11 May 2018 14:15:09 +0200 Subject: [11] RFR(XS) 8202773: Unhandled oop in JavaThread::collect_counters In-Reply-To: <4376e8c6-2e77-1928-e159-ebd4ded45fe1@oracle.com> References: <4376e8c6-2e77-1928-e159-ebd4ded45fe1@oracle.com> Message-ID: <19caab92-c114-5450-6510-d6eb2b6d366f@oracle.com> Looks good, thanks for fixing! /Robbin On 2018-05-10 01:47, Vladimir Kozlov wrote: > https://bugs.openjdk.java.net/browse/JDK-8202773 > > Based on discussion in bug report MutexLocker should be removed from this code: > > diff -r ae0ebd3cf949 src/hotspot/share/runtime/thread.cpp > --- a/src/hotspot/share/runtime/thread.cpp > +++ b/src/hotspot/share/runtime/thread.cpp > @@ -1467,7 +1467,6 @@ > > ?void JavaThread::collect_counters(typeArrayOop array) { > ?? if (JVMCICounterSize > 0) { > -??? MutexLocker tl(Threads_lock); > ???? JavaThreadIteratorWithHandle jtiwh; > ???? for (int i = 0; i < array->length(); i++) { > ?????? array->long_at_put(i, _jvmci_old_thread_counters[i]); > > Tested with tier1,tier2,tier2-graal,hs-precheckin-comp > From vladimir.kozlov at oracle.com Fri May 11 12:21:49 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 11 May 2018 05:21:49 -0700 Subject: [11] RFR(XS) 8202773: Unhandled oop in JavaThread::collect_counters In-Reply-To: <19caab92-c114-5450-6510-d6eb2b6d366f@oracle.com> References: <4376e8c6-2e77-1928-e159-ebd4ded45fe1@oracle.com> <19caab92-c114-5450-6510-d6eb2b6d366f@oracle.com> Message-ID: Thank you, Robbin, for review. Vladimir K On 5/11/18 5:15 AM, Robbin Ehn wrote: > Looks good, thanks for fixing! > > /Robbin > > On 2018-05-10 01:47, Vladimir Kozlov wrote: >> https://bugs.openjdk.java.net/browse/JDK-8202773 >> >> Based on discussion in bug report MutexLocker should be removed from this code: >> >> diff -r ae0ebd3cf949 src/hotspot/share/runtime/thread.cpp >> --- a/src/hotspot/share/runtime/thread.cpp >> +++ b/src/hotspot/share/runtime/thread.cpp >> @@ -1467,7 +1467,6 @@ >> >> ??void JavaThread::collect_counters(typeArrayOop array) { >> ??? if (JVMCICounterSize > 0) { >> -??? MutexLocker tl(Threads_lock); >> ????? JavaThreadIteratorWithHandle jtiwh; >> ????? for (int i = 0; i < array->length(); i++) { >> ??????? array->long_at_put(i, _jvmci_old_thread_counters[i]); >> >> Tested with tier1,tier2,tier2-graal,hs-precheckin-comp >> From vladimir.kozlov at oracle.com Fri May 11 12:25:31 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 11 May 2018 05:25:31 -0700 Subject: RFR: 8202993: Add support for x86 testptr/testq with register and address In-Reply-To: <3afb8fa8-ae8d-1e8c-b042-fb61d0b9e9f3@oracle.com> References: <3afb8fa8-ae8d-1e8c-b042-fb61d0b9e9f3@oracle.com> Message-ID: <842bb81b-6f17-c633-c764-12de35c43ccd@oracle.com> Looks good. Thanks, Vladimir On 5/11/18 4:16 AM, Per Liden wrote: > On x86, we currently only have MacroAssembler::testptr() overloads for testing registers and immediate values. In ZGC, > we need a version taking a register and an address. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202993 > Webrev: http://cr.openjdk.java.net/~pliden/8202993/webrev.0 > > /Per From christoph.langer at sap.com Fri May 11 12:33:46 2018 From: christoph.langer at sap.com (Langer, Christoph) Date: Fri, 11 May 2018 12:33:46 +0000 Subject: [11] RFR for JDK-8202544: Hide unused exports in libzip In-Reply-To: <168390df-4124-6a92-0d30-0f34f6a48815@oracle.com> References: <4b0e27bb-8167-4ae5-c5e5-6ec054015d30@oracle.com> <426447b2-974e-bbf6-e1df-3ce235930252@oracle.com> <4cfd517954d1462e9c8e4fe5f34aada5@sap.com> <168390df-4124-6a92-0d30-0f34f6a48815@oracle.com> Message-ID: <6f9e6834441241b4869369c28dc1e546@sap.com> Hi Alexey, good catch, I missed that. Best regards Christoph > -----Original Message----- > From: Alexey Ivanov [mailto:alexey.ivanov at oracle.com] > Sent: Freitag, 11. Mai 2018 13:21 > To: Langer, Christoph ; core-libs dev at openjdk.java.net>; hotspot-dev ; > build-dev ; Magnus Ihse Bursie > > Subject: Re: [11] RFR for JDK-8202544: Hide unused exports in libzip > > Hi Christoph, > > Thank you for your review. > > I was checking the changeset before pushing and noticed I had forgotten > to remove JNIEXPORT modifier from ZIP_GetEntry in zip_util.h. > > Here's the updated webrev: > http://cr.openjdk.java.net/~aivanov/8202544/jdk11/webrev.1/ > > The only difference with previous one is in this snippet in zip_util.h: > > -JNIEXPORT jzentry * > +jzentry * > ?ZIP_GetEntry(jzfile *zip, char *name, jint ulen); > > > Thank you! > > Regards, > Alexey > > On 10/05/2018 22:11, Langer, Christoph wrote: > > Hi Alexey, > > > > looks good to me. Symbols don't seem to be needed outside libzip > (java.base). > > > > Best regards > > Christoph > > > >> -----Original Message----- > >> From: build-dev [mailto:build-dev-bounces at openjdk.java.net] On Behalf > Of > >> Alexey Ivanov > >> Sent: Mittwoch, 9. Mai 2018 16:35 > >> To: core-libs ; hotspot-dev >> dev at openjdk.java.net> > >> Cc: build-dev > >> Subject: Re: [11] RFR for JDK-8202544: Hide unused exports in libzip > >> > >> Any volunteers from core-libs and/or hotspot? > >> > >> Thank you, > >> Alexey > >> > >> On 02/05/2018 13:02, Magnus Ihse Bursie wrote: > >>> Looks good to me, FWIW. > >>> > >>> /Magnus > >>> > >>>> 2 maj 2018 kl. 13:52 skrev Alexey Ivanov : > >>>> > >>>> Hi, > >>>> > >>>> Could you please review the following fix for jdk11? > >>>> > >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8202544 > >>>> webrev: > http://cr.openjdk.java.net/~aivanov/8202544/jdk11/webrev.0/ > >>>> > >>>> The following exported functions in libzip are not used: > >>>> ZIP_GetEntry, ZIP_FreeEntry, ZIP_Lock, ZIP_Unlock, ZIP_Read > >>>> > >>>> I removed JNIEXPORT modifiers from these functions. With the fix, > >> they're not exported on Windows; on Linux they're listed as Local rather > than > >> Global. > >>>> I ran tests, no failures. > >>>> > >>>> > >>>> Thank you in advance. > >>>> > >>>> Regards, > >>>> Alexey From per.liden at oracle.com Fri May 11 12:40:47 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 11 May 2018 14:40:47 +0200 Subject: RFR: 8202993: Add support for x86 testptr/testq with register and address In-Reply-To: <842bb81b-6f17-c633-c764-12de35c43ccd@oracle.com> References: <3afb8fa8-ae8d-1e8c-b042-fb61d0b9e9f3@oracle.com> <842bb81b-6f17-c633-c764-12de35c43ccd@oracle.com> Message-ID: <383e707a-ced0-00af-4078-7cdd574cb854@oracle.com> Thanks for reviewing, Vladimir! /Per On 05/11/2018 02:25 PM, Vladimir Kozlov wrote: > Looks good. > > Thanks, > Vladimir > > On 5/11/18 4:16 AM, Per Liden wrote: >> On x86, we currently only have MacroAssembler::testptr() overloads for >> testing registers and immediate values. In ZGC, we need a version >> taking a register and an address. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202993 >> Webrev: http://cr.openjdk.java.net/~pliden/8202993/webrev.0 >> >> /Per From mikhailo.seledtsov at oracle.com Fri May 11 13:52:42 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Fri, 11 May 2018 06:52:42 -0700 Subject: RFR(M) : 8202392: [TESTBUG] open source vm testbase heapdump tests In-Reply-To: References: Message-ID: <5AF5A02A.7010107@oracle.com> Looks good to me, Misha On 5/9/18, 5:14 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8202392/webrev.00/index.html >> 1396 lines changed: 1396 ins; 0 del; 0 mod; > Hi all, > > could you please review his patch which open sources heapdump tests from so-called VM testbase? as it's obvious from test names, they test heap dumping using jmap and a number of JVM flags. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > webrev: http://cr.openjdk.java.net/~iignatyev//8202392/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8202392 > testing: :vmTestbase_vm_heapdump test group > > Thanks, > -- Igor From mikhailo.seledtsov at oracle.com Fri May 11 16:10:35 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Fri, 11 May 2018 09:10:35 -0700 Subject: 8199271: [TESTBUG] open source VM testbase stress tests In-Reply-To: <1BC403EC-BDF0-4CA2-B87F-B38A92B6765F@oracle.com> References: <1BC403EC-BDF0-4CA2-B87F-B38A92B6765F@oracle.com> Message-ID: <5AF5C07B.1080909@oracle.com> Looks good to me, Misha On 5/8/18, 2:23 PM, Leonid Mesnik wrote: > Hi > > Please review this change open sourcing vm testbase stress tests. These tests have been developed a long time ago for internal test harness and don't looks very nice now. > They are open sourced with minimal changes only. The long term plan is to review and improve them moving out of vmTestbase directory in corresponding components. > > The fix introduce vmTestbase_nsk_stress test group and have make files changes required to build native part of tests. > > I've verified that "make -- run-test TEST=:vmTestbase_nsk_stress" pass on my linux locally. > > webrev: http://cr.openjdk.java.net/~lmesnik/8199271/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8199271 From jcbeyler at google.com Fri May 11 16:27:31 2018 From: jcbeyler at google.com (JC Beyler) Date: Fri, 11 May 2018 09:27:31 -0700 Subject: RFR(L) : 8199375 : [TESTBUG] Open source vm testbase monitoring tests In-Reply-To: <8fd2e124-47c9-98ff-90e8-104f1a78ed28@oracle.com> References: <0BDC93A6-7D0E-4B4B-A797-936A58A296B9@oracle.com> <8fd2e124-47c9-98ff-90e8-104f1a78ed28@oracle.com> Message-ID: Just wanted to chime in and say that this is great, the more the tests are all in the open, the easier it will be for all to be able to test what is needed and not be surprised by closed test failures :) So thank you for doing this and for the other open sourcing of tests efforts! Jc On Wed, May 2, 2018 at 2:19 PM serguei.spitsyn at oracle.com < serguei.spitsyn at oracle.com> wrote: > Hi Igor, > > It looks good. > > Thanks, > Serguei > > > On 5/1/18 19:10, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8199375/webrev.00/index.html > >> 41276 lines changed: 41274 ins; 1 del; 1 mod; > > Hi all, > > > > could you please review the patch which open sources monitoring tests > from vm testbase? > > > > The tests were developed to test hotspot related JMX functionality. as > w/ common VM testbase code, these tests are old, they have been run from > hotspot testing for a long period of time. originally, these tests were run > by a test harness different from jtreg and had different build and > execution schemes, some parts couldn't be easily translated to jtreg, so > tests might have actions or pieces of code which look weird. in a long > term, we are planning to rework them. > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199375 > > webrev: > http://cr.openjdk.java.net/~iignatyev/8199375/webrev.00/index.html > > testing: vmTestbase_nsk_monitoring test group > > > > Thanks, > > -- Igor > > From robbin.ehn at oracle.com Fri May 11 19:54:22 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 11 May 2018 21:54:22 +0200 Subject: RFR(L): 8195098: Low latency hashtable for read-mostly scenarios In-Reply-To: References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <8defc9dd-e908-1bd5-8fdb-929905773ab0@oracle.com> Message-ID: <20d4cc82-37b1-f449-b53f-9fb68059942f@oracle.com> Hi Gerard, On 2018-05-10 19:55, Gerard Ziemski wrote: > hi Robbin, > > There are 2 minor issues that caught my eye, anything else we can fix in followup issues: > > #1 > > // This method looks on the _invisible_epoch field and > // does a write_synchronize if needed. > void write_synchonize_on_visible_epoch(Thread* thread); > > > a) I think in the word ?looks? in the comment should be ?locks? I don't like lock, sine it implies mutual exclusiveness. I added an explanation, hopefully you like it. > b) Why is the method name using word ?visible? when it locks on an ?invisible? epoch? The method only does write_synchronize if the the 'epoch'/generation was seen by another thread. Hence write_synchonize_on_visible_epoch, e.g. it only does wite_synchronize if hashtable state was publish to another thread. Hopefully my explanation added above makes it much clearer. > > > #2 ScopedCS and MultiGetHandle look identical? ScopedCS is an internal class only and MultiGetHandle an interface class. But since they provide same type scoped with identical code I let MultiGetHandle inherit from ScopedCS instead and removed the code duplication, thanks! Please see incremental: http://cr.openjdk.java.net/~rehn/8195098/v3/inc/ For reference full: http://cr.openjdk.java.net/~rehn/8195098/v3/full/webrev/ Thanks, Robbin > > // ScopedCS > template > inline ConcurrentHashTable:: > ScopedCS::ScopedCS(Thread* thread, ConcurrentHashTable* cht) > : _thread(thread), _cht(cht) > { > GlobalCounter::critical_section_begin(_thread); > // This version is published now. > if (OrderAccess::load_acquire(&_cht->_invisible_epoch) != NULL) { > OrderAccess::release_store_fence(&_cht->_invisible_epoch, (Thread*)NULL); > } > } > > template > inline ConcurrentHashTable:: > ScopedCS::~ScopedCS() > { > GlobalCounter::critical_section_end(_thread); > } > > // MultiGetHandle > template > inline ConcurrentHashTable:: > MultiGetHandle::MultiGetHandle(Thread* thread, > ConcurrentHashTable* cht) > : _thread(thread), _cht(cht) > { > GlobalCounter::critical_section_begin(_thread); > // This version is published now. > if (OrderAccess::load_acquire(&_cht->_invisible_epoch) != NULL) { > OrderAccess::release_store_fence(&_cht->_invisible_epoch, (Thread*)NULL); > } > } > > template > inline ConcurrentHashTable:: > MultiGetHandle::~MultiGetHandle() > { > GlobalCounter::critical_section_end(_thread); > } > > > cheers > >> On May 2, 2018, at 4:03 AM, Robbin Ehn wrote: >> >> Hi all, >> >> Here is an update with Gerard's and Coleen's input. >> >> Inc: >> http://cr.openjdk.java.net/~rehn/8195098/v1/inc/webrev/ >> Full: >> http://cr.openjdk.java.net/~rehn/8195098/v1/full/webrev/ >> >> Thanks, Robbin > From serguei.spitsyn at oracle.com Sat May 12 01:10:57 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 11 May 2018 18:10:57 -0700 Subject: RFR(M) : 8202392: [TESTBUG] open source vm testbase heapdump tests In-Reply-To: <5AF5A02A.7010107@oracle.com> References: <5AF5A02A.7010107@oracle.com> Message-ID: +1 Thanks, Serguei On 5/11/18 06:52, Mikhailo Seledtsov wrote: > Looks good to me, > > Misha > > On 5/9/18, 5:14 PM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8202392/webrev.00/index.html >>> 1396 lines changed: 1396 ins; 0 del; 0 mod; >> Hi all, >> >> could you please review his patch which open sources heapdump tests >> from so-called VM testbase? as it's obvious from test names, they >> test heap dumping using jmap and a number of JVM flags. >> >> As usually w/ VM testbase code, these tests are old, they have been >> run in hotspot testing for a long period of time. Originally, these >> tests were run by a test harness different from jtreg and had >> different build and execution schemes, some parts couldn't be easily >> translated to jtreg, so tests might have actions or pieces of code >> which look weird. In a long term, we are planning to rework them. >> >> webrev: >> http://cr.openjdk.java.net/~iignatyev//8202392/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8202392 >> testing: :vmTestbase_vm_heapdump test group >> >> Thanks, >> -- Igor From per.liden at oracle.com Mon May 14 06:24:51 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 14 May 2018 08:24:51 +0200 Subject: RFR: 8202479: Add missing try_resolve_jobject_in_native calls In-Reply-To: <5AE88118.7080203@oracle.com> References: <5AE88118.7080203@oracle.com> Message-ID: <0e980a41-0481-c41b-7f4b-743f25916488@oracle.com> On 05/01/2018 05:00 PM, Erik ?sterlund wrote: > Hi, > > There are some missing calls to try_resolve_jobject_in_native for the > jni fast get field optimization. > > On x86_64, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT and > T_LONG, but is missing for T_FLOAT and T_DOUBLE. > On SPARC, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT, but > is missing for T_LONG, T_FLOAT and T_DOUBLE. > On AArch64, it is used for all types. > > Here is a patch to add the missing calls to try_resolve_jobject_in_native. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8202479/webrev.00/ Looks good to me. /Per > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8202479 > > Thanks, > /Erik From erik.osterlund at oracle.com Mon May 14 07:00:18 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Mon, 14 May 2018 08:00:18 +0100 Subject: RFR: 8202479: Add missing try_resolve_jobject_in_native calls In-Reply-To: <0e980a41-0481-c41b-7f4b-743f25916488@oracle.com> References: <5AE88118.7080203@oracle.com> <0e980a41-0481-c41b-7f4b-743f25916488@oracle.com> Message-ID: Hi Per, Thanks for the review! /Erik > On 14 May 2018, at 07:24, Per Liden wrote: > >> On 05/01/2018 05:00 PM, Erik ?sterlund wrote: >> Hi, >> There are some missing calls to try_resolve_jobject_in_native for the jni fast get field optimization. >> On x86_64, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT and T_LONG, but is missing for T_FLOAT and T_DOUBLE. >> On SPARC, it is used for T_BOOLEAN, T_BYTE, T_CHAR, T_SHORT, T_INT, but is missing for T_LONG, T_FLOAT and T_DOUBLE. >> On AArch64, it is used for all types. >> Here is a patch to add the missing calls to try_resolve_jobject_in_native. >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8202479/webrev.00/ > > Looks good to me. > > /Per > >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8202479 >> Thanks, >> /Erik From robbin.ehn at oracle.com Mon May 14 09:12:57 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 14 May 2018 11:12:57 +0200 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: References: Message-ID: <5cdb2496-bad5-fa4a-f98e-37ae6a853f6b@oracle.com> Hi JC, I found a .19 which I looked at: src/hotspot/share/gc/shared/collectedHeap.inline.hpp CollectedHeap::allocate_memory This is the only place I found which calls the ~JvmtiSampledObjectAllocEventCollector It is not intuitive with creating a handle for the destructor, I suggest something like collector.sample(THREAD, obj_h); instead. open/src/hotspot/share/runtime/threadHeapSampler.hpp Don't include inline.hpp in hpp. This means you need to move the two methods using orderAccess to cpp (or a inline.hpp). As general note, not your doing, setting a pointer in a heap allocated object to a stack allocated object is a really bad pattern. JvmtiThreadState -> collector Thanks, Robbin On 05/08/2018 03:10 AM, JC Beyler wrote: > Hi all, > > With the awesome help of Serguei Spitsyn, we have moved forward on the > implementation for JEP-331 and have the following webrev for review: > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.18/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8171119 > > It is based on jdk/jdk so should patch well with a recent tip. > > Could we please have some reviews for the webrev? It would be greatly > appreciated! > > Thanks for all your help! > Jc > From rkennke at redhat.com Mon May 14 10:15:52 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 14 May 2018 12:15:52 +0200 Subject: RFR: JDK-8202016: Use obj+offset in interpreter array access In-Reply-To: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> References: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> Message-ID: <041699e9-9c5b-0a8b-1b94-45a181018b47@redhat.com> Am 07.05.2018 um 21:47 schrieb Roman Kennke: > In the TemplateTable::aastore() and > InterpreterMacroAssembler::load_resolved_reference_at_index(), the > element address is flattened, and then passed to the BarrierSetAssembler > for actual access. GCs like Shenandoah need the original obj though. > > The proposed change replaces the flattening with base+index+disp > addressing, and removes the shift and add computations. In the case of > aastore, we need to re-fetch the rcx/index from the stack because it > gets trashed on the way. > > Testing: passed: hotspot/jtreg:tier1 > > Can I please get a review? > Thanks, Roman > Ping? Thanks, Roman From rkennke at redhat.com Mon May 14 10:16:24 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 14 May 2018 12:16:24 +0200 Subject: RFR: JDK-8200623: Primitive heap access for interpreter BarrierSetAssembler/x86 In-Reply-To: References: Message-ID: <542449a6-84a5-ba69-16dd-2fd2fcd587c0@redhat.com> Am 07.05.2018 um 22:31 schrieb Roman Kennke: > JDK-8199417 added better modularization for interpreter barriers. > Shenandoah and possibly future GCs also need barriers for primitive access. > > Some notes on implementation: > - float/double/long access produced some headaches for the following > reasons: > > - float and double would either take XMMRegister which is not > compatible with Register > - or load-from/store-to the floating point stack (see > MacroAssembler::load/store_float/double) > - long access on x86_32 would load-into/store-from 2 registers, or > else use a trick via the floating point stack to do atomic access > > None of this seemed easy/nice to do with the API. I helped myself by > accepting noreg as dst/src argument, which means the corresponding tos > (i.e. ltos, ftos, dtos) and the BSA would then access from/to > xmm0/float-stack in case of float/double or the double-reg/float-stack > in case of long/32bit, which is all that we ever need. > > I'm passing MO_RELAXED to long access calls to hint that we want atomic > access or not. I hope that is ok. > > > Tested: hotspot/jtreg:tier1 > > http://cr.openjdk.java.net/~rkennke/JDK-8200623/webrev.00/ > > Can I please get a review? > > Thanks, Roman > Ping? Thanks, Roman From adam.farley at uk.ibm.com Mon May 14 10:21:29 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Mon, 14 May 2018 11:21:29 +0100 Subject: RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers Message-ID: Bump. Best Regards Adam Farley > On 13/04/2018 15:14, Adam Farley8 wrote: > >> Hi Alan, Peter, > >> > >> I see that native memory is tracked in java.nio.Bits, but that only includes what the user thinks they are allocating. > >> > >> When the VM adds extra memory to the allocation amount this extra bit is not represented in the Bits total. > >> A cursory glance shows, minimum, that we round the requested memory quantity up to the heap word size in > >> the Unsafe.allocateMemory code, and something to do with nmt_header_size in os:malloc() (os.cpp) too. > > Is the align_up(sz, HeapWordSize) really causing an issue? > > Coupled with the nmt_header_size business, it makes the Bits values wrong. The more DBB allocations, the more > inaccurate those numbers will be. > > > > > We could change Bits to align with HotSpot. The BufferPoolMXBean API allows the capacity and memory usage to differ (when we originally added this, direct buffers were page aligned) so doing this would mean it more accurately reflects the memory allocated to direct buffers. > > > > -Alan > > I agree with you that the "+x" information should be added to only one of the two atomic longs. > > To get "X", it seems to me that the best option would be to introduce an native method in Bits that fetches > "X" directly from Hotspot, using the same code that Hotspot uses (so we'd have to abstract-out the Hotspot logic > that adds X to the memory quantity). > > This way, anyone modifying the Hotspot logic won't risk rendering the Bits logic wrong again. > > Best Regards > > Adam Farley Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From adinn at redhat.com Mon May 14 11:19:17 2018 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 14 May 2018 12:19:17 +0100 Subject: RFR: JDK-8202016: Use obj+offset in interpreter array access In-Reply-To: <041699e9-9c5b-0a8b-1b94-45a181018b47@redhat.com> References: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> <041699e9-9c5b-0a8b-1b94-45a181018b47@redhat.com> Message-ID: <8b16fb83-25c4-8fcf-7ef6-13d3943f3fd7@redhat.com> On 14/05/18 11:15, Roman Kennke wrote: > Am 07.05.2018 um 21:47 schrieb Roman Kennke: >> In the TemplateTable::aastore() and >> InterpreterMacroAssembler::load_resolved_reference_at_index(), the >> element address is flattened, and then passed to the BarrierSetAssembler >> for actual access. GCs like Shenandoah need the original obj though. >> >> The proposed change replaces the flattening with base+index+disp >> addressing, and removes the shift and add computations. In the case of >> aastore, we need to re-fetch the rcx/index from the stack because it >> gets trashed on the way. >> >> Testing: passed: hotspot/jtreg:tier1 >> >> Can I please get a review? >> Thanks, Roman Well, that x86 code change looks ok/ish/ -- although the need to reload from the stack is a tad disappointing. However, I am left wondering why this is not a problem on other architectures? Indeed, looking at AArch64 I believe it is actually a serious headache. Method aastore looks like this: Address element_address(r4, arrayOopDesc::base_offset_in_bytes(T_OBJECT)); index_check(r3, r2); // kills r1 __ lea(r4, Address(r3, r2, Address::uxtw(UseCompressedOops? 2 : 3))); . . . // Get the value we will store __ ldr(r0, at_tos()); // Now store using the appropriate barrier do_oop_store(_masm, element_address, r0, IN_HEAP | IN_HEAP_ARRAY); The Address for r4 only includes a displacement and the index register offset has to be explicitly added to the base register using an lea. Of course, that's because AArch64 doesn't like an Address with both a disp and a index register. You can have one or the other but not both. So, if the abstraction you are relying on is to rely on do_oop_store to unpack an object base address, offset and index register from its Address argument then I am afraid you are on to a loser. What do you propose to do about that? and should it not be done as part of this fix? regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From erik.helin at oracle.com Mon May 14 11:23:43 2018 From: erik.helin at oracle.com (Erik Helin) Date: Mon, 14 May 2018 13:23:43 +0200 Subject: RFR(L) : 8199370: [TESTBUG] Open source vm testbase GC tests In-Reply-To: References: Message-ID: On 05/08/2018 12:35 AM, Igor Ignatyev wrote: > Hi all, Hi Igor, On 05/08/2018 12:35 AM, Igor Ignatyev wrote: > could you please review the patch which open sources GC tests from vm testbase? it introduces the following test groups: > - vmTestbase_vm_g1classunloading > - vmTestbase_vm_gc_compact > - vmTestbase_vm_gc_concurrent > - vmTestbase_vm_gc_container > - vmTestbase_vm_gc_juggle > - vmTestbase_vm_gc_locker > - vmTestbase_vm_gc_ref > - vmTestbase_vm_gc_misc This is a very welcome, and pretty massive, change :) I won't be able to read through each test, there are simple too many, but I can sample a few of them and have a look. On 05/08/2018 12:35 AM, Igor Ignatyev wrote: > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. I'm also assuming that to help the open sourcing of these tests, most comments will likely be deferred until later? If so, that is fine with me. On 05/08/2018 12:35 AM, Igor Ignatyev wrote: > JBS: https://bugs.openjdk.java.net/browse/JDK-8199370 > webrev: http://cr.openjdk.java.net/~iignatyev/8199370/webrev.00/index.html Many of the tests are a bit cryptic, but that can be refactored later. Could you please file bugs for the two tests written in shell? Particularly parOld/test.sh should be trivial to rewrite in Java. It seems like a lot of tests contains a TEST.properties file with the content `exclusiveAccess.dirs=.`. Could this become the default value somewhere, so we don't need all those TEST.properties files? Thanks, Erik > Thanks, > -- Igor > From rkennke at redhat.com Mon May 14 11:34:18 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 14 May 2018 13:34:18 +0200 Subject: RFR: JDK-8202016: Use obj+offset in interpreter array access In-Reply-To: <8b16fb83-25c4-8fcf-7ef6-13d3943f3fd7@redhat.com> References: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> <041699e9-9c5b-0a8b-1b94-45a181018b47@redhat.com> <8b16fb83-25c4-8fcf-7ef6-13d3943f3fd7@redhat.com> Message-ID: Am 14.05.2018 um 13:19 schrieb Andrew Dinn: > On 14/05/18 11:15, Roman Kennke wrote: >> Am 07.05.2018 um 21:47 schrieb Roman Kennke: >>> In the TemplateTable::aastore() and >>> InterpreterMacroAssembler::load_resolved_reference_at_index(), the >>> element address is flattened, and then passed to the BarrierSetAssembler >>> for actual access. GCs like Shenandoah need the original obj though. >>> >>> The proposed change replaces the flattening with base+index+disp >>> addressing, and removes the shift and add computations. In the case of >>> aastore, we need to re-fetch the rcx/index from the stack because it >>> gets trashed on the way. >>> >>> Testing: passed: hotspot/jtreg:tier1 >>> >>> Can I please get a review? >>> Thanks, Roman > Well, that x86 code change looks ok/ish/ -- although the need to reload > from the stack is a tad disappointing. > > However, I am left wondering why this is not a problem on other > architectures? Indeed, looking at AArch64 I believe it is actually a > serious headache. It is, and I have a solution here: cr.openjdk.java.net/~rkennke/interp_primitives_aarch64/webrev.00 I will propose this for upstream soon. Method aastore looks like this: > > Address element_address(r4, arrayOopDesc::base_offset_in_bytes(T_OBJECT)); > > index_check(r3, r2); // kills r1 > __ lea(r4, Address(r3, r2, Address::uxtw(UseCompressedOops? 2 : 3))); > . . . > > // Get the value we will store > __ ldr(r0, at_tos()); > // Now store using the appropriate barrier > do_oop_store(_masm, element_address, r0, IN_HEAP | IN_HEAP_ARRAY); > > The Address for r4 only includes a displacement and the index register > offset has to be explicitly added to the base register using an lea. Of > course, that's because AArch64 doesn't like an Address with both a disp > and a index register. You can have one or the other but not both. > > So, if the abstraction you are relying on is to rely on do_oop_store to > unpack an object base address, offset and index register from its > Address argument then I am afraid you are on to a loser. What do you > propose to do about that? and should it not be done as part of this fix? For AArch64 it will basically reshuffle address calculation from currently (base + index) + disp (which we can't resolve) to base + (index + disp) which we can. The alternative would be to explicitely drag the base_obj through all the BarrierSetAssembler calls. So far I could live without it (x86 and aarch64). Roman From rkennke at redhat.com Mon May 14 11:40:39 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 14 May 2018 13:40:39 +0200 Subject: RFR: JDK-8202776: Modularize GC allocations in runtime Message-ID: Currently, GCs only get to see (and modify) 'large' allocations, e.g. allocations of TLAB blocks (via CH::allocate_new_tlab()) or non-TLAB objects (via CH::mem_allocate()). I think GCs need to own the whole allocation path, including allocations *from* TLABs. For example, Shenandoah needs to allocate one extra word per object, and do some per-object initialization to set up the forwarding pointer. More generally speaking, I believe the interface between GC and rest of the world should just be 'allocate me a chunk of X words' and it should be totally to the GCs decretion how and where it allocates that, how objects are laid out and whatever pre- or post-processing needs to be done. For runtime we're already mostly there in the form of CollectedHeap::mem_allocate(). However, currently, TLAB allocation is done outside of this path, and GCs cannot currently control it. This patch propose to move the TLAB allocation to inside of mem_allocate(). On a more-or-less related not, it also makes CollectedHeap::fill_with_object_impl() virtual, so that GCs can intercept that too. (For example, Shenandoah needs to write a fwd ptr, and then fill with one word less.) http://cr.openjdk.java.net/~rkennke/JDK-8202776/webrev.00/ Passes: hotspot/tier1 tests Can I please get a review? Thanks, Roman From rkennke at redhat.com Mon May 14 12:02:51 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 14 May 2018 14:02:51 +0200 Subject: RFR: JDK-8202714: Create a MacroAssembler::access_load/store_at wrapper for AArch64 Message-ID: <233d81d0-d79f-dd2e-7052-6990e5654095@redhat.com> This reshuffles AArch64 code around BarrierSetAssembler calls follow the equivalent code in x86: http://cr.openjdk.java.net/~rkennke/JDK-8202714/webrev.00/ No regressions in hotspot/tier1 Can I please get a review? Thanks, Roman From nils.eliasson at oracle.com Mon May 14 11:50:14 2018 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Mon, 14 May 2018 13:50:14 +0200 Subject: RFR: 8202993: Add support for x86 testptr/testq with register and address In-Reply-To: <3afb8fa8-ae8d-1e8c-b042-fb61d0b9e9f3@oracle.com> References: <3afb8fa8-ae8d-1e8c-b042-fb61d0b9e9f3@oracle.com> Message-ID: <299595b2-9234-9604-2538-5e024d84e0c6@oracle.com> Looks good! // Nils On 2018-05-11 13:16, Per Liden wrote: > On x86, we currently only have MacroAssembler::testptr() overloads for > testing registers and immediate values. In ZGC, we need a version > taking a register and an address. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202993 > Webrev: http://cr.openjdk.java.net/~pliden/8202993/webrev.0 > > /Per From per.liden at oracle.com Mon May 14 12:15:36 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 14 May 2018 14:15:36 +0200 Subject: RFR: 8202993: Add support for x86 testptr/testq with register and address In-Reply-To: <299595b2-9234-9604-2538-5e024d84e0c6@oracle.com> References: <3afb8fa8-ae8d-1e8c-b042-fb61d0b9e9f3@oracle.com> <299595b2-9234-9604-2538-5e024d84e0c6@oracle.com> Message-ID: Thanks Nils! /Per On 05/14/2018 01:50 PM, Nils Eliasson wrote: > Looks good! > > // Nils > > > On 2018-05-11 13:16, Per Liden wrote: >> On x86, we currently only have MacroAssembler::testptr() overloads for >> testing registers and immediate values. In ZGC, we need a version >> taking a register and an address. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202993 >> Webrev: http://cr.openjdk.java.net/~pliden/8202993/webrev.0 >> >> /Per > From adinn at redhat.com Mon May 14 13:25:55 2018 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 14 May 2018 14:25:55 +0100 Subject: RFR: JDK-8202016: Use obj+offset in interpreter array access In-Reply-To: References: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> <041699e9-9c5b-0a8b-1b94-45a181018b47@redhat.com> <8b16fb83-25c4-8fcf-7ef6-13d3943f3fd7@redhat.com> Message-ID: <31370613-99ef-06cb-9241-dfc459d3b913@redhat.com> On 14/05/18 12:34, Roman Kennke wrote: > Am 14.05.2018 um 13:19 schrieb Andrew Dinn: . . . >> Well, that x86 code change looks ok/ish/ -- although the need to reload >> from the stack is a tad disappointing. >> >> However, I am left wondering why this is not a problem on other >> architectures? Indeed, looking at AArch64 I believe it is actually a >> serious headache. > > It is, and I have a solution here: > cr.openjdk.java.net/~rkennke/interp_primitives_aarch64/webrev.0 Yuck, how repulsive :] > I will propose this for upstream soon. Ok, so ack the original x86 change. > For AArch64 it will basically reshuffle address calculation from > currently (base + index) + disp (which we can't resolve) to base + > (index + disp) which we can. > > The alternative would be to explicitely drag the base_obj through all > the BarrierSetAssembler calls. So far I could live without it (x86 and > aarch64). Yes, even more repulsive, I guess. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adinn at redhat.com Mon May 14 13:31:04 2018 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 14 May 2018 14:31:04 +0100 Subject: RFR: JDK-8202714: Create a MacroAssembler::access_load/store_at wrapper for AArch64 In-Reply-To: <233d81d0-d79f-dd2e-7052-6990e5654095@redhat.com> References: <233d81d0-d79f-dd2e-7052-6990e5654095@redhat.com> Message-ID: <0c759020-5b11-8d68-f4b9-136c7247e10e@redhat.com> On 14/05/18 13:02, Roman Kennke wrote: > This reshuffles AArch64 code around BarrierSetAssembler calls follow the > equivalent code in x86: > > http://cr.openjdk.java.net/~rkennke/JDK-8202714/webrev.00/ > > No regressions in hotspot/tier1 > > Can I please get a review? Reviewed :-) regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adinn at redhat.com Mon May 14 13:35:56 2018 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 14 May 2018 14:35:56 +0100 Subject: RFR: JDK-8200623: Primitive heap access for interpreter BarrierSetAssembler/x86 In-Reply-To: <542449a6-84a5-ba69-16dd-2fd2fcd587c0@redhat.com> References: <542449a6-84a5-ba69-16dd-2fd2fcd587c0@redhat.com> Message-ID: <929fcf27-523c-0769-fa9c-93aa2d396b1e@redhat.com> On 14/05/18 11:16, Roman Kennke wrote: > Am 07.05.2018 um 22:31 schrieb Roman Kennke: >> JDK-8199417 added better modularization for interpreter barriers. >> Shenandoah and possibly future GCs also need barriers for primitive access. >> >> Some notes on implementation: >> - float/double/long access produced some headaches for the following >> reasons: >> >> - float and double would either take XMMRegister which is not >> compatible with Register >> - or load-from/store-to the floating point stack (see >> MacroAssembler::load/store_float/double) >> - long access on x86_32 would load-into/store-from 2 registers, or >> else use a trick via the floating point stack to do atomic access >> >> None of this seemed easy/nice to do with the API. I helped myself by >> accepting noreg as dst/src argument, which means the corresponding tos >> (i.e. ltos, ftos, dtos) and the BSA would then access from/to >> xmm0/float-stack in case of float/double or the double-reg/float-stack >> in case of long/32bit, which is all that we ever need. >> >> I'm passing MO_RELAXED to long access calls to hint that we want atomic >> access or not. I hope that is ok. >> >> >> Tested: hotspot/jtreg:tier1 >> >> http://cr.openjdk.java.net/~rkennke/JDK-8200623/webrev.00/ >> >> Can I please get a review? >> >> Thanks, Roman >> > > Ping? Also reviewed. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From erik.gahlin at oracle.com Mon May 14 14:36:56 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Mon, 14 May 2018 16:36:56 +0200 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: <5AE0612F.8060200@oracle.com> References: <5AE0612F.8060200@oracle.com> Message-ID: <5AF99F08.6060408@oracle.com> Here is an updated webrev: http://cr.openjdk.java.net/~egahlin/8199712.1/ [1] that incorporates: - build changes - new event prefix, i.e. "com.oracle.jdk.CPULoad" becomes "jdk.CPULoad" - obsolete command line options EnableTracing and UseLockedTracing - fixed typos in the Javadoc - simplified #include files RFEs have been filed for other issues, CSR is approved and tests pass. Erik and Markus [1] Parent: changeset: 50092:0e42d3120e51 user: clanger date: Sat May 12 10:26:42 2018 +0200 summary: 8202915: [JAXP] Performance enhancements and cleanups in com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator > Greetings, > > Could I have a review of 8199712: Flight Recorder > > As mentioned in the preview [1] the tracing backend has been removed. > Event metadata has been consolidated into a single XML file and event > classes are now generated by GenerateJfrFiles.java. > > Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. > > For details about the feature, see the JEP: > https://bugs.openjdk.java.net/browse/JDK-8193393 > > Webrev: > http://cr.openjdk.java.net/~egahlin/8199712.0/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8199712 > > [1] > http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359.html > > Thanks > Erik and Markus From jcbeyler at google.com Mon May 14 15:09:30 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 14 May 2018 08:09:30 -0700 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: <5cdb2496-bad5-fa4a-f98e-37ae6a853f6b@oracle.com> References: <5cdb2496-bad5-fa4a-f98e-37ae6a853f6b@oracle.com> Message-ID: Hi Robbin, First off: Thanks for looking! There were 3 comments here and I'll try to address all three :) >From easy to more difficult: - The thread state keeping a pointer of the collector: yes I agree but it follows the other collector implementations and with Serguei we tried to keep that implementation in sync. - Done for the orderAccess, I'll send a new webrev when we solve the next conversation: Now the hardest one (or the one that might generate the most conversation): There are now three different implementations for putting the collector in place: 1) Minimum change to the collectedHeap.inline.hpp but the collectors are not symmetrical anymore: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.15/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html -> That looks like what you had. Pro: no big change to the collectedHeap code, easy to see no overhead when disabled Con: collectors are not symmetrical anymore 2) Small change to the collectedHeap.inline.hpp and collectors remain symmetrical: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.16/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html Pro: small change all around Con: not clear that having a handle created on each slowpath does not add any overhead when disabled. 3) Bigger change to collectedHeap.inline.hpp, collectors remain symmetrical: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.19/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html That is the one that you reviewed. Pro: code is a bit bigger for the collectedHeap but you have no overhead again if disabled, the collectors remain symmetrical Con: bigger change to the collectedHeap. So what I see here is that we have to get a consensus for which implementation is better. I don't like the (2), I worry about the overhead of always doing a Handle in the slowpath. So I have a tendency to prefer (1) or (3). With Serguei, we preferred (3). What do you and the community think? Thanks again for your review! Jc On Mon, May 14, 2018 at 2:13 AM Robbin Ehn wrote: > Hi JC, I found a .19 which I looked at: > > src/hotspot/share/gc/shared/collectedHeap.inline.hpp > CollectedHeap::allocate_memory > > This is the only place I found which calls the > ~JvmtiSampledObjectAllocEventCollector > It is not intuitive with creating a handle for the destructor, I suggest > something like collector.sample(THREAD, obj_h); instead. > > open/src/hotspot/share/runtime/threadHeapSampler.hpp > Don't include inline.hpp in hpp. > This means you need to move the two methods using orderAccess to cpp > (or a inline.hpp). > > As general note, not your doing, setting a pointer in a heap allocated > object to > a stack allocated object is a really bad pattern. > JvmtiThreadState -> collector > > Thanks, Robbin > > On 05/08/2018 03:10 AM, JC Beyler wrote: > > Hi all, > > > > With the awesome help of Serguei Spitsyn, we have moved forward on the > > implementation for JEP-331 and have the following webrev for review: > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.18/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8171119 > > > > It is based on jdk/jdk so should patch well with a recent tip. > > > > Could we please have some reviews for the webrev? It would be greatly > > appreciated! > > > > Thanks for all your help! > > Jc > > > From erik.joelsson at oracle.com Mon May 14 16:05:21 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 14 May 2018 09:05:21 -0700 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: <5AF99F08.6060408@oracle.com> References: <5AE0612F.8060200@oracle.com> <5AF99F08.6060408@oracle.com> Message-ID: Oh, I missed the new makefiles last time I looked at this. in Copy-jdk.jfr.gmk, everything looks like it's indented an extra 4 steps. I'm assuming this is because it used to be conditional in the previous closed file. GensrcJfr.gmk, line 94, please move )) to the left. Looking closer at GensrcJfr.gmk, The macro SetupJfrGeneration looks like it is only called once. This could be greatly simplified by just taking the body of the macro and inlining all the inputs. This of course unless you see a need in the future to generate additional files using the jfr tool. /Erik On 2018-05-14 07:36, Erik Gahlin wrote: > Here is an updated webrev: > > http://cr.openjdk.java.net/~egahlin/8199712.1/ [1] > > that incorporates: > > - build changes > - new event prefix, i.e. "com.oracle.jdk.CPULoad" becomes "jdk.CPULoad" > - obsolete command line options EnableTracing and UseLockedTracing > - fixed typos in the Javadoc > - simplified #include files > > RFEs have been filed for other issues, CSR is approved and tests pass. > > Erik and Markus > > [1] Parent: > > changeset:?? 50092:0e42d3120e51 > > user:??????? clanger > date:??????? Sat May 12 10:26:42 2018 +0200 > summary:???? 8202915: [JAXP] Performance enhancements and cleanups in > com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator > > >> Greetings, >> >> Could I have a review of 8199712: Flight Recorder >> >> As mentioned in the preview [1] the tracing backend has been removed. >> Event metadata has been consolidated into a single XML file and event >> classes are now generated by GenerateJfrFiles.java. >> >> Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. >> >> For details about the feature, see the JEP: >> https://bugs.openjdk.java.net/browse/JDK-8193393 >> >> Webrev: >> http://cr.openjdk.java.net/~egahlin/8199712.0/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8199712 >> >> [1] >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359.html >> >> Thanks >> Erik and Markus > From gerard.ziemski at oracle.com Mon May 14 16:58:35 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Mon, 14 May 2018 11:58:35 -0500 Subject: RFR(L): 8195098: Low latency hashtable for read-mostly scenarios In-Reply-To: <20d4cc82-37b1-f449-b53f-9fb68059942f@oracle.com> References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <8defc9dd-e908-1bd5-8fdb-929905773ab0@oracle.com> <20d4cc82-37b1-f449-b53f-9fb68059942f@oracle.com> Message-ID: Thank you Robbin for addressing all my concerns and answering all my questions. I will probably have more later, but we can handle that then. Very impressive hashtable. cheers > On May 11, 2018, at 2:54 PM, Robbin Ehn wrote: > > Hi Gerard, > > On 2018-05-10 19:55, Gerard Ziemski wrote: >> hi Robbin, >> There are 2 minor issues that caught my eye, anything else we can fix in followup issues: >> #1 >> // This method looks on the _invisible_epoch field and >> // does a write_synchronize if needed. >> void write_synchonize_on_visible_epoch(Thread* thread); >> a) I think in the word ?looks? in the comment should be ?locks? > > I don't like lock, sine it implies mutual exclusiveness. I added an explanation, hopefully you like it. > >> b) Why is the method name using word ?visible? when it locks on an ?invisible? epoch? > > The method only does write_synchronize if the the 'epoch'/generation was seen by another thread. Hence write_synchonize_on_visible_epoch, e.g. it only does > wite_synchronize if hashtable state was publish to another thread. Hopefully my > explanation added above makes it much clearer. > >> #2 ScopedCS and MultiGetHandle look identical? > > ScopedCS is an internal class only and MultiGetHandle an interface class. > But since they provide same type scoped with identical code I let > MultiGetHandle inherit from ScopedCS instead and removed the code duplication, thanks! > > Please see incremental: > http://cr.openjdk.java.net/~rehn/8195098/v3/inc/ > > For reference full: > http://cr.openjdk.java.net/~rehn/8195098/v3/full/webrev/ > > Thanks, Robbin > >> // ScopedCS >> template >> inline ConcurrentHashTable:: >> ScopedCS::ScopedCS(Thread* thread, ConcurrentHashTable* cht) >> : _thread(thread), _cht(cht) >> { >> GlobalCounter::critical_section_begin(_thread); >> // This version is published now. >> if (OrderAccess::load_acquire(&_cht->_invisible_epoch) != NULL) { >> OrderAccess::release_store_fence(&_cht->_invisible_epoch, (Thread*)NULL); >> } >> } >> template >> inline ConcurrentHashTable:: >> ScopedCS::~ScopedCS() >> { >> GlobalCounter::critical_section_end(_thread); >> } >> // MultiGetHandle >> template >> inline ConcurrentHashTable:: >> MultiGetHandle::MultiGetHandle(Thread* thread, >> ConcurrentHashTable* cht) >> : _thread(thread), _cht(cht) >> { >> GlobalCounter::critical_section_begin(_thread); >> // This version is published now. >> if (OrderAccess::load_acquire(&_cht->_invisible_epoch) != NULL) { >> OrderAccess::release_store_fence(&_cht->_invisible_epoch, (Thread*)NULL); >> } >> } >> template >> inline ConcurrentHashTable:: >> MultiGetHandle::~MultiGetHandle() >> { >> GlobalCounter::critical_section_end(_thread); >> } >> cheers >>> On May 2, 2018, at 4:03 AM, Robbin Ehn wrote: >>> >>> Hi all, >>> >>> Here is an update with Gerard's and Coleen's input. >>> >>> Inc: >>> http://cr.openjdk.java.net/~rehn/8195098/v1/inc/webrev/ >>> Full: >>> http://cr.openjdk.java.net/~rehn/8195098/v1/full/webrev/ >>> >>> Thanks, Robbin From mikhailo.seledtsov at oracle.com Mon May 14 17:54:36 2018 From: mikhailo.seledtsov at oracle.com (mikhailo) Date: Mon, 14 May 2018 10:54:36 -0700 Subject: RFR(L) : 8199384 : [TESTBUG] Open source VM testbase MLVM tests In-Reply-To: <53E4CA40-DE86-41A2-A1BE-239BE1D85069@oracle.com> References: <53E4CA40-DE86-41A2-A1BE-239BE1D85069@oracle.com> Message-ID: Changes look good to me, Misha On 05/09/2018 02:09 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8199384/webrev.00/index.html >> 61414 lines changed: 61414 ins; 0 del; 0 mod; > Hi all, > > could you please review this patch which open sources MLVM tests from VM testbase? > > these tests were developed in early days of JSR292 to test different aspects of MethodHandles and invokedynamic. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199384 > webrev: http://cr.openjdk.java.net/~iignatyev//8199384/webrev.00/index.html > testing: :vmTestbase_vm_mlvm test group > > Thanks, > -- Igor > From rkennke at redhat.com Mon May 14 17:59:42 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 14 May 2018 19:59:42 +0200 Subject: RFR: JDK-8202016: Use obj+offset in interpreter array access In-Reply-To: <31370613-99ef-06cb-9241-dfc459d3b913@redhat.com> References: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> <041699e9-9c5b-0a8b-1b94-45a181018b47@redhat.com> <8b16fb83-25c4-8fcf-7ef6-13d3943f3fd7@redhat.com> <31370613-99ef-06cb-9241-dfc459d3b913@redhat.com> Message-ID: <0fc6d80a-d097-f49d-f2f8-45510d91c368@redhat.com> Thanks for the reviews, Andrew! Roman > On 14/05/18 12:34, Roman Kennke wrote: >> Am 14.05.2018 um 13:19 schrieb Andrew Dinn: > . . . >>> Well, that x86 code change looks ok/ish/ -- although the need to reload >>> from the stack is a tad disappointing. >>> >>> However, I am left wondering why this is not a problem on other >>> architectures? Indeed, looking at AArch64 I believe it is actually a >>> serious headache. >> >> It is, and I have a solution here: >> cr.openjdk.java.net/~rkennke/interp_primitives_aarch64/webrev.0 > > Yuck, how repulsive :] > >> I will propose this for upstream soon. > > Ok, so ack the original x86 change. > >> For AArch64 it will basically reshuffle address calculation from >> currently (base + index) + disp (which we can't resolve) to base + >> (index + disp) which we can. >> >> The alternative would be to explicitely drag the base_obj through all >> the BarrierSetAssembler calls. So far I could live without it (x86 and >> aarch64). > Yes, even more repulsive, I guess. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From robbin.ehn at oracle.com Mon May 14 18:47:36 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 14 May 2018 20:47:36 +0200 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: References: <5cdb2496-bad5-fa4a-f98e-37ae6a853f6b@oracle.com> Message-ID: <56bbfb13-3527-669a-b155-d699be1ef42e@oracle.com> Hi JC, On 2018-05-14 17:09, JC Beyler wrote: > Hi Robbin, > > First off: Thanks for looking! There were 3 comments here and I'll try to > address all three :) > > From easy to more difficult: > - The thread state keeping a pointer of the collector: yes I agree but it > follows the other collector implementations and with Serguei we tried to keep > that implementation in sync. I agree that this is the correct approach for now. > - Done for the orderAccess, I'll send a new webrev when we solve the next > conversation: Thanks > > Now the hardest one (or the one that might generate the most conversation): // If we want to be sampling, protect the allocated object with a Handle // before doing the callback. obj_h = Handle(THREAD, (oop) obj); Can you just add a comment somewhere here and say that callback in done in the destructor of collector or similar? Thanks, Robbin > > There are now three different implementations for putting the collector in place: > ? 1) Minimum change to the collectedHeap.inline.hpp but the collectors are not > symmetrical anymore: > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.15/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html > ? ? ?-> That looks like what you had. > > Pro: no big change to the collectedHeap code, easy to see no overhead when disabled > Con: collectors are not symmetrical anymore > > ? ?2) Small change to the collectedHeap.inline.hpp and collectors remain > symmetrical: > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.16/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html > > Pro: small change all around > Con: not clear that having a handle created on each slowpath does not add any > overhead when disabled. > > ? ?3) Bigger change to collectedHeap.inline.hpp, collectors remain symmetrical: > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.19/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html > > That is the one that you reviewed. > > Pro: code is a bit bigger for the collectedHeap but you have no overhead again > if disabled, the collectors remain symmetrical > Con: bigger change to the collectedHeap. > > So what I see here is that we have to get a consensus for which implementation > is better. I don't like the (2), I worry about the overhead of always doing a > Handle in the slowpath. So I have a tendency to prefer (1) or (3). With Serguei, > we preferred (3). > > What?do you and the community think? > > Thanks again for your review! > Jc > > On Mon, May 14, 2018 at 2:13 AM Robbin Ehn > wrote: > > Hi JC, I found a .19 which I looked at: > > src/hotspot/share/gc/shared/collectedHeap.inline.hpp > CollectedHeap::allocate_memory > > This is the only place I found which calls the > ~JvmtiSampledObjectAllocEventCollector > It is not intuitive with creating a handle for the destructor, I suggest > something like collector.sample(THREAD, obj_h); instead. > > open/src/hotspot/share/runtime/threadHeapSampler.hpp > Don't include inline.hpp in hpp. > This means you need to move the two methods using orderAccess to cpp > (or a inline.hpp). > > As general note, not your doing, setting a pointer in a heap allocated > object to > a stack allocated object is a really bad pattern. > JvmtiThreadState -> collector > > Thanks, Robbin > > On 05/08/2018 03:10 AM, JC Beyler wrote: > > Hi all, > > > > With the awesome help of Serguei Spitsyn, we have moved forward on the > > implementation for JEP-331 and have the following webrev for review: > > > > Webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.18/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8171119 > > > > It is based on jdk/jdk so should patch well with a recent tip. > > > > Could we please have some reviews for the webrev? It would be greatly > > appreciated! > > > > Thanks for all your help! > > Jc > > > From rkennke at redhat.com Mon May 14 19:19:20 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 14 May 2018 21:19:20 +0200 Subject: RFR: JDK-8203157: Object equals abstraction for BarrierSetAssembler Message-ID: <61fcbf38-ecf2-3d19-9f96-5331a1fb65ff@redhat.com> Similar to what's done in the runtime already, GCs might need a say about object equality (e.g. Shenandoah GC). This adds the required abstraction to x86 and aarch64 assembler code. In x86 it ends up a bit ugly because of the existing variations of cmpoop() which take several combinations of Register, Address and jobject as argument, and even worse, varies between 64 and 32 bit builds. In aarch64, I added the MacroAssembler::cmpoop() indirection to make it more like the x86 implementation. http://cr.openjdk.java.net/~rkennke/JDK-8203157/webrev.00/ Passes hotspot/tier1 tests for x86_64/x86_32/aarch64 Can I please get a review? Thanks, Roman From jcbeyler at google.com Mon May 14 19:37:36 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 14 May 2018 12:37:36 -0700 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: <56bbfb13-3527-669a-b155-d699be1ef42e@oracle.com> References: <5cdb2496-bad5-fa4a-f98e-37ae6a853f6b@oracle.com> <56bbfb13-3527-669a-b155-d699be1ef42e@oracle.com> Message-ID: Hi Robbin, I did this then, which should be explicit enough, no? // If we want to be sampling, protect the allocated object with a Handle // before doing the callback. The callback is done in the destructor of // the JvmtiSampledObjectAllocEventCollector. obj_h = Handle(THREAD, (oop) obj); Do you have any other concerns? If not, I'll generate a new webrev that incorporates all your comments. In the meantime, is there anyone else who would be motivated to review the webrev please? :) http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.19/ Thanks a lot! Jc On Mon, May 14, 2018 at 11:47 AM Robbin Ehn wrote: > Hi JC, > > On 2018-05-14 17:09, JC Beyler wrote: > > Hi Robbin, > > > > First off: Thanks for looking! There were 3 comments here and I'll try > to > > address all three :) > > > > From easy to more difficult: > > - The thread state keeping a pointer of the collector: yes I agree but > it > > follows the other collector implementations and with Serguei we tried to > keep > > that implementation in sync. > > I agree that this is the correct approach for now. > > > - Done for the orderAccess, I'll send a new webrev when we solve the > next > > conversation: > > Thanks > > > > > Now the hardest one (or the one that might generate the most > conversation): > > // If we want to be sampling, protect the allocated object with a > Handle > // before doing the callback. > obj_h = Handle(THREAD, (oop) obj); > > Can you just add a comment somewhere here and say that callback in done in > the > destructor of collector or similar? > > Thanks, Robbin > > > > > There are now three different implementations for putting the collector > in place: > > 1) Minimum change to the collectedHeap.inline.hpp but the collectors > are not > > symmetrical anymore: > > > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.15/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html > > -> That looks like what you had. > > > > Pro: no big change to the collectedHeap code, easy to see no overhead > when disabled > > Con: collectors are not symmetrical anymore > > > > 2) Small change to the collectedHeap.inline.hpp and collectors > remain > > symmetrical: > > > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.16/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html > > > > Pro: small change all around > > Con: not clear that having a handle created on each slowpath does not > add any > > overhead when disabled. > > > > 3) Bigger change to collectedHeap.inline.hpp, collectors remain > symmetrical: > > > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.19/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html > > > > That is the one that you reviewed. > > > > Pro: code is a bit bigger for the collectedHeap but you have no overhead > again > > if disabled, the collectors remain symmetrical > > Con: bigger change to the collectedHeap. > > > > So what I see here is that we have to get a consensus for which > implementation > > is better. I don't like the (2), I worry about the overhead of always > doing a > > Handle in the slowpath. So I have a tendency to prefer (1) or (3). With > Serguei, > > we preferred (3). > > > > What do you and the community think? > > > > Thanks again for your review! > > Jc > > > > On Mon, May 14, 2018 at 2:13 AM Robbin Ehn > > wrote: > > > > Hi JC, I found a .19 which I looked at: > > > > src/hotspot/share/gc/shared/collectedHeap.inline.hpp > > CollectedHeap::allocate_memory > > > > This is the only place I found which calls the > > ~JvmtiSampledObjectAllocEventCollector > > It is not intuitive with creating a handle for the destructor, I > suggest > > something like collector.sample(THREAD, obj_h); instead. > > > > open/src/hotspot/share/runtime/threadHeapSampler.hpp > > Don't include inline.hpp in hpp. > > This means you need to move the two methods using orderAccess to cpp > > (or a inline.hpp). > > > > As general note, not your doing, setting a pointer in a heap > allocated > > object to > > a stack allocated object is a really bad pattern. > > JvmtiThreadState -> collector > > > > Thanks, Robbin > > > > On 05/08/2018 03:10 AM, JC Beyler wrote: > > > Hi all, > > > > > > With the awesome help of Serguei Spitsyn, we have moved forward > on the > > > implementation for JEP-331 and have the following webrev for > review: > > > > > > Webrev: > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.18/ > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8171119 > > > > > > It is based on jdk/jdk so should patch well with a recent tip. > > > > > > Could we please have some reviews for the webrev? It would be > greatly > > > appreciated! > > > > > > Thanks for all your help! > > > Jc > > > > > > From robbin.ehn at oracle.com Mon May 14 19:43:09 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Mon, 14 May 2018 21:43:09 +0200 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: References: <5cdb2496-bad5-fa4a-f98e-37ae6a853f6b@oracle.com> <56bbfb13-3527-669a-b155-d699be1ef42e@oracle.com> Message-ID: <3c1d88fd-a3bc-350a-15fb-ec33118b7626@oracle.com> On 2018-05-14 21:37, JC Beyler wrote: > Hi Robbin, > > I did this then, which should be explicit enough, no? > > ? ? ? ?// If we want to be sampling, protect the allocated object with a Handle > ? ? ? // before doing the callback. The callback is done in the destructor of > ? ? ? // the JvmtiSampledObjectAllocEventCollector. > ? ? ? obj_h = Handle(THREAD, (oop) obj); > > Do you have any other concerns? If not, I'll generate a new webrev that > incorporates all your comments. Hi JC, thanks for all your work, ship it! /Robbin > > In the meantime, is there anyone else who would be motivated to review the > webrev please? :) > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.19/ > > Thanks a lot! > Jc > > > > > > On Mon, May 14, 2018 at 11:47 AM Robbin Ehn > wrote: > > Hi JC, > > On 2018-05-14 17:09, JC Beyler wrote: > > Hi Robbin, > > > > First off: Thanks for looking! There were 3 comments here and I'll try to > > address all three :) > > > >? From easy to more difficult: > > - The thread state keeping a pointer of the collector: yes I agree but it > > follows the other collector implementations and with Serguei we tried to > keep > > that implementation in sync. > > I agree that this is the correct approach for now. > > > - Done for the orderAccess, I'll send a new webrev when we solve the next > > conversation: > > Thanks > > > > > Now the hardest one (or the one that might generate the most conversation): > > ? ? ? ?// If we want to be sampling, protect the allocated object with a Handle > ? ? ? ?// before doing the callback. > ? ? ? ?obj_h = Handle(THREAD, (oop) obj); > > Can you just add a comment somewhere here and say that callback in done in the > destructor of collector or similar? > > Thanks, Robbin > > > > > There are now three different implementations for putting the collector > in place: > >? ? 1) Minimum change to the collectedHeap.inline.hpp but the collectors > are not > > symmetrical anymore: > > > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.15/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html > >? ? ? ?-> That looks like what you had. > > > > Pro: no big change to the collectedHeap code, easy to see no overhead > when disabled > > Con: collectors are not symmetrical anymore > > > >? ? ?2) Small change to the collectedHeap.inline.hpp and collectors remain > > symmetrical: > > > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.16/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html > > > > Pro: small change all around > > Con: not clear that having a handle created on each slowpath does not add > any > > overhead when disabled. > > > >? ? ?3) Bigger change to collectedHeap.inline.hpp, collectors remain > symmetrical: > > > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.19/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html > > > > That is the one that you reviewed. > > > > Pro: code is a bit bigger for the collectedHeap but you have no overhead > again > > if disabled, the collectors remain symmetrical > > Con: bigger change to the collectedHeap. > > > > So what I see here is that we have to get a consensus for which > implementation > > is better. I don't like the (2), I worry about the overhead of always > doing a > > Handle in the slowpath. So I have a tendency to prefer (1) or (3). With > Serguei, > > we preferred (3). > > > > What?do you and the community think? > > > > Thanks again for your review! > > Jc > > > > On Mon, May 14, 2018 at 2:13 AM Robbin Ehn > > >> wrote: > > > >? ? ?Hi JC, I found a .19 which I looked at: > > > >? ? ?src/hotspot/share/gc/shared/collectedHeap.inline.hpp > >? ? ?CollectedHeap::allocate_memory > > > >? ? ?This is the only place I found which calls the > >? ? ?~JvmtiSampledObjectAllocEventCollector > >? ? ?It is not intuitive with creating a handle for the destructor, I suggest > >? ? ?something like collector.sample(THREAD, obj_h); instead. > > > >? ? ?open/src/hotspot/share/runtime/threadHeapSampler.hpp > >? ? ?Don't include inline.hpp in hpp. > >? ? ?This means you need to move the two methods using orderAccess to cpp > >? ? ?(or a inline.hpp). > > > >? ? ?As general note, not your doing, setting a pointer in a heap allocated > >? ? ?object to > >? ? ?a stack allocated object is a really bad pattern. > >? ? ?JvmtiThreadState -> collector > > > >? ? ?Thanks, Robbin > > > >? ? ?On 05/08/2018 03:10 AM, JC Beyler wrote: > >? ? ? > Hi all, > >? ? ? > > >? ? ? > With the awesome help of Serguei Spitsyn, we have moved forward on the > >? ? ? > implementation for JEP-331 and have the following webrev for review: > >? ? ? > > >? ? ? > Webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.18/ > >? ? ? > Bug: https://bugs.openjdk.java.net/browse/JDK-8171119 > >? ? ? > > >? ? ? > It is based on jdk/jdk so should patch well with a recent tip. > >? ? ? > > >? ? ? > Could we please have some reviews for the webrev? It would be greatly > >? ? ? > appreciated! > >? ? ? > > >? ? ? > Thanks for all your help! > >? ? ? > Jc > >? ? ? > > > > From jcbeyler at google.com Mon May 14 20:02:17 2018 From: jcbeyler at google.com (JC Beyler) Date: Mon, 14 May 2018 13:02:17 -0700 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: <3c1d88fd-a3bc-350a-15fb-ec33118b7626@oracle.com> References: <5cdb2496-bad5-fa4a-f98e-37ae6a853f6b@oracle.com> <56bbfb13-3527-669a-b155-d699be1ef42e@oracle.com> <3c1d88fd-a3bc-350a-15fb-ec33118b7626@oracle.com> Message-ID: Hi Robbin and all, Thank you for your continuous help! Done then! Here is the new webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.20/ and the incremental is: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.19_20/ Thanks again all! Jc On Mon, May 14, 2018 at 12:43 PM Robbin Ehn wrote: > On 2018-05-14 21:37, JC Beyler wrote: > > Hi Robbin, > > > > I did this then, which should be explicit enough, no? > > > > // If we want to be sampling, protect the allocated object with > a Handle > > // before doing the callback. The callback is done in the > destructor of > > // the JvmtiSampledObjectAllocEventCollector. > > obj_h = Handle(THREAD, (oop) obj); > > > > Do you have any other concerns? If not, I'll generate a new webrev that > > incorporates all your comments. > > Hi JC, thanks for all your work, ship it! > > /Robbin > > > > > In the meantime, is there anyone else who would be motivated to review > the > > webrev please? :) > > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.19/ > > > > Thanks a lot! > > Jc > > > > > > > > > > > > On Mon, May 14, 2018 at 11:47 AM Robbin Ehn > > wrote: > > > > Hi JC, > > > > On 2018-05-14 17:09, JC Beyler wrote: > > > Hi Robbin, > > > > > > First off: Thanks for looking! There were 3 comments here and > I'll try to > > > address all three :) > > > > > > From easy to more difficult: > > > - The thread state keeping a pointer of the collector: yes I > agree but it > > > follows the other collector implementations and with Serguei we > tried to > > keep > > > that implementation in sync. > > > > I agree that this is the correct approach for now. > > > > > - Done for the orderAccess, I'll send a new webrev when we solve > the next > > > conversation: > > > > Thanks > > > > > > > > Now the hardest one (or the one that might generate the most > conversation): > > > > // If we want to be sampling, protect the allocated object > with a Handle > > // before doing the callback. > > obj_h = Handle(THREAD, (oop) obj); > > > > Can you just add a comment somewhere here and say that callback in > done in the > > destructor of collector or similar? > > > > Thanks, Robbin > > > > > > > > There are now three different implementations for putting the > collector > > in place: > > > 1) Minimum change to the collectedHeap.inline.hpp but the > collectors > > are not > > > symmetrical anymore: > > > > > > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.15/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html > > > -> That looks like what you had. > > > > > > Pro: no big change to the collectedHeap code, easy to see no > overhead > > when disabled > > > Con: collectors are not symmetrical anymore > > > > > > 2) Small change to the collectedHeap.inline.hpp and > collectors remain > > > symmetrical: > > > > > > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.16/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html > > > > > > Pro: small change all around > > > Con: not clear that having a handle created on each slowpath does > not add > > any > > > overhead when disabled. > > > > > > 3) Bigger change to collectedHeap.inline.hpp, collectors > remain > > symmetrical: > > > > > > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.19/src/hotspot/share/gc/shared/collectedHeap.inline.hpp.udiff.html > > > > > > That is the one that you reviewed. > > > > > > Pro: code is a bit bigger for the collectedHeap but you have no > overhead > > again > > > if disabled, the collectors remain symmetrical > > > Con: bigger change to the collectedHeap. > > > > > > So what I see here is that we have to get a consensus for which > > implementation > > > is better. I don't like the (2), I worry about the overhead of > always > > doing a > > > Handle in the slowpath. So I have a tendency to prefer (1) or > (3). With > > Serguei, > > > we preferred (3). > > > > > > What do you and the community think? > > > > > > Thanks again for your review! > > > Jc > > > > > > On Mon, May 14, 2018 at 2:13 AM Robbin Ehn > > > > >> > wrote: > > > > > > Hi JC, I found a .19 which I looked at: > > > > > > src/hotspot/share/gc/shared/collectedHeap.inline.hpp > > > CollectedHeap::allocate_memory > > > > > > This is the only place I found which calls the > > > ~JvmtiSampledObjectAllocEventCollector > > > It is not intuitive with creating a handle for the > destructor, I suggest > > > something like collector.sample(THREAD, obj_h); instead. > > > > > > open/src/hotspot/share/runtime/threadHeapSampler.hpp > > > Don't include inline.hpp in hpp. > > > This means you need to move the two methods using orderAccess > to cpp > > > (or a inline.hpp). > > > > > > As general note, not your doing, setting a pointer in a heap > allocated > > > object to > > > a stack allocated object is a really bad pattern. > > > JvmtiThreadState -> collector > > > > > > Thanks, Robbin > > > > > > On 05/08/2018 03:10 AM, JC Beyler wrote: > > > > Hi all, > > > > > > > > With the awesome help of Serguei Spitsyn, we have moved > forward on the > > > > implementation for JEP-331 and have the following webrev > for review: > > > > > > > > Webrev: > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.18/ > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8171119 > > > > > > > > It is based on jdk/jdk so should patch well with a recent > tip. > > > > > > > > Could we please have some reviews for the webrev? It would > be greatly > > > > appreciated! > > > > > > > > Thanks for all your help! > > > > Jc > > > > > > > > > > From rkennke at redhat.com Mon May 14 20:23:12 2018 From: rkennke at redhat.com (Roman Kennke) Date: Mon, 14 May 2018 22:23:12 +0200 Subject: RFR: JDK-8203172: Primitive heap access for interpreter BarrierSetAssembler/aarch64 Message-ID: Similar to x86 (http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-May/032114.html) here comes the primitive heap access changes for aarch64: http://cr.openjdk.java.net/~rkennke/JDK-8203172/webrev.00/ Some notes: - array access used to compute base_obj + index, and then use indexed addressing with base_offset. This means we cannot get base_obj in the BarrierSetAssembler API, but we need that, e.g. for resolving the target object via forwarding pointer. I changed (base_obj+index)+base_offset to base_obj+(index+base_offset) in all the relevant places. - in jniFastGetField_aarch64.cpp, we are using a trick to ensure correct ordering field-load with the load of the safepoint counter: we make them address dependend. For float and double loads this meant to load the value as int/long, and then later moving those into v0. This doesn't work when going through the BarrierSetAssembler API: it loads straight to v0. Instead I am inserting a LoadLoad membar for float/double (which should be rare enough anyway). Other than that it's pretty much analogous to x86. Testing: no regressions in hotspot/tier1 Can I please get a review? Thanks, Roman From leonid.mesnik at oracle.com Mon May 14 21:04:18 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Mon, 14 May 2018 14:04:18 -0700 Subject: 8199271: [TESTBUG] open source VM testbase stress tests In-Reply-To: <5AF5C07B.1080909@oracle.com> References: <1BC403EC-BDF0-4CA2-B87F-B38A92B6765F@oracle.com> <5AF5C07B.1080909@oracle.com> Message-ID: <8BA2B023-E947-4EF8-9D97-EBA28BAA07E2@oracle.com> Misha Thank you for review. I still need one more review from 'R'eviewer. Leonid > On May 11, 2018, at 9:10 AM, Mikhailo Seledtsov wrote: > > Looks good to me, > > Misha > > On 5/8/18, 2:23 PM, Leonid Mesnik wrote: >> Hi >> >> Please review this change open sourcing vm testbase stress tests. These tests have been developed a long time ago for internal test harness and don't looks very nice now. >> They are open sourced with minimal changes only. The long term plan is to review and improve them moving out of vmTestbase directory in corresponding components. >> >> The fix introduce vmTestbase_nsk_stress test group and have make files changes required to build native part of tests. >> >> I've verified that "make -- run-test TEST=:vmTestbase_nsk_stress" pass on my linux locally. >> >> webrev: http://cr.openjdk.java.net/~lmesnik/8199271/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8199271 From leonid.mesnik at oracle.com Mon May 14 21:04:30 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Mon, 14 May 2018 14:04:30 -0700 Subject: 8199271: [TESTBUG] open source VM testbase stress tests In-Reply-To: <4f0ea090-94c9-ffd7-3852-a9a9d14e8af8@oracle.com> References: <1BC403EC-BDF0-4CA2-B87F-B38A92B6765F@oracle.com> <4f0ea090-94c9-ffd7-3852-a9a9d14e8af8@oracle.com> Message-ID: <8FBFB3E8-2E13-42F8-82F3-6E601E900AED@oracle.com> Thank you for review. Leonid > On May 8, 2018, at 2:41 PM, Erik Joelsson wrote: > > Build changes look good. > > /Erik > > > On 2018-05-08 14:23, Leonid Mesnik wrote: >> Hi >> >> Please review this change open sourcing vm testbase stress tests. These tests have been developed a long time ago for internal test harness and don't looks very nice now. >> They are open sourced with minimal changes only. The long term plan is to review and improve them moving out of vmTestbase directory in corresponding components. >> >> The fix introduce vmTestbase_nsk_stress test group and have make files changes required to build native part of tests. >> >> I've verified that "make -- run-test TEST=:vmTestbase_nsk_stress" pass on my linux locally. >> >> webrev: http://cr.openjdk.java.net/~lmesnik/8199271/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8199271 > From bsrbnd at gmail.com Mon May 14 21:47:58 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 14 May 2018 23:47:58 +0200 Subject: RFR: JDK-8202016: Use obj+offset in interpreter array access In-Reply-To: <8b16fb83-25c4-8fcf-7ef6-13d3943f3fd7@redhat.com> References: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> <041699e9-9c5b-0a8b-1b94-45a181018b47@redhat.com> <8b16fb83-25c4-8fcf-7ef6-13d3943f3fd7@redhat.com> Message-ID: Hi, On 14 May 2018 at 13:19, Andrew Dinn wrote: > On 14/05/18 11:15, Roman Kennke wrote: >> Am 07.05.2018 um 21:47 schrieb Roman Kennke: >>> In the TemplateTable::aastore() and >>> InterpreterMacroAssembler::load_resolved_reference_at_index(), the >>> element address is flattened, and then passed to the BarrierSetAssembler >>> for actual access. GCs like Shenandoah need the original obj though. >>> >>> The proposed change replaces the flattening with base+index+disp >>> addressing, and removes the shift and add computations. In the case of >>> aastore, we need to re-fetch the rcx/index from the stack because it >>> gets trashed on the way. >>> >>> Testing: passed: hotspot/jtreg:tier1 >>> >>> Can I please get a review? >>> Thanks, Roman > Well, that x86 code change looks ok/ish/ -- although the need to reload > from the stack is a tad disappointing. Maybe I'm a bit late, but I think we could use r8 & r9 to avoid some unnecessary stack access, as next. What do you think (tier1 seems OK)? Regards, Bernard diff -r 24151f48582b src/hotspot/cpu/x86/templateTable_x86.cpp --- a/src/hotspot/cpu/x86/templateTable_x86.cpp Mon May 14 21:56:07 2018 +0200 +++ b/src/hotspot/cpu/x86/templateTable_x86.cpp Mon May 14 23:03:40 2018 +0200 @@ -1098,6 +1098,11 @@ __ movl(rcx, at_tos_p1()); // index __ movptr(rdx, at_tos_p2()); // array +#ifdef _LP64 + __ movptr(r8, rax); + __ movl(r9, rcx); +#endif + Address element_address(rdx, rcx, UseCompressedOops? Address::times_4 : Address::times_ptr, arrayOopDesc::base_offset_in_bytes(T_OBJECT)); @@ -1125,8 +1130,13 @@ __ bind(ok_is_subtype); // Get the value we will store +#ifdef _LP64 + __ movptr(rax, r8); + __ movl(rcx, r9); +#else __ movptr(rax, at_tos()); __ movl(rcx, at_tos_p1()); // index +#endif // Now store using the appropriate barrier do_oop_store(_masm, element_address, rax, IN_HEAP_ARRAY); __ jmp(done); > [...] > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From magnus.ihse.bursie at oracle.com Mon May 14 21:53:58 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 14 May 2018 23:53:58 +0200 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: References: <5AE0612F.8060200@oracle.com> <5AF99F08.6060408@oracle.com> Message-ID: On 2018-05-14 18:05, Erik Joelsson wrote: > Oh, I missed the new makefiles last time I looked at this. > > in Copy-jdk.jfr.gmk, everything looks like it's indented an extra 4 > steps. I'm assuming this is because it used to be conditional in the > previous closed file. > > GensrcJfr.gmk, line 94, please move )) to the left. > > Looking closer at GensrcJfr.gmk, The macro SetupJfrGeneration looks > like it is only called once. This could be greatly simplified by just > taking the body of the macro and inlining all the inputs. This of > course unless you see a need in the future to generate additional > files using the jfr tool. Also, the JFR_DEPS variable does not seem to be used. Is it a left-over from a previous attempt to get proper dependencies? (The current solution looks like it should work.) /Magnus > > /Erik > > On 2018-05-14 07:36, Erik Gahlin wrote: >> Here is an updated webrev: >> >> http://cr.openjdk.java.net/~egahlin/8199712.1/ [1] >> >> that incorporates: >> >> - build changes >> - new event prefix, i.e. "com.oracle.jdk.CPULoad" becomes "jdk.CPULoad" >> - obsolete command line options EnableTracing and UseLockedTracing >> - fixed typos in the Javadoc >> - simplified #include files >> >> RFEs have been filed for other issues, CSR is approved and tests pass. >> >> Erik and Markus >> >> [1] Parent: >> >> changeset:?? 50092:0e42d3120e51 >> >> user:??????? clanger >> date:??????? Sat May 12 10:26:42 2018 +0200 >> summary:???? 8202915: [JAXP] Performance enhancements and cleanups in >> com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator >> >> >>> Greetings, >>> >>> Could I have a review of 8199712: Flight Recorder >>> >>> As mentioned in the preview [1] the tracing backend has been >>> removed. Event metadata has been consolidated into a single XML file >>> and event classes are now generated by GenerateJfrFiles.java. >>> >>> Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. >>> >>> For details about the feature, see the JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8193393 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~egahlin/8199712.0/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8199712 >>> >>> [1] >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359.html >>> >>> Thanks >>> Erik and Markus >> > From david.holmes at oracle.com Tue May 15 00:52:18 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 15 May 2018 10:52:18 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control Message-ID: This review is being spread across four groups: langtools, core-libs, hotspot and serviceability. This is the specific review thread for hotspot - webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ See below for full details - including annotated full webrev guiding the review. The intent is to have JEP-181 targeted and integrated by the end of this month. Thanks, David ----- The nestmates project (JEP-181) introduces new classfile attributes to identify classes and interfaces in the same nest, so that the VM can perform access control based on those attributes and so allow direct private access between nestmates without requiring javac to generate synthetic accessor methods. These access control changes also extend to core reflection and the MethodHandle.Lookup contexts. Direct private calls between nestmates requires a more general calling context than is permitted by invokespecial, and so the JVMS is updated to allow, and javac updated to use, invokevirtual and invokeinterface for private class and interface method calls respectively. These changed semantics also extend to MethodHandle findXXX operations. At this time we are only concerned with static nest definitions, which map to a top-level class/interface as the nest-host and all its nested types as nest-members. Please see the JEP for further details. JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 All of the specification changes have been previously been worked out by the Valhalla Project Expert Group, and the implementation reviewed by the various contributors and discussed on the valhalla-dev mailing list. Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan Master webrev of all changes: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ Annotated master webrev index: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html Performance: this is expected to be performance neutral in a general sense. Benchmarking and performance runs are about to start. Testing Discussion: ------------------ The testing for nestmates can be broken into four main groups: - New tests specifically related to nestmates and currently in the runtime/Nestmates directory - New tests to complement existing tests by adding in testcases not previously expressible. - For example java/lang/invoke/SpecialInterfaceCall.java tests use of invokespecial for private interface methods and performing receiver typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do similar tests for invokeinterface. - New JVM TI tests to verify the spec changes related to nest attributes. - Existing tests significantly affected by the nestmates changes, primarily: - runtime/SelectionResolution In most cases the nestmate changes makes certain invocations that were illegal, legal (e.g. not requiring invokespecial to invoke private interface methods; allowing access to private members via reflection/Methodhandles that were previously not allowed). - Existing tests incidentally affected by the nestmate changes This includes tests of things utilising class redefinition/retransformation to alter nested types but which unintentionally alter nest relationships (which is not permitted). There are still a number of tests problem-listed with issues filed against them to have them adapted to work with nestmates. Some of these are intended to be addressed in the short-term, while some (such as the runtime/SelectionResolution test changes) may not eventuate. - https://bugs.openjdk.java.net/browse/JDK-8203033 - https://bugs.openjdk.java.net/browse/JDK-8199450 - https://bugs.openjdk.java.net/browse/JDK-8196855 - https://bugs.openjdk.java.net/browse/JDK-8194857 - https://bugs.openjdk.java.net/browse/JDK-8187655 There is also further test work still to be completed (the JNI and JDI invocation tests): - https://bugs.openjdk.java.net/browse/JDK-8191117 which will continue in parallel with the main RFR. Pre-integration Testing: - General: - Mach5: hs/jdk tier1,2 - Mach5: hs-nightly (tiers 1 -3) - Targetted - nashorn (for asm changes) - hotspot: runtime/* serviceability/* compiler/* vmTestbase/* - jdk: java/lang/invoke/* java/lang/reflect/* java/lang/instrument/* java/lang/Class/* java/lang/management/* - langtools: tools/javac tools/javap From markus.gronlund at oracle.com Tue May 15 07:14:14 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Tue, 15 May 2018 00:14:14 -0700 (PDT) Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: References: <5AE0612F.8060200@oracle.com> <5AF99F08.6060408@oracle.com> Message-ID: Thanks Magnus, About JFR_DEPS, it is somewhat of a remnant from the previous system. Looks like the previous system eventually took the local variable and appended it to DEPS (is this a global where all dependencies are tracked?) Can we do the following instead to ensure file edits trigger recompilation? DEPS := $(METADATA_XML) DEPS := $(METADATA_XSD) Thanks Markus -----Original Message----- From: Magnus Ihse Bursie Sent: den 14 maj 2018 23:54 To: Erik Joelsson ; Erik Gahlin ; hotspot-dev Source Developers Cc: hotspot-jfr-dev at openjdk.java.net; build-dev at openjdk.java.net Subject: Re: RFR(XL): 8199712: Flight Recorder On 2018-05-14 18:05, Erik Joelsson wrote: > Oh, I missed the new makefiles last time I looked at this. > > in Copy-jdk.jfr.gmk, everything looks like it's indented an extra 4 > steps. I'm assuming this is because it used to be conditional in the > previous closed file. > > GensrcJfr.gmk, line 94, please move )) to the left. > > Looking closer at GensrcJfr.gmk, The macro SetupJfrGeneration looks > like it is only called once. This could be greatly simplified by just > taking the body of the macro and inlining all the inputs. This of > course unless you see a need in the future to generate additional > files using the jfr tool. Also, the JFR_DEPS variable does not seem to be used. Is it a left-over from a previous attempt to get proper dependencies? (The current solution looks like it should work.) /Magnus > > /Erik > > On 2018-05-14 07:36, Erik Gahlin wrote: >> Here is an updated webrev: >> >> http://cr.openjdk.java.net/~egahlin/8199712.1/ [1] >> >> that incorporates: >> >> - build changes >> - new event prefix, i.e. "com.oracle.jdk.CPULoad" becomes "jdk.CPULoad" >> - obsolete command line options EnableTracing and UseLockedTracing >> - fixed typos in the Javadoc >> - simplified #include files >> >> RFEs have been filed for other issues, CSR is approved and tests pass. >> >> Erik and Markus >> >> [1] Parent: >> >> changeset:?? 50092:0e42d3120e51 >> >> user:??????? clanger >> date:??????? Sat May 12 10:26:42 2018 +0200 >> summary:???? 8202915: [JAXP] Performance enhancements and cleanups in >> com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator >> >> >>> Greetings, >>> >>> Could I have a review of 8199712: Flight Recorder >>> >>> As mentioned in the preview [1] the tracing backend has been >>> removed. Event metadata has been consolidated into a single XML file >>> and event classes are now generated by GenerateJfrFiles.java. >>> >>> Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. >>> >>> For details about the feature, see the JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8193393 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~egahlin/8199712.0/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8199712 >>> >>> [1] >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359 >>> .html >>> >>> Thanks >>> Erik and Markus >> > From thomas.schatzl at oracle.com Tue May 15 07:23:03 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 15 May 2018 09:23:03 +0200 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: References: <5cdb2496-bad5-fa4a-f98e-37ae6a853f6b@oracle.com> <56bbfb13-3527-669a-b155-d699be1ef42e@oracle.com> <3c1d88fd-a3bc-350a-15fb-ec33118b7626@oracle.com> Message-ID: Hi, On Mon, 2018-05-14 at 13:02 -0700, JC Beyler wrote: > Hi Robbin and all, > > Thank you for your continuous help! > > Done then! Here is the new webrev: > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.20/ > > and the incremental is: > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.19_20/ > > Thanks again all! > Jc > looks good to me. Two minor issues that you might want to fix before pushing: - collectedHeap.hpp: The declaration of CollectedHeap::allocate_memory() should be private. - collectedHeap.inline.hpp: in CollectedHeap::allocate_memory() there is this completely duplicated code which you might factor out into another (private) method: if (init_memory) { obj = ... } else { obj = ... } post_setup(klass, ...); Thanks, Thomas From stuart.monteith at linaro.org Tue May 15 08:16:24 2018 From: stuart.monteith at linaro.org (Stuart Monteith) Date: Tue, 15 May 2018 09:16:24 +0100 Subject: RFR: JDK-8202714: Create a MacroAssembler::access_load/store_at wrapper for AArch64 In-Reply-To: <233d81d0-d79f-dd2e-7052-6990e5654095@redhat.com> References: <233d81d0-d79f-dd2e-7052-6990e5654095@redhat.com> Message-ID: Hi, I tried something similar for the ZGC tree (although I stepped back from them). Your changes look good to me. I'm not a Reviewer. BR, Stuart On 14 May 2018 at 13:02, Roman Kennke wrote: > This reshuffles AArch64 code around BarrierSetAssembler calls follow the > equivalent code in x86: > > http://cr.openjdk.java.net/~rkennke/JDK-8202714/webrev.00/ > > No regressions in hotspot/tier1 > > Can I please get a review? > > Thanks, Roman > From rkennke at redhat.com Tue May 15 08:18:33 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 15 May 2018 10:18:33 +0200 Subject: RFR: JDK-8202714: Create a MacroAssembler::access_load/store_at wrapper for AArch64 In-Reply-To: References: <233d81d0-d79f-dd2e-7052-6990e5654095@redhat.com> Message-ID: <9d153da3-4e56-d087-0a2d-b37a9666bc12@redhat.com> Thanks, Stuart! Roman > Hi, > I tried something similar for the ZGC tree (although I stepped back > from them). Your changes look good to me. I'm not a Reviewer. > > BR, > Stuart > > On 14 May 2018 at 13:02, Roman Kennke wrote: >> This reshuffles AArch64 code around BarrierSetAssembler calls follow the >> equivalent code in x86: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8202714/webrev.00/ >> >> No regressions in hotspot/tier1 >> >> Can I please get a review? >> >> Thanks, Roman >> From rkennke at redhat.com Tue May 15 09:26:06 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 15 May 2018 11:26:06 +0200 Subject: RFR: JDK-8202016: Use obj+offset in interpreter array access In-Reply-To: References: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> <041699e9-9c5b-0a8b-1b94-45a181018b47@redhat.com> <8b16fb83-25c4-8fcf-7ef6-13d3943f3fd7@redhat.com> Message-ID: <6cd6372a-b527-0b6d-6a61-72a3d1eb7319@redhat.com> Am 14.05.2018 um 23:47 schrieb B. Blaser: > Hi, > > On 14 May 2018 at 13:19, Andrew Dinn wrote: >> On 14/05/18 11:15, Roman Kennke wrote: >>> Am 07.05.2018 um 21:47 schrieb Roman Kennke: >>>> In the TemplateTable::aastore() and >>>> InterpreterMacroAssembler::load_resolved_reference_at_index(), the >>>> element address is flattened, and then passed to the BarrierSetAssembler >>>> for actual access. GCs like Shenandoah need the original obj though. >>>> >>>> The proposed change replaces the flattening with base+index+disp >>>> addressing, and removes the shift and add computations. In the case of >>>> aastore, we need to re-fetch the rcx/index from the stack because it >>>> gets trashed on the way. >>>> >>>> Testing: passed: hotspot/jtreg:tier1 >>>> >>>> Can I please get a review? >>>> Thanks, Roman >> Well, that x86 code change looks ok/ish/ -- although the need to reload >> from the stack is a tad disappointing. > > Maybe I'm a bit late, but I think we could use r8 & r9 to avoid some > unnecessary stack access, as next. > What do you think (tier1 seems OK)? > Yes, that is a good idea and I believe it would work. I think it can be improved even further by only moving it out to r8/r9 once, and use those registers for addressing. Do you want to file a followup-issue and do that? Thanks, Roman > Regards, > Bernard > > > diff -r 24151f48582b src/hotspot/cpu/x86/templateTable_x86.cpp > --- a/src/hotspot/cpu/x86/templateTable_x86.cpp Mon May 14 21:56:07 > 2018 +0200 > +++ b/src/hotspot/cpu/x86/templateTable_x86.cpp Mon May 14 23:03:40 > 2018 +0200 > @@ -1098,6 +1098,11 @@ > __ movl(rcx, at_tos_p1()); // index > __ movptr(rdx, at_tos_p2()); // array > > +#ifdef _LP64 > + __ movptr(r8, rax); > + __ movl(r9, rcx); > +#endif > + > Address element_address(rdx, rcx, > UseCompressedOops? Address::times_4 : > Address::times_ptr, > arrayOopDesc::base_offset_in_bytes(T_OBJECT)); > @@ -1125,8 +1130,13 @@ > __ bind(ok_is_subtype); > > // Get the value we will store > +#ifdef _LP64 > + __ movptr(rax, r8); > + __ movl(rcx, r9); > +#else > __ movptr(rax, at_tos()); > __ movl(rcx, at_tos_p1()); // index > +#endif > // Now store using the appropriate barrier > do_oop_store(_masm, element_address, rax, IN_HEAP_ARRAY); > __ jmp(done); > > >> [...] >> >> regards, >> >> >> Andrew Dinn >> ----------- >> Senior Principal Software Engineer >> Red Hat UK Ltd >> Registered in England and Wales under Company Registration No. 03798903 >> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adinn at redhat.com Tue May 15 09:34:27 2018 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 15 May 2018 10:34:27 +0100 Subject: RFR: JDK-8203172: Primitive heap access for interpreter BarrierSetAssembler/aarch64 In-Reply-To: References: Message-ID: <230347c6-9d85-0693-c703-09a79151473e@redhat.com> On 14/05/18 21:23, Roman Kennke wrote: > . . . > Can I please get a review? I have only one small question regarding this change in templateTable_aarch64.cpp: @@ -2763,12 +2756,11 @@ // ztos { __ pop(ztos); if (!is_static) pop_and_check_object(obj); - __ andw(r0, r0, 0x1); - __ strb(r0, field); + __ access_store_at(T_BOOLEAN, IN_HEAP, field, r0, noreg, noreg); if (rc == may_rewrite) { patch_bytecode(Bytecodes::_fast_zputfield, bc, r1, true, byte_no); } __ b(Done); } The implementation of access_store_at (when the call finally lands in in BarrierSetAssembler) for the T_BOOLEAN case simply does an strb. So, the andw(r0, r0, 0x1) in the original has been elided in the replacement. Was that definitely redundant in the original? Otherwise all looks fine. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From per.liden at oracle.com Tue May 15 09:37:36 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 15 May 2018 11:37:36 +0200 Subject: RFR: JDK-8200623: Primitive heap access for interpreter BarrierSetAssembler/x86 In-Reply-To: <542449a6-84a5-ba69-16dd-2fd2fcd587c0@redhat.com> References: <542449a6-84a5-ba69-16dd-2fd2fcd587c0@redhat.com> Message-ID: Hi Roman, I think it would be good if Erik ?sterlund reviewed these Access-related changes. He'll be back in the office on Thursday. cheers, Per On 05/14/2018 12:16 PM, Roman Kennke wrote: > Am 07.05.2018 um 22:31 schrieb Roman Kennke: >> JDK-8199417 added better modularization for interpreter barriers. >> Shenandoah and possibly future GCs also need barriers for primitive access. >> >> Some notes on implementation: >> - float/double/long access produced some headaches for the following >> reasons: >> >> - float and double would either take XMMRegister which is not >> compatible with Register >> - or load-from/store-to the floating point stack (see >> MacroAssembler::load/store_float/double) >> - long access on x86_32 would load-into/store-from 2 registers, or >> else use a trick via the floating point stack to do atomic access >> >> None of this seemed easy/nice to do with the API. I helped myself by >> accepting noreg as dst/src argument, which means the corresponding tos >> (i.e. ltos, ftos, dtos) and the BSA would then access from/to >> xmm0/float-stack in case of float/double or the double-reg/float-stack >> in case of long/32bit, which is all that we ever need. >> >> I'm passing MO_RELAXED to long access calls to hint that we want atomic >> access or not. I hope that is ok. >> >> >> Tested: hotspot/jtreg:tier1 >> >> http://cr.openjdk.java.net/~rkennke/JDK-8200623/webrev.00/ >> >> Can I please get a review? >> >> Thanks, Roman >> > > Ping? > > Thanks, Roman > From rkennke at redhat.com Tue May 15 09:38:52 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 15 May 2018 11:38:52 +0200 Subject: RFR: JDK-8203172: Primitive heap access for interpreter BarrierSetAssembler/aarch64 In-Reply-To: <230347c6-9d85-0693-c703-09a79151473e@redhat.com> References: <230347c6-9d85-0693-c703-09a79151473e@redhat.com> Message-ID: Hi Andrew, > On 14/05/18 21:23, Roman Kennke wrote: >> . . . >> Can I please get a review? > I have only one small question regarding this change in > templateTable_aarch64.cpp: > > @@ -2763,12 +2756,11 @@ > > // ztos > { > __ pop(ztos); > if (!is_static) pop_and_check_object(obj); > - __ andw(r0, r0, 0x1); > - __ strb(r0, field); > + __ access_store_at(T_BOOLEAN, IN_HEAP, field, r0, noreg, noreg); > if (rc == may_rewrite) { > patch_bytecode(Bytecodes::_fast_zputfield, bc, r1, true, byte_no); > } > __ b(Done); > } > > The implementation of access_store_at (when the call finally lands in in > BarrierSetAssembler) for the T_BOOLEAN case simply does an strb. So, the > andw(r0, r0, 0x1) in the original has been elided in the replacement. > Was that definitely redundant in the original? http://cr.openjdk.java.net/~rkennke/JDK-8203172/webrev.00/src/hotspot/cpu/aarch64/gc/shared/barrierSetAssembler_aarch64.cpp.udiff.html + case T_BOOLEAN: + __ andw(val, val, 0x1); // boolean is true if LSB is 1 + __ strb(val, dst); + break; I think it does what you want? Cheers, Roman From rkennke at redhat.com Tue May 15 09:42:35 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 15 May 2018 11:42:35 +0200 Subject: RFR: JDK-8200623: Primitive heap access for interpreter BarrierSetAssembler/x86 In-Reply-To: References: <542449a6-84a5-ba69-16dd-2fd2fcd587c0@redhat.com> Message-ID: Hi Per, yes, I will hold back non-simple access-related changes until he's back. Unfortunately, I'll be out starting Friday for at least one week, so all this stuff will have to wait. Thanks, Roman > Hi Roman, > > I think it would be good if Erik ?sterlund reviewed these Access-related > changes. He'll be back in the office on Thursday. > > cheers, > Per > > On 05/14/2018 12:16 PM, Roman Kennke wrote: >> Am 07.05.2018 um 22:31 schrieb Roman Kennke: >>> JDK-8199417 added better modularization for interpreter barriers. >>> Shenandoah and possibly future GCs also need barriers for primitive >>> access. >>> >>> Some notes on implementation: >>> - float/double/long access produced some headaches for the following >>> reasons: >>> >>> ?? - float and double would either take XMMRegister which is not >>> compatible with Register >>> ?? - or load-from/store-to the floating point stack (see >>> MacroAssembler::load/store_float/double) >>> ?? - long access on x86_32 would load-into/store-from 2 registers, or >>> else use a trick via the floating point stack to do atomic access >>> >>> None of this seemed easy/nice to do with the API. I helped myself by >>> accepting noreg as dst/src argument, which means the corresponding tos >>> (i.e. ltos, ftos, dtos) and the BSA would then access from/to >>> xmm0/float-stack in case of float/double or the double-reg/float-stack >>> in case of long/32bit, which is all that we ever need. >>> >>> I'm passing MO_RELAXED to long access calls to hint that we want atomic >>> access or not. I hope that is ok. >>> >>> >>> Tested: hotspot/jtreg:tier1 >>> >>> http://cr.openjdk.java.net/~rkennke/JDK-8200623/webrev.00/ >>> >>> Can I please get a review? >>> >>> Thanks, Roman >>> >> >> Ping? >> >> Thanks, Roman >> From adinn at redhat.com Tue May 15 10:31:20 2018 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 15 May 2018 11:31:20 +0100 Subject: RFR: JDK-8203172: Primitive heap access for interpreter BarrierSetAssembler/aarch64 In-Reply-To: References: <230347c6-9d85-0693-c703-09a79151473e@redhat.com> Message-ID: <037d7d8e-70df-6e73-218d-31f47d895bc4@redhat.com> On 15/05/18 10:38, Roman Kennke wrote: > http://cr.openjdk.java.net/~rkennke/JDK-8203172/webrev.00/src/hotspot/cpu/aarch64/gc/shared/barrierSetAssembler_aarch64.cpp.udiff.html > > + case T_BOOLEAN: > + __ andw(val, val, 0x1); // boolean is true if LSB is 1 > + __ strb(val, dst); > + break; > > I think it does what you want? Yes it does :-) All looks good then. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From bsrbnd at gmail.com Tue May 15 10:43:15 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Tue, 15 May 2018 12:43:15 +0200 Subject: RFR: JDK-8202016: Use obj+offset in interpreter array access In-Reply-To: <6cd6372a-b527-0b6d-6a61-72a3d1eb7319@redhat.com> References: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> <041699e9-9c5b-0a8b-1b94-45a181018b47@redhat.com> <8b16fb83-25c4-8fcf-7ef6-13d3943f3fd7@redhat.com> <6cd6372a-b527-0b6d-6a61-72a3d1eb7319@redhat.com> Message-ID: On 15 May 2018 at 11:26, Roman Kennke wrote: > Am 14.05.2018 um 23:47 schrieb B. Blaser: >> Hi, >> >> On 14 May 2018 at 13:19, Andrew Dinn wrote: >>> On 14/05/18 11:15, Roman Kennke wrote: >>>> Am 07.05.2018 um 21:47 schrieb Roman Kennke: >>>>> In the TemplateTable::aastore() and >>>>> InterpreterMacroAssembler::load_resolved_reference_at_index(), the >>>>> element address is flattened, and then passed to the BarrierSetAssembler >>>>> for actual access. GCs like Shenandoah need the original obj though. >>>>> >>>>> The proposed change replaces the flattening with base+index+disp >>>>> addressing, and removes the shift and add computations. In the case of >>>>> aastore, we need to re-fetch the rcx/index from the stack because it >>>>> gets trashed on the way. >>>>> >>>>> Testing: passed: hotspot/jtreg:tier1 >>>>> >>>>> Can I please get a review? >>>>> Thanks, Roman >>> Well, that x86 code change looks ok/ish/ -- although the need to reload >>> from the stack is a tad disappointing. >> >> Maybe I'm a bit late, but I think we could use r8 & r9 to avoid some >> unnecessary stack access, as next. >> What do you think (tier1 seems OK)? >> > Yes, that is a good idea and I believe it would work. I think it can be > improved even further by only moving it out to r8/r9 once, and use those > registers for addressing. Do you want to file a followup-issue and do that? Yes, good suggestion. I'll provide an improved patch and create a JBS issue. Thanks for your feedback, Bernard > Thanks, Roman > > >> Regards, >> Bernard >> >> >> diff -r 24151f48582b src/hotspot/cpu/x86/templateTable_x86.cpp >> --- a/src/hotspot/cpu/x86/templateTable_x86.cpp Mon May 14 21:56:07 >> 2018 +0200 >> +++ b/src/hotspot/cpu/x86/templateTable_x86.cpp Mon May 14 23:03:40 >> 2018 +0200 >> @@ -1098,6 +1098,11 @@ >> __ movl(rcx, at_tos_p1()); // index >> __ movptr(rdx, at_tos_p2()); // array >> >> +#ifdef _LP64 >> + __ movptr(r8, rax); >> + __ movl(r9, rcx); >> +#endif >> + >> Address element_address(rdx, rcx, >> UseCompressedOops? Address::times_4 : >> Address::times_ptr, >> arrayOopDesc::base_offset_in_bytes(T_OBJECT)); >> @@ -1125,8 +1130,13 @@ >> __ bind(ok_is_subtype); >> >> // Get the value we will store >> +#ifdef _LP64 >> + __ movptr(rax, r8); >> + __ movl(rcx, r9); >> +#else >> __ movptr(rax, at_tos()); >> __ movl(rcx, at_tos_p1()); // index >> +#endif >> // Now store using the appropriate barrier >> do_oop_store(_masm, element_address, rax, IN_HEAP_ARRAY); >> __ jmp(done); >> >> >>> [...] >>> >>> regards, >>> >>> >>> Andrew Dinn >>> ----------- >>> Senior Principal Software Engineer >>> Red Hat UK Ltd >>> Registered in England and Wales under Company Registration No. 03798903 >>> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > > From aph at redhat.com Tue May 15 13:07:18 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 15 May 2018 14:07:18 +0100 Subject: RFR: JDK-8202714: Create a MacroAssembler::access_load/store_at wrapper for AArch64 In-Reply-To: References: <233d81d0-d79f-dd2e-7052-6990e5654095@redhat.com> Message-ID: > On 14 May 2018 at 13:02, Roman Kennke wrote: >> This reshuffles AArch64 code around BarrierSetAssembler calls follow the >> equivalent code in x86: >> >> http://cr.openjdk.java.net/~rkennke/JDK-8202714/webrev.00/ >> >> No regressions in hotspot/tier1 >> >> Can I please get a review? Looks OK. This use of an explict base class override in MacroAssembler is rather nasty: 3990 void MacroAssembler::access_store_at(BasicType type, DecoratorSet decorators, 3991 Address dst, Register src, 3992 Register tmp1, Register thread_tmp) { 3993 BarrierSetAssembler *bs = BarrierSet::barrier_set()->barrier_set_assembler(); 3994 bool as_raw = (decorators & AS_RAW) != 0; 3995 if (as_raw) { 3996 bs->BarrierSetAssembler::store_at(this, decorators, type, dst, src, tmp1, thread_tmp); 3997 } else { 3998 bs->store_at(this, decorators, type, dst, src, tmp1, thread_tmp); 3999 } 4000 } 4001 -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From erik.gahlin at oracle.com Tue May 15 13:12:25 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Tue, 15 May 2018 15:12:25 +0200 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: References: <5AE0612F.8060200@oracle.com> <5AF99F08.6060408@oracle.com> Message-ID: <5AFADCB9.4060904@oracle.com> On 2018-05-14 18:05, Erik Joelsson wrote: > Oh, I missed the new makefiles last time I looked at this. > > in Copy-jdk.jfr.gmk, everything looks like it's indented an extra 4 > steps. I'm assuming this is because it used to be conditional in the > previous closed file. Will fix! > > GensrcJfr.gmk, line 94, please move )) to the left. > Will fix! > Looking closer at GensrcJfr.gmk, The macro SetupJfrGeneration looks > like it is only called once. This could be greatly simplified by just > taking the body of the macro and inlining all the inputs. This of > course unless you see a need in the future to generate additional > files using the jfr tool. Will fix this in a follow up bug: https://bugs.openjdk.java.net/browse/JDK-8203221 Thanks Erik > > /Erik > > On 2018-05-14 07:36, Erik Gahlin wrote: >> Here is an updated webrev: >> >> http://cr.openjdk.java.net/~egahlin/8199712.1/ [1] >> >> that incorporates: >> >> - build changes >> - new event prefix, i.e. "com.oracle.jdk.CPULoad" becomes "jdk.CPULoad" >> - obsolete command line options EnableTracing and UseLockedTracing >> - fixed typos in the Javadoc >> - simplified #include files >> >> RFEs have been filed for other issues, CSR is approved and tests pass. >> >> Erik and Markus >> >> [1] Parent: >> >> changeset: 50092:0e42d3120e51 >> >> user: clanger >> date: Sat May 12 10:26:42 2018 +0200 >> summary: 8202915: [JAXP] Performance enhancements and cleanups in >> com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator >> >> >>> Greetings, >>> >>> Could I have a review of 8199712: Flight Recorder >>> >>> As mentioned in the preview [1] the tracing backend has been >>> removed. Event metadata has been consolidated into a single XML file >>> and event classes are now generated by GenerateJfrFiles.java. >>> >>> Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. >>> >>> For details about the feature, see the JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8193393 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~egahlin/8199712.0/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8199712 >>> >>> [1] >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359.html >>> >>> Thanks >>> Erik and Markus >> > From per.liden at oracle.com Tue May 15 13:31:56 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 15 May 2018 15:31:56 +0200 Subject: RFR: 8203220: Introduce ATTRIBUTE_ALIGNED macro Message-ID: <8882c62a-bdd0-fb7b-89ee-ecfae3a38cea@oracle.com> This patch introduces a global ATTRIBUTE_ALIGNED(x) macro, which expands to __attribute__((aligned(x)) or the equivalent of that. We already use this attribute is some parts of the code, but we haven't had a macro in globalDefinitions.hpp for this. In the src/hotspot/cpu/x86/macroAssembler_x86_*.cpp files I chose to keep the ALIGNED_(x) and just define it as ATTRIBUTE_ALIGNED(x), instead of changing all uses of it. Also, for GCC I've included a workaround for a known bug (see the comment in the file). I thought it would be nice to include that since it might otherwise affect quite a few systems. And finally, we don't really have a uniform naming standard for these types of attributes. E.g. we have NOINLINE/ALWAYSINLINE and we have ATTRIBUTE_PRINTF. I chose to use the ATTRIBUTE_ prefix, instead of just ALIGNED, since ALIGNED is somewhat generic and we already have at least one place where a variable is named ALIGNED, so it would conflict. Bug: https://bugs.openjdk.java.net/browse/JDK-8203220 Webrev: http://cr.openjdk.java.net/~pliden/8203220/webrev.0 Testing: hs-tier{1,2} /Per From rkennke at redhat.com Tue May 15 13:40:56 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 15 May 2018 15:40:56 +0200 Subject: RFR: JDK-8202714: Create a MacroAssembler::access_load/store_at wrapper for AArch64 In-Reply-To: References: <233d81d0-d79f-dd2e-7052-6990e5654095@redhat.com> Message-ID: <86a87e81-3e4d-a699-7f13-cac9941b8164@redhat.com> Am 15.05.2018 um 15:07 schrieb Andrew Haley: > >> On 14 May 2018 at 13:02, Roman Kennke wrote: >>> This reshuffles AArch64 code around BarrierSetAssembler calls follow the >>> equivalent code in x86: >>> >>> http://cr.openjdk.java.net/~rkennke/JDK-8202714/webrev.00/ >>> >>> No regressions in hotspot/tier1 >>> >>> Can I please get a review? > > Looks OK. This use of an explict base class override in MacroAssembler > is rather nasty: > > 3990 void MacroAssembler::access_store_at(BasicType type, DecoratorSet decorators, > 3991 Address dst, Register src, > 3992 Register tmp1, Register thread_tmp) { > 3993 BarrierSetAssembler *bs = BarrierSet::barrier_set()->barrier_set_assembler(); > 3994 bool as_raw = (decorators & AS_RAW) != 0; > 3995 if (as_raw) { > 3996 bs->BarrierSetAssembler::store_at(this, decorators, type, dst, src, tmp1, thread_tmp); > 3997 } else { > 3998 bs->store_at(this, decorators, type, dst, src, tmp1, thread_tmp); > 3999 } > 4000 } > 4001 > It just mirrors what is done in x86 and elsewhere. :-) Roman From aph at redhat.com Tue May 15 14:35:24 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 15 May 2018 15:35:24 +0100 Subject: RFR: JDK-8202714: Create a MacroAssembler::access_load/store_at wrapper for AArch64 In-Reply-To: <86a87e81-3e4d-a699-7f13-cac9941b8164@redhat.com> References: <233d81d0-d79f-dd2e-7052-6990e5654095@redhat.com> <86a87e81-3e4d-a699-7f13-cac9941b8164@redhat.com> Message-ID: <8f82dcf8-9ea2-8e77-19b8-b173145f6349@redhat.com> On 05/15/2018 02:40 PM, Roman Kennke wrote: > Am 15.05.2018 um 15:07 schrieb Andrew Haley: >> >>> On 14 May 2018 at 13:02, Roman Kennke wrote: >>>> This reshuffles AArch64 code around BarrierSetAssembler calls follow the >>>> equivalent code in x86: >>>> >>>> http://cr.openjdk.java.net/~rkennke/JDK-8202714/webrev.00/ >>>> >>>> No regressions in hotspot/tier1 >>>> >>>> Can I please get a review? >> >> Looks OK. This use of an explict base class override in MacroAssembler >> is rather nasty: >> >> 3990 void MacroAssembler::access_store_at(BasicType type, DecoratorSet decorators, >> 3991 Address dst, Register src, >> 3992 Register tmp1, Register thread_tmp) { >> 3993 BarrierSetAssembler *bs = BarrierSet::barrier_set()->barrier_set_assembler(); >> 3994 bool as_raw = (decorators & AS_RAW) != 0; >> 3995 if (as_raw) { >> 3996 bs->BarrierSetAssembler::store_at(this, decorators, type, dst, src, tmp1, thread_tmp); >> 3997 } else { >> 3998 bs->store_at(this, decorators, type, dst, src, tmp1, thread_tmp); >> 3999 } >> 4000 } >> 4001 > > > It just mirrors what is done in x86 and elsewhere. :-) Yeah, I'm sure it does, but one class using internal knowledge of another class's inheritance hierarchy is an anti-pattern. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From kim.barrett at oracle.com Tue May 15 14:36:26 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 15 May 2018 10:36:26 -0400 Subject: RFR: 8203220: Introduce ATTRIBUTE_ALIGNED macro In-Reply-To: <8882c62a-bdd0-fb7b-89ee-ecfae3a38cea@oracle.com> References: <8882c62a-bdd0-fb7b-89ee-ecfae3a38cea@oracle.com> Message-ID: > On May 15, 2018, at 9:31 AM, Per Liden wrote: > > This patch introduces a global ATTRIBUTE_ALIGNED(x) macro, which expands to __attribute__((aligned(x)) or the equivalent of that. > > We already use this attribute is some parts of the code, but we haven't had a macro in globalDefinitions.hpp for this. > > In the src/hotspot/cpu/x86/macroAssembler_x86_*.cpp files I chose to keep the ALIGNED_(x) and just define it as ATTRIBUTE_ALIGNED(x), instead of changing all uses of it. > > Also, for GCC I've included a workaround for a known bug (see the comment in the file). I thought it would be nice to include that since it might otherwise affect quite a few systems. > > And finally, we don't really have a uniform naming standard for these types of attributes. E.g. we have NOINLINE/ALWAYSINLINE and we have ATTRIBUTE_PRINTF. I chose to use the ATTRIBUTE_ prefix, instead of just ALIGNED, since ALIGNED is somewhat generic and we already have at least one place where a variable is named ALIGNED, so it would conflict. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203220 > Webrev: http://cr.openjdk.java.net/~pliden/8203220/webrev.0 > > Testing: hs-tier{1,2} > > /Per Looks good. From per.liden at oracle.com Tue May 15 14:50:17 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 15 May 2018 16:50:17 +0200 Subject: RFR: 8203220: Introduce ATTRIBUTE_ALIGNED macro In-Reply-To: References: <8882c62a-bdd0-fb7b-89ee-ecfae3a38cea@oracle.com> Message-ID: On 05/15/2018 04:36 PM, Kim Barrett wrote: >> On May 15, 2018, at 9:31 AM, Per Liden wrote: >> >> This patch introduces a global ATTRIBUTE_ALIGNED(x) macro, which expands to __attribute__((aligned(x)) or the equivalent of that. >> >> We already use this attribute is some parts of the code, but we haven't had a macro in globalDefinitions.hpp for this. >> >> In the src/hotspot/cpu/x86/macroAssembler_x86_*.cpp files I chose to keep the ALIGNED_(x) and just define it as ATTRIBUTE_ALIGNED(x), instead of changing all uses of it. >> >> Also, for GCC I've included a workaround for a known bug (see the comment in the file). I thought it would be nice to include that since it might otherwise affect quite a few systems. >> >> And finally, we don't really have a uniform naming standard for these types of attributes. E.g. we have NOINLINE/ALWAYSINLINE and we have ATTRIBUTE_PRINTF. I chose to use the ATTRIBUTE_ prefix, instead of just ALIGNED, since ALIGNED is somewhat generic and we already have at least one place where a variable is named ALIGNED, so it would conflict. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203220 >> Webrev: http://cr.openjdk.java.net/~pliden/8203220/webrev.0 >> >> Testing: hs-tier{1,2} >> >> /Per > > Looks good. > Thanks for reviewing, Kim! /Per From rkennke at redhat.com Tue May 15 14:53:43 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 15 May 2018 16:53:43 +0200 Subject: RFR: JDK-8202714: Create a MacroAssembler::access_load/store_at wrapper for AArch64 In-Reply-To: <8f82dcf8-9ea2-8e77-19b8-b173145f6349@redhat.com> References: <233d81d0-d79f-dd2e-7052-6990e5654095@redhat.com> <86a87e81-3e4d-a699-7f13-cac9941b8164@redhat.com> <8f82dcf8-9ea2-8e77-19b8-b173145f6349@redhat.com> Message-ID: Am 15.05.2018 um 16:35 schrieb Andrew Haley: > On 05/15/2018 02:40 PM, Roman Kennke wrote: >> Am 15.05.2018 um 15:07 schrieb Andrew Haley: >>> >>>> On 14 May 2018 at 13:02, Roman Kennke wrote: >>>>> This reshuffles AArch64 code around BarrierSetAssembler calls follow the >>>>> equivalent code in x86: >>>>> >>>>> http://cr.openjdk.java.net/~rkennke/JDK-8202714/webrev.00/ >>>>> >>>>> No regressions in hotspot/tier1 >>>>> >>>>> Can I please get a review? >>> >>> Looks OK. This use of an explict base class override in MacroAssembler >>> is rather nasty: >>> >>> 3990 void MacroAssembler::access_store_at(BasicType type, DecoratorSet decorators, >>> 3991 Address dst, Register src, >>> 3992 Register tmp1, Register thread_tmp) { >>> 3993 BarrierSetAssembler *bs = BarrierSet::barrier_set()->barrier_set_assembler(); >>> 3994 bool as_raw = (decorators & AS_RAW) != 0; >>> 3995 if (as_raw) { >>> 3996 bs->BarrierSetAssembler::store_at(this, decorators, type, dst, src, tmp1, thread_tmp); >>> 3997 } else { >>> 3998 bs->store_at(this, decorators, type, dst, src, tmp1, thread_tmp); >>> 3999 } >>> 4000 } >>> 4001 >> >> >> It just mirrors what is done in x86 and elsewhere. :-) > Yeah, I'm sure it does, but one class using internal knowledge of another > class's inheritance hierarchy is an anti-pattern. > Right. I guess we could have all the implementations check for AS_RAW, and if so, pass the call up the inheritance hierarchy until it hits the base class BarrierSetAssembler. Sounds like more boilerplate but more clean. I'd do that under a separate RFE though, the goal of this one was to align aarch64 with x86 implementation, changing the call chain would affect all the implementations. Roman From aph at redhat.com Tue May 15 14:59:25 2018 From: aph at redhat.com (Andrew Haley) Date: Tue, 15 May 2018 15:59:25 +0100 Subject: RFR: JDK-8202714: Create a MacroAssembler::access_load/store_at wrapper for AArch64 In-Reply-To: References: <233d81d0-d79f-dd2e-7052-6990e5654095@redhat.com> <86a87e81-3e4d-a699-7f13-cac9941b8164@redhat.com> <8f82dcf8-9ea2-8e77-19b8-b173145f6349@redhat.com> Message-ID: <92944a03-fa15-0e32-3c0c-b0fb72c3ca8e@redhat.com> On 05/15/2018 03:53 PM, Roman Kennke wrote: > Right. I guess we could have all the implementations check for AS_RAW, > and if so, pass the call up the inheritance hierarchy until it hits the > base class BarrierSetAssembler. Sounds like more boilerplate but more > clean. I'd do that under a separate RFE though, the goal of this one was > to align aarch64 with x86 implementation, changing the call chain would > affect all the implementations. OK, under protest. I think the time to stop this kind of thing is at review time, but I can live with it. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From thomas.schatzl at oracle.com Tue May 15 15:07:29 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 15 May 2018 17:07:29 +0200 Subject: RFR: 8203220: Introduce ATTRIBUTE_ALIGNED macro In-Reply-To: <8882c62a-bdd0-fb7b-89ee-ecfae3a38cea@oracle.com> References: <8882c62a-bdd0-fb7b-89ee-ecfae3a38cea@oracle.com> Message-ID: <5ccebd70f7377107d9c1d41e04b7fd4acbae6d48.camel@oracle.com> Hi, On Tue, 2018-05-15 at 15:31 +0200, Per Liden wrote: > This patch introduces a global ATTRIBUTE_ALIGNED(x) macro, which > expands > to __attribute__((aligned(x)) or the equivalent of that. > > We already use this attribute is some parts of the code, but we > haven't > had a macro in globalDefinitions.hpp for this. > > In the src/hotspot/cpu/x86/macroAssembler_x86_*.cpp files I chose to > keep the ALIGNED_(x) and just define it as ATTRIBUTE_ALIGNED(x), > instead of changing all uses of it. > > Also, for GCC I've included a workaround for a known bug (see the > comment in the file). I thought it would be nice to include that > since it might otherwise affect quite a few systems. > > And finally, we don't really have a uniform naming standard for > these types of attributes. E.g. we have NOINLINE/ALWAYSINLINE and we > have ATTRIBUTE_PRINTF. I chose to use the ATTRIBUTE_ prefix, instead > of just ALIGNED, since ALIGNED is somewhat generic and we already > have at least one place where a variable is named ALIGNED, so it > would conflict. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203220 > Webrev: http://cr.openjdk.java.net/~pliden/8203220/webrev.0 - it would have been nice to have the ALIGNED_ macro in macroAssembler_* replaced by the new ATTRIBUTE_ALIGNED - there are only a few uses of it and it does not seem worth to have an extra macro in all these places. Feel free to ignore, but if you change that I do not need to re-review. Looks good. Thanks, Thomas From per.liden at oracle.com Tue May 15 15:07:35 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 15 May 2018 17:07:35 +0200 Subject: RFR: 8203227: Introduce os::processor_id() for Linux and Solaris Message-ID: <327220bf-ea05-1921-59e1-2bd0c2c4dea8@oracle.com> This patch introduces os::processor_id() to get the id of the CPU that the caller is currently executing on. Obviously, the returned id should only be viewed as a strong hint, since the information might be instantly out-of-date. We're using this in ZGC to do various CPU-local operations. We currently only need this for Linux and Solaris, which is what this patch adds. Support for additional platforms can be added when needed. I didn't add dummy os::processor_id() to the not-yet-implemented platforms, since I think it's better to get a early compile-time error instead of a Unimplemented() at runtime, if you try to use them. Bug: https://bugs.openjdk.java.net/browse/JDK-8203227 Webrev: http://cr.openjdk.java.net/~pliden/8203227/webrev.0 /Per From per.liden at oracle.com Tue May 15 15:45:20 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 15 May 2018 17:45:20 +0200 Subject: RFR: 8203220: Introduce ATTRIBUTE_ALIGNED macro In-Reply-To: <5ccebd70f7377107d9c1d41e04b7fd4acbae6d48.camel@oracle.com> References: <8882c62a-bdd0-fb7b-89ee-ecfae3a38cea@oracle.com> <5ccebd70f7377107d9c1d41e04b7fd4acbae6d48.camel@oracle.com> Message-ID: <032701d9-f838-cc41-ae26-863db9949c89@oracle.com> On 05/15/2018 05:07 PM, Thomas Schatzl wrote: > Hi, > > On Tue, 2018-05-15 at 15:31 +0200, Per Liden wrote: >> This patch introduces a global ATTRIBUTE_ALIGNED(x) macro, which >> expands >> to __attribute__((aligned(x)) or the equivalent of that. >> >> We already use this attribute is some parts of the code, but we >> haven't >> had a macro in globalDefinitions.hpp for this. >> >> In the src/hotspot/cpu/x86/macroAssembler_x86_*.cpp files I chose to >> keep the ALIGNED_(x) and just define it as ATTRIBUTE_ALIGNED(x), >> instead of changing all uses of it. >> >> Also, for GCC I've included a workaround for a known bug (see the >> comment in the file). I thought it would be nice to include that >> since it might otherwise affect quite a few systems. >> >> And finally, we don't really have a uniform naming standard for >> these types of attributes. E.g. we have NOINLINE/ALWAYSINLINE and we >> have ATTRIBUTE_PRINTF. I chose to use the ATTRIBUTE_ prefix, instead >> of just ALIGNED, since ALIGNED is somewhat generic and we already >> have at least one place where a variable is named ALIGNED, so it >> would conflict. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203220 >> Webrev: http://cr.openjdk.java.net/~pliden/8203220/webrev.0 > > - it would have been nice to have the ALIGNED_ macro in > macroAssembler_* replaced by the new ATTRIBUTE_ALIGNED - there are only > a few uses of it and it does not seem worth to have an extra macro in > all these places. > > Feel free to ignore, but if you change that I do not need to re-review. There seem to be ~100 uses of ALIGNED_, but I don't mind changing that. Inc: http://cr.openjdk.java.net/~pliden/8203220/webrev.0vs1 Full: http://cr.openjdk.java.net/~pliden/8203220/webrev.1 > > Looks good. Thanks for reviewing, Thomas! /Per > > Thanks, > Thomas > From thomas.schatzl at oracle.com Tue May 15 16:44:03 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 15 May 2018 18:44:03 +0200 Subject: RFR: 8203220: Introduce ATTRIBUTE_ALIGNED macro In-Reply-To: <032701d9-f838-cc41-ae26-863db9949c89@oracle.com> References: <8882c62a-bdd0-fb7b-89ee-ecfae3a38cea@oracle.com> <5ccebd70f7377107d9c1d41e04b7fd4acbae6d48.camel@oracle.com> <032701d9-f838-cc41-ae26-863db9949c89@oracle.com> Message-ID: <9232d02d70ed89ffde3963fdec0eeacc05bddaa1.camel@oracle.com> Hi, On Tue, 2018-05-15 at 17:45 +0200, Per Liden wrote: > On 05/15/2018 05:07 PM, Thomas Schatzl wrote: > > Hi, > > > > On Tue, 2018-05-15 at 15:31 +0200, Per Liden wrote: > > > This patch introduces a global ATTRIBUTE_ALIGNED(x) macro, which > > > expands > > > to __attribute__((aligned(x)) or the equivalent of that. > > > > > > [...] > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203220 > > > Webrev: http://cr.openjdk.java.net/~pliden/8203220/webrev.0 > > > > - it would have been nice to have the ALIGNED_ macro in > > macroAssembler_* replaced by the new ATTRIBUTE_ALIGNED - there are > > only a few uses of it and it does not seem worth to have an extra > > macro in all these places. > > > > Feel free to ignore, but if you change that I do not need to re- > > review. > > There seem to be ~100 uses of ALIGNED_, but I don't mind changing > that. I admit I only looked at the first two files or so, and they did not use the ALIGNED_ macro a lot. > > Inc: http://cr.openjdk.java.net/~pliden/8203220/webrev.0vs1 > Full: http://cr.openjdk.java.net/~pliden/8203220/webrev.1 > > > > > Looks good. > > Thanks for reviewing, Thomas! Still good. Thomas From jcbeyler at google.com Tue May 15 16:57:42 2018 From: jcbeyler at google.com (JC Beyler) Date: Tue, 15 May 2018 09:57:42 -0700 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: References: <5cdb2496-bad5-fa4a-f98e-37ae6a853f6b@oracle.com> <56bbfb13-3527-669a-b155-d699be1ef42e@oracle.com> <3c1d88fd-a3bc-350a-15fb-ec33118b7626@oracle.com> Message-ID: Hi Thomas, Thanks for the review! I've done your two fixes for the next webrev. I'll perhaps wait for the next review to combine webrev updates? Your two change requests were minimal it might make sense to wait before re-generating a new webrev. Anybody else have any comments/suggestions? Here are the latest links: Webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.20/ JEP-331 Issue: https://bugs.openjdk.java.net/browse/JDK-8171119 CSR: https://bugs.openjdk.java.net/browse/JDK-8194905 Test Plan: https://bugs.openjdk.java.net/browse/JDK-8202566 Thanks again all! Jc On Tue, May 15, 2018 at 12:23 AM Thomas Schatzl wrote: > Hi, > > On Mon, 2018-05-14 at 13:02 -0700, JC Beyler wrote: > > Hi Robbin and all, > > > > Thank you for your continuous help! > > > > Done then! Here is the new webrev: > > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.20/ > > > > and the incremental is: > > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.19_20/ > > > > Thanks again all! > > Jc > > > > looks good to me. > > Two minor issues that you might want to fix before pushing: > > - collectedHeap.hpp: The declaration of > CollectedHeap::allocate_memory() should be private. > > - collectedHeap.inline.hpp: in CollectedHeap::allocate_memory() there > is this completely duplicated code which you might factor out into > another (private) method: > > if (init_memory) { > obj = ... > } else { > obj = ... > } > post_setup(klass, ...); > > Thanks, > Thomas > > From alexander.harlap at oracle.com Tue May 15 16:59:39 2018 From: alexander.harlap at oracle.com (Alexander Harlap) Date: Tue, 15 May 2018 12:59:39 -0400 Subject: RFR: JDK-8189271: Metaspace::_capacity_until_GC should be size_t Message-ID: Small cleanup suggested by Coleen. Webrev: http://cr.openjdk.java.net/~aharlap/8189271/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8189271 Tested by running standard build and test Alex From thomas.stuefe at gmail.com Tue May 15 17:06:19 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 15 May 2018 19:06:19 +0200 Subject: RFR: JDK-8189271: Metaspace::_capacity_until_GC should be size_t In-Reply-To: References: Message-ID: Looks fine, good cleanup. Thanks, Thomas On May 15, 2018 19:00, "Alexander Harlap" wrote: Small cleanup suggested by Coleen. Webrev: http://cr.openjdk.java.net/~aharlap/8189271/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8189271 Tested by running standard build and test Alex From rkennke at redhat.com Tue May 15 17:15:04 2018 From: rkennke at redhat.com (Roman Kennke) Date: Tue, 15 May 2018 19:15:04 +0200 Subject: RFR: JDK-8202714: Create a MacroAssembler::access_load/store_at wrapper for AArch64 In-Reply-To: <92944a03-fa15-0e32-3c0c-b0fb72c3ca8e@redhat.com> References: <233d81d0-d79f-dd2e-7052-6990e5654095@redhat.com> <86a87e81-3e4d-a699-7f13-cac9941b8164@redhat.com> <8f82dcf8-9ea2-8e77-19b8-b173145f6349@redhat.com> <92944a03-fa15-0e32-3c0c-b0fb72c3ca8e@redhat.com> Message-ID: Am 15.05.2018 um 16:59 schrieb Andrew Haley: > On 05/15/2018 03:53 PM, Roman Kennke wrote: >> Right. I guess we could have all the implementations check for AS_RAW, >> and if so, pass the call up the inheritance hierarchy until it hits the >> base class BarrierSetAssembler. Sounds like more boilerplate but more >> clean. I'd do that under a separate RFE though, the goal of this one was >> to align aarch64 with x86 implementation, changing the call chain would >> affect all the implementations. > > OK, under protest. I think the time to stop this kind of thing is > at review time, but I can live with it. > https://bugs.openjdk.java.net/browse/JDK-8203232 Will take care of it (x86 and aarch64) right now. Roman From thomas.schatzl at oracle.com Tue May 15 17:56:00 2018 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 15 May 2018 19:56:00 +0200 Subject: RFR: JDK-8189271: Metaspace::_capacity_until_GC should be size_t In-Reply-To: References: Message-ID: <55575d3b3e46e9701c6f365378e463cbae16c038.camel@oracle.com> Hi, On Tue, 2018-05-15 at 12:59 -0400, Alexander Harlap wrote: > Small cleanup suggested by Coleen. > > Webrev: http://cr.openjdk.java.net/~aharlap/8189271/webrev.00/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8189271 > > Tested by running standard build and test > looks good. Thomas From kim.barrett at oracle.com Tue May 15 17:56:43 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 15 May 2018 13:56:43 -0400 Subject: RFR: 8203220: Introduce ATTRIBUTE_ALIGNED macro In-Reply-To: <032701d9-f838-cc41-ae26-863db9949c89@oracle.com> References: <8882c62a-bdd0-fb7b-89ee-ecfae3a38cea@oracle.com> <5ccebd70f7377107d9c1d41e04b7fd4acbae6d48.camel@oracle.com> <032701d9-f838-cc41-ae26-863db9949c89@oracle.com> Message-ID: <2755767A-7EF5-4C36-8E56-A0EEBB4D8D14@oracle.com> > On May 15, 2018, at 11:45 AM, Per Liden wrote: > > On 05/15/2018 05:07 PM, Thomas Schatzl wrote: >> Hi, >> On Tue, 2018-05-15 at 15:31 +0200, Per Liden wrote: >>> This patch introduces a global ATTRIBUTE_ALIGNED(x) macro, which >>> expands >>> to __attribute__((aligned(x)) or the equivalent of that. >>> >>> [?] >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203220 >>> Webrev: http://cr.openjdk.java.net/~pliden/8203220/webrev.0 >> - it would have been nice to have the ALIGNED_ macro in >> macroAssembler_* replaced by the new ATTRIBUTE_ALIGNED - there are only >> a few uses of it and it does not seem worth to have an extra macro in >> all these places. >> Feel free to ignore, but if you change that I do not need to re-review. > > There seem to be ~100 uses of ALIGNED_, but I don't mind changing that. > > Inc: http://cr.openjdk.java.net/~pliden/8203220/webrev.0vs1 > Full: http://cr.openjdk.java.net/~pliden/8203220/webrev.1 Still looks good. Remember to update the copyrights. From igor.ignatyev at oracle.com Tue May 15 18:58:48 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 15 May 2018 11:58:48 -0700 Subject: RFR(L) : 8199370: [TESTBUG] Open source vm testbase GC tests In-Reply-To: References: Message-ID: <8D49F7BE-84F3-4E3D-B9DC-30233E005263@oracle.com> Hi Erik, please see my answers inline. can I consider this RFR as reviewed by you? Thanks, -- Igor > On May 14, 2018, at 4:23 AM, Erik Helin wrote: > > On 05/08/2018 12:35 AM, Igor Ignatyev wrote: >> Hi all, > > Hi Igor, > > On 05/08/2018 12:35 AM, Igor Ignatyev wrote: >> could you please review the patch which open sources GC tests from vm testbase? it introduces the following test groups: >> - vmTestbase_vm_g1classunloading >> - vmTestbase_vm_gc_compact >> - vmTestbase_vm_gc_concurrent >> - vmTestbase_vm_gc_container >> - vmTestbase_vm_gc_juggle >> - vmTestbase_vm_gc_locker >> - vmTestbase_vm_gc_ref >> - vmTestbase_vm_gc_misc > > This is a very welcome, and pretty massive, change :) I won't be able to read through each test, there are simple too many, but I can sample a few of them and have a look. > > On 05/08/2018 12:35 AM, Igor Ignatyev wrote: >> As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > I'm also assuming that to help the open sourcing of these tests, most comments will likely be deferred until later? If so, that is fine with me. y, unless it is something very important and cost of delaying changes is high, I'd prefer to defer all comments/improvements till later. > > On 05/08/2018 12:35 AM, Igor Ignatyev wrote: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8199370 >> webrev: http://cr.openjdk.java.net/~iignatyev/8199370/webrev.00/index.html > > Many of the tests are a bit cryptic, but that can be refactored later. Could you please file bugs for the two tests written in shell? Particularly parOld/test.sh should be trivial to rewrite in Java. sure, I've filed 8203239 and 8203238. > > It seems like a lot of tests contains a TEST.properties file with the content `exclusiveAccess.dirs=.`. Could this become the default value somewhere, so we don't need all those TEST.properties files? y, but this will require finding a most top directory whose all tests have this TEST.properties file and it will also make it harder to understand how a test is executed. there was/is ongoing discussing w/ Jon on moving 'exclusiveAccess' to test description. anyhow I've filed an RFE to clean that up -- 8203241. > > Thanks, > Erik > >> Thanks, >> -- Igor From igor.ignatyev at oracle.com Tue May 15 19:24:03 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 15 May 2018 12:24:03 -0700 Subject: RFR(M) : 8202392: [TESTBUG] open source vm testbase heapdump tests In-Reply-To: References: <5AF5A02A.7010107@oracle.com> Message-ID: <78837961-81B0-4F39-987E-0F9A8444EDEE@oracle.com> Misha, Serguei, thank you for your review. Cheers, -- Igor > On May 11, 2018, at 6:10 PM, serguei.spitsyn at oracle.com wrote: > > +1 > > Thanks, > Serguei > > On 5/11/18 06:52, Mikhailo Seledtsov wrote: >> Looks good to me, >> >> Misha >> >> On 5/9/18, 5:14 PM, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev//8202392/webrev.00/index.html >>>> 1396 lines changed: 1396 ins; 0 del; 0 mod; >>> Hi all, >>> >>> could you please review his patch which open sources heapdump tests from so-called VM testbase? as it's obvious from test names, they test heap dumping using jmap and a number of JVM flags. >>> >>> As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. >>> >>> webrev: http://cr.openjdk.java.net/~iignatyev//8202392/webrev.00/index.html >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8202392 >>> testing: :vmTestbase_vm_heapdump test group >>> >>> Thanks, >>> -- Igor > From poonam.bajaj at oracle.com Tue May 15 20:46:22 2018 From: poonam.bajaj at oracle.com (Poonam Parhar) Date: Tue, 15 May 2018 13:46:22 -0700 Subject: RFR (8u): JDK-8146115: Improve docker container detection and resource configuration usage Message-ID: <3968d009-c1a2-87e7-bc22-70c348ee5b69@oracle.com> Hello, Please review the docker container support changes backported to JDK 8u. These changes include the backport of the following enhancement and the follow-on bug fixes done on top of that in jdk 10 and 11. Webrev: http://cr.openjdk.java.net/~poonam/8146115/webrev.00/ Enhancement:JDK-8146115: Improve docker container detection and resource configuration usage The changes also include the fixes for the following two bugs: Bug JDK-8186248: Allow more flexibility in selecting Heap % of available RAM BugJDK-8190283: Default heap sizing options select a MaxHeapSize larger than available physical memory in some cases These changes add a new JVM option 'PrintContainerInfo' for tracing container related information which is specific to jdk 8. Testing results (with -XX:+UnlockDiagnosticVMOptions -XX:+PrintContainerInfo -XX:+PrintActiveCpus JVM options): -------------- poonam at poonam-VirtualBox:~/docker-image$ java TestCPUsMemory Number of Processors: 3 Max Memory: 921174016 poonam at poonam-VirtualBox:~/docker-image$ sudo docker run --cpus 1 -m1024m? --rm myimage [sudo] password for poonam: WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. OSContainer::init: Initializing Container Support Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes Memory Limit is: 1073741824 active_processor_count: sched_getaffinity processor count: 3 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: 100000 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 1024 CPU Quota count based on quota/period: 1 OSContainer::active_processor_count: 1 active_processor_count: determined by OSContainer: 1 active_processor_count: sched_getaffinity processor count: 3 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: 100000 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 1024 CPU Quota count based on quota/period: 1 OSContainer::active_processor_count: 1 active_processor_count: determined by OSContainer: 1 Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes Memory Limit is: 1073741824 total container memory: 1073741824 active_processor_count: sched_getaffinity processor count: 3 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: 100000 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 1024 CPU Quota count based on quota/period: 1 OSContainer::active_processor_count: 1 active_processor_count: determined by OSContainer: 1 active_processor_count: sched_getaffinity processor count: 3 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: 100000 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 1024 CPU Quota count based on quota/period: 1 OSContainer::active_processor_count: 1 active_processor_count: determined by OSContainer: 1 active_processor_count: sched_getaffinity processor count: 3 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: 100000 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 1024 CPU Quota count based on quota/period: 1 OSContainer::active_processor_count: 1 active_processor_count: determined by OSContainer: 1 Number of Processors: 1 Max Memory: 259522560 poonam at poonam-VirtualBox:~/docker-image$ sudo docker run --cpu-shares 2048 -m1024m? --rm myimage WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. OSContainer::init: Initializing Container Support Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes Memory Limit is: 1073741824 active_processor_count: sched_getaffinity processor count: 3 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: -1 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 2048 CPU Share count based on shares: 2 OSContainer::active_processor_count: 2 active_processor_count: determined by OSContainer: 2 active_processor_count: sched_getaffinity processor count: 3 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: -1 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 2048 CPU Share count based on shares: 2 OSContainer::active_processor_count: 2 active_processor_count: determined by OSContainer: 2 Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes Memory Limit is: 1073741824 total container memory: 1073741824 Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes Memory Limit is: 1073741824 total container memory: 1073741824 active_processor_count: sched_getaffinity processor count: 3 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: -1 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 2048 CPU Share count based on shares: 2 OSContainer::active_processor_count: 2 active_processor_count: determined by OSContainer: 2 active_processor_count: sched_getaffinity processor count: 3 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: -1 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 2048 CPU Share count based on shares: 2 OSContainer::active_processor_count: 2 active_processor_count: determined by OSContainer: 2 active_processor_count: sched_getaffinity processor count: 3 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: -1 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 2048 CPU Share count based on shares: 2 OSContainer::active_processor_count: 2 active_processor_count: determined by OSContainer: 2 Number of Processors: 2 Max Memory: 259522560 poonam at poonam-VirtualBox:~/docker-image$ sudo docker run --cpuset-cpus 0-1 -m1024m? --rm myimage WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. OSContainer::init: Initializing Container Support Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes Memory Limit is: 1073741824 active_processor_count: sched_getaffinity processor count: 2 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: -1 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 1024 OSContainer::active_processor_count: 2 active_processor_count: determined by OSContainer: 2 active_processor_count: sched_getaffinity processor count: 2 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: -1 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 1024 OSContainer::active_processor_count: 2 active_processor_count: determined by OSContainer: 2 Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes Memory Limit is: 1073741824 total container memory: 1073741824 Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes Memory Limit is: 1073741824 total container memory: 1073741824 active_processor_count: sched_getaffinity processor count: 2 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: -1 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 1024 OSContainer::active_processor_count: 2 active_processor_count: determined by OSContainer: 2 active_processor_count: sched_getaffinity processor count: 2 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: -1 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 1024 OSContainer::active_processor_count: 2 active_processor_count: determined by OSContainer: 2 active_processor_count: sched_getaffinity processor count: 2 Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us CPU Quota is: -1 Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us CPU Period is: 100000 Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares CPU Shares is: 1024 OSContainer::active_processor_count: 2 active_processor_count: determined by OSContainer: 2 Number of Processors: 2 Max Memory: 259522560 ------------------ Thanks, Poonam From shade at redhat.com Tue May 15 21:19:32 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 15 May 2018 23:19:32 +0200 Subject: RFR (XS) 8203251: Non-PCH build failed after JDK-8199712 (Flight Recorder) Message-ID: A few missing includes cause this: https://bugs.openjdk.java.net/browse/JDK-8203251 This fixes x86_64: # HG changeset patch # User shade # Date 1526419080 -7200 # Tue May 15 23:18:00 2018 +0200 # Node ID 2957fddc6b4ef6b51cb093fb525cb90f952d4fdd # Parent d2cfda6a00de3afb9fe96db0a640e5cd4c597bfc 8203251: Non-PCH build failed after JDK-8199712 (Flight Recorder) Reviewed-by: XXX diff -r d2cfda6a00de -r 2957fddc6b4e src/hotspot/share/jfr/recorder/checkpoint/jfrCheckpointManager.cpp --- a/src/hotspot/share/jfr/recorder/checkpoint/jfrCheckpointManager.cpp Tue May 15 13:28:08 2018 -0700 +++ b/src/hotspot/share/jfr/recorder/checkpoint/jfrCheckpointManager.cpp Tue May 15 23:18:00 2018 +0200 @@ -39,6 +39,7 @@ #include "memory/resourceArea.hpp" #include "runtime/mutexLocker.hpp" #include "runtime/orderAccess.inline.hpp" +#include "runtime/os.inline.hpp" #include "runtime/safepoint.hpp" typedef JfrCheckpointManager::Buffer* BufferPtr; diff -r d2cfda6a00de -r 2957fddc6b4e src/hotspot/share/jfr/recorder/repository/jfrChunkWriter.cpp --- a/src/hotspot/share/jfr/recorder/repository/jfrChunkWriter.cpp Tue May 15 13:28:08 2018 -0700 +++ b/src/hotspot/share/jfr/recorder/repository/jfrChunkWriter.cpp Tue May 15 23:18:00 2018 +0200 @@ -30,6 +30,7 @@ #include "jfr/utilities/jfrTypes.hpp" #include "runtime/mutexLocker.hpp" #include "runtime/os.hpp" +#include "runtime/os.inline.hpp" const u2 JFR_VERSION_MAJOR = 2; const u2 JFR_VERSION_MINOR = 0; diff -r d2cfda6a00de -r 2957fddc6b4e src/hotspot/share/jfr/recorder/stacktrace/jfrStackTraceRepository.cpp --- a/src/hotspot/share/jfr/recorder/stacktrace/jfrStackTraceRepository.cpp Tue May 15 13:28:08 2018 -0700 +++ b/src/hotspot/share/jfr/recorder/stacktrace/jfrStackTraceRepository.cpp Tue May 15 23:18:00 2018 +0200 @@ -32,6 +32,7 @@ #include "memory/allocation.inline.hpp" #include "runtime/mutexLocker.hpp" #include "runtime/safepoint.hpp" +#include "runtime/os.inline.hpp" #include "runtime/task.hpp" #include "runtime/vframe.inline.hpp" diff -r d2cfda6a00de -r 2957fddc6b4e src/hotspot/share/jfr/recorder/storage/jfrStorage.cpp --- a/src/hotspot/share/jfr/recorder/storage/jfrStorage.cpp Tue May 15 13:28:08 2018 -0700 +++ b/src/hotspot/share/jfr/recorder/storage/jfrStorage.cpp Tue May 15 23:18:00 2018 +0200 @@ -39,6 +39,7 @@ #include "logging/log.hpp" #include "runtime/mutexLocker.hpp" #include "runtime/orderAccess.inline.hpp" +#include "runtime/os.inline.hpp" #include "runtime/safepoint.hpp" #include "runtime/thread.hpp" Thanks, -Aleksey From igor.ignatyev at oracle.com Tue May 15 21:21:42 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 15 May 2018 14:21:42 -0700 Subject: RFR(L) : 8199379 : [TESTBUG] Open source vm testbase JDB tests Message-ID: <6D060F7C-C265-48CF-BB21-532848461B2A@oracle.com> http://cr.openjdk.java.net/~iignatyev//8199379/webrev.00/index.html > 18000 lines changed: 17999 ins; 0 del; 1 mod; Hi all, could you please review this patch which open sources JDB tests from VM testbase? these tests were developed to test 'jdb' tool and JDI thru it. As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. JBS: https://bugs.openjdk.java.net/browse/JDK-8199379 webrev: http://cr.openjdk.java.net/~iignatyev//8199379/webrev.00/index.html testing: :vmTestbase_nsk_jdb test group Thanks, -- Igor From markus.gronlund at oracle.com Tue May 15 21:25:33 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Tue, 15 May 2018 23:25:33 +0200 Subject: RFR (XS) 8203251: Non-PCH build failed after JDK-8199712 (Flight Recorder) In-Reply-To: References: Message-ID: <5617ED29-136D-40FA-BC42-3778117BD226@oracle.com> Hi Aleksey, Thanks for being so responsive and sorry for this inconvenience. Trivial, please put it back. Many thanks Markus Ps will follow on this why this was not caught. Sent from my iPhone > On 15 May 2018, at 23:19, Aleksey Shipilev wrote: > > A few missing includes cause this: > https://bugs.openjdk.java.net/browse/JDK-8203251 > > This fixes x86_64: > > # HG changeset patch > # User shade > # Date 1526419080 -7200 > # Tue May 15 23:18:00 2018 +0200 > # Node ID 2957fddc6b4ef6b51cb093fb525cb90f952d4fdd > # Parent d2cfda6a00de3afb9fe96db0a640e5cd4c597bfc > 8203251: Non-PCH build failed after JDK-8199712 (Flight Recorder) > Reviewed-by: XXX > > diff -r d2cfda6a00de -r 2957fddc6b4e src/hotspot/share/jfr/recorder/checkpoint/jfrCheckpointManager.cpp > --- a/src/hotspot/share/jfr/recorder/checkpoint/jfrCheckpointManager.cpp Tue May 15 13:28:08 2018 -0700 > +++ b/src/hotspot/share/jfr/recorder/checkpoint/jfrCheckpointManager.cpp Tue May 15 23:18:00 2018 +0200 > @@ -39,6 +39,7 @@ > #include "memory/resourceArea.hpp" > #include "runtime/mutexLocker.hpp" > #include "runtime/orderAccess.inline.hpp" > +#include "runtime/os.inline.hpp" > #include "runtime/safepoint.hpp" > > typedef JfrCheckpointManager::Buffer* BufferPtr; > diff -r d2cfda6a00de -r 2957fddc6b4e src/hotspot/share/jfr/recorder/repository/jfrChunkWriter.cpp > --- a/src/hotspot/share/jfr/recorder/repository/jfrChunkWriter.cpp Tue May 15 13:28:08 2018 -0700 > +++ b/src/hotspot/share/jfr/recorder/repository/jfrChunkWriter.cpp Tue May 15 23:18:00 2018 +0200 > @@ -30,6 +30,7 @@ > #include "jfr/utilities/jfrTypes.hpp" > #include "runtime/mutexLocker.hpp" > #include "runtime/os.hpp" > +#include "runtime/os.inline.hpp" > > const u2 JFR_VERSION_MAJOR = 2; > const u2 JFR_VERSION_MINOR = 0; > diff -r d2cfda6a00de -r 2957fddc6b4e > src/hotspot/share/jfr/recorder/stacktrace/jfrStackTraceRepository.cpp > --- a/src/hotspot/share/jfr/recorder/stacktrace/jfrStackTraceRepository.cpp Tue May 15 13:28:08 2018 > -0700 > +++ b/src/hotspot/share/jfr/recorder/stacktrace/jfrStackTraceRepository.cpp Tue May 15 23:18:00 2018 > +0200 > @@ -32,6 +32,7 @@ > #include "memory/allocation.inline.hpp" > #include "runtime/mutexLocker.hpp" > #include "runtime/safepoint.hpp" > +#include "runtime/os.inline.hpp" > #include "runtime/task.hpp" > #include "runtime/vframe.inline.hpp" > > diff -r d2cfda6a00de -r 2957fddc6b4e src/hotspot/share/jfr/recorder/storage/jfrStorage.cpp > --- a/src/hotspot/share/jfr/recorder/storage/jfrStorage.cpp Tue May 15 13:28:08 2018 -0700 > +++ b/src/hotspot/share/jfr/recorder/storage/jfrStorage.cpp Tue May 15 23:18:00 2018 +0200 > @@ -39,6 +39,7 @@ > #include "logging/log.hpp" > #include "runtime/mutexLocker.hpp" > #include "runtime/orderAccess.inline.hpp" > +#include "runtime/os.inline.hpp" > #include "runtime/safepoint.hpp" > #include "runtime/thread.hpp" > > Thanks, > -Aleksey > From shade at redhat.com Tue May 15 21:38:22 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 15 May 2018 23:38:22 +0200 Subject: RFR (XS) 8203251: Non-PCH build failed after JDK-8199712 (Flight Recorder) In-Reply-To: <5617ED29-136D-40FA-BC42-3778117BD226@oracle.com> References: <5617ED29-136D-40FA-BC42-3778117BD226@oracle.com> Message-ID: <2dcd25b4-0fb2-fc80-874b-8b75186d454f@redhat.com> On 05/15/2018 11:25 PM, Markus Gronlund wrote: > Thanks for being so responsive and sorry for this inconvenience. > Trivial, please put it back. No problem, pushed. I expect more breakages tomorrow morning, after other platforms/configurations complete their builds. > Ps will follow on this why this was not caught. That's easy: we have been asking Oracle folks to teach their CIs to build with --disable-precompiled-headers in order to capture header weirdness early. While there is a reason to build with PCH on developer machines saving human time, there seem to be no reason not to force robots to pay a little extra. Thanks, -Aleksey From igor.ignatyev at oracle.com Tue May 15 23:16:17 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 15 May 2018 16:16:17 -0700 Subject: RFR(M) : 8202946 : [TESTBUG] Open source VM testbase OOM tests Message-ID: <2DD8F9C6-8471-4BF6-8573-0DA3F2B6C66B@oracle.com> http://cr.openjdk.java.net/~iignatyev//8202946/webrev.00/index.html > 1619 lines changed: 1619 ins; 0 del; 0 mod; Hi all, could you please review this patch which open sources OOM tests from VM testbase? these tests test OutOfMemoryError throwing in different scenarios. As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. JBS: https://bugs.openjdk.java.net/browse/JDK-8202946 webrev: http://cr.openjdk.java.net/~iignatyev//8202946/webrev.00/index.html testing: :vmTestbase_vm_oom test group Thanks, -- Igor From david.holmes at oracle.com Wed May 16 02:24:56 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 16 May 2018 12:24:56 +1000 Subject: RFR: 8203227: Introduce os::processor_id() for Linux and Solaris In-Reply-To: <327220bf-ea05-1921-59e1-2bd0c2c4dea8@oracle.com> References: <327220bf-ea05-1921-59e1-2bd0c2c4dea8@oracle.com> Message-ID: <13cbc7ca-58eb-ae0b-9640-42be8e5fb3fd@oracle.com> Hi Per, On 16/05/2018 1:07 AM, Per Liden wrote: > This patch introduces os::processor_id() to get the id of the CPU that > the caller is currently executing on. Obviously, the returned id should > only be viewed as a strong hint, since the information might be > instantly out-of-date. Or even wrong to begin with: https://stackoverflow.com/questions/36934736/is-sched-getcpu-reliable-on-linux caveat-emptor! :) > We're using this in ZGC to do various CPU-local operations. We currently So there's a requirement that the CPU value returned by processor_id() is compatible with some other API's that take a processor id? Does that need documenting in some form? > only need this for Linux and Solaris, which is what this patch adds. Are the API's that take the processor id defined in os as well? Otherwise it seems this functionality should just be localized to the specific os_.cpp file from which it is used (and/or the os:: class). > Support for additional platforms can be added when needed. I didn't add > dummy os::processor_id() to the not-yet-implemented platforms, since I > think it's better to get a early compile-time error instead of a > Unimplemented() at runtime, if you try to use them. I'd rather not see the runtime support of our primary platforms of interest fragmented like this. Are there technical challenges to doing this? > Bug: https://bugs.openjdk.java.net/browse/JDK-8203227 > Webrev: http://cr.openjdk.java.net/~pliden/8203227/webrev.0 For Solaris there is no need to introduce Solaris::getcpuid() just because Linux already has that. (Unless this ends up not being needed in the os class of course :) ). Are you allowing for the -1 return value on Linux or assuming it will never happen in practice? If the former then that should be documented as part of the semantics of os::processor_id(). If the latter then you should probably add an assert or guarantee. I suspect this is a historical restriction no longer relevant to us - in which case we can probably clean out the dynamic lookup/linking!). Thanks, David > /Per From per.liden at oracle.com Wed May 16 05:48:23 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 16 May 2018 07:48:23 +0200 Subject: RFR: 8203220: Introduce ATTRIBUTE_ALIGNED macro In-Reply-To: <9232d02d70ed89ffde3963fdec0eeacc05bddaa1.camel@oracle.com> References: <8882c62a-bdd0-fb7b-89ee-ecfae3a38cea@oracle.com> <5ccebd70f7377107d9c1d41e04b7fd4acbae6d48.camel@oracle.com> <032701d9-f838-cc41-ae26-863db9949c89@oracle.com> <9232d02d70ed89ffde3963fdec0eeacc05bddaa1.camel@oracle.com> Message-ID: On 05/15/2018 06:44 PM, Thomas Schatzl wrote: > Hi, > > On Tue, 2018-05-15 at 17:45 +0200, Per Liden wrote: >> On 05/15/2018 05:07 PM, Thomas Schatzl wrote: >>> Hi, >>> >>> On Tue, 2018-05-15 at 15:31 +0200, Per Liden wrote: >>>> This patch introduces a global ATTRIBUTE_ALIGNED(x) macro, which >>>> expands >>>> to __attribute__((aligned(x)) or the equivalent of that. >>>> >>>> [...] >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203220 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8203220/webrev.0 >>> >>> - it would have been nice to have the ALIGNED_ macro in >>> macroAssembler_* replaced by the new ATTRIBUTE_ALIGNED - there are >>> only a few uses of it and it does not seem worth to have an extra >>> macro in all these places. >>> >>> Feel free to ignore, but if you change that I do not need to re- >>> review. >> >> There seem to be ~100 uses of ALIGNED_, but I don't mind changing >> that. > > I admit I only looked at the first two files or so, and they did not > use the ALIGNED_ macro a lot. > >> >> Inc: http://cr.openjdk.java.net/~pliden/8203220/webrev.0vs1 >> Full: http://cr.openjdk.java.net/~pliden/8203220/webrev.1 >> >>> >>> Looks good. >> >> Thanks for reviewing, Thomas! > > Still good. Thanks Thomas! /Per From per.liden at oracle.com Wed May 16 05:48:52 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 16 May 2018 07:48:52 +0200 Subject: RFR: 8203220: Introduce ATTRIBUTE_ALIGNED macro In-Reply-To: <2755767A-7EF5-4C36-8E56-A0EEBB4D8D14@oracle.com> References: <8882c62a-bdd0-fb7b-89ee-ecfae3a38cea@oracle.com> <5ccebd70f7377107d9c1d41e04b7fd4acbae6d48.camel@oracle.com> <032701d9-f838-cc41-ae26-863db9949c89@oracle.com> <2755767A-7EF5-4C36-8E56-A0EEBB4D8D14@oracle.com> Message-ID: <2b05941b-97d4-487c-2595-b92adcdd3f13@oracle.com> On 05/15/2018 07:56 PM, Kim Barrett wrote: >> On May 15, 2018, at 11:45 AM, Per Liden wrote: >> >> On 05/15/2018 05:07 PM, Thomas Schatzl wrote: >>> Hi, >>> On Tue, 2018-05-15 at 15:31 +0200, Per Liden wrote: >>>> This patch introduces a global ATTRIBUTE_ALIGNED(x) macro, which >>>> expands >>>> to __attribute__((aligned(x)) or the equivalent of that. >>>> >>>> [?] >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203220 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8203220/webrev.0 >>> - it would have been nice to have the ALIGNED_ macro in >>> macroAssembler_* replaced by the new ATTRIBUTE_ALIGNED - there are only >>> a few uses of it and it does not seem worth to have an extra macro in >>> all these places. >>> Feel free to ignore, but if you change that I do not need to re-review. >> >> There seem to be ~100 uses of ALIGNED_, but I don't mind changing that. >> >> Inc: http://cr.openjdk.java.net/~pliden/8203220/webrev.0vs1 >> Full: http://cr.openjdk.java.net/~pliden/8203220/webrev.1 > > Still looks good. > > Remember to update the copyrights. > Thanks Kim! /Per From shade at redhat.com Wed May 16 07:59:20 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 16 May 2018 09:59:20 +0200 Subject: RFR (S) 8203278: AArch64/PPC64 build failures after JDK-8199712 (Flight Recorder) Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8203278 The trouble with AArch64 and PPC64 is that they define thread_state specially: http://hg.openjdk.java.net/jdk/jdk/file/bf9177eac58d/src/hotspot/share/runtime/thread.hpp#l1162 So most paths that use non-trivial things from thread.hpp have to include thread.inline.hpp. It became exposed after JFR push, I have no clear understanding how. I tried to make the fix contained by only changing .cpp files, but after 30 minutes of whack-a-mole game, I have given up and added it to resourceArea.hpp, where most build failures originate. Testing: x86_64 build, aarch64 build Fix: # HG changeset patch # User shade # Date 1526457236 -7200 # Wed May 16 09:53:56 2018 +0200 # Node ID 69d14a6872028b27c207586351ec9293c00b49f3 # Parent fb66b2959eafb060ed835686c3aed08e343d6775 8203278: AArch64/PPC64 build failures after JDK-8199712 (Flight Recorder) Reviewed-by: XXX diff -r fb66b2959eaf -r 69d14a687202 src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp --- a/src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp Tue May 15 23:37:37 2018 +0200 +++ b/src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp Wed May 16 09:53:56 2018 +0200 @@ -34,6 +34,7 @@ #include "runtime/atomic.hpp" #include "runtime/orderAccess.inline.hpp" #include "runtime/thread.hpp" +#include "runtime/thread.inline.hpp" #include "runtime/vmThread.hpp" #include "utilities/ostream.hpp" diff -r fb66b2959eaf -r 69d14a687202 src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeSetUtils.cpp --- a/src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeSetUtils.cpp Tue May 15 23:37:37 2018 +0200 +++ b/src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeSetUtils.cpp Wed May 16 09:53:56 2018 +0200 @@ -27,6 +27,7 @@ #include "oops/instanceKlass.hpp" #include "oops/oop.inline.hpp" #include "oops/symbol.hpp" +#include "runtime/thread.inline.hpp" JfrSymbolId::JfrSymbolId() : _symbol_id_counter(0), _sym_table(new SymbolTable(this)), _cstring_table(new CStringTable(this)) { assert(_sym_table != NULL, "invariant"); diff -r fb66b2959eaf -r 69d14a687202 src/hotspot/share/jfr/support/jfrEventClass.cpp --- a/src/hotspot/share/jfr/support/jfrEventClass.cpp Tue May 15 23:37:37 2018 +0200 +++ b/src/hotspot/share/jfr/support/jfrEventClass.cpp Wed May 16 09:53:56 2018 +0200 @@ -25,6 +25,7 @@ #include "precompiled.hpp" #include "jfr/recorder/checkpoint/types/traceid/jfrTraceId.inline.hpp" #include "jfr/support/jfrEventClass.hpp" +#include "runtime/thread.inline.hpp" bool JdkJfrEvent::is(const Klass* k) { return JfrTraceId::is_jdk_jfr_event(k); diff -r fb66b2959eaf -r 69d14a687202 src/hotspot/share/memory/resourceArea.hpp --- a/src/hotspot/share/memory/resourceArea.hpp Tue May 15 23:37:37 2018 +0200 +++ b/src/hotspot/share/memory/resourceArea.hpp Wed May 16 09:53:56 2018 +0200 @@ -27,6 +27,7 @@ #include "memory/allocation.hpp" #include "runtime/thread.hpp" +#include "runtime/thread.inline.hpp" // The resource area holds temporary data structures in the VM. // The actual allocation areas are thread local. Typical usage: From markus.gronlund at oracle.com Wed May 16 08:59:43 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 16 May 2018 01:59:43 -0700 (PDT) Subject: RFR (S) 8203278: AArch64/PPC64 build failures after JDK-8199712 (Flight Recorder) In-Reply-To: References: Message-ID: <51e1559e-7892-468f-8a64-b1065d7914c1@default> Hi again Aleksey, Again, our apologies for having created this extra work for you. Let's see if I can assist you here: If it is the non-definition of thread->thread_state() that is causing problems, I see these three suspect JFR related areas where it is called without (explicitly) including "runtime/thread.inline.hpp (e.g. they only have "runtime/thread.hpp): 1) jfr/jni/jfrJNIMethodRegistration.cpp 2) jfr/leakprofiler/leakprofiler.cpp 3) jfr/checkpoint/types/traceid/jfrTraceId.inline.hpp <<-- probably the most culpable location, causing spread. Can you please try to update these locations, changing from: #include "runtime/thread.hpp" to #include "runtime/thread.inline.hpp" Maybe this allows us to contain the fix a bit. Cheers Markus PS not yet tested (will do now), hoping you do not end up circular. -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: den 16 maj 2018 09:59 To: hotspot-dev at openjdk.java.net Subject: RFR (S) 8203278: AArch64/PPC64 build failures after JDK-8199712 (Flight Recorder) Bug: https://bugs.openjdk.java.net/browse/JDK-8203278 The trouble with AArch64 and PPC64 is that they define thread_state specially: http://hg.openjdk.java.net/jdk/jdk/file/bf9177eac58d/src/hotspot/share/runtime/thread.hpp#l1162 So most paths that use non-trivial things from thread.hpp have to include thread.inline.hpp. It became exposed after JFR push, I have no clear understanding how. I tried to make the fix contained by only changing .cpp files, but after 30 minutes of whack-a-mole game, I have given up and added it to resourceArea.hpp, where most build failures originate. Testing: x86_64 build, aarch64 build Fix: # HG changeset patch # User shade # Date 1526457236 -7200 # Wed May 16 09:53:56 2018 +0200 # Node ID 69d14a6872028b27c207586351ec9293c00b49f3 # Parent fb66b2959eafb060ed835686c3aed08e343d6775 8203278: AArch64/PPC64 build failures after JDK-8199712 (Flight Recorder) Reviewed-by: XXX diff -r fb66b2959eaf -r 69d14a687202 src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp --- a/src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp Tue May 15 23:37:37 2018 +0200 +++ b/src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp Wed May 16 09:53:56 2018 +0200 @@ -34,6 +34,7 @@ #include "runtime/atomic.hpp" #include "runtime/orderAccess.inline.hpp" #include "runtime/thread.hpp" +#include "runtime/thread.inline.hpp" #include "runtime/vmThread.hpp" #include "utilities/ostream.hpp" diff -r fb66b2959eaf -r 69d14a687202 src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeSetUtils.cpp --- a/src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeSetUtils.cpp Tue May 15 23:37:37 2018 +0200 +++ b/src/hotspot/share/jfr/recorder/checkpoint/types/jfrTypeSetUtils.cpp Wed May 16 09:53:56 2018 +0200 @@ -27,6 +27,7 @@ #include "oops/instanceKlass.hpp" #include "oops/oop.inline.hpp" #include "oops/symbol.hpp" +#include "runtime/thread.inline.hpp" JfrSymbolId::JfrSymbolId() : _symbol_id_counter(0), _sym_table(new SymbolTable(this)), _cstring_table(new CStringTable(this)) { assert(_sym_table != NULL, "invariant"); diff -r fb66b2959eaf -r 69d14a687202 src/hotspot/share/jfr/support/jfrEventClass.cpp --- a/src/hotspot/share/jfr/support/jfrEventClass.cpp Tue May 15 23:37:37 2018 +0200 +++ b/src/hotspot/share/jfr/support/jfrEventClass.cpp Wed May 16 09:53:56 2018 +0200 @@ -25,6 +25,7 @@ #include "precompiled.hpp" #include "jfr/recorder/checkpoint/types/traceid/jfrTraceId.inline.hpp" #include "jfr/support/jfrEventClass.hpp" +#include "runtime/thread.inline.hpp" bool JdkJfrEvent::is(const Klass* k) { return JfrTraceId::is_jdk_jfr_event(k); diff -r fb66b2959eaf -r 69d14a687202 src/hotspot/share/memory/resourceArea.hpp --- a/src/hotspot/share/memory/resourceArea.hpp Tue May 15 23:37:37 2018 +0200 +++ b/src/hotspot/share/memory/resourceArea.hpp Wed May 16 09:53:56 2018 +0200 @@ -27,6 +27,7 @@ #include "memory/allocation.hpp" #include "runtime/thread.hpp" +#include "runtime/thread.inline.hpp" // The resource area holds temporary data structures in the VM. // The actual allocation areas are thread local. Typical usage: From shade at redhat.com Wed May 16 09:08:23 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 16 May 2018 11:08:23 +0200 Subject: RFR (S): 8203274: 32-bit build failures after JDK-8199712 (Flight Recorder) Message-ID: <5f5f6c52-3301-3fb9-b868-7929c8dd8d30@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8203274 Testing: x86_64 build, x86_32 build, arm32-hflt build Fix: diff -r 2e886b7b3d72 -r a7c343a2f1c4 src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.cpp --- a/src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.cpp Wed May 16 09:58:24 2018 +0200 +++ b/src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.cpp Wed May 16 11:00:04 2018 +0200 @@ -1232,7 +1232,7 @@ // This means the original constant pool contents are copied unmodified writer.bytes(orig_stream->buffer(), orig_access_flag_offset); assert(writer.is_valid(), "invariant"); - assert(writer.current_offset() == orig_access_flag_offset, "invariant"); // same positions + assert(writer.current_offset() == (intptr_t)orig_access_flag_offset, "invariant"); // same positions // Our writer now sits just after the last original constant pool entry. // I.e. we are in a good position to append new constant pool entries // This array will contain the resolved indexes diff -r 2e886b7b3d72 -r a7c343a2f1c4 src/hotspot/share/jfr/recorder/stringpool/jfrStringPoolBuffer.cpp --- a/src/hotspot/share/jfr/recorder/stringpool/jfrStringPoolBuffer.cpp Wed May 16 09:58:24 2018 +0200 +++ b/src/hotspot/share/jfr/recorder/stringpool/jfrStringPoolBuffer.cpp Wed May 16 11:00:04 2018 +0200 @@ -55,7 +55,17 @@ } void JfrStringPoolBuffer::increment(uint64_t value) { +#if !(defined(ARM) || defined(IA32)) Atomic::add(value, &_string_count_pos); +#else + // TODO: This should be fixed in Atomic::add handling for 32-bit platforms, + // see JDK-8203283. We workaround the absence of support right here. + uint64_t cur, val; + do { + cur = Atomic::load(&_string_count_top); + val = cur + value; + } while (Atomic::cmpxchg(val, &_string_count_pos, cur) != cur); +#endif } void JfrStringPoolBuffer::set_string_top(uint64_t value) { diff -r 2e886b7b3d72 -r a7c343a2f1c4 src/hotspot/share/jfr/utilities/jfrAllocation.cpp --- a/src/hotspot/share/jfr/utilities/jfrAllocation.cpp Wed May 16 09:58:24 2018 +0200 +++ b/src/hotspot/share/jfr/utilities/jfrAllocation.cpp Wed May 16 11:00:04 2018 +0200 @@ -66,8 +66,8 @@ const size_t total_deallocated = atomic_add_jlong(dealloc_size, &_deallocated_bytes); const size_t current_live_set = atomic_add_jlong(dealloc_size * -1, &_live_set_bytes); log_trace(jfr, system)("Deallocation: [" SIZE_FORMAT "] bytes", dealloc_size); - log_trace(jfr, system)("Total dealloc [" JLONG_FORMAT "] bytes", total_deallocated); - log_trace(jfr, system)("Liveset: [" JLONG_FORMAT "] bytes", current_live_set); + log_trace(jfr, system)("Total dealloc [" SIZE_FORMAT "] bytes", total_deallocated); + log_trace(jfr, system)("Liveset: [" SIZE_FORMAT "] bytes", current_live_set); } } Thanks, -Aleksey From shade at redhat.com Wed May 16 09:27:27 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 16 May 2018 11:27:27 +0200 Subject: RFR (S) 8203278: AArch64/PPC64 build failures after JDK-8199712 (Flight Recorder) In-Reply-To: <51e1559e-7892-468f-8a64-b1065d7914c1@default> References: <51e1559e-7892-468f-8a64-b1065d7914c1@default> Message-ID: On 05/16/2018 10:59 AM, Markus Gronlund wrote: > Again, our apologies for having created this extra work for you. Yeah, as I said yesterday, it was bound to happen :) > Let's see if I can assist you here: > > If it is the non-definition of thread->thread_state() that is causing problems, I see these three suspect JFR related areas where it is called without (explicitly) including "runtime/thread.inline.hpp (e.g. they only have "runtime/thread.hpp): > > 1) jfr/jni/jfrJNIMethodRegistration.cpp > 2) jfr/leakprofiler/leakprofiler.cpp > 3) jfr/checkpoint/types/traceid/jfrTraceId.inline.hpp <<-- probably the most culpable location, causing spread. Oh, I get it now! Basically, only (2) and (3) are needed to build on AArch64: diff -r bf9177eac58d src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp --- a/src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp Tue May 15 19:26:00 2018 -0400 +++ b/src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp Wed May 16 11:25:39 2018 +0200 @@ -33,7 +33,7 @@ #include "oops/oop.hpp" #include "runtime/atomic.hpp" #include "runtime/orderAccess.inline.hpp" -#include "runtime/thread.hpp" +#include "runtime/thread.inline.hpp" #include "runtime/vmThread.hpp" #include "utilities/ostream.hpp" diff -r bf9177eac58d src/hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceId.inline.hpp --- a/src/hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceId.inline.hpp Tue May 15 19:26:00 2018 -0400 +++ b/src/hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceId.inline.hpp Wed May 16 11:25:39 2018 +0200 @@ -34,7 +34,7 @@ #include "oops/klass.hpp" #include "oops/instanceKlass.hpp" #include "oops/method.hpp" -#include "runtime/thread.hpp" +#include "runtime/thread.inline.hpp" #include "utilities/debug.hpp" template Thanks, -Aleksey From markus.gronlund at oracle.com Wed May 16 10:10:30 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 16 May 2018 03:10:30 -0700 (PDT) Subject: RFR (S) 8203278: AArch64/PPC64 build failures after JDK-8199712 (Flight Recorder) In-Reply-To: References: <51e1559e-7892-468f-8a64-b1065d7914c1@default> Message-ID: <05a588ec-4389-40e3-bf99-9a504af361bf@default> Looks good :) Markus -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: den 16 maj 2018 11:27 To: Markus Gronlund Cc: hotspot-dev at openjdk.java.net Subject: Re: RFR (S) 8203278: AArch64/PPC64 build failures after JDK-8199712 (Flight Recorder) On 05/16/2018 10:59 AM, Markus Gronlund wrote: > Again, our apologies for having created this extra work for you. Yeah, as I said yesterday, it was bound to happen :) > Let's see if I can assist you here: > > If it is the non-definition of thread->thread_state() that is causing problems, I see these three suspect JFR related areas where it is called without (explicitly) including "runtime/thread.inline.hpp (e.g. they only have "runtime/thread.hpp): > > 1) jfr/jni/jfrJNIMethodRegistration.cpp > 2) jfr/leakprofiler/leakprofiler.cpp > 3) jfr/checkpoint/types/traceid/jfrTraceId.inline.hpp <<-- probably the most culpable location, causing spread. Oh, I get it now! Basically, only (2) and (3) are needed to build on AArch64: diff -r bf9177eac58d src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp --- a/src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp Tue May 15 19:26:00 2018 -0400 +++ b/src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp Wed May 16 11:25:39 2018 +0200 @@ -33,7 +33,7 @@ #include "oops/oop.hpp" #include "runtime/atomic.hpp" #include "runtime/orderAccess.inline.hpp" -#include "runtime/thread.hpp" +#include "runtime/thread.inline.hpp" #include "runtime/vmThread.hpp" #include "utilities/ostream.hpp" diff -r bf9177eac58d src/hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceId.inline.hpp --- a/src/hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceId.inline.hpp Tue May 15 19:26:00 2018 -0400 +++ b/src/hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceId.inline.hpp Wed May 16 11:25:39 2018 +0200 @@ -34,7 +34,7 @@ #include "oops/klass.hpp" #include "oops/instanceKlass.hpp" #include "oops/method.hpp" -#include "runtime/thread.hpp" +#include "runtime/thread.inline.hpp" #include "utilities/debug.hpp" template Thanks, -Aleksey From markus.gronlund at oracle.com Wed May 16 10:24:02 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 16 May 2018 03:24:02 -0700 (PDT) Subject: RFR (S): 8203274: 32-bit build failures after JDK-8199712 (Flight Recorder) In-Reply-To: <5f5f6c52-3301-3fb9-b868-7929c8dd8d30@redhat.com> References: <5f5f6c52-3301-3fb9-b868-7929c8dd8d30@redhat.com> Message-ID: <464e4102-b36c-483d-83b3-acec42dab677@default> Thanks again Aleksey, Will create a follow-up bug/rfe about better convergence of primitive types. Also, there is an invariant that will allow us to not use CAS in the StringPoolBuffer, I am testing it out now, will send the patch to you when ready. That patch will also incorporate the changes you have done for jfrEventClassTransformer.cpp and jfrAllocation.cpp as well. Thanks Markus -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: den 16 maj 2018 11:08 To: hotspot-dev at openjdk.java.net Subject: RFR (S): 8203274: 32-bit build failures after JDK-8199712 (Flight Recorder) Bug: https://bugs.openjdk.java.net/browse/JDK-8203274 Testing: x86_64 build, x86_32 build, arm32-hflt build Fix: diff -r 2e886b7b3d72 -r a7c343a2f1c4 src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.cpp --- a/src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.cpp Wed May 16 09:58:24 2018 +0200 +++ b/src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.cpp Wed May 16 11:00:04 2018 +0200 @@ -1232,7 +1232,7 @@ // This means the original constant pool contents are copied unmodified writer.bytes(orig_stream->buffer(), orig_access_flag_offset); assert(writer.is_valid(), "invariant"); - assert(writer.current_offset() == orig_access_flag_offset, "invariant"); // same positions + assert(writer.current_offset() == (intptr_t)orig_access_flag_offset, + "invariant"); // same positions // Our writer now sits just after the last original constant pool entry. // I.e. we are in a good position to append new constant pool entries // This array will contain the resolved indexes diff -r 2e886b7b3d72 -r a7c343a2f1c4 src/hotspot/share/jfr/recorder/stringpool/jfrStringPoolBuffer.cpp --- a/src/hotspot/share/jfr/recorder/stringpool/jfrStringPoolBuffer.cpp Wed May 16 09:58:24 2018 +0200 +++ b/src/hotspot/share/jfr/recorder/stringpool/jfrStringPoolBuffer.cpp Wed May 16 11:00:04 2018 +0200 @@ -55,7 +55,17 @@ } void JfrStringPoolBuffer::increment(uint64_t value) { +#if !(defined(ARM) || defined(IA32)) Atomic::add(value, &_string_count_pos); +#else + // TODO: This should be fixed in Atomic::add handling for 32-bit +platforms, + // see JDK-8203283. We workaround the absence of support right here. + uint64_t cur, val; + do { + cur = Atomic::load(&_string_count_top); + val = cur + value; + } while (Atomic::cmpxchg(val, &_string_count_pos, cur) != cur); +#endif } void JfrStringPoolBuffer::set_string_top(uint64_t value) { diff -r 2e886b7b3d72 -r a7c343a2f1c4 src/hotspot/share/jfr/utilities/jfrAllocation.cpp --- a/src/hotspot/share/jfr/utilities/jfrAllocation.cpp Wed May 16 09:58:24 2018 +0200 +++ b/src/hotspot/share/jfr/utilities/jfrAllocation.cpp Wed May 16 11:00:04 2018 +0200 @@ -66,8 +66,8 @@ const size_t total_deallocated = atomic_add_jlong(dealloc_size, &_deallocated_bytes); const size_t current_live_set = atomic_add_jlong(dealloc_size * -1, &_live_set_bytes); log_trace(jfr, system)("Deallocation: [" SIZE_FORMAT "] bytes", dealloc_size); - log_trace(jfr, system)("Total dealloc [" JLONG_FORMAT "] bytes", total_deallocated); - log_trace(jfr, system)("Liveset: [" JLONG_FORMAT "] bytes", current_live_set); + log_trace(jfr, system)("Total dealloc [" SIZE_FORMAT "] bytes", total_deallocated); + log_trace(jfr, system)("Liveset: [" SIZE_FORMAT "] bytes", current_live_set); } } Thanks, -Aleksey From shade at redhat.com Wed May 16 10:35:04 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 16 May 2018 12:35:04 +0200 Subject: RFR (S): 8203274: 32-bit build failures after JDK-8199712 (Flight Recorder) In-Reply-To: <464e4102-b36c-483d-83b3-acec42dab677@default> References: <5f5f6c52-3301-3fb9-b868-7929c8dd8d30@redhat.com> <464e4102-b36c-483d-83b3-acec42dab677@default> Message-ID: <19285686-e3a7-c1c3-e2b9-18a4502a7a76@redhat.com> On 05/16/2018 12:24 PM, Markus Gronlund wrote: > Thanks again Aleksey, > > Will create a follow-up bug/rfe about better convergence of primitive types. > > Also, there is an invariant that will allow us to not use CAS in the StringPoolBuffer, I am > testing it out now, will send the patch to you when ready. > > That patch will also incorporate the changes you have done for jfrEventClassTransformer.cpp and > jfrAllocation.cpp as well. Ok, I'll push this build fix shortly, and then you can build on top? -Aleksey From markus.gronlund at oracle.com Wed May 16 10:36:42 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 16 May 2018 03:36:42 -0700 (PDT) Subject: RFR (S): 8203274: 32-bit build failures after JDK-8199712 (Flight Recorder) In-Reply-To: <19285686-e3a7-c1c3-e2b9-18a4502a7a76@redhat.com> References: <5f5f6c52-3301-3fb9-b868-7929c8dd8d30@redhat.com> <464e4102-b36c-483d-83b3-acec42dab677@default> <19285686-e3a7-c1c3-e2b9-18a4502a7a76@redhat.com> Message-ID: Absolutetly. Looks good. Markus -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: den 16 maj 2018 12:35 To: Markus Gronlund Cc: hotspot-dev at openjdk.java.net Subject: Re: RFR (S): 8203274: 32-bit build failures after JDK-8199712 (Flight Recorder) On 05/16/2018 12:24 PM, Markus Gronlund wrote: > Thanks again Aleksey, > > Will create a follow-up bug/rfe about better convergence of primitive types. > > Also, there is an invariant that will allow us to not use CAS in the > StringPoolBuffer, I am testing it out now, will send the patch to you when ready. > > That patch will also incorporate the changes you have done for > jfrEventClassTransformer.cpp and jfrAllocation.cpp as well. Ok, I'll push this build fix shortly, and then you can build on top? -Aleksey From per.liden at oracle.com Wed May 16 10:55:54 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 16 May 2018 12:55:54 +0200 Subject: RFR: 8203227: Introduce os::processor_id() for Linux and Solaris In-Reply-To: <13cbc7ca-58eb-ae0b-9640-42be8e5fb3fd@oracle.com> References: <327220bf-ea05-1921-59e1-2bd0c2c4dea8@oracle.com> <13cbc7ca-58eb-ae0b-9640-42be8e5fb3fd@oracle.com> Message-ID: <3eac8bb2-fc7a-42b2-154b-e7f6cd63b20b@oracle.com> Hi David, Thanks for looking at this. On 05/16/2018 04:24 AM, David Holmes wrote: > Hi Per, > > On 16/05/2018 1:07 AM, Per Liden wrote: >> This patch introduces os::processor_id() to get the id of the CPU that >> the caller is currently executing on. Obviously, the returned id >> should only be viewed as a strong hint, since the information might be >> instantly out-of-date. > > Or even wrong to begin with: > > https://stackoverflow.com/questions/36934736/is-sched-getcpu-reliable-on-linux > > > caveat-emptor! :) > >> We're using this in ZGC to do various CPU-local operations. We currently > > So there's a requirement that the CPU value returned by processor_id() > is compatible with some other API's that take a processor id? Does that > need documenting in some form? Good point, this should be made more clear. We currently assume that os::processor_id() returns a value from 0 to (os::processor_count() - 1). Adding a comment in os.hpp clarifying that. > >> only need this for Linux and Solaris, which is what this patch adds. > > Are the API's that take the processor id defined in os as well? > Otherwise it seems this functionality should just be localized to the > specific os_.cpp file from which it is used (and/or the os:: > class). The os::processor_id() call-sites we have are in shared os-agnostic code. This function is also fairly closely related to os::processor_count(), so it seems like os:: would be the natural place for this. > >> Support for additional platforms can be added when needed. I didn't >> add dummy os::processor_id() to the not-yet-implemented platforms, >> since I think it's better to get a early compile-time error instead of >> a Unimplemented() at runtime, if you try to use them. > > I'd rather not see the runtime support of our primary platforms of > interest fragmented like this. Are there technical challenges to doing > this? I'm a bit reluctant to add this for other platforms at this point, since some platforms might require non-trivial implementations, which would go unused/untested. For macOS I believe we would need something like this: http://cr.openjdk.java.net/~gziemski/zgc_os_processor_id/webrev/src/hotspot/os/bsd/os_bsd.cpp.udiff.html (I should add that I don't know if this patch does the right thing, but Gerard apparently had some success with it) For Windows, I believe a call to GetCurrentProcessorNumber() should be enough (we don't seem to handle/support logical processor groups in os::processor_count() so should be safe to do the same thing here). The plan has been to add support for these if/when ZGC supports these platforms (unless someone else needs and adds them before that). But adding them now makes me a bit uneasy, especially given the non-trivial nature of the macOS implementation. > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203227 >> Webrev: http://cr.openjdk.java.net/~pliden/8203227/webrev.0 > > For Solaris there is no need to introduce Solaris::getcpuid() just > because Linux already has that. (Unless this ends up not being needed in > the os class of course :) ). Check. > > Are you allowing for the -1 return value on Linux or assuming it will > never happen in practice? If the former then that should be documented > as part of the semantics of os::processor_id(). If the latter then you > should probably add an assert or guarantee. I suspect this is a > historical restriction no longer relevant to us - in which case we can > probably clean out the dynamic lookup/linking!). We're assuming it will never happen in practice, and this is currently guarded with a guarantee() during initialization of ZGC. sched_getcpu() has "only" been around since glibc 2.14 (June 2011). However, we can also receive -1 if the kernel doesn't have the getcpu() syscall (first introduced in Linux 2.6.19, Nov 2006). So, it seems safe to at least assume that the getcpu() syscall is available, and only have an single check for that during startup (with a vm_exit_during_initialization() if it's unavailable). Here's an updated webrev for continued discussion: http://cr.openjdk.java.net/~pliden/8203227/webrev.1 cheers, Per > > Thanks, > David > >> /Per From shade at redhat.com Wed May 16 10:58:15 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 16 May 2018 12:58:15 +0200 Subject: RFR 8203285: Minimal VM fails to build after JDK-8199712 (Flight Recorder) Message-ID: <029c7a1a-8f03-1903-1fbf-9f367e2a7999@redhat.com> JFR is not supported for Minimal VM, but it still fails to build. Bug (with build failure examples): https://bugs.openjdk.java.net/browse/JDK-8203285 Testing: minimal build, x86_64 build Fix: diff -r f222eba39694 src/hotspot/share/jfr/metadata/GenerateJfrFiles.java --- a/src/hotspot/share/jfr/metadata/GenerateJfrFiles.java Wed May 16 12:38:35 2018 +0200 +++ b/src/hotspot/share/jfr/metadata/GenerateJfrFiles.java Wed May 16 12:55:34 2018 +0200 @@ -450,6 +450,7 @@ out.write("#ifndef JFRFILES_JFREVENTCLASSES_HPP"); out.write("#define JFRFILES_JFREVENTCLASSES_HPP"); out.write(""); + out.write("#include \"oops/klass.hpp\""); out.write("#include \"jfrfiles/jfrTypes.hpp\""); out.write("#include \"jfr/utilities/jfrTypes.hpp\""); out.write("#include \"utilities/macros.hpp\""); @@ -689,4 +690,4 @@ private static void printField(Printer out, FieldElement field) { out.write(" " + field.getFieldType() + " _" + field.name + ";"); } -} \ No newline at end of file +} diff -r f222eba39694 src/hotspot/share/jfr/support/jfrThreadId.hpp --- a/src/hotspot/share/jfr/support/jfrThreadId.hpp Wed May 16 12:38:35 2018 +0200 +++ b/src/hotspot/share/jfr/support/jfrThreadId.hpp Wed May 16 12:55:34 2018 +0200 @@ -26,6 +26,7 @@ #define SHARE_VM_JFR_SUPPORT_JFRTHREADID_HPP #include "utilities/macros.hpp" +#include "utilities/globalDefinitions.hpp" #if INCLUDE_JFR #include "jfr/support/jfrThreadLocal.hpp" diff -r f222eba39694 src/hotspot/share/runtime/java.cpp --- a/src/hotspot/share/runtime/java.cpp Wed May 16 12:38:35 2018 +0200 +++ b/src/hotspot/share/runtime/java.cpp Wed May 16 12:55:34 2018 +0200 @@ -32,6 +32,7 @@ #include "compiler/compileBroker.hpp" #include "compiler/compilerOracle.hpp" #include "interpreter/bytecodeHistogram.hpp" +#include "jfr/jfrEvents.hpp" #include "jfr/support/jfrThreadId.hpp" #if INCLUDE_JVMCI #include "jvmci/jvmciCompiler.hpp" Thanks, -Aleksey From markus.gronlund at oracle.com Wed May 16 11:13:54 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 16 May 2018 04:13:54 -0700 (PDT) Subject: RFR 8203285: Minimal VM fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <029c7a1a-8f03-1903-1fbf-9f367e2a7999@redhat.com> References: <029c7a1a-8f03-1903-1fbf-9f367e2a7999@redhat.com> Message-ID: <25f592a4-55ec-42b0-b181-c10bcce27b3d@default> Looks good. Thank you very much for fixing this. Markus -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: den 16 maj 2018 12:58 To: hotspot-dev at openjdk.java.net Subject: RFR 8203285: Minimal VM fails to build after JDK-8199712 (Flight Recorder) JFR is not supported for Minimal VM, but it still fails to build. Bug (with build failure examples): https://bugs.openjdk.java.net/browse/JDK-8203285 Testing: minimal build, x86_64 build Fix: diff -r f222eba39694 src/hotspot/share/jfr/metadata/GenerateJfrFiles.java --- a/src/hotspot/share/jfr/metadata/GenerateJfrFiles.java Wed May 16 12:38:35 2018 +0200 +++ b/src/hotspot/share/jfr/metadata/GenerateJfrFiles.java Wed May 16 12:55:34 2018 +0200 @@ -450,6 +450,7 @@ out.write("#ifndef JFRFILES_JFREVENTCLASSES_HPP"); out.write("#define JFRFILES_JFREVENTCLASSES_HPP"); out.write(""); + out.write("#include \"oops/klass.hpp\""); out.write("#include \"jfrfiles/jfrTypes.hpp\""); out.write("#include \"jfr/utilities/jfrTypes.hpp\""); out.write("#include \"utilities/macros.hpp\""); @@ -689,4 +690,4 @@ private static void printField(Printer out, FieldElement field) { out.write(" " + field.getFieldType() + " _" + field.name + ";"); } -} \ No newline at end of file +} diff -r f222eba39694 src/hotspot/share/jfr/support/jfrThreadId.hpp --- a/src/hotspot/share/jfr/support/jfrThreadId.hpp Wed May 16 12:38:35 2018 +0200 +++ b/src/hotspot/share/jfr/support/jfrThreadId.hpp Wed May 16 12:55:34 2018 +0200 @@ -26,6 +26,7 @@ #define SHARE_VM_JFR_SUPPORT_JFRTHREADID_HPP #include "utilities/macros.hpp" +#include "utilities/globalDefinitions.hpp" #if INCLUDE_JFR #include "jfr/support/jfrThreadLocal.hpp" diff -r f222eba39694 src/hotspot/share/runtime/java.cpp --- a/src/hotspot/share/runtime/java.cpp Wed May 16 12:38:35 2018 +0200 +++ b/src/hotspot/share/runtime/java.cpp Wed May 16 12:55:34 2018 +0200 @@ -32,6 +32,7 @@ #include "compiler/compileBroker.hpp" #include "compiler/compilerOracle.hpp" #include "interpreter/bytecodeHistogram.hpp" +#include "jfr/jfrEvents.hpp" #include "jfr/support/jfrThreadId.hpp" #if INCLUDE_JVMCI #include "jvmci/jvmciCompiler.hpp" Thanks, -Aleksey From bsrbnd at gmail.com Wed May 16 11:27:13 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 16 May 2018 13:27:13 +0200 Subject: RFR: JDK-8202016: Use obj+offset in interpreter array access In-Reply-To: References: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> <041699e9-9c5b-0a8b-1b94-45a181018b47@redhat.com> <8b16fb83-25c4-8fcf-7ef6-13d3943f3fd7@redhat.com> <6cd6372a-b527-0b6d-6a61-72a3d1eb7319@redhat.com> Message-ID: On 15 May 2018 at 12:43, B. Blaser wrote: > On 15 May 2018 at 11:26, Roman Kennke wrote: >> Am 14.05.2018 um 23:47 schrieb B. Blaser: >>> Hi, >>> >>> On 14 May 2018 at 13:19, Andrew Dinn wrote: >>>> On 14/05/18 11:15, Roman Kennke wrote: >>>>> Am 07.05.2018 um 21:47 schrieb Roman Kennke: >>>>>> In the TemplateTable::aastore() and >>>>>> InterpreterMacroAssembler::load_resolved_reference_at_index(), the >>>>>> element address is flattened, and then passed to the BarrierSetAssembler >>>>>> for actual access. GCs like Shenandoah need the original obj though. >>>>>> >>>>>> The proposed change replaces the flattening with base+index+disp >>>>>> addressing, and removes the shift and add computations. In the case of >>>>>> aastore, we need to re-fetch the rcx/index from the stack because it >>>>>> gets trashed on the way. >>>>>> >>>>>> Testing: passed: hotspot/jtreg:tier1 >>>>>> >>>>>> Can I please get a review? >>>>>> Thanks, Roman >>>> Well, that x86 code change looks ok/ish/ -- although the need to reload >>>> from the stack is a tad disappointing. >>> >>> Maybe I'm a bit late, but I think we could use r8 & r9 to avoid some >>> unnecessary stack access, as next. >>> What do you think (tier1 seems OK)? >>> >> Yes, that is a good idea and I believe it would work. I think it can be >> improved even further by only moving it out to r8/r9 once, and use those >> registers for addressing. Do you want to file a followup-issue and do that? > > Yes, good suggestion. > I'll provide an improved patch and create a JBS issue. I've created https://bugs.openjdk.java.net/browse/JDK-8203286 The following patch uses r9 (=rcx) directly for addressing, but do_oop_store() still expects value in rax, see [1]. Passed: TEST="jtreg:test/hotspot/jtreg:tier1" I'd need someone to review & push the fix, would you be available for this? Thanks, Bernard [1] http://hg.openjdk.java.net/jdk/jdk/file/f222eba39694/src/hotspot/cpu/x86/templateTable_x86.cpp#l155 diff -r 24151f48582b src/hotspot/cpu/x86/templateTable_x86.cpp --- a/src/hotspot/cpu/x86/templateTable_x86.cpp Mon May 14 21:56:07 2018 +0200 +++ b/src/hotspot/cpu/x86/templateTable_x86.cpp Wed May 16 12:18:50 2018 +0200 @@ -1098,7 +1098,12 @@ __ movl(rcx, at_tos_p1()); // index __ movptr(rdx, at_tos_p2()); // array - Address element_address(rdx, rcx, +#ifdef _LP64 + __ movptr(r8, rax); + __ movl(r9, rcx); +#endif + + Address element_address(rdx, LP64_ONLY(r9) NOT_LP64(rcx), UseCompressedOops? Address::times_4 : Address::times_ptr, arrayOopDesc::base_offset_in_bytes(T_OBJECT)); @@ -1125,8 +1130,12 @@ __ bind(ok_is_subtype); // Get the value we will store +#ifdef _LP64 + __ movptr(rax, r8); // do_oop_store() expects value in rax +#else __ movptr(rax, at_tos()); __ movl(rcx, at_tos_p1()); // index +#endif // Now store using the appropriate barrier do_oop_store(_masm, element_address, rax, IN_HEAP_ARRAY); __ jmp(done); From david.holmes at oracle.com Wed May 16 11:48:26 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 16 May 2018 21:48:26 +1000 Subject: RFR (S) 8203278: AArch64/PPC64 build failures after JDK-8199712 (Flight Recorder) In-Reply-To: References: <51e1559e-7892-468f-8a64-b1065d7914c1@default> Message-ID: <68d13bc8-826d-ce0e-ef3c-531a28cc1f8d@oracle.com> That looks better. :) Thanks, David On 16/05/2018 7:27 PM, Aleksey Shipilev wrote: > On 05/16/2018 10:59 AM, Markus Gronlund wrote: >> Again, our apologies for having created this extra work for you. > > Yeah, as I said yesterday, it was bound to happen :) > >> Let's see if I can assist you here: >> >> If it is the non-definition of thread->thread_state() that is causing problems, I see these three suspect JFR related areas where it is called without (explicitly) including "runtime/thread.inline.hpp (e.g. they only have "runtime/thread.hpp): >> >> 1) jfr/jni/jfrJNIMethodRegistration.cpp >> 2) jfr/leakprofiler/leakprofiler.cpp >> 3) jfr/checkpoint/types/traceid/jfrTraceId.inline.hpp <<-- probably the most culpable location, causing spread. > > Oh, I get it now! Basically, only (2) and (3) are needed to build on AArch64: > > diff -r bf9177eac58d src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp > --- a/src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp Tue May 15 19:26:00 2018 -0400 > +++ b/src/hotspot/share/jfr/leakprofiler/leakProfiler.cpp Wed May 16 11:25:39 2018 +0200 > @@ -33,7 +33,7 @@ > #include "oops/oop.hpp" > #include "runtime/atomic.hpp" > #include "runtime/orderAccess.inline.hpp" > -#include "runtime/thread.hpp" > +#include "runtime/thread.inline.hpp" > #include "runtime/vmThread.hpp" > #include "utilities/ostream.hpp" > > diff -r bf9177eac58d src/hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceId.inline.hpp > --- a/src/hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceId.inline.hpp Tue May 15 > 19:26:00 2018 -0400 > +++ b/src/hotspot/share/jfr/recorder/checkpoint/types/traceid/jfrTraceId.inline.hpp Wed May 16 > 11:25:39 2018 +0200 > @@ -34,7 +34,7 @@ > #include "oops/klass.hpp" > #include "oops/instanceKlass.hpp" > #include "oops/method.hpp" > -#include "runtime/thread.hpp" > +#include "runtime/thread.inline.hpp" > #include "utilities/debug.hpp" > > template > > > Thanks, > -Aleksey > From david.holmes at oracle.com Wed May 16 11:55:19 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 16 May 2018 21:55:19 +1000 Subject: RFR (S): 8203274: 32-bit build failures after JDK-8199712 (Flight Recorder) In-Reply-To: <5f5f6c52-3301-3fb9-b868-7929c8dd8d30@redhat.com> References: <5f5f6c52-3301-3fb9-b868-7929c8dd8d30@redhat.com> Message-ID: On 16/05/2018 7:08 PM, Aleksey Shipilev wrote: > Bug: > https://bugs.openjdk.java.net/browse/JDK-8203274 > > Testing: x86_64 build, x86_32 build, arm32-hflt build > > Fix: > > diff -r 2e886b7b3d72 -r a7c343a2f1c4 src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.cpp > --- a/src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.cpp Wed May 16 09:58:24 2018 +0200 > +++ b/src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.cpp Wed May 16 11:00:04 2018 +0200 > @@ -1232,7 +1232,7 @@ > // This means the original constant pool contents are copied unmodified > writer.bytes(orig_stream->buffer(), orig_access_flag_offset); > assert(writer.is_valid(), "invariant"); > - assert(writer.current_offset() == orig_access_flag_offset, "invariant"); // same positions > + assert(writer.current_offset() == (intptr_t)orig_access_flag_offset, "invariant"); // same positions > // Our writer now sits just after the last original constant pool entry. > // I.e. we are in a good position to append new constant pool entries > // This array will contain the resolved indexes > diff -r 2e886b7b3d72 -r a7c343a2f1c4 src/hotspot/share/jfr/recorder/stringpool/jfrStringPoolBuffer.cpp > --- a/src/hotspot/share/jfr/recorder/stringpool/jfrStringPoolBuffer.cpp Wed May 16 09:58:24 2018 +0200 > +++ b/src/hotspot/share/jfr/recorder/stringpool/jfrStringPoolBuffer.cpp Wed May 16 11:00:04 2018 +0200 > @@ -55,7 +55,17 @@ > } > > void JfrStringPoolBuffer::increment(uint64_t value) { > +#if !(defined(ARM) || defined(IA32)) > Atomic::add(value, &_string_count_pos); > +#else > + // TODO: This should be fixed in Atomic::add handling for 32-bit platforms, > + // see JDK-8203283. We workaround the absence of support right here. I'm somewhat annoyed about this. I thought after the templatization of Atomic everything was still setup to correctly handle the fallback to using CAS for 64-bit Atomic ops on 32-bit, for x86 at least. David > + uint64_t cur, val; > + do { > + cur = Atomic::load(&_string_count_top); > + val = cur + value; > + } while (Atomic::cmpxchg(val, &_string_count_pos, cur) != cur); > +#endif > } > > void JfrStringPoolBuffer::set_string_top(uint64_t value) { > diff -r 2e886b7b3d72 -r a7c343a2f1c4 src/hotspot/share/jfr/utilities/jfrAllocation.cpp > --- a/src/hotspot/share/jfr/utilities/jfrAllocation.cpp Wed May 16 09:58:24 2018 +0200 > +++ b/src/hotspot/share/jfr/utilities/jfrAllocation.cpp Wed May 16 11:00:04 2018 +0200 > @@ -66,8 +66,8 @@ > const size_t total_deallocated = atomic_add_jlong(dealloc_size, &_deallocated_bytes); > const size_t current_live_set = atomic_add_jlong(dealloc_size * -1, &_live_set_bytes); > log_trace(jfr, system)("Deallocation: [" SIZE_FORMAT "] bytes", dealloc_size); > - log_trace(jfr, system)("Total dealloc [" JLONG_FORMAT "] bytes", total_deallocated); > - log_trace(jfr, system)("Liveset: [" JLONG_FORMAT "] bytes", current_live_set); > + log_trace(jfr, system)("Total dealloc [" SIZE_FORMAT "] bytes", total_deallocated); > + log_trace(jfr, system)("Liveset: [" SIZE_FORMAT "] bytes", current_live_set); > } > } > > > Thanks, > -Aleksey > > From david.holmes at oracle.com Wed May 16 12:16:04 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 16 May 2018 22:16:04 +1000 Subject: RFR: 8203227: Introduce os::processor_id() for Linux and Solaris In-Reply-To: <3eac8bb2-fc7a-42b2-154b-e7f6cd63b20b@oracle.com> References: <327220bf-ea05-1921-59e1-2bd0c2c4dea8@oracle.com> <13cbc7ca-58eb-ae0b-9640-42be8e5fb3fd@oracle.com> <3eac8bb2-fc7a-42b2-154b-e7f6cd63b20b@oracle.com> Message-ID: <8905be7b-eadc-be5f-c48c-611a17a62edf@oracle.com> Hi Per, On 16/05/2018 8:55 PM, Per Liden wrote: > Hi David, > > Thanks for looking at this. > > On 05/16/2018 04:24 AM, David Holmes wrote: >> Hi Per, >> >> On 16/05/2018 1:07 AM, Per Liden wrote: >>> This patch introduces os::processor_id() to get the id of the CPU >>> that the caller is currently executing on. Obviously, the returned id >>> should only be viewed as a strong hint, since the information might >>> be instantly out-of-date. >> >> Or even wrong to begin with: >> >> https://stackoverflow.com/questions/36934736/is-sched-getcpu-reliable-on-linux >> >> >> caveat-emptor! :) >> >>> We're using this in ZGC to do various CPU-local operations. We currently >> >> So there's a requirement that the CPU value returned by processor_id() >> is compatible with some other API's that take a processor id? Does >> that need documenting in some form? > > Good point, this should be made more clear. We currently assume that > os::processor_id() returns a value from 0 to (os::processor_count() - > 1). Adding a comment in os.hpp clarifying that. Thanks. Not every value in that range may be a valid processor id, but every valid processor id should be within that range. >>> only need this for Linux and Solaris, which is what this patch adds. >> >> Are the API's that take the processor id defined in os as well? >> Otherwise it seems this functionality should just be localized to the >> specific os_.cpp file from which it is used (and/or the os:: >> class). > > The os::processor_id() call-sites we have are in shared os-agnostic > code. This function is also fairly closely related to > os::processor_count(), so it seems like os:: would be the natural place > for this. I'm curious as to what functions in os-agnostic code consume the processor id value? Or are you storing/caching it in the shared code and only using it later from OS specific code? >> >>> Support for additional platforms can be added when needed. I didn't >>> add dummy os::processor_id() to the not-yet-implemented platforms, >>> since I think it's better to get a early compile-time error instead >>> of a Unimplemented() at runtime, if you try to use them. >> >> I'd rather not see the runtime support of our primary platforms of >> interest fragmented like this. Are there technical challenges to doing >> this? > > I'm a bit reluctant to add this for other platforms at this point, since > some platforms might require non-trivial implementations, which would go > unused/untested. For macOS I believe we would need something like this: > > http://cr.openjdk.java.net/~gziemski/zgc_os_processor_id/webrev/src/hotspot/os/bsd/os_bsd.cpp.udiff.html Yeah direct use of cpuid is not pleasant. > (I should add that I don't know if this patch does the right thing, but > Gerard apparently had some success with it) > > For Windows, I believe a call to GetCurrentProcessorNumber() should be > enough (we don't seem to handle/support logical processor groups in > os::processor_count() so should be safe to do the same thing here). The current processor should always be correct regardless of what logical/virtual processor configuration exists. > The plan has been to add support for these if/when ZGC supports these > platforms (unless someone else needs and adds them before that). But > adding them now makes me a bit uneasy, especially given the non-trivial > nature of the macOS implementation. Okay. I'd prefer to see all platforms covered, but not with the risk of untested code being used. >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203227 >>> Webrev: http://cr.openjdk.java.net/~pliden/8203227/webrev.0 >> >> For Solaris there is no need to introduce Solaris::getcpuid() just >> because Linux already has that. (Unless this ends up not being needed >> in the os class of course :) ). > > Check. > >> >> Are you allowing for the -1 return value on Linux or assuming it will >> never happen in practice? If the former then that should be documented >> as part of the semantics of os::processor_id(). If the latter then you >> should probably add an assert or guarantee. I suspect this is a >> historical restriction no longer relevant to us - in which case we can >> probably clean out the dynamic lookup/linking!). > > We're assuming it will never happen in practice, and this is currently > guarded with a guarantee() during initialization of ZGC. sched_getcpu() > has "only" been around since glibc 2.14 (June 2011). However, we can > also receive -1 if the kernel doesn't have the getcpu() syscall (first > introduced in Linux 2.6.19, Nov 2006). > > So, it seems safe to at least assume that the getcpu() syscall is > available, and only have an single check for that during startup (with a > vm_exit_during_initialization() if it's unavailable). Okay. We should clean up the syscall and dynamic lookup code use here. > Here's an updated webrev for continued discussion: > > http://cr.openjdk.java.net/~pliden/8203227/webrev.1 Looks fine to me. Thanks, David > cheers, > Per > >> >> Thanks, >> David >> >>> /Per From stefan.johansson at oracle.com Wed May 16 12:42:56 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Wed, 16 May 2018 14:42:56 +0200 Subject: RFR: 8202813: Move vm_weak processing from SystemDictionary::do_unloading to WeakProcessor In-Reply-To: References: Message-ID: <10a99bf5-d3cc-9d5b-8a38-5ba1672f9f8b@oracle.com> Hi Kim, On 2018-05-08 21:34, Kim Barrett wrote: > Please review this change to move the clearing of dead entries in > vm_weak_oop_storage from SystemDictionary::do_unloading to preceeding > calls to WeakProcessor::weak_oops_do. > > This involves conditionalizing WeakProcessor invocations, to allow > different calls to process different subsets of the native weak > references. For example, vm_weak_oop_storage is treated as strong > rather than weak when ClassUnloading is disabled. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8202813 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8202813/open.00/ > Thanks for doing this work to unify weak root processing. I might be missing the bigger picture, so before starting to commenting the code I have a couple of questions: 1. With this patch SystemDictionary::vm_weak_oop_storage is now processed both by the WeakProcessor in weak_oops_do and the SystemDictionary in oops_do() and roots_oops_do(). Shouldn't all processing be moved to the WeakProcessor to make it clear who is responsible for doing the processing? Or why do we want to treat do_unloading special? 2. For this patch I don't see the need for the PhaseSet class and the serial and parallel phases distinction. Couldn't we just pass in a bool to weak_oops_do saying if we should handle the vm_weak_oop_storage or not? I understand that for upcoming patches this might be needed, but then I think this should be added in that change. Thanks, Stefan > Testing: > Mach5 jdk-tier{1-2}, hs-tier{1-5} > Locally runthese30m. > From robbin.ehn at oracle.com Wed May 16 12:53:13 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Wed, 16 May 2018 14:53:13 +0200 Subject: RFR(L): 8195098: Low latency hashtable for read-mostly scenarios In-Reply-To: References: <1cc85bdc-98cb-115d-a904-4cf1c94259b2@oracle.com> <8defc9dd-e908-1bd5-8fdb-929905773ab0@oracle.com> <20d4cc82-37b1-f449-b53f-9fb68059942f@oracle.com> Message-ID: <0148edb4-c0e7-653c-1e65-7e5b8657470b@oracle.com> Thanks Gerard, Robbin On 05/14/2018 06:58 PM, Gerard Ziemski wrote: > Thank you Robbin for addressing all my concerns and answering all my questions. I will probably have more later, but we can handle that then. > > Very impressive hashtable. > > > cheers > > >> On May 11, 2018, at 2:54 PM, Robbin Ehn wrote: >> >> Hi Gerard, >> >> On 2018-05-10 19:55, Gerard Ziemski wrote: >>> hi Robbin, >>> There are 2 minor issues that caught my eye, anything else we can fix in followup issues: >>> #1 >>> // This method looks on the _invisible_epoch field and >>> // does a write_synchronize if needed. >>> void write_synchonize_on_visible_epoch(Thread* thread); >>> a) I think in the word ?looks? in the comment should be ?locks? >> >> I don't like lock, sine it implies mutual exclusiveness. I added an explanation, hopefully you like it. >> >>> b) Why is the method name using word ?visible? when it locks on an ?invisible? epoch? >> >> The method only does write_synchronize if the the 'epoch'/generation was seen by another thread. Hence write_synchonize_on_visible_epoch, e.g. it only does >> wite_synchronize if hashtable state was publish to another thread. Hopefully my >> explanation added above makes it much clearer. >> >>> #2 ScopedCS and MultiGetHandle look identical? >> >> ScopedCS is an internal class only and MultiGetHandle an interface class. >> But since they provide same type scoped with identical code I let >> MultiGetHandle inherit from ScopedCS instead and removed the code duplication, thanks! >> >> Please see incremental: >> http://cr.openjdk.java.net/~rehn/8195098/v3/inc/ >> >> For reference full: >> http://cr.openjdk.java.net/~rehn/8195098/v3/full/webrev/ >> >> Thanks, Robbin >> >>> // ScopedCS >>> template >>> inline ConcurrentHashTable:: >>> ScopedCS::ScopedCS(Thread* thread, ConcurrentHashTable* cht) >>> : _thread(thread), _cht(cht) >>> { >>> GlobalCounter::critical_section_begin(_thread); >>> // This version is published now. >>> if (OrderAccess::load_acquire(&_cht->_invisible_epoch) != NULL) { >>> OrderAccess::release_store_fence(&_cht->_invisible_epoch, (Thread*)NULL); >>> } >>> } >>> template >>> inline ConcurrentHashTable:: >>> ScopedCS::~ScopedCS() >>> { >>> GlobalCounter::critical_section_end(_thread); >>> } >>> // MultiGetHandle >>> template >>> inline ConcurrentHashTable:: >>> MultiGetHandle::MultiGetHandle(Thread* thread, >>> ConcurrentHashTable* cht) >>> : _thread(thread), _cht(cht) >>> { >>> GlobalCounter::critical_section_begin(_thread); >>> // This version is published now. >>> if (OrderAccess::load_acquire(&_cht->_invisible_epoch) != NULL) { >>> OrderAccess::release_store_fence(&_cht->_invisible_epoch, (Thread*)NULL); >>> } >>> } >>> template >>> inline ConcurrentHashTable:: >>> MultiGetHandle::~MultiGetHandle() >>> { >>> GlobalCounter::critical_section_end(_thread); >>> } >>> cheers >>>> On May 2, 2018, at 4:03 AM, Robbin Ehn wrote: >>>> >>>> Hi all, >>>> >>>> Here is an update with Gerard's and Coleen's input. >>>> >>>> Inc: >>>> http://cr.openjdk.java.net/~rehn/8195098/v1/inc/webrev/ >>>> Full: >>>> http://cr.openjdk.java.net/~rehn/8195098/v1/full/webrev/ >>>> >>>> Thanks, Robbin > From per.liden at oracle.com Wed May 16 12:54:26 2018 From: per.liden at oracle.com (Per Liden) Date: Wed, 16 May 2018 14:54:26 +0200 Subject: RFR: 8203227: Introduce os::processor_id() for Linux and Solaris In-Reply-To: <8905be7b-eadc-be5f-c48c-611a17a62edf@oracle.com> References: <327220bf-ea05-1921-59e1-2bd0c2c4dea8@oracle.com> <13cbc7ca-58eb-ae0b-9640-42be8e5fb3fd@oracle.com> <3eac8bb2-fc7a-42b2-154b-e7f6cd63b20b@oracle.com> <8905be7b-eadc-be5f-c48c-611a17a62edf@oracle.com> Message-ID: <1c382d62-1d65-4bba-5027-773a374967d5@oracle.com> Hi David, On 05/16/2018 02:16 PM, David Holmes wrote: [...] >>>> We're using this in ZGC to do various CPU-local operations. We >>>> currently >>> >>> So there's a requirement that the CPU value returned by >>> processor_id() is compatible with some other API's that take a >>> processor id? Does that need documenting in some form? >> >> Good point, this should be made more clear. We currently assume that >> os::processor_id() returns a value from 0 to (os::processor_count() - >> 1). Adding a comment in os.hpp clarifying that. > > Thanks. Not every value in that range may be a valid processor id, but > every valid processor id should be within that range. Yep. > >>>> only need this for Linux and Solaris, which is what this patch adds. >>> >>> Are the API's that take the processor id defined in os as well? >>> Otherwise it seems this functionality should just be localized to the >>> specific os_.cpp file from which it is used (and/or the os:: >>> class). >> >> The os::processor_id() call-sites we have are in shared os-agnostic >> code. This function is also fairly closely related to >> os::processor_count(), so it seems like os:: would be the natural >> place for this. > > I'm curious as to what functions in os-agnostic code consume the > processor id value? Or are you storing/caching it in the shared code and > only using it later from OS specific code? The typical way we use this in ZGC is to have CPU-local data, conceptually equivalent to this toy example: // Allocate per-CPU data MyData* my_per_cpu_data = alloc(sizeof(MyData) * os::processor_count()); // Read CPU-local data ... = my_per_cpu_data[os::processor_id()]; Have a look at the ZPerCPU template class if you're interested in more details: http://hg.openjdk.java.net/zgc/zgc/file/118cb3e3517d/src/hotspot/share/gc/z/zValue.hpp This template basically allows us to just say: // Allocate per-CPU data ZPerCPU my_per_cpu_data; // Read CPU-local data ... = my_per_cpu_data.get(); It should be noted that ZGC is only ever built (controlled by sh configure) for supported platforms (Linux & Solaris at this moment), which is why it's fine to have os::processor_id() calls in this shared code. > >>> >>>> Support for additional platforms can be added when needed. I didn't >>>> add dummy os::processor_id() to the not-yet-implemented platforms, >>>> since I think it's better to get a early compile-time error instead >>>> of a Unimplemented() at runtime, if you try to use them. >>> >>> I'd rather not see the runtime support of our primary platforms of >>> interest fragmented like this. Are there technical challenges to >>> doing this? >> >> I'm a bit reluctant to add this for other platforms at this point, >> since some platforms might require non-trivial implementations, which >> would go unused/untested. For macOS I believe we would need something >> like this: >> >> http://cr.openjdk.java.net/~gziemski/zgc_os_processor_id/webrev/src/hotspot/os/bsd/os_bsd.cpp.udiff.html > > > Yeah direct use of cpuid is not pleasant. > >> (I should add that I don't know if this patch does the right thing, >> but Gerard apparently had some success with it) >> >> For Windows, I believe a call to GetCurrentProcessorNumber() should be >> enough (we don't seem to handle/support logical processor groups in >> os::processor_count() so should be safe to do the same thing here). > > The current processor should always be correct regardless of what > logical/virtual processor configuration exists. Check. > >> The plan has been to add support for these if/when ZGC supports these >> platforms (unless someone else needs and adds them before that). But >> adding them now makes me a bit uneasy, especially given the >> non-trivial nature of the macOS implementation. > > Okay. I'd prefer to see all platforms covered, but not with the risk of > untested code being used. > >>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203227 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8203227/webrev.0 >>> >>> For Solaris there is no need to introduce Solaris::getcpuid() just >>> because Linux already has that. (Unless this ends up not being needed >>> in the os class of course :) ). >> >> Check. >> >>> >>> Are you allowing for the -1 return value on Linux or assuming it will >>> never happen in practice? If the former then that should be >>> documented as part of the semantics of os::processor_id(). If the >>> latter then you should probably add an assert or guarantee. I suspect >>> this is a historical restriction no longer relevant to us - in which >>> case we can probably clean out the dynamic lookup/linking!). >> >> We're assuming it will never happen in practice, and this is currently >> guarded with a guarantee() during initialization of ZGC. >> sched_getcpu() has "only" been around since glibc 2.14 (June 2011). >> However, we can also receive -1 if the kernel doesn't have the >> getcpu() syscall (first introduced in Linux 2.6.19, Nov 2006). >> >> So, it seems safe to at least assume that the getcpu() syscall is >> available, and only have an single check for that during startup (with >> a vm_exit_during_initialization() if it's unavailable). > > Okay. We should clean up the syscall and dynamic lookup code use here. > >> Here's an updated webrev for continued discussion: >> >> http://cr.openjdk.java.net/~pliden/8203227/webrev.1 > > Looks fine to me. Ok, thanks David! /Per > > Thanks, > David > >> cheers, >> Per >> >>> >>> Thanks, >>> David >>> >>>> /Per From matthias.baesken at sap.com Wed May 16 13:12:56 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 16 May 2018 13:12:56 +0000 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: Message-ID: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> Ping : could I get a second review ? Bug : https://bugs.openjdk.java.net/browse/JDK-8202427 Change : http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > We could also just print the MB value , let's see what other think. > Another option might be to have a flexible output (kB for smaller memory > values , MB (or GB) for larger ) ? Martin suggested to just print the MB values, what do you think ? Thanks, Matthias > -----Original Message----- > From: Baesken, Matthias > Sent: Mittwoch, 2. Mai 2018 13:00 > To: Doerr, Martin ; 'hotspot- > dev at openjdk.java.net' ; hotspot- > runtime-dev at openjdk.java.net > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > > Hi Martin, thanks for your input . > I add hotspot-runtime-dev . > > > > > I wonder if we really need the sizes in kB in addition to MB. Maybe other > > reviewers would like to comment on this, too. > > > > We could also just print the MB value , let's see what other think. > Another option might be to have a flexible output (kB for smaller memory > values , MB (or GB) for larger ) ? > > Best regards, Matthias > > > > -----Original Message----- > > From: Doerr, Martin > > Sent: Mittwoch, 2. Mai 2018 12:53 > > To: Baesken, Matthias ; 'hotspot- > > dev at openjdk.java.net' > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > > > > Hi Matthias, > > > > looks like a nice enhancement. We can get substantially more information. > > > > I wonder if we really need the sizes in kB in addition to MB. Maybe other > > reviewers would like to comment on this, too. > > > > We should have line breaks. > > > > Best regards, > > Martin > > > > > > -----Original Message----- > > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > > Behalf Of Baesken, Matthias > > Sent: Montag, 30. April 2018 16:53 > > To: 'hotspot-dev at openjdk.java.net' > > Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows > > > > On Windows, > > the os::print_memory_info misses a few memory-related infos that might > > be interesting : > > - current and peak WorkingSet size (= physical memory assigned to the > > process) > > - current and peak commit charge (also known as "private bytes" in > Windows > > tools) > > - on 32bit Windows : > > user-mode portion/free user mode portion of virtual address-space > > (Total/AvailVirtual) because it shows how close do we get to the 2-4 GB per > > process border. > > > > > > - the current naming of "swap/free-swap" memory is a bit misleading; > > in the Windows world swap is a special page file used for UWP apps. > > (see Windows Internals : "The swap file" (chapter 5 Memory > management) > > Windows 8 added another page file called a swap file. It is ... exclusively > used > > for UWP (Universal Windows Platform) apps. > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the UWP > > related values, it is about page file sizes > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just virtual > > memory might be more appropriate). > > > > > > https://msdn.microsoft.com/de- > > de/library/windows/desktop/aa366770(v=vs.85).aspx > > > > documents it in the following way: > > ullTotalPageFile: > > The current committed memory limit for the system or the current process, > > whichever is smaller,... > > > > ullAvailPageFile : > > The maximum amount of memory the current process can commit, in > bytes. > > This value is equal to or smaller than the system-wide available commit > value > > > > > > > > Aditionally I suggest having output of the various memory-values in M > > (megabyte) as well , the k (kilobyte) output sometimes gives very high and > > unreadable numbers). > > > > > > Could you please review my change ? > > > > > > Bug : > > > > https://bugs.openjdk.java.net/browse/JDK-8202427 > > > > > > Change : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > > > > > > Thanks, Matthias > > From glaubitz at physik.fu-berlin.de Wed May 16 13:32:04 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Wed, 16 May 2018 15:32:04 +0200 Subject: Hotspot broken on linux-sparc, most likely after JDK-8199712 (Flight Recorder) Message-ID: <68849898-3aae-c3c9-aa35-952be7ecdccf@physik.fu-berlin.de> Hello! JDK-8199712 (Flight Recorder) most likely broke the build on linux-sparc and it seems we're just missing a source file, either through a header or a Makefile (see below). I assume we need to build hotspot/cpu/sparc/vm_version_ext_sparc.cpp to get this issue fixed. I will look into this later unless someone is faster than me. Adrian === Output from failing command(s) repeated here === /usr/bin/printf "* For target hotspot_variant-server_libjvm_objs_os_perf_linux.o:\n" * For target hotspot_variant-server_libjvm_objs_os_perf_linux.o: (/bin/grep -v -e "^Note: including file:" < /srv/glaubitz/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs/hotspot_variant-server_libjvm_objs_os_perf_linux.o.log || true) | /usr/bin/head -n 12 /srv/glaubitz/jdk/src/hotspot/os/linux/os_perf_linux.cpp: In member function ?bool CPUInformationInterface::initialize()?: /srv/glaubitz/jdk/src/hotspot/os/linux/os_perf_linux.cpp:1028:45: error: ?VM_Version_Ext? has not been declared _cpu_info->set_number_of_hardware_threads(VM_Version_Ext::number_of_threads()); ^~~~~~~~~~~~~~ /srv/glaubitz/jdk/src/hotspot/os/linux/os_perf_linux.cpp:1029:34: error: ?VM_Version_Ext? has not been declared _cpu_info->set_number_of_cores(VM_Version_Ext::number_of_cores()); ^~~~~~~~~~~~~~ /srv/glaubitz/jdk/src/hotspot/os/linux/os_perf_linux.cpp:1030:36: error: ?VM_Version_Ext? has not been declared _cpu_info->set_number_of_sockets(VM_Version_Ext::number_of_sockets()); ^~~~~~~~~~~~~~ /srv/glaubitz/jdk/src/hotspot/os/linux/os_perf_linux.cpp:1031:27: error: ?VM_Version_Ext? has not been declared _cpu_info->set_cpu_name(VM_Version_Ext::cpu_name()); if test `/usr/bin/wc -l < /srv/glaubitz/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs/hotspot_variant-server_libjvm_objs_os_perf_linux.o.log` -gt 12; then /bin/echo " ... (rest of output omitted)" ; fi ... (rest of output omitted) /usr/bin/printf "\n* All command lines available in /srv/glaubitz/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs.\n" * All command lines available in /srv/glaubitz/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs. /usr/bin/printf "=== End of repeated output ===\n" === End of repeated output === -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From shade at redhat.com Wed May 16 13:39:28 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 16 May 2018 15:39:28 +0200 Subject: Hotspot broken on linux-sparc, most likely after JDK-8199712 (Flight Recorder) In-Reply-To: <68849898-3aae-c3c9-aa35-952be7ecdccf@physik.fu-berlin.de> References: <68849898-3aae-c3c9-aa35-952be7ecdccf@physik.fu-berlin.de> Message-ID: On 05/16/2018 03:32 PM, John Paul Adrian Glaubitz wrote: > Hello! > > JDK-8199712 (Flight Recorder) most likely broke the build on linux-sparc > and it seems we're just missing a source file, either through a header > or a Makefile (see below). > > I assume we need to build hotspot/cpu/sparc/vm_version_ext_sparc.cpp to > get this issue fixed. It is there already: ./src/hotspot/cpu/sparc/vm_version_ext_sparc.cpp ...but you probably need: diff -r 781f36c0831e src/hotspot/os/linux/os_perf_linux.cpp --- a/src/hotspot/os/linux/os_perf_linux.cpp Wed May 16 13:14:58 2018 +0200 +++ b/src/hotspot/os/linux/os_perf_linux.cpp Wed May 16 15:39:14 2018 +0200 @@ -40,6 +40,9 @@ #include "vm_version_ext_aarch64.hpp" #endif #endif +#ifdef SPARC +#include "vm_version_ext_sparc.hpp" +#endif #include #include -Aleksey From goetz.lindenmaier at sap.com Wed May 16 13:48:48 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 16 May 2018 13:48:48 +0000 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> References: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> Message-ID: Hi Matthias, I appreciate the additional information in the hs_err file. Could you please add an example output (of the final version) to the bug description? As Martin, I would like more line feeds, both in the code and the output. Currently, the output in the hs_err file looks like this (where the first line is too long): Memory: 4k page, system-wide physical 16776692k [16383M] (8795032k [8588M] free), TotalPageFile size 49949460k [48778M] (AvailPageFile size 38942152k [38029M]), user-mode portion of virtual address-space 2097024k [2047M] (6940k [6M] free) current process WorkingSet (physical memory assigned to process): 991804k [968M], peak: 991804k [968M] current process commit charge ("private bytes"): 1523632k [1487M], peak: 1523632k [1487M] I also think that one number is enough. Adaptive numbers would be great. I know about src/hotspot/share/memory/metaspace/metaspaceCommon.hpp: print_human_readable_size() for this, but this seems a bit hidden in metaspace. I don't like usage of _M_IX86. Common in this file seems #ifndef _WIN64 for 32-bit windows dependent code. But don't care here. Best regards, Goetz. > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > Behalf Of Baesken, Matthias > Sent: Mittwoch, 16. Mai 2018 15:13 > To: Doerr, Martin ; 'hotspot- > dev at openjdk.java.net' ; hotspot- > runtime-dev at openjdk.java.net > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > > Ping : could I get a second review ? > > Bug : > https://bugs.openjdk.java.net/browse/JDK-8202427 > Change : > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > > > > We could also just print the MB value , let's see what other think. > > Another option might be to have a flexible output (kB for smaller > memory > > values , MB (or GB) for larger ) ? > > Martin suggested to just print the MB values, what do you think ? > > > Thanks, Matthias > > > > -----Original Message----- > > From: Baesken, Matthias > > Sent: Mittwoch, 2. Mai 2018 13:00 > > To: Doerr, Martin ; 'hotspot- > > dev at openjdk.java.net' ; hotspot- > > runtime-dev at openjdk.java.net > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > > > > Hi Martin, thanks for your input . > > I add hotspot-runtime-dev . > > > > > > > > I wonder if we really need the sizes in kB in addition to MB. Maybe other > > > reviewers would like to comment on this, too. > > > > > > > We could also just print the MB value , let's see what other think. > > Another option might be to have a flexible output (kB for smaller > memory > > values , MB (or GB) for larger ) ? > > > > Best regards, Matthias > > > > > > > -----Original Message----- > > > From: Doerr, Martin > > > Sent: Mittwoch, 2. Mai 2018 12:53 > > > To: Baesken, Matthias ; 'hotspot- > > > dev at openjdk.java.net' > > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > > > > > > Hi Matthias, > > > > > > looks like a nice enhancement. We can get substantially more > information. > > > > > > I wonder if we really need the sizes in kB in addition to MB. Maybe other > > > reviewers would like to comment on this, too. > > > > > > We should have line breaks. > > > > > > Best regards, > > > Martin > > > > > > > > > -----Original Message----- > > > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > > > Behalf Of Baesken, Matthias > > > Sent: Montag, 30. April 2018 16:53 > > > To: 'hotspot-dev at openjdk.java.net' > > > Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows > > > > > > On Windows, > > > the os::print_memory_info misses a few memory-related infos that > might > > > be interesting : > > > - current and peak WorkingSet size (= physical memory assigned to the > > > process) > > > - current and peak commit charge (also known as "private bytes" in > > Windows > > > tools) > > > - on 32bit Windows : > > > user-mode portion/free user mode portion of virtual address-space > > > (Total/AvailVirtual) because it shows how close do we get to the 2-4 GB > per > > > process border. > > > > > > > > > - the current naming of "swap/free-swap" memory is a bit misleading; > > > in the Windows world swap is a special page file used for UWP apps. > > > (see Windows Internals : "The swap file" (chapter 5 Memory > > management) > > > Windows 8 added another page file called a swap file. It is ... exclusively > > used > > > for UWP (Universal Windows Platform) apps. > > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the UWP > > > related values, it is about page file sizes > > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just > virtual > > > memory might be more appropriate). > > > > > > > > > https://msdn.microsoft.com/de- > > > de/library/windows/desktop/aa366770(v=vs.85).aspx > > > > > > documents it in the following way: > > > ullTotalPageFile: > > > The current committed memory limit for the system or the current > process, > > > whichever is smaller,... > > > > > > ullAvailPageFile : > > > The maximum amount of memory the current process can commit, in > > bytes. > > > This value is equal to or smaller than the system-wide available commit > > value > > > > > > > > > > > > Aditionally I suggest having output of the various memory-values in M > > > (megabyte) as well , the k (kilobyte) output sometimes gives very high > and > > > unreadable numbers). > > > > > > > > > Could you please review my change ? > > > > > > > > > Bug : > > > > > > https://bugs.openjdk.java.net/browse/JDK-8202427 > > > > > > > > > Change : > > > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > > > > > > > > > Thanks, Matthias > > > From glaubitz at physik.fu-berlin.de Wed May 16 13:49:52 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Wed, 16 May 2018 15:49:52 +0200 Subject: Hotspot broken on linux-sparc, most likely after JDK-8199712 (Flight Recorder) In-Reply-To: References: <68849898-3aae-c3c9-aa35-952be7ecdccf@physik.fu-berlin.de> Message-ID: On 05/16/2018 03:39 PM, Aleksey Shipilev wrote: > It is there already: > ./src/hotspot/cpu/sparc/vm_version_ext_sparc.cpp > > ...but you probably need: Yes, I saw that. Sorry if that wasn't clear. I just meant that while it's there, it's not being built for linux-sparc. > diff -r 781f36c0831e src/hotspot/os/linux/os_perf_linux.cpp > --- a/src/hotspot/os/linux/os_perf_linux.cpp Wed May 16 13:14:58 2018 +0200 > +++ b/src/hotspot/os/linux/os_perf_linux.cpp Wed May 16 15:39:14 2018 +0200 > @@ -40,6 +40,9 @@ > #include "vm_version_ext_aarch64.hpp" > #endif > #endif > +#ifdef SPARC > +#include "vm_version_ext_sparc.hpp" > +#endif > > #include > #include Yes, this seems to be the case. However, it seems that vm_version_ext_sparc.hpp seems to be written with Solaris-only in mind :( === Output from failing command(s) repeated here === /usr/bin/printf "* For target hotspot_variant-server_libjvm_objs_os_perf_linux.o:\n" * For target hotspot_variant-server_libjvm_objs_os_perf_linux.o: (/bin/grep -v -e "^Note: including file:" < /srv/glaubitz/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs/hotspot_variant-server_libjvm_objs_os_perf_linux.o.log || true) | /usr/bin/head -n 12 In file included from /srv/glaubitz/jdk/src/hotspot/os/linux/os_perf_linux.cpp:43:0: /srv/glaubitz/jdk/src/hotspot/cpu/sparc/vm_version_ext_sparc.hpp:30:10: fatal error: kstat.h: No such file or directory #include ^~~~~~~~~ compilation terminated. if test `/usr/bin/wc -l < /srv/glaubitz/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs/hotspot_variant-server_libjvm_objs_os_perf_linux.o.log` -gt 12; then /bin/echo " ... (rest of output omitted)" ; fi /usr/bin/printf "\n* All command lines available in /srv/glaubitz/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs.\n" * All command lines available in /srv/glaubitz/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs. /usr/bin/printf "=== End of repeated output ===\n" === End of repeated output === I assume this will need some more work as just commenting out this include results in: === Output from failing command(s) repeated here === /usr/bin/printf "* For target hotspot_variant-server_libjvm_objs_os_perf_linux.o:\n" * For target hotspot_variant-server_libjvm_objs_os_perf_linux.o: (/bin/grep -v -e "^Note: including file:" < /srv/glaubitz/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs/hotspot_variant-server_libjvm_objs_os_perf_linux.o.log || true) | /usr/bin/head -n 12 In file included from /srv/glaubitz/jdk/src/hotspot/os/linux/os_perf_linux.cpp:43:0: /srv/glaubitz/jdk/src/hotspot/cpu/sparc/vm_version_ext_sparc.hpp:50:10: error: ?kid_t? does not name a type; did you mean ?id_t?? static kid_t _kcid; ^~~~~ id_t if test `/usr/bin/wc -l < /srv/glaubitz/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs/hotspot_variant-server_libjvm_objs_os_perf_linux.o.log` -gt 12; then /bin/echo " ... (rest of output omitted)" ; fi /usr/bin/printf "\n* All command lines available in /srv/glaubitz/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs.\n" * All command lines available in /srv/glaubitz/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs. /usr/bin/printf "=== End of repeated output ===\n" === End of repeated output === Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From glaubitz at physik.fu-berlin.de Wed May 16 14:00:49 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Wed, 16 May 2018 16:00:49 +0200 Subject: Hotspot broken on linux-sparc, most likely after JDK-8199712 (Flight Recorder) In-Reply-To: References: <68849898-3aae-c3c9-aa35-952be7ecdccf@physik.fu-berlin.de> Message-ID: <83ac5657-66a7-212a-d62f-fa823faab165@physik.fu-berlin.de> On 05/16/2018 03:49 PM, John Paul Adrian Glaubitz wrote: > Yes, this seems to be the case. However, it seems that vm_version_ext_sparc.hpp > seems to be written with Solaris-only in mind :( > > === Output from failing command(s) repeated here === > === End of repeated output === > > I assume this will need some more work as just commenting out this include > results in: > > === Output from failing command(s) repeated here === > === End of repeated output === I have opened a bug report for this [1]. Adrian > [1] https://bugs.openjdk.java.net/browse/JDK-8203301 -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From shade at redhat.com Wed May 16 14:07:13 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 16 May 2018 16:07:13 +0200 Subject: Hotspot broken on linux-sparc, most likely after JDK-8199712 (Flight Recorder) In-Reply-To: <83ac5657-66a7-212a-d62f-fa823faab165@physik.fu-berlin.de> References: <68849898-3aae-c3c9-aa35-952be7ecdccf@physik.fu-berlin.de> <83ac5657-66a7-212a-d62f-fa823faab165@physik.fu-berlin.de> Message-ID: <54153bef-6d37-e204-be71-bc693090e9fb@redhat.com> On 05/16/2018 04:00 PM, John Paul Adrian Glaubitz wrote: > On 05/16/2018 03:49 PM, John Paul Adrian Glaubitz wrote: >> Yes, this seems to be the case. However, it seems that vm_version_ext_sparc.hpp >> seems to be written with Solaris-only in mind :( >> >> === Output from failing command(s) repeated here === >> === End of repeated output === >> >> I assume this will need some more work as just commenting out this include >> results in: >> >> === Output from failing command(s) repeated here === >> === End of repeated output === > > I have opened a bug report for this [1]. >> [1] https://bugs.openjdk.java.net/browse/JDK-8203301 Renamed and linked it up to the rest of build issues caused by JFR integration. -Aleksey From gromero at linux.vnet.ibm.com Wed May 16 14:08:10 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Wed, 16 May 2018 11:08:10 -0300 Subject: RFR(M) 8188165: PPC64: Optimize Unsafe.copyMemory and arraycopy In-Reply-To: References: Message-ID: <698c78e0-008c-8a4b-411d-6dce56911438@linux.vnet.ibm.com> Hi, >[Ping] > >On 9/29/17 4:00 PM, Matthew Brandyberry wrote: >> This is specific to PPC64LE only. >> >> The emphasis in the proposed code is on minimizing branches. Thus, >> this code makes no attempt to avoid misaligned accesses and each block >> is designed to copy as many elements as possible. >> >> As one data point, this yields as much as a 13x improvement in >> jbyte_disjoint_arraycopy for certain misaligned scenarios. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8188165 >> Webrev: http://cr.openjdk.java.net/~mbrandy/8188165/jdk10/v1/ For the records, Martin already reviewed that change and pointed out some aspects that need to be improved / addressed in that change. Mainly: - Avoid adding another stub - It should not be just a solution for LE (it must work for both endianness) Additionally, I tested the change using SPECjbb2015 and found a slight regression. On a POWER8, SMT=8, score of 10 runs: Mean max-jOPS Mean critical-jOPS SD max-jOPS SD critical-jOPS +--------+---------------+--------------------+-------------+------------------+ w/ opto | 24387.6 | 3905.4 | 972.960 | 323.4238 | +------------------------------------------------------------------------------+ wo/ opto | 24891.7 | 3933.1 | 227.518 | 293.6455 | +--------+---------------+--------------------+-------------+------------------+ I'll try to address it before OpenJDK 11 rampdown. Best regards, Gustavo [RUNs] -- w/ opto -- root at pub:~/SPECjbb2015-1.00# for i in 18-04-10_222315-not-reverted 18-04-11_211557-not-reverted 18-04-11_233332-not-reverted 18-04-12_014507-not-reverted 18-04-14_132617-not-reverted 18-04-14_154517-not-reverted 18-04-14_181417-not-reverted 18-04-14_203008-not-reverted 18-04-14_224917-not-reverted 18-04-15_010546-not-reverted; do fgrep "jOPS =" $i/controller.out;done RUN RESULT: hbIR (max attempted) = 27674, hbIR (settled) = 26176, max-jOPS = 22693, critical-jOPS = 3386 RUN RESULT: hbIR (max attempted) = 27674, hbIR (settled) = 26793, max-jOPS = 24907, critical-jOPS = 4070 RUN RESULT: hbIR (max attempted) = 28420, hbIR (settled) = 27532, max-jOPS = 24725, critical-jOPS = 4281 RUN RESULT: hbIR (max attempted) = 27674, hbIR (settled) = 26176, max-jOPS = 22416, critical-jOPS = 4321 RUN RESULT: hbIR (max attempted) = 27674, hbIR (settled) = 26793, max-jOPS = 24907, critical-jOPS = 3777 RUN RESULT: hbIR (max attempted) = 27674, hbIR (settled) = 26176, max-jOPS = 24907, critical-jOPS = 4117 RUN RESULT: hbIR (max attempted) = 27674, hbIR (settled) = 26793, max-jOPS = 24907, critical-jOPS = 4124 RUN RESULT: hbIR (max attempted) = 27674, hbIR (settled) = 26176, max-jOPS = 24630, critical-jOPS = 3516 RUN RESULT: hbIR (max attempted) = 27674, hbIR (settled) = 26176, max-jOPS = 24907, critical-jOPS = 3663 RUN RESULT: hbIR (max attempted) = 33169, hbIR (settled) = 27674, max-jOPS = 24877, critical-jOPS = 3799 -- wo/ opto -- root at pub:~/SPECjbb2015-1.00# for i in 18-04-11_004013-reverted 18-04-12_040118-reverted 18-04-12_061741-reverted 18-04-12_083821-reverted 18-04-15_025546-reverted 18-04-15_050857-reverted 18-04-15_072246-reverted 18-04-15_093501-reverted 18-04-15_115234-reverted 18-04-15_141256-reverted; do fgrep "jOPS =" $i/controller.out;done RUN RESULT: hbIR (max attempted) = 27674, hbIR (settled) = 26176, max-jOPS = 24630, critical-jOPS = 4258 RUN RESULT: hbIR (max attempted) = 29485, hbIR (settled) = 28420, max-jOPS = 25062, critical-jOPS = 3839 RUN RESULT: hbIR (max attempted) = 27674, hbIR (settled) = 26793, max-jOPS = 24907, critical-jOPS = 4056 RUN RESULT: hbIR (max attempted) = 33169, hbIR (settled) = 27674, max-jOPS = 24877, critical-jOPS = 3985 RUN RESULT: hbIR (max attempted) = 27674, hbIR (settled) = 26793, max-jOPS = 24907, critical-jOPS = 3897 RUN RESULT: hbIR (max attempted) = 28420, hbIR (settled) = 27532, max-jOPS = 25010, critical-jOPS = 4308 RUN RESULT: hbIR (max attempted) = 29485, hbIR (settled) = 28420, max-jOPS = 25357, critical-jOPS = 4096 RUN RESULT: hbIR (max attempted) = 27674, hbIR (settled) = 26176, max-jOPS = 24907, critical-jOPS = 3798 RUN RESULT: hbIR (max attempted) = 27674, hbIR (settled) = 26176, max-jOPS = 24630, critical-jOPS = 3829 RUN RESULT: hbIR (max attempted) = 27674, hbIR (settled) = 26176, max-jOPS = 24630, critical-jOPS = 3265 From thomas.stuefe at gmail.com Wed May 16 14:35:21 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 16 May 2018 16:35:21 +0200 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> Message-ID: On Wed, May 16, 2018 at 3:48 PM, Lindenmaier, Goetz wrote: > Hi Matthias, > > I appreciate the additional information in the hs_err file. > Could you please add an example output (of the final version) to the bug description? > > As Martin, I would like more line feeds, both in the code and the output. > Currently, the output in the hs_err file looks like this (where the first line is too long): > Memory: 4k page, system-wide physical 16776692k [16383M] (8795032k [8588M] free), TotalPageFile size 49949460k [48778M] (AvailPageFile size 38942152k [38029M]), user-mode portion of virtual address-space 2097024k [2047M] (6940k [6M] free) > current process WorkingSet (physical memory assigned to process): 991804k [968M], peak: 991804k [968M] > > current process commit charge ("private bytes"): 1523632k [1487M], peak: 1523632k [1487M] > > I also think that one number is enough. Adaptive numbers would be great. > I know about > src/hotspot/share/memory/metaspace/metaspaceCommon.hpp: print_human_readable_size() > for this, but this seems a bit hidden in metaspace. > Guys, lets not worry about human readable numbers for now. We can move print_human_readable_size() out from metaspace to something generic and use it to improve this code (and others) in a later patch. That was my original intention when adding print_human_readable_size(). The patch is good. Thanks for doing this. ..Thoams > I don't like usage of _M_IX86. Common in this file seems #ifndef _WIN64 for 32-bit windows > dependent code. But don't care here. > > Best regards, > Goetz. > > >> -----Original Message----- >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >> Behalf Of Baesken, Matthias >> Sent: Mittwoch, 16. Mai 2018 15:13 >> To: Doerr, Martin ; 'hotspot- >> dev at openjdk.java.net' ; hotspot- >> runtime-dev at openjdk.java.net >> Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows >> >> Ping : could I get a second review ? >> >> Bug : >> https://bugs.openjdk.java.net/browse/JDK-8202427 >> Change : >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ >> >> >> > We could also just print the MB value , let's see what other think. >> > Another option might be to have a flexible output (kB for smaller >> memory >> > values , MB (or GB) for larger ) ? >> >> Martin suggested to just print the MB values, what do you think ? >> >> >> Thanks, Matthias >> >> >> > -----Original Message----- >> > From: Baesken, Matthias >> > Sent: Mittwoch, 2. Mai 2018 13:00 >> > To: Doerr, Martin ; 'hotspot- >> > dev at openjdk.java.net' ; hotspot- >> > runtime-dev at openjdk.java.net >> > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows >> > >> > Hi Martin, thanks for your input . >> > I add hotspot-runtime-dev . >> > >> > > >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe other >> > > reviewers would like to comment on this, too. >> > > >> > >> > We could also just print the MB value , let's see what other think. >> > Another option might be to have a flexible output (kB for smaller >> memory >> > values , MB (or GB) for larger ) ? >> > >> > Best regards, Matthias >> > >> > >> > > -----Original Message----- >> > > From: Doerr, Martin >> > > Sent: Mittwoch, 2. Mai 2018 12:53 >> > > To: Baesken, Matthias ; 'hotspot- >> > > dev at openjdk.java.net' >> > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows >> > > >> > > Hi Matthias, >> > > >> > > looks like a nice enhancement. We can get substantially more >> information. >> > > >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe other >> > > reviewers would like to comment on this, too. >> > > >> > > We should have line breaks. >> > > >> > > Best regards, >> > > Martin >> > > >> > > >> > > -----Original Message----- >> > > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >> > > Behalf Of Baesken, Matthias >> > > Sent: Montag, 30. April 2018 16:53 >> > > To: 'hotspot-dev at openjdk.java.net' >> > > Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows >> > > >> > > On Windows, >> > > the os::print_memory_info misses a few memory-related infos that >> might >> > > be interesting : >> > > - current and peak WorkingSet size (= physical memory assigned to the >> > > process) >> > > - current and peak commit charge (also known as "private bytes" in >> > Windows >> > > tools) >> > > - on 32bit Windows : >> > > user-mode portion/free user mode portion of virtual address-space >> > > (Total/AvailVirtual) because it shows how close do we get to the 2-4 GB >> per >> > > process border. >> > > >> > > >> > > - the current naming of "swap/free-swap" memory is a bit misleading; >> > > in the Windows world swap is a special page file used for UWP apps. >> > > (see Windows Internals : "The swap file" (chapter 5 Memory >> > management) >> > > Windows 8 added another page file called a swap file. It is ... exclusively >> > used >> > > for UWP (Universal Windows Platform) apps. >> > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the UWP >> > > related values, it is about page file sizes >> > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just >> virtual >> > > memory might be more appropriate). >> > > >> > > >> > > https://msdn.microsoft.com/de- >> > > de/library/windows/desktop/aa366770(v=vs.85).aspx >> > > >> > > documents it in the following way: >> > > ullTotalPageFile: >> > > The current committed memory limit for the system or the current >> process, >> > > whichever is smaller,... >> > > >> > > ullAvailPageFile : >> > > The maximum amount of memory the current process can commit, in >> > bytes. >> > > This value is equal to or smaller than the system-wide available commit >> > value >> > > >> > > >> > > >> > > Aditionally I suggest having output of the various memory-values in M >> > > (megabyte) as well , the k (kilobyte) output sometimes gives very high >> and >> > > unreadable numbers). >> > > >> > > >> > > Could you please review my change ? >> > > >> > > >> > > Bug : >> > > >> > > https://bugs.openjdk.java.net/browse/JDK-8202427 >> > > >> > > >> > > Change : >> > > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ >> > > >> > > >> > > Thanks, Matthias >> > > > From martin.doerr at sap.com Wed May 16 14:49:10 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 16 May 2018 14:49:10 +0000 Subject: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) Message-ID: <2252a635cba543a8b61dafe28b199327@sap.com> Hi, can I please get reviews for a build fix for PPC64 and s390? It contains a fix in os_perf_linux.cpp which is needed for any linux platforms not supported by Oracle. Webrev: http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.00/ Note gtest is still broken on AIX: jdk/hotspot/variant-server/libjvm/gtest/objs/libjvm.loadmap for more information. ld: 0711-317 ERROR: Undefined symbol: .SystemProcessInterface::system_processes(SystemProcess**,int*) const ld: 0711-317 ERROR: Undefined symbol: .os::print_os_info_brief(outputStream*) ld: 0711-317 ERROR: Undefined symbol: .CPUPerformanceInterface::cpu_loads_process(double*,double*,double*) const ld: 0711-317 ERROR: Undefined symbol: .CPUPerformanceInterface::cpu_load_total_process(double*) const ld: 0711-317 ERROR: Undefined symbol: .CPUPerformanceInterface::context_switch_rate(double*) const ld: 0711-317 ERROR: Undefined symbol: .CPUPerformanceInterface::cpu_load(int,double*) const ld: 0711-317 ERROR: Undefined symbol: .CPUInformationInterface::cpu_information(CPUInformation&) ld: 0711-317 ERROR: Undefined symbol: .CPUInformationInterface::CPUInformationInterface() ld: 0711-317 ERROR: Undefined symbol: .CPUInformationInterface::initialize() ld: 0711-317 ERROR: Undefined symbol: .CPUPerformanceInterface::CPUPerformanceInterface() Best regards, Martin From sgehwolf at redhat.com Wed May 16 14:58:22 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 16 May 2018 16:58:22 +0200 Subject: RFR: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) Message-ID: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> Hi, Could I please get a review of this build fix for Zero? After the Flight Recorder integration it fails do build. Post-patch is builds again (Zero release/fastdebug config). Many thanks to Aleksey Shipilev who contributed parts of the patch. Bug: https://bugs.openjdk.java.net/browse/JDK-8203287 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.01/ Testing: Zero builds on linux x86_64, server VM on linux x86_64 + jfr tests. jdk-submit job is running. Thanks, Severin From martin.doerr at sap.com Wed May 16 15:04:03 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 16 May 2018 15:04:03 +0000 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> Message-ID: <80f45f83fddf4718ae4b4c911448fa8b@sap.com> Hi Severin, JDK-8203288 contains a change of os_perf_linux.cpp using: #include CPU_HEADER(vm_version_ext) (See http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.00/) Can you check if that works for you, please? Best regards, Martin -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Severin Gehwolf Sent: Mittwoch, 16. Mai 2018 16:58 To: hotspot-dev Subject: RFR: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) Hi, Could I please get a review of this build fix for Zero? After the Flight Recorder integration it fails do build. Post-patch is builds again (Zero release/fastdebug config). Many thanks to Aleksey Shipilev who contributed parts of the patch. Bug: https://bugs.openjdk.java.net/browse/JDK-8203287 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.01/ Testing: Zero builds on linux x86_64, server VM on linux x86_64 + jfr tests. jdk-submit job is running. Thanks, Severin From boris.ulasevich at bell-sw.com Wed May 16 15:05:47 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Wed, 16 May 2018 18:05:47 +0300 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: Message-ID: Hi David, Let us look to the change in templateTable_arm.cpp. I have several notes. 1. We have compilation error because check_klass_subtype call needs one more temporary register parameter. I think correct change is: check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); -> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); 2. R0_tmp holds mdp used for profilation several lines below, so we can't spoil it. I think correct change is: check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); -> check_klass_subtype(Rklass, Rinterf, R1_tmp, R2_tmp, noreg, subtype); 3. We can't jump to Rindex. I believe it was supposed to jump to Rmethod: jump_from_interpreted(Rindex); -> jump_from_interpreted(Rmethod); 4. Please note than Rklass and Rflags reuses same register. Before the change their life ranges had no intersection. I would propose to use R2 for Rklass (same with Rrecv) and move load_klass call after invokevirtual_helper call to avoid life range intersecton. See my patch against original templateTable_arm.cpp version in attach - with this change ARM build compiles and passes most of runtime/Nestmates tests Ok. regards, Boris On 15.05.2018 03:52, David Holmes wrote: > This review is being spread across four groups: langtools, core-libs, > hotspot and serviceability. This is the specific review thread for > hotspot - webrev: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ > > See below for full details - including annotated full webrev guiding the > review. > > The intent is to have JEP-181 targeted and integrated by the end of this > month. > > Thanks, > David > ----- > > The nestmates project (JEP-181) introduces new classfile attributes to > identify classes and interfaces in the same nest, so that the VM can > perform access control based on those attributes and so allow direct > private access between nestmates without requiring javac to generate > synthetic accessor methods. These access control changes also extend to > core reflection and the MethodHandle.Lookup contexts. > > Direct private calls between nestmates requires a more general calling > context than is permitted by invokespecial, and so the JVMS is updated > to allow, and javac updated to use, invokevirtual and invokeinterface > for private class and interface method calls respectively. These changed > semantics also extend to MethodHandle findXXX operations. > > At this time we are only concerned with static nest definitions, which > map to a top-level class/interface as the nest-host and all its nested > types as nest-members. > > Please see the JEP for further details. > > JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 > Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 > CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 > > All of the specification changes have been previously been worked out by > the Valhalla Project Expert Group, and the implementation reviewed by > the various contributors and discussed on the valhalla-dev mailing list. > > Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, > Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir > Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan > > Master webrev of all changes: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ > > Annotated master webrev index: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html > > Performance: this is expected to be performance neutral in a general > sense. Benchmarking and performance runs are about to start. > > Testing Discussion: > ------------------ > > The testing for nestmates can be broken into four main groups: > > -? New tests specifically related to nestmates and currently in the > runtime/Nestmates directory > > - New tests to complement existing tests by adding in testcases not > previously expressible. > ? -? For example java/lang/invoke/SpecialInterfaceCall.java tests use > of invokespecial for private interface methods and performing receiver > typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do > similar tests for invokeinterface. > > -? New JVM TI tests to verify the spec changes related to nest attributes. > > -? Existing tests significantly affected by the nestmates changes, > primarily: > ?? -? runtime/SelectionResolution > > ?? In most cases the nestmate changes makes certain invocations that > were illegal, legal (e.g. not requiring invokespecial to invoke private > interface methods; allowing access to private members via > reflection/Methodhandles that were previously not allowed). > > - Existing tests incidentally affected by the nestmate changes > > ? This includes tests of things utilising class > redefinition/retransformation to alter nested types but which > unintentionally alter nest relationships (which is not permitted). > > There are still a number of tests problem-listed with issues filed > against them to have them adapted to work with nestmates. Some of these > are intended to be addressed in the short-term, while some (such as the > runtime/SelectionResolution test changes) may not eventuate. > > - https://bugs.openjdk.java.net/browse/JDK-8203033 > - https://bugs.openjdk.java.net/browse/JDK-8199450 > - https://bugs.openjdk.java.net/browse/JDK-8196855 > - https://bugs.openjdk.java.net/browse/JDK-8194857 > - https://bugs.openjdk.java.net/browse/JDK-8187655 > > There is also further test work still to be completed (the JNI and JDI > invocation tests): > - https://bugs.openjdk.java.net/browse/JDK-8191117 > which will continue in parallel with the main RFR. > > Pre-integration Testing: > ?- General: > ??? - Mach5: hs/jdk tier1,2 > ??? - Mach5: hs-nightly (tiers 1 -3) > ?- Targetted > ?? - nashorn (for asm changes) > ?? - hotspot: runtime/* > ????????????? serviceability/* > ????????????? compiler/* > ????????????? vmTestbase/* > ?? - jdk: java/lang/invoke/* > ????????? java/lang/reflect/* > ????????? java/lang/instrument/* > ????????? java/lang/Class/* > ????????? java/lang/management/* > ? - langtools: tools/javac > ?????????????? tools/javap > -------------- next part -------------- diff -r 3d98842c8677 src/hotspot/cpu/arm/templateTable_arm.cpp --- a/src/hotspot/cpu/arm/templateTable_arm.cpp Tue May 15 05:33:26 2018 -0400 +++ b/src/hotspot/cpu/arm/templateTable_arm.cpp Wed May 16 17:35:40 2018 +0300 @@ -4276,25 +4276,42 @@ const Register Rinterf = R5_tmp; const Register Rindex = R4_tmp; const Register Rflags = R3_tmp; - const Register Rklass = R3_tmp; + const Register Rklass = R2_tmp; // Note! Same register with Rrecv prepare_invoke(byte_no, Rinterf, Rmethod, Rrecv, Rflags); + // First check for Object case, then private interface method, + // then regular interface method. + // Special case of invokeinterface called for virtual method of - // java.lang.Object. See cpCacheOop.cpp for details. - // This code isn't produced by javac, but could be produced by - // another compliant java compiler. - Label notMethod; - __ tbz(Rflags, ConstantPoolCacheEntry::is_forced_virtual_shift, notMethod); - + // java.lang.Object. See cpCache.cpp for details. + Label notObjectMethod; + __ tbz(Rflags, ConstantPoolCacheEntry::is_forced_virtual_shift, notObjectMethod); invokevirtual_helper(Rmethod, Rrecv, Rflags); - __ bind(notMethod); + __ bind(notObjectMethod); // Get receiver klass into Rklass - also a null check __ load_klass(Rklass, Rrecv); + // Check for private method invocation - indicated by vfinal Label no_such_interface; + Label notVFinal; + __ tbz(Rflags, ConstantPoolCacheEntry::is_vfinal_shift, notVFinal); + + Label subtype; + __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R2_tmp, noreg, subtype); + + // If we get here the typecheck failed + __ b(no_such_interface); + __ bind(subtype); + + // do the call - the index is actually the method to call + __ profile_final_call(R0_tmp); + __ jump_from_interpreted(Rmethod); + + __ bind(notVFinal); + // Receiver subtype check against REFC. __ lookup_interface_method(// inputs: rec. class, interface Rklass, Rinterf, noreg, From shade at redhat.com Wed May 16 15:06:54 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 16 May 2018 17:06:54 +0200 Subject: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) In-Reply-To: <2252a635cba543a8b61dafe28b199327@sap.com> References: <2252a635cba543a8b61dafe28b199327@sap.com> Message-ID: <48176edc-b498-3877-8bbc-ad24b064b487@redhat.com> On 05/16/2018 04:49 PM, Doerr, Martin wrote: > It contains a fix in os_perf_linux.cpp which is needed for any linux platforms not supported by Oracle. > http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.00/ Looks fine. I wonder if #include CPU_HEADER would break other platforms, but we shall see. Does it still build fine on Linux/x86_64? -Aleksey From shade at redhat.com Wed May 16 15:08:22 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 16 May 2018 17:08:22 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <80f45f83fddf4718ae4b4c911448fa8b@sap.com> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> Message-ID: <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> On 05/16/2018 05:04 PM, Doerr, Martin wrote: > JDK-8203288 contains a change of os_perf_linux.cpp using: > #include CPU_HEADER(vm_version_ext) > (See http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.00/) > > Can you check if that works for you, please? +1 The rest of the patch looks good, Severin. Thanks, -Aleksey From rene.schuenemann at gmail.com Wed May 16 15:10:53 2018 From: rene.schuenemann at gmail.com (=?UTF-8?B?UmVuw6kgU2Now7xuZW1hbm4=?=) Date: Wed, 16 May 2018 17:10:53 +0200 Subject: RFR 8203292: Print non-default flags to the hs_err file Message-ID: Hi, can I please get a review for the following change: Bug: https://bugs.openjdk.java.net/browse/JDK-8203292 Webrev: http://cr.openjdk.java.net/~simonis/webrevs/2018/8203292 With this change, all flags with a non-default value will be printed to the hs_error file. While the hs_err file already contains the full command line parameters, flags can also be changed by ergonomics or various serviceability tools. Thank you, Rene From aph at redhat.com Wed May 16 15:14:25 2018 From: aph at redhat.com (Andrew Haley) Date: Wed, 16 May 2018 16:14:25 +0100 Subject: RFR: JDK-8202016: Use obj+offset in interpreter array access In-Reply-To: References: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> <041699e9-9c5b-0a8b-1b94-45a181018b47@redhat.com> <8b16fb83-25c4-8fcf-7ef6-13d3943f3fd7@redhat.com> Message-ID: <7aeaa2ef-12ca-db86-0698-ab5b3a5521ff@redhat.com> On 05/14/2018 12:34 PM, Roman Kennke wrote: > The alternative would be to explicitely drag the base_obj through all > the BarrierSetAssembler calls. I wonder if that might be a better idea, though. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From martin.doerr at sap.com Wed May 16 15:18:39 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 16 May 2018 15:18:39 +0000 Subject: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) In-Reply-To: <48176edc-b498-3877-8bbc-ad24b064b487@redhat.com> References: <2252a635cba543a8b61dafe28b199327@sap.com> <48176edc-b498-3877-8bbc-ad24b064b487@redhat.com> Message-ID: <93c1a964c7ee41d4972ef78b6958a5b3@sap.com> Hi Aleksey, thanks for reviewing. Yes, linux x86_64 still works. I've also pushed it to the submission forest. Waiting for a reply ... Best regards, Martin -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: Mittwoch, 16. Mai 2018 17:07 To: Doerr, Martin ; hotspot-dev developers (hotspot-dev at openjdk.java.net) Cc: Thomas St?fe ; John Paul Adrian Glaubitz Subject: Re: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) On 05/16/2018 04:49 PM, Doerr, Martin wrote: > It contains a fix in os_perf_linux.cpp which is needed for any linux platforms not supported by Oracle. > http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.00/ Looks fine. I wonder if #include CPU_HEADER would break other platforms, but we shall see. Does it still build fine on Linux/x86_64? -Aleksey From rkennke at redhat.com Wed May 16 15:20:59 2018 From: rkennke at redhat.com (Roman Kennke) Date: Wed, 16 May 2018 17:20:59 +0200 Subject: RFR: JDK-8202016: Use obj+offset in interpreter array access In-Reply-To: <7aeaa2ef-12ca-db86-0698-ab5b3a5521ff@redhat.com> References: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> <041699e9-9c5b-0a8b-1b94-45a181018b47@redhat.com> <8b16fb83-25c4-8fcf-7ef6-13d3943f3fd7@redhat.com> <7aeaa2ef-12ca-db86-0698-ab5b3a5521ff@redhat.com> Message-ID: Am 16.05.2018 um 17:14 schrieb Andrew Haley: > On 05/14/2018 12:34 PM, Roman Kennke wrote: >> The alternative would be to explicitely drag the base_obj through all >> the BarrierSetAssembler calls. > > I wonder if that might be a better idea, though. > This would mean to also drag index and/or offset and/or disp through the calls, some of which are optional/only used for specific types of accesses. Seeing that all this stuff is already encapsulated in Address, we can just as well use Address. Right? Roman From gromero at linux.vnet.ibm.com Wed May 16 15:34:42 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Wed, 16 May 2018 12:34:42 -0300 Subject: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) In-Reply-To: <93c1a964c7ee41d4972ef78b6958a5b3@sap.com> References: <2252a635cba543a8b61dafe28b199327@sap.com> <48176edc-b498-3877-8bbc-ad24b064b487@redhat.com> <93c1a964c7ee41d4972ef78b6958a5b3@sap.com> Message-ID: <06185936-9af1-ae90-b69d-6e0270fc8192@linux.vnet.ibm.com> Hi Martin, Even after applying the fix I see the following warning on POWER8 LE: In file included from /home/gromero/hg/jdk11/jdk/src/hotspot/share/jfr/recorder/checkpoint/jfrCheckpointManager.cpp:36:0: /home/gromero/hg/jdk11/jdk/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp:108:4: warning: #warning "Unconfigured platform" [-Wcpp] #warning "Unconfigured platform" ^ In file included from /home/gromero/hg/jdk11/jdk/src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.cpp:38:0: /home/gromero/hg/jdk11/jdk/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp:108:4: warning: #warning "Unconfigured platform" [-Wcpp] #warning "Unconfigured platform" Would that be the root cause? diff -r 37e6f6f50da0 src/hotspot/share/jfr/utilities/jfrBigEndian.hpp --- a/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp Wed May 16 11:11:03 2018 -0400 +++ b/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp Wed May 16 11:27:53 2018 -0400 @@ -100,7 +100,7 @@ } inline bool JfrBigEndian::platform_supports_unaligned_reads(void) { -#if defined(IA32) || defined(AMD64) +#if defined(IA32) || defined(AMD64) || defined(PPC64) return true; #elif defined(SPARC) || defined(ARM) || defined(AARCH64) return false; Best regards, Gustavo On 05/16/2018 12:18 PM, Doerr, Martin wrote: > Hi Aleksey, > > thanks for reviewing. Yes, linux x86_64 still works. I've also pushed it to the submission forest. Waiting for a reply ... > > Best regards, > Martin > > > -----Original Message----- > From: Aleksey Shipilev [mailto:shade at redhat.com] > Sent: Mittwoch, 16. Mai 2018 17:07 > To: Doerr, Martin ; hotspot-dev developers (hotspot-dev at openjdk.java.net) > Cc: Thomas St?fe ; John Paul Adrian Glaubitz > Subject: Re: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) > > On 05/16/2018 04:49 PM, Doerr, Martin wrote: >> It contains a fix in os_perf_linux.cpp which is needed for any linux platforms not supported by Oracle. > >> http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.00/ > > Looks fine. I wonder if #include CPU_HEADER would break other platforms, but we shall see. > Does it still build fine on Linux/x86_64? > > -Aleksey > From sgehwolf at redhat.com Wed May 16 15:38:30 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 16 May 2018 17:38:30 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> Message-ID: <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> Hi Martin, Aleksey, On Wed, 2018-05-16 at 17:08 +0200, Aleksey Shipilev wrote: > On 05/16/2018 05:04 PM, Doerr, Martin wrote: > > JDK-8203288 contains a change of os_perf_linux.cpp using: > > #include CPU_HEADER(vm_version_ext) > > (See http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.00/) > > > > Can you check if that works for you, please? That works, thanks! How do you want to coordinate getting both patches integrated? Shall I wait for your change to get integrated or should I get it in as is first and you'll resolve the merge conflict changing to your version in os_perf_linux.cpp? > +1 > > The rest of the patch looks good, Severin. Thanks for the reviews! Cheers, Severin From shade at redhat.com Wed May 16 15:40:15 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 16 May 2018 17:40:15 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> Message-ID: <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> On 05/16/2018 05:38 PM, Severin Gehwolf wrote: > Hi Martin, Aleksey, > > On Wed, 2018-05-16 at 17:08 +0200, Aleksey Shipilev wrote: >> On 05/16/2018 05:04 PM, Doerr, Martin wrote: >>> JDK-8203288 contains a change of os_perf_linux.cpp using: >>> #include CPU_HEADER(vm_version_ext) >>> (See http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.00/) >>> >>> Can you check if that works for you, please? > > That works, thanks! > > How do you want to coordinate getting both patches integrated? Shall I > wait for your change to get integrated or should I get it in as is > first and you'll resolve the merge conflict changing to your version in > os_perf_linux.cpp? Personally, I'd prefer PPC64/S390 to go in first, because it makes more sense to have CPU_HEADER. Then Zero can piggyback on it. -Aleksey From glaubitz at physik.fu-berlin.de Wed May 16 15:42:40 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Wed, 16 May 2018 17:42:40 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> Message-ID: On 05/16/2018 05:38 PM, Severin Gehwolf wrote: > On Wed, 2018-05-16 at 17:08 +0200, Aleksey Shipilev wrote: >> On 05/16/2018 05:04 PM, Doerr, Martin wrote: >>> JDK-8203288 contains a change of os_perf_linux.cpp using: >>> #include CPU_HEADER(vm_version_ext) >>> (See http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.00/) >>> >>> Can you check if that works for you, please? > > That works, thanks! > > How do you want to coordinate getting both patches integrated? Shall I > wait for your change to get integrated or should I get it in as is > first and you'll resolve the merge conflict changing to your version in > os_perf_linux.cpp? And I'll try to whip up a patch for linux-sparc after those two have been merged. Btw, shouldn't have these regressions been caught by the submit-hs repository? Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From martin.doerr at sap.com Wed May 16 15:46:39 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 16 May 2018 15:46:39 +0000 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> Message-ID: Hi Severin, I'd prefer to push PPC64/s390 first because it contains the final solution. If you want to push earlier, please just omit the change to os_perf_linux.cpp. Or you can change that file to the final solution. I'd be fine with that, too. Best regards, Martin -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: Mittwoch, 16. Mai 2018 17:40 To: Severin Gehwolf ; Doerr, Martin ; hotspot-dev Subject: Re: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) On 05/16/2018 05:38 PM, Severin Gehwolf wrote: > Hi Martin, Aleksey, > > On Wed, 2018-05-16 at 17:08 +0200, Aleksey Shipilev wrote: >> On 05/16/2018 05:04 PM, Doerr, Martin wrote: >>> JDK-8203288 contains a change of os_perf_linux.cpp using: >>> #include CPU_HEADER(vm_version_ext) >>> (See http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.00/) >>> >>> Can you check if that works for you, please? > > That works, thanks! > > How do you want to coordinate getting both patches integrated? Shall I > wait for your change to get integrated or should I get it in as is > first and you'll resolve the merge conflict changing to your version in > os_perf_linux.cpp? Personally, I'd prefer PPC64/S390 to go in first, because it makes more sense to have CPU_HEADER. Then Zero can piggyback on it. -Aleksey From sgehwolf at redhat.com Wed May 16 15:50:09 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Wed, 16 May 2018 17:50:09 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> Message-ID: Hi Martin, On Wed, 2018-05-16 at 15:46 +0000, Doerr, Martin wrote: > Hi Severin, > > I'd prefer to push PPC64/s390 first because it contains the final solution. > If you want to push earlier, please just omit the change to os_perf_linux.cpp. > Or you can change that file to the final solution. I'd be fine with that, too. OK. I'll wait for 8203288 to go in first and get the Zero parts in after. Thanks, Severin > -----Original Message----- > From: Aleksey Shipilev [mailto:shade at redhat.com] > Sent: Mittwoch, 16. Mai 2018 17:40 > To: Severin Gehwolf ; Doerr, Martin ; hotspot-dev > Subject: Re: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) > > On 05/16/2018 05:38 PM, Severin Gehwolf wrote: > > Hi Martin, Aleksey, > > > > On Wed, 2018-05-16 at 17:08 +0200, Aleksey Shipilev wrote: > > > On 05/16/2018 05:04 PM, Doerr, Martin wrote: > > > > JDK-8203288 contains a change of os_perf_linux.cpp using: > > > > #include CPU_HEADER(vm_version_ext) > > > > (See http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.00/) > > > > > > > > Can you check if that works for you, please? > > > > That works, thanks! > > > > How do you want to coordinate getting both patches integrated? Shall I > > wait for your change to get integrated or should I get it in as is > > first and you'll resolve the merge conflict changing to your version in > > os_perf_linux.cpp? > > Personally, I'd prefer PPC64/S390 to go in first, because it makes more sense to have CPU_HEADER. > Then Zero can piggyback on it. > > -Aleksey > From gromero at linux.vnet.ibm.com Wed May 16 16:02:18 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Wed, 16 May 2018 13:02:18 -0300 Subject: RFR(XS): 8203305: PPC64: Improve TM detection for enabling RTM on Linux / POWER9 Message-ID: <894f02c6-868e-d178-fe5e-1f1aefb979f0@linux.vnet.ibm.com> Hi, Could the following small change be reviewed please? It improves the TM detection for enabling (or not) RTM on Linux. With that change JVM uses TM capability flags advertised by the kernel instead of parsing major/minor kernel version. That way is better to address TM on POWER9 which now supports a new TM mode called "TM with no Suspend Mode" and that is not mature yet (so I'm disabling it for now). The new mode is just available on POWER9 DD2.1 NV (baremetal) and so TM on POWER9 VMs behave as on POWER8 hence RTM is enabled on POWER9 VM only. Nothing changes regarding AIX. bug: https://bugs.openjdk.java.net/browse/JDK-8203305 webrev: http://cr.openjdk.java.net/~gromero/POWER9/8203305/v1 Thank you. Best regards, Gustavo From shade at redhat.com Wed May 16 16:23:47 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 16 May 2018 18:23:47 +0200 Subject: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) In-Reply-To: <93c1a964c7ee41d4972ef78b6958a5b3@sap.com> References: <2252a635cba543a8b61dafe28b199327@sap.com> <48176edc-b498-3877-8bbc-ad24b064b487@redhat.com> <93c1a964c7ee41d4972ef78b6958a5b3@sap.com> Message-ID: <5f213774-94d7-e32e-c59c-8ea81fb402a8@redhat.com> On 05/16/2018 05:18 PM, Doerr, Martin wrote: > thanks for reviewing. Yes, linux x86_64 still works. I've also pushed it to the submission forest. Waiting for a reply ... Realized I can actually build the patch myself. So, CPU_HEADER seems to work, but the patch itself is incomplete. Tested: --- x86_64-server: builds fine --- x86_32-server: builds fine --- x86_32-minimal: builds fine --- aarch64-server: builds fine --- arm32-hflt-server: builds fine --- ppc64le-server: still fails with: /home/shade/jdk-jdk/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp:108:4: warning: #warning "Unconfigured platform" [-Wcpp] #warning "Unconfigured platform" ^~~~~~~ In file included from /home/shade/jdk-jdk/src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.cpp:38:0: /home/shade/jdk-jdk/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp:108:4: warning: #warning "Unconfigured platform" [-Wcpp] #warning "Unconfigured platform" ^~~~~~~ --- s390x-server: still fails with: In file included from /home/shade/jdk-jdk/src/hotspot/share/jfr/recorder/checkpoint/jfrCheckpointManager.cpp:36:0: /home/shade/jdk-jdk/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp:108:4: warning: #warning "Unconfigured platform" [-Wcpp] #warning "Unconfigured platform" ^~~~~~~ In file included from /home/shade/jdk-jdk/src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.cpp:38:0: /home/shade/jdk-jdk/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp:108:4: warning: #warning "Unconfigured platform" [-Wcpp] #warning "Unconfigured platform" ^~~~~~~ -Aleksey From martin.doerr at sap.com Wed May 16 16:29:14 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 16 May 2018 16:29:14 +0000 Subject: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) In-Reply-To: <5f213774-94d7-e32e-c59c-8ea81fb402a8@redhat.com> References: <2252a635cba543a8b61dafe28b199327@sap.com> <48176edc-b498-3877-8bbc-ad24b064b487@redhat.com> <93c1a964c7ee41d4972ef78b6958a5b3@sap.com> <5f213774-94d7-e32e-c59c-8ea81fb402a8@redhat.com> Message-ID: <38311dc4306147b8ba303ab447b4f61e@sap.com> Hi Aleksey, I have used --disable-warnings-as-errors as workaround. Maybe I can fix the warning tomorrow, too. Best regards, Martin -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: Mittwoch, 16. Mai 2018 18:24 To: Doerr, Martin ; hotspot-dev developers (hotspot-dev at openjdk.java.net) Cc: Thomas St?fe ; John Paul Adrian Glaubitz Subject: Re: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) On 05/16/2018 05:18 PM, Doerr, Martin wrote: > thanks for reviewing. Yes, linux x86_64 still works. I've also pushed it to the submission forest. Waiting for a reply ... Realized I can actually build the patch myself. So, CPU_HEADER seems to work, but the patch itself is incomplete. Tested: --- x86_64-server: builds fine --- x86_32-server: builds fine --- x86_32-minimal: builds fine --- aarch64-server: builds fine --- arm32-hflt-server: builds fine --- ppc64le-server: still fails with: /home/shade/jdk-jdk/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp:108:4: warning: #warning "Unconfigured platform" [-Wcpp] #warning "Unconfigured platform" ^~~~~~~ In file included from /home/shade/jdk-jdk/src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.cpp:38:0: /home/shade/jdk-jdk/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp:108:4: warning: #warning "Unconfigured platform" [-Wcpp] #warning "Unconfigured platform" ^~~~~~~ --- s390x-server: still fails with: In file included from /home/shade/jdk-jdk/src/hotspot/share/jfr/recorder/checkpoint/jfrCheckpointManager.cpp:36:0: /home/shade/jdk-jdk/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp:108:4: warning: #warning "Unconfigured platform" [-Wcpp] #warning "Unconfigured platform" ^~~~~~~ In file included from /home/shade/jdk-jdk/src/hotspot/share/jfr/instrumentation/jfrEventClassTransformer.cpp:38:0: /home/shade/jdk-jdk/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp:108:4: warning: #warning "Unconfigured platform" [-Wcpp] #warning "Unconfigured platform" ^~~~~~~ -Aleksey From shade at redhat.com Wed May 16 16:43:09 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 16 May 2018 18:43:09 +0200 Subject: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) In-Reply-To: <38311dc4306147b8ba303ab447b4f61e@sap.com> References: <2252a635cba543a8b61dafe28b199327@sap.com> <48176edc-b498-3877-8bbc-ad24b064b487@redhat.com> <93c1a964c7ee41d4972ef78b6958a5b3@sap.com> <5f213774-94d7-e32e-c59c-8ea81fb402a8@redhat.com> <38311dc4306147b8ba303ab447b4f61e@sap.com> Message-ID: <117e46dc-5e00-cb47-2de0-2451cef2e6b4@redhat.com> On 05/16/2018 06:29 PM, Doerr, Martin wrote: > I have used --disable-warnings-as-errors as workaround. Maybe I can fix the warning tomorrow, too. That warning is fixable on both PPC64 and S390 with "just": diff -r 708db0a05b27 src/hotspot/share/jfr/utilities/jfrBigEndian.hpp --- a/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp Wed May 16 16:34:52 2018 +0200 +++ b/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp Wed May 16 18:42:53 2018 +0200 @@ -102,7 +102,7 @@ inline bool JfrBigEndian::platform_supports_unaligned_reads(void) { #if defined(IA32) || defined(AMD64) return true; -#elif defined(SPARC) || defined(ARM) || defined(AARCH64) +#elif defined(SPARC) || defined(ARM) || defined(AARCH64) || defined(PPC64) || defined(S390) return false; #else #warning "Unconfigured platform" ...which returns the same "false" as the default/warned path, and it fixes normal builds. Thanks, -Aleksey From gerard.ziemski at oracle.com Wed May 16 19:04:03 2018 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Wed, 16 May 2018 14:04:03 -0500 Subject: RFR 8203292: Print non-default flags to the hs_err file In-Reply-To: References: Message-ID: <8326D964-D653-446D-87B2-F791C796A6F1@oracle.com> hi Rene, Nice enhancement and thank you for skipping the default valued flags like David Holmes suggested. I can sponsor this fix for you and will run some basic sanity tests, even though the change looks small and simple. cheers > On May 16, 2018, at 10:10 AM, Ren? Sch?nemann wrote: > > Hi, > > can I please get a review for the following change: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203292 > Webrev: http://cr.openjdk.java.net/~simonis/webrevs/2018/8203292 > > With this change, all flags with a non-default value will be printed > to the hs_error file. > While the hs_err file already contains the full command line parameters, > flags can also be changed by ergonomics or various serviceability tools. > > Thank you, > Rene From david.holmes at oracle.com Wed May 16 21:23:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 17 May 2018 07:23:30 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: Message-ID: Hi Boris, Many thanks for looking at this and working through the ARM code. I will study your patch in detail but am concerned by the "passes most of runtime/Nestmates tests Ok."! What tests are not passing? David On 17/05/2018 1:05 AM, Boris Ulasevich wrote: > Hi David, > > Let us look to the change in templateTable_arm.cpp. I have several notes. > > 1. We have compilation error because check_klass_subtype call needs one > more temporary register parameter. I think correct change is: > check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); > -> > check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); > > 2. R0_tmp holds mdp used for profilation several lines below, so we > can't spoil it. I think correct change is: > check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); > -> > check_klass_subtype(Rklass, Rinterf, R1_tmp, R2_tmp, noreg, subtype); > > 3. We can't jump to Rindex. I believe it was supposed to jump to Rmethod: > jump_from_interpreted(Rindex); > -> > jump_from_interpreted(Rmethod); > > 4. Please note than Rklass and Rflags reuses same register. Before the > change their life ranges had no intersection. I would propose to use R2 > for Rklass (same with Rrecv) and move load_klass call after > invokevirtual_helper call to avoid life range intersecton. > > See my patch against original templateTable_arm.cpp version in attach - > with this change ARM build compiles and passes most of runtime/Nestmates > tests Ok. > > regards, > Boris > > On 15.05.2018 03:52, David Holmes wrote: >> This review is being spread across four groups: langtools, core-libs, >> hotspot and serviceability. This is the specific review thread for >> hotspot - webrev: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >> >> See below for full details - including annotated full webrev guiding >> the review. >> >> The intent is to have JEP-181 targeted and integrated by the end of >> this month. >> >> Thanks, >> David >> ----- >> >> The nestmates project (JEP-181) introduces new classfile attributes to >> identify classes and interfaces in the same nest, so that the VM can >> perform access control based on those attributes and so allow direct >> private access between nestmates without requiring javac to generate >> synthetic accessor methods. These access control changes also extend >> to core reflection and the MethodHandle.Lookup contexts. >> >> Direct private calls between nestmates requires a more general calling >> context than is permitted by invokespecial, and so the JVMS is updated >> to allow, and javac updated to use, invokevirtual and invokeinterface >> for private class and interface method calls respectively. These >> changed semantics also extend to MethodHandle findXXX operations. >> >> At this time we are only concerned with static nest definitions, which >> map to a top-level class/interface as the nest-host and all its nested >> types as nest-members. >> >> Please see the JEP for further details. >> >> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >> >> All of the specification changes have been previously been worked out >> by the Valhalla Project Expert Group, and the implementation reviewed >> by the various contributors and discussed on the valhalla-dev mailing >> list. >> >> Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, >> Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir >> Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan >> >> Master webrev of all changes: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >> >> Annotated master webrev index: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >> >> Performance: this is expected to be performance neutral in a general >> sense. Benchmarking and performance runs are about to start. >> >> Testing Discussion: >> ------------------ >> >> The testing for nestmates can be broken into four main groups: >> >> -? New tests specifically related to nestmates and currently in the >> runtime/Nestmates directory >> >> - New tests to complement existing tests by adding in testcases not >> previously expressible. >> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests use >> of invokespecial for private interface methods and performing receiver >> typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do >> similar tests for invokeinterface. >> >> -? New JVM TI tests to verify the spec changes related to nest >> attributes. >> >> -? Existing tests significantly affected by the nestmates changes, >> primarily: >> ??? -? runtime/SelectionResolution >> >> ??? In most cases the nestmate changes makes certain invocations that >> were illegal, legal (e.g. not requiring invokespecial to invoke >> private interface methods; allowing access to private members via >> reflection/Methodhandles that were previously not allowed). >> >> - Existing tests incidentally affected by the nestmate changes >> >> ?? This includes tests of things utilising class >> redefinition/retransformation to alter nested types but which >> unintentionally alter nest relationships (which is not permitted). >> >> There are still a number of tests problem-listed with issues filed >> against them to have them adapted to work with nestmates. Some of >> these are intended to be addressed in the short-term, while some (such >> as the runtime/SelectionResolution test changes) may not eventuate. >> >> - https://bugs.openjdk.java.net/browse/JDK-8203033 >> - https://bugs.openjdk.java.net/browse/JDK-8199450 >> - https://bugs.openjdk.java.net/browse/JDK-8196855 >> - https://bugs.openjdk.java.net/browse/JDK-8194857 >> - https://bugs.openjdk.java.net/browse/JDK-8187655 >> >> There is also further test work still to be completed (the JNI and JDI >> invocation tests): >> - https://bugs.openjdk.java.net/browse/JDK-8191117 >> which will continue in parallel with the main RFR. >> >> Pre-integration Testing: >> ??- General: >> ???? - Mach5: hs/jdk tier1,2 >> ???? - Mach5: hs-nightly (tiers 1 -3) >> ??- Targetted >> ??? - nashorn (for asm changes) >> ??? - hotspot: runtime/* >> ?????????????? serviceability/* >> ?????????????? compiler/* >> ?????????????? vmTestbase/* >> ??? - jdk: java/lang/invoke/* >> ?????????? java/lang/reflect/* >> ?????????? java/lang/instrument/* >> ?????????? java/lang/Class/* >> ?????????? java/lang/management/* >> ?? - langtools: tools/javac >> ??????????????? tools/javap >> From david.holmes at oracle.com Wed May 16 21:26:52 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 17 May 2018 07:26:52 +1000 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> Message-ID: On 17/05/2018 1:42 AM, John Paul Adrian Glaubitz wrote: > On 05/16/2018 05:38 PM, Severin Gehwolf wrote: >> On Wed, 2018-05-16 at 17:08 +0200, Aleksey Shipilev wrote: >>> On 05/16/2018 05:04 PM, Doerr, Martin wrote: >>>> JDK-8203288 contains a change of os_perf_linux.cpp using: >>>> #include CPU_HEADER(vm_version_ext) >>>> (See http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.00/) >>>> >>>> Can you check if that works for you, please? >> >> That works, thanks! >> >> How do you want to coordinate getting both patches integrated? Shall I >> wait for your change to get integrated or should I get it in as is >> first and you'll resolve the merge conflict changing to your version in >> os_perf_linux.cpp? > > And I'll try to whip up a patch for linux-sparc after those two have been merged. > > Btw, shouldn't have these regressions been caught by the submit-hs repository? No - it only builds the Oracle supported platforms. David > Adrian > From glaubitz at physik.fu-berlin.de Wed May 16 21:29:15 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Wed, 16 May 2018 23:29:15 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> Message-ID: <7a01f7c5-70ca-cf8d-3371-826a13f467f5@physik.fu-berlin.de> On 05/16/2018 11:26 PM, David Holmes wrote: >> And I'll try to whip up a patch for linux-sparc after those two have been merged. >> >> Btw, shouldn't have these regressions been caught by the submit-hs repository? > > No - it only builds the Oracle supported platforms. Does this exclude ppc64 and s390x? Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From david.holmes at oracle.com Wed May 16 21:59:23 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 17 May 2018 07:59:23 +1000 Subject: RFR 8203292: Print non-default flags to the hs_err file In-Reply-To: References: Message-ID: Looks good Rene! :) Thanks, David On 17/05/2018 1:10 AM, Ren? Sch?nemann wrote: > Hi, > > can I please get a review for the following change: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203292 > Webrev: http://cr.openjdk.java.net/~simonis/webrevs/2018/8203292 > > With this change, all flags with a non-default value will be printed > to the hs_error file. > While the hs_err file already contains the full command line parameters, > flags can also be changed by ergonomics or various serviceability tools. > > Thank you, > Rene > From david.holmes at oracle.com Wed May 16 22:14:22 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 17 May 2018 08:14:22 +1000 Subject: RFR: 8203227: Introduce os::processor_id() for Linux and Solaris In-Reply-To: <1c382d62-1d65-4bba-5027-773a374967d5@oracle.com> References: <327220bf-ea05-1921-59e1-2bd0c2c4dea8@oracle.com> <13cbc7ca-58eb-ae0b-9640-42be8e5fb3fd@oracle.com> <3eac8bb2-fc7a-42b2-154b-e7f6cd63b20b@oracle.com> <8905be7b-eadc-be5f-c48c-611a17a62edf@oracle.com> <1c382d62-1d65-4bba-5027-773a374967d5@oracle.com> Message-ID: <5b39f546-1081-9cd9-49e6-a6ecdc8ccf10@oracle.com> On 16/05/2018 10:54 PM, Per Liden wrote: > Hi David, > > On 05/16/2018 02:16 PM, David Holmes wrote: > [...] >>>>> We're using this in ZGC to do various CPU-local operations. We >>>>> currently >>>> >>>> So there's a requirement that the CPU value returned by >>>> processor_id() is compatible with some other API's that take a >>>> processor id? Does that need documenting in some form? >>> >>> Good point, this should be made more clear. We currently assume that >>> os::processor_id() returns a value from 0 to (os::processor_count() - >>> 1). Adding a comment in os.hpp clarifying that. >> >> Thanks. Not every value in that range may be a valid processor id, but >> every valid processor id should be within that range. > > Yep. > >> >>>>> only need this for Linux and Solaris, which is what this patch adds. >>>> >>>> Are the API's that take the processor id defined in os as well? >>>> Otherwise it seems this functionality should just be localized to >>>> the specific os_.cpp file from which it is used (and/or the >>>> os:: class). >>> >>> The os::processor_id() call-sites we have are in shared os-agnostic >>> code. This function is also fairly closely related to >>> os::processor_count(), so it seems like os:: would be the natural >>> place for this. >> >> I'm curious as to what functions in os-agnostic code consume the >> processor id value? Or are you storing/caching it in the shared code >> and only using it later from OS specific code? > > The typical way we use this in ZGC is to have CPU-local data, > conceptually equivalent to this toy example: > > // Allocate per-CPU data > MyData* my_per_cpu_data = alloc(sizeof(MyData) * os::processor_count()); > > // Read CPU-local data > ... = my_per_cpu_data[os::processor_id()]; Ah! I see. Not what I initially expected - I thought you'd end up using the id to pass to some other OS specific function to "operate" with respect to a given cpu. But it's simply an index for "cpu-local" data. Thanks, David ------ > Have a look at the ZPerCPU template class if you're interested in more > details: > http://hg.openjdk.java.net/zgc/zgc/file/118cb3e3517d/src/hotspot/share/gc/z/zValue.hpp > > > This template basically allows us to just say: > > // Allocate per-CPU data > ZPerCPU my_per_cpu_data; > > // Read CPU-local data > ... = my_per_cpu_data.get(); > > It should be noted that ZGC is only ever built (controlled by sh > configure) for supported platforms (Linux & Solaris at this moment), > which is why it's fine to have os::processor_id() calls in this shared > code. > >> >>>> >>>>> Support for additional platforms can be added when needed. I didn't >>>>> add dummy os::processor_id() to the not-yet-implemented platforms, >>>>> since I think it's better to get a early compile-time error instead >>>>> of a Unimplemented() at runtime, if you try to use them. >>>> >>>> I'd rather not see the runtime support of our primary platforms of >>>> interest fragmented like this. Are there technical challenges to >>>> doing this? >>> >>> I'm a bit reluctant to add this for other platforms at this point, >>> since some platforms might require non-trivial implementations, which >>> would go unused/untested. For macOS I believe we would need something >>> like this: >>> >>> http://cr.openjdk.java.net/~gziemski/zgc_os_processor_id/webrev/src/hotspot/os/bsd/os_bsd.cpp.udiff.html >> >> >> >> Yeah direct use of cpuid is not pleasant. >> >>> (I should add that I don't know if this patch does the right thing, >>> but Gerard apparently had some success with it) >>> >>> For Windows, I believe a call to GetCurrentProcessorNumber() should >>> be enough (we don't seem to handle/support logical processor groups >>> in os::processor_count() so should be safe to do the same thing here). >> >> The current processor should always be correct regardless of what >> logical/virtual processor configuration exists. > > Check. > >> >>> The plan has been to add support for these if/when ZGC supports these >>> platforms (unless someone else needs and adds them before that). But >>> adding them now makes me a bit uneasy, especially given the >>> non-trivial nature of the macOS implementation. >> >> Okay. I'd prefer to see all platforms covered, but not with the risk >> of untested code being used. >> >>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203227 >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8203227/webrev.0 >>>> >>>> For Solaris there is no need to introduce Solaris::getcpuid() just >>>> because Linux already has that. (Unless this ends up not being >>>> needed in the os class of course :) ). >>> >>> Check. >>> >>>> >>>> Are you allowing for the -1 return value on Linux or assuming it >>>> will never happen in practice? If the former then that should be >>>> documented as part of the semantics of os::processor_id(). If the >>>> latter then you should probably add an assert or guarantee. I >>>> suspect this is a historical restriction no longer relevant to us - >>>> in which case we can probably clean out the dynamic lookup/linking!). >>> >>> We're assuming it will never happen in practice, and this is >>> currently guarded with a guarantee() during initialization of ZGC. >>> sched_getcpu() has "only" been around since glibc 2.14 (June 2011). >>> However, we can also receive -1 if the kernel doesn't have the >>> getcpu() syscall (first introduced in Linux 2.6.19, Nov 2006). >>> >>> So, it seems safe to at least assume that the getcpu() syscall is >>> available, and only have an single check for that during startup >>> (with a vm_exit_during_initialization() if it's unavailable). >> >> Okay. We should clean up the syscall and dynamic lookup code use here. >> >>> Here's an updated webrev for continued discussion: >>> >>> http://cr.openjdk.java.net/~pliden/8203227/webrev.1 >> >> Looks fine to me. > > Ok, thanks David! > > /Per > >> >> Thanks, >> David >> >>> cheers, >>> Per >>> >>>> >>>> Thanks, >>>> David >>>> >>>>> /Per From leonid.mesnik at oracle.com Wed May 16 22:14:31 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Wed, 16 May 2018 15:14:31 -0700 Subject: RFR: 8199064: Test applications/jcstress/other/Test.java#id1108 fails on Sparc Message-ID: <4A1B218C-66B9-433A-9BA9-BA495CDF4EA9@oracle.com> Hi Could you please review following patch which increase timeout for jcstress test wrappers in jtreg. The default jtreg timeout is not enough for a lot of jcstress tests. The longest tests take more then 1800 seconds on typical VM used for testing (2 CPU/16GB of Memory) with default VM options. These tests are not executed in tier1/2 testing so execution time in the case if test really hang is not very critical. I just set timeout to 2400 seconds for all test wrappers. The timeoutfactor could be used to adjust timeouts for slow platforms and/or VM options. webrev: http://cr.openjdk.java.net/~lmesnik/8199064/webrev.00/ bug: https://bugs.openjdk.java.net/browse/JDK-8199064 Leonid From rachmato at redhat.com Wed May 16 21:30:39 2018 From: rachmato at redhat.com (Richard Achmatowicz) Date: Wed, 16 May 2018 17:30:39 -0400 Subject: Enabling use of hugepages with java In-Reply-To: <5b5c7e1d-a6aa-a468-590e-9cbea17314be@oracle.com> References: <9e71448b-2d3d-acb6-79e8-47d1d0987da3@oracle.com> <5b5c7e1d-a6aa-a468-590e-9cbea17314be@oracle.com> Message-ID: <0cd776a9-ff79-4b5e-4e33-5ef30477abf4@redhat.com> Hi Stefan Thanks very much for your reply. I wish I had seen it before wading through the JDK sources (virtualspace.cpp, os_linux.cpp) :-) Through running some tests, I have found that when using the option -XX:+UseSHM, in addition to raising shmmax, the process requesting the large pages also needs to be a member of the group vm.hugetlb_shm_group: // Running as me (who is not in group 0) [nrla at localhost hugepages]$ sysctl -a | grep shm kernel.shm_next_id = -1 kernel.shm_rmid_forced = 0 kernel.shmall = 18446744073692774399 kernel.shmmax = 18446744073692774399 kernel.shmmni = 4096 vm.hugetlb_shm_group = 0 [nrla at localhost hugepages]$ java -Xmx7G -XX:+UseLargePages -XX:+UseSHM HugePagesTest Java HotSpot(TM) 64-Bit Server VM warning: Failed to reserve shared memory. (error = 1) Java HotSpot(TM) 64-Bit Server VM warning: Failed to reserve shared memory. (error = 1) Allocating (int) array of 4294967296 bytes (4.0 Gb). Allocated (int) array successfully. Press "ENTER" to continue... Shutting down. // Running as root (who is in group 0) [root at localhost hugepages]# java -Xmx7G -XX:+UseLargePages -XX:+UseSHM HugePagesTest Allocating (int) array of 4294967296 bytes (4.0 Gb). Allocated (int) array successfully. Press "ENTER" to continue... Shutting down. The large memory usage, after successful allocation and then after termination: [nrla at localhost ~]$ cat /proc/meminfo | grep Huge AnonHugePages:???????? 0 kB ShmemHugePages:??????? 0 kB HugePages_Total:??? 4096 HugePages_Free:???? 2045 HugePages_Rsvd:???? 1645 HugePages_Surp:??????? 0 Hugepagesize:?????? 2048 kB [nrla at localhost ~]$ cat /proc/meminfo | grep Huge AnonHugePages:???????? 0 kB ShmemHugePages:??????? 0 kB HugePages_Total:??? 4096 HugePages_Free:???? 4096 HugePages_Rsvd:??????? 0 HugePages_Surp:??????? 0 Hugepagesize:?????? 2048 kB Richard On 03/06/2018 03:30 AM, Stefan Karlsson wrote: > Hi Richard, > > On 2018-02-28 23:04, David Holmes wrote: >> Hi Richard, >> >> Moving to hotspot-dev as the appropriate list. >> >> David >> >> On 1/03/2018 1:20 AM, Richard Achmatowicz wrote: >>> Hi >>> >>> I hope that I am directing this question to the correct mailing list. >>> >>> I have a question concerning the OS setup on Linux required for >>> correct use of the java option -XX:+UseLargePages in JDK 8. >>> >>> Official Oracle documentation >>> (http://www.oracle.com/technetwork/java/javase/tech/largememory-jsp-137182.html) >>> suggests that in order to make use of large memory pages, in >>> addition to setting the flag -XX:+UseLargePages, an OS option shmmax >>> needs to be tuned to be larger than the java heap size. >>> >>> ?From looking at the java documentation, there are various ways of >>> enabling the use of huge pages: -XX:+UseHugeTLBFS, >>> -XX:+UseTransparentHugePages, -XX:+UseSHM and, if I understand >>> correctly, these correspond in part to making use of different >>> OS-level APIs for accessing huge pages (via shared memory, >>> hugetlbfs, and other means). >>> >>> My question is this: is setting the shmmax OS value only relevant if >>> we are using -XX:+UseSHM? In other words, if we are using >>> -XX:+UseHugeTLBFS to enable use of hugepages by the JVM, is it the >>> case that setting the shmmax OS setting has no effect on the use of >>> hugepages by the JVM? > > Yes, your understanding is correct. > > The document you link to seems to be from a time before > -XX:+UseHugeTLBFS was added. > > This document clarifies this a bit more: > ?https://docs.oracle.com/javase/8/docs/technotes/tools/unix/java.html#large_pages > > > ?"If you are using the option -XX:+UseSHM (instead of > -XX:+UseHugeTLBFS), then increase the SHMMAX value." > > Cheers, > StefanK > >>> >>> Thanks in advance >>> >>> Richard >>> > From david.holmes at oracle.com Wed May 16 22:43:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 17 May 2018 08:43:30 +1000 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <7a01f7c5-70ca-cf8d-3371-826a13f467f5@physik.fu-berlin.de> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7a01f7c5-70ca-cf8d-3371-826a13f467f5@physik.fu-berlin.de> Message-ID: <7be461bc-e070-2c4a-7ecd-3222285a5e23@oracle.com> On 17/05/2018 7:29 AM, John Paul Adrian Glaubitz wrote: > On 05/16/2018 11:26 PM, David Holmes wrote: >>> And I'll try to whip up a patch for linux-sparc after those two have been merged. >>> >>> Btw, shouldn't have these regressions been caught by the submit-hs repository? >> >> No - it only builds the Oracle supported platforms. > > Does this exclude ppc64 and s390x? Yes - and ARM and Aarch64, and x86-32. And AIX, BSD, ... The Oracle supported platforms are currently: - linux x64 - windows x64 - Mac OS x64 - Solaris sparc David > Adrian > From glaubitz at physik.fu-berlin.de Wed May 16 22:50:31 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Thu, 17 May 2018 00:50:31 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <7be461bc-e070-2c4a-7ecd-3222285a5e23@oracle.com> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7a01f7c5-70ca-cf8d-3371-826a13f467f5@physik.fu-berlin.de> <7be461bc-e070-2c4a-7ecd-3222285a5e23@oracle.com> Message-ID: On 05/17/2018 12:43 AM, David Holmes wrote: >> Does this exclude ppc64 and s390x? > > Yes - and ARM and Aarch64, and x86-32. And AIX, BSD, ... > > The Oracle supported platforms are currently: > - linux x64 > - windows x64 > - Mac OS x64 > - Solaris sparc Ah, ok. Might be an idea then to set up some CI jobs on the various servers I have access to to provide automated builds for more targets. I have had this idea for some time now to help find regressions earlier. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From david.holmes at oracle.com Wed May 16 22:54:05 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 17 May 2018 08:54:05 +1000 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7a01f7c5-70ca-cf8d-3371-826a13f467f5@physik.fu-berlin.de> <7be461bc-e070-2c4a-7ecd-3222285a5e23@oracle.com> Message-ID: Hi Adrian, On 17/05/2018 8:50 AM, John Paul Adrian Glaubitz wrote: > On 05/17/2018 12:43 AM, David Holmes wrote: >>> Does this exclude ppc64 and s390x? >> >> Yes - and ARM and Aarch64, and x86-32. And AIX, BSD, ... >> >> The Oracle supported platforms are currently: >> - linux x64 >> - windows x64 >> - Mac OS x64 >> - Solaris sparc > > Ah, ok. Might be an idea then to set up some CI jobs on the various servers > I have access to to provide automated builds for more targets. I have had this > idea for some time now to help find regressions earlier. See attached for existing community efforts in this area. David > Adrian > -------------- next part -------------- An embedded message was scrubbed... From: Martijn Verburg Subject: Re: JDK submit repo Date: Wed, 7 Mar 2018 21:37:11 +0000 Size: 6419 URL: From glaubitz at physik.fu-berlin.de Wed May 16 22:58:06 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Thu, 17 May 2018 00:58:06 +0200 Subject: AdoptOpenJDK build farm - was: Re: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7a01f7c5-70ca-cf8d-3371-826a13f467f5@physik.fu-berlin.de> <7be461bc-e070-2c4a-7ecd-3222285a5e23@oracle.com> Message-ID: <029e83e0-bd1f-7d31-6eb2-1d7fe995e5e3@physik.fu-berlin.de> On 05/17/2018 12:54 AM, David Holmes wrote: >> Ah, ok. Might be an idea then to set up some CI jobs on the various servers >> I have access to to provide automated builds for more targets. I have had this >> idea for some time now to help find regressions earlier. > > See attached for existing community efforts in this area. Oh, that's nice. Missing my beloved Linux on SPARC though ;). I will get in touch with those guys. Thanks for the heads up! Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From kim.barrett at oracle.com Wed May 16 23:51:50 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 16 May 2018 19:51:50 -0400 Subject: RFR: 8202813: Move vm_weak processing from SystemDictionary::do_unloading to WeakProcessor In-Reply-To: <10a99bf5-d3cc-9d5b-8a38-5ba1672f9f8b@oracle.com> References: <10a99bf5-d3cc-9d5b-8a38-5ba1672f9f8b@oracle.com> Message-ID: <4E15250A-9872-4195-892B-C1E0078A80FD@oracle.com> > On May 16, 2018, at 8:42 AM, Stefan Johansson wrote: > > Hi Kim, > > On 2018-05-08 21:34, Kim Barrett wrote: >> Please review this change to move the clearing of dead entries in >> vm_weak_oop_storage from SystemDictionary::do_unloading to preceeding >> calls to WeakProcessor::weak_oops_do. >> This involves conditionalizing WeakProcessor invocations, to allow >> different calls to process different subsets of the native weak >> references. For example, vm_weak_oop_storage is treated as strong >> rather than weak when ClassUnloading is disabled. >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8202813 >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8202813/open.00/ > Thanks for doing this work to unify weak root processing. Thanks for looking at it. > I might be missing the bigger picture, so before starting to commenting the code I have a couple of questions: > 1. With this patch SystemDictionary::vm_weak_oop_storage is now processed both by the WeakProcessor in weak_oops_do and the SystemDictionary in oops_do() and roots_oops_do(). Shouldn't all processing be moved to the WeakProcessor to make it clear who is responsible for doing the processing? Or why do we want to treat do_unloading special? Yes, it should be moved completely. Baby steps. I started to change (remove, actually, and update callers) SystemDictionary::oops_do, but discovered jfr was using it, and didn't want to touch that in the middle of the jfr review. And there are some preliminary cleanups I'm working on around roots_oops_do, but haven't finished them yet. > 2. For this patch I don't see the need for the PhaseSet class and the serial and parallel phases distinction. Couldn't we just pass in a bool to weak_oops_do saying if we should handle the vm_weak_oop_storage or not? I understand that for upcoming patches this might be needed, but then I think this should be added in that change. I agree it's over-engineered for the needs of this change set. However, further conditionalization beyond a boolean for class-unloading is going to be needed RSN for the new StringTable (based on the new concurrent hash table + oopstorage). StringTable processing is another place where the various GCs and their sub-parts vary about where it's treated as strong or weak. That should be showing up shortly, and is being worked on by folks who aren't me. So unless you want me to separate that out into a separate change that will likely be my next one to support that work... From vladimir.x.ivanov at oracle.com Thu May 17 00:01:16 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 16 May 2018 17:01:16 -0700 Subject: RFR(L) : 8199384 : [TESTBUG] Open source VM testbase MLVM tests In-Reply-To: <53E4CA40-DE86-41A2-A1BE-239BE1D85069@oracle.com> References: <53E4CA40-DE86-41A2-A1BE-239BE1D85069@oracle.com> Message-ID: <320aedf5-40b9-c718-19b1-290211879e7f@oracle.com> Looks good. Best regards, Vladimir Ivanov On 5/9/18 14:09, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8199384/webrev.00/index.html >> 61414 lines changed: 61414 ins; 0 del; 0 mod; > > Hi all, > > could you please review this patch which open sources MLVM tests from VM testbase? > > these tests were developed in early days of JSR292 to test different aspects of MethodHandles and invokedynamic. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199384 > webrev: http://cr.openjdk.java.net/~iignatyev//8199384/webrev.00/index.html > testing: :vmTestbase_vm_mlvm test group > > Thanks, > -- Igor > From markus.gronlund at oracle.com Thu May 17 01:13:12 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 16 May 2018 18:13:12 -0700 (PDT) Subject: Flight Recorder and non-Oracle platforms Message-ID: Hello again, many thanks to all of you for your war efforts yesterday. I would like to pre-empt some about what you can expect once you start getting your platforms building again. The short "I just got my system building again, please don't have me run into more issues"-version: We currently have only one specific platform listed that is explicitly excluded out-of-the box for JFR (in addition to MINIMAL target), and that is linux-sparc. Please see [1] # Enable JFR by default, except on linux-sparcv9 and on minimal. if test "x$OPENJDK_TARGET_OS" != xlinux || test "x$OPENJDK_TARGET_CPU" != xsparcv9; then NON_MINIMAL_FEATURES="$NON_MINIMAL_FEATURES jfr" fi This code section is ported from closed, and as you can see, it is missing some of the open platforms here. You might want to add your os / cpu here to exclude JFR from building by default (it controls the INCLUDE_JFR macro). Excluded INCLUDE_JFR will also take out the command-line options so one should not even be able to attempt to start the functionality. In addition, and a more dynamic way to exclude JFR from a build, is to use the JVM_FEATURE configure option: --with-jvm-features= -jfr feature_1 feature2 (note the minus sign) Still interested? There are two platform dependent areas that might require some work to get the full functionality of JFR supported on a specific platform. 1. CPU information "runtime/os_perf.hpp" includes a few interfaces that is used for uniformly pulling out (periodically, at intervals) OS / CPU specific information, for example: CPUInformationInterface (informational) Impl: cpu//VM_Version_Ext.cpp files has been added for this purpose (for some CPUs) CPUPerformanceInterface (counters) Impl: os//os_flavor.cpp which some have seen today as part of our mayhem. A good thing with the CPUPerformanceInterface related information is that JFR has "support" for FUNCTIONALITY_NOT_IMPLEMENTED, for example see [2]: The FUNCTIONALITY_NOT_IMPLEMENTED macro is defined in "runtime/os_perf.hpp" It is therefore possible "stub out" one area of platform dependent functionality (I think our tests are lenient about this too). One last thing about CPU Information - if JFR functionality be requested, say via command-line: -XX:StartFlightRecording Then the CPU Information interface implementation is expected to be successfully initialized for the VM to start. The notion of "successfully initialized" here is very basic indeed, the only thing needed is that: bool initialize () routines return "true" (since false now indicates something is wrong and the VM will not start (command-line) or JFR will do a transactional "rollback" if attempted dynamically) 2. Thread sampling You might already know that JFR share, or have similar, stack walking code as AsynchGetCallTrace / Forte. And just as Forte had its: bool pd_get_top_frame_for_signal_handler(frame* fr_addr, void* ucontext, bool isInJava); // usually implemented (or not) in os_cpu/thread_.cpp For example, see [3] JFR has introduced its own: bool pd_get_top_frame_for_profiling(frame* fr_addr, void* ucontext, bool isInJava); pretty much in the same location as the first top_frame_for_signal_handler. Why not use the same? We do things slightly different in JFR compared to Forte, we have a tighter collaboration between the suspender and the suspendee (we can also support non sigprof platforms, such as Windows). Tying back to the linked example for thread_linux_sparc.cpp above, this was the reason that linux-sparc was excluded from building by default if I remember correctly, there was not enough time to craft a profiling () method. It should be possible to look at some x86 platform to see what the relevant differences are as a basis for adding this support. The invocation of pd_get_top_frame_for_profiling(frame* fr_addr, void* ucontext, bool isInJava) comes from here, see [4] and [5] It is also possible to "stub out" this code; if pd_get_top_frame_for_profiling(frame* fr_addr, void* ucontext, bool isInJava) returns false, the sampling attempt is skipped. That is totally fine. There is some additional platform specific support that might need to be implemented in the context of thread sampling, for example "crash protection" please lookup: os::ThreadCrashProtection Now this has been in open for some time, so its status might be better than some of the other parts. Summary: If you are not interested in the JFR functionality, and / or want it excluded from the builds, please exclude your platform combo from the default build via hotspot.m4 [1] Platform specific sensitive areas for JFR are around CPU Information / CPU Performance counters as well as thread sampling (e.g. stack walking / traversal). The JFR system should be resilient to withstand running with only a subset of events / implementations in most parts, but the most critical will be the ones listed above. Hoping for a better day today Best regards for now Markus [1] http://hg.openjdk.java.net/jdk/jdk/file/9010b580d8a9/make/autoconf/hotspot.m4#l312 [2] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/os/bsd/os_perf_bsd.cpp#l98 [3] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/os_cpu/linux_sparc/thread_linux_sparc.cpp#l40 [4] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/share/jfr/periodic/sampling/jfrThreadSampler.cpp#l180 [5] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/share/jfr/periodic/sampling/jfrCallTrace.cpp#l98 From serguei.spitsyn at oracle.com Thu May 17 03:48:47 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 16 May 2018 20:48:47 -0700 Subject: 8199271: [TESTBUG] open source VM testbase stress tests In-Reply-To: <8BA2B023-E947-4EF8-9D97-EBA28BAA07E2@oracle.com> References: <1BC403EC-BDF0-4CA2-B87F-B38A92B6765F@oracle.com> <5AF5C07B.1080909@oracle.com> <8BA2B023-E947-4EF8-9D97-EBA28BAA07E2@oracle.com> Message-ID: <58bfdd2c-9163-2464-93bd-9ee1e265d97e@oracle.com> Hi Leonid, Looks good to me too. Thanks, Serguei On 5/14/18 14:04, Leonid Mesnik wrote: > Misha > > Thank you for review. I still need one more review from 'R'eviewer. > > Leonid > >> On May 11, 2018, at 9:10 AM, Mikhailo Seledtsov wrote: >> >> Looks good to me, >> >> Misha >> >> On 5/8/18, 2:23 PM, Leonid Mesnik wrote: >>> Hi >>> >>> Please review this change open sourcing vm testbase stress tests. These tests have been developed a long time ago for internal test harness and don't looks very nice now. >>> They are open sourced with minimal changes only. The long term plan is to review and improve them moving out of vmTestbase directory in corresponding components. >>> >>> The fix introduce vmTestbase_nsk_stress test group and have make files changes required to build native part of tests. >>> >>> I've verified that "make -- run-test TEST=:vmTestbase_nsk_stress" pass on my linux locally. >>> >>> webrev: http://cr.openjdk.java.net/~lmesnik/8199271/webrev.00/ >>> bug: https://bugs.openjdk.java.net/browse/JDK-8199271 From thomas.stuefe at gmail.com Thu May 17 06:22:37 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 17 May 2018 08:22:37 +0200 Subject: Flight Recorder and non-Oracle platforms In-Reply-To: References: Message-ID: Hi Markus, is the JFR in openjdk11 compatible with existing Mission Control binaries, e.g. from an oracle jdk 10? So, that we could port JFR and already play around with it without having to wait for porting MC? As far as I understand (not an expert), MC can be used remotely too, cross-platform? So if the answer is "yes", one should be able to use e.g. a windows MC to connect remotely to openjdk+JFR on a porting platform? Thanks, Thomas On Thu, May 17, 2018 at 3:13 AM, Markus Gronlund wrote: > Hello again, > > many thanks to all of you for your war efforts yesterday. > > I would like to pre-empt some about what you can expect once you start getting your platforms building again. > > The short "I just got my system building again, please don't have me run into more issues"-version: > > We currently have only one specific platform listed that is explicitly excluded out-of-the box for JFR (in addition to MINIMAL target), and that is linux-sparc. > > Please see [1] > > # Enable JFR by default, except on linux-sparcv9 and on minimal. > > if test "x$OPENJDK_TARGET_OS" != xlinux || test "x$OPENJDK_TARGET_CPU" != xsparcv9; then > > NON_MINIMAL_FEATURES="$NON_MINIMAL_FEATURES jfr" > > fi > > This code section is ported from closed, and as you can see, it is missing some of the open platforms here. You might want to add your os / cpu here to exclude JFR from building by default (it controls the INCLUDE_JFR macro). Excluded INCLUDE_JFR will also take out the command-line options so one should not even be able to attempt to start the functionality. > > In addition, and a more dynamic way to exclude JFR from a build, is to use the JVM_FEATURE configure option: > > --with-jvm-features= -jfr feature_1 feature2 (note the minus sign) > > > > Still interested? > > There are two platform dependent areas that might require some work to get the full functionality of JFR supported on a specific platform. > > 1. CPU information > > "runtime/os_perf.hpp" includes a few interfaces that is used for uniformly pulling out (periodically, at intervals) OS / CPU specific information, for example: > > CPUInformationInterface (informational) > > Impl: > > cpu//VM_Version_Ext.cpp files has been added for this purpose (for some CPUs) > > CPUPerformanceInterface (counters) > > Impl: > > os//os_flavor.cpp which some have seen today as part of our mayhem. > > A good thing with the CPUPerformanceInterface related information is that JFR has "support" for FUNCTIONALITY_NOT_IMPLEMENTED, for example see [2]: > > The FUNCTIONALITY_NOT_IMPLEMENTED macro is defined in "runtime/os_perf.hpp" > > It is therefore possible "stub out" one area of platform dependent functionality (I think our tests are lenient about this too). > > One last thing about CPU Information - if JFR functionality be requested, say via command-line: > > -XX:StartFlightRecording > > Then the CPU Information interface implementation is expected to be successfully initialized for the VM to start. The notion of "successfully initialized" here is very basic indeed, the only thing needed is that: > > bool initialize () > > routines return "true" (since false now indicates something is wrong and the VM will not start (command-line) or JFR will do a transactional "rollback" if attempted dynamically) > > 2. Thread sampling > > You might already know that JFR share, or have similar, stack walking code as AsynchGetCallTrace / Forte. And just as Forte had its: > > bool pd_get_top_frame_for_signal_handler(frame* fr_addr, void* ucontext, bool isInJava); > > // usually implemented (or not) in os_cpu/thread_.cpp > > For example, see [3] > > JFR has introduced its own: > > bool pd_get_top_frame_for_profiling(frame* fr_addr, void* ucontext, bool isInJava); > > pretty much in the same location as the first top_frame_for_signal_handler. > > Why not use the same? We do things slightly different in JFR compared to Forte, we have a tighter collaboration between the suspender and the suspendee (we can also support non sigprof platforms, such as Windows). > > Tying back to the linked example for thread_linux_sparc.cpp above, this was the reason that linux-sparc was excluded from building by default if I remember correctly, there was not enough time to craft a profiling () method. It should be possible to look at some x86 platform to see what the relevant differences are as a basis for adding this support. > > The invocation of pd_get_top_frame_for_profiling(frame* fr_addr, void* ucontext, bool isInJava) comes from here, see [4] and [5] > > It is also possible to "stub out" this code; if pd_get_top_frame_for_profiling(frame* fr_addr, void* ucontext, bool isInJava) returns false, the sampling attempt is skipped. That is totally fine. > > There is some additional platform specific support that might need to be implemented in the context of thread sampling, for example "crash protection" > > please lookup: > > os::ThreadCrashProtection > > Now this has been in open for some time, so its status might be better than some of the other parts. > > Summary: > > If you are not interested in the JFR functionality, and / or want it excluded from the builds, please exclude your platform combo from the default build via hotspot.m4 [1] > > Platform specific sensitive areas for JFR are around CPU Information / CPU Performance counters as well as thread sampling (e.g. stack walking / traversal). > > The JFR system should be resilient to withstand running with only a subset of events / implementations in most parts, but the most critical will be the ones listed above. > > Hoping for a better day today > > Best regards for now > > Markus > > > > [1] http://hg.openjdk.java.net/jdk/jdk/file/9010b580d8a9/make/autoconf/hotspot.m4#l312 > > [2] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/os/bsd/os_perf_bsd.cpp#l98 > > [3] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/os_cpu/linux_sparc/thread_linux_sparc.cpp#l40 > > [4] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/share/jfr/periodic/sampling/jfrThreadSampler.cpp#l180 > > [5] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/share/jfr/periodic/sampling/jfrCallTrace.cpp#l98 > > > > > > > > From erik.gahlin at oracle.com Thu May 17 07:11:24 2018 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Thu, 17 May 2018 09:11:24 +0200 Subject: Flight Recorder and non-Oracle platforms In-Reply-To: References: Message-ID: <03DD48FB-E919-471E-A6A5-C7DC7D79E186@oracle.com> It?s may possible to connect using MC, but you will not be able to open a recording. The file format version was bumped. It?s not that it has changed, but events in JDK 11 are prefixed with ?jdk.? while events in Oracle JDK 10 were prefixed with ?com.oracle.jdk.? Flight Recorder comes with an API and it?s easy to pretty print a recording if you want to play around. There is an example in the JEP [1] Erik [1] http://openjdk.java.net/jeps/328 > On 17 May 2018, at 08:22, Thomas St?fe wrote: > > Hi Markus, > > is the JFR in openjdk11 compatible with existing Mission Control > binaries, e.g. from an oracle jdk 10? So, that we could port JFR and > already play around with it without having to wait for porting MC? > > As far as I understand (not an expert), MC can be used remotely too, > cross-platform? So if the answer is "yes", one should be able to use > e.g. a windows MC to connect remotely to openjdk+JFR on a porting > platform? > > Thanks, Thomas > > > On Thu, May 17, 2018 at 3:13 AM, Markus Gronlund > wrote: >> Hello again, >> >> many thanks to all of you for your war efforts yesterday. >> >> I would like to pre-empt some about what you can expect once you start getting your platforms building again. >> >> The short "I just got my system building again, please don't have me run into more issues"-version: >> >> We currently have only one specific platform listed that is explicitly excluded out-of-the box for JFR (in addition to MINIMAL target), and that is linux-sparc. >> >> Please see [1] >> >> # Enable JFR by default, except on linux-sparcv9 and on minimal. >> >> if test "x$OPENJDK_TARGET_OS" != xlinux || test "x$OPENJDK_TARGET_CPU" != xsparcv9; then >> >> NON_MINIMAL_FEATURES="$NON_MINIMAL_FEATURES jfr" >> >> fi >> >> This code section is ported from closed, and as you can see, it is missing some of the open platforms here. You might want to add your os / cpu here to exclude JFR from building by default (it controls the INCLUDE_JFR macro). Excluded INCLUDE_JFR will also take out the command-line options so one should not even be able to attempt to start the functionality. >> >> In addition, and a more dynamic way to exclude JFR from a build, is to use the JVM_FEATURE configure option: >> >> --with-jvm-features= -jfr feature_1 feature2 (note the minus sign) >> >> >> >> Still interested? >> >> There are two platform dependent areas that might require some work to get the full functionality of JFR supported on a specific platform. >> >> 1. CPU information >> >> "runtime/os_perf.hpp" includes a few interfaces that is used for uniformly pulling out (periodically, at intervals) OS / CPU specific information, for example: >> >> CPUInformationInterface (informational) >> >> Impl: >> >> cpu//VM_Version_Ext.cpp files has been added for this purpose (for some CPUs) >> >> CPUPerformanceInterface (counters) >> >> Impl: >> >> os//os_flavor.cpp which some have seen today as part of our mayhem. >> >> A good thing with the CPUPerformanceInterface related information is that JFR has "support" for FUNCTIONALITY_NOT_IMPLEMENTED, for example see [2]: >> >> The FUNCTIONALITY_NOT_IMPLEMENTED macro is defined in "runtime/os_perf.hpp" >> >> It is therefore possible "stub out" one area of platform dependent functionality (I think our tests are lenient about this too). >> >> One last thing about CPU Information - if JFR functionality be requested, say via command-line: >> >> -XX:StartFlightRecording >> >> Then the CPU Information interface implementation is expected to be successfully initialized for the VM to start. The notion of "successfully initialized" here is very basic indeed, the only thing needed is that: >> >> bool initialize () >> >> routines return "true" (since false now indicates something is wrong and the VM will not start (command-line) or JFR will do a transactional "rollback" if attempted dynamically) >> >> 2. Thread sampling >> >> You might already know that JFR share, or have similar, stack walking code as AsynchGetCallTrace / Forte. And just as Forte had its: >> >> bool pd_get_top_frame_for_signal_handler(frame* fr_addr, void* ucontext, bool isInJava); >> >> // usually implemented (or not) in os_cpu/thread_.cpp >> >> For example, see [3] >> >> JFR has introduced its own: >> >> bool pd_get_top_frame_for_profiling(frame* fr_addr, void* ucontext, bool isInJava); >> >> pretty much in the same location as the first top_frame_for_signal_handler. >> >> Why not use the same? We do things slightly different in JFR compared to Forte, we have a tighter collaboration between the suspender and the suspendee (we can also support non sigprof platforms, such as Windows). >> >> Tying back to the linked example for thread_linux_sparc.cpp above, this was the reason that linux-sparc was excluded from building by default if I remember correctly, there was not enough time to craft a profiling () method. It should be possible to look at some x86 platform to see what the relevant differences are as a basis for adding this support. >> >> The invocation of pd_get_top_frame_for_profiling(frame* fr_addr, void* ucontext, bool isInJava) comes from here, see [4] and [5] >> >> It is also possible to "stub out" this code; if pd_get_top_frame_for_profiling(frame* fr_addr, void* ucontext, bool isInJava) returns false, the sampling attempt is skipped. That is totally fine. >> >> There is some additional platform specific support that might need to be implemented in the context of thread sampling, for example "crash protection" >> >> please lookup: >> >> os::ThreadCrashProtection >> >> Now this has been in open for some time, so its status might be better than some of the other parts. >> >> Summary: >> >> If you are not interested in the JFR functionality, and / or want it excluded from the builds, please exclude your platform combo from the default build via hotspot.m4 [1] >> >> Platform specific sensitive areas for JFR are around CPU Information / CPU Performance counters as well as thread sampling (e.g. stack walking / traversal). >> >> The JFR system should be resilient to withstand running with only a subset of events / implementations in most parts, but the most critical will be the ones listed above. >> >> Hoping for a better day today >> >> Best regards for now >> >> Markus >> >> >> >> [1] http://hg.openjdk.java.net/jdk/jdk/file/9010b580d8a9/make/autoconf/hotspot.m4#l312 >> >> [2] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/os/bsd/os_perf_bsd.cpp#l98 >> >> [3] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/os_cpu/linux_sparc/thread_linux_sparc.cpp#l40 >> >> [4] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/share/jfr/periodic/sampling/jfrThreadSampler.cpp#l180 >> >> [5] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/share/jfr/periodic/sampling/jfrCallTrace.cpp#l98 >> >> >> >> >> >> >> >> From thomas.stuefe at gmail.com Thu May 17 07:12:32 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 17 May 2018 09:12:32 +0200 Subject: Flight Recorder and non-Oracle platforms In-Reply-To: <03DD48FB-E919-471E-A6A5-C7DC7D79E186@oracle.com> References: <03DD48FB-E919-471E-A6A5-C7DC7D79E186@oracle.com> Message-ID: Thanks Erik! On Thu, May 17, 2018 at 9:11 AM, Erik Gahlin wrote: > It?s may possible to connect using MC, but you will not be able to open a recording. > > The file format version was bumped. It?s not that it has changed, but events in JDK 11 are prefixed with ?jdk.? while events in Oracle JDK 10 were prefixed with ?com.oracle.jdk.? > > Flight Recorder comes with an API and it?s easy to pretty print a recording if you want to play around. There is an example in the JEP [1] > > Erik > > [1] http://openjdk.java.net/jeps/328 > > >> On 17 May 2018, at 08:22, Thomas St?fe wrote: >> >> Hi Markus, >> >> is the JFR in openjdk11 compatible with existing Mission Control >> binaries, e.g. from an oracle jdk 10? So, that we could port JFR and >> already play around with it without having to wait for porting MC? >> >> As far as I understand (not an expert), MC can be used remotely too, >> cross-platform? So if the answer is "yes", one should be able to use >> e.g. a windows MC to connect remotely to openjdk+JFR on a porting >> platform? >> >> Thanks, Thomas >> >> >> On Thu, May 17, 2018 at 3:13 AM, Markus Gronlund >> wrote: >>> Hello again, >>> >>> many thanks to all of you for your war efforts yesterday. >>> >>> I would like to pre-empt some about what you can expect once you start getting your platforms building again. >>> >>> The short "I just got my system building again, please don't have me run into more issues"-version: >>> >>> We currently have only one specific platform listed that is explicitly excluded out-of-the box for JFR (in addition to MINIMAL target), and that is linux-sparc. >>> >>> Please see [1] >>> >>> # Enable JFR by default, except on linux-sparcv9 and on minimal. >>> >>> if test "x$OPENJDK_TARGET_OS" != xlinux || test "x$OPENJDK_TARGET_CPU" != xsparcv9; then >>> >>> NON_MINIMAL_FEATURES="$NON_MINIMAL_FEATURES jfr" >>> >>> fi >>> >>> This code section is ported from closed, and as you can see, it is missing some of the open platforms here. You might want to add your os / cpu here to exclude JFR from building by default (it controls the INCLUDE_JFR macro). Excluded INCLUDE_JFR will also take out the command-line options so one should not even be able to attempt to start the functionality. >>> >>> In addition, and a more dynamic way to exclude JFR from a build, is to use the JVM_FEATURE configure option: >>> >>> --with-jvm-features= -jfr feature_1 feature2 (note the minus sign) >>> >>> >>> >>> Still interested? >>> >>> There are two platform dependent areas that might require some work to get the full functionality of JFR supported on a specific platform. >>> >>> 1. CPU information >>> >>> "runtime/os_perf.hpp" includes a few interfaces that is used for uniformly pulling out (periodically, at intervals) OS / CPU specific information, for example: >>> >>> CPUInformationInterface (informational) >>> >>> Impl: >>> >>> cpu//VM_Version_Ext.cpp files has been added for this purpose (for some CPUs) >>> >>> CPUPerformanceInterface (counters) >>> >>> Impl: >>> >>> os//os_flavor.cpp which some have seen today as part of our mayhem. >>> >>> A good thing with the CPUPerformanceInterface related information is that JFR has "support" for FUNCTIONALITY_NOT_IMPLEMENTED, for example see [2]: >>> >>> The FUNCTIONALITY_NOT_IMPLEMENTED macro is defined in "runtime/os_perf.hpp" >>> >>> It is therefore possible "stub out" one area of platform dependent functionality (I think our tests are lenient about this too). >>> >>> One last thing about CPU Information - if JFR functionality be requested, say via command-line: >>> >>> -XX:StartFlightRecording >>> >>> Then the CPU Information interface implementation is expected to be successfully initialized for the VM to start. The notion of "successfully initialized" here is very basic indeed, the only thing needed is that: >>> >>> bool initialize () >>> >>> routines return "true" (since false now indicates something is wrong and the VM will not start (command-line) or JFR will do a transactional "rollback" if attempted dynamically) >>> >>> 2. Thread sampling >>> >>> You might already know that JFR share, or have similar, stack walking code as AsynchGetCallTrace / Forte. And just as Forte had its: >>> >>> bool pd_get_top_frame_for_signal_handler(frame* fr_addr, void* ucontext, bool isInJava); >>> >>> // usually implemented (or not) in os_cpu/thread_.cpp >>> >>> For example, see [3] >>> >>> JFR has introduced its own: >>> >>> bool pd_get_top_frame_for_profiling(frame* fr_addr, void* ucontext, bool isInJava); >>> >>> pretty much in the same location as the first top_frame_for_signal_handler. >>> >>> Why not use the same? We do things slightly different in JFR compared to Forte, we have a tighter collaboration between the suspender and the suspendee (we can also support non sigprof platforms, such as Windows). >>> >>> Tying back to the linked example for thread_linux_sparc.cpp above, this was the reason that linux-sparc was excluded from building by default if I remember correctly, there was not enough time to craft a profiling () method. It should be possible to look at some x86 platform to see what the relevant differences are as a basis for adding this support. >>> >>> The invocation of pd_get_top_frame_for_profiling(frame* fr_addr, void* ucontext, bool isInJava) comes from here, see [4] and [5] >>> >>> It is also possible to "stub out" this code; if pd_get_top_frame_for_profiling(frame* fr_addr, void* ucontext, bool isInJava) returns false, the sampling attempt is skipped. That is totally fine. >>> >>> There is some additional platform specific support that might need to be implemented in the context of thread sampling, for example "crash protection" >>> >>> please lookup: >>> >>> os::ThreadCrashProtection >>> >>> Now this has been in open for some time, so its status might be better than some of the other parts. >>> >>> Summary: >>> >>> If you are not interested in the JFR functionality, and / or want it excluded from the builds, please exclude your platform combo from the default build via hotspot.m4 [1] >>> >>> Platform specific sensitive areas for JFR are around CPU Information / CPU Performance counters as well as thread sampling (e.g. stack walking / traversal). >>> >>> The JFR system should be resilient to withstand running with only a subset of events / implementations in most parts, but the most critical will be the ones listed above. >>> >>> Hoping for a better day today >>> >>> Best regards for now >>> >>> Markus >>> >>> >>> >>> [1] http://hg.openjdk.java.net/jdk/jdk/file/9010b580d8a9/make/autoconf/hotspot.m4#l312 >>> >>> [2] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/os/bsd/os_perf_bsd.cpp#l98 >>> >>> [3] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/os_cpu/linux_sparc/thread_linux_sparc.cpp#l40 >>> >>> [4] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/share/jfr/periodic/sampling/jfrThreadSampler.cpp#l180 >>> >>> [5] http://hg.openjdk.java.net/jdk/jdk/file/3595bd343b65/src/hotspot/share/jfr/periodic/sampling/jfrCallTrace.cpp#l98 >>> >>> >>> >>> >>> >>> >>> >>> > From stefan.johansson at oracle.com Thu May 17 07:13:03 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Thu, 17 May 2018 09:13:03 +0200 Subject: RFR: 8202813: Move vm_weak processing from SystemDictionary::do_unloading to WeakProcessor In-Reply-To: <4E15250A-9872-4195-892B-C1E0078A80FD@oracle.com> References: <10a99bf5-d3cc-9d5b-8a38-5ba1672f9f8b@oracle.com> <4E15250A-9872-4195-892B-C1E0078A80FD@oracle.com> Message-ID: <6dc7d3f8-dd9f-0990-860d-3b9423c1e271@oracle.com> On 2018-05-17 01:51, Kim Barrett wrote: >> On May 16, 2018, at 8:42 AM, Stefan Johansson wrote: >> >> Hi Kim, >> >> On 2018-05-08 21:34, Kim Barrett wrote: >>> Please review this change to move the clearing of dead entries in >>> vm_weak_oop_storage from SystemDictionary::do_unloading to preceeding >>> calls to WeakProcessor::weak_oops_do. >>> This involves conditionalizing WeakProcessor invocations, to allow >>> different calls to process different subsets of the native weak >>> references. For example, vm_weak_oop_storage is treated as strong >>> rather than weak when ClassUnloading is disabled. >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8202813 >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8202813/open.00/ >> Thanks for doing this work to unify weak root processing. > > Thanks for looking at it. > >> I might be missing the bigger picture, so before starting to commenting the code I have a couple of questions: >> 1. With this patch SystemDictionary::vm_weak_oop_storage is now processed both by the WeakProcessor in weak_oops_do and the SystemDictionary in oops_do() and roots_oops_do(). Shouldn't all processing be moved to the WeakProcessor to make it clear who is responsible for doing the processing? Or why do we want to treat do_unloading special? > > Yes, it should be moved completely. Baby steps. > > I started to change (remove, actually, and update callers) > SystemDictionary::oops_do, but discovered jfr was using it, and didn't > want to touch that in the middle of the jfr review. > > And there are some preliminary cleanups I'm working on around > roots_oops_do, but haven't finished them yet. > I agree that baby steps is often good, but now when JFR is in would it make sense to include more in this change? >> 2. For this patch I don't see the need for the PhaseSet class and the serial and parallel phases distinction. Couldn't we just pass in a bool to weak_oops_do saying if we should handle the vm_weak_oop_storage or not? I understand that for upcoming patches this might be needed, but then I think this should be added in that change. > > I agree it's over-engineered for the needs of this change set. > However, further conditionalization beyond a boolean for > class-unloading is going to be needed RSN for the new StringTable > (based on the new concurrent hash table + oopstorage). StringTable > processing is another place where the various GCs and their sub-parts > vary about where it's treated as strong or weak. That should be > showing up shortly, and is being worked on by folks who aren't me. So > unless you want me to separate that out into a separate change that > will likely be my next one to support that work... > I would like it to be added when needed, to make sure we only include the parts we really need. Right now it would be hard for me to review those parts since I can't verify that they work as intended. I'm mostly concerned about the parts around serial/parallel phases, which is currently unused. Thanks, Stefan From erik.helin at oracle.com Thu May 17 07:12:17 2018 From: erik.helin at oracle.com (Erik Helin) Date: Thu, 17 May 2018 09:12:17 +0200 Subject: RFR(L) : 8199370: [TESTBUG] Open source vm testbase GC tests In-Reply-To: <8D49F7BE-84F3-4E3D-B9DC-30233E005263@oracle.com> References: <8D49F7BE-84F3-4E3D-B9DC-30233E005263@oracle.com> Message-ID: <38097225-0570-5865-0177-b1f8c230733c@oracle.com> On 05/15/2018 08:58 PM, Igor Ignatyev wrote: > Hi Erik, > > please see my answers inline. > > can I consider this RFR as reviewed by you? Yep! > Thanks, > -- Igor > >> On May 14, 2018, at 4:23 AM, Erik Helin wrote: >> >> On 05/08/2018 12:35 AM, Igor Ignatyev wrote: >>> Hi all, >> >> Hi Igor, >> >> On 05/08/2018 12:35 AM, Igor Ignatyev wrote: >>> could you please review the patch which open sources GC tests from vm testbase? it introduces the following test groups: >>> - vmTestbase_vm_g1classunloading >>> - vmTestbase_vm_gc_compact >>> - vmTestbase_vm_gc_concurrent >>> - vmTestbase_vm_gc_container >>> - vmTestbase_vm_gc_juggle >>> - vmTestbase_vm_gc_locker >>> - vmTestbase_vm_gc_ref >>> - vmTestbase_vm_gc_misc >> >> This is a very welcome, and pretty massive, change :) I won't be able to read through each test, there are simple too many, but I can sample a few of them and have a look. >> >> On 05/08/2018 12:35 AM, Igor Ignatyev wrote: >>> As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. >> >> I'm also assuming that to help the open sourcing of these tests, most comments will likely be deferred until later? If so, that is fine with me. > y, unless it is something very important and cost of delaying changes is high, I'd prefer to defer all comments/improvements till later. Ok, normally I would like it to be cleaned up prior to pushing, but I'm also aware of the history of these tests. Right now it seems more valuable to push these tests so we can collaborate on cleaning them up in upstream. >> >> On 05/08/2018 12:35 AM, Igor Ignatyev wrote: >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8199370 >>> webrev: http://cr.openjdk.java.net/~iignatyev/8199370/webrev.00/index.html >> >> Many of the tests are a bit cryptic, but that can be refactored later. Could you please file bugs for the two tests written in shell? Particularly parOld/test.sh should be trivial to rewrite in Java. > sure, I've filed 8203239 and 8203238. Thanks! >> It seems like a lot of tests contains a TEST.properties file with the content `exclusiveAccess.dirs=.`. Could this become the default value somewhere, so we don't need all those TEST.properties files? > y, but this will require finding a most top directory whose all tests have this TEST.properties file and it will also make it harder to understand how a test is executed. there was/is ongoing discussing w/ Jon on moving 'exclusiveAccess' to test description. anyhow I've filed an RFE to clean that up -- 8203241. Ok, we can continue the discussion in 8203241. Thanks, Erik >> >> Thanks, >> Erik >> >>> Thanks, >>> -- Igor > From stefan.karlsson at oracle.com Thu May 17 07:23:03 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 17 May 2018 09:23:03 +0200 Subject: RFR: 8203339: Add oopDesc::field_offset() Message-ID: <080bc03c-19ff-c2d0-5a85-418aa76a42c8@oracle.com> Hi all, Please review this patch to add a oopDesc::field_offset() convenience function. http://cr.openjdk.java.net/~stefank/8203339/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8203339 We use it in ZGC to extract the offset to oop fields in some of our OopClosures. The OopClosures provide the interior pointers to oop fields, and together with the object pointer the offset can be calculated. See the following ZGC use-case: http://hg.openjdk.java.net/zgc/zgc/file/3a52c8361e20/src/hotspot/share/gc/z/zHeapIterator.cpp#l87 class ZHeapIteratorPushOopClosure : public ExtendedOopClosure { private: ZHeapIterator* const _iter; const oop _base; public: ZHeapIteratorPushOopClosure(ZHeapIterator* iter, oop base) : _iter(iter), _base(base) {} void do_oop_nv(oop* p) { const oop obj = HeapAccess::oop_load_at(_base, _base->field_offset(p)); _iter->push(obj); } Thanks, StefanK From stefan.karlsson at oracle.com Thu May 17 07:46:30 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 17 May 2018 09:46:30 +0200 Subject: 8203341: Add a safepoint-aware Semaphore Message-ID: <2f8412d5-2fe4-2c8f-077c-cf96e8f9bebf@oracle.com> Hi all, Please review this patch to add a safepoint-aware Semaphore (and refactor the semaphore usage in thread-local handshakes). http://cr.openjdk.java.net/~stefank/8203341/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8203341 Both ZGC and the thread-local handshakes need to move the JavaThread to a blocked state before waiting on a Semaphore. I propose that we introduce a new SemaphoreSafepointAware wrapper around Semaphore. There are two things to consider with this patch: 1) In ZGC, where this patch originates from, we set the JavaThread's osthread state with osthread->set_state(CONDVAR_WAIT) (using OSThreadWaitState). This means that we get the following printed when the threads are dumped: "waiting on condition ". I've kept this behavior for the SemaphoreSafepointAware class. This means that we now change the osthread state for JavaThreads blocking in HandshakeState::process_self_inner. 2) The thread-local handshakes uses trywait before entering wait: if (!_semaphore.trywait()) { - ThreadBlockInVM tbivm(thread); _semaphore.wait(); } At the time when SemaphoreSafepointAware was written, we didn't have trywait on all platforms, now we do. Maybe we should always do the trywait dance inside SemaphoreSafepointAware::wait() ? inline void SemaphoreSafepointAware::wait() { if (Thread::current()->is_Java_thread()) { if (!_semaphore.trywait) { JavaThread* thread = JavaThread::current(); // Prepare to block and allow safepoints while blocked ThreadBlockInVM tbivm(thread); OSThreadWaitState osts(thread->osthread(), false /* not in Object.wait() */); // Wait for value _semaphore.wait(); } Thanks, StefanK From rkennke at redhat.com Thu May 17 08:03:45 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 17 May 2018 10:03:45 +0200 Subject: RFR: 8203339: Add oopDesc::field_offset() In-Reply-To: <080bc03c-19ff-c2d0-5a85-418aa76a42c8@oracle.com> References: <080bc03c-19ff-c2d0-5a85-418aa76a42c8@oracle.com> Message-ID: > > Please review this patch to add a oopDesc::field_offset() convenience > function. > > http://cr.openjdk.java.net/~stefank/8203339/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8203339 > > We use it in ZGC to extract the offset to oop fields in some of our > OopClosures. The OopClosures provide the interior pointers to oop > fields, and together with the object pointer the offset can be calculated. > > See the following ZGC use-case: > > http://hg.openjdk.java.net/zgc/zgc/file/3a52c8361e20/src/hotspot/share/gc/z/zHeapIterator.cpp#l87 > > > class ZHeapIteratorPushOopClosure : public ExtendedOopClosure { > private: > ? ZHeapIterator* const _iter; > ? const oop??????????? _base; > > public: > ? ZHeapIteratorPushOopClosure(ZHeapIterator* iter, oop base) : > ????? _iter(iter), > ????? _base(base) {} > > ? void do_oop_nv(oop* p) { > ??? const oop obj = HeapAccess::oop_load_at(_base, > _base->field_offset(p)); > ??? _iter->push(obj); > ? } > Seems useful. Patch looks good to me. Roman From rene.schuenemann at gmail.com Thu May 17 08:10:09 2018 From: rene.schuenemann at gmail.com (=?UTF-8?B?UmVuw6kgU2Now7xuZW1hbm4=?=) Date: Thu, 17 May 2018 10:10:09 +0200 Subject: RFR 8203292: Print non-default flags to the hs_err file In-Reply-To: References: Message-ID: Thank you Gerard and David :-) On Wed, May 16, 2018 at 11:59 PM, David Holmes wrote: > Looks good Rene! :) > > Thanks, > David > > > On 17/05/2018 1:10 AM, Ren? Sch?nemann wrote: >> >> Hi, >> >> can I please get a review for the following change: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203292 >> Webrev: http://cr.openjdk.java.net/~simonis/webrevs/2018/8203292 >> >> With this change, all flags with a non-default value will be printed >> to the hs_error file. >> While the hs_err file already contains the full command line parameters, >> flags can also be changed by ergonomics or various serviceability tools. >> >> Thank you, >> Rene >> > From boris.ulasevich at bell-sw.com Thu May 17 08:13:05 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Thu, 17 May 2018 11:13:05 +0300 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: Message-ID: <53feed71-0ab9-7743-6034-3c95bb3b20e8@bell-sw.com> Hi David, I see three tests failing: > NullPointerException at TestInterfaceMethodSelection.doInvoke(TestInterfaceMethodSelection.java:191) > NullPointerException at TestInvoke.access_priv(TestInvoke.java:54) > InvocationTargetException at TestReflection.access_priv(TestReflection.java:61) I will send you test details in a separate mail. Boris On 17.05.2018 00:23, David Holmes wrote: > Hi Boris, > > Many thanks for looking at this and working through the ARM code. > > I will study your patch in detail but am concerned by the "passes most > of runtime/Nestmates tests Ok."! What tests are not passing? > > David > > On 17/05/2018 1:05 AM, Boris Ulasevich wrote: >> Hi David, >> >> Let us look to the change in templateTable_arm.cpp. I have several notes. >> >> 1. We have compilation error because check_klass_subtype call needs >> one more temporary register parameter. I think correct change is: >> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >> -> >> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); >> >> 2. R0_tmp holds mdp used for profilation several lines below, so we >> can't spoil it. I think correct change is: >> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); >> -> >> check_klass_subtype(Rklass, Rinterf, R1_tmp, R2_tmp, noreg, subtype); >> >> 3. We can't jump to Rindex. I believe it was supposed to jump to Rmethod: >> jump_from_interpreted(Rindex); >> -> >> jump_from_interpreted(Rmethod); >> >> 4. Please note than Rklass and Rflags reuses same register. Before the >> change their life ranges had no intersection. I would propose to use >> R2 for Rklass (same with Rrecv) and move load_klass call after >> invokevirtual_helper call to avoid life range intersecton. >> >> See my patch against original templateTable_arm.cpp version in attach >> - with this change ARM build compiles and passes most of >> runtime/Nestmates tests Ok. >> >> regards, >> Boris >> >> On 15.05.2018 03:52, David Holmes wrote: >>> This review is being spread across four groups: langtools, core-libs, >>> hotspot and serviceability. This is the specific review thread for >>> hotspot - webrev: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>> >>> See below for full details - including annotated full webrev guiding >>> the review. >>> >>> The intent is to have JEP-181 targeted and integrated by the end of >>> this month. >>> >>> Thanks, >>> David >>> ----- >>> >>> The nestmates project (JEP-181) introduces new classfile attributes >>> to identify classes and interfaces in the same nest, so that the VM >>> can perform access control based on those attributes and so allow >>> direct private access between nestmates without requiring javac to >>> generate synthetic accessor methods. These access control changes >>> also extend to core reflection and the MethodHandle.Lookup contexts. >>> >>> Direct private calls between nestmates requires a more general >>> calling context than is permitted by invokespecial, and so the JVMS >>> is updated to allow, and javac updated to use, invokevirtual and >>> invokeinterface for private class and interface method calls >>> respectively. These changed semantics also extend to MethodHandle >>> findXXX operations. >>> >>> At this time we are only concerned with static nest definitions, >>> which map to a top-level class/interface as the nest-host and all its >>> nested types as nest-members. >>> >>> Please see the JEP for further details. >>> >>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>> >>> All of the specification changes have been previously been worked out >>> by the Valhalla Project Expert Group, and the implementation reviewed >>> by the various contributors and discussed on the valhalla-dev mailing >>> list. >>> >>> Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, >>> Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, >>> Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan >>> >>> Master webrev of all changes: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>> >>> Annotated master webrev index: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>> >>> Performance: this is expected to be performance neutral in a general >>> sense. Benchmarking and performance runs are about to start. >>> >>> Testing Discussion: >>> ------------------ >>> >>> The testing for nestmates can be broken into four main groups: >>> >>> -? New tests specifically related to nestmates and currently in the >>> runtime/Nestmates directory >>> >>> - New tests to complement existing tests by adding in testcases not >>> previously expressible. >>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>> use of invokespecial for private interface methods and performing >>> receiver typechecks, so we add >>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>> invokeinterface. >>> >>> -? New JVM TI tests to verify the spec changes related to nest >>> attributes. >>> >>> -? Existing tests significantly affected by the nestmates changes, >>> primarily: >>> ??? -? runtime/SelectionResolution >>> >>> ??? In most cases the nestmate changes makes certain invocations that >>> were illegal, legal (e.g. not requiring invokespecial to invoke >>> private interface methods; allowing access to private members via >>> reflection/Methodhandles that were previously not allowed). >>> >>> - Existing tests incidentally affected by the nestmate changes >>> >>> ?? This includes tests of things utilising class >>> redefinition/retransformation to alter nested types but which >>> unintentionally alter nest relationships (which is not permitted). >>> >>> There are still a number of tests problem-listed with issues filed >>> against them to have them adapted to work with nestmates. Some of >>> these are intended to be addressed in the short-term, while some >>> (such as the runtime/SelectionResolution test changes) may not >>> eventuate. >>> >>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>> >>> There is also further test work still to be completed (the JNI and >>> JDI invocation tests): >>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>> which will continue in parallel with the main RFR. >>> >>> Pre-integration Testing: >>> ??- General: >>> ???? - Mach5: hs/jdk tier1,2 >>> ???? - Mach5: hs-nightly (tiers 1 -3) >>> ??- Targetted >>> ??? - nashorn (for asm changes) >>> ??? - hotspot: runtime/* >>> ?????????????? serviceability/* >>> ?????????????? compiler/* >>> ?????????????? vmTestbase/* >>> ??? - jdk: java/lang/invoke/* >>> ?????????? java/lang/reflect/* >>> ?????????? java/lang/instrument/* >>> ?????????? java/lang/Class/* >>> ?????????? java/lang/management/* >>> ?? - langtools: tools/javac >>> ??????????????? tools/javap >>> From stefan.karlsson at oracle.com Thu May 17 08:13:39 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 17 May 2018 10:13:39 +0200 Subject: RFR: 8203339: Add oopDesc::field_offset() In-Reply-To: References: <080bc03c-19ff-c2d0-5a85-418aa76a42c8@oracle.com> Message-ID: Thanks, Roman. StefanK On 2018-05-17 10:03, Roman Kennke wrote: > >> >> Please review this patch to add a oopDesc::field_offset() convenience >> function. >> >> http://cr.openjdk.java.net/~stefank/8203339/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8203339 >> >> We use it in ZGC to extract the offset to oop fields in some of our >> OopClosures. The OopClosures provide the interior pointers to oop >> fields, and together with the object pointer the offset can be calculated. >> >> See the following ZGC use-case: >> >> http://hg.openjdk.java.net/zgc/zgc/file/3a52c8361e20/src/hotspot/share/gc/z/zHeapIterator.cpp#l87 >> >> >> class ZHeapIteratorPushOopClosure : public ExtendedOopClosure { >> private: >> ? ZHeapIterator* const _iter; >> ? const oop??????????? _base; >> >> public: >> ? ZHeapIteratorPushOopClosure(ZHeapIterator* iter, oop base) : >> ????? _iter(iter), >> ????? _base(base) {} >> >> ? void do_oop_nv(oop* p) { >> ??? const oop obj = HeapAccess::oop_load_at(_base, >> _base->field_offset(p)); >> ??? _iter->push(obj); >> ? } >> > > Seems useful. Patch looks good to me. > > Roman > From erik.osterlund at oracle.com Thu May 17 08:19:49 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 17 May 2018 10:19:49 +0200 Subject: RFR: 8203339: Add oopDesc::field_offset() In-Reply-To: <080bc03c-19ff-c2d0-5a85-418aa76a42c8@oracle.com> References: <080bc03c-19ff-c2d0-5a85-418aa76a42c8@oracle.com> Message-ID: <5AFD3B25.9030409@oracle.com> Hi Stefan, Looks good. Thanks, /Erik On 2018-05-17 09:23, Stefan Karlsson wrote: > Hi all, > > Please review this patch to add a oopDesc::field_offset() convenience > function. > > http://cr.openjdk.java.net/~stefank/8203339/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8203339 > > We use it in ZGC to extract the offset to oop fields in some of our > OopClosures. The OopClosures provide the interior pointers to oop > fields, and together with the object pointer the offset can be > calculated. > > See the following ZGC use-case: > > http://hg.openjdk.java.net/zgc/zgc/file/3a52c8361e20/src/hotspot/share/gc/z/zHeapIterator.cpp#l87 > > > class ZHeapIteratorPushOopClosure : public ExtendedOopClosure { > private: > ZHeapIterator* const _iter; > const oop _base; > > public: > ZHeapIteratorPushOopClosure(ZHeapIterator* iter, oop base) : > _iter(iter), > _base(base) {} > > void do_oop_nv(oop* p) { > const oop obj = HeapAccess::oop_load_at(_base, > _base->field_offset(p)); > _iter->push(obj); > } > > Thanks, > StefanK From stefan.karlsson at oracle.com Thu May 17 08:20:19 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 17 May 2018 10:20:19 +0200 Subject: RFR: 8203339: Add oopDesc::field_offset() In-Reply-To: <5AFD3B25.9030409@oracle.com> References: <080bc03c-19ff-c2d0-5a85-418aa76a42c8@oracle.com> <5AFD3B25.9030409@oracle.com> Message-ID: Thanks, Erik. StefanK On 2018-05-17 10:19, Erik ?sterlund wrote: > Hi Stefan, > > Looks good. > > Thanks, > /Erik > > On 2018-05-17 09:23, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to add a oopDesc::field_offset() convenience >> function. >> >> http://cr.openjdk.java.net/~stefank/8203339/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8203339 >> >> We use it in ZGC to extract the offset to oop fields in some of our >> OopClosures. The OopClosures provide the interior pointers to oop >> fields, and together with the object pointer the offset can be >> calculated. >> >> See the following ZGC use-case: >> >> http://hg.openjdk.java.net/zgc/zgc/file/3a52c8361e20/src/hotspot/share/gc/z/zHeapIterator.cpp#l87 >> >> >> class ZHeapIteratorPushOopClosure : public ExtendedOopClosure { >> private: >> ? ZHeapIterator* const _iter; >> ? const oop??????????? _base; >> >> public: >> ? ZHeapIteratorPushOopClosure(ZHeapIterator* iter, oop base) : >> ????? _iter(iter), >> ????? _base(base) {} >> >> ? void do_oop_nv(oop* p) { >> ??? const oop obj = HeapAccess::oop_load_at(_base, >> _base->field_offset(p)); >> ??? _iter->push(obj); >> ? } >> >> Thanks, >> StefanK > From david.holmes at oracle.com Thu May 17 08:24:51 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 17 May 2018 18:24:51 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <53feed71-0ab9-7743-6034-3c95bb3b20e8@bell-sw.com> References: <53feed71-0ab9-7743-6034-3c95bb3b20e8@bell-sw.com> Message-ID: On 17/05/2018 6:13 PM, Boris Ulasevich wrote: > Hi David, > > I see three tests failing: > > NullPointerException at > TestInterfaceMethodSelection.doInvoke(TestInterfaceMethodSelection.java:191) > > > NullPointerException at TestInvoke.access_priv(TestInvoke.java:54) > > InvocationTargetException at > TestReflection.access_priv(TestReflection.java:61) > > I will send you test details in a separate mail. Ok. This indicates a bug in the assembly code. The NPE's will likely be SEGVs caused by a zero register. Thanks, David > Boris > > On 17.05.2018 00:23, David Holmes wrote: >> Hi Boris, >> >> Many thanks for looking at this and working through the ARM code. >> >> I will study your patch in detail but am concerned by the "passes most >> of runtime/Nestmates tests Ok."! What tests are not passing? >> >> David >> >> On 17/05/2018 1:05 AM, Boris Ulasevich wrote: >>> Hi David, >>> >>> Let us look to the change in templateTable_arm.cpp. I have several >>> notes. >>> >>> 1. We have compilation error because check_klass_subtype call needs >>> one more temporary register parameter. I think correct change is: >>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >>> -> >>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); >>> >>> 2. R0_tmp holds mdp used for profilation several lines below, so we >>> can't spoil it. I think correct change is: >>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); >>> -> >>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R2_tmp, noreg, subtype); >>> >>> 3. We can't jump to Rindex. I believe it was supposed to jump to >>> Rmethod: >>> jump_from_interpreted(Rindex); >>> -> >>> jump_from_interpreted(Rmethod); >>> >>> 4. Please note than Rklass and Rflags reuses same register. Before >>> the change their life ranges had no intersection. I would propose to >>> use R2 for Rklass (same with Rrecv) and move load_klass call after >>> invokevirtual_helper call to avoid life range intersecton. >>> >>> See my patch against original templateTable_arm.cpp version in attach >>> - with this change ARM build compiles and passes most of >>> runtime/Nestmates tests Ok. >>> >>> regards, >>> Boris >>> >>> On 15.05.2018 03:52, David Holmes wrote: >>>> This review is being spread across four groups: langtools, >>>> core-libs, hotspot and serviceability. This is the specific review >>>> thread for hotspot - webrev: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>> >>>> See below for full details - including annotated full webrev guiding >>>> the review. >>>> >>>> The intent is to have JEP-181 targeted and integrated by the end of >>>> this month. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> The nestmates project (JEP-181) introduces new classfile attributes >>>> to identify classes and interfaces in the same nest, so that the VM >>>> can perform access control based on those attributes and so allow >>>> direct private access between nestmates without requiring javac to >>>> generate synthetic accessor methods. These access control changes >>>> also extend to core reflection and the MethodHandle.Lookup contexts. >>>> >>>> Direct private calls between nestmates requires a more general >>>> calling context than is permitted by invokespecial, and so the JVMS >>>> is updated to allow, and javac updated to use, invokevirtual and >>>> invokeinterface for private class and interface method calls >>>> respectively. These changed semantics also extend to MethodHandle >>>> findXXX operations. >>>> >>>> At this time we are only concerned with static nest definitions, >>>> which map to a top-level class/interface as the nest-host and all >>>> its nested types as nest-members. >>>> >>>> Please see the JEP for further details. >>>> >>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>> >>>> All of the specification changes have been previously been worked >>>> out by the Valhalla Project Expert Group, and the implementation >>>> reviewed by the various contributors and discussed on the >>>> valhalla-dev mailing list. >>>> >>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen >>>> Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, >>>> Kumar Srinivasan >>>> >>>> Master webrev of all changes: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>> >>>> Annotated master webrev index: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>> >>>> Performance: this is expected to be performance neutral in a general >>>> sense. Benchmarking and performance runs are about to start. >>>> >>>> Testing Discussion: >>>> ------------------ >>>> >>>> The testing for nestmates can be broken into four main groups: >>>> >>>> -? New tests specifically related to nestmates and currently in the >>>> runtime/Nestmates directory >>>> >>>> - New tests to complement existing tests by adding in testcases not >>>> previously expressible. >>>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>>> use of invokespecial for private interface methods and performing >>>> receiver typechecks, so we add >>>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>>> invokeinterface. >>>> >>>> -? New JVM TI tests to verify the spec changes related to nest >>>> attributes. >>>> >>>> -? Existing tests significantly affected by the nestmates changes, >>>> primarily: >>>> ??? -? runtime/SelectionResolution >>>> >>>> ??? In most cases the nestmate changes makes certain invocations >>>> that were illegal, legal (e.g. not requiring invokespecial to invoke >>>> private interface methods; allowing access to private members via >>>> reflection/Methodhandles that were previously not allowed). >>>> >>>> - Existing tests incidentally affected by the nestmate changes >>>> >>>> ?? This includes tests of things utilising class >>>> redefinition/retransformation to alter nested types but which >>>> unintentionally alter nest relationships (which is not permitted). >>>> >>>> There are still a number of tests problem-listed with issues filed >>>> against them to have them adapted to work with nestmates. Some of >>>> these are intended to be addressed in the short-term, while some >>>> (such as the runtime/SelectionResolution test changes) may not >>>> eventuate. >>>> >>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>> >>>> There is also further test work still to be completed (the JNI and >>>> JDI invocation tests): >>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>> which will continue in parallel with the main RFR. >>>> >>>> Pre-integration Testing: >>>> ??- General: >>>> ???? - Mach5: hs/jdk tier1,2 >>>> ???? - Mach5: hs-nightly (tiers 1 -3) >>>> ??- Targetted >>>> ??? - nashorn (for asm changes) >>>> ??? - hotspot: runtime/* >>>> ?????????????? serviceability/* >>>> ?????????????? compiler/* >>>> ?????????????? vmTestbase/* >>>> ??? - jdk: java/lang/invoke/* >>>> ?????????? java/lang/reflect/* >>>> ?????????? java/lang/instrument/* >>>> ?????????? java/lang/Class/* >>>> ?????????? java/lang/management/* >>>> ?? - langtools: tools/javac >>>> ??????????????? tools/javap >>>> From boris.ulasevich at bell-sw.com Thu May 17 09:23:53 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Thu, 17 May 2018 12:23:53 +0300 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <53feed71-0ab9-7743-6034-3c95bb3b20e8@bell-sw.com> Message-ID: Hi David, You are right! My bad, R2_tmp parameter conflicts with Rklass on check_klass_subtype(..) call. See correct patch in attach. Now all runtime/Nestmates tests passed! :) regards, Boris On 17.05.2018 11:24, David Holmes wrote: > On 17/05/2018 6:13 PM, Boris Ulasevich wrote: >> Hi David, >> >> I see three tests failing: >> ?> NullPointerException at >> TestInterfaceMethodSelection.doInvoke(TestInterfaceMethodSelection.java:191) >> >> ?> NullPointerException at TestInvoke.access_priv(TestInvoke.java:54) >> ?> InvocationTargetException at >> TestReflection.access_priv(TestReflection.java:61) >> >> I will send you test details in a separate mail. > > Ok. This indicates a bug in the assembly code. The NPE's will likely be > SEGVs caused by a zero register. > > Thanks, > David > > >> Boris >> >> On 17.05.2018 00:23, David Holmes wrote: >>> Hi Boris, >>> >>> Many thanks for looking at this and working through the ARM code. >>> >>> I will study your patch in detail but am concerned by the "passes >>> most of runtime/Nestmates tests Ok."! What tests are not passing? >>> >>> David >>> >>> On 17/05/2018 1:05 AM, Boris Ulasevich wrote: >>>> Hi David, >>>> >>>> Let us look to the change in templateTable_arm.cpp. I have several >>>> notes. >>>> >>>> 1. We have compilation error because check_klass_subtype call needs >>>> one more temporary register parameter. I think correct change is: >>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >>>> -> >>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); >>>> >>>> 2. R0_tmp holds mdp used for profilation several lines below, so we >>>> can't spoil it. I think correct change is: >>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); >>>> -> >>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R2_tmp, noreg, subtype); >>>> >>>> 3. We can't jump to Rindex. I believe it was supposed to jump to >>>> Rmethod: >>>> jump_from_interpreted(Rindex); >>>> -> >>>> jump_from_interpreted(Rmethod); >>>> >>>> 4. Please note than Rklass and Rflags reuses same register. Before >>>> the change their life ranges had no intersection. I would propose to >>>> use R2 for Rklass (same with Rrecv) and move load_klass call after >>>> invokevirtual_helper call to avoid life range intersecton. >>>> >>>> See my patch against original templateTable_arm.cpp version in >>>> attach - with this change ARM build compiles and passes most of >>>> runtime/Nestmates tests Ok. >>>> >>>> regards, >>>> Boris >>>> >>>> On 15.05.2018 03:52, David Holmes wrote: >>>>> This review is being spread across four groups: langtools, >>>>> core-libs, hotspot and serviceability. This is the specific review >>>>> thread for hotspot - webrev: >>>>> >>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>>> >>>>> See below for full details - including annotated full webrev >>>>> guiding the review. >>>>> >>>>> The intent is to have JEP-181 targeted and integrated by the end of >>>>> this month. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>> The nestmates project (JEP-181) introduces new classfile attributes >>>>> to identify classes and interfaces in the same nest, so that the VM >>>>> can perform access control based on those attributes and so allow >>>>> direct private access between nestmates without requiring javac to >>>>> generate synthetic accessor methods. These access control changes >>>>> also extend to core reflection and the MethodHandle.Lookup contexts. >>>>> >>>>> Direct private calls between nestmates requires a more general >>>>> calling context than is permitted by invokespecial, and so the JVMS >>>>> is updated to allow, and javac updated to use, invokevirtual and >>>>> invokeinterface for private class and interface method calls >>>>> respectively. These changed semantics also extend to MethodHandle >>>>> findXXX operations. >>>>> >>>>> At this time we are only concerned with static nest definitions, >>>>> which map to a top-level class/interface as the nest-host and all >>>>> its nested types as nest-members. >>>>> >>>>> Please see the JEP for further details. >>>>> >>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>>> >>>>> All of the specification changes have been previously been worked >>>>> out by the Valhalla Project Expert Group, and the implementation >>>>> reviewed by the various contributors and discussed on the >>>>> valhalla-dev mailing list. >>>>> >>>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen >>>>> Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, >>>>> Kumar Srinivasan >>>>> >>>>> Master webrev of all changes: >>>>> >>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>>> >>>>> Annotated master webrev index: >>>>> >>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>>> >>>>> Performance: this is expected to be performance neutral in a >>>>> general sense. Benchmarking and performance runs are about to start. >>>>> >>>>> Testing Discussion: >>>>> ------------------ >>>>> >>>>> The testing for nestmates can be broken into four main groups: >>>>> >>>>> -? New tests specifically related to nestmates and currently in the >>>>> runtime/Nestmates directory >>>>> >>>>> - New tests to complement existing tests by adding in testcases not >>>>> previously expressible. >>>>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>>>> use of invokespecial for private interface methods and performing >>>>> receiver typechecks, so we add >>>>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>>>> invokeinterface. >>>>> >>>>> -? New JVM TI tests to verify the spec changes related to nest >>>>> attributes. >>>>> >>>>> -? Existing tests significantly affected by the nestmates changes, >>>>> primarily: >>>>> ??? -? runtime/SelectionResolution >>>>> >>>>> ??? In most cases the nestmate changes makes certain invocations >>>>> that were illegal, legal (e.g. not requiring invokespecial to >>>>> invoke private interface methods; allowing access to private >>>>> members via reflection/Methodhandles that were previously not >>>>> allowed). >>>>> >>>>> - Existing tests incidentally affected by the nestmate changes >>>>> >>>>> ?? This includes tests of things utilising class >>>>> redefinition/retransformation to alter nested types but which >>>>> unintentionally alter nest relationships (which is not permitted). >>>>> >>>>> There are still a number of tests problem-listed with issues filed >>>>> against them to have them adapted to work with nestmates. Some of >>>>> these are intended to be addressed in the short-term, while some >>>>> (such as the runtime/SelectionResolution test changes) may not >>>>> eventuate. >>>>> >>>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>>> >>>>> There is also further test work still to be completed (the JNI and >>>>> JDI invocation tests): >>>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>>> which will continue in parallel with the main RFR. >>>>> >>>>> Pre-integration Testing: >>>>> ??- General: >>>>> ???? - Mach5: hs/jdk tier1,2 >>>>> ???? - Mach5: hs-nightly (tiers 1 -3) >>>>> ??- Targetted >>>>> ??? - nashorn (for asm changes) >>>>> ??? - hotspot: runtime/* >>>>> ?????????????? serviceability/* >>>>> ?????????????? compiler/* >>>>> ?????????????? vmTestbase/* >>>>> ??? - jdk: java/lang/invoke/* >>>>> ?????????? java/lang/reflect/* >>>>> ?????????? java/lang/instrument/* >>>>> ?????????? java/lang/Class/* >>>>> ?????????? java/lang/management/* >>>>> ?? - langtools: tools/javac >>>>> ??????????????? tools/javap >>>>> -------------- next part -------------- diff -r 3d98842c8677 src/hotspot/cpu/arm/templateTable_arm.cpp --- a/src/hotspot/cpu/arm/templateTable_arm.cpp Tue May 15 05:33:26 2018 -0400 +++ b/src/hotspot/cpu/arm/templateTable_arm.cpp Thu May 17 12:03:52 2018 +0300 @@ -4276,25 +4276,42 @@ const Register Rinterf = R5_tmp; const Register Rindex = R4_tmp; const Register Rflags = R3_tmp; - const Register Rklass = R3_tmp; + const Register Rklass = R2_tmp; // Note! Same register with Rrecv prepare_invoke(byte_no, Rinterf, Rmethod, Rrecv, Rflags); + // First check for Object case, then private interface method, + // then regular interface method. + // Special case of invokeinterface called for virtual method of - // java.lang.Object. See cpCacheOop.cpp for details. - // This code isn't produced by javac, but could be produced by - // another compliant java compiler. - Label notMethod; - __ tbz(Rflags, ConstantPoolCacheEntry::is_forced_virtual_shift, notMethod); - + // java.lang.Object. See cpCache.cpp for details. + Label notObjectMethod; + __ tbz(Rflags, ConstantPoolCacheEntry::is_forced_virtual_shift, notObjectMethod); invokevirtual_helper(Rmethod, Rrecv, Rflags); - __ bind(notMethod); + __ bind(notObjectMethod); // Get receiver klass into Rklass - also a null check __ load_klass(Rklass, Rrecv); + // Check for private method invocation - indicated by vfinal Label no_such_interface; + Label notVFinal; + __ tbz(Rflags, ConstantPoolCacheEntry::is_vfinal_shift, notVFinal); + + Label subtype; + __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R3_tmp, noreg, subtype); + + // If we get here the typecheck failed + __ b(no_such_interface); + __ bind(subtype); + + // do the call - the index is actually the method to call + __ profile_final_call(R0_tmp); + __ jump_from_interpreted(Rmethod); + + __ bind(notVFinal); + // Receiver subtype check against REFC. __ lookup_interface_method(// inputs: rec. class, interface Rklass, Rinterf, noreg, From martin.doerr at sap.com Thu May 17 09:28:34 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 17 May 2018 09:28:34 +0000 Subject: RFR(XL): 8199712: Flight Recorder In-Reply-To: <5AFADCB9.4060904@oracle.com> References: <5AE0612F.8060200@oracle.com> <5AF99F08.6060408@oracle.com> <5AFADCB9.4060904@oracle.com> Message-ID: <2963dda67711466b9abbe767a145de2a@sap.com> Hi Erik, we get link errors on Windows 32 bit because of inconsistent signature of "jfr_add_string_constant". I've created a bug: https://bugs.openjdk.java.net/browse/JDK-8203346 Please take a look. Thanks and best regards, Martin -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Erik Gahlin Sent: Dienstag, 15. Mai 2018 15:12 To: Erik Joelsson ; hotspot-dev Source Developers Cc: hotspot-jfr-dev at openjdk.java.net; build-dev at openjdk.java.net Subject: Re: RFR(XL): 8199712: Flight Recorder On 2018-05-14 18:05, Erik Joelsson wrote: > Oh, I missed the new makefiles last time I looked at this. > > in Copy-jdk.jfr.gmk, everything looks like it's indented an extra 4 > steps. I'm assuming this is because it used to be conditional in the > previous closed file. Will fix! > > GensrcJfr.gmk, line 94, please move )) to the left. > Will fix! > Looking closer at GensrcJfr.gmk, The macro SetupJfrGeneration looks > like it is only called once. This could be greatly simplified by just > taking the body of the macro and inlining all the inputs. This of > course unless you see a need in the future to generate additional > files using the jfr tool. Will fix this in a follow up bug: https://bugs.openjdk.java.net/browse/JDK-8203221 Thanks Erik > > /Erik > > On 2018-05-14 07:36, Erik Gahlin wrote: >> Here is an updated webrev: >> >> http://cr.openjdk.java.net/~egahlin/8199712.1/ [1] >> >> that incorporates: >> >> - build changes >> - new event prefix, i.e. "com.oracle.jdk.CPULoad" becomes "jdk.CPULoad" >> - obsolete command line options EnableTracing and UseLockedTracing >> - fixed typos in the Javadoc >> - simplified #include files >> >> RFEs have been filed for other issues, CSR is approved and tests pass. >> >> Erik and Markus >> >> [1] Parent: >> >> changeset: 50092:0e42d3120e51 >> >> user: clanger >> date: Sat May 12 10:26:42 2018 +0200 >> summary: 8202915: [JAXP] Performance enhancements and cleanups in >> com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator >> >> >>> Greetings, >>> >>> Could I have a review of 8199712: Flight Recorder >>> >>> As mentioned in the preview [1] the tracing backend has been >>> removed. Event metadata has been consolidated into a single XML file >>> and event classes are now generated by GenerateJfrFiles.java. >>> >>> Tests have been run on Linux-x64, Windows-x64 and MaxOSX-x64. >>> >>> For details about the feature, see the JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8193393 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~egahlin/8199712.0/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8199712 >>> >>> [1] >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-April/031359.html >>> >>> Thanks >>> Erik and Markus >> > From erik.osterlund at oracle.com Thu May 17 09:48:11 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 17 May 2018 11:48:11 +0200 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: References: <5cdb2496-bad5-fa4a-f98e-37ae6a853f6b@oracle.com> <56bbfb13-3527-669a-b155-d699be1ef42e@oracle.com> <3c1d88fd-a3bc-350a-15fb-ec33118b7626@oracle.com> Message-ID: <5AFD4FDB.9090509@oracle.com> Hi JC, I have looked through your changes. In threadLocalAllocBuffer.cpp: 349 HeapWord* ThreadLocalAllocBuffer::allocate_sampled_object(size_t size) { 350 Thread* thread = myThread(); 351 thread->tlab().set_back_allocation_end(); 352 HeapWord* result = thread->tlab().allocate(size); 353 354 if (result) { 355 thread->heap_sampler().check_for_sampling(result, size * HeapWordSize, _bytes_since_last_sample_point); 356 thread->tlab().set_sample_end(); 357 } 358 359 return result; 360 } ...it looks like you are fetching myThread() from the TLAB, but use this fetched thread only to fetch the TLAB. But That TLAB is in fact "this" in this context. I would prefer not calling x() instead of thread->tlab()->x(). I don't need another webrev for this. The rest looks good to me. Thank you for doing this, and thank you for breaking out the per-thread logic into its own class. Thanks, /Erik On 2018-05-15 18:57, JC Beyler wrote: > Hi Thomas, > > Thanks for the review! I've done your two fixes for the next webrev. > I'll perhaps wait for the next review to combine webrev updates? Your > two change requests were minimal it might make sense to wait before > re-generating a new webrev. > > Anybody else have any comments/suggestions? > > Here are the latest links: > Webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.20/ > > JEP-331 Issue: https://bugs.openjdk.java.net/browse/JDK-8171119 > CSR: https://bugs.openjdk.java.net/browse/JDK-8194905 > Test Plan: https://bugs.openjdk.java.net/browse/JDK-8202566 > > Thanks again all! > Jc > > On Tue, May 15, 2018 at 12:23 AM Thomas Schatzl > > wrote: > > Hi, > > On Mon, 2018-05-14 at 13:02 -0700, JC Beyler wrote: > > Hi Robbin and all, > > > > Thank you for your continuous help! > > > > Done then! Here is the new webrev: > > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.20/ > > > > > and the incremental is: > > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.19_20/ > > > > > Thanks again all! > > Jc > > > > looks good to me. > > Two minor issues that you might want to fix before pushing: > > - collectedHeap.hpp: The declaration of > CollectedHeap::allocate_memory() should be private. > > - collectedHeap.inline.hpp: in CollectedHeap::allocate_memory() there > is this completely duplicated code which you might factor out into > another (private) method: > > if (init_memory) { > obj = ... > } else { > obj = ... > } > post_setup(klass, ...); > > Thanks, > Thomas > From martin.doerr at sap.com Thu May 17 10:10:41 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 17 May 2018 10:10:41 +0000 Subject: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) In-Reply-To: <117e46dc-5e00-cb47-2de0-2451cef2e6b4@redhat.com> References: <2252a635cba543a8b61dafe28b199327@sap.com> <48176edc-b498-3877-8bbc-ad24b064b487@redhat.com> <93c1a964c7ee41d4972ef78b6958a5b3@sap.com> <5f213774-94d7-e32e-c59c-8ea81fb402a8@redhat.com> <38311dc4306147b8ba303ab447b4f61e@sap.com> <117e46dc-5e00-cb47-2de0-2451cef2e6b4@redhat.com> Message-ID: Hi Aleksey, PPC and s390 can perform unaligned accesses. I think we could also use UseUnalignedAccesses for this function. But I have changed the code to tread PPC and s390 like the Intel/AMD platforms. I have also fixed the AIX build. Please review: http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.01/ Best regards, Martin -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: Mittwoch, 16. Mai 2018 18:43 To: Doerr, Martin ; hotspot-dev developers (hotspot-dev at openjdk.java.net) Cc: Thomas St?fe ; John Paul Adrian Glaubitz Subject: Re: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) On 05/16/2018 06:29 PM, Doerr, Martin wrote: > I have used --disable-warnings-as-errors as workaround. Maybe I can fix the warning tomorrow, too. That warning is fixable on both PPC64 and S390 with "just": diff -r 708db0a05b27 src/hotspot/share/jfr/utilities/jfrBigEndian.hpp --- a/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp Wed May 16 16:34:52 2018 +0200 +++ b/src/hotspot/share/jfr/utilities/jfrBigEndian.hpp Wed May 16 18:42:53 2018 +0200 @@ -102,7 +102,7 @@ inline bool JfrBigEndian::platform_supports_unaligned_reads(void) { #if defined(IA32) || defined(AMD64) return true; -#elif defined(SPARC) || defined(ARM) || defined(AARCH64) +#elif defined(SPARC) || defined(ARM) || defined(AARCH64) || defined(PPC64) || defined(S390) return false; #else #warning "Unconfigured platform" ...which returns the same "false" as the default/warned path, and it fixes normal builds. Thanks, -Aleksey From shade at redhat.com Thu May 17 10:35:33 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 17 May 2018 12:35:33 +0200 Subject: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) In-Reply-To: References: <2252a635cba543a8b61dafe28b199327@sap.com> <48176edc-b498-3877-8bbc-ad24b064b487@redhat.com> <93c1a964c7ee41d4972ef78b6958a5b3@sap.com> <5f213774-94d7-e32e-c59c-8ea81fb402a8@redhat.com> <38311dc4306147b8ba303ab447b4f61e@sap.com> <117e46dc-5e00-cb47-2de0-2451cef2e6b4@redhat.com> Message-ID: <9d8f5203-8cb8-56d1-8163-b5d8c599eca3@redhat.com> On 05/17/2018 12:10 PM, Doerr, Martin wrote: > Please review: > http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.01/ *) Stray comment in os_aix.cpp, remove before pushing? 1392 // print_libversion_info(st); Otherwise looks good. I rechecked ppc64el and s390x build fine with this patch. -Aleksey From markus.gronlund at oracle.com Thu May 17 10:40:59 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Thu, 17 May 2018 03:40:59 -0700 (PDT) Subject: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant Message-ID: Greetings, Please see this tiny fix for the following bug: https://bugs.openjdk.java.net/browse/JDK-8203346 Change: # HG changeset patch # User mgronlun # Date 1526553151 -7200 # Thu May 17 12:32:31 2018 +0200 # Node ID 9ac9ae21cc4c872ab40bb2c171dda3a921aa25fb # Parent 8e4fcfb4cfe48e6b481c6aa1751d916014a826af 8203346: JFR: Inconsistent signature of jfr_add_string_constant Reviewed-by: diff --git a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp --- a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp +++ b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp @@ -117,7 +117,7 @@ jlong JNICALL jfr_get_epoch_address(JNIEnv* env, jobject jvm); -jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jlong gen, jlong id, jstring string); +jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jboolean epoch, jlong id, jstring string); void JNICALL jfr_uncaught_exception(JNIEnv* env, jobject jvm, jobject thread, jthrowable throwable); Thanks to Martin for reporting, sorry for the inconvenience Markus From david.holmes at oracle.com Thu May 17 10:50:18 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 17 May 2018 20:50:18 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <53feed71-0ab9-7743-6034-3c95bb3b20e8@bell-sw.com> Message-ID: On 17/05/2018 7:23 PM, Boris Ulasevich wrote: > Hi David, > > You are right! My bad, R2_tmp parameter conflicts with Rklass on > check_klass_subtype(..) call. See correct patch in attach. Now all > runtime/Nestmates tests passed! :) Excellent! That's great news. Thanks Boris. I'll get the updated patch applied as part of the RFR. David > regards, > Boris > > On 17.05.2018 11:24, David Holmes wrote: >> On 17/05/2018 6:13 PM, Boris Ulasevich wrote: >>> Hi David, >>> >>> I see three tests failing: >>> ?> NullPointerException at >>> TestInterfaceMethodSelection.doInvoke(TestInterfaceMethodSelection.java:191) >>> >>> ?> NullPointerException at TestInvoke.access_priv(TestInvoke.java:54) >>> ?> InvocationTargetException at >>> TestReflection.access_priv(TestReflection.java:61) >>> >>> I will send you test details in a separate mail. >> >> Ok. This indicates a bug in the assembly code. The NPE's will likely >> be SEGVs caused by a zero register. >> >> Thanks, >> David >> >> >>> Boris >>> >>> On 17.05.2018 00:23, David Holmes wrote: >>>> Hi Boris, >>>> >>>> Many thanks for looking at this and working through the ARM code. >>>> >>>> I will study your patch in detail but am concerned by the "passes >>>> most of runtime/Nestmates tests Ok."! What tests are not passing? >>>> >>>> David >>>> >>>> On 17/05/2018 1:05 AM, Boris Ulasevich wrote: >>>>> Hi David, >>>>> >>>>> Let us look to the change in templateTable_arm.cpp. I have several >>>>> notes. >>>>> >>>>> 1. We have compilation error because check_klass_subtype call needs >>>>> one more temporary register parameter. I think correct change is: >>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >>>>> -> >>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); >>>>> >>>>> 2. R0_tmp holds mdp used for profilation several lines below, so we >>>>> can't spoil it. I think correct change is: >>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); >>>>> -> >>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R2_tmp, noreg, subtype); >>>>> >>>>> 3. We can't jump to Rindex. I believe it was supposed to jump to >>>>> Rmethod: >>>>> jump_from_interpreted(Rindex); >>>>> -> >>>>> jump_from_interpreted(Rmethod); >>>>> >>>>> 4. Please note than Rklass and Rflags reuses same register. Before >>>>> the change their life ranges had no intersection. I would propose >>>>> to use R2 for Rklass (same with Rrecv) and move load_klass call >>>>> after invokevirtual_helper call to avoid life range intersecton. >>>>> >>>>> See my patch against original templateTable_arm.cpp version in >>>>> attach - with this change ARM build compiles and passes most of >>>>> runtime/Nestmates tests Ok. >>>>> >>>>> regards, >>>>> Boris >>>>> >>>>> On 15.05.2018 03:52, David Holmes wrote: >>>>>> This review is being spread across four groups: langtools, >>>>>> core-libs, hotspot and serviceability. This is the specific review >>>>>> thread for hotspot - webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>>>> >>>>>> See below for full details - including annotated full webrev >>>>>> guiding the review. >>>>>> >>>>>> The intent is to have JEP-181 targeted and integrated by the end >>>>>> of this month. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>> The nestmates project (JEP-181) introduces new classfile >>>>>> attributes to identify classes and interfaces in the same nest, so >>>>>> that the VM can perform access control based on those attributes >>>>>> and so allow direct private access between nestmates without >>>>>> requiring javac to generate synthetic accessor methods. These >>>>>> access control changes also extend to core reflection and the >>>>>> MethodHandle.Lookup contexts. >>>>>> >>>>>> Direct private calls between nestmates requires a more general >>>>>> calling context than is permitted by invokespecial, and so the >>>>>> JVMS is updated to allow, and javac updated to use, invokevirtual >>>>>> and invokeinterface for private class and interface method calls >>>>>> respectively. These changed semantics also extend to MethodHandle >>>>>> findXXX operations. >>>>>> >>>>>> At this time we are only concerned with static nest definitions, >>>>>> which map to a top-level class/interface as the nest-host and all >>>>>> its nested types as nest-members. >>>>>> >>>>>> Please see the JEP for further details. >>>>>> >>>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>>>> >>>>>> All of the specification changes have been previously been worked >>>>>> out by the Valhalla Project Expert Group, and the implementation >>>>>> reviewed by the various contributors and discussed on the >>>>>> valhalla-dev mailing list. >>>>>> >>>>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen >>>>>> Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, >>>>>> Kumar Srinivasan >>>>>> >>>>>> Master webrev of all changes: >>>>>> >>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>>>> >>>>>> Annotated master webrev index: >>>>>> >>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>>>> >>>>>> Performance: this is expected to be performance neutral in a >>>>>> general sense. Benchmarking and performance runs are about to start. >>>>>> >>>>>> Testing Discussion: >>>>>> ------------------ >>>>>> >>>>>> The testing for nestmates can be broken into four main groups: >>>>>> >>>>>> -? New tests specifically related to nestmates and currently in >>>>>> the runtime/Nestmates directory >>>>>> >>>>>> - New tests to complement existing tests by adding in testcases >>>>>> not previously expressible. >>>>>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>>>>> use of invokespecial for private interface methods and performing >>>>>> receiver typechecks, so we add >>>>>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>>>>> invokeinterface. >>>>>> >>>>>> -? New JVM TI tests to verify the spec changes related to nest >>>>>> attributes. >>>>>> >>>>>> -? Existing tests significantly affected by the nestmates changes, >>>>>> primarily: >>>>>> ??? -? runtime/SelectionResolution >>>>>> >>>>>> ??? In most cases the nestmate changes makes certain invocations >>>>>> that were illegal, legal (e.g. not requiring invokespecial to >>>>>> invoke private interface methods; allowing access to private >>>>>> members via reflection/Methodhandles that were previously not >>>>>> allowed). >>>>>> >>>>>> - Existing tests incidentally affected by the nestmate changes >>>>>> >>>>>> ?? This includes tests of things utilising class >>>>>> redefinition/retransformation to alter nested types but which >>>>>> unintentionally alter nest relationships (which is not permitted). >>>>>> >>>>>> There are still a number of tests problem-listed with issues filed >>>>>> against them to have them adapted to work with nestmates. Some of >>>>>> these are intended to be addressed in the short-term, while some >>>>>> (such as the runtime/SelectionResolution test changes) may not >>>>>> eventuate. >>>>>> >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>>>> >>>>>> There is also further test work still to be completed (the JNI and >>>>>> JDI invocation tests): >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>>>> which will continue in parallel with the main RFR. >>>>>> >>>>>> Pre-integration Testing: >>>>>> ??- General: >>>>>> ???? - Mach5: hs/jdk tier1,2 >>>>>> ???? - Mach5: hs-nightly (tiers 1 -3) >>>>>> ??- Targetted >>>>>> ??? - nashorn (for asm changes) >>>>>> ??? - hotspot: runtime/* >>>>>> ?????????????? serviceability/* >>>>>> ?????????????? compiler/* >>>>>> ?????????????? vmTestbase/* >>>>>> ??? - jdk: java/lang/invoke/* >>>>>> ?????????? java/lang/reflect/* >>>>>> ?????????? java/lang/instrument/* >>>>>> ?????????? java/lang/Class/* >>>>>> ?????????? java/lang/management/* >>>>>> ?? - langtools: tools/javac >>>>>> ??????????????? tools/javap >>>>>> From shade at redhat.com Thu May 17 10:52:10 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 17 May 2018 12:52:10 +0200 Subject: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant In-Reply-To: References: Message-ID: On 05/17/2018 12:40 PM, Markus Gronlund wrote: > Please see this tiny fix for the following bug: > > https://bugs.openjdk.java.net/browse/JDK-8203346 > diff --git a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp > --- a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp > +++ b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp > > @@ -117,7 +117,7 @@ > jlong JNICALL jfr_get_epoch_address(JNIEnv* env, jobject jvm); > > -jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jlong gen, jlong id, jstring string); > +jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jboolean epoch, jlong id, jstring string); > > void JNICALL jfr_uncaught_exception(JNIEnv* env, jobject jvm, jobject thread, jthrowable throwable); Looks good, but see: $ ack jfr_add_string_constant src/hotspot/ src/hotspot/share/jfr/jni/jfrJniMethod.cpp 300:JVM_ENTRY_NO_ENV(jboolean, jfr_add_string_constant(JNIEnv* env, jclass jvm, jboolean epoch, jlong id, jstring string)) src/hotspot/share/jfr/jni/jfrJniMethod.hpp 120:jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jlong gen, jlong id, jstring string); src/hotspot/share/jfr/jni/jfrJniMethodRegistration.cpp 76: (char*)"addStringConstant", (char*)"(ZJLjava/lang/String;)Z", (void*)jfr_add_string_constant, In jfrJniMethodRegistration.cpp, the signature should now be "(ZZLjava/lang/String;)Z" -Aleksey P.S. You might want to fix your mailer, it produces excess newlines: http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-May/032352.html From shade at redhat.com Thu May 17 10:56:51 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 17 May 2018 12:56:51 +0200 Subject: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant In-Reply-To: References: Message-ID: On 05/17/2018 12:52 PM, Aleksey Shipilev wrote: > On 05/17/2018 12:40 PM, Markus Gronlund wrote: >> Please see this tiny fix for the following bug: >> >> https://bugs.openjdk.java.net/browse/JDK-8203346 >> diff --git a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >> --- a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >> +++ b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >> >> @@ -117,7 +117,7 @@ >> jlong JNICALL jfr_get_epoch_address(JNIEnv* env, jobject jvm); >> >> -jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jlong gen, jlong id, jstring string); >> +jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jboolean epoch, jlong id, jstring string); >> >> void JNICALL jfr_uncaught_exception(JNIEnv* env, jobject jvm, jobject thread, jthrowable throwable); > > Looks good, but see: > > $ ack jfr_add_string_constant src/hotspot/ > > src/hotspot/share/jfr/jni/jfrJniMethod.cpp > 300:JVM_ENTRY_NO_ENV(jboolean, jfr_add_string_constant(JNIEnv* env, jclass jvm, jboolean epoch, > jlong id, jstring string)) > > src/hotspot/share/jfr/jni/jfrJniMethod.hpp > 120:jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jlong gen, jlong id, jstring string); > > src/hotspot/share/jfr/jni/jfrJniMethodRegistration.cpp > 76: (char*)"addStringConstant", (char*)"(ZJLjava/lang/String;)Z", (void*)jfr_add_string_constant, > > > In jfrJniMethodRegistration.cpp, the signature should now be "(ZZLjava/lang/String;)Z" No, wait, I confused myself. Java method indeed is (boolean, long, String), so current signature is correct. public static native boolean addStringConstant(boolean epoch, long id, String s); Thanks, -Aleksey From david.holmes at oracle.com Thu May 17 11:03:25 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 17 May 2018 21:03:25 +1000 Subject: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant In-Reply-To: References: Message-ID: <29755bcd-1e92-478f-586f-d7a02d839ee8@oracle.com> I'm intrigued as to how/why this would only cause a problem on 32-bit windows ?? What version of Visual Studio was it? David On 17/05/2018 8:40 PM, Markus Gronlund wrote: > Greetings, > > > > Please see this tiny fix for the following bug: > > > > https://bugs.openjdk.java.net/browse/JDK-8203346 > > > > Change: > > > > # HG changeset patch > > # User mgronlun > > # Date 1526553151 -7200 > > # Thu May 17 12:32:31 2018 +0200 > > # Node ID 9ac9ae21cc4c872ab40bb2c171dda3a921aa25fb > > # Parent 8e4fcfb4cfe48e6b481c6aa1751d916014a826af > > 8203346: JFR: Inconsistent signature of jfr_add_string_constant > > Reviewed-by: > > > > diff --git a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp > > --- a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp > > +++ b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp > > @@ -117,7 +117,7 @@ > > > > jlong JNICALL jfr_get_epoch_address(JNIEnv* env, jobject jvm); > > > > -jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jlong gen, jlong id, jstring string); > > +jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jboolean epoch, jlong id, jstring string); > > > > void JNICALL jfr_uncaught_exception(JNIEnv* env, jobject jvm, jobject thread, jthrowable throwable); > > > > > > Thanks to Martin for reporting, sorry for the inconvenience > > > > Markus > From shade at redhat.com Thu May 17 11:06:13 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 17 May 2018 13:06:13 +0200 Subject: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant In-Reply-To: References: Message-ID: On 05/17/2018 12:56 PM, Aleksey Shipilev wrote: > On 05/17/2018 12:52 PM, Aleksey Shipilev wrote: >> On 05/17/2018 12:40 PM, Markus Gronlund wrote: >>> Please see this tiny fix for the following bug: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203346 >>> diff --git a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> --- a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> +++ b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> >>> @@ -117,7 +117,7 @@ >>> jlong JNICALL jfr_get_epoch_address(JNIEnv* env, jobject jvm); >>> >>> -jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jlong gen, jlong id, jstring string); >>> +jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jboolean epoch, jlong id, jstring string); >>> >>> void JNICALL jfr_uncaught_exception(JNIEnv* env, jobject jvm, jobject thread, jthrowable throwable); >> >> Looks good, but see: >> >> $ ack jfr_add_string_constant src/hotspot/ >> >> src/hotspot/share/jfr/jni/jfrJniMethod.cpp >> 300:JVM_ENTRY_NO_ENV(jboolean, jfr_add_string_constant(JNIEnv* env, jclass jvm, jboolean epoch, >> jlong id, jstring string)) >> >> src/hotspot/share/jfr/jni/jfrJniMethod.hpp >> 120:jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jlong gen, jlong id, jstring string); >> >> src/hotspot/share/jfr/jni/jfrJniMethodRegistration.cpp >> 76: (char*)"addStringConstant", (char*)"(ZJLjava/lang/String;)Z", (void*)jfr_add_string_constant, >> >> >> In jfrJniMethodRegistration.cpp, the signature should now be "(ZZLjava/lang/String;)Z" > > No, wait, I confused myself. Java method indeed is (boolean, long, String), so current signature is > correct. > > public static native boolean addStringConstant(boolean epoch, long id, String s); Ugh. I still wonder why the return type is "jlong" in native parts. Should be jboolean too? -Aleksey From markus.gronlund at oracle.com Thu May 17 11:13:31 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Thu, 17 May 2018 04:13:31 -0700 (PDT) Subject: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant In-Reply-To: References: Message-ID: " Ugh. I still wonder why the return type is "jlong" in native parts. Should be jboolean too?" Yes, same thing there, looks like the .hpp declaration did not get properly updated when moving from a jlong generational counter to a boolean epoch. Will fix it as well. Thanks Markus -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: den 17 maj 2018 13:06 To: Markus Gronlund ; hotspot-dev at openjdk.java.net; Doerr, Martin Subject: Re: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant On 05/17/2018 12:56 PM, Aleksey Shipilev wrote: > On 05/17/2018 12:52 PM, Aleksey Shipilev wrote: >> On 05/17/2018 12:40 PM, Markus Gronlund wrote: >>> Please see this tiny fix for the following bug: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203346 >>> diff --git a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> --- a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> +++ b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> >>> @@ -117,7 +117,7 @@ >>> jlong JNICALL jfr_get_epoch_address(JNIEnv* env, jobject jvm); >>> >>> -jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, >>> jlong gen, jlong id, jstring string); >>> +jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, >>> +jboolean epoch, jlong id, jstring string); >>> >>> void JNICALL jfr_uncaught_exception(JNIEnv* env, jobject jvm, >>> jobject thread, jthrowable throwable); >> >> Looks good, but see: >> >> $ ack jfr_add_string_constant src/hotspot/ >> >> src/hotspot/share/jfr/jni/jfrJniMethod.cpp >> 300:JVM_ENTRY_NO_ENV(jboolean, jfr_add_string_constant(JNIEnv* env, >> jclass jvm, jboolean epoch, jlong id, jstring string)) >> >> src/hotspot/share/jfr/jni/jfrJniMethod.hpp >> 120:jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, >> jlong gen, jlong id, jstring string); >> >> src/hotspot/share/jfr/jni/jfrJniMethodRegistration.cpp >> 76: (char*)"addStringConstant", (char*)"(ZJLjava/lang/String;)Z", (void*)jfr_add_string_constant, >> >> >> In jfrJniMethodRegistration.cpp, the signature should now be "(ZZLjava/lang/String;)Z" > > No, wait, I confused myself. Java method indeed is (boolean, long, > String), so current signature is correct. > > public static native boolean addStringConstant(boolean epoch, long > id, String s); Ugh. I still wonder why the return type is "jlong" in native parts. Should be jboolean too? -Aleksey From markus.gronlund at oracle.com Thu May 17 11:47:13 2018 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Thu, 17 May 2018 04:47:13 -0700 (PDT) Subject: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant In-Reply-To: References: Message-ID: <3adff83f-c7cd-4ac9-8681-f46968a4d56b@default> Here is an update (with some fixes to a few small things in the vicinity as well): # HG changeset patch # User mgronlun # Date 1526557216 -7200 # Thu May 17 13:40:16 2018 +0200 # Node ID aa550bc8a5ddd1f0caa6d7e639c04b66b5cb8a2d # Parent 8e4fcfb4cfe48e6b481c6aa1751d916014a826af 8203346: JFR: Inconsistent signature of jfr_add_string_constant Reviewed-by: diff --git a/src/hotspot/share/jfr/jni/jfrJniMethod.cpp b/src/hotspot/share/jfr/jni/jfrJniMethod.cpp --- a/src/hotspot/share/jfr/jni/jfrJniMethod.cpp +++ b/src/hotspot/share/jfr/jni/jfrJniMethod.cpp @@ -298,11 +298,11 @@ JVM_END JVM_ENTRY_NO_ENV(jboolean, jfr_add_string_constant(JNIEnv* env, jclass jvm, jboolean epoch, jlong id, jstring string)) - return JfrStringPool::add(epoch == JNI_TRUE, id, string, thread); + return JfrStringPool::add(epoch == JNI_TRUE, id, string, thread) ? JNI_TRUE : JNI_FALSE; JVM_END JVM_ENTRY_NO_ENV(void, jfr_set_force_instrumentation(JNIEnv* env, jobject jvm, jboolean force_instrumentation)) - JfrEventClassTransformer::set_force_instrumentation(force_instrumentation == JNI_TRUE ? true : false); + JfrEventClassTransformer::set_force_instrumentation(force_instrumentation == JNI_TRUE); JVM_END JVM_ENTRY_NO_ENV(void, jfr_emit_old_object_samples(JNIEnv* env, jobject jvm, jlong cutoff_ticks, jboolean emit_all)) diff --git a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp --- a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp +++ b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp @@ -117,7 +117,7 @@ jlong JNICALL jfr_get_epoch_address(JNIEnv* env, jobject jvm); -jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jlong gen, jlong id, jstring string); +jboolean JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jboolean epoch, jlong id, jstring string); void JNICALL jfr_uncaught_exception(JNIEnv* env, jobject jvm, jobject thread, jthrowable throwable); Thanks Markus -----Original Message----- From: Markus Gronlund Sent: den 17 maj 2018 13:14 To: Aleksey Shipilev ; hotspot-dev at openjdk.java.net; Doerr, Martin Subject: RE: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant " Ugh. I still wonder why the return type is "jlong" in native parts. Should be jboolean too?" Yes, same thing there, looks like the .hpp declaration did not get properly updated when moving from a jlong generational counter to a boolean epoch. Will fix it as well. Thanks Markus -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: den 17 maj 2018 13:06 To: Markus Gronlund ; hotspot-dev at openjdk.java.net; Doerr, Martin Subject: Re: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant On 05/17/2018 12:56 PM, Aleksey Shipilev wrote: > On 05/17/2018 12:52 PM, Aleksey Shipilev wrote: >> On 05/17/2018 12:40 PM, Markus Gronlund wrote: >>> Please see this tiny fix for the following bug: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203346 >>> diff --git a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> --- a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> +++ b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> >>> @@ -117,7 +117,7 @@ >>> jlong JNICALL jfr_get_epoch_address(JNIEnv* env, jobject jvm); >>> >>> -jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, >>> jlong gen, jlong id, jstring string); >>> +jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, >>> +jboolean epoch, jlong id, jstring string); >>> >>> void JNICALL jfr_uncaught_exception(JNIEnv* env, jobject jvm, >>> jobject thread, jthrowable throwable); >> >> Looks good, but see: >> >> $ ack jfr_add_string_constant src/hotspot/ >> >> src/hotspot/share/jfr/jni/jfrJniMethod.cpp >> 300:JVM_ENTRY_NO_ENV(jboolean, jfr_add_string_constant(JNIEnv* env, >> jclass jvm, jboolean epoch, jlong id, jstring string)) >> >> src/hotspot/share/jfr/jni/jfrJniMethod.hpp >> 120:jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, >> jlong gen, jlong id, jstring string); >> >> src/hotspot/share/jfr/jni/jfrJniMethodRegistration.cpp >> 76: (char*)"addStringConstant", (char*)"(ZJLjava/lang/String;)Z", (void*)jfr_add_string_constant, >> >> >> In jfrJniMethodRegistration.cpp, the signature should now be "(ZZLjava/lang/String;)Z" > > No, wait, I confused myself. Java method indeed is (boolean, long, > String), so current signature is correct. > > public static native boolean addStringConstant(boolean epoch, long > id, String s); Ugh. I still wonder why the return type is "jlong" in native parts. Should be jboolean too? -Aleksey From shade at redhat.com Thu May 17 12:17:11 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 17 May 2018 14:17:11 +0200 Subject: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant In-Reply-To: <3adff83f-c7cd-4ac9-8681-f46968a4d56b@default> References: <3adff83f-c7cd-4ac9-8681-f46968a4d56b@default> Message-ID: <657f3aa9-c151-d475-7f48-c4ffcdf6627c@redhat.com> On 05/17/2018 01:47 PM, Markus Gronlund wrote: > Here is an update (with some fixes to a few small things in the vicinity as well): > > # HG changeset patch > # User mgronlun > # Date 1526557216 -7200 > # Thu May 17 13:40:16 2018 +0200 > # Node ID aa550bc8a5ddd1f0caa6d7e639c04b66b5cb8a2d > # Parent 8e4fcfb4cfe48e6b481c6aa1751d916014a826af > 8203346: JFR: Inconsistent signature of jfr_add_string_constant > Reviewed-by: > > diff --git a/src/hotspot/share/jfr/jni/jfrJniMethod.cpp b/src/hotspot/share/jfr/jni/jfrJniMethod.cpp > --- a/src/hotspot/share/jfr/jni/jfrJniMethod.cpp > +++ b/src/hotspot/share/jfr/jni/jfrJniMethod.cpp > @@ -298,11 +298,11 @@ > JVM_END > > JVM_ENTRY_NO_ENV(jboolean, jfr_add_string_constant(JNIEnv* env, jclass jvm, jboolean epoch, jlong id, jstring string)) > - return JfrStringPool::add(epoch == JNI_TRUE, id, string, thread); > + return JfrStringPool::add(epoch == JNI_TRUE, id, string, thread) ? JNI_TRUE : JNI_FALSE; > JVM_END > > JVM_ENTRY_NO_ENV(void, jfr_set_force_instrumentation(JNIEnv* env, jobject jvm, jboolean force_instrumentation)) > - JfrEventClassTransformer::set_force_instrumentation(force_instrumentation == JNI_TRUE ? true : false); > + JfrEventClassTransformer::set_force_instrumentation(force_instrumentation == JNI_TRUE); > JVM_END > > JVM_ENTRY_NO_ENV(void, jfr_emit_old_object_samples(JNIEnv* env, jobject jvm, jlong cutoff_ticks, jboolean emit_all)) > diff --git a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp > --- a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp > +++ b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp > @@ -117,7 +117,7 @@ > > jlong JNICALL jfr_get_epoch_address(JNIEnv* env, jobject jvm); > > -jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jlong gen, jlong id, jstring string); > +jboolean JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jboolean epoch, jlong id, jstring string); > > void JNICALL jfr_uncaught_exception(JNIEnv* env, jobject jvm, jobject thread, jthrowable throwable); This looks good. -Aleksey From matthias.baesken at sap.com Thu May 17 12:23:29 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 17 May 2018 12:23:29 +0000 Subject: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant Message-ID: <446aaa390b7940aea2eb4a5b46f57c52@sap.com> Hi David , it was VS2013 . We wondered a bit as well why we do not see it on other OS . Best Regards, Matthias >I'm intrigued as to how/why this would only cause a problem on 32-bit >windows ?? What version of Visual Studio was it? > >David > >On 17/05/2018 8:40 PM, Markus Gronlund wrote: >> Greetings, >> >> >> >> Please see this tiny fix for the following bug: >> >> >> >> https://bugs.openjdk.java.net/browse/JDK-8203346 >> >> .... From martin.doerr at sap.com Thu May 17 12:24:21 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 17 May 2018 12:24:21 +0000 Subject: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) In-Reply-To: <9d8f5203-8cb8-56d1-8163-b5d8c599eca3@redhat.com> References: <2252a635cba543a8b61dafe28b199327@sap.com> <48176edc-b498-3877-8bbc-ad24b064b487@redhat.com> <93c1a964c7ee41d4972ef78b6958a5b3@sap.com> <5f213774-94d7-e32e-c59c-8ea81fb402a8@redhat.com> <38311dc4306147b8ba303ab447b4f61e@sap.com> <117e46dc-5e00-cb47-2de0-2451cef2e6b4@redhat.com> <9d8f5203-8cb8-56d1-8163-b5d8c599eca3@redhat.com> Message-ID: Hi Aleksey, thanks for the review. Improved comment and pushed. Best regards, Martin -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: Donnerstag, 17. Mai 2018 12:36 To: Doerr, Martin ; hotspot-dev developers (hotspot-dev at openjdk.java.net) Cc: Thomas St?fe ; John Paul Adrian Glaubitz Subject: Re: RFR: 8203288: PPC64 and s390 fail to build after JDK-8199712 (Flight Recorder) On 05/17/2018 12:10 PM, Doerr, Martin wrote: > Please review: > http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.01/ *) Stray comment in os_aix.cpp, remove before pushing? 1392 // print_libversion_info(st); Otherwise looks good. I rechecked ppc64el and s390x build fine with this patch. -Aleksey From martin.doerr at sap.com Thu May 17 12:25:12 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 17 May 2018 12:25:12 +0000 Subject: RFR(XS): 8203305: PPC64: Improve TM detection for enabling RTM on Linux / POWER9 In-Reply-To: <894f02c6-868e-d178-fe5e-1f1aefb979f0@linux.vnet.ibm.com> References: <894f02c6-868e-d178-fe5e-1f1aefb979f0@linux.vnet.ibm.com> Message-ID: Hi Gustavo, thanks for the contribution. Reviewed and pushed. Best regards, Martin -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Gustavo Romero Sent: Mittwoch, 16. Mai 2018 18:02 To: hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net Subject: RFR(XS): 8203305: PPC64: Improve TM detection for enabling RTM on Linux / POWER9 Hi, Could the following small change be reviewed please? It improves the TM detection for enabling (or not) RTM on Linux. With that change JVM uses TM capability flags advertised by the kernel instead of parsing major/minor kernel version. That way is better to address TM on POWER9 which now supports a new TM mode called "TM with no Suspend Mode" and that is not mature yet (so I'm disabling it for now). The new mode is just available on POWER9 DD2.1 NV (baremetal) and so TM on POWER9 VMs behave as on POWER8 hence RTM is enabled on POWER9 VM only. Nothing changes regarding AIX. bug: https://bugs.openjdk.java.net/browse/JDK-8203305 webrev: http://cr.openjdk.java.net/~gromero/POWER9/8203305/v1 Thank you. Best regards, Gustavo From gromero at linux.vnet.ibm.com Thu May 17 12:32:33 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Thu, 17 May 2018 09:32:33 -0300 Subject: RFR(XS): 8203305: PPC64: Improve TM detection for enabling RTM on Linux / POWER9 In-Reply-To: References: <894f02c6-868e-d178-fe5e-1f1aefb979f0@linux.vnet.ibm.com> Message-ID: <1121128d-ba8f-2d47-f787-c815a584b6b5@linux.vnet.ibm.com> Hi Martin, On 05/17/2018 09:25 AM, Doerr, Martin wrote: > Hi Gustavo, > > thanks for the contribution. Reviewed and pushed. Thanks a lot for reviewing and pushing it. Best regards, Gustavo > Best regards, > Martin > > > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Gustavo Romero > Sent: Mittwoch, 16. Mai 2018 18:02 > To: hotspot-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net > Subject: RFR(XS): 8203305: PPC64: Improve TM detection for enabling RTM on Linux / POWER9 > > Hi, > > Could the following small change be reviewed please? > > It improves the TM detection for enabling (or not) RTM on Linux. > > With that change JVM uses TM capability flags advertised by the kernel > instead of parsing major/minor kernel version. > > That way is better to address TM on POWER9 which now supports a new TM mode > called "TM with no Suspend Mode" and that is not mature yet (so I'm > disabling it for now). > > The new mode is just available on POWER9 DD2.1 NV (baremetal) and so TM on > POWER9 VMs behave as on POWER8 hence RTM is enabled on POWER9 VM only. > > Nothing changes regarding AIX. > > bug: https://bugs.openjdk.java.net/browse/JDK-8203305 > webrev: http://cr.openjdk.java.net/~gromero/POWER9/8203305/v1 > > Thank you. > > > Best regards, > Gustavo > From adam.farley at uk.ibm.com Thu May 17 12:57:37 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Thu, 17 May 2018 13:57:37 +0100 Subject: RFR: JDK-8190187: Follow-up. Message-ID: Hi Alan, As requested. Any ideas? - Adam > On 16/05/2018 16:51, Adam Farley8 wrote: > > > Hi Alan, > > > > As I'm calling time on this task (no responses in several months), I wanted to ask if you > > could think of any way this code (or some other code to a similar effect) could be committed > > to openjdk. > > > > I'm out of ideas. > > Can you send this to the mailing list and I will reply there? > > -Alan Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From rwestrel at redhat.com Thu May 17 12:58:25 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Thu, 17 May 2018 14:58:25 +0200 Subject: 8202377: Modularize C2 GC barriers In-Reply-To: <5AF30676.8050909@oracle.com> References: <5AE86C53.4070708@oracle.com> <5AF30676.8050909@oracle.com> Message-ID: Hi Erik, > http://cr.openjdk.java.net/~eosterlund/8202377/webrev.01/ Overall, this looks good to me. I suppose: loop_optimize_gc_barrier() find_dominating_barriers() are there for ZGC and they don't feel generic enough to me to be included in this API. That is if I were to add another GC or let say change the G1 implementation so it's better suited for optimizations, I think it's quite unlikely I would reuse either hook. It would be better to remove them in my opinion. Roland. From martin.doerr at sap.com Thu May 17 13:10:14 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 17 May 2018 13:10:14 +0000 Subject: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant In-Reply-To: <3adff83f-c7cd-4ac9-8681-f46968a4d56b@default> References: <3adff83f-c7cd-4ac9-8681-f46968a4d56b@default> Message-ID: <646be14c6c7247949854ee195d1593c9@sap.com> Hi Markus, thanks for fixing it so quickly. Best regards, Martin -----Original Message----- From: Markus Gronlund [mailto:markus.gronlund at oracle.com] Sent: Donnerstag, 17. Mai 2018 13:47 To: Aleksey Shipilev ; hotspot-dev at openjdk.java.net; Doerr, Martin Subject: RE: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant Here is an update (with some fixes to a few small things in the vicinity as well): # HG changeset patch # User mgronlun # Date 1526557216 -7200 # Thu May 17 13:40:16 2018 +0200 # Node ID aa550bc8a5ddd1f0caa6d7e639c04b66b5cb8a2d # Parent 8e4fcfb4cfe48e6b481c6aa1751d916014a826af 8203346: JFR: Inconsistent signature of jfr_add_string_constant Reviewed-by: diff --git a/src/hotspot/share/jfr/jni/jfrJniMethod.cpp b/src/hotspot/share/jfr/jni/jfrJniMethod.cpp --- a/src/hotspot/share/jfr/jni/jfrJniMethod.cpp +++ b/src/hotspot/share/jfr/jni/jfrJniMethod.cpp @@ -298,11 +298,11 @@ JVM_END JVM_ENTRY_NO_ENV(jboolean, jfr_add_string_constant(JNIEnv* env, jclass jvm, jboolean epoch, jlong id, jstring string)) - return JfrStringPool::add(epoch == JNI_TRUE, id, string, thread); + return JfrStringPool::add(epoch == JNI_TRUE, id, string, thread) ? JNI_TRUE : JNI_FALSE; JVM_END JVM_ENTRY_NO_ENV(void, jfr_set_force_instrumentation(JNIEnv* env, jobject jvm, jboolean force_instrumentation)) - JfrEventClassTransformer::set_force_instrumentation(force_instrumentation == JNI_TRUE ? true : false); + JfrEventClassTransformer::set_force_instrumentation(force_instrumentation == JNI_TRUE); JVM_END JVM_ENTRY_NO_ENV(void, jfr_emit_old_object_samples(JNIEnv* env, jobject jvm, jlong cutoff_ticks, jboolean emit_all)) diff --git a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp --- a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp +++ b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp @@ -117,7 +117,7 @@ jlong JNICALL jfr_get_epoch_address(JNIEnv* env, jobject jvm); -jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jlong gen, jlong id, jstring string); +jboolean JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, jboolean epoch, jlong id, jstring string); void JNICALL jfr_uncaught_exception(JNIEnv* env, jobject jvm, jobject thread, jthrowable throwable); Thanks Markus -----Original Message----- From: Markus Gronlund Sent: den 17 maj 2018 13:14 To: Aleksey Shipilev ; hotspot-dev at openjdk.java.net; Doerr, Martin Subject: RE: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant " Ugh. I still wonder why the return type is "jlong" in native parts. Should be jboolean too?" Yes, same thing there, looks like the .hpp declaration did not get properly updated when moving from a jlong generational counter to a boolean epoch. Will fix it as well. Thanks Markus -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: den 17 maj 2018 13:06 To: Markus Gronlund ; hotspot-dev at openjdk.java.net; Doerr, Martin Subject: Re: RFR(XXS): 8203346: JFR: Inconsistent signature of jfr_add_string_constant On 05/17/2018 12:56 PM, Aleksey Shipilev wrote: > On 05/17/2018 12:52 PM, Aleksey Shipilev wrote: >> On 05/17/2018 12:40 PM, Markus Gronlund wrote: >>> Please see this tiny fix for the following bug: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8203346 >>> diff --git a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> --- a/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> +++ b/src/hotspot/share/jfr/jni/jfrJniMethod.hpp >>> >>> @@ -117,7 +117,7 @@ >>> jlong JNICALL jfr_get_epoch_address(JNIEnv* env, jobject jvm); >>> >>> -jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, >>> jlong gen, jlong id, jstring string); >>> +jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, >>> +jboolean epoch, jlong id, jstring string); >>> >>> void JNICALL jfr_uncaught_exception(JNIEnv* env, jobject jvm, >>> jobject thread, jthrowable throwable); >> >> Looks good, but see: >> >> $ ack jfr_add_string_constant src/hotspot/ >> >> src/hotspot/share/jfr/jni/jfrJniMethod.cpp >> 300:JVM_ENTRY_NO_ENV(jboolean, jfr_add_string_constant(JNIEnv* env, >> jclass jvm, jboolean epoch, jlong id, jstring string)) >> >> src/hotspot/share/jfr/jni/jfrJniMethod.hpp >> 120:jlong JNICALL jfr_add_string_constant(JNIEnv* env, jclass jvm, >> jlong gen, jlong id, jstring string); >> >> src/hotspot/share/jfr/jni/jfrJniMethodRegistration.cpp >> 76: (char*)"addStringConstant", (char*)"(ZJLjava/lang/String;)Z", (void*)jfr_add_string_constant, >> >> >> In jfrJniMethodRegistration.cpp, the signature should now be "(ZZLjava/lang/String;)Z" > > No, wait, I confused myself. Java method indeed is (boolean, long, > String), so current signature is correct. > > public static native boolean addStringConstant(boolean epoch, long > id, String s); Ugh. I still wonder why the return type is "jlong" in native parts. Should be jboolean too? -Aleksey From sgehwolf at redhat.com Thu May 17 13:12:05 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 17 May 2018 15:12:05 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> Message-ID: <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> Hi Martin, Aleksey, Since JDK-8203288 has been pushed here is the latest webrev for Zero. It no longer contains changes in os_perf_linux.cpp. http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.02/ How does it look? Thanks, Severin On Wed, 2018-05-16 at 15:46 +0000, Doerr, Martin wrote: > Hi Severin, > > I'd prefer to push PPC64/s390 first because it contains the final solution. > If you want to push earlier, please just omit the change to os_perf_linux.cpp. > Or you can change that file to the final solution. I'd be fine with that, too. > > Best regards, > Martin > > > -----Original Message----- > From: Aleksey Shipilev [mailto:shade at redhat.com] > Sent: Mittwoch, 16. Mai 2018 17:40 > To: Severin Gehwolf ; Doerr, Martin ; hotspot-dev > Subject: Re: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) > > On 05/16/2018 05:38 PM, Severin Gehwolf wrote: > > Hi Martin, Aleksey, > > > > On Wed, 2018-05-16 at 17:08 +0200, Aleksey Shipilev wrote: > > > On 05/16/2018 05:04 PM, Doerr, Martin wrote: > > > > JDK-8203288 contains a change of os_perf_linux.cpp using: > > > > #include CPU_HEADER(vm_version_ext) > > > > (See http://cr.openjdk.java.net/~mdoerr/8203288_ppc64_s390_build/webrev.00/) > > > > > > > > Can you check if that works for you, please? > > > > That works, thanks! > > > > How do you want to coordinate getting both patches integrated? Shall I > > wait for your change to get integrated or should I get it in as is > > first and you'll resolve the merge conflict changing to your version in > > os_perf_linux.cpp? > > Personally, I'd prefer PPC64/S390 to go in first, because it makes more sense to have CPU_HEADER. > Then Zero can piggyback on it. > > -Aleksey > From shade at redhat.com Thu May 17 13:27:32 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 17 May 2018 15:27:32 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> Message-ID: On 05/17/2018 03:12 PM, Severin Gehwolf wrote: > Hi Martin, Aleksey, > > Since JDK-8203288 has been pushed here is the latest webrev for Zero. > It no longer contains changes in os_perf_linux.cpp. > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.02/ > > How does it look? Looks good, but I used parentheses inconsistently with the rest of Hotspot in the original patch: #if (defined X86 && !defined ZERO) Should be: #if defined(X86) && !defined (ZERO) -Aleksey From erik.osterlund at oracle.com Thu May 17 13:48:45 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 17 May 2018 15:48:45 +0200 Subject: 8202377: Modularize C2 GC barriers In-Reply-To: References: <5AE86C53.4070708@oracle.com> <5AF30676.8050909@oracle.com> Message-ID: <5AFD883D.1040604@oracle.com> Hi Roland, Thank you for having a look at these changes. On 2018-05-17 14:58, Roland Westrelin wrote: > Hi Erik, > >> http://cr.openjdk.java.net/~eosterlund/8202377/webrev.01/ > Overall, this looks good to me. > > I suppose: > > loop_optimize_gc_barrier() > find_dominating_barriers() > > are there for ZGC and they don't feel generic enough to me to be > included in this API. That is if I were to add another GC or let say > change the G1 implementation so it's better suited for optimizations, I > think it's quite unlikely I would reuse either hook. It would be better > to remove them in my opinion. Yes they are indeed hooks used only by ZGC at the moment, and I agree they might be an over-generalization at the moment. I am okay with removing those hooks now, and then we see what we might have in common with Shenandoah later when we have integrated them both. Incremental webrev: http://cr.openjdk.java.net/~eosterlund/8202377/webrev.01_02/ Full webrev: http://cr.openjdk.java.net/~eosterlund/8202377/webrev.02/ Again, thanks a lot for looking at these changes. Thanks, /Erik > Roland. From rwestrel at redhat.com Thu May 17 13:50:47 2018 From: rwestrel at redhat.com (Roland Westrelin) Date: Thu, 17 May 2018 15:50:47 +0200 Subject: 8202377: Modularize C2 GC barriers In-Reply-To: <5AFD883D.1040604@oracle.com> References: <5AE86C53.4070708@oracle.com> <5AF30676.8050909@oracle.com> <5AFD883D.1040604@oracle.com> Message-ID: > http://cr.openjdk.java.net/~eosterlund/8202377/webrev.02/ That looks good to me. Roland. From erik.osterlund at oracle.com Thu May 17 13:56:08 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 17 May 2018 15:56:08 +0200 Subject: 8202377: Modularize C2 GC barriers In-Reply-To: References: <5AE86C53.4070708@oracle.com> <5AF30676.8050909@oracle.com> <5AFD883D.1040604@oracle.com> Message-ID: <5AFD89F8.5000303@oracle.com> Hi Roland, Thank you for the review. :) /Erik On 2018-05-17 15:50, Roland Westrelin wrote: >> http://cr.openjdk.java.net/~eosterlund/8202377/webrev.02/ > That looks good to me. > > Roland. From sgehwolf at redhat.com Thu May 17 14:02:34 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 17 May 2018 16:02:34 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> Message-ID: On Thu, 2018-05-17 at 15:27 +0200, Aleksey Shipilev wrote: > On 05/17/2018 03:12 PM, Severin Gehwolf wrote: > > Hi Martin, Aleksey, > > > > Since JDK-8203288 has been pushed here is the latest webrev for Zero. > > It no longer contains changes in os_perf_linux.cpp. > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.02/ > > > > How does it look? > > Looks good, but I used parentheses inconsistently with the rest of Hotspot in the original patch: > > #if (defined X86 && !defined ZERO) > > Should be: > > #if defined(X86) && !defined (ZERO) OK. Here it goes: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.03/ Thoughts? Cheers, Severin From shade at redhat.com Thu May 17 14:14:29 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 17 May 2018 16:14:29 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> Message-ID: <0a09b984-a617-af45-4033-43086fe818a4@redhat.com> On 05/17/2018 04:02 PM, Severin Gehwolf wrote: > On Thu, 2018-05-17 at 15:27 +0200, Aleksey Shipilev wrote: >> On 05/17/2018 03:12 PM, Severin Gehwolf wrote: >>> Hi Martin, Aleksey, >>> >>> Since JDK-8203288 has been pushed here is the latest webrev for Zero. >>> It no longer contains changes in os_perf_linux.cpp. >>> >>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.02/ >>> >>> How does it look? >> >> Looks good, but I used parentheses inconsistently with the rest of Hotspot in the original patch: >> >> #if (defined X86 && !defined ZERO) >> >> Should be: >> >> #if defined(X86) && !defined (ZERO) > > OK. Here it goes: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.03/ Notice the parentheses, it is done like that in other places: #if defined(X86) && !defined(ZERO) Apply this patch to correct: http://cr.openjdk.java.net/~shade/8203287/parent.patch I checked it builds with x86_64 and Zero. -Aleksey From erik.osterlund at oracle.com Thu May 17 14:19:18 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 17 May 2018 16:19:18 +0200 Subject: RFR: 8203353: Fixup inferred decorators in the interpreter Message-ID: <5AFD8F66.1070006@oracle.com> Hi, All subsystems that the access API touches, except the interpreter, infers a set of decorators according to rules specified in accessDecorators.hpp. These are typically sane defaults for things like reference strength, memory ordering, as well as inferring that IN_HEAP_ARRAY is IN_HEAP, and IN_CONCURRENT_ROOT is IN_ROOT. The interpreter should do that too. Webrev: http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8203353 Thanks, /Erik From martin.doerr at sap.com Thu May 17 14:17:04 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 17 May 2018 14:17:04 +0000 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <0a09b984-a617-af45-4033-43086fe818a4@redhat.com> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> <0a09b984-a617-af45-4033-43086fe818a4@redhat.com> Message-ID: Hi Severin, looks good to me with Aleksey's patch. Thanks for waiting. Best regards, Martin -----Original Message----- From: Aleksey Shipilev [mailto:shade at redhat.com] Sent: Donnerstag, 17. Mai 2018 16:14 To: Severin Gehwolf ; Doerr, Martin ; hotspot-dev Subject: Re: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) On 05/17/2018 04:02 PM, Severin Gehwolf wrote: > On Thu, 2018-05-17 at 15:27 +0200, Aleksey Shipilev wrote: >> On 05/17/2018 03:12 PM, Severin Gehwolf wrote: >>> Hi Martin, Aleksey, >>> >>> Since JDK-8203288 has been pushed here is the latest webrev for Zero. >>> It no longer contains changes in os_perf_linux.cpp. >>> >>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.02/ >>> >>> How does it look? >> >> Looks good, but I used parentheses inconsistently with the rest of Hotspot in the original patch: >> >> #if (defined X86 && !defined ZERO) >> >> Should be: >> >> #if defined(X86) && !defined (ZERO) > > OK. Here it goes: > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.03/ Notice the parentheses, it is done like that in other places: #if defined(X86) && !defined(ZERO) Apply this patch to correct: http://cr.openjdk.java.net/~shade/8203287/parent.patch I checked it builds with x86_64 and Zero. -Aleksey From sgehwolf at redhat.com Thu May 17 14:38:36 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Thu, 17 May 2018 16:38:36 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> <0a09b984-a617-af45-4033-43086fe818a4@redhat.com> Message-ID: Hi Martin, Aleksey, Ugh, sorry, my mistake :-/ Thanks for the reviews! This is what I'm going to push once jdk-submit comes back clean: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.04/ Thanks, Severin On Thu, 2018-05-17 at 14:17 +0000, Doerr, Martin wrote: > Hi Severin, > > looks good to me with Aleksey's patch. Thanks for waiting. > > Best regards, > Martin > > > -----Original Message----- > From: Aleksey Shipilev [mailto:shade at redhat.com] > Sent: Donnerstag, 17. Mai 2018 16:14 > To: Severin Gehwolf ; Doerr, Martin ; hotspot-dev > Subject: Re: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) > > On 05/17/2018 04:02 PM, Severin Gehwolf wrote: > > On Thu, 2018-05-17 at 15:27 +0200, Aleksey Shipilev wrote: > > > On 05/17/2018 03:12 PM, Severin Gehwolf wrote: > > > > Hi Martin, Aleksey, > > > > > > > > Since JDK-8203288 has been pushed here is the latest webrev for Zero. > > > > It no longer contains changes in os_perf_linux.cpp. > > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.02/ > > > > > > > > How does it look? > > > > > > Looks good, but I used parentheses inconsistently with the rest of Hotspot in the original patch: > > > > > > #if (defined X86 && !defined ZERO) > > > > > > Should be: > > > > > > #if defined(X86) && !defined (ZERO) > > > > OK. Here it goes: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.03/ > > Notice the parentheses, it is done like that in other places: > > #if defined(X86) && !defined(ZERO) > > Apply this patch to correct: > http://cr.openjdk.java.net/~shade/8203287/parent.patch > > I checked it builds with x86_64 and Zero. > > -Aleksey > From matthias.baesken at sap.com Thu May 17 15:20:29 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 17 May 2018 15:20:29 +0000 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> Message-ID: > > > I also think that one number is enough. Adaptive numbers would be great. Hi Thomas/Goetz/Martin , thanks for looking into it. Adaptive numbers could be done in a follow-up as suggested by Thomas , this would also touch the other platforms . I added more line feeds to the output and reduced output to just printing M . Example output from a Windows desktop PC : Memory: 4k page, system-wide physical 32520M (20165M free) TotalPageFile size 34568M (AvailPageFile size 18170M) current process WorkingSet (physical memory assigned to process): 79M, peak: 79M current process commit charge ("private bytes"): 148M, peak: 148M new webrev : http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.1/ Best regards, Matthias > -----Original Message----- > From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > Sent: Mittwoch, 16. Mai 2018 16:35 > To: Lindenmaier, Goetz > Cc: Baesken, Matthias ; Doerr, Martin > ; hotspot-dev at openjdk.java.net; hotspot- > runtime-dev at openjdk.java.net > Subject: Re: [RFR] 8202427: Enhance os::print_memory_info on Windows > > On Wed, May 16, 2018 at 3:48 PM, Lindenmaier, Goetz > wrote: > > Hi Matthias, > > > > I appreciate the additional information in the hs_err file. > > Could you please add an example output (of the final version) to the bug > description? > > > > As Martin, I would like more line feeds, both in the code and the output. > > Currently, the output in the hs_err file looks like this (where the first line is > too long): > > Memory: 4k page, system-wide physical 16776692k [16383M] (8795032k > [8588M] free), TotalPageFile size 49949460k [48778M] (AvailPageFile size > 38942152k [38029M]), user-mode portion of virtual address-space 2097024k > [2047M] (6940k [6M] free) > > current process WorkingSet (physical memory assigned to process): > 991804k [968M], peak: 991804k [968M] > > > > current process commit charge ("private bytes"): 1523632k [1487M], > peak: 1523632k [1487M] > > > > I also think that one number is enough. Adaptive numbers would be great. > > I know about > > src/hotspot/share/memory/metaspace/metaspaceCommon.hpp: > print_human_readable_size() > > for this, but this seems a bit hidden in metaspace. > > > > Guys, lets not worry about human readable numbers for now. We can move > print_human_readable_size() out from metaspace to something generic > and use it to improve this code (and others) in a later patch. That > was my original intention when adding print_human_readable_size(). > > The patch is good. Thanks for doing this. > > ..Thoams > > > I don't like usage of _M_IX86. Common in this file seems #ifndef _WIN64 > for 32-bit windows > > dependent code. But don't care here. > > > > Best regards, > > Goetz. > > > > > >> -----Original Message----- > >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > >> Behalf Of Baesken, Matthias > >> Sent: Mittwoch, 16. Mai 2018 15:13 > >> To: Doerr, Martin ; 'hotspot- > >> dev at openjdk.java.net' ; hotspot- > >> runtime-dev at openjdk.java.net > >> Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > >> > >> Ping : could I get a second review ? > >> > >> Bug : > >> https://bugs.openjdk.java.net/browse/JDK-8202427 > >> Change : > >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > >> > >> > >> > We could also just print the MB value , let's see what other think. > >> > Another option might be to have a flexible output (kB for smaller > >> memory > >> > values , MB (or GB) for larger ) ? > >> > >> Martin suggested to just print the MB values, what do you think ? > >> > >> > >> Thanks, Matthias > >> > >> > >> > -----Original Message----- > >> > From: Baesken, Matthias > >> > Sent: Mittwoch, 2. Mai 2018 13:00 > >> > To: Doerr, Martin ; 'hotspot- > >> > dev at openjdk.java.net' ; hotspot- > >> > runtime-dev at openjdk.java.net > >> > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > Windows > >> > > >> > Hi Martin, thanks for your input . > >> > I add hotspot-runtime-dev . > >> > > >> > > > >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe > other > >> > > reviewers would like to comment on this, too. > >> > > > >> > > >> > We could also just print the MB value , let's see what other think. > >> > Another option might be to have a flexible output (kB for smaller > >> memory > >> > values , MB (or GB) for larger ) ? > >> > > >> > Best regards, Matthias > >> > > >> > > >> > > -----Original Message----- > >> > > From: Doerr, Martin > >> > > Sent: Mittwoch, 2. Mai 2018 12:53 > >> > > To: Baesken, Matthias ; 'hotspot- > >> > > dev at openjdk.java.net' > >> > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > Windows > >> > > > >> > > Hi Matthias, > >> > > > >> > > looks like a nice enhancement. We can get substantially more > >> information. > >> > > > >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe > other > >> > > reviewers would like to comment on this, too. > >> > > > >> > > We should have line breaks. > >> > > > >> > > Best regards, > >> > > Martin > >> > > > >> > > > >> > > -----Original Message----- > >> > > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] > On > >> > > Behalf Of Baesken, Matthias > >> > > Sent: Montag, 30. April 2018 16:53 > >> > > To: 'hotspot-dev at openjdk.java.net' dev at openjdk.java.net> > >> > > Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows > >> > > > >> > > On Windows, > >> > > the os::print_memory_info misses a few memory-related infos that > >> might > >> > > be interesting : > >> > > - current and peak WorkingSet size (= physical memory assigned to > the > >> > > process) > >> > > - current and peak commit charge (also known as "private bytes" in > >> > Windows > >> > > tools) > >> > > - on 32bit Windows : > >> > > user-mode portion/free user mode portion of virtual address-space > >> > > (Total/AvailVirtual) because it shows how close do we get to the 2-4 > GB > >> per > >> > > process border. > >> > > > >> > > > >> > > - the current naming of "swap/free-swap" memory is a bit misleading; > >> > > in the Windows world swap is a special page file used for UWP apps. > >> > > (see Windows Internals : "The swap file" (chapter 5 Memory > >> > management) > >> > > Windows 8 added another page file called a swap file. It is ... > exclusively > >> > used > >> > > for UWP (Universal Windows Platform) apps. > >> > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the > UWP > >> > > related values, it is about page file sizes > >> > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just > >> virtual > >> > > memory might be more appropriate). > >> > > > >> > > > >> > > https://msdn.microsoft.com/de- > >> > > de/library/windows/desktop/aa366770(v=vs.85).aspx > >> > > > >> > > documents it in the following way: > >> > > ullTotalPageFile: > >> > > The current committed memory limit for the system or the current > >> process, > >> > > whichever is smaller,... > >> > > > >> > > ullAvailPageFile : > >> > > The maximum amount of memory the current process can commit, in > >> > bytes. > >> > > This value is equal to or smaller than the system-wide available > commit > >> > value > >> > > > >> > > > >> > > > >> > > Aditionally I suggest having output of the various memory-values in M > >> > > (megabyte) as well , the k (kilobyte) output sometimes gives very > high > >> and > >> > > unreadable numbers). > >> > > > >> > > > >> > > Could you please review my change ? > >> > > > >> > > > >> > > Bug : > >> > > > >> > > https://bugs.openjdk.java.net/browse/JDK-8202427 > >> > > > >> > > > >> > > Change : > >> > > > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > >> > > > >> > > > >> > > Thanks, Matthias > >> > > > > From erik.osterlund at oracle.com Thu May 17 15:38:11 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 17 May 2018 17:38:11 +0200 Subject: 8203341: Add a safepoint-aware Semaphore In-Reply-To: <2f8412d5-2fe4-2c8f-077c-cf96e8f9bebf@oracle.com> References: <2f8412d5-2fe4-2c8f-077c-cf96e8f9bebf@oracle.com> Message-ID: <5AFDA1E3.6010401@oracle.com> Hi Stefan, I wonder if it would make sense to let Semaphore::wait have a bool safepoint_check, similar to Mutex (possibly with a default value), instead of introducing a new class with the same API, that only differs in this property. Thanks, /Erik On 2018-05-17 09:46, Stefan Karlsson wrote: > Hi all, > > Please review this patch to add a safepoint-aware Semaphore (and > refactor the semaphore usage in thread-local handshakes). > > http://cr.openjdk.java.net/~stefank/8203341/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8203341 > > Both ZGC and the thread-local handshakes need to move the JavaThread > to a blocked state before waiting on a Semaphore. I propose that we > introduce a new SemaphoreSafepointAware wrapper around Semaphore. > > There are two things to consider with this patch: > > 1) In ZGC, where this patch originates from, we set the JavaThread's > osthread state with osthread->set_state(CONDVAR_WAIT) (using > OSThreadWaitState). This means that we get the following printed when > the threads are dumped: "waiting on condition ". I've kept this > behavior for the SemaphoreSafepointAware class. This means that we now > change the osthread state for JavaThreads blocking in > HandshakeState::process_self_inner. > > 2) The thread-local handshakes uses trywait before entering wait: > if (!_semaphore.trywait()) { > - ThreadBlockInVM tbivm(thread); > _semaphore.wait(); > } > > At the time when SemaphoreSafepointAware was written, we didn't have > trywait on all platforms, now we do. Maybe we should always do the > trywait dance inside SemaphoreSafepointAware::wait() ? > > inline void SemaphoreSafepointAware::wait() { > if (Thread::current()->is_Java_thread()) { > if (!_semaphore.trywait) { > JavaThread* thread = JavaThread::current(); > > // Prepare to block and allow safepoints while blocked > ThreadBlockInVM tbivm(thread); > OSThreadWaitState osts(thread->osthread(), false /* not in > Object.wait() */); > > // Wait for value > _semaphore.wait(); > } > > Thanks, > StefanK From stefan.karlsson at oracle.com Thu May 17 15:59:51 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 17 May 2018 17:59:51 +0200 Subject: 8203341: Add a safepoint-aware Semaphore In-Reply-To: <5AFDA1E3.6010401@oracle.com> References: <2f8412d5-2fe4-2c8f-077c-cf96e8f9bebf@oracle.com> <5AFDA1E3.6010401@oracle.com> Message-ID: On 2018-05-17 17:38, Erik ?sterlund wrote: > Hi Stefan, > > I wonder if it would make sense to let Semaphore::wait have a bool > safepoint_check, similar to Mutex (possibly with a default value), > instead of introducing a new class with the same API, that only > differs in this property. That sounds reasonable to me. I also need to change my patch so that the thread-local handshake code pass in the existing JavaThread, so that we don't have to call Thread::current()->is_JavaThread() unnecessarily. Thanks, StefanK > > Thanks, > /Erik > > On 2018-05-17 09:46, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to add a safepoint-aware Semaphore (and >> refactor the semaphore usage in thread-local handshakes). >> >> http://cr.openjdk.java.net/~stefank/8203341/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8203341 >> >> Both ZGC and the thread-local handshakes need to move the JavaThread >> to a blocked state before waiting on a Semaphore. I propose that we >> introduce a new SemaphoreSafepointAware wrapper around Semaphore. >> >> There are two things to consider with this patch: >> >> 1) In ZGC, where this patch originates from, we set the JavaThread's >> osthread state with osthread->set_state(CONDVAR_WAIT) (using >> OSThreadWaitState). This means that we get the following printed when >> the threads are dumped: "waiting on condition ". I've kept this >> behavior for the SemaphoreSafepointAware class. This means that we >> now change the osthread state for JavaThreads blocking in >> HandshakeState::process_self_inner. >> >> 2) The thread-local handshakes uses trywait before entering wait: >> ?? if (!_semaphore.trywait()) { >> -??? ThreadBlockInVM tbivm(thread); >> ???? _semaphore.wait(); >> ?? } >> >> At the time when SemaphoreSafepointAware was written, we didn't have >> trywait on all platforms, now we do. Maybe we should always do the >> trywait dance inside SemaphoreSafepointAware::wait() ? >> >> inline void SemaphoreSafepointAware::wait() { >> ? if (Thread::current()->is_Java_thread()) { >> ??? if (!_semaphore.trywait) { >> ????? JavaThread* thread = JavaThread::current(); >> >> ????? // Prepare to block and allow safepoints while blocked >> ????? ThreadBlockInVM tbivm(thread); >> ????? OSThreadWaitState osts(thread->osthread(), false /* not in >> Object.wait() */); >> >> ????? // Wait for value >> ???? _semaphore.wait(); >> ?? } >> >> Thanks, >> StefanK > From Alan.Bateman at oracle.com Thu May 17 16:27:03 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 17 May 2018 17:27:03 +0100 Subject: RFR: JDK-8190187: Follow-up. In-Reply-To: References: Message-ID: <460d4e0b-ee30-3d32-b482-2efb571f18db@oracle.com> On 17/05/2018 13:57, Adam Farley8 wrote: > Hi Alan, > > As requested. > > Any ideas? The issue here is that the patch is just one part of changing long standing behavior. There is still work needed to flush out other interactions in the JVM TI spec, e.g. the multiple agent case and whether the Agent_OnUnLoad should be invoked for agents loaded before some other agent returns JNI_SILENT_EXIT. Also the late-binding agent case needs examination to see if wording is needed for the case that Agent_OnAttach returns this status. There is also an update needed to the JNI spec as we discussed. As per the mails on core-libs-dev, it's not clear that it's worth it. In any case, I don't know if anyone cycles to work with you on this. I think the help you are looking for is to ensure that all the spec issues are covered, maybe help with the tests and testing, and then the process work that is the CSR and release note. -Alan From jcbeyler at google.com Thu May 17 17:29:06 2018 From: jcbeyler at google.com (JC Beyler) Date: Thu, 17 May 2018 10:29:06 -0700 Subject: RFR 8171119: Low-Overhead Heap Profiling In-Reply-To: <5AFD4FDB.9090509@oracle.com> References: <5cdb2496-bad5-fa4a-f98e-37ae6a853f6b@oracle.com> <56bbfb13-3527-669a-b155-d699be1ef42e@oracle.com> <3c1d88fd-a3bc-350a-15fb-ec33118b7626@oracle.com> <5AFD4FDB.9090509@oracle.com> Message-ID: Hi Erik, Thanks for the review. I changed this method to: HeapWord* ThreadLocalAllocBuffer::allocate_sampled_object(size_t size) { - Thread* thread = myThread(); - thread->tlab().set_back_allocation_end(); - HeapWord* result = thread->tlab().allocate(size); + set_back_allocation_end(); + HeapWord* result = allocate(size); if (result) { - thread->heap_sampler().check_for_sampling(result, size * HeapWordSize, _bytes_since_last_sample_point); - thread->tlab().set_sample_end(); + myThread()->heap_sampler().check_for_sampling(result, size * HeapWordSize, _bytes_since_last_sample_point); + set_sample_end(); } I still have a myThread() call due to calling the heap_sampler but moved it below in the if (result) and then did the calls to x() directly instead of thread->tlab()->x() as requested. Thanks for the review and your help! I'll post a new webrev in a bit after it builds and passes my HeapMonitor tests, Jc On Thu, May 17, 2018 at 2:45 AM Erik ?sterlund wrote: > Hi JC, > > I have looked through your changes. > > In threadLocalAllocBuffer.cpp: > > 349 HeapWord* ThreadLocalAllocBuffer::allocate_sampled_object(size_t > size) { > 350 Thread* thread = myThread(); > 351 thread->tlab().set_back_allocation_end(); > 352 HeapWord* result = thread->tlab().allocate(size); > 353 > 354 if (result) { > 355 thread->heap_sampler().check_for_sampling(result, size * > HeapWordSize, _bytes_since_last_sample_point); > 356 thread->tlab().set_sample_end(); > 357 } > 358 > 359 return result; > 360 } > > ...it looks like you are fetching myThread() from the TLAB, but use this > fetched thread only to fetch the TLAB. But That TLAB is in fact "this" in > this context. I would prefer not calling x() instead of thread->tlab()->x(). > > I don't need another webrev for this. The rest looks good to me. Thank you > for doing this, and thank you for breaking out the per-thread logic into > its own class. > > Thanks, > /Erik > > On 2018-05-15 18:57, JC Beyler wrote: > > Hi Thomas, > > Thanks for the review! I've done your two fixes for the next webrev. I'll > perhaps wait for the next review to combine webrev updates? Your two change > requests were minimal it might make sense to wait before re-generating a > new webrev. > > Anybody else have any comments/suggestions? > > Here are the latest links: > Webrev: http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.20/ > JEP-331 Issue: https://bugs.openjdk.java.net/browse/JDK-8171119 > CSR: https://bugs.openjdk.java.net/browse/JDK-8194905 > Test Plan: https://bugs.openjdk.java.net/browse/JDK-8202566 > > Thanks again all! > Jc > > On Tue, May 15, 2018 at 12:23 AM Thomas Schatzl > wrote: > >> Hi, >> >> On Mon, 2018-05-14 at 13:02 -0700, JC Beyler wrote: >> > Hi Robbin and all, >> > >> > Thank you for your continuous help! >> > >> > Done then! Here is the new webrev: >> > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.20/ >> > >> > and the incremental is: >> > http://cr.openjdk.java.net/~jcbeyler/8171119/heap_event.19_20/ >> > >> > Thanks again all! >> > Jc >> > >> >> looks good to me. >> >> Two minor issues that you might want to fix before pushing: >> >> - collectedHeap.hpp: The declaration of >> CollectedHeap::allocate_memory() should be private. >> >> - collectedHeap.inline.hpp: in CollectedHeap::allocate_memory() there >> is this completely duplicated code which you might factor out into >> another (private) method: >> >> if (init_memory) { >> obj = ... >> } else { >> obj = ... >> } >> post_setup(klass, ...); >> >> Thanks, >> Thomas >> >> > From bob.vandette at oracle.com Thu May 17 18:12:46 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Thu, 17 May 2018 14:12:46 -0400 Subject: RFR (8u): JDK-8146115: Improve docker container detection and resource configuration usage In-Reply-To: <3968d009-c1a2-87e7-bc22-70c348ee5b69@oracle.com> References: <3968d009-c1a2-87e7-bc22-70c348ee5b69@oracle.com> Message-ID: <69915730-5963-41AE-AB75-B0C223E39035@oracle.com> The backport of my changes look pretty good. If the new PrintContainerInfo option is only referenced on Linux platforms, you might want to move it to globals_linux.hpp. Is there a reason PrintActiveCpus is a diagnostic flag but PrintContainerInfo is not? Is it acceptable to add these new VM flags in a backport that won?t be supported in the latest release? Bob. > On May 15, 2018, at 4:46 PM, Poonam Parhar wrote: > > Hello, > > Please review the docker container support changes backported to JDK 8u. These changes include the backport of the following enhancement and the follow-on bug fixes done on top of that in jdk 10 and 11. > > Webrev: http://cr.openjdk.java.net/~poonam/8146115/webrev.00/ > > Enhancement:JDK-8146115: Improve docker container detection and resource configuration usage > > The changes also include the fixes for the following two bugs: > Bug JDK-8186248: Allow more flexibility in selecting Heap % of available RAM > BugJDK-8190283: Default heap sizing options select a MaxHeapSize larger than available physical memory in some cases > > These changes add a new JVM option 'PrintContainerInfo' for tracing container related information which is specific to jdk 8. > > Testing results (with -XX:+UnlockDiagnosticVMOptions -XX:+PrintContainerInfo -XX:+PrintActiveCpus JVM options): > -------------- > poonam at poonam-VirtualBox:~/docker-image$ java TestCPUsMemory > Number of Processors: 3 > Max Memory: 921174016 > poonam at poonam-VirtualBox:~/docker-image$ sudo docker run --cpus 1 -m1024m --rm myimage > [sudo] password for poonam: > WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. > OSContainer::init: Initializing Container Support > Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes > Memory Limit is: 1073741824 > active_processor_count: sched_getaffinity processor count: 3 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: 100000 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 1024 > CPU Quota count based on quota/period: 1 > OSContainer::active_processor_count: 1 > active_processor_count: determined by OSContainer: 1 > active_processor_count: sched_getaffinity processor count: 3 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: 100000 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 1024 > CPU Quota count based on quota/period: 1 > OSContainer::active_processor_count: 1 > active_processor_count: determined by OSContainer: 1 > Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes > Memory Limit is: 1073741824 > total container memory: 1073741824 > active_processor_count: sched_getaffinity processor count: 3 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: 100000 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 1024 > CPU Quota count based on quota/period: 1 > OSContainer::active_processor_count: 1 > active_processor_count: determined by OSContainer: 1 > active_processor_count: sched_getaffinity processor count: 3 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: 100000 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 1024 > CPU Quota count based on quota/period: 1 > OSContainer::active_processor_count: 1 > active_processor_count: determined by OSContainer: 1 > > active_processor_count: sched_getaffinity processor count: 3 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: 100000 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 1024 > CPU Quota count based on quota/period: 1 > OSContainer::active_processor_count: 1 > active_processor_count: determined by OSContainer: 1 > Number of Processors: 1 > Max Memory: 259522560 > > poonam at poonam-VirtualBox:~/docker-image$ sudo docker run --cpu-shares 2048 -m1024m --rm myimage > WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. > OSContainer::init: Initializing Container Support > Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes > Memory Limit is: 1073741824 > active_processor_count: sched_getaffinity processor count: 3 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: -1 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 2048 > CPU Share count based on shares: 2 > OSContainer::active_processor_count: 2 > active_processor_count: determined by OSContainer: 2 > active_processor_count: sched_getaffinity processor count: 3 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: -1 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 2048 > CPU Share count based on shares: 2 > OSContainer::active_processor_count: 2 > active_processor_count: determined by OSContainer: 2 > Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes > Memory Limit is: 1073741824 > total container memory: 1073741824 > Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes > Memory Limit is: 1073741824 > total container memory: 1073741824 > active_processor_count: sched_getaffinity processor count: 3 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: -1 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 2048 > CPU Share count based on shares: 2 > OSContainer::active_processor_count: 2 > active_processor_count: determined by OSContainer: 2 > active_processor_count: sched_getaffinity processor count: 3 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: -1 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 2048 > CPU Share count based on shares: 2 > OSContainer::active_processor_count: 2 > active_processor_count: determined by OSContainer: 2 > > active_processor_count: sched_getaffinity processor count: 3 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: -1 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 2048 > CPU Share count based on shares: 2 > OSContainer::active_processor_count: 2 > active_processor_count: determined by OSContainer: 2 > Number of Processors: 2 > Max Memory: 259522560 > > poonam at poonam-VirtualBox:~/docker-image$ sudo docker run --cpuset-cpus 0-1 -m1024m --rm myimage > WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. > OSContainer::init: Initializing Container Support > Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes > Memory Limit is: 1073741824 > active_processor_count: sched_getaffinity processor count: 2 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: -1 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 1024 > OSContainer::active_processor_count: 2 > active_processor_count: determined by OSContainer: 2 > active_processor_count: sched_getaffinity processor count: 2 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: -1 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 1024 > OSContainer::active_processor_count: 2 > active_processor_count: determined by OSContainer: 2 > Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes > Memory Limit is: 1073741824 > total container memory: 1073741824 > Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes > Memory Limit is: 1073741824 > total container memory: 1073741824 > active_processor_count: sched_getaffinity processor count: 2 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: -1 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 1024 > OSContainer::active_processor_count: 2 > active_processor_count: determined by OSContainer: 2 > active_processor_count: sched_getaffinity processor count: 2 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: -1 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 1024 > OSContainer::active_processor_count: 2 > active_processor_count: determined by OSContainer: 2 > > active_processor_count: sched_getaffinity processor count: 2 > Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us > CPU Quota is: -1 > Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us > CPU Period is: 100000 > Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares > CPU Shares is: 1024 > OSContainer::active_processor_count: 2 > active_processor_count: determined by OSContainer: 2 > Number of Processors: 2 > Max Memory: 259522560 > ------------------ > > Thanks, > Poonam > From kim.barrett at oracle.com Thu May 17 19:17:52 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 17 May 2018 15:17:52 -0400 Subject: RFR: 8202813: Move vm_weak processing from SystemDictionary::do_unloading to WeakProcessor In-Reply-To: <6dc7d3f8-dd9f-0990-860d-3b9423c1e271@oracle.com> References: <10a99bf5-d3cc-9d5b-8a38-5ba1672f9f8b@oracle.com> <4E15250A-9872-4195-892B-C1E0078A80FD@oracle.com> <6dc7d3f8-dd9f-0990-860d-3b9423c1e271@oracle.com> Message-ID: > On May 17, 2018, at 3:13 AM, Stefan Johansson wrote: > > > > On 2018-05-17 01:51, Kim Barrett wrote: >>> On May 16, 2018, at 8:42 AM, Stefan Johansson wrote: >>> >>> Hi Kim, >>> >>> On 2018-05-08 21:34, Kim Barrett wrote: >>>> Please review this change to move the clearing of dead entries in >>>> vm_weak_oop_storage from SystemDictionary::do_unloading to preceeding >>>> calls to WeakProcessor::weak_oops_do. >>>> This involves conditionalizing WeakProcessor invocations, to allow >>>> different calls to process different subsets of the native weak >>>> references. For example, vm_weak_oop_storage is treated as strong >>>> rather than weak when ClassUnloading is disabled. >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8202813 >>>> Webrev: >>>> http://cr.openjdk.java.net/~kbarrett/8202813/open.00/ >>> Thanks for doing this work to unify weak root processing. >> Thanks for looking at it. >>> I might be missing the bigger picture, so before starting to commenting the code I have a couple of questions: >>> 1. With this patch SystemDictionary::vm_weak_oop_storage is now processed both by the WeakProcessor in weak_oops_do and the SystemDictionary in oops_do() and roots_oops_do(). Shouldn't all processing be moved to the WeakProcessor to make it clear who is responsible for doing the processing? Or why do we want to treat do_unloading special? >> Yes, it should be moved completely. Baby steps. >> I started to change (remove, actually, and update callers) >> SystemDictionary::oops_do, but discovered jfr was using it, and didn't >> want to touch that in the middle of the jfr review. >> And there are some preliminary cleanups I'm working on around >> roots_oops_do, but haven't finished them yet. > I agree that baby steps is often good, but now when JFR is in would it make sense to include more in this change? OK, I can do that. > >>> 2. For this patch I don't see the need for the PhaseSet class and the serial and parallel phases distinction. Couldn't we just pass in a bool to weak_oops_do saying if we should handle the vm_weak_oop_storage or not? I understand that for upcoming patches this might be needed, but then I think this should be added in that change. >> I agree it's over-engineered for the needs of this change set. >> However, further conditionalization beyond a boolean for >> class-unloading is going to be needed RSN for the new StringTable >> (based on the new concurrent hash table + oopstorage). StringTable >> processing is another place where the various GCs and their sub-parts >> vary about where it's treated as strong or weak. That should be >> showing up shortly, and is being worked on by folks who aren't me. So >> unless you want me to separate that out into a separate change that >> will likely be my next one to support that work... > I would like it to be added when needed, to make sure we only include the parts we really need. Right now it would be hard for me to review those parts since I can't verify that they work as intended. I'm mostly concerned about the parts around serial/parallel phases, which is currently unused. Since you and (privately) Coleen are both complaining about this, I?ll reconsider. Withdrawing this change for now. From leonid.mesnik at oracle.com Thu May 17 19:59:46 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Thu, 17 May 2018 12:59:46 -0700 Subject: 8199271: [TESTBUG] open source VM testbase stress tests In-Reply-To: <58bfdd2c-9163-2464-93bd-9ee1e265d97e@oracle.com> References: <1BC403EC-BDF0-4CA2-B87F-B38A92B6765F@oracle.com> <5AF5C07B.1080909@oracle.com> <8BA2B023-E947-4EF8-9D97-EBA28BAA07E2@oracle.com> <58bfdd2c-9163-2464-93bd-9ee1e265d97e@oracle.com> Message-ID: <175FD6F0-AAF1-4985-A899-002320444F3B@oracle.com> Thank you for review. Leonid > On May 16, 2018, at 8:48 PM, serguei.spitsyn at oracle.com wrote: > > Hi Leonid, > > Looks good to me too. > > Thanks, > Serguei > > > On 5/14/18 14:04, Leonid Mesnik wrote: >> Misha >> >> Thank you for review. I still need one more review from 'R'eviewer. >> >> Leonid >> >>> On May 11, 2018, at 9:10 AM, Mikhailo Seledtsov wrote: >>> >>> Looks good to me, >>> >>> Misha >>> >>> On 5/8/18, 2:23 PM, Leonid Mesnik wrote: >>>> Hi >>>> >>>> Please review this change open sourcing vm testbase stress tests. These tests have been developed a long time ago for internal test harness and don't looks very nice now. >>>> They are open sourced with minimal changes only. The long term plan is to review and improve them moving out of vmTestbase directory in corresponding components. >>>> >>>> The fix introduce vmTestbase_nsk_stress test group and have make files changes required to build native part of tests. >>>> >>>> I've verified that "make -- run-test TEST=:vmTestbase_nsk_stress" pass on my linux locally. >>>> >>>> webrev: http://cr.openjdk.java.net/~lmesnik/8199271/webrev.00/ >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8199271 > From bsrbnd at gmail.com Thu May 17 20:10:37 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Thu, 17 May 2018 22:10:37 +0200 Subject: RFR 8202835: jfr/event/os/TestSystemProcess.java fails on missing events Message-ID: Hi, A recent change in 'os::readdir()' to use 'readdir' instead of the deprecated 'readdir_r' implies to copy the returned 'dirent' if necessary because 'readdir' uses its own buffer and not the one provided by the caller as it was done before with 'readdir_r'. The following fix is only intended to correct 'ProcessIterator', further improvements will be addressed as part of: https://bugs.openjdk.java.net/browse/JDK-8202353 Tier1 is OK. Any comment is welcome. Thanks, Bernard JBS: https://bugs.openjdk.java.net/browse/JDK-8202835 Suggested fix: diff -r 8e4fcfb4cfe4 src/hotspot/os/linux/os_perf_linux.cpp --- a/src/hotspot/os/linux/os_perf_linux.cpp Thu May 17 10:32:26 2018 +0200 +++ b/src/hotspot/os/linux/os_perf_linux.cpp Thu May 17 20:33:33 2018 +0200 @@ -721,6 +721,8 @@ char _exeName[PATH_MAX]; char _exePath[PATH_MAX]; + static const size_t _entry_size = sizeof(struct dirent) + NAME_MAX + 1; + ProcessIterator(); ~ProcessIterator(); bool initialize(); @@ -921,6 +923,7 @@ _valid = false; return OS_ERR; } + memcpy(_entry, entry, _entry_size); } while(!is_valid_entry(_entry)); _valid = true; @@ -935,7 +938,7 @@ bool SystemProcessInterface::SystemProcesses::ProcessIterator::initialize() { _dir = opendir("/proc"); - _entry = (struct dirent*)NEW_C_HEAP_ARRAY(char, sizeof(struct dirent) + NAME_MAX + 1, mtInternal); + _entry = (struct dirent*)NEW_C_HEAP_ARRAY(char, _entry_size, mtInternal); if (NULL == _entry) { return false; } diff -r 8e4fcfb4cfe4 test/jdk/ProblemList.txt --- a/test/jdk/ProblemList.txt Thu May 17 10:32:26 2018 +0200 +++ b/test/jdk/ProblemList.txt Thu May 17 20:33:33 2018 +0200 @@ -829,5 +829,4 @@ jdk/jfr/event/io/TestInstrumentation.java 8202142 generic-all jdk/jfr/event/sampling/TestNative.java 8202142 generic-all -jdk/jfr/event/os/TestSystemProcess.java 8202835 linux-all -jdk/jfr/event/runtime/TestBiasedLockRevocationEvents.java 8203237 generic-all \ No newline at end of file +jdk/jfr/event/runtime/TestBiasedLockRevocationEvents.java 8203237 generic-all From kim.barrett at oracle.com Thu May 17 20:33:51 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 17 May 2018 16:33:51 -0400 Subject: RFR: 8202863: Rename OopStorage inner collection classes Message-ID: [Resending hotspot-dev too, so Coleen will see it, since she requested this change.] Please review this simple renaming of some private inner classes of OopStorage. The renamings were suggested during the review of JDK-8200557, but were deferred to a separate change to avoid adding complexity to that review. The following classes are being renamed: BlockArray => ActiveArray BlockList => AllocateList BlockEntry => AllocateEntry CR: https://bugs.openjdk.java.net/browse/JDK-8202863 Webrev: http://cr.openjdk.java.net/~kbarrett/8202863/open.00/ Testing: Mach5 CI equivalent. From coleen.phillimore at oracle.com Thu May 17 20:44:44 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 17 May 2018 16:44:44 -0400 Subject: RFR: 8202863: Rename OopStorage inner collection classes In-Reply-To: References: Message-ID: Looks good! Coleen On 5/17/18 4:33 PM, Kim Barrett wrote: > [Resending hotspot-dev too, so Coleen will see it, since she requested this change.] > > Please review this simple renaming of some private inner classes of > OopStorage. The renamings were suggested during the review of > JDK-8200557, but were deferred to a separate change to avoid adding > complexity to that review. The following classes are being renamed: > > BlockArray => ActiveArray > BlockList => AllocateList > BlockEntry => AllocateEntry > > CR: > https://bugs.openjdk.java.net/browse/JDK-8202863 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8202863/open.00/ > > Testing: > Mach5 CI equivalent. > From kim.barrett at oracle.com Thu May 17 21:34:34 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 17 May 2018 17:34:34 -0400 Subject: RFR: 8202863: Rename OopStorage inner collection classes In-Reply-To: References: Message-ID: > On May 17, 2018, at 4:44 PM, coleen.phillimore at oracle.com wrote: > > Looks good! > Coleen Thanks. > > On 5/17/18 4:33 PM, Kim Barrett wrote: >> [Resending hotspot-dev too, so Coleen will see it, since she requested this change.] >> >> Please review this simple renaming of some private inner classes of >> OopStorage. The renamings were suggested during the review of >> JDK-8200557, but were deferred to a separate change to avoid adding >> complexity to that review. The following classes are being renamed: >> >> BlockArray => ActiveArray >> BlockList => AllocateList >> BlockEntry => AllocateEntry >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8202863 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8202863/open.00/ >> >> Testing: >> Mach5 CI equivalent. From kevin.walls at oracle.com Thu May 17 22:21:47 2018 From: kevin.walls at oracle.com (Kevin Walls) Date: Thu, 17 May 2018 23:21:47 +0100 Subject: [8u] 8203349: 8u hotspot should recognise later Windows compilers Message-ID: <672231ce-9a74-af4e-88db-8044e4a74ebe@oracle.com> Hi, I'd like to get a review for this 8u/hotspot build change for Windows, to loosen the restriction on what compilers we can use. JBS: 8203349: 8u hotspot should recognise later Windows compilers https://bugs.openjdk.java.net/browse/JDK-8203349 Webrev: http://cr.openjdk.java.net/~kevinw/8203349/webrev.00/ Changes in the places we (sanity) check version numbers, set some compile options, and expand the check in vm.make to make sure we put the precompiled object? _build_pch_file.obj on the jvm link command. In compile.make I added blocks for VS2013 and 17, and left them as separate, duplicated, blocks of settings to make them easier to change independently. This doesn't change anything about what is "supported" or documented as working. Thanks! Kevin From erik.joelsson at oracle.com Thu May 17 22:32:58 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Thu, 17 May 2018 15:32:58 -0700 Subject: [8u] 8203349: 8u hotspot should recognise later Windows compilers In-Reply-To: <672231ce-9a74-af4e-88db-8044e4a74ebe@oracle.com> References: <672231ce-9a74-af4e-88db-8044e4a74ebe@oracle.com> Message-ID: Hello Kevin, In JDK 11, vm_version.cpp we have _MSC_VER == 1900 translating to VS2015 and 1912 to VS2017. Is it really 1900 in nmake? Otherwise I think this looks ok. /Erik On 2018-05-17 15:21, Kevin Walls wrote: > Hi, > > I'd like to get a review for this 8u/hotspot build change for > Windows, to loosen the restriction on what compilers we can use. > > JBS: > 8203349: 8u hotspot should recognise later Windows compilers > https://bugs.openjdk.java.net/browse/JDK-8203349 > > Webrev: http://cr.openjdk.java.net/~kevinw/8203349/webrev.00/ > > Changes in the places we (sanity) check version numbers, set some compile > options, and expand the check in vm.make to make sure we put the > precompiled > object? _build_pch_file.obj on the jvm link command. > > In compile.make I added blocks for VS2013 and 17, and left them as > separate, > duplicated, blocks of settings to make them easier to change > independently. > > This doesn't change anything about what is "supported" or documented > as working. > > Thanks! > Kevin > > From paul.sandoz at oracle.com Fri May 18 01:41:14 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 17 May 2018 18:41:14 -0700 Subject: RFR: 8199064: Test applications/jcstress/other/Test.java#id1108 fails on Sparc In-Reply-To: <4A1B218C-66B9-433A-9BA9-BA495CDF4EA9@oracle.com> References: <4A1B218C-66B9-433A-9BA9-BA495CDF4EA9@oracle.com> Message-ID: <5830D038-FF0D-40FE-BA69-BF1D40AFD94B@oracle.com> Looks good. I was initially taken aback by all the changes but quickly realized they are all generated, the important change being in TestGenerator.java. Do you have a sense of the range of times based on the JcstressGroup? I wonder if we could be a little smarter on timeout value. Paul. > On May 16, 2018, at 3:14 PM, Leonid Mesnik wrote: > > Hi > > Could you please review following patch which increase timeout for jcstress test wrappers in jtreg. > > The default jtreg timeout is not enough for a lot of jcstress tests. The longest tests take more then 1800 seconds on typical VM used for testing (2 CPU/16GB of Memory) with default VM options. > These tests are not executed in tier1/2 testing so execution time in the case if test really hang is not very critical. I just set timeout to 2400 seconds for all test wrappers. > The timeoutfactor could be used to adjust timeouts for slow platforms and/or VM options. > > > webrev: http://cr.openjdk.java.net/~lmesnik/8199064/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8199064 > > Leonid From david.holmes at oracle.com Fri May 18 04:17:08 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 18 May 2018 14:17:08 +1000 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> <0a09b984-a617-af45-4033-43086fe818a4@redhat.com> Message-ID: <64914635-0d59-7aed-d52c-7a5a4fc16658@oracle.com> It doesn't look right to me that vm_version_ext_zero.cpp is zero specific - it really should be CPU specific. The zero version is setting up a bunch of dummy values for threads/cores/sockets etc, but if something else uses them in calculations then you'd get some weird numbers. David On 18/05/2018 12:38 AM, Severin Gehwolf wrote: > Hi Martin, Aleksey, > > Ugh, sorry, my mistake :-/ Thanks for the reviews! > > This is what I'm going to push once jdk-submit comes back clean: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.04/ > > Thanks, > Severin > > On Thu, 2018-05-17 at 14:17 +0000, Doerr, Martin wrote: >> Hi Severin, >> >> looks good to me with Aleksey's patch. Thanks for waiting. >> >> Best regards, >> Martin >> >> >> -----Original Message----- >> From: Aleksey Shipilev [mailto:shade at redhat.com] >> Sent: Donnerstag, 17. Mai 2018 16:14 >> To: Severin Gehwolf ; Doerr, Martin ; hotspot-dev >> Subject: Re: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) >> >> On 05/17/2018 04:02 PM, Severin Gehwolf wrote: >>> On Thu, 2018-05-17 at 15:27 +0200, Aleksey Shipilev wrote: >>>> On 05/17/2018 03:12 PM, Severin Gehwolf wrote: >>>>> Hi Martin, Aleksey, >>>>> >>>>> Since JDK-8203288 has been pushed here is the latest webrev for Zero. >>>>> It no longer contains changes in os_perf_linux.cpp. >>>>> >>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.02/ >>>>> >>>>> How does it look? >>>> >>>> Looks good, but I used parentheses inconsistently with the rest of Hotspot in the original patch: >>>> >>>> #if (defined X86 && !defined ZERO) >>>> >>>> Should be: >>>> >>>> #if defined(X86) && !defined (ZERO) >>> >>> OK. Here it goes: >>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.03/ >> >> Notice the parentheses, it is done like that in other places: >> >> #if defined(X86) && !defined(ZERO) >> >> Apply this patch to correct: >> http://cr.openjdk.java.net/~shade/8203287/parent.patch >> >> I checked it builds with x86_64 and Zero. >> >> -Aleksey >> From goetz.lindenmaier at sap.com Fri May 18 05:45:07 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 18 May 2018 05:45:07 +0000 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> Message-ID: <1420518caee1469bb7ce92fbebf0b6a9@sap.com> Hi Matthias, The output looks good now. Reviewed. Could you still please break the lines of the st->print statements in the code? Just after the format string for example. I don't need a new webrev for that. Best regards, Goetz > -----Original Message----- > From: Baesken, Matthias > Sent: Thursday, May 17, 2018 5:20 PM > To: Thomas St?fe ; Lindenmaier, Goetz > > Cc: Doerr, Martin ; hotspot-dev at openjdk.java.net; > hotspot-runtime-dev at openjdk.java.net > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows > > > > > I also think that one number is enough. Adaptive numbers would be > great. > > > Hi Thomas/Goetz/Martin , thanks for looking into it. > Adaptive numbers could be done in a follow-up as suggested by Thomas , > this would also touch the other platforms . > > I added more line feeds to the output and reduced output to just printing > M . > > > Example output from a Windows desktop PC : > > Memory: 4k page, system-wide physical 32520M (20165M free) > TotalPageFile size 34568M (AvailPageFile size 18170M) > current process WorkingSet (physical memory assigned to process): 79M, > peak: 79M > current process commit charge ("private bytes"): 148M, peak: 148M > > > new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.1/ > > > Best regards, Matthias > > > > > -----Original Message----- > > From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > > Sent: Mittwoch, 16. Mai 2018 16:35 > > To: Lindenmaier, Goetz > > Cc: Baesken, Matthias ; Doerr, Martin > > ; hotspot-dev at openjdk.java.net; hotspot- > > runtime-dev at openjdk.java.net > > Subject: Re: [RFR] 8202427: Enhance os::print_memory_info on Windows > > > > On Wed, May 16, 2018 at 3:48 PM, Lindenmaier, Goetz > > wrote: > > > Hi Matthias, > > > > > > I appreciate the additional information in the hs_err file. > > > Could you please add an example output (of the final version) to the bug > > description? > > > > > > As Martin, I would like more line feeds, both in the code and the output. > > > Currently, the output in the hs_err file looks like this (where the first line > is > > too long): > > > Memory: 4k page, system-wide physical 16776692k [16383M] (8795032k > > [8588M] free), TotalPageFile size 49949460k [48778M] (AvailPageFile size > > 38942152k [38029M]), user-mode portion of virtual address-space 2097024k > > [2047M] (6940k [6M] free) > > > current process WorkingSet (physical memory assigned to process): > > 991804k [968M], peak: 991804k [968M] > > > > > > current process commit charge ("private bytes"): 1523632k [1487M], > > peak: 1523632k [1487M] > > > > > > I also think that one number is enough. Adaptive numbers would be > great. > > > I know about > > > src/hotspot/share/memory/metaspace/metaspaceCommon.hpp: > > print_human_readable_size() > > > for this, but this seems a bit hidden in metaspace. > > > > > > > Guys, lets not worry about human readable numbers for now. We can > move > > print_human_readable_size() out from metaspace to something generic > > and use it to improve this code (and others) in a later patch. That > > was my original intention when adding print_human_readable_size(). > > > > The patch is good. Thanks for doing this. > > > > ..Thoams > > > > > I don't like usage of _M_IX86. Common in this file seems #ifndef _WIN64 > > for 32-bit windows > > > dependent code. But don't care here. > > > > > > Best regards, > > > Goetz. > > > > > > > > >> -----Original Message----- > > >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] > On > > >> Behalf Of Baesken, Matthias > > >> Sent: Mittwoch, 16. Mai 2018 15:13 > > >> To: Doerr, Martin ; 'hotspot- > > >> dev at openjdk.java.net' ; hotspot- > > >> runtime-dev at openjdk.java.net > > >> Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > Windows > > >> > > >> Ping : could I get a second review ? > > >> > > >> Bug : > > >> https://bugs.openjdk.java.net/browse/JDK-8202427 > > >> Change : > > >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > > >> > > >> > > >> > We could also just print the MB value , let's see what other think. > > >> > Another option might be to have a flexible output (kB for smaller > > >> memory > > >> > values , MB (or GB) for larger ) ? > > >> > > >> Martin suggested to just print the MB values, what do you think ? > > >> > > >> > > >> Thanks, Matthias > > >> > > >> > > >> > -----Original Message----- > > >> > From: Baesken, Matthias > > >> > Sent: Mittwoch, 2. Mai 2018 13:00 > > >> > To: Doerr, Martin ; 'hotspot- > > >> > dev at openjdk.java.net' ; hotspot- > > >> > runtime-dev at openjdk.java.net > > >> > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > > Windows > > >> > > > >> > Hi Martin, thanks for your input . > > >> > I add hotspot-runtime-dev . > > >> > > > >> > > > > >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe > > other > > >> > > reviewers would like to comment on this, too. > > >> > > > > >> > > > >> > We could also just print the MB value , let's see what other think. > > >> > Another option might be to have a flexible output (kB for smaller > > >> memory > > >> > values , MB (or GB) for larger ) ? > > >> > > > >> > Best regards, Matthias > > >> > > > >> > > > >> > > -----Original Message----- > > >> > > From: Doerr, Martin > > >> > > Sent: Mittwoch, 2. Mai 2018 12:53 > > >> > > To: Baesken, Matthias ; 'hotspot- > > >> > > dev at openjdk.java.net' > > >> > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > > Windows > > >> > > > > >> > > Hi Matthias, > > >> > > > > >> > > looks like a nice enhancement. We can get substantially more > > >> information. > > >> > > > > >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe > > other > > >> > > reviewers would like to comment on this, too. > > >> > > > > >> > > We should have line breaks. > > >> > > > > >> > > Best regards, > > >> > > Martin > > >> > > > > >> > > > > >> > > -----Original Message----- > > >> > > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] > > On > > >> > > Behalf Of Baesken, Matthias > > >> > > Sent: Montag, 30. April 2018 16:53 > > >> > > To: 'hotspot-dev at openjdk.java.net' > dev at openjdk.java.net> > > >> > > Subject: [RFR] 8202427: Enhance os::print_memory_info on > Windows > > >> > > > > >> > > On Windows, > > >> > > the os::print_memory_info misses a few memory-related infos > that > > >> might > > >> > > be interesting : > > >> > > - current and peak WorkingSet size (= physical memory assigned to > > the > > >> > > process) > > >> > > - current and peak commit charge (also known as "private bytes" in > > >> > Windows > > >> > > tools) > > >> > > - on 32bit Windows : > > >> > > user-mode portion/free user mode portion of virtual address-space > > >> > > (Total/AvailVirtual) because it shows how close do we get to the 2-4 > > GB > > >> per > > >> > > process border. > > >> > > > > >> > > > > >> > > - the current naming of "swap/free-swap" memory is a bit > misleading; > > >> > > in the Windows world swap is a special page file used for UWP apps. > > >> > > (see Windows Internals : "The swap file" (chapter 5 Memory > > >> > management) > > >> > > Windows 8 added another page file called a swap file. It is ... > > exclusively > > >> > used > > >> > > for UWP (Universal Windows Platform) apps. > > >> > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the > > UWP > > >> > > related values, it is about page file sizes > > >> > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just > > >> virtual > > >> > > memory might be more appropriate). > > >> > > > > >> > > > > >> > > https://msdn.microsoft.com/de- > > >> > > de/library/windows/desktop/aa366770(v=vs.85).aspx > > >> > > > > >> > > documents it in the following way: > > >> > > ullTotalPageFile: > > >> > > The current committed memory limit for the system or the current > > >> process, > > >> > > whichever is smaller,... > > >> > > > > >> > > ullAvailPageFile : > > >> > > The maximum amount of memory the current process can commit, > in > > >> > bytes. > > >> > > This value is equal to or smaller than the system-wide available > > commit > > >> > value > > >> > > > > >> > > > > >> > > > > >> > > Aditionally I suggest having output of the various memory-values in > M > > >> > > (megabyte) as well , the k (kilobyte) output sometimes gives very > > high > > >> and > > >> > > unreadable numbers). > > >> > > > > >> > > > > >> > > Could you please review my change ? > > >> > > > > >> > > > > >> > > Bug : > > >> > > > > >> > > https://bugs.openjdk.java.net/browse/JDK-8202427 > > >> > > > > >> > > > > >> > > Change : > > >> > > > > >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > > >> > > > > >> > > > > >> > > Thanks, Matthias > > >> > > > > > From thomas.stuefe at gmail.com Fri May 18 06:16:10 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 18 May 2018 08:16:10 +0200 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> Message-ID: Hi Matthias, I took another closer look and have some minor nits. Sorry for not looking better the first time, my head was not clear :) Since that was my fault I leave it up to you if you follow up. The fix in its current form is also fine to me. - I am not a big fan of ">> 20". I would change that to " / M" to make the code more readable. - Strictly speaking GetProcessMemoryInfo could fail (I have never seen that though) and return FALSE. In which case we should probably not output the results. Best Regards, Thomas On Thu, May 17, 2018 at 5:20 PM, Baesken, Matthias wrote: >> > > I also think that one number is enough. Adaptive numbers would be great. > > > Hi Thomas/Goetz/Martin , thanks for looking into it. > Adaptive numbers could be done in a follow-up as suggested by Thomas , this would also touch the other platforms . > > I added more line feeds to the output and reduced output to just printing M . > > > Example output from a Windows desktop PC : > > Memory: 4k page, system-wide physical 32520M (20165M free) > TotalPageFile size 34568M (AvailPageFile size 18170M) > current process WorkingSet (physical memory assigned to process): 79M, peak: 79M > current process commit charge ("private bytes"): 148M, peak: 148M > > > new webrev : > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.1/ > > > Best regards, Matthias > > > >> -----Original Message----- >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] >> Sent: Mittwoch, 16. Mai 2018 16:35 >> To: Lindenmaier, Goetz >> Cc: Baesken, Matthias ; Doerr, Martin >> ; hotspot-dev at openjdk.java.net; hotspot- >> runtime-dev at openjdk.java.net >> Subject: Re: [RFR] 8202427: Enhance os::print_memory_info on Windows >> >> On Wed, May 16, 2018 at 3:48 PM, Lindenmaier, Goetz >> wrote: >> > Hi Matthias, >> > >> > I appreciate the additional information in the hs_err file. >> > Could you please add an example output (of the final version) to the bug >> description? >> > >> > As Martin, I would like more line feeds, both in the code and the output. >> > Currently, the output in the hs_err file looks like this (where the first line is >> too long): >> > Memory: 4k page, system-wide physical 16776692k [16383M] (8795032k >> [8588M] free), TotalPageFile size 49949460k [48778M] (AvailPageFile size >> 38942152k [38029M]), user-mode portion of virtual address-space 2097024k >> [2047M] (6940k [6M] free) >> > current process WorkingSet (physical memory assigned to process): >> 991804k [968M], peak: 991804k [968M] >> > >> > current process commit charge ("private bytes"): 1523632k [1487M], >> peak: 1523632k [1487M] >> > >> > I also think that one number is enough. Adaptive numbers would be great. >> > I know about >> > src/hotspot/share/memory/metaspace/metaspaceCommon.hpp: >> print_human_readable_size() >> > for this, but this seems a bit hidden in metaspace. >> > >> >> Guys, lets not worry about human readable numbers for now. We can move >> print_human_readable_size() out from metaspace to something generic >> and use it to improve this code (and others) in a later patch. That >> was my original intention when adding print_human_readable_size(). >> >> The patch is good. Thanks for doing this. >> >> ..Thoams >> >> > I don't like usage of _M_IX86. Common in this file seems #ifndef _WIN64 >> for 32-bit windows >> > dependent code. But don't care here. >> > >> > Best regards, >> > Goetz. >> > >> > >> >> -----Original Message----- >> >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On >> >> Behalf Of Baesken, Matthias >> >> Sent: Mittwoch, 16. Mai 2018 15:13 >> >> To: Doerr, Martin ; 'hotspot- >> >> dev at openjdk.java.net' ; hotspot- >> >> runtime-dev at openjdk.java.net >> >> Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on Windows >> >> >> >> Ping : could I get a second review ? >> >> >> >> Bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8202427 >> >> Change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ >> >> >> >> >> >> > We could also just print the MB value , let's see what other think. >> >> > Another option might be to have a flexible output (kB for smaller >> >> memory >> >> > values , MB (or GB) for larger ) ? >> >> >> >> Martin suggested to just print the MB values, what do you think ? >> >> >> >> >> >> Thanks, Matthias >> >> >> >> >> >> > -----Original Message----- >> >> > From: Baesken, Matthias >> >> > Sent: Mittwoch, 2. Mai 2018 13:00 >> >> > To: Doerr, Martin ; 'hotspot- >> >> > dev at openjdk.java.net' ; hotspot- >> >> > runtime-dev at openjdk.java.net >> >> > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on >> Windows >> >> > >> >> > Hi Martin, thanks for your input . >> >> > I add hotspot-runtime-dev . >> >> > >> >> > > >> >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe >> other >> >> > > reviewers would like to comment on this, too. >> >> > > >> >> > >> >> > We could also just print the MB value , let's see what other think. >> >> > Another option might be to have a flexible output (kB for smaller >> >> memory >> >> > values , MB (or GB) for larger ) ? >> >> > >> >> > Best regards, Matthias >> >> > >> >> > >> >> > > -----Original Message----- >> >> > > From: Doerr, Martin >> >> > > Sent: Mittwoch, 2. Mai 2018 12:53 >> >> > > To: Baesken, Matthias ; 'hotspot- >> >> > > dev at openjdk.java.net' >> >> > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on >> Windows >> >> > > >> >> > > Hi Matthias, >> >> > > >> >> > > looks like a nice enhancement. We can get substantially more >> >> information. >> >> > > >> >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe >> other >> >> > > reviewers would like to comment on this, too. >> >> > > >> >> > > We should have line breaks. >> >> > > >> >> > > Best regards, >> >> > > Martin >> >> > > >> >> > > >> >> > > -----Original Message----- >> >> > > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] >> On >> >> > > Behalf Of Baesken, Matthias >> >> > > Sent: Montag, 30. April 2018 16:53 >> >> > > To: 'hotspot-dev at openjdk.java.net' > dev at openjdk.java.net> >> >> > > Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows >> >> > > >> >> > > On Windows, >> >> > > the os::print_memory_info misses a few memory-related infos that >> >> might >> >> > > be interesting : >> >> > > - current and peak WorkingSet size (= physical memory assigned to >> the >> >> > > process) >> >> > > - current and peak commit charge (also known as "private bytes" in >> >> > Windows >> >> > > tools) >> >> > > - on 32bit Windows : >> >> > > user-mode portion/free user mode portion of virtual address-space >> >> > > (Total/AvailVirtual) because it shows how close do we get to the 2-4 >> GB >> >> per >> >> > > process border. >> >> > > >> >> > > >> >> > > - the current naming of "swap/free-swap" memory is a bit misleading; >> >> > > in the Windows world swap is a special page file used for UWP apps. >> >> > > (see Windows Internals : "The swap file" (chapter 5 Memory >> >> > management) >> >> > > Windows 8 added another page file called a swap file. It is ... >> exclusively >> >> > used >> >> > > for UWP (Universal Windows Platform) apps. >> >> > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the >> UWP >> >> > > related values, it is about page file sizes >> >> > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just >> >> virtual >> >> > > memory might be more appropriate). >> >> > > >> >> > > >> >> > > https://msdn.microsoft.com/de- >> >> > > de/library/windows/desktop/aa366770(v=vs.85).aspx >> >> > > >> >> > > documents it in the following way: >> >> > > ullTotalPageFile: >> >> > > The current committed memory limit for the system or the current >> >> process, >> >> > > whichever is smaller,... >> >> > > >> >> > > ullAvailPageFile : >> >> > > The maximum amount of memory the current process can commit, in >> >> > bytes. >> >> > > This value is equal to or smaller than the system-wide available >> commit >> >> > value >> >> > > >> >> > > >> >> > > >> >> > > Aditionally I suggest having output of the various memory-values in M >> >> > > (megabyte) as well , the k (kilobyte) output sometimes gives very >> high >> >> and >> >> > > unreadable numbers). >> >> > > >> >> > > >> >> > > Could you please review my change ? >> >> > > >> >> > > >> >> > > Bug : >> >> > > >> >> > > https://bugs.openjdk.java.net/browse/JDK-8202427 >> >> > > >> >> > > >> >> > > Change : >> >> > > >> >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ >> >> > > >> >> > > >> >> > > Thanks, Matthias >> >> > > >> > From david.holmes at oracle.com Fri May 18 06:27:58 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 18 May 2018 16:27:58 +1000 Subject: RFR: JDK-8190187: Follow-up. In-Reply-To: <460d4e0b-ee30-3d32-b482-2efb571f18db@oracle.com> References: <460d4e0b-ee30-3d32-b482-2efb571f18db@oracle.com> Message-ID: <8c61fe81-ff96-286e-63cb-b1af24a5981c@oracle.com> On 18/05/2018 2:27 AM, Alan Bateman wrote: > On 17/05/2018 13:57, Adam Farley8 wrote: >> Hi Alan, >> >> As requested. >> >> Any ideas? > The issue here is that the patch is just one part of changing long > standing behavior. There is still work needed to flush out other > interactions in the JVM TI spec, e.g. the multiple agent case and > whether the Agent_OnUnLoad should be invoked for agents loaded before > some other agent returns JNI_SILENT_EXIT. Also the late-binding agent > case needs examination to see if wording is needed for the case that > Agent_OnAttach returns this status. There is also an update needed to > the JNI spec as we discussed. As per the mails on core-libs-dev, it's > not clear that it's worth it. In any case, I don't know if anyone cycles > to work with you on this. I think the help you are looking for is to > ensure that all the spec issues are covered, maybe help with the tests > and testing, and then the process work that is the CSR and release note. +1 I'm not convinced this really needs fixing (as per previous discussions). A proper fix is more involved than what has been proposed and needs to be given due indepth consideration - none of which I have time for. Even reviewing this would be a considerable time investment to be sure that all the issues had indeed been properly examined and covered. David > -Alan From sgehwolf at redhat.com Fri May 18 08:20:42 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 18 May 2018 10:20:42 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <64914635-0d59-7aed-d52c-7a5a4fc16658@oracle.com> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> <0a09b984-a617-af45-4033-43086fe818a4@redhat.com> <64914635-0d59-7aed-d52c-7a5a4fc16658@oracle.com> Message-ID: Hi David, On Fri, 2018-05-18 at 14:17 +1000, David Holmes wrote: > It doesn't look right to me that vm_version_ext_zero.cpp is zero > specific - it really should be CPU specific. The zero version is setting > up a bunch of dummy values for threads/cores/sockets etc, but if > something else uses them in calculations then you'd get some weird numbers. Right. Do you expect this to be used outside JFR? Note that Zero is supposed to be buildable on a wide range of architectures. Basically all that have GCC and libffi. That would mean we'd have to add CPU specific files for all such architectures. Examples would be mips, ppc (32bit), s390 (31 bit) and others. It seems more reasonable to add a zero specific version of it and produce dummy values than requiring one to add CPU specific vm_version_ext files on architectures where one happens to build Zero on just to get it to build. What's more, it only seems to be used by JFR. The trouble comes from os_perf_linux.cpp which got added in shared code for JFR. I don't expect JFR to work on Zero, TBH. It seems not worth having for Zero. But that's for later. Right now, the Zero build is broken and we need to get it fixed. FWIW, I've tried this to disable the jfr feature by default on Zero, but that still attempts to build os_perf_linux.cpp: diff --git a/make/autoconf/hotspot.m4 b/make/autoconf/hotspot.m4 --- a/make/autoconf/hotspot.m4 +++ b/make/autoconf/hotspot.m4 @@ -309,9 +309,11 @@ AC_MSG_ERROR([Specified JVM feature 'cmsgc' requires feature 'serialgc']) fi - # Enable JFR by default, except on linux-sparcv9 and on minimal. - if test "x$OPENJDK_TARGET_OS" != xlinux || test "x$OPENJDK_TARGET_CPU" != xsparcv9; then - NON_MINIMAL_FEATURES="$NON_MINIMAL_FEATURES jfr" + # Enable JFR by default, except for Zero, linux-sparcv9 and on minimal. + if test "x$HOTSPOT_TARGET_CPU" != xzero; then + if test "x$OPENJDK_TARGET_OS" != xlinux || test "x$OPENJDK_TARGET_CPU" != xsparcv9; then + NON_MINIMAL_FEATURES="$NON_MINIMAL_FEATURES jfr" + fi fi # Turn on additional features based on other parts of configure Thoughts? Thanks, Severin > David > > On 18/05/2018 12:38 AM, Severin Gehwolf wrote: > > Hi Martin, Aleksey, > > > > Ugh, sorry, my mistake :-/ Thanks for the reviews! > > > > This is what I'm going to push once jdk-submit comes back clean: > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.04/ > > > > Thanks, > > Severin > > > > On Thu, 2018-05-17 at 14:17 +0000, Doerr, Martin wrote: > > > Hi Severin, > > > > > > looks good to me with Aleksey's patch. Thanks for waiting. > > > > > > Best regards, > > > Martin > > > > > > > > > -----Original Message----- > > > From: Aleksey Shipilev [mailto:shade at redhat.com] > > > Sent: Donnerstag, 17. Mai 2018 16:14 > > > To: Severin Gehwolf ; Doerr, Martin ; hotspot-dev > > > Subject: Re: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) > > > > > > On 05/17/2018 04:02 PM, Severin Gehwolf wrote: > > > > On Thu, 2018-05-17 at 15:27 +0200, Aleksey Shipilev wrote: > > > > > On 05/17/2018 03:12 PM, Severin Gehwolf wrote: > > > > > > Hi Martin, Aleksey, > > > > > > > > > > > > Since JDK-8203288 has been pushed here is the latest webrev for Zero. > > > > > > It no longer contains changes in os_perf_linux.cpp. > > > > > > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.02/ > > > > > > > > > > > > How does it look? > > > > > > > > > > Looks good, but I used parentheses inconsistently with the rest of Hotspot in the original patch: > > > > > > > > > > #if (defined X86 && !defined ZERO) > > > > > > > > > > Should be: > > > > > > > > > > #if defined(X86) && !defined (ZERO) > > > > > > > > OK. Here it goes: > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.03/ > > > > > > Notice the parentheses, it is done like that in other places: > > > > > > #if defined(X86) && !defined(ZERO) > > > > > > Apply this patch to correct: > > > http://cr.openjdk.java.net/~shade/8203287/parent.patch > > > > > > I checked it builds with x86_64 and Zero. > > > > > > -Aleksey > > > From kevin.walls at oracle.com Fri May 18 08:51:45 2018 From: kevin.walls at oracle.com (Kevin Walls) Date: Fri, 18 May 2018 09:51:45 +0100 Subject: [8u] 8203349: 8u hotspot should recognise later Windows compilers In-Reply-To: References: <672231ce-9a74-af4e-88db-8044e4a74ebe@oracle.com> Message-ID: <13e5ccc0-cdfb-aa02-1cfd-da966bfe4b4e@oracle.com> Hi Erik - Quite right, thanks.? Actually I should include recognising VS2015 while I'm doing this. Update: http://cr.openjdk.java.net/~kevinw/8203349/webrev.01/ Also, update Abstract_VM_Version::internal_vm_info_string()? in src/share/vm/runtime/vm_version.cpp so the long version string is readable for these compilers. In the sanity checks for VS2015, I am predicting it has a linker version of 1900 as it's between the 1800 and 1900 that I have seen myself for the other versions I have. I have a local VS2013 build, and builds using the unchanged "official" compiler are still OK. Thanks Kevin On 17/05/2018 23:32, Erik Joelsson wrote: > Hello Kevin, > > In JDK 11, vm_version.cpp we have _MSC_VER == 1900 translating to > VS2015 and 1912 to VS2017. Is it really 1900 in nmake? > > Otherwise I think this looks ok. > > /Erik > > > On 2018-05-17 15:21, Kevin Walls wrote: >> Hi, >> >> I'd like to get a review for this 8u/hotspot build change for >> Windows, to loosen the restriction on what compilers we can use. >> >> JBS: >> 8203349: 8u hotspot should recognise later Windows compilers >> https://bugs.openjdk.java.net/browse/JDK-8203349 >> >> Webrev: http://cr.openjdk.java.net/~kevinw/8203349/webrev.00/ >> >> Changes in the places we (sanity) check version numbers, set some >> compile >> options, and expand the check in vm.make to make sure we put the >> precompiled >> object? _build_pch_file.obj on the jvm link command. >> >> In compile.make I added blocks for VS2013 and 17, and left them as >> separate, >> duplicated, blocks of settings to make them easier to change >> independently. >> >> This doesn't change anything about what is "supported" or documented >> as working. >> >> Thanks! >> Kevin >> >> > From david.holmes at oracle.com Fri May 18 09:24:36 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 18 May 2018 19:24:36 +1000 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> <0a09b984-a617-af45-4033-43086fe818a4@redhat.com> <64914635-0d59-7aed-d52c-7a5a4fc16658@oracle.com> Message-ID: <436c2684-92b1-f07c-283f-6e93de7eaf31@oracle.com> Hi Severin, It sounds like parts of os_perf should be conditional on INCLUDE_JFR (which I think replaced INCLUDE_TRACE). But I haven't been able to look at the Flight Recorder changes in detail. You obviously need to do what you need to move forward but something seems wrong here. David On 18/05/2018 6:20 PM, Severin Gehwolf wrote: > Hi David, > > On Fri, 2018-05-18 at 14:17 +1000, David Holmes wrote: >> It doesn't look right to me that vm_version_ext_zero.cpp is zero >> specific - it really should be CPU specific. The zero version is setting >> up a bunch of dummy values for threads/cores/sockets etc, but if >> something else uses them in calculations then you'd get some weird numbers. > > Right. Do you expect this to be used outside JFR? > > Note that Zero is supposed to be buildable on a wide range of > architectures. Basically all that have GCC and libffi. That would mean > we'd have to add CPU specific files for all such architectures. > Examples would be mips, ppc (32bit), s390 (31 bit) and others. It seems > more reasonable to add a zero specific version of it and produce dummy > values than requiring one to add CPU specific vm_version_ext files on > architectures where one happens to build Zero on just to get it to > build. What's more, it only seems to be used by JFR. > > The trouble comes from os_perf_linux.cpp which got added in shared code > for JFR. I don't expect JFR to work on Zero, TBH. It seems not worth > having for Zero. But that's for later. Right now, the Zero build is > broken and we need to get it fixed. FWIW, I've tried this to disable > the jfr feature by default on Zero, but that still attempts to build > os_perf_linux.cpp: > > diff --git a/make/autoconf/hotspot.m4 b/make/autoconf/hotspot.m4 > --- a/make/autoconf/hotspot.m4 > +++ b/make/autoconf/hotspot.m4 > @@ -309,9 +309,11 @@ > AC_MSG_ERROR([Specified JVM feature 'cmsgc' requires feature 'serialgc']) > fi > > - # Enable JFR by default, except on linux-sparcv9 and on minimal. > - if test "x$OPENJDK_TARGET_OS" != xlinux || test "x$OPENJDK_TARGET_CPU" != xsparcv9; then > - NON_MINIMAL_FEATURES="$NON_MINIMAL_FEATURES jfr" > + # Enable JFR by default, except for Zero, linux-sparcv9 and on minimal. > + if test "x$HOTSPOT_TARGET_CPU" != xzero; then > + if test "x$OPENJDK_TARGET_OS" != xlinux || test "x$OPENJDK_TARGET_CPU" != xsparcv9; then > + NON_MINIMAL_FEATURES="$NON_MINIMAL_FEATURES jfr" > + fi > fi > > # Turn on additional features based on other parts of configure > > Thoughts? > > Thanks, > Severin > >> David >> >> On 18/05/2018 12:38 AM, Severin Gehwolf wrote: >>> Hi Martin, Aleksey, >>> >>> Ugh, sorry, my mistake :-/ Thanks for the reviews! >>> >>> This is what I'm going to push once jdk-submit comes back clean: >>> >>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.04/ >>> >>> Thanks, >>> Severin >>> >>> On Thu, 2018-05-17 at 14:17 +0000, Doerr, Martin wrote: >>>> Hi Severin, >>>> >>>> looks good to me with Aleksey's patch. Thanks for waiting. >>>> >>>> Best regards, >>>> Martin >>>> >>>> >>>> -----Original Message----- >>>> From: Aleksey Shipilev [mailto:shade at redhat.com] >>>> Sent: Donnerstag, 17. Mai 2018 16:14 >>>> To: Severin Gehwolf ; Doerr, Martin ; hotspot-dev >>>> Subject: Re: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) >>>> >>>> On 05/17/2018 04:02 PM, Severin Gehwolf wrote: >>>>> On Thu, 2018-05-17 at 15:27 +0200, Aleksey Shipilev wrote: >>>>>> On 05/17/2018 03:12 PM, Severin Gehwolf wrote: >>>>>>> Hi Martin, Aleksey, >>>>>>> >>>>>>> Since JDK-8203288 has been pushed here is the latest webrev for Zero. >>>>>>> It no longer contains changes in os_perf_linux.cpp. >>>>>>> >>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.02/ >>>>>>> >>>>>>> How does it look? >>>>>> >>>>>> Looks good, but I used parentheses inconsistently with the rest of Hotspot in the original patch: >>>>>> >>>>>> #if (defined X86 && !defined ZERO) >>>>>> >>>>>> Should be: >>>>>> >>>>>> #if defined(X86) && !defined (ZERO) >>>>> >>>>> OK. Here it goes: >>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.03/ >>>> >>>> Notice the parentheses, it is done like that in other places: >>>> >>>> #if defined(X86) && !defined(ZERO) >>>> >>>> Apply this patch to correct: >>>> http://cr.openjdk.java.net/~shade/8203287/parent.patch >>>> >>>> I checked it builds with x86_64 and Zero. >>>> >>>> -Aleksey >>>> From glaubitz at physik.fu-berlin.de Fri May 18 09:41:59 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Fri, 18 May 2018 11:41:59 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <436c2684-92b1-f07c-283f-6e93de7eaf31@oracle.com> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> <0a09b984-a617-af45-4033-43086fe818a4@redhat.com> <64914635-0d59-7aed-d52c-7a5a4fc16658@oracle.com> <436c2684-92b1-f07c-283f-6e93de7eaf31@oracle.com> Message-ID: I agree. Please don?t break portability of Zero here. I haven?t had the time to look into this yet, but I will try to catch up the next days and see where I can help. Adrian > On May 18, 2018, at 11:24 AM, David Holmes wrote: > > Hi Severin, > > It sounds like parts of os_perf should be conditional on INCLUDE_JFR (which I think replaced INCLUDE_TRACE). But I haven't been able to look at the Flight Recorder changes in detail. > > You obviously need to do what you need to move forward but something seems wrong here. > > David > >> On 18/05/2018 6:20 PM, Severin Gehwolf wrote: >> Hi David, >>> On Fri, 2018-05-18 at 14:17 +1000, David Holmes wrote: >>> It doesn't look right to me that vm_version_ext_zero.cpp is zero >>> specific - it really should be CPU specific. The zero version is setting >>> up a bunch of dummy values for threads/cores/sockets etc, but if >>> something else uses them in calculations then you'd get some weird numbers. >> Right. Do you expect this to be used outside JFR? >> Note that Zero is supposed to be buildable on a wide range of >> architectures. Basically all that have GCC and libffi. That would mean >> we'd have to add CPU specific files for all such architectures. >> Examples would be mips, ppc (32bit), s390 (31 bit) and others. It seems >> more reasonable to add a zero specific version of it and produce dummy >> values than requiring one to add CPU specific vm_version_ext files on >> architectures where one happens to build Zero on just to get it to >> build. What's more, it only seems to be used by JFR. >> The trouble comes from os_perf_linux.cpp which got added in shared code >> for JFR. I don't expect JFR to work on Zero, TBH. It seems not worth >> having for Zero. But that's for later. Right now, the Zero build is >> broken and we need to get it fixed. FWIW, I've tried this to disable >> the jfr feature by default on Zero, but that still attempts to build >> os_perf_linux.cpp: >> diff --git a/make/autoconf/hotspot.m4 b/make/autoconf/hotspot.m4 >> --- a/make/autoconf/hotspot.m4 >> +++ b/make/autoconf/hotspot.m4 >> @@ -309,9 +309,11 @@ >> AC_MSG_ERROR([Specified JVM feature 'cmsgc' requires feature 'serialgc']) >> fi >> - # Enable JFR by default, except on linux-sparcv9 and on minimal. >> - if test "x$OPENJDK_TARGET_OS" != xlinux || test "x$OPENJDK_TARGET_CPU" != xsparcv9; then >> - NON_MINIMAL_FEATURES="$NON_MINIMAL_FEATURES jfr" >> + # Enable JFR by default, except for Zero, linux-sparcv9 and on minimal. >> + if test "x$HOTSPOT_TARGET_CPU" != xzero; then >> + if test "x$OPENJDK_TARGET_OS" != xlinux || test "x$OPENJDK_TARGET_CPU" != xsparcv9; then >> + NON_MINIMAL_FEATURES="$NON_MINIMAL_FEATURES jfr" >> + fi >> fi >> # Turn on additional features based on other parts of configure >> Thoughts? >> Thanks, >> Severin >>> David >>> >>>> On 18/05/2018 12:38 AM, Severin Gehwolf wrote: >>>> Hi Martin, Aleksey, >>>> >>>> Ugh, sorry, my mistake :-/ Thanks for the reviews! >>>> >>>> This is what I'm going to push once jdk-submit comes back clean: >>>> >>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.04/ >>>> >>>> Thanks, >>>> Severin >>>> >>>>> On Thu, 2018-05-17 at 14:17 +0000, Doerr, Martin wrote: >>>>> Hi Severin, >>>>> >>>>> looks good to me with Aleksey's patch. Thanks for waiting. >>>>> >>>>> Best regards, >>>>> Martin >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: Aleksey Shipilev [mailto:shade at redhat.com] >>>>> Sent: Donnerstag, 17. Mai 2018 16:14 >>>>> To: Severin Gehwolf ; Doerr, Martin ; hotspot-dev >>>>> Subject: Re: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) >>>>> >>>>>> On 05/17/2018 04:02 PM, Severin Gehwolf wrote: >>>>>>> On Thu, 2018-05-17 at 15:27 +0200, Aleksey Shipilev wrote: >>>>>>>> On 05/17/2018 03:12 PM, Severin Gehwolf wrote: >>>>>>>> Hi Martin, Aleksey, >>>>>>>> >>>>>>>> Since JDK-8203288 has been pushed here is the latest webrev for Zero. >>>>>>>> It no longer contains changes in os_perf_linux.cpp. >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.02/ >>>>>>>> >>>>>>>> How does it look? >>>>>>> >>>>>>> Looks good, but I used parentheses inconsistently with the rest of Hotspot in the original patch: >>>>>>> >>>>>>> #if (defined X86 && !defined ZERO) >>>>>>> >>>>>>> Should be: >>>>>>> >>>>>>> #if defined(X86) && !defined (ZERO) >>>>>> >>>>>> OK. Here it goes: >>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8203287/webrev.03/ >>>>> >>>>> Notice the parentheses, it is done like that in other places: >>>>> >>>>> #if defined(X86) && !defined(ZERO) >>>>> >>>>> Apply this patch to correct: >>>>> http://cr.openjdk.java.net/~shade/8203287/parent.patch >>>>> >>>>> I checked it builds with x86_64 and Zero. >>>>> >>>>> -Aleksey >>>>> From tobias.hartmann at oracle.com Fri May 18 10:54:01 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 18 May 2018 12:54:01 +0200 Subject: [11] RFR(S): 8202848: -XX:+ExecuteInternalVMTests asserts with "assert(cd.valid() == true) failed: failed on a valid DirectivesParser string" Message-ID: <0b3c8315-160e-52df-3758-e54ee2e455d8@oracle.com> Hi, please review the following patch: https://bugs.openjdk.java.net/browse/JDK-8202848 http://cr.openjdk.java.net/~thartmann/8202848/webrev.00/ The problem is that with some locale settings (for example, LC_NUMERIC = de_DE.UTF-8), sscanf in JSON::parse_json_number() treats "," as the decimal separator instead of ".". As a result, the value of "VectorizeDebug: 1," in one of the DirectivesParser tests is read as "1," and we skip the comma at the end of the line. This only affects VM internal tests, directive files specified via -XX:CompilerDirectivesFile= are parsed before the locale is set to the systems default. We should therefore explicitly set the "C" locale for the VM internal DirectivesParser tests. Thanks, Tobias From david.holmes at oracle.com Fri May 18 11:01:17 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 18 May 2018 21:01:17 +1000 Subject: [11] RFR(S): 8202848: -XX:+ExecuteInternalVMTests asserts with "assert(cd.valid() == true) failed: failed on a valid DirectivesParser string" In-Reply-To: <0b3c8315-160e-52df-3758-e54ee2e455d8@oracle.com> References: <0b3c8315-160e-52df-3758-e54ee2e455d8@oracle.com> Message-ID: Hi Tobias, Seems fine to me. Thanks, David On 18/05/2018 8:54 PM, Tobias Hartmann wrote: > Hi, > > please review the following patch: > https://bugs.openjdk.java.net/browse/JDK-8202848 > http://cr.openjdk.java.net/~thartmann/8202848/webrev.00/ > > The problem is that with some locale settings (for example, LC_NUMERIC = de_DE.UTF-8), sscanf in > JSON::parse_json_number() treats "," as the decimal separator instead of ".". As a result, the value > of "VectorizeDebug: 1," in one of the DirectivesParser tests is read as "1," and we skip the comma > at the end of the line. > > This only affects VM internal tests, directive files specified via -XX:CompilerDirectivesFile= are > parsed before the locale is set to the systems default. We should therefore explicitly set the "C" > locale for the VM internal DirectivesParser tests. > > Thanks, > Tobias > From tobias.hartmann at oracle.com Fri May 18 11:02:49 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 18 May 2018 13:02:49 +0200 Subject: [11] RFR(S): 8202848: -XX:+ExecuteInternalVMTests asserts with "assert(cd.valid() == true) failed: failed on a valid DirectivesParser string" In-Reply-To: References: <0b3c8315-160e-52df-3758-e54ee2e455d8@oracle.com> Message-ID: <542977cd-b587-9123-515d-35b9b5385e47@oracle.com> Thanks, David! Best regards, Tobias On 18.05.2018 13:01, David Holmes wrote: > Hi Tobias, > > Seems fine to me. > > Thanks, > David > > On 18/05/2018 8:54 PM, Tobias Hartmann wrote: >> Hi, >> >> please review the following patch: >> https://bugs.openjdk.java.net/browse/JDK-8202848 >> http://cr.openjdk.java.net/~thartmann/8202848/webrev.00/ >> >> The problem is that with some locale settings (for example, LC_NUMERIC = de_DE.UTF-8), sscanf in >> JSON::parse_json_number() treats "," as the decimal separator instead of ".". As a result, the value >> of "VectorizeDebug: 1," in one of the DirectivesParser tests is read as "1," and we skip the comma >> at the end of the line. >> >> This only affects VM internal tests, directive files specified via -XX:CompilerDirectivesFile= are >> parsed before the locale is set to the systems default. We should therefore explicitly set the "C" >> locale for the VM internal DirectivesParser tests. >> >> Thanks, >> Tobias >> From shade at redhat.com Fri May 18 11:10:25 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 18 May 2018 13:10:25 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> <0a09b984-a617-af45-4033-43086fe818a4@redhat.com> <64914635-0d59-7aed-d52c-7a5a4fc16658@oracle.com> <436c2684-92b1-f07c-283f-6e93de7eaf31@oracle.com> Message-ID: <59ed28ff-9d69-501b-5fe3-deab0f60abbd@redhat.com> But Zero is already broken, and there are two fixes: 1. Make Zero build again. 2. Make Zero compatible with JFR. I see no sense in waiting for (2), when (1) is ready. New code does not break the Zero portability per se, it renders Zero+JFR non-working on some platforms, but that is for (2). -Aleksey On 05/18/2018 11:41 AM, John Paul Adrian Glaubitz wrote: > I agree. Please don?t break portability of Zero here. > > I haven?t had the time to look into this yet, but I will try to catch up the next days and see where I can help. > > Adrian > >> On May 18, 2018, at 11:24 AM, David Holmes wrote: >> >> Hi Severin, >> >> It sounds like parts of os_perf should be conditional on INCLUDE_JFR (which I think replaced INCLUDE_TRACE). But I haven't been able to look at the Flight Recorder changes in detail. >> >> You obviously need to do what you need to move forward but something seems wrong here. From glaubitz at physik.fu-berlin.de Fri May 18 11:14:04 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Fri, 18 May 2018 13:14:04 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <59ed28ff-9d69-501b-5fe3-deab0f60abbd@redhat.com> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> <0a09b984-a617-af45-4033-43086fe818a4@redhat.com> <64914635-0d59-7aed-d52c-7a5a4fc16658@oracle.com> <436c2684-92b1-f07c-283f-6e93de7eaf31@oracle.com> <59ed28ff-9d69-501b-5fe3-deab0f60abbd@redhat.com> Message-ID: Hi! On 05/18/2018 01:10 PM, Aleksey Shipilev wrote: > But Zero is already broken, and there are two fixes: > 1. Make Zero build again. > 2. Make Zero compatible with JFR. > > I see no sense in waiting for (2), when (1) is ready. New code does not break the Zero portability > per se, it renders Zero+JFR non-working on some platforms, but that is for (2). I was not saying that the Zero build shouldn't be fixed. I just said that to keep in mind that Zero is primarily useful for uncommon architectures. If a change is committed now that makes Zero build on Oracle's supported platforms only, we would loose the main advantage of Zero. Disclaimer: I haven't looked into what Flight Recorder exactly is. I I just know that it currently broke several targets. Thanks, Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From sgehwolf at redhat.com Fri May 18 11:27:44 2018 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Fri, 18 May 2018 13:27:44 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <59ed28ff-9d69-501b-5fe3-deab0f60abbd@redhat.com> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> <0a09b984-a617-af45-4033-43086fe818a4@redhat.com> <64914635-0d59-7aed-d52c-7a5a4fc16658@oracle.com> <436c2684-92b1-f07c-283f-6e93de7eaf31@oracle.com> <59ed28ff-9d69-501b-5fe3-deab0f60abbd@redhat.com> Message-ID: <3eba82e77d89147971d3abe63455db925f530682.camel@redhat.com> On Fri, 2018-05-18 at 13:10 +0200, Aleksey Shipilev wrote: > But Zero is already broken, and there are two fixes: > 1. Make Zero build again. > 2. Make Zero compatible with JFR. > > I see no sense in waiting for (2), when (1) is ready. New code does not break the Zero portability > per se, it renders Zero+JFR non-working on some platforms, but that is for (2). Exactly, I'm going to push this fix now as a broken build opens the door for more Zero build breakage and leave the rest for (2). Thanks, Severin > -Aleksey > > On 05/18/2018 11:41 AM, John Paul Adrian Glaubitz wrote: > > I agree. Please don?t break portability of Zero here. > > > > I haven?t had the time to look into this yet, but I will try to > > catch up the next days and see where I can help. > > > > Adrian > > > > > On May 18, 2018, at 11:24 AM, David Holmes > > om> wrote: > > > > > > Hi Severin, > > > > > > It sounds like parts of os_perf should be conditional on > > > INCLUDE_JFR (which I think replaced INCLUDE_TRACE). But I haven't > > > been able to look at the Flight Recorder changes in detail. > > > > > > You obviously need to do what you need to move forward but > > > something seems wrong here. > > > From glaubitz at physik.fu-berlin.de Fri May 18 11:30:10 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Fri, 18 May 2018 13:30:10 +0200 Subject: 8203287: Zero fails to build after JDK-8199712 (Flight Recorder) In-Reply-To: <3eba82e77d89147971d3abe63455db925f530682.camel@redhat.com> References: <6979fed024cdce4ab38fc81477bb9927ab48302c.camel@redhat.com> <80f45f83fddf4718ae4b4c911448fa8b@sap.com> <422a9916-6075-0059-198d-997c55a48a6c@redhat.com> <97eb4b9df45821c2cdfa5d608e504619a233b695.camel@redhat.com> <7300a4ac-3106-e135-f889-a91db111effd@redhat.com> <897c116a15024d1fae2f1078408d9b9ecdc304c9.camel@redhat.com> <0a09b984-a617-af45-4033-43086fe818a4@redhat.com> <64914635-0d59-7aed-d52c-7a5a4fc16658@oracle.com> <436c2684-92b1-f07c-283f-6e93de7eaf31@oracle.com> <59ed28ff-9d69-501b-5fe3-deab0f60abbd@redhat.com> <3eba82e77d89147971d3abe63455db925f530682.camel@redhat.com> Message-ID: <882a9b5f-3611-ef47-5a7e-13931251043a@physik.fu-berlin.de> On 05/18/2018 01:27 PM, Severin Gehwolf wrote: > On Fri, 2018-05-18 at 13:10 +0200, Aleksey Shipilev wrote: >> But Zero is already broken, and there are two fixes: >> 1. Make Zero build again. >> 2. Make Zero compatible with JFR. >> >> I see no sense in waiting for (2), when (1) is ready. New code does not break the Zero portability >> per se, it renders Zero+JFR non-working on some platforms, but that is for (2). > > Exactly, I'm going to push this fix now as a broken build opens the > door for more Zero build breakage and leave the rest for (2). Sounds good. Once I have the time to fully understand Flight Recorder, I might whip up a patch to make Zero work with it. Thanks for taking care of Zero! Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From stefan.karlsson at oracle.com Fri May 18 12:04:01 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 18 May 2018 14:04:01 +0200 Subject: RFR: 8202989: Add missing decorators in calls to to arraycopy_prologue/epilogue In-Reply-To: References: Message-ID: <6a451d81-2280-b2de-abe0-617320c8bf37@oracle.com> Looks good. StefanK On 2018-05-11 10:47, Per Liden wrote: > Calls to BarrierSetAssambler::arraycopy_prologue/epilogue() are missing > the IN_HEAP and IN_HEAP_ARRAY access decorators. This is sort of ok, > since the context (we're doing an arraycopy) imply these. However, by > not explicitly specifying them we loose the ability to do straight > forward checks/asserts on expected/allowed decorators later on in the GC > specific BarrierSetAssermbler backends. To avoid having the backends > know about this implicit relationship I propose that we explicitly > specify them at the call-sites. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202989 > Webrev: http://cr.openjdk.java.net/~pliden/8202989/webrev.0 > Testing: hs-tier{1,2} > > /Per From per.liden at oracle.com Fri May 18 12:07:12 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 18 May 2018 14:07:12 +0200 Subject: RFR: 8202989: Add missing decorators in calls to to arraycopy_prologue/epilogue In-Reply-To: <6a451d81-2280-b2de-abe0-617320c8bf37@oracle.com> References: <6a451d81-2280-b2de-abe0-617320c8bf37@oracle.com> Message-ID: <44b009e7-a6c3-c658-c8cc-6eb55772548d@oracle.com> Thanks Stefan. /Per On 05/18/2018 02:04 PM, Stefan Karlsson wrote: > Looks good. > > StefanK > > On 2018-05-11 10:47, Per Liden wrote: >> Calls to BarrierSetAssambler::arraycopy_prologue/epilogue() are >> missing the IN_HEAP and IN_HEAP_ARRAY access decorators. This is sort >> of ok, since the context (we're doing an arraycopy) imply these. >> However, by not explicitly specifying them we loose the ability to do >> straight forward checks/asserts on expected/allowed decorators later >> on in the GC specific BarrierSetAssermbler backends. To avoid having >> the backends know about this implicit relationship I propose that we >> explicitly specify them at the call-sites. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202989 >> Webrev: http://cr.openjdk.java.net/~pliden/8202989/webrev.0 >> Testing: hs-tier{1,2} >> >> /Per From erik.osterlund at oracle.com Fri May 18 12:15:06 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 18 May 2018 14:15:06 +0200 Subject: RFR: 8202989: Add missing decorators in calls to to arraycopy_prologue/epilogue In-Reply-To: References: Message-ID: <5AFEC3CA.6090500@oracle.com> Hi Per, Looks good. Thanks, /Erik On 2018-05-11 10:47, Per Liden wrote: > Calls to BarrierSetAssambler::arraycopy_prologue/epilogue() are > missing the IN_HEAP and IN_HEAP_ARRAY access decorators. This is sort > of ok, since the context (we're doing an arraycopy) imply these. > However, by not explicitly specifying them we loose the ability to do > straight forward checks/asserts on expected/allowed decorators later > on in the GC specific BarrierSetAssermbler backends. To avoid having > the backends know about this implicit relationship I propose that we > explicitly specify them at the call-sites. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8202989 > Webrev: http://cr.openjdk.java.net/~pliden/8202989/webrev.0 > Testing: hs-tier{1,2} > > /Per From per.liden at oracle.com Fri May 18 12:13:29 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 18 May 2018 14:13:29 +0200 Subject: RFR: 8202989: Add missing decorators in calls to to arraycopy_prologue/epilogue In-Reply-To: <5AFEC3CA.6090500@oracle.com> References: <5AFEC3CA.6090500@oracle.com> Message-ID: Thanks Erik! /Per On 05/18/2018 02:15 PM, Erik ?sterlund wrote: > Hi Per, > > Looks good. > > Thanks, > /Erik > > On 2018-05-11 10:47, Per Liden wrote: >> Calls to BarrierSetAssambler::arraycopy_prologue/epilogue() are >> missing the IN_HEAP and IN_HEAP_ARRAY access decorators. This is sort >> of ok, since the context (we're doing an arraycopy) imply these. >> However, by not explicitly specifying them we loose the ability to do >> straight forward checks/asserts on expected/allowed decorators later >> on in the GC specific BarrierSetAssermbler backends. To avoid having >> the backends know about this implicit relationship I propose that we >> explicitly specify them at the call-sites. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8202989 >> Webrev: http://cr.openjdk.java.net/~pliden/8202989/webrev.0 >> Testing: hs-tier{1,2} >> >> /Per > From stefan.karlsson at oracle.com Fri May 18 12:13:12 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 18 May 2018 14:13:12 +0200 Subject: 8203341: Add a safepoint-aware Semaphore In-Reply-To: References: <2f8412d5-2fe4-2c8f-077c-cf96e8f9bebf@oracle.com> <5AFDA1E3.6010401@oracle.com> Message-ID: <385fc0f8-461d-5f0f-d678-1dcb69b173cf@oracle.com> Updated webrevs: http://cr.openjdk.java.net/~stefank/8203341/webrev.02/ http://cr.openjdk.java.net/~stefank/8203341/webrev.02.delta/ I removed the SemaphoreSafepointAware class, as Erik requested, but instead of adding a bool I added a new function for the safepoint aware waits. Thanks, StefanK On 2018-05-17 17:59, Stefan Karlsson wrote: > On 2018-05-17 17:38, Erik ?sterlund wrote: >> Hi Stefan, >> >> I wonder if it would make sense to let Semaphore::wait have a bool >> safepoint_check, similar to Mutex (possibly with a default value), >> instead of introducing a new class with the same API, that only >> differs in this property. > > That sounds reasonable to me. > > I also need to change my patch so that the thread-local handshake code > pass in the existing JavaThread, so that we don't have to call > Thread::current()->is_JavaThread() unnecessarily. > > Thanks, > StefanK > >> >> Thanks, >> /Erik >> >> On 2018-05-17 09:46, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to add a safepoint-aware Semaphore (and >>> refactor the semaphore usage in thread-local handshakes). >>> >>> http://cr.openjdk.java.net/~stefank/8203341/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8203341 >>> >>> Both ZGC and the thread-local handshakes need to move the JavaThread >>> to a blocked state before waiting on a Semaphore. I propose that we >>> introduce a new SemaphoreSafepointAware wrapper around Semaphore. >>> >>> There are two things to consider with this patch: >>> >>> 1) In ZGC, where this patch originates from, we set the JavaThread's >>> osthread state with osthread->set_state(CONDVAR_WAIT) (using >>> OSThreadWaitState). This means that we get the following printed when >>> the threads are dumped: "waiting on condition ". I've kept this >>> behavior for the SemaphoreSafepointAware class. This means that we >>> now change the osthread state for JavaThreads blocking in >>> HandshakeState::process_self_inner. >>> >>> 2) The thread-local handshakes uses trywait before entering wait: >>> ?? if (!_semaphore.trywait()) { >>> -??? ThreadBlockInVM tbivm(thread); >>> ???? _semaphore.wait(); >>> ?? } >>> >>> At the time when SemaphoreSafepointAware was written, we didn't have >>> trywait on all platforms, now we do. Maybe we should always do the >>> trywait dance inside SemaphoreSafepointAware::wait() ? >>> >>> inline void SemaphoreSafepointAware::wait() { >>> ? if (Thread::current()->is_Java_thread()) { >>> ??? if (!_semaphore.trywait) { >>> ????? JavaThread* thread = JavaThread::current(); >>> >>> ????? // Prepare to block and allow safepoints while blocked >>> ????? ThreadBlockInVM tbivm(thread); >>> ????? OSThreadWaitState osts(thread->osthread(), false /* not in >>> Object.wait() */); >>> >>> ????? // Wait for value >>> ???? _semaphore.wait(); >>> ?? } >>> >>> Thanks, >>> StefanK >> > From erik.osterlund at oracle.com Fri May 18 12:21:11 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 18 May 2018 14:21:11 +0200 Subject: 8203341: Add a safepoint-aware Semaphore In-Reply-To: <385fc0f8-461d-5f0f-d678-1dcb69b173cf@oracle.com> References: <2f8412d5-2fe4-2c8f-077c-cf96e8f9bebf@oracle.com> <5AFDA1E3.6010401@oracle.com> <385fc0f8-461d-5f0f-d678-1dcb69b173cf@oracle.com> Message-ID: <5AFEC537.4050304@oracle.com> Hi Stefan, Looks good. Thanks, /Erik On 2018-05-18 14:13, Stefan Karlsson wrote: > Updated webrevs: > http://cr.openjdk.java.net/~stefank/8203341/webrev.02/ > http://cr.openjdk.java.net/~stefank/8203341/webrev.02.delta/ > > I removed the SemaphoreSafepointAware class, as Erik requested, but > instead of adding a bool I added a new function for the safepoint > aware waits. > > Thanks, > StefanK > > On 2018-05-17 17:59, Stefan Karlsson wrote: >> On 2018-05-17 17:38, Erik ?sterlund wrote: >>> Hi Stefan, >>> >>> I wonder if it would make sense to let Semaphore::wait have a bool >>> safepoint_check, similar to Mutex (possibly with a default value), >>> instead of introducing a new class with the same API, that only >>> differs in this property. >> >> That sounds reasonable to me. >> >> I also need to change my patch so that the thread-local handshake >> code pass in the existing JavaThread, so that we don't have to call >> Thread::current()->is_JavaThread() unnecessarily. >> >> Thanks, >> StefanK >> >>> >>> Thanks, >>> /Erik >>> >>> On 2018-05-17 09:46, Stefan Karlsson wrote: >>>> Hi all, >>>> >>>> Please review this patch to add a safepoint-aware Semaphore (and >>>> refactor the semaphore usage in thread-local handshakes). >>>> >>>> http://cr.openjdk.java.net/~stefank/8203341/webrev.01/ >>>> https://bugs.openjdk.java.net/browse/JDK-8203341 >>>> >>>> Both ZGC and the thread-local handshakes need to move the >>>> JavaThread to a blocked state before waiting on a Semaphore. I >>>> propose that we introduce a new SemaphoreSafepointAware wrapper >>>> around Semaphore. >>>> >>>> There are two things to consider with this patch: >>>> >>>> 1) In ZGC, where this patch originates from, we set the >>>> JavaThread's osthread state with osthread->set_state(CONDVAR_WAIT) >>>> (using OSThreadWaitState). This means that we get the following >>>> printed when the threads are dumped: "waiting on condition ". I've >>>> kept this behavior for the SemaphoreSafepointAware class. This >>>> means that we now change the osthread state for JavaThreads >>>> blocking in HandshakeState::process_self_inner. >>>> >>>> 2) The thread-local handshakes uses trywait before entering wait: >>>> if (!_semaphore.trywait()) { >>>> - ThreadBlockInVM tbivm(thread); >>>> _semaphore.wait(); >>>> } >>>> >>>> At the time when SemaphoreSafepointAware was written, we didn't >>>> have trywait on all platforms, now we do. Maybe we should always do >>>> the trywait dance inside SemaphoreSafepointAware::wait() ? >>>> >>>> inline void SemaphoreSafepointAware::wait() { >>>> if (Thread::current()->is_Java_thread()) { >>>> if (!_semaphore.trywait) { >>>> JavaThread* thread = JavaThread::current(); >>>> >>>> // Prepare to block and allow safepoints while blocked >>>> ThreadBlockInVM tbivm(thread); >>>> OSThreadWaitState osts(thread->osthread(), false /* not in >>>> Object.wait() */); >>>> >>>> // Wait for value >>>> _semaphore.wait(); >>>> } >>>> >>>> Thanks, >>>> StefanK >>> >> From robbin.ehn at oracle.com Fri May 18 12:25:01 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 18 May 2018 14:25:01 +0200 Subject: RFR: 8203227: Introduce os::processor_id() for Linux and Solaris In-Reply-To: <3eac8bb2-fc7a-42b2-154b-e7f6cd63b20b@oracle.com> References: <327220bf-ea05-1921-59e1-2bd0c2c4dea8@oracle.com> <13cbc7ca-58eb-ae0b-9640-42be8e5fb3fd@oracle.com> <3eac8bb2-fc7a-42b2-154b-e7f6cd63b20b@oracle.com> Message-ID: <5a9752cf-3309-b970-ec52-8fca17c66101@oracle.com> > > Here's an updated webrev for continued discussion: > > http://cr.openjdk.java.net/~pliden/8203227/webrev.1 Looks good, Robbin > > cheers, > Per > >> >> Thanks, >> David >> >>> /Per From stefan.karlsson at oracle.com Fri May 18 12:45:09 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Fri, 18 May 2018 14:45:09 +0200 Subject: 8203341: Add a safepoint-aware Semaphore In-Reply-To: <5AFEC537.4050304@oracle.com> References: <2f8412d5-2fe4-2c8f-077c-cf96e8f9bebf@oracle.com> <5AFDA1E3.6010401@oracle.com> <385fc0f8-461d-5f0f-d678-1dcb69b173cf@oracle.com> <5AFEC537.4050304@oracle.com> Message-ID: <322b2a9c-4dfe-9790-1c1d-b1c7cfa4a8ff@oracle.com> I got offline suggestions that I should rename the function to wait_with_safepoint_check: http://cr.openjdk.java.net/~stefank/8203341/webrev.03/ http://cr.openjdk.java.net/~stefank/8203341/webrev.03.delta/ Thanks, StefanK On 2018-05-18 14:21, Erik ?sterlund wrote: > Hi Stefan, > > Looks good. > > Thanks, > /Erik > > On 2018-05-18 14:13, Stefan Karlsson wrote: >> Updated webrevs: >> ?http://cr.openjdk.java.net/~stefank/8203341/webrev.02/ >> ?http://cr.openjdk.java.net/~stefank/8203341/webrev.02.delta/ >> >> I removed the SemaphoreSafepointAware class, as Erik requested, but >> instead of adding a bool I added a new function for the safepoint >> aware waits. >> >> Thanks, >> StefanK >> >> On 2018-05-17 17:59, Stefan Karlsson wrote: >>> On 2018-05-17 17:38, Erik ?sterlund wrote: >>>> Hi Stefan, >>>> >>>> I wonder if it would make sense to let Semaphore::wait have a bool >>>> safepoint_check, similar to Mutex (possibly with a default value), >>>> instead of introducing a new class with the same API, that only >>>> differs in this property. >>> >>> That sounds reasonable to me. >>> >>> I also need to change my patch so that the thread-local handshake >>> code pass in the existing JavaThread, so that we don't have to call >>> Thread::current()->is_JavaThread() unnecessarily. >>> >>> Thanks, >>> StefanK >>> >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-05-17 09:46, Stefan Karlsson wrote: >>>>> Hi all, >>>>> >>>>> Please review this patch to add a safepoint-aware Semaphore (and >>>>> refactor the semaphore usage in thread-local handshakes). >>>>> >>>>> http://cr.openjdk.java.net/~stefank/8203341/webrev.01/ >>>>> https://bugs.openjdk.java.net/browse/JDK-8203341 >>>>> >>>>> Both ZGC and the thread-local handshakes need to move the >>>>> JavaThread to a blocked state before waiting on a Semaphore. I >>>>> propose that we introduce a new SemaphoreSafepointAware wrapper >>>>> around Semaphore. >>>>> >>>>> There are two things to consider with this patch: >>>>> >>>>> 1) In ZGC, where this patch originates from, we set the >>>>> JavaThread's osthread state with osthread->set_state(CONDVAR_WAIT) >>>>> (using OSThreadWaitState). This means that we get the following >>>>> printed when the threads are dumped: "waiting on condition ". I've >>>>> kept this behavior for the SemaphoreSafepointAware class. This >>>>> means that we now change the osthread state for JavaThreads >>>>> blocking in HandshakeState::process_self_inner. >>>>> >>>>> 2) The thread-local handshakes uses trywait before entering wait: >>>>> ?? if (!_semaphore.trywait()) { >>>>> -??? ThreadBlockInVM tbivm(thread); >>>>> ???? _semaphore.wait(); >>>>> ?? } >>>>> >>>>> At the time when SemaphoreSafepointAware was written, we didn't >>>>> have trywait on all platforms, now we do. Maybe we should always do >>>>> the trywait dance inside SemaphoreSafepointAware::wait() ? >>>>> >>>>> inline void SemaphoreSafepointAware::wait() { >>>>> ? if (Thread::current()->is_Java_thread()) { >>>>> ??? if (!_semaphore.trywait) { >>>>> ????? JavaThread* thread = JavaThread::current(); >>>>> >>>>> ????? // Prepare to block and allow safepoints while blocked >>>>> ????? ThreadBlockInVM tbivm(thread); >>>>> ????? OSThreadWaitState osts(thread->osthread(), false /* not in >>>>> Object.wait() */); >>>>> >>>>> ????? // Wait for value >>>>> ???? _semaphore.wait(); >>>>> ?? } >>>>> >>>>> Thanks, >>>>> StefanK >>>> >>> > From coleen.phillimore at oracle.com Fri May 18 12:50:20 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 18 May 2018 08:50:20 -0400 Subject: [11] RFR(S): 8202848: -XX:+ExecuteInternalVMTests asserts with "assert(cd.valid() == true) failed: failed on a valid DirectivesParser string" In-Reply-To: <0b3c8315-160e-52df-3758-e54ee2e455d8@oracle.com> References: <0b3c8315-160e-52df-3758-e54ee2e455d8@oracle.com> Message-ID: Thank you for fixing this!? Looks good. Coleen On 5/18/18 6:54 AM, Tobias Hartmann wrote: > Hi, > > please review the following patch: > https://bugs.openjdk.java.net/browse/JDK-8202848 > http://cr.openjdk.java.net/~thartmann/8202848/webrev.00/ > > The problem is that with some locale settings (for example, LC_NUMERIC = de_DE.UTF-8), sscanf in > JSON::parse_json_number() treats "," as the decimal separator instead of ".". As a result, the value > of "VectorizeDebug: 1," in one of the DirectivesParser tests is read as "1," and we skip the comma > at the end of the line. > > This only affects VM internal tests, directive files specified via -XX:CompilerDirectivesFile= are > parsed before the locale is set to the systems default. We should therefore explicitly set the "C" > locale for the VM internal DirectivesParser tests. > > Thanks, > Tobias From per.liden at oracle.com Fri May 18 13:14:37 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 18 May 2018 15:14:37 +0200 Subject: RFR: 8203227: Introduce os::processor_id() for Linux and Solaris In-Reply-To: <5a9752cf-3309-b970-ec52-8fca17c66101@oracle.com> References: <327220bf-ea05-1921-59e1-2bd0c2c4dea8@oracle.com> <13cbc7ca-58eb-ae0b-9640-42be8e5fb3fd@oracle.com> <3eac8bb2-fc7a-42b2-154b-e7f6cd63b20b@oracle.com> <5a9752cf-3309-b970-ec52-8fca17c66101@oracle.com> Message-ID: <9b7303b6-bda2-37f7-fbf0-19e85ab966e6@oracle.com> Thanks Robbin! /Per On 05/18/2018 02:25 PM, Robbin Ehn wrote: >> >> Here's an updated webrev for continued discussion: >> >> http://cr.openjdk.java.net/~pliden/8203227/webrev.1 > > Looks good, Robbin > >> >> cheers, >> Per >> >>> >>> Thanks, >>> David >>> >>>> /Per From per.liden at oracle.com Fri May 18 13:17:05 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 18 May 2018 15:17:05 +0200 Subject: 8203341: Add a safepoint-aware Semaphore In-Reply-To: <322b2a9c-4dfe-9790-1c1d-b1c7cfa4a8ff@oracle.com> References: <2f8412d5-2fe4-2c8f-077c-cf96e8f9bebf@oracle.com> <5AFDA1E3.6010401@oracle.com> <385fc0f8-461d-5f0f-d678-1dcb69b173cf@oracle.com> <5AFEC537.4050304@oracle.com> <322b2a9c-4dfe-9790-1c1d-b1c7cfa4a8ff@oracle.com> Message-ID: <0582d5e2-1c33-b88d-da39-a2cccc24373e@oracle.com> Looks good! /Per On 05/18/2018 02:45 PM, Stefan Karlsson wrote: > I got offline suggestions that I should rename the function to > wait_with_safepoint_check: > > http://cr.openjdk.java.net/~stefank/8203341/webrev.03/ > http://cr.openjdk.java.net/~stefank/8203341/webrev.03.delta/ > > Thanks, > StefanK > > On 2018-05-18 14:21, Erik ?sterlund wrote: >> Hi Stefan, >> >> Looks good. >> >> Thanks, >> /Erik >> >> On 2018-05-18 14:13, Stefan Karlsson wrote: >>> Updated webrevs: >>> ?http://cr.openjdk.java.net/~stefank/8203341/webrev.02/ >>> ?http://cr.openjdk.java.net/~stefank/8203341/webrev.02.delta/ >>> >>> I removed the SemaphoreSafepointAware class, as Erik requested, but >>> instead of adding a bool I added a new function for the safepoint >>> aware waits. >>> >>> Thanks, >>> StefanK >>> >>> On 2018-05-17 17:59, Stefan Karlsson wrote: >>>> On 2018-05-17 17:38, Erik ?sterlund wrote: >>>>> Hi Stefan, >>>>> >>>>> I wonder if it would make sense to let Semaphore::wait have a bool >>>>> safepoint_check, similar to Mutex (possibly with a default value), >>>>> instead of introducing a new class with the same API, that only >>>>> differs in this property. >>>> >>>> That sounds reasonable to me. >>>> >>>> I also need to change my patch so that the thread-local handshake >>>> code pass in the existing JavaThread, so that we don't have to call >>>> Thread::current()->is_JavaThread() unnecessarily. >>>> >>>> Thanks, >>>> StefanK >>>> >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2018-05-17 09:46, Stefan Karlsson wrote: >>>>>> Hi all, >>>>>> >>>>>> Please review this patch to add a safepoint-aware Semaphore (and >>>>>> refactor the semaphore usage in thread-local handshakes). >>>>>> >>>>>> http://cr.openjdk.java.net/~stefank/8203341/webrev.01/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8203341 >>>>>> >>>>>> Both ZGC and the thread-local handshakes need to move the >>>>>> JavaThread to a blocked state before waiting on a Semaphore. I >>>>>> propose that we introduce a new SemaphoreSafepointAware wrapper >>>>>> around Semaphore. >>>>>> >>>>>> There are two things to consider with this patch: >>>>>> >>>>>> 1) In ZGC, where this patch originates from, we set the >>>>>> JavaThread's osthread state with osthread->set_state(CONDVAR_WAIT) >>>>>> (using OSThreadWaitState). This means that we get the following >>>>>> printed when the threads are dumped: "waiting on condition ". I've >>>>>> kept this behavior for the SemaphoreSafepointAware class. This >>>>>> means that we now change the osthread state for JavaThreads >>>>>> blocking in HandshakeState::process_self_inner. >>>>>> >>>>>> 2) The thread-local handshakes uses trywait before entering wait: >>>>>> ?? if (!_semaphore.trywait()) { >>>>>> -??? ThreadBlockInVM tbivm(thread); >>>>>> ???? _semaphore.wait(); >>>>>> ?? } >>>>>> >>>>>> At the time when SemaphoreSafepointAware was written, we didn't >>>>>> have trywait on all platforms, now we do. Maybe we should always >>>>>> do the trywait dance inside SemaphoreSafepointAware::wait() ? >>>>>> >>>>>> inline void SemaphoreSafepointAware::wait() { >>>>>> ? if (Thread::current()->is_Java_thread()) { >>>>>> ??? if (!_semaphore.trywait) { >>>>>> ????? JavaThread* thread = JavaThread::current(); >>>>>> >>>>>> ????? // Prepare to block and allow safepoints while blocked >>>>>> ????? ThreadBlockInVM tbivm(thread); >>>>>> ????? OSThreadWaitState osts(thread->osthread(), false /* not in >>>>>> Object.wait() */); >>>>>> >>>>>> ????? // Wait for value >>>>>> ???? _semaphore.wait(); >>>>>> ?? } >>>>>> >>>>>> Thanks, >>>>>> StefanK >>>>> >>>> >> From tobias.hartmann at oracle.com Fri May 18 13:18:47 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 18 May 2018 15:18:47 +0200 Subject: [11] RFR(S): 8202848: -XX:+ExecuteInternalVMTests asserts with "assert(cd.valid() == true) failed: failed on a valid DirectivesParser string" In-Reply-To: References: <0b3c8315-160e-52df-3758-e54ee2e455d8@oracle.com> Message-ID: <1c7b3e9e-2d8b-9987-6c76-6d621d486b07@oracle.com> Thanks, Coleen! Best regards, Tobias On 18.05.2018 14:50, coleen.phillimore at oracle.com wrote: > > Thank you for fixing this!? Looks good. > Coleen > > On 5/18/18 6:54 AM, Tobias Hartmann wrote: >> Hi, >> >> please review the following patch: >> https://bugs.openjdk.java.net/browse/JDK-8202848 >> http://cr.openjdk.java.net/~thartmann/8202848/webrev.00/ >> >> The problem is that with some locale settings (for example, LC_NUMERIC = de_DE.UTF-8), sscanf in >> JSON::parse_json_number() treats "," as the decimal separator instead of ".". As a result, the value >> of "VectorizeDebug: 1," in one of the DirectivesParser tests is read as "1," and we skip the comma >> at the end of the line. >> >> This only affects VM internal tests, directive files specified via -XX:CompilerDirectivesFile= are >> parsed before the locale is set to the systems default. We should therefore explicitly set the "C" >> locale for the VM internal DirectivesParser tests. >> >> Thanks, >> Tobias > From poonam.bajaj at oracle.com Fri May 18 13:30:38 2018 From: poonam.bajaj at oracle.com (Poonam Parhar) Date: Fri, 18 May 2018 06:30:38 -0700 Subject: RFR (8u): JDK-8146115: Improve docker container detection and resource configuration usage In-Reply-To: <69915730-5963-41AE-AB75-B0C223E39035@oracle.com> References: <3968d009-c1a2-87e7-bc22-70c348ee5b69@oracle.com> <69915730-5963-41AE-AB75-B0C223E39035@oracle.com> Message-ID: <456bd0f8-c89b-1cd0-8369-9ecf0ac3b0e4@oracle.com> Hello Bob, Thanks a lot for reviewing the changes! On 5/17/18 11:12 AM, Bob Vandette wrote: > The backport of my changes look pretty good. > > If the new PrintContainerInfo option is only referenced on Linux platforms, you might > want to move it to globals_linux.hpp. Yes, currently it is being used only on Linux platforms, but I think it is a general option and might be used on other platforms at a later date.? So I think we can leave it in globals.hpp. > > Is there a reason PrintActiveCpus is a diagnostic flag but PrintContainerInfo is not? Yes, PrintContainerInfo should also be a diagnostic option. I have changed it. Updated webrev: http://cr.openjdk.java.net/~poonam/8146115/webrev.01/ > > Is it acceptable to add these new VM flags in a backport that won?t be supported in the latest release? Since JDK 9 and later releases use Unified JVM logging, and we don't have that in JDK8, a new option is required in 8 to log the information which is being logged under 'container' tag in 10 and 11. We will need to have a CSR request approved for the new JVM options added as part of this backport. Thanks, Poonam > > Bob. > > >> On May 15, 2018, at 4:46 PM, Poonam Parhar wrote: >> >> Hello, >> >> Please review the docker container support changes backported to JDK 8u. These changes include the backport of the following enhancement and the follow-on bug fixes done on top of that in jdk 10 and 11. >> >> Webrev: http://cr.openjdk.java.net/~poonam/8146115/webrev.00/ >> >> Enhancement:JDK-8146115: Improve docker container detection and resource configuration usage >> >> The changes also include the fixes for the following two bugs: >> Bug JDK-8186248: Allow more flexibility in selecting Heap % of available RAM >> BugJDK-8190283: Default heap sizing options select a MaxHeapSize larger than available physical memory in some cases >> >> These changes add a new JVM option 'PrintContainerInfo' for tracing container related information which is specific to jdk 8. >> >> Testing results (with -XX:+UnlockDiagnosticVMOptions -XX:+PrintContainerInfo -XX:+PrintActiveCpus JVM options): >> -------------- >> poonam at poonam-VirtualBox:~/docker-image$ java TestCPUsMemory >> Number of Processors: 3 >> Max Memory: 921174016 >> poonam at poonam-VirtualBox:~/docker-image$ sudo docker run --cpus 1 -m1024m --rm myimage >> [sudo] password for poonam: >> WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. >> OSContainer::init: Initializing Container Support >> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes >> Memory Limit is: 1073741824 >> active_processor_count: sched_getaffinity processor count: 3 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: 100000 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 1024 >> CPU Quota count based on quota/period: 1 >> OSContainer::active_processor_count: 1 >> active_processor_count: determined by OSContainer: 1 >> active_processor_count: sched_getaffinity processor count: 3 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: 100000 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 1024 >> CPU Quota count based on quota/period: 1 >> OSContainer::active_processor_count: 1 >> active_processor_count: determined by OSContainer: 1 >> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes >> Memory Limit is: 1073741824 >> total container memory: 1073741824 >> active_processor_count: sched_getaffinity processor count: 3 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: 100000 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 1024 >> CPU Quota count based on quota/period: 1 >> OSContainer::active_processor_count: 1 >> active_processor_count: determined by OSContainer: 1 >> active_processor_count: sched_getaffinity processor count: 3 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: 100000 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 1024 >> CPU Quota count based on quota/period: 1 >> OSContainer::active_processor_count: 1 >> active_processor_count: determined by OSContainer: 1 >> >> active_processor_count: sched_getaffinity processor count: 3 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: 100000 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 1024 >> CPU Quota count based on quota/period: 1 >> OSContainer::active_processor_count: 1 >> active_processor_count: determined by OSContainer: 1 >> Number of Processors: 1 >> Max Memory: 259522560 >> >> poonam at poonam-VirtualBox:~/docker-image$ sudo docker run --cpu-shares 2048 -m1024m --rm myimage >> WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. >> OSContainer::init: Initializing Container Support >> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes >> Memory Limit is: 1073741824 >> active_processor_count: sched_getaffinity processor count: 3 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: -1 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 2048 >> CPU Share count based on shares: 2 >> OSContainer::active_processor_count: 2 >> active_processor_count: determined by OSContainer: 2 >> active_processor_count: sched_getaffinity processor count: 3 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: -1 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 2048 >> CPU Share count based on shares: 2 >> OSContainer::active_processor_count: 2 >> active_processor_count: determined by OSContainer: 2 >> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes >> Memory Limit is: 1073741824 >> total container memory: 1073741824 >> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes >> Memory Limit is: 1073741824 >> total container memory: 1073741824 >> active_processor_count: sched_getaffinity processor count: 3 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: -1 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 2048 >> CPU Share count based on shares: 2 >> OSContainer::active_processor_count: 2 >> active_processor_count: determined by OSContainer: 2 >> active_processor_count: sched_getaffinity processor count: 3 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: -1 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 2048 >> CPU Share count based on shares: 2 >> OSContainer::active_processor_count: 2 >> active_processor_count: determined by OSContainer: 2 >> >> active_processor_count: sched_getaffinity processor count: 3 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: -1 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 2048 >> CPU Share count based on shares: 2 >> OSContainer::active_processor_count: 2 >> active_processor_count: determined by OSContainer: 2 >> Number of Processors: 2 >> Max Memory: 259522560 >> >> poonam at poonam-VirtualBox:~/docker-image$ sudo docker run --cpuset-cpus 0-1 -m1024m --rm myimage >> WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. >> OSContainer::init: Initializing Container Support >> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes >> Memory Limit is: 1073741824 >> active_processor_count: sched_getaffinity processor count: 2 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: -1 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 1024 >> OSContainer::active_processor_count: 2 >> active_processor_count: determined by OSContainer: 2 >> active_processor_count: sched_getaffinity processor count: 2 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: -1 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 1024 >> OSContainer::active_processor_count: 2 >> active_processor_count: determined by OSContainer: 2 >> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes >> Memory Limit is: 1073741824 >> total container memory: 1073741824 >> Path to /memory.limit_in_bytes is /sys/fs/cgroup/memory/memory.limit_in_bytes >> Memory Limit is: 1073741824 >> total container memory: 1073741824 >> active_processor_count: sched_getaffinity processor count: 2 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: -1 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 1024 >> OSContainer::active_processor_count: 2 >> active_processor_count: determined by OSContainer: 2 >> active_processor_count: sched_getaffinity processor count: 2 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: -1 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 1024 >> OSContainer::active_processor_count: 2 >> active_processor_count: determined by OSContainer: 2 >> >> active_processor_count: sched_getaffinity processor count: 2 >> Path to /cpu.cfs_quota_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us >> CPU Quota is: -1 >> Path to /cpu.cfs_period_us is /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_period_us >> CPU Period is: 100000 >> Path to /cpu.shares is /sys/fs/cgroup/cpu,cpuacct/cpu.shares >> CPU Shares is: 1024 >> OSContainer::active_processor_count: 2 >> active_processor_count: determined by OSContainer: 2 >> Number of Processors: 2 >> Max Memory: 259522560 >> ------------------ >> >> Thanks, >> Poonam >> From thomas.stuefe at gmail.com Fri May 18 13:42:29 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 18 May 2018 15:42:29 +0200 Subject: [11] RFR(S): 8202848: -XX:+ExecuteInternalVMTests asserts with "assert(cd.valid() == true) failed: failed on a valid DirectivesParser string" In-Reply-To: <0b3c8315-160e-52df-3758-e54ee2e455d8@oracle.com> References: <0b3c8315-160e-52df-3758-e54ee2e455d8@oracle.com> Message-ID: Thanks for fixing, Tobias! Looks good. ..Thomas On Fri, May 18, 2018 at 12:54 PM, Tobias Hartmann wrote: > Hi, > > please review the following patch: > https://bugs.openjdk.java.net/browse/JDK-8202848 > http://cr.openjdk.java.net/~thartmann/8202848/webrev.00/ > > The problem is that with some locale settings (for example, LC_NUMERIC = de_DE.UTF-8), sscanf in > JSON::parse_json_number() treats "," as the decimal separator instead of ".". As a result, the value > of "VectorizeDebug: 1," in one of the DirectivesParser tests is read as "1," and we skip the comma > at the end of the line. > > This only affects VM internal tests, directive files specified via -XX:CompilerDirectivesFile= are > parsed before the locale is set to the systems default. We should therefore explicitly set the "C" > locale for the VM internal DirectivesParser tests. > > Thanks, > Tobias From tobias.hartmann at oracle.com Fri May 18 14:26:21 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 18 May 2018 16:26:21 +0200 Subject: [11] RFR(S): 8202848: -XX:+ExecuteInternalVMTests asserts with "assert(cd.valid() == true) failed: failed on a valid DirectivesParser string" In-Reply-To: References: <0b3c8315-160e-52df-3758-e54ee2e455d8@oracle.com> Message-ID: Thanks, Thomas! Best regards, Tobias On 18.05.2018 15:42, Thomas St?fe wrote: > Thanks for fixing, Tobias! Looks good. > > ..Thomas > > On Fri, May 18, 2018 at 12:54 PM, Tobias Hartmann > wrote: >> Hi, >> >> please review the following patch: >> https://bugs.openjdk.java.net/browse/JDK-8202848 >> http://cr.openjdk.java.net/~thartmann/8202848/webrev.00/ >> >> The problem is that with some locale settings (for example, LC_NUMERIC = de_DE.UTF-8), sscanf in >> JSON::parse_json_number() treats "," as the decimal separator instead of ".". As a result, the value >> of "VectorizeDebug: 1," in one of the DirectivesParser tests is read as "1," and we skip the comma >> at the end of the line. >> >> This only affects VM internal tests, directive files specified via -XX:CompilerDirectivesFile= are >> parsed before the locale is set to the systems default. We should therefore explicitly set the "C" >> locale for the VM internal DirectivesParser tests. >> >> Thanks, >> Tobias From daniel.daugherty at oracle.com Fri May 18 14:42:37 2018 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 18 May 2018 07:42:37 -0700 (PDT) Subject: [11] RFR(S): 8202848: -XX:+ExecuteInternalVMTests asserts with "assert(cd.valid() == true) failed: failed on a valid DirectivesParser string" In-Reply-To: <0b3c8315-160e-52df-3758-e54ee2e455d8@oracle.com> References: <0b3c8315-160e-52df-3758-e54ee2e455d8@oracle.com> Message-ID: <270f83ba-2208-535a-e7ad-4b0f7affaa28@oracle.com> On 5/18/18 6:54 AM, Tobias Hartmann wrote: > Hi, > > please review the following patch: > https://bugs.openjdk.java.net/browse/JDK-8202848 > http://cr.openjdk.java.net/~thartmann/8202848/webrev.00/ src/hotspot/share/utilities/internalVMTests.cpp ??? No comments. Thumbs up! Dan > > The problem is that with some locale settings (for example, LC_NUMERIC = de_DE.UTF-8), sscanf in > JSON::parse_json_number() treats "," as the decimal separator instead of ".". As a result, the value > of "VectorizeDebug: 1," in one of the DirectivesParser tests is read as "1," and we skip the comma > at the end of the line. > > This only affects VM internal tests, directive files specified via -XX:CompilerDirectivesFile= are > parsed before the locale is set to the systems default. We should therefore explicitly set the "C" > locale for the VM internal DirectivesParser tests. > > Thanks, > Tobias From tobias.hartmann at oracle.com Fri May 18 14:46:09 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Fri, 18 May 2018 16:46:09 +0200 Subject: [11] RFR(S): 8202848: -XX:+ExecuteInternalVMTests asserts with "assert(cd.valid() == true) failed: failed on a valid DirectivesParser string" In-Reply-To: <270f83ba-2208-535a-e7ad-4b0f7affaa28@oracle.com> References: <0b3c8315-160e-52df-3758-e54ee2e455d8@oracle.com> <270f83ba-2208-535a-e7ad-4b0f7affaa28@oracle.com> Message-ID: <8642ba29-75dd-88b0-789b-32d1eba04ebc@oracle.com> Thanks, Daniel! Tobias On 18.05.2018 16:42, Daniel D. Daugherty wrote: > On 5/18/18 6:54 AM, Tobias Hartmann wrote: >> Hi, >> >> please review the following patch: >> https://bugs.openjdk.java.net/browse/JDK-8202848 >> http://cr.openjdk.java.net/~thartmann/8202848/webrev.00/ > > src/hotspot/share/utilities/internalVMTests.cpp > ??? No comments. > > Thumbs up! > > Dan > > >> >> The problem is that with some locale settings (for example, LC_NUMERIC = de_DE.UTF-8), sscanf in >> JSON::parse_json_number() treats "," as the decimal separator instead of ".". As a result, the value >> of "VectorizeDebug: 1," in one of the DirectivesParser tests is read as "1," and we skip the comma >> at the end of the line. >> >> This only affects VM internal tests, directive files specified via -XX:CompilerDirectivesFile= are >> parsed before the locale is set to the systems default. We should therefore explicitly set the "C" >> locale for the VM internal DirectivesParser tests. >> >> Thanks, >> Tobias > From erik.osterlund at oracle.com Fri May 18 15:03:52 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 18 May 2018 17:03:52 +0200 Subject: RFR: 8202547: Move G1 runtime calls used by generated code to G1BarrierSetRuntime Message-ID: <5AFEEB58.3080404@oracle.com> Hi, Generated code occasionally needs to call into the runtime for G1 barriers. Some of these slow-path runtime calls are in SharedRuntime, some are in G1BarrierSet. It would be nice to have them collected in the same place. This patch move these slow-path C++ functions for G1 into a new class, g1BarrierSetRuntime. Webrev: http://cr.openjdk.java.net/~eosterlund/8202547/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8202547 Thanks, /Erik From matthias.baesken at sap.com Fri May 18 15:10:48 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 18 May 2018 15:10:48 +0000 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: References: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> Message-ID: <219b9940eaa14dd6805ac5360a1733aa@sap.com> Hi Thomas , thanks looking into it. Indeed GetProcessMemoryInfo could fail , and GetProcessMemoryInfo could fail as well . MSDN says for both functions . https://msdn.microsoft.com/de-de/library/windows/desktop/ms683219(v=vs.85).aspx https://msdn.microsoft.com/de-de/library/windows/desktop/aa366589(v=vs.85).aspx .......... Return value If the function succeeds, the return value is nonzero. If the function fails, the return value is zero. I add return code checking for failure cases . (plus some line feeds in the coding which is desired by Goetz ). For GlobalMemoryStatusEx there seem to be quite a few places where return code checking is missing , this should be addressed in a follow up change : OperatingSystemImpl.c 103 GlobalMemoryStatusEx(&ms); 113 GlobalMemoryStatusEx(&ms); 142 GlobalMemoryStatusEx(&ms); 152 GlobalMemoryStatusEx(&ms); os_windows.cpp 736 GlobalMemoryStatusEx(&ms); 748 GlobalMemoryStatusEx(&ms); 1748 GlobalMemoryStatusEx(&ms); 3669 GlobalMemoryStatusEx(&ms); Best regards, Matthias > -----Original Message----- > From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > Sent: Freitag, 18. Mai 2018 08:16 > To: Baesken, Matthias > Cc: Lindenmaier, Goetz ; Doerr, Martin > ; hotspot-dev at openjdk.java.net; hotspot- > runtime-dev at openjdk.java.net > Subject: Re: [RFR] 8202427: Enhance os::print_memory_info on Windows > > Hi Matthias, > > I took another closer look and have some minor nits. Sorry for not > looking better the first time, my head was not clear :) > > Since that was my fault I leave it up to you if you follow up. The fix > in its current form is also fine to me. > > - I am not a big fan of ">> 20". I would change that to " / M" to make > the code more readable. > - Strictly speaking GetProcessMemoryInfo could fail (I have never seen > that though) and return FALSE. In which case we should probably not > output the results. > > Best Regards, Thomas > > > On Thu, May 17, 2018 at 5:20 PM, Baesken, Matthias > wrote: > >> > > I also think that one number is enough. Adaptive numbers would be > great. > > > > > > Hi Thomas/Goetz/Martin , thanks for looking into it. > > Adaptive numbers could be done in a follow-up as suggested by Thomas , > this would also touch the other platforms . > > > > I added more line feeds to the output and reduced output to just printing > M . > > > > > > Example output from a Windows desktop PC : > > > > Memory: 4k page, system-wide physical 32520M (20165M free) > > TotalPageFile size 34568M (AvailPageFile size 18170M) > > current process WorkingSet (physical memory assigned to process): 79M, > peak: 79M > > current process commit charge ("private bytes"): 148M, peak: 148M > > > > > > new webrev : > > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.1/ > > > > > > Best regards, Matthias > > > > > > > >> -----Original Message----- > >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] > >> Sent: Mittwoch, 16. Mai 2018 16:35 > >> To: Lindenmaier, Goetz > >> Cc: Baesken, Matthias ; Doerr, Martin > >> ; hotspot-dev at openjdk.java.net; hotspot- > >> runtime-dev at openjdk.java.net > >> Subject: Re: [RFR] 8202427: Enhance os::print_memory_info on Windows > >> > >> On Wed, May 16, 2018 at 3:48 PM, Lindenmaier, Goetz > >> wrote: > >> > Hi Matthias, > >> > > >> > I appreciate the additional information in the hs_err file. > >> > Could you please add an example output (of the final version) to the > bug > >> description? > >> > > >> > As Martin, I would like more line feeds, both in the code and the output. > >> > Currently, the output in the hs_err file looks like this (where the first > line is > >> too long): > >> > Memory: 4k page, system-wide physical 16776692k [16383M] > (8795032k > >> [8588M] free), TotalPageFile size 49949460k [48778M] (AvailPageFile size > >> 38942152k [38029M]), user-mode portion of virtual address-space > 2097024k > >> [2047M] (6940k [6M] free) > >> > current process WorkingSet (physical memory assigned to process): > >> 991804k [968M], peak: 991804k [968M] > >> > > >> > current process commit charge ("private bytes"): 1523632k [1487M], > >> peak: 1523632k [1487M] > >> > > >> > I also think that one number is enough. Adaptive numbers would be > great. > >> > I know about > >> > src/hotspot/share/memory/metaspace/metaspaceCommon.hpp: > >> print_human_readable_size() > >> > for this, but this seems a bit hidden in metaspace. > >> > > >> > >> Guys, lets not worry about human readable numbers for now. We can > move > >> print_human_readable_size() out from metaspace to something generic > >> and use it to improve this code (and others) in a later patch. That > >> was my original intention when adding print_human_readable_size(). > >> > >> The patch is good. Thanks for doing this. > >> > >> ..Thoams > >> > >> > I don't like usage of _M_IX86. Common in this file seems #ifndef > _WIN64 > >> for 32-bit windows > >> > dependent code. But don't care here. > >> > > >> > Best regards, > >> > Goetz. > >> > > >> > > >> >> -----Original Message----- > >> >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] > On > >> >> Behalf Of Baesken, Matthias > >> >> Sent: Mittwoch, 16. Mai 2018 15:13 > >> >> To: Doerr, Martin ; 'hotspot- > >> >> dev at openjdk.java.net' ; hotspot- > >> >> runtime-dev at openjdk.java.net > >> >> Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > Windows > >> >> > >> >> Ping : could I get a second review ? > >> >> > >> >> Bug : > >> >> https://bugs.openjdk.java.net/browse/JDK-8202427 > >> >> Change : > >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > >> >> > >> >> > >> >> > We could also just print the MB value , let's see what other think. > >> >> > Another option might be to have a flexible output (kB for smaller > >> >> memory > >> >> > values , MB (or GB) for larger ) ? > >> >> > >> >> Martin suggested to just print the MB values, what do you think ? > >> >> > >> >> > >> >> Thanks, Matthias > >> >> > >> >> > >> >> > -----Original Message----- > >> >> > From: Baesken, Matthias > >> >> > Sent: Mittwoch, 2. Mai 2018 13:00 > >> >> > To: Doerr, Martin ; 'hotspot- > >> >> > dev at openjdk.java.net' ; hotspot- > >> >> > runtime-dev at openjdk.java.net > >> >> > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > >> Windows > >> >> > > >> >> > Hi Martin, thanks for your input . > >> >> > I add hotspot-runtime-dev . > >> >> > > >> >> > > > >> >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe > >> other > >> >> > > reviewers would like to comment on this, too. > >> >> > > > >> >> > > >> >> > We could also just print the MB value , let's see what other think. > >> >> > Another option might be to have a flexible output (kB for smaller > >> >> memory > >> >> > values , MB (or GB) for larger ) ? > >> >> > > >> >> > Best regards, Matthias > >> >> > > >> >> > > >> >> > > -----Original Message----- > >> >> > > From: Doerr, Martin > >> >> > > Sent: Mittwoch, 2. Mai 2018 12:53 > >> >> > > To: Baesken, Matthias ; 'hotspot- > >> >> > > dev at openjdk.java.net' > >> >> > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on > >> Windows > >> >> > > > >> >> > > Hi Matthias, > >> >> > > > >> >> > > looks like a nice enhancement. We can get substantially more > >> >> information. > >> >> > > > >> >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe > >> other > >> >> > > reviewers would like to comment on this, too. > >> >> > > > >> >> > > We should have line breaks. > >> >> > > > >> >> > > Best regards, > >> >> > > Martin > >> >> > > > >> >> > > > >> >> > > -----Original Message----- > >> >> > > From: hotspot-dev [mailto:hotspot-dev- > bounces at openjdk.java.net] > >> On > >> >> > > Behalf Of Baesken, Matthias > >> >> > > Sent: Montag, 30. April 2018 16:53 > >> >> > > To: 'hotspot-dev at openjdk.java.net' >> dev at openjdk.java.net> > >> >> > > Subject: [RFR] 8202427: Enhance os::print_memory_info on > Windows > >> >> > > > >> >> > > On Windows, > >> >> > > the os::print_memory_info misses a few memory-related infos > that > >> >> might > >> >> > > be interesting : > >> >> > > - current and peak WorkingSet size (= physical memory assigned to > >> the > >> >> > > process) > >> >> > > - current and peak commit charge (also known as "private bytes" in > >> >> > Windows > >> >> > > tools) > >> >> > > - on 32bit Windows : > >> >> > > user-mode portion/free user mode portion of virtual address- > space > >> >> > > (Total/AvailVirtual) because it shows how close do we get to the 2- > 4 > >> GB > >> >> per > >> >> > > process border. > >> >> > > > >> >> > > > >> >> > > - the current naming of "swap/free-swap" memory is a bit > misleading; > >> >> > > in the Windows world swap is a special page file used for UWP > apps. > >> >> > > (see Windows Internals : "The swap file" (chapter 5 Memory > >> >> > management) > >> >> > > Windows 8 added another page file called a swap file. It is ... > >> exclusively > >> >> > used > >> >> > > for UWP (Universal Windows Platform) apps. > >> >> > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the > >> UWP > >> >> > > related values, it is about page file sizes > >> >> > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just > >> >> virtual > >> >> > > memory might be more appropriate). > >> >> > > > >> >> > > > >> >> > > https://msdn.microsoft.com/de- > >> >> > > de/library/windows/desktop/aa366770(v=vs.85).aspx > >> >> > > > >> >> > > documents it in the following way: > >> >> > > ullTotalPageFile: > >> >> > > The current committed memory limit for the system or the current > >> >> process, > >> >> > > whichever is smaller,... > >> >> > > > >> >> > > ullAvailPageFile : > >> >> > > The maximum amount of memory the current process can commit, > in > >> >> > bytes. > >> >> > > This value is equal to or smaller than the system-wide available > >> commit > >> >> > value > >> >> > > > >> >> > > > >> >> > > > >> >> > > Aditionally I suggest having output of the various memory-values > in M > >> >> > > (megabyte) as well , the k (kilobyte) output sometimes gives very > >> high > >> >> and > >> >> > > unreadable numbers). > >> >> > > > >> >> > > > >> >> > > Could you please review my change ? > >> >> > > > >> >> > > > >> >> > > Bug : > >> >> > > > >> >> > > https://bugs.openjdk.java.net/browse/JDK-8202427 > >> >> > > > >> >> > > > >> >> > > Change : > >> >> > > > >> >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ > >> >> > > > >> >> > > > >> >> > > Thanks, Matthias > >> >> > > > >> > From HORIE at jp.ibm.com Fri May 18 15:12:42 2018 From: HORIE at jp.ibm.com (Michihiro Horie) Date: Sat, 19 May 2018 00:12:42 +0900 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: References: , <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> Message-ID: Dear all, I update the webrev: http://cr.openjdk.java.net/~mhorie/8154736/webrev.09/ With the release barrier before the CAS, new_obj can be observed from other threads. If the CAS succeeds, the current thread can use new_obj without barriers. If the CAS fails, "o->forwardee()" is ordered with respect to CAS by accessing the same memory location "_mark", so no barriers needed. The order of (1) access to the forwardee and (2) access to forwardee's fields is preserved due to Release-Consume ordering on supported platforms. (The ordering between "new_obj = o->forwardee();" and logging or other usages is not changed.) Regarding the maintainability, the requirement is CAS(memory_order_release) as Release-Consume to be consistent with C++11. This requirement is necessary when a weaker platform like DEC Alpha is to be supported. On currently supported platforms, code change can be safe if the code meats this requirement (and the order of (1) access to the forwardee and (2) access to forwardee's fields is the natural way of coding). oop PSPromotionManager::copy_to_survivor_space(oop o) { oop new_obj = NULL; markOop test_mark = o->mark_raw(); if (!test_mark->is_marked()) { // The same test as "o->is_forwarded()" : Copy::aligned_disjoint_words((HeapWord*)o, (HeapWord*)new_obj, ...); if (o->cas_forward_to(new_obj, test_mark)) { // CAS succeeds : new_obj->push_contents(this); } else { // CAS fails : new_obj = o->forwardee(); } } else { : new_obj = o->forwardee(); } log_develop_trace(gc, scavenge)(..., new_obj->klass()->internal_name(), p2i((void *)o), p2i((void *)new_obj), new_obj->size()); return new_obj; } Best regards, -- Michihiro, IBM Research - Tokyo ----- Original message ----- From: Kim Barrett To: David Holmes Cc: Michihiro Horie , ppc-aix-port-dev at openjdk.java.net, hotspot-dev at openjdk.java.net, hotspot-gc-dev at openjdk.java.net, Hiroshi H Horii Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 Date: Fri, Apr 27, 2018 6:03 AM > On Apr 25, 2018, at 8:45 AM, David Holmes wrote: > > Hi Michihiro, > > On 23/04/2018 8:33 PM, Michihiro Horie wrote: >> Dear all, >> I would like to ask reviews on 8154736 ?enhancement of cmpxchg and >> copy_to_survivor?. The change adds options to avoid expensive syncs with >> compare-and-exchange. An experiment by using SPECjbb2015 showed 6% >> improvement in critical-jOPS. This change focuses on ppc64 but would be >> potentially beneficial for aarch64. >> Although discussions stopped at >> http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-October/002718.html >> , I would like to restart the review by taking over Hiroshi's work if the >> discussion is still open. > > So the very last comment there was about not implicitly assuming memory_order_consume, yet that has not been addressed in the proposal. > > Further the discussion on hotspot-runtime-dev through September and October was far more illuminating. I think my post here: > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-October/021617.html > > and the closely following one from Thomas Schatzl summed up the concerns about the proposed changes. > > AFAICS the restarted proposal addresses none of those concerns but simply takes up where the previous implementation suggestion left off. > > This is a proposal to change the memory ordering semantics of part of the shared GC code _not_ just the PPC64 implementation, but I have seen no analysis to demonstrate the correctness of such a proposal. I agree with David here. So far we've seen no such analysis. All we have seen is a series of proposed changes and non-failing test results, all of which have then been shown to have holes. (Among other things, this suggests the set of tests being applied is inadequate.) Part of the author's job is to convince reviewers that the proposed change is correct. I'm not expecting a formal proof, but I am expecting a lot more than has been provided so far. In this latest proposal, the conditional acquire doesn't look right to me. If the logging is disabled so there is no acquire, the object is then returned to the callers, who might do what with it? Is that safe? For all callers? And is it stable through future maintenance? This is not to say that I think making those acquires unconditional is sufficient. From erik.joelsson at oracle.com Fri May 18 15:29:43 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 18 May 2018 08:29:43 -0700 Subject: [8u] 8203349: 8u hotspot should recognise later Windows compilers In-Reply-To: <13e5ccc0-cdfb-aa02-1cfd-da966bfe4b4e@oracle.com> References: <672231ce-9a74-af4e-88db-8044e4a74ebe@oracle.com> <13e5ccc0-cdfb-aa02-1cfd-da966bfe4b4e@oracle.com> Message-ID: Hello Kevin, I have a machine with both 2015 and 2017 installed and checked the link version numbers: VS2015: C:\Program Files (x86)\Microsoft Visual Studio 14.0>link /? Microsoft (R) Incremental Linker Version 14.00.24215.1 Copyright (C) Microsoft Corporation.? All rights reserved. VS2017: C:\Users\erik\source>link /? Microsoft (R) Incremental Linker Version 14.12.25835.0 Copyright (C) Microsoft Corporation.? All rights reserved. So it looks like you need to add 1412 to checkLink in sanity.make. /Erik On 2018-05-18 01:51, Kevin Walls wrote: > Hi Erik - > > Quite right, thanks.? Actually I should include recognising VS2015 > while I'm doing this. > > Update: http://cr.openjdk.java.net/~kevinw/8203349/webrev.01/ > > Also, update Abstract_VM_Version::internal_vm_info_string()? in > src/share/vm/runtime/vm_version.cpp so the long version string is > readable for these compilers. > > In the sanity checks for VS2015, I am predicting it has a linker > version of 1900 as it's between the 1800 and 1900 that I have seen > myself for the other versions I have. > > I have a local VS2013 build, and builds using the unchanged "official" > compiler are still OK. > > Thanks > Kevin > > > > On 17/05/2018 23:32, Erik Joelsson wrote: >> Hello Kevin, >> >> In JDK 11, vm_version.cpp we have _MSC_VER == 1900 translating to >> VS2015 and 1912 to VS2017. Is it really 1900 in nmake? >> >> Otherwise I think this looks ok. >> >> /Erik >> >> >> On 2018-05-17 15:21, Kevin Walls wrote: >>> Hi, >>> >>> I'd like to get a review for this 8u/hotspot build change for >>> Windows, to loosen the restriction on what compilers we can use. >>> >>> JBS: >>> 8203349: 8u hotspot should recognise later Windows compilers >>> https://bugs.openjdk.java.net/browse/JDK-8203349 >>> >>> Webrev: http://cr.openjdk.java.net/~kevinw/8203349/webrev.00/ >>> >>> Changes in the places we (sanity) check version numbers, set some >>> compile >>> options, and expand the check in vm.make to make sure we put the >>> precompiled >>> object? _build_pch_file.obj on the jvm link command. >>> >>> In compile.make I added blocks for VS2013 and 17, and left them as >>> separate, >>> duplicated, blocks of settings to make them easier to change >>> independently. >>> >>> This doesn't change anything about what is "supported" or documented >>> as working. >>> >>> Thanks! >>> Kevin >>> >>> >> > From ioi.lam at oracle.com Fri May 18 16:36:28 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 18 May 2018 09:36:28 -0700 Subject: RFR 8203381 Replace InstanceKlass::allocate_instance_handle with JavaCalls::construct_new_instance Message-ID: <45eb3ef9-88c3-1bda-bb9e-122eb888dc35@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8203381 http://cr.openjdk.java.net/~iklam/jdk11/8203381-construct-new-instance.v01/ This patch removes a lot of the boilerplate code for allocating a Java object and calling its constructor. Hopefully the code is cleaner and easier to read. Also: Added assert(InstanceKlass::is_initialized(), ...) in case where we cannot call JavaCalls::construct_new_instance for thread objects. Replaced unnecessary SystemDictionary::resolve_or_null() calls with SystemDictionary::XXX_klass(). Tested with hs-tier[1,2,3,4,5] Thanks - Ioi From kevin.walls at oracle.com Fri May 18 16:53:05 2018 From: kevin.walls at oracle.com (Kevin Walls) Date: Fri, 18 May 2018 17:53:05 +0100 Subject: [8u] 8203349: 8u hotspot should recognise later Windows compilers In-Reply-To: References: <672231ce-9a74-af4e-88db-8044e4a74ebe@oracle.com> <13e5ccc0-cdfb-aa02-1cfd-da966bfe4b4e@oracle.com> Message-ID: <09c35443-c51d-68f9-d963-5be80692eb4e@oracle.com> Great, thanks, rebuilt the webrev. If we add many more there we might want to rethink or reword the sanity warning, but I'd like to save that for another time. 8-) Thanks Kevin On 18/05/2018 16:29, Erik Joelsson wrote: > Hello Kevin, > > I have a machine with both 2015 and 2017 installed and checked the > link version numbers: > > VS2015: > > C:\Program Files (x86)\Microsoft Visual Studio 14.0>link /? > Microsoft (R) Incremental Linker Version 14.00.24215.1 > Copyright (C) Microsoft Corporation.? All rights reserved. > > VS2017: > > C:\Users\erik\source>link /? > Microsoft (R) Incremental Linker Version 14.12.25835.0 > Copyright (C) Microsoft Corporation.? All rights reserved. > > So it looks like you need to add 1412 to checkLink in sanity.make. > > /Erik > > > On 2018-05-18 01:51, Kevin Walls wrote: >> Hi Erik - >> >> Quite right, thanks.? Actually I should include recognising VS2015 >> while I'm doing this. >> >> Update: http://cr.openjdk.java.net/~kevinw/8203349/webrev.01/ >> >> Also, update Abstract_VM_Version::internal_vm_info_string()? in >> src/share/vm/runtime/vm_version.cpp so the long version string is >> readable for these compilers. >> >> In the sanity checks for VS2015, I am predicting it has a linker >> version of 1900 as it's between the 1800 and 1900 that I have seen >> myself for the other versions I have. >> >> I have a local VS2013 build, and builds using the unchanged >> "official" compiler are still OK. >> >> Thanks >> Kevin >> >> >> >> On 17/05/2018 23:32, Erik Joelsson wrote: >>> Hello Kevin, >>> >>> In JDK 11, vm_version.cpp we have _MSC_VER == 1900 translating to >>> VS2015 and 1912 to VS2017. Is it really 1900 in nmake? >>> >>> Otherwise I think this looks ok. >>> >>> /Erik >>> >>> >>> On 2018-05-17 15:21, Kevin Walls wrote: >>>> Hi, >>>> >>>> I'd like to get a review for this 8u/hotspot build change for >>>> Windows, to loosen the restriction on what compilers we can use. >>>> >>>> JBS: >>>> 8203349: 8u hotspot should recognise later Windows compilers >>>> https://bugs.openjdk.java.net/browse/JDK-8203349 >>>> >>>> Webrev: http://cr.openjdk.java.net/~kevinw/8203349/webrev.00/ >>>> >>>> Changes in the places we (sanity) check version numbers, set some >>>> compile >>>> options, and expand the check in vm.make to make sure we put the >>>> precompiled >>>> object? _build_pch_file.obj on the jvm link command. >>>> >>>> In compile.make I added blocks for VS2013 and 17, and left them as >>>> separate, >>>> duplicated, blocks of settings to make them easier to change >>>> independently. >>>> >>>> This doesn't change anything about what is "supported" or >>>> documented as working. >>>> >>>> Thanks! >>>> Kevin >>>> >>>> >>> >> > From erik.joelsson at oracle.com Fri May 18 16:54:30 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 18 May 2018 09:54:30 -0700 Subject: [8u] 8203349: 8u hotspot should recognise later Windows compilers In-Reply-To: <09c35443-c51d-68f9-d963-5be80692eb4e@oracle.com> References: <672231ce-9a74-af4e-88db-8044e4a74ebe@oracle.com> <13e5ccc0-cdfb-aa02-1cfd-da966bfe4b4e@oracle.com> <09c35443-c51d-68f9-d963-5be80692eb4e@oracle.com> Message-ID: Looks good. /Erik On 2018-05-18 09:53, Kevin Walls wrote: > Great, thanks, rebuilt the webrev. > > If we add many more there we might want to rethink or reword the > sanity warning, but I'd like to save that for another time. 8-) > > Thanks > Kevin > > > On 18/05/2018 16:29, Erik Joelsson wrote: >> Hello Kevin, >> >> I have a machine with both 2015 and 2017 installed and checked the >> link version numbers: >> >> VS2015: >> >> C:\Program Files (x86)\Microsoft Visual Studio 14.0>link /? >> Microsoft (R) Incremental Linker Version 14.00.24215.1 >> Copyright (C) Microsoft Corporation.? All rights reserved. >> >> VS2017: >> >> C:\Users\erik\source>link /? >> Microsoft (R) Incremental Linker Version 14.12.25835.0 >> Copyright (C) Microsoft Corporation.? All rights reserved. >> >> So it looks like you need to add 1412 to checkLink in sanity.make. >> >> /Erik >> >> >> On 2018-05-18 01:51, Kevin Walls wrote: >>> Hi Erik - >>> >>> Quite right, thanks.? Actually I should include recognising VS2015 >>> while I'm doing this. >>> >>> Update: http://cr.openjdk.java.net/~kevinw/8203349/webrev.01/ >>> >>> Also, update Abstract_VM_Version::internal_vm_info_string() in >>> src/share/vm/runtime/vm_version.cpp so the long version string is >>> readable for these compilers. >>> >>> In the sanity checks for VS2015, I am predicting it has a linker >>> version of 1900 as it's between the 1800 and 1900 that I have seen >>> myself for the other versions I have. >>> >>> I have a local VS2013 build, and builds using the unchanged >>> "official" compiler are still OK. >>> >>> Thanks >>> Kevin >>> >>> >>> >>> On 17/05/2018 23:32, Erik Joelsson wrote: >>>> Hello Kevin, >>>> >>>> In JDK 11, vm_version.cpp we have _MSC_VER == 1900 translating to >>>> VS2015 and 1912 to VS2017. Is it really 1900 in nmake? >>>> >>>> Otherwise I think this looks ok. >>>> >>>> /Erik >>>> >>>> >>>> On 2018-05-17 15:21, Kevin Walls wrote: >>>>> Hi, >>>>> >>>>> I'd like to get a review for this 8u/hotspot build change for >>>>> Windows, to loosen the restriction on what compilers we can use. >>>>> >>>>> JBS: >>>>> 8203349: 8u hotspot should recognise later Windows compilers >>>>> https://bugs.openjdk.java.net/browse/JDK-8203349 >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~kevinw/8203349/webrev.00/ >>>>> >>>>> Changes in the places we (sanity) check version numbers, set some >>>>> compile >>>>> options, and expand the check in vm.make to make sure we put the >>>>> precompiled >>>>> object? _build_pch_file.obj on the jvm link command. >>>>> >>>>> In compile.make I added blocks for VS2013 and 17, and left them as >>>>> separate, >>>>> duplicated, blocks of settings to make them easier to change >>>>> independently. >>>>> >>>>> This doesn't change anything about what is "supported" or >>>>> documented as working. >>>>> >>>>> Thanks! >>>>> Kevin >>>>> >>>>> >>>> >>> >> > From leonid.mesnik at oracle.com Fri May 18 17:07:58 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Fri, 18 May 2018 10:07:58 -0700 Subject: RFR: 8199064: Test applications/jcstress/other/Test.java#id1108 fails on Sparc In-Reply-To: <5830D038-FF0D-40FE-BA69-BF1D40AFD94B@oracle.com> References: <4A1B218C-66B9-433A-9BA9-BA495CDF4EA9@oracle.com> <5830D038-FF0D-40FE-BA69-BF1D40AFD94B@oracle.com> Message-ID: <866B8FB1-2E73-4EED-A6CC-569727F4FEDC@oracle.com> Paul Thank you for review. You are right, I just forget to mention that most if files are generated. I agree that this timeout is very high for most of tests. However please note that: 1) It affect execution time only when test really hang and killed by timeout. 2) These tests are executed in hotspot tier7 only. They are executed once a week during PIT only and during ATR testing only. So I think that we don't need to optimize them right now. Please file RFE if you think that it is worth to improve it. Leonid > On May 17, 2018, at 6:41 PM, Paul Sandoz wrote: > > Looks good. > > I was initially taken aback by all the changes but quickly realized they are all generated, the important change being in TestGenerator.java. > > Do you have a sense of the range of times based on the JcstressGroup? I wonder if we could be a little smarter on timeout value. > > Paul. > >> On May 16, 2018, at 3:14 PM, Leonid Mesnik wrote: >> >> Hi >> >> Could you please review following patch which increase timeout for jcstress test wrappers in jtreg. >> >> The default jtreg timeout is not enough for a lot of jcstress tests. The longest tests take more then 1800 seconds on typical VM used for testing (2 CPU/16GB of Memory) with default VM options. >> These tests are not executed in tier1/2 testing so execution time in the case if test really hang is not very critical. I just set timeout to 2400 seconds for all test wrappers. >> The timeoutfactor could be used to adjust timeouts for slow platforms and/or VM options. >> >> >> webrev: http://cr.openjdk.java.net/~lmesnik/8199064/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8199064 >> >> Leonid > From paul.sandoz at oracle.com Fri May 18 17:31:19 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 18 May 2018 10:31:19 -0700 Subject: RFR: 8199064: Test applications/jcstress/other/Test.java#id1108 fails on Sparc In-Reply-To: <866B8FB1-2E73-4EED-A6CC-569727F4FEDC@oracle.com> References: <4A1B218C-66B9-433A-9BA9-BA495CDF4EA9@oracle.com> <5830D038-FF0D-40FE-BA69-BF1D40AFD94B@oracle.com> <866B8FB1-2E73-4EED-A6CC-569727F4FEDC@oracle.com> Message-ID: > On May 18, 2018, at 10:07 AM, Leonid Mesnik wrote: > > Paul > > Thank you for review. You are right, I just forget to mention that most if files are generated. > I agree that this timeout is very high for most of tests. However please note that: > 1) It affect execution time only when test really hang and killed by timeout. > 2) These tests are executed in hotspot tier7 only. They are executed once a week during PIT only and during ATR testing only. > > So I think that we don't need to optimize them right now. Ok, that's fine, Paul. > Please file RFE if you think that it is worth to improve it. > From igor.ignatyev at oracle.com Fri May 18 17:50:24 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 18 May 2018 10:50:24 -0700 Subject: RFR(L) : 8202812 : [TESTBUG] Open source VM testbase compiler tests Message-ID: http://cr.openjdk.java.net/~iignatyev//8202812/webrev.00/index.html > 56733 lines changed: 56732 ins; 0 del; 1 mod; 1 Hi all, could you please review the patch which open sources compiler tests from vm testbase? these tests were developed in different time to cover different parts of JITs. As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. JBS: https://bugs.openjdk.java.net/browse/JDK-8202812 webrev: http://cr.openjdk.java.net/~iignatyev/8202812/webrev.00/index.html testing: :vmTestbase_vm_compiler test group Thanks, -- Igor From yumin.qi at gmail.com Fri May 18 18:04:45 2018 From: yumin.qi at gmail.com (yumin qi) Date: Fri, 18 May 2018 11:04:45 -0700 Subject: RFR 8203381 Replace InstanceKlass::allocate_instance_handle with JavaCalls::construct_new_instance In-Reply-To: <45eb3ef9-88c3-1bda-bb9e-122eb888dc35@oracle.com> References: <45eb3ef9-88c3-1bda-bb9e-122eb888dc35@oracle.com> Message-ID: Hi, Ioi Very nice clean-up for Thread creation. One question, should we replace all the JavaCalls::call_special with JavaCalls::construct_new_instance (there are three versions) and make JavaCalls::call_special an internal implementation? I did see many places using JavaCalls::call_special. Thanks Yumin On Fri, May 18, 2018 at 9:36 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8203381 > http://cr.openjdk.java.net/~iklam/jdk11/8203381-construct-ne > w-instance.v01/ > > This patch removes a lot of the boilerplate code for allocating a > Java object and calling its constructor. Hopefully the code is cleaner > and easier to read. > > Also: > > Added assert(InstanceKlass::is_initialized(), ...) in case where > we cannot call JavaCalls::construct_new_instance for thread objects. > > Replaced unnecessary SystemDictionary::resolve_or_null() calls with > SystemDictionary::XXX_klass(). > > Tested with hs-tier[1,2,3,4,5] > > Thanks > - Ioi > > > > > > From igor.ignatyev at oracle.com Fri May 18 18:38:12 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 18 May 2018 11:38:12 -0700 Subject: RFR(XS) : 8203437 : 8199370 broke build on linux-ppc64le (w/ GCC 4.8.5.) Message-ID: <31AF805E-B9BE-4BCC-BE81-6111216B2D93@oracle.com> http://cr.openjdk.java.net/~iignatyev//8203437/webrev.00/index.html > 1 line changed: 0 ins; 0 del; 1 mod; Hi all, could you please review the one-line patch which fix builds on gcc 4.8? JBS: https://bugs.openjdk.java.net/browse/JDK-8203437 webrev: http://cr.openjdk.java.net/~iignatyev//8203437/webrev.00 testing: 'make test-image' w/ gcc4.8.2. Thanks, -- Igor From shade at redhat.com Fri May 18 18:48:02 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 18 May 2018 20:48:02 +0200 Subject: RFR(XS) : 8203437 : 8199370 broke build on linux-ppc64le (w/ GCC 4.8.5.) In-Reply-To: <31AF805E-B9BE-4BCC-BE81-6111216B2D93@oracle.com> References: <31AF805E-B9BE-4BCC-BE81-6111216B2D93@oracle.com> Message-ID: <9957f149-ac01-a069-cdc9-ac70c73b70eb@redhat.com> On 05/18/2018 08:38 PM, Igor Ignatyev wrote: > JBS: https://bugs.openjdk.java.net/browse/JDK-8203437 > webrev: http://cr.openjdk.java.net/~iignatyev//8203437/webrev.00 Looks OK. Not immediately evident why only those two fields require initialization. > testing: 'make test-image' w/ gcc4.8.2. Interestingly, my ppc64le builds are fine without this fix. Do I need to add "test-image" to CI scripts to compile the tests too? -Aleksey From kishor.kharbas at intel.com Fri May 18 19:10:28 2018 From: kishor.kharbas at intel.com (Kharbas, Kishor) Date: Fri, 18 May 2018 19:10:28 +0000 Subject: RFR(M): 8202286: Allocation of Old generation of Java Heap on alternate memory devices. Message-ID: Hi! I have proposed a JEP to add an experimental feature to allocate Old generation of Java Heap on alternate memory devices such as NV-DIMM memory. The link to the JEP is: https://bugs.openjdk.java.net/browse/JDK-8202286 Best regards, Kishor From igor.ignatyev at oracle.com Fri May 18 19:38:25 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 18 May 2018 12:38:25 -0700 Subject: RFR(XS) : 8203437 : 8199370 broke build on linux-ppc64le (w/ GCC 4.8.5.) In-Reply-To: <9957f149-ac01-a069-cdc9-ac70c73b70eb@redhat.com> References: <31AF805E-B9BE-4BCC-BE81-6111216B2D93@oracle.com> <9957f149-ac01-a069-cdc9-ac70c73b70eb@redhat.com> Message-ID: > On May 18, 2018, at 11:48 AM, Aleksey Shipilev wrote: > > On 05/18/2018 08:38 PM, Igor Ignatyev wrote: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8203437 >> webrev: http://cr.openjdk.java.net/~iignatyev//8203437/webrev.00 > > Looks OK. Not immediately evident why only those two fields require initialization. > >> testing: 'make test-image' w/ gcc4.8.2. > > Interestingly, my ppc64le builds are fine without this fix. Do I need to add "test-image" to CI > scripts to compile the tests too? I've double-checked 'make images' doesn't build test image, actually it's just an alias for 'product-images', so yes 'test-image' is needed to compile native part of tests. alternatively, you can use 'all-images' target which builds product, test and doc images. -- Igor > > -Aleksey > From coleen.phillimore at oracle.com Fri May 18 22:36:16 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 18 May 2018 18:36:16 -0400 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: Message-ID: <8fe8a029-04f8-415a-efc6-3adef875255c@oracle.com> Hi, I haven't been following along or read through all of this, but I had a look through the code and have some questions about it. http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/src/hotspot/share/prims/jvmtiRedefineClasses.cpp.udiff.html There is commented out code at line 806 in compare_and_normalize_class_versions.?? Could you make the added code a separate function defined above so this isn't the longest function in the jvm? 739 JvmtiThreadState *state = JvmtiThreadState::state_for((JavaThread*)thread); 740 RedefineVerifyMark rvm(the_class, scratch_class, state); Why does this need RedefineVerifyMark?? It doesn't call the verifier. http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/src/hotspot/share/runtime/reflection.cpp.frames.html 720 bool access = cur_ik->has_nestmate_access_to(field_ik, THREAD); 721 if (HAS_PENDING_EXCEPTION) { 722 return false; 723 } This looks suspicious.?? Do you want to CLEAR_PENDING_EXCEPTION? This looks like the combination of these functions drops exceptions, that could show up in inconvenient places.? Or add TRAPS to reflection::verify_field_access so that the callers properly handle the pending exception? This is CHECK_false. and also maybe has_nestmate_access_to, should pass CHECK_false to its nest_host() calls, so it doesn't look so weird. http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/src/hotspot/share/oops/instanceKlass.cpp.udiff.html + // names match so check actual klass - this may trigger class loading if + // it doesn't match (but that should be impossible) + Klass* k2 = _constants->klass_at(cp_index, CHECK_false); If throwing an exception is impossible, this should probably have a CATCH.? This is code is odd about exceptions.? It looks like has_nestmate_access() + } + else { Oh please can you fix these?? The hotspot style is the? } else { all on one line (OTBS). Since has_nest_member is only used in InstanceKlass, it should have private access. + bool has_nest_member(InstanceKlass* k, TRAPS) const; Looks like raw_nest_host() is uncalled (and shouldn't be public). That's all I can do for now.? I think you have good reviewers for the feature itself, but if you want more, I'll spend more time with this. thanks, Coleen On 5/14/18 8:52 PM, David Holmes wrote: > This review is being spread across four groups: langtools, core-libs, > hotspot and serviceability. This is the specific review thread for > hotspot - webrev: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ > > See below for full details - including annotated full webrev guiding > the review. > > The intent is to have JEP-181 targeted and integrated by the end of > this month. > > Thanks, > David > ----- > > The nestmates project (JEP-181) introduces new classfile attributes to > identify classes and interfaces in the same nest, so that the VM can > perform access control based on those attributes and so allow direct > private access between nestmates without requiring javac to generate > synthetic accessor methods. These access control changes also extend > to core reflection and the MethodHandle.Lookup contexts. > > Direct private calls between nestmates requires a more general calling > context than is permitted by invokespecial, and so the JVMS is updated > to allow, and javac updated to use, invokevirtual and invokeinterface > for private class and interface method calls respectively. These > changed semantics also extend to MethodHandle findXXX operations. > > At this time we are only concerned with static nest definitions, which > map to a top-level class/interface as the nest-host and all its nested > types as nest-members. > > Please see the JEP for further details. > > JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 > Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 > CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 > > All of the specification changes have been previously been worked out > by the Valhalla Project Expert Group, and the implementation reviewed > by the various contributors and discussed on the valhalla-dev mailing > list. > > Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, > Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir > Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan > > Master webrev of all changes: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ > > Annotated master webrev index: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html > > Performance: this is expected to be performance neutral in a general > sense. Benchmarking and performance runs are about to start. > > Testing Discussion: > ------------------ > > The testing for nestmates can be broken into four main groups: > > -? New tests specifically related to nestmates and currently in the > runtime/Nestmates directory > > - New tests to complement existing tests by adding in testcases not > previously expressible. > ? -? For example java/lang/invoke/SpecialInterfaceCall.java tests use > of invokespecial for private interface methods and performing receiver > typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do > similar tests for invokeinterface. > > -? New JVM TI tests to verify the spec changes related to nest > attributes. > > -? Existing tests significantly affected by the nestmates changes, > primarily: > ?? -? runtime/SelectionResolution > > ?? In most cases the nestmate changes makes certain invocations that > were illegal, legal (e.g. not requiring invokespecial to invoke > private interface methods; allowing access to private members via > reflection/Methodhandles that were previously not allowed). > > - Existing tests incidentally affected by the nestmate changes > > ? This includes tests of things utilising class > redefinition/retransformation to alter nested types but which > unintentionally alter nest relationships (which is not permitted). > > There are still a number of tests problem-listed with issues filed > against them to have them adapted to work with nestmates. Some of > these are intended to be addressed in the short-term, while some (such > as the runtime/SelectionResolution test changes) may not eventuate. > > - https://bugs.openjdk.java.net/browse/JDK-8203033 > - https://bugs.openjdk.java.net/browse/JDK-8199450 > - https://bugs.openjdk.java.net/browse/JDK-8196855 > - https://bugs.openjdk.java.net/browse/JDK-8194857 > - https://bugs.openjdk.java.net/browse/JDK-8187655 > > There is also further test work still to be completed (the JNI and JDI > invocation tests): > - https://bugs.openjdk.java.net/browse/JDK-8191117 > which will continue in parallel with the main RFR. > > Pre-integration Testing: > ?- General: > ??? - Mach5: hs/jdk tier1,2 > ??? - Mach5: hs-nightly (tiers 1 -3) > ?- Targetted > ?? - nashorn (for asm changes) > ?? - hotspot: runtime/* > ????????????? serviceability/* > ????????????? compiler/* > ????????????? vmTestbase/* > ?? - jdk: java/lang/invoke/* > ????????? java/lang/reflect/* > ????????? java/lang/instrument/* > ????????? java/lang/Class/* > ????????? java/lang/management/* > ? - langtools: tools/javac > ?????????????? tools/javap > From coleen.phillimore at oracle.com Fri May 18 23:14:47 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 18 May 2018 19:14:47 -0400 Subject: RFR: 8202863: Rename OopStorage inner collection classes In-Reply-To: References: Message-ID: Also, this looks trivial enough to push with one reviewer. Coleen On 5/17/18 5:34 PM, Kim Barrett wrote: >> On May 17, 2018, at 4:44 PM, coleen.phillimore at oracle.com wrote: >> >> Looks good! >> Coleen > Thanks. > >> On 5/17/18 4:33 PM, Kim Barrett wrote: >>> [Resending hotspot-dev too, so Coleen will see it, since she requested this change.] >>> >>> Please review this simple renaming of some private inner classes of >>> OopStorage. The renamings were suggested during the review of >>> JDK-8200557, but were deferred to a separate change to avoid adding >>> complexity to that review. The following classes are being renamed: >>> >>> BlockArray => ActiveArray >>> BlockList => AllocateList >>> BlockEntry => AllocateEntry >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8202863 >>> >>> Webrev: >>> http://cr.openjdk.java.net/~kbarrett/8202863/open.00/ >>> >>> Testing: >>> Mach5 CI equivalent. > From serguei.spitsyn at oracle.com Sat May 19 01:42:54 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Fri, 18 May 2018 18:42:54 -0700 Subject: RFR(L) : 8199379 : [TESTBUG] Open source vm testbase JDB tests In-Reply-To: <6D060F7C-C265-48CF-BB21-532848461B2A@oracle.com> References: <6D060F7C-C265-48CF-BB21-532848461B2A@oracle.com> Message-ID: Hi Igor, This looks good to me. Thank you a lot for taking care about this move from closed to open! Thanks, Serguei On 5/15/18 14:21, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8199379/webrev.00/index.html >> 18000 lines changed: 17999 ins; 0 del; 1 mod; > > Hi all, > > could you please review this patch which open sources JDB tests from VM testbase? > > these tests were developed to test 'jdb' tool and JDI thru it. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199379 > webrev: http://cr.openjdk.java.net/~iignatyev//8199379/webrev.00/index.html > testing: :vmTestbase_nsk_jdb test group > > Thanks, > -- Igor From ioi.lam at oracle.com Sat May 19 01:46:32 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 18 May 2018 18:46:32 -0700 Subject: RFR 8203381 Replace InstanceKlass::allocate_instance_handle with JavaCalls::construct_new_instance In-Reply-To: References: <45eb3ef9-88c3-1bda-bb9e-122eb888dc35@oracle.com> Message-ID: <7acb71f5-584f-9447-e7c5-e7bc3c06ee99@oracle.com> Hi Yumin, Thanks for the review. In some cases, such as create_initial_thread, we must do ik->allocate_instance_handle() and JavaCalls::call_special in 2 steps, and set some thread states in between the 2 steps. Also, there are other use of JavaCalls::call_special for invoke instance methods that are not constructors. Therefore, we cannot hide JavaCalls::call_special. Thanks - Ioi On 5/18/18 11:04 AM, yumin qi wrote: > Hi, Ioi > > ? Very nice clean-up for Thread creation. > ? One question, should we replace all? the JavaCalls::call_special > with JavaCalls::construct_new_instance (there are three versions) and > make JavaCalls::call_special an internal implementation? I did see > many places using JavaCalls::call_special. > > Thanks > Yumin > > On Fri, May 18, 2018 at 9:36 AM, Ioi Lam > wrote: > > https://bugs.openjdk.java.net/browse/JDK-8203381 > > http://cr.openjdk.java.net/~iklam/jdk11/8203381-construct-new-instance.v01/ > > > This patch removes a lot of the boilerplate code for allocating a > Java object and calling its constructor. Hopefully the code is cleaner > and easier to read. > > Also: > > Added assert(InstanceKlass::is_initialized(), ...) in case where > we cannot call JavaCalls::construct_new_instance for thread objects. > > Replaced unnecessary SystemDictionary::resolve_or_null() calls with > SystemDictionary::XXX_klass(). > > Tested with hs-tier[1,2,3,4,5] > > Thanks > - Ioi > > > > > > From thomas.stuefe at gmail.com Sat May 19 05:24:06 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 19 May 2018 07:24:06 +0200 Subject: [RFR] 8202427: Enhance os::print_memory_info on Windows In-Reply-To: <219b9940eaa14dd6805ac5360a1733aa@sap.com> References: <8e76b12f2d0e4b04acbda1a3c0d356a3@sap.com> <219b9940eaa14dd6805ac5360a1733aa@sap.com> Message-ID: Hi Matthias, sounds good. ..Thomas On Fri, May 18, 2018 at 5:10 PM, Baesken, Matthias wrote: > Hi Thomas , thanks looking into it. > Indeed GetProcessMemoryInfo could fail , and GetProcessMemoryInfo could fail as well . > > MSDN says for both functions . > > https://msdn.microsoft.com/de-de/library/windows/desktop/ms683219(v=vs.85).aspx > > https://msdn.microsoft.com/de-de/library/windows/desktop/aa366589(v=vs.85).aspx > > .......... > > Return value > If the function succeeds, the return value is nonzero. > If the function fails, the return value is zero. > > > I add return code checking for failure cases . (plus some line feeds in the coding which is desired by Goetz ). > > > For GlobalMemoryStatusEx there seem to be quite a few places where return code checking is missing , this should be addressed in a follow up change : > > > OperatingSystemImpl.c > 103 GlobalMemoryStatusEx(&ms); > 113 GlobalMemoryStatusEx(&ms); > 142 GlobalMemoryStatusEx(&ms); > 152 GlobalMemoryStatusEx(&ms); > > > os_windows.cpp > 736 GlobalMemoryStatusEx(&ms); > 748 GlobalMemoryStatusEx(&ms); > 1748 GlobalMemoryStatusEx(&ms); > 3669 GlobalMemoryStatusEx(&ms); > > > Best regards, Matthias > >> -----Original Message----- >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] >> Sent: Freitag, 18. Mai 2018 08:16 >> To: Baesken, Matthias >> Cc: Lindenmaier, Goetz ; Doerr, Martin >> ; hotspot-dev at openjdk.java.net; hotspot- >> runtime-dev at openjdk.java.net >> Subject: Re: [RFR] 8202427: Enhance os::print_memory_info on Windows >> >> Hi Matthias, >> >> I took another closer look and have some minor nits. Sorry for not >> looking better the first time, my head was not clear :) >> >> Since that was my fault I leave it up to you if you follow up. The fix >> in its current form is also fine to me. >> >> - I am not a big fan of ">> 20". I would change that to " / M" to make >> the code more readable. >> - Strictly speaking GetProcessMemoryInfo could fail (I have never seen >> that though) and return FALSE. In which case we should probably not >> output the results. >> >> Best Regards, Thomas >> >> >> On Thu, May 17, 2018 at 5:20 PM, Baesken, Matthias >> wrote: >> >> > > I also think that one number is enough. Adaptive numbers would be >> great. >> > >> > >> > Hi Thomas/Goetz/Martin , thanks for looking into it. >> > Adaptive numbers could be done in a follow-up as suggested by Thomas , >> this would also touch the other platforms . >> > >> > I added more line feeds to the output and reduced output to just printing >> M . >> > >> > >> > Example output from a Windows desktop PC : >> > >> > Memory: 4k page, system-wide physical 32520M (20165M free) >> > TotalPageFile size 34568M (AvailPageFile size 18170M) >> > current process WorkingSet (physical memory assigned to process): 79M, >> peak: 79M >> > current process commit charge ("private bytes"): 148M, peak: 148M >> > >> > >> > new webrev : >> > >> > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.1/ >> > >> > >> > Best regards, Matthias >> > >> > >> > >> >> -----Original Message----- >> >> From: Thomas St?fe [mailto:thomas.stuefe at gmail.com] >> >> Sent: Mittwoch, 16. Mai 2018 16:35 >> >> To: Lindenmaier, Goetz >> >> Cc: Baesken, Matthias ; Doerr, Martin >> >> ; hotspot-dev at openjdk.java.net; hotspot- >> >> runtime-dev at openjdk.java.net >> >> Subject: Re: [RFR] 8202427: Enhance os::print_memory_info on Windows >> >> >> >> On Wed, May 16, 2018 at 3:48 PM, Lindenmaier, Goetz >> >> wrote: >> >> > Hi Matthias, >> >> > >> >> > I appreciate the additional information in the hs_err file. >> >> > Could you please add an example output (of the final version) to the >> bug >> >> description? >> >> > >> >> > As Martin, I would like more line feeds, both in the code and the output. >> >> > Currently, the output in the hs_err file looks like this (where the first >> line is >> >> too long): >> >> > Memory: 4k page, system-wide physical 16776692k [16383M] >> (8795032k >> >> [8588M] free), TotalPageFile size 49949460k [48778M] (AvailPageFile size >> >> 38942152k [38029M]), user-mode portion of virtual address-space >> 2097024k >> >> [2047M] (6940k [6M] free) >> >> > current process WorkingSet (physical memory assigned to process): >> >> 991804k [968M], peak: 991804k [968M] >> >> > >> >> > current process commit charge ("private bytes"): 1523632k [1487M], >> >> peak: 1523632k [1487M] >> >> > >> >> > I also think that one number is enough. Adaptive numbers would be >> great. >> >> > I know about >> >> > src/hotspot/share/memory/metaspace/metaspaceCommon.hpp: >> >> print_human_readable_size() >> >> > for this, but this seems a bit hidden in metaspace. >> >> > >> >> >> >> Guys, lets not worry about human readable numbers for now. We can >> move >> >> print_human_readable_size() out from metaspace to something generic >> >> and use it to improve this code (and others) in a later patch. That >> >> was my original intention when adding print_human_readable_size(). >> >> >> >> The patch is good. Thanks for doing this. >> >> >> >> ..Thoams >> >> >> >> > I don't like usage of _M_IX86. Common in this file seems #ifndef >> _WIN64 >> >> for 32-bit windows >> >> > dependent code. But don't care here. >> >> > >> >> > Best regards, >> >> > Goetz. >> >> > >> >> > >> >> >> -----Original Message----- >> >> >> From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] >> On >> >> >> Behalf Of Baesken, Matthias >> >> >> Sent: Mittwoch, 16. Mai 2018 15:13 >> >> >> To: Doerr, Martin ; 'hotspot- >> >> >> dev at openjdk.java.net' ; hotspot- >> >> >> runtime-dev at openjdk.java.net >> >> >> Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on >> Windows >> >> >> >> >> >> Ping : could I get a second review ? >> >> >> >> >> >> Bug : >> >> >> https://bugs.openjdk.java.net/browse/JDK-8202427 >> >> >> Change : >> >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ >> >> >> >> >> >> >> >> >> > We could also just print the MB value , let's see what other think. >> >> >> > Another option might be to have a flexible output (kB for smaller >> >> >> memory >> >> >> > values , MB (or GB) for larger ) ? >> >> >> >> >> >> Martin suggested to just print the MB values, what do you think ? >> >> >> >> >> >> >> >> >> Thanks, Matthias >> >> >> >> >> >> >> >> >> > -----Original Message----- >> >> >> > From: Baesken, Matthias >> >> >> > Sent: Mittwoch, 2. Mai 2018 13:00 >> >> >> > To: Doerr, Martin ; 'hotspot- >> >> >> > dev at openjdk.java.net' ; hotspot- >> >> >> > runtime-dev at openjdk.java.net >> >> >> > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on >> >> Windows >> >> >> > >> >> >> > Hi Martin, thanks for your input . >> >> >> > I add hotspot-runtime-dev . >> >> >> > >> >> >> > > >> >> >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe >> >> other >> >> >> > > reviewers would like to comment on this, too. >> >> >> > > >> >> >> > >> >> >> > We could also just print the MB value , let's see what other think. >> >> >> > Another option might be to have a flexible output (kB for smaller >> >> >> memory >> >> >> > values , MB (or GB) for larger ) ? >> >> >> > >> >> >> > Best regards, Matthias >> >> >> > >> >> >> > >> >> >> > > -----Original Message----- >> >> >> > > From: Doerr, Martin >> >> >> > > Sent: Mittwoch, 2. Mai 2018 12:53 >> >> >> > > To: Baesken, Matthias ; 'hotspot- >> >> >> > > dev at openjdk.java.net' >> >> >> > > Subject: RE: [RFR] 8202427: Enhance os::print_memory_info on >> >> Windows >> >> >> > > >> >> >> > > Hi Matthias, >> >> >> > > >> >> >> > > looks like a nice enhancement. We can get substantially more >> >> >> information. >> >> >> > > >> >> >> > > I wonder if we really need the sizes in kB in addition to MB. Maybe >> >> other >> >> >> > > reviewers would like to comment on this, too. >> >> >> > > >> >> >> > > We should have line breaks. >> >> >> > > >> >> >> > > Best regards, >> >> >> > > Martin >> >> >> > > >> >> >> > > >> >> >> > > -----Original Message----- >> >> >> > > From: hotspot-dev [mailto:hotspot-dev- >> bounces at openjdk.java.net] >> >> On >> >> >> > > Behalf Of Baesken, Matthias >> >> >> > > Sent: Montag, 30. April 2018 16:53 >> >> >> > > To: 'hotspot-dev at openjdk.java.net' > >> dev at openjdk.java.net> >> >> >> > > Subject: [RFR] 8202427: Enhance os::print_memory_info on >> Windows >> >> >> > > >> >> >> > > On Windows, >> >> >> > > the os::print_memory_info misses a few memory-related infos >> that >> >> >> might >> >> >> > > be interesting : >> >> >> > > - current and peak WorkingSet size (= physical memory assigned to >> >> the >> >> >> > > process) >> >> >> > > - current and peak commit charge (also known as "private bytes" in >> >> >> > Windows >> >> >> > > tools) >> >> >> > > - on 32bit Windows : >> >> >> > > user-mode portion/free user mode portion of virtual address- >> space >> >> >> > > (Total/AvailVirtual) because it shows how close do we get to the 2- >> 4 >> >> GB >> >> >> per >> >> >> > > process border. >> >> >> > > >> >> >> > > >> >> >> > > - the current naming of "swap/free-swap" memory is a bit >> misleading; >> >> >> > > in the Windows world swap is a special page file used for UWP >> apps. >> >> >> > > (see Windows Internals : "The swap file" (chapter 5 Memory >> >> >> > management) >> >> >> > > Windows 8 added another page file called a swap file. It is ... >> >> exclusively >> >> >> > used >> >> >> > > for UWP (Universal Windows Platform) apps. >> >> >> > > Our output (ullTotalPageFile and ullAvailPageFile) is NOT about the >> >> UWP >> >> >> > > related values, it is about page file sizes >> >> >> > > (so calling it TotalPageFile/AvailPageFile in "Windows-speak" or just >> >> >> virtual >> >> >> > > memory might be more appropriate). >> >> >> > > >> >> >> > > >> >> >> > > https://msdn.microsoft.com/de- >> >> >> > > de/library/windows/desktop/aa366770(v=vs.85).aspx >> >> >> > > >> >> >> > > documents it in the following way: >> >> >> > > ullTotalPageFile: >> >> >> > > The current committed memory limit for the system or the current >> >> >> process, >> >> >> > > whichever is smaller,... >> >> >> > > >> >> >> > > ullAvailPageFile : >> >> >> > > The maximum amount of memory the current process can commit, >> in >> >> >> > bytes. >> >> >> > > This value is equal to or smaller than the system-wide available >> >> commit >> >> >> > value >> >> >> > > >> >> >> > > >> >> >> > > >> >> >> > > Aditionally I suggest having output of the various memory-values >> in M >> >> >> > > (megabyte) as well , the k (kilobyte) output sometimes gives very >> >> high >> >> >> and >> >> >> > > unreadable numbers). >> >> >> > > >> >> >> > > >> >> >> > > Could you please review my change ? >> >> >> > > >> >> >> > > >> >> >> > > Bug : >> >> >> > > >> >> >> > > https://bugs.openjdk.java.net/browse/JDK-8202427 >> >> >> > > >> >> >> > > >> >> >> > > Change : >> >> >> > > >> >> >> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8202427.0/ >> >> >> > > >> >> >> > > >> >> >> > > Thanks, Matthias >> >> >> > > >> >> > From bsrbnd at gmail.com Sat May 19 16:31:40 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sat, 19 May 2018 18:31:40 +0200 Subject: RFR 8202835: jfr/event/os/TestSystemProcess.java fails on missing events In-Reply-To: References: Message-ID: On 17 May 2018 at 22:10, B. Blaser wrote: > Hi, > > A recent change in 'os::readdir()' to use 'readdir' instead of the > deprecated 'readdir_r' implies to copy the returned 'dirent' if > necessary because 'readdir' uses its own buffer and not the one > provided by the caller as it was done before with 'readdir_r'. > The following fix is only intended to correct 'ProcessIterator', > further improvements will be addressed as part of: > https://bugs.openjdk.java.net/browse/JDK-8202353 > > Tier1 is OK. Looking closer at 'ProcessIterator', I note that copying 'dirent' is unnecessary. Here is the updated fix. Any comment or review is welcome. Thanks, Bernard diff -r 8e4fcfb4cfe4 src/hotspot/os/linux/os_perf_linux.cpp --- a/src/hotspot/os/linux/os_perf_linux.cpp Thu May 17 10:32:26 2018 +0200 +++ b/src/hotspot/os/linux/os_perf_linux.cpp Sat May 19 17:22:18 2018 +0200 @@ -903,21 +903,14 @@ } int SystemProcessInterface::SystemProcesses::ProcessIterator::next_process() { - struct dirent* entry; - if (!is_valid()) { return OS_ERR; } do { - entry = os::readdir(_dir, _entry); - if (entry == NULL) { - // error - _valid = false; - return OS_ERR; - } + _entry = os::readdir(_dir, NULL); if (_entry == NULL) { - // reached end + // reached end or error _valid = false; return OS_ERR; } @@ -935,10 +928,6 @@ bool SystemProcessInterface::SystemProcesses::ProcessIterator::initialize() { _dir = opendir("/proc"); - _entry = (struct dirent*)NEW_C_HEAP_ARRAY(char, sizeof(struct dirent) + NAME_MAX + 1, mtInternal); - if (NULL == _entry) { - return false; - } _valid = true; next_process(); @@ -946,9 +935,6 @@ } SystemProcessInterface::SystemProcesses::ProcessIterator::~ProcessIterator() { - if (_entry != NULL) { - FREE_C_HEAP_ARRAY(char, _entry); - } if (_dir != NULL) { closedir(_dir); } diff -r 8e4fcfb4cfe4 test/jdk/ProblemList.txt --- a/test/jdk/ProblemList.txt Thu May 17 10:32:26 2018 +0200 +++ b/test/jdk/ProblemList.txt Sat May 19 17:22:18 2018 +0200 @@ -829,5 +829,4 @@ jdk/jfr/event/io/TestInstrumentation.java 8202142 generic-all jdk/jfr/event/sampling/TestNative.java 8202142 generic-all -jdk/jfr/event/os/TestSystemProcess.java 8202835 linux-all -jdk/jfr/event/runtime/TestBiasedLockRevocationEvents.java 8203237 generic-all \ No newline at end of file +jdk/jfr/event/runtime/TestBiasedLockRevocationEvents.java 8203237 generic-all diff -r 8e4fcfb4cfe4 test/jdk/TEST.groups --- a/test/jdk/TEST.groups Thu May 17 10:32:26 2018 +0200 +++ b/test/jdk/TEST.groups Sat May 19 17:22:18 2018 +0200 @@ -465,6 +465,7 @@ jdk/jfr/api/recording/event/TestLoadEventAfterStart.java \ jdk/jfr/api/recording/state/TestState.java \ jdk/jfr/event/os/TestCPULoad.java \ + jdk/jfr/event/os/TestSystemProcess.java \ jdk/jfr/event/compiler/TestAllocInNewTLAB.java \ jdk/jfr/jcmd/TestJcmdStartStopDefault.java \ jdk/jfr/event/io/TestFileStreamEvents.java \ From david.holmes at oracle.com Mon May 21 00:35:19 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 21 May 2018 10:35:19 +1000 Subject: 8203341: Add a safepoint-aware Semaphore In-Reply-To: <322b2a9c-4dfe-9790-1c1d-b1c7cfa4a8ff@oracle.com> References: <2f8412d5-2fe4-2c8f-077c-cf96e8f9bebf@oracle.com> <5AFDA1E3.6010401@oracle.com> <385fc0f8-461d-5f0f-d678-1dcb69b173cf@oracle.com> <5AFEC537.4050304@oracle.com> <322b2a9c-4dfe-9790-1c1d-b1c7cfa4a8ff@oracle.com> Message-ID: <89f62752-c90f-02c0-ab5a-c3c1714cc3ae@oracle.com> Hi Stefan, This updated webrev only came out late Friday night for me so I did not get a chance to review before you pushed it. I would have argued for keeping consistency with existing similar API's and use the "bool safepoint_check" parameter instead of adding a new variant of the "wait" method. src/hotspot/share/runtime/semaphore.inline.hpp 2 * Copyright (c) 2015, 2018, How does a new file have a 2015 copyright? 41 inline void Semaphore::wait_with_safepoint_check() { 42 Thread* thread = Thread::current(); 43 if (thread->is_Java_thread()) { 44 wait_with_safepoint_check(static_cast(thread)); 45 } else { 46 // Wait for value 47 _impl.wait(); 48 } 49 } This seems misnamed - if not a JavaThread then there is no safepoint check. It seems confusing to me to now have three wait variants with no clear indicator as to whether specific types of threads should only call specific variants (and no assertions checking that). I would expect JavaThreads to always call wait_with_safepoint_check(JavaThread), while non-JavaThread's should only call wait(). The parameter-less wait_with_safepoint_check() doesn't seem desirable. I'm guessing it is there to allow the dispatching to be done by the semaphore code rather than having the callers call Thread::current() and dispatch themselves? But unless we may sometimes want JavaThreads to not perform safepoint checks (do we??), this seems overly complicated. We could have simply defined wait to take a Thread parameter (default NULL) and then decide whether to use ThreadBlockInVM or not based on whether it was a JavaThread. At most two variants seem needed (if you don't like potentially calling Thread::current() if you don't need the Thread) rather than three. Cheers, David ----- On 18/05/2018 10:45 PM, Stefan Karlsson wrote: > I got offline suggestions that I should rename the function to > wait_with_safepoint_check: > > http://cr.openjdk.java.net/~stefank/8203341/webrev.03/ > http://cr.openjdk.java.net/~stefank/8203341/webrev.03.delta/ > > Thanks, > StefanK > > On 2018-05-18 14:21, Erik ?sterlund wrote: >> Hi Stefan, >> >> Looks good. >> >> Thanks, >> /Erik >> >> On 2018-05-18 14:13, Stefan Karlsson wrote: >>> Updated webrevs: >>> ?http://cr.openjdk.java.net/~stefank/8203341/webrev.02/ >>> ?http://cr.openjdk.java.net/~stefank/8203341/webrev.02.delta/ >>> >>> I removed the SemaphoreSafepointAware class, as Erik requested, but >>> instead of adding a bool I added a new function for the safepoint >>> aware waits. >>> >>> Thanks, >>> StefanK >>> >>> On 2018-05-17 17:59, Stefan Karlsson wrote: >>>> On 2018-05-17 17:38, Erik ?sterlund wrote: >>>>> Hi Stefan, >>>>> >>>>> I wonder if it would make sense to let Semaphore::wait have a bool >>>>> safepoint_check, similar to Mutex (possibly with a default value), >>>>> instead of introducing a new class with the same API, that only >>>>> differs in this property. >>>> >>>> That sounds reasonable to me. >>>> >>>> I also need to change my patch so that the thread-local handshake >>>> code pass in the existing JavaThread, so that we don't have to call >>>> Thread::current()->is_JavaThread() unnecessarily. >>>> >>>> Thanks, >>>> StefanK >>>> >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2018-05-17 09:46, Stefan Karlsson wrote: >>>>>> Hi all, >>>>>> >>>>>> Please review this patch to add a safepoint-aware Semaphore (and >>>>>> refactor the semaphore usage in thread-local handshakes). >>>>>> >>>>>> http://cr.openjdk.java.net/~stefank/8203341/webrev.01/ >>>>>> https://bugs.openjdk.java.net/browse/JDK-8203341 >>>>>> >>>>>> Both ZGC and the thread-local handshakes need to move the >>>>>> JavaThread to a blocked state before waiting on a Semaphore. I >>>>>> propose that we introduce a new SemaphoreSafepointAware wrapper >>>>>> around Semaphore. >>>>>> >>>>>> There are two things to consider with this patch: >>>>>> >>>>>> 1) In ZGC, where this patch originates from, we set the >>>>>> JavaThread's osthread state with osthread->set_state(CONDVAR_WAIT) >>>>>> (using OSThreadWaitState). This means that we get the following >>>>>> printed when the threads are dumped: "waiting on condition ". I've >>>>>> kept this behavior for the SemaphoreSafepointAware class. This >>>>>> means that we now change the osthread state for JavaThreads >>>>>> blocking in HandshakeState::process_self_inner. >>>>>> >>>>>> 2) The thread-local handshakes uses trywait before entering wait: >>>>>> ?? if (!_semaphore.trywait()) { >>>>>> -??? ThreadBlockInVM tbivm(thread); >>>>>> ???? _semaphore.wait(); >>>>>> ?? } >>>>>> >>>>>> At the time when SemaphoreSafepointAware was written, we didn't >>>>>> have trywait on all platforms, now we do. Maybe we should always >>>>>> do the trywait dance inside SemaphoreSafepointAware::wait() ? >>>>>> >>>>>> inline void SemaphoreSafepointAware::wait() { >>>>>> ? if (Thread::current()->is_Java_thread()) { >>>>>> ??? if (!_semaphore.trywait) { >>>>>> ????? JavaThread* thread = JavaThread::current(); >>>>>> >>>>>> ????? // Prepare to block and allow safepoints while blocked >>>>>> ????? ThreadBlockInVM tbivm(thread); >>>>>> ????? OSThreadWaitState osts(thread->osthread(), false /* not in >>>>>> Object.wait() */); >>>>>> >>>>>> ????? // Wait for value >>>>>> ???? _semaphore.wait(); >>>>>> ?? } >>>>>> >>>>>> Thanks, >>>>>> StefanK >>>>> >>>> >> From david.holmes at oracle.com Mon May 21 02:16:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 21 May 2018 12:16:12 +1000 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: References: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> Message-ID: <74ba1586-4d0e-c8bb-c1b6-0273de88bedc@oracle.com> Hi Michihiro, An initial question ... Can you clarify the assumptions/expectations with regard to Release-Consume given it has been deprecated by the C++ committee? memory_order_release is expected to be C++ compatible. Thanks, David On 19/05/2018 1:12 AM, Michihiro Horie wrote: > Dear all, > > I update the webrev: http://cr.openjdk.java.net/~mhorie/8154736/webrev.09/ > > With the release barrier before the CAS, new_obj can be observed from > other threads. If the CAS succeeds, the current thread can use new_obj > without barriers. If the CAS fails, "o->forwardee()" is ordered with > respect to CAS by accessing the same memory location "_mark", so no > barriers needed. The order of (1) access to the forwardee and (2) access > to forwardee's fields is preserved due to Release-Consume ordering on > supported platforms. (The ordering between "new_obj = o->forwardee();" > and logging or other usages is not changed.) > > Regarding the maintainability, the requirement is > CAS(memory_order_release) as Release-Consume to be consistent with > C++11. This requirement is necessary when a weaker platform like DEC > Alpha is to be supported. On currently supported platforms, code change > can be safe if the code meats this requirement (and the order of (1) > access to the forwardee and (2) access to forwardee's fields is the > natural way of coding). > > > oop PSPromotionManager::copy_to_survivor_space(oop o) { > oop new_obj = NULL; > markOop test_mark = o->mark_raw(); > > if (!test_mark->is_marked()) { // The same test as "o->is_forwarded()" > : > Copy::aligned_disjoint_words((HeapWord*)o, (HeapWord*)new_obj, ...); > > if (o->cas_forward_to(new_obj, test_mark)) { // CAS succeeds > : > new_obj->push_contents(this); > > } else { // CAS fails > : > new_obj = o->forwardee(); > } > } else { > : > new_obj = o->forwardee(); > } > > log_develop_trace(gc, scavenge)(..., new_obj->klass()->internal_name(), > p2i((void *)o), p2i((void *)new_obj), new_obj->size()); > return new_obj; > } > > > Best regards, > -- > Michihiro, > IBM Research - Tokyo > > > ----- Original message ----- > From: Kim Barrett > To: David Holmes > Cc: Michihiro Horie , > ppc-aix-port-dev at openjdk.java.net, hotspot-dev at openjdk.java.net, > hotspot-gc-dev at openjdk.java.net, Hiroshi H Horii > Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and > copy_to_survivor for ppc64 > Date: Fri, Apr 27, 2018 6:03 AM > > > On Apr 25, 2018, at 8:45 AM, David Holmes > wrote: > > > > Hi Michihiro, > > > > On 23/04/2018 8:33 PM, Michihiro Horie wrote: > >> Dear all, > >> I would like to ask reviews on 8154736 ?enhancement of cmpxchg and > >> copy_to_survivor?. The change adds options to avoid expensive syncs with > >> compare-and-exchange. An experiment by using SPECjbb2015 showed 6% > >> improvement in critical-jOPS. This change focuses on ppc64 but would be > >> potentially beneficial for aarch64. > >> Although discussions stopped at > >> > _http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-October/002718.html_ > >> , I would like to restart the review by taking over Hiroshi's work > if the > >> discussion is still open. > > > > So the very last comment there was about not implicitly assuming > memory_order_consume, yet that has not been addressed in the proposal. > > > > Further the discussion on hotspot-runtime-dev through September and > October was far more illuminating. I think my post here: > > > > > _http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-October/021617.html_ > > > > and the closely following one from Thomas Schatzl summed up the > concerns about the proposed changes. > > > > AFAICS the restarted proposal addresses none of those concerns but > simply takes up where the previous implementation suggestion left off. > > > > This is a proposal to change the memory ordering semantics of part of > the shared GC code _not_ just the PPC64 implementation, but I have seen > no analysis to demonstrate the correctness of such a proposal. > > I agree with David here. So far we've seen no such analysis. All we > have seen is a series of proposed changes and non-failing test > results, all of which have then been shown to have holes. (Among other > things, this suggests the set of tests being applied is inadequate.) > Part of the author's job is to convince reviewers that the proposed > change is correct. I'm not expecting a formal proof, but I am > expecting a lot more than has been provided so far. > > In this latest proposal, the conditional acquire doesn't look right to > me. If the logging is disabled so there is no acquire, the object is > then returned to the callers, who might do what with it? Is that safe? > For all callers? And is it stable through future maintenance? This is > not to say that I think making those acquires unconditional is > sufficient. > > From kim.barrett at oracle.com Mon May 21 03:59:47 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 21 May 2018 05:59:47 +0200 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: References: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> Message-ID: > On May 18, 2018, at 5:12 PM, Michihiro Horie wrote: > > Dear all, > > I update the webrev: http://cr.openjdk.java.net/~mhorie/8154736/webrev.09/ > > With the release barrier before the CAS, new_obj can be observed from other threads. If the CAS succeeds, the current thread can use new_obj without barriers. If the CAS fails, "o->forwardee()" is ordered with respect to CAS by accessing the same memory location "_mark", so no barriers needed. The order of (1) access to the forwardee and (2) access to forwardee's fields is preserved due to Release-Consume ordering on supported platforms. (The ordering between "new_obj = o->forwardee();" and logging or other usages is not changed.) > > Regarding the maintainability, the requirement is CAS(memory_order_release) as Release-Consume to be consistent with C++11. This requirement is necessary when a weaker platform like DEC Alpha is to be supported. On currently supported platforms, code change can be safe if the code meats this requirement (and the order of (1) access to the forwardee and (2) access to forwardee's fields is the natural way of coding). Relying on implicit consume has been been discussed and rejected, in the earlier thread on this topic and I think elsewhere too. http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-October/021538.html From adinn at redhat.com Mon May 21 09:47:46 2018 From: adinn at redhat.com (Andrew Dinn) Date: Mon, 21 May 2018 10:47:46 +0100 Subject: RFC: Experiment in accessing/managing persistent memory from Java Message-ID: <84bc61e0-9311-ee10-3de7-424e567f22a5@redhat.com> I have been helping one of my Red Hat colleagues, Jonathan Halliday, to investigate provision of a Java equivalent to Intel's libpmem suite of C libraries [1]. This approach avoids the significant cost of using the Intel libraries from Java via JNI (or, worse, as a virtual driver for a persistent memory device). Jonathan has modified the JVM/JDK to allow a MappedByteBuffer to be mapped over persistent memory, providing equivalent function to libpmem itself. On top of this he implemented a Java journaled log class providing equivalent functionality to one of the Intel client libs, libpmemlog, built over libpmem. The modified MappedByteBuffer can be configured to use either i) a registered native method or ii) a JIT intrinsic to perform the critical task of cache line writeback i.e. the persistence step (the intrinsic is my contribution). Jonathan's tests compare use of JNI, registered native and intrinsic with an equivalent C program to write a large swathe of records to a journaled log file stored in persistent memory. Performance is worse than C when relying on JNI and significantly better with JVM/JDK support. Indeed, as one might reasonably expect, use of the JIT intrinsic almost completely eliminates writeback costs. The journaled log code, jdk dev tree patch, build instructions, test code plus C equivalent and test results are all available from Jonathan's git repo [2]. For those who do not want to look at the actual code, the README file [3] provides background to use of persistent memory, an overview of the design, and summary details of the test process and results. [1] https://pmem.io/pmdk/ [2] https://github.com/jhalliday/pmem [3] http://github.com://jhalliday/pmem/README.md n.b. Jonathan has experimented with using this same prototype to replace the journaled log used in the Red Hat Narayana transaction manager. It provides a significant improvement on the current disk file based log, both for throughput and latency (the code is not yet available as getting it to work involved some horrible hacking of the build to migrate up to jdk11). regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From coleen.phillimore at oracle.com Mon May 21 12:08:56 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 21 May 2018 08:08:56 -0400 Subject: RFR 8203381 Replace InstanceKlass::allocate_instance_handle with JavaCalls::construct_new_instance In-Reply-To: <45eb3ef9-88c3-1bda-bb9e-122eb888dc35@oracle.com> References: <45eb3ef9-88c3-1bda-bb9e-122eb888dc35@oracle.com> Message-ID: <4fb140c9-f9c7-5ad7-e4fc-40f369b237d1@oracle.com> This change came out nicely. On 5/18/18 12:36 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8203381 > http://cr.openjdk.java.net/~iklam/jdk11/8203381-construct-new-instance.v01/ > > > This patch removes a lot of the boilerplate code for allocating a > Java object and calling its constructor. Hopefully the code is cleaner > and easier to read. > > Also: > > Added assert(InstanceKlass::is_initialized(), ...) in case where > we cannot call JavaCalls::construct_new_instance for thread objects. > > Replaced unnecessary SystemDictionary::resolve_or_null() calls with > SystemDictionary::XXX_klass(). Great, there are a lot of these that should be replaced in various places. Thanks, Coleen > > Tested with hs-tier[1,2,3,4,5] > > Thanks > - Ioi > > > > > From stefan.karlsson at oracle.com Mon May 21 14:42:34 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 21 May 2018 16:42:34 +0200 Subject: 8203341: Add a safepoint-aware Semaphore In-Reply-To: <89f62752-c90f-02c0-ab5a-c3c1714cc3ae@oracle.com> References: <2f8412d5-2fe4-2c8f-077c-cf96e8f9bebf@oracle.com> <5AFDA1E3.6010401@oracle.com> <385fc0f8-461d-5f0f-d678-1dcb69b173cf@oracle.com> <5AFEC537.4050304@oracle.com> <322b2a9c-4dfe-9790-1c1d-b1c7cfa4a8ff@oracle.com> <89f62752-c90f-02c0-ab5a-c3c1714cc3ae@oracle.com> Message-ID: <8d23706b-8afe-3367-e3b7-fc8ea3cd9957@oracle.com> Hi David, On 2018-05-21 02:35, David Holmes wrote: > Hi Stefan, > > This updated webrev only came out late Friday night for me so I did > not get a chance to review before you pushed it. Let's continue the discussion here, and push an update to the code as a new RFE. > I would have argued for keeping consistency with existing similar > API's and use the "bool safepoint_check" parameter instead of adding a > new variant of the "wait" method. > > src/hotspot/share/runtime/semaphore.inline.hpp > > ?2? * Copyright (c) 2015, 2018, > > How does a new file have a 2015 copyright? This code comes from the ZGC repository, and was created 2015. > > ? 41 inline void Semaphore::wait_with_safepoint_check() { > ? 42?? Thread* thread = Thread::current(); > ? 43?? if (thread->is_Java_thread()) { > ? 44 wait_with_safepoint_check(static_cast(thread)); > ? 45?? } else { > ? 46???? // Wait for value > ? 47???? _impl.wait(); > ? 48?? } > ? 49 } > > This seems misnamed - if not a JavaThread then there is no safepoint > check. > > It seems confusing to me to now have three wait variants with no clear > indicator as to whether specific types of threads should only call > specific variants (and no assertions checking that). I would expect > JavaThreads to always call wait_with_safepoint_check(JavaThread), > while non-JavaThread's should only call wait(). The parameter-less > wait_with_safepoint_check() doesn't seem desirable. I'm guessing it is > there to allow the dispatching to be done by the semaphore code rather > than having the callers call Thread::current() and dispatch themselves? Yes, this is what the code in ZGC looks like: http://hg.openjdk.java.net/zgc/zgc/file/d99bcf9de8f3/src/hotspot/share/gc/z/zFuture.inline.hpp This code can be called from JavaThreads and GC threads (non-JavaThreads), and after this change this would start to use wait_with_safepoint_check(). > > But unless we may sometimes want JavaThreads to not perform safepoint > checks (do we??), > this seems overly complicated. We could have simply defined wait to > take a Thread parameter (default NULL) and then decide whether to use > ThreadBlockInVM or not based on whether it was a JavaThread. This would introduce a virtual call in the handshake code. > At most two variants seem needed (if you don't like potentially > calling Thread::current() if you don't need the Thread) rather than > three. We have these use-cases (maybe there are more): 1) GC threads using Semaphores - using Semaphore::wait - no safepoint checking 2) Java threads executing handshake code - Semaphore::wait_with_safepoint_check(JavaThread*) - with safepoint checking 3) ZGC code that are executed by both GC threads and Java threads -? using Semaphore::wait_with_safepoint_check() -? dispatching to either (1) or (2). We could easily move the code in Semaphore::wait_with_safepoint_check() to ZGC, if you don't think it's usable by other parts of the VM. That would also solve the naming problem. http://cr.openjdk.java.net/~stefank/8203341/webrev.04.delta/ http://cr.openjdk.java.net/~stefank/8203341/webrev.04/ Is this OK, or do you still want a bool to mimic the Mutex API? I think most usages of Semaphores want to use the wait() function (without safepoint check), but the default for Mutex is to lock with safepoint check. Thanks, StefanK > > Cheers, > David > ----- > > On 18/05/2018 10:45 PM, Stefan Karlsson wrote: >> I got offline suggestions that I should rename the function to >> wait_with_safepoint_check: >> >> http://cr.openjdk.java.net/~stefank/8203341/webrev.03/ >> http://cr.openjdk.java.net/~stefank/8203341/webrev.03.delta/ >> >> Thanks, >> StefanK >> >> On 2018-05-18 14:21, Erik ?sterlund wrote: >>> Hi Stefan, >>> >>> Looks good. >>> >>> Thanks, >>> /Erik >>> >>> On 2018-05-18 14:13, Stefan Karlsson wrote: >>>> Updated webrevs: >>>> ?http://cr.openjdk.java.net/~stefank/8203341/webrev.02/ >>>> ?http://cr.openjdk.java.net/~stefank/8203341/webrev.02.delta/ >>>> >>>> I removed the SemaphoreSafepointAware class, as Erik requested, but >>>> instead of adding a bool I added a new function for the safepoint >>>> aware waits. >>>> >>>> Thanks, >>>> StefanK >>>> >>>> On 2018-05-17 17:59, Stefan Karlsson wrote: >>>>> On 2018-05-17 17:38, Erik ?sterlund wrote: >>>>>> Hi Stefan, >>>>>> >>>>>> I wonder if it would make sense to let Semaphore::wait have a >>>>>> bool safepoint_check, similar to Mutex (possibly with a default >>>>>> value), instead of introducing a new class with the same API, >>>>>> that only differs in this property. >>>>> >>>>> That sounds reasonable to me. >>>>> >>>>> I also need to change my patch so that the thread-local handshake >>>>> code pass in the existing JavaThread, so that we don't have to >>>>> call Thread::current()->is_JavaThread() unnecessarily. >>>>> >>>>> Thanks, >>>>> StefanK >>>>> >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>> On 2018-05-17 09:46, Stefan Karlsson wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> Please review this patch to add a safepoint-aware Semaphore (and >>>>>>> refactor the semaphore usage in thread-local handshakes). >>>>>>> >>>>>>> http://cr.openjdk.java.net/~stefank/8203341/webrev.01/ >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8203341 >>>>>>> >>>>>>> Both ZGC and the thread-local handshakes need to move the >>>>>>> JavaThread to a blocked state before waiting on a Semaphore. I >>>>>>> propose that we introduce a new SemaphoreSafepointAware wrapper >>>>>>> around Semaphore. >>>>>>> >>>>>>> There are two things to consider with this patch: >>>>>>> >>>>>>> 1) In ZGC, where this patch originates from, we set the >>>>>>> JavaThread's osthread state with >>>>>>> osthread->set_state(CONDVAR_WAIT) (using OSThreadWaitState). >>>>>>> This means that we get the following printed when the threads >>>>>>> are dumped: "waiting on condition ". I've kept this behavior for >>>>>>> the SemaphoreSafepointAware class. This means that we now change >>>>>>> the osthread state for JavaThreads blocking in >>>>>>> HandshakeState::process_self_inner. >>>>>>> >>>>>>> 2) The thread-local handshakes uses trywait before entering wait: >>>>>>> ?? if (!_semaphore.trywait()) { >>>>>>> -??? ThreadBlockInVM tbivm(thread); >>>>>>> ???? _semaphore.wait(); >>>>>>> ?? } >>>>>>> >>>>>>> At the time when SemaphoreSafepointAware was written, we didn't >>>>>>> have trywait on all platforms, now we do. Maybe we should always >>>>>>> do the trywait dance inside SemaphoreSafepointAware::wait() ? >>>>>>> >>>>>>> inline void SemaphoreSafepointAware::wait() { >>>>>>> ? if (Thread::current()->is_Java_thread()) { >>>>>>> ??? if (!_semaphore.trywait) { >>>>>>> ????? JavaThread* thread = JavaThread::current(); >>>>>>> >>>>>>> ????? // Prepare to block and allow safepoints while blocked >>>>>>> ????? ThreadBlockInVM tbivm(thread); >>>>>>> ????? OSThreadWaitState osts(thread->osthread(), false /* not in >>>>>>> Object.wait() */); >>>>>>> >>>>>>> ????? // Wait for value >>>>>>> ???? _semaphore.wait(); >>>>>>> ?? } >>>>>>> >>>>>>> Thanks, >>>>>>> StefanK >>>>>> >>>>> >>> From daniel.smith at oracle.com Mon May 21 15:19:38 2018 From: daniel.smith at oracle.com (Dan Smith) Date: Mon, 21 May 2018 09:19:38 -0600 Subject: General Registration -- 2018 JVM Language Summit Message-ID: GENERAL REGISTRATION -- JVM LANGUAGE SUMMIT, JULY 2018 General registration for the 2018 JVM Language Summit is now open. The event will be held at Oracle's Santa Clara campus on July 30-31, 2018. Speaker registration remains open through May 25. If you have an interesting topic to speak about, please submit an abstract! Many JVM Language Summit speakers and attendees are also deeply involved in the OpenJDK Community, where they do much of their technical work in their roles as OpenJDK Committers. This year, as an experiment, we're going to host the first OpenJDK Committers' Workshop right after the JVM Language Summit. To fit everything into the week, and allow time for travel, there will be two (rather than the usual three) days of the JVM Language Summit followed immediately by two days of the OpenJDK Committers' Workshop, at the same location. In short, OpenJDK Committers in attendance can look forward to four days of intense, inspiring collaboration around the JVM this year. Overview The JVM Language Summit is an open technical collaboration among language designers, compiler writers, tool builders, runtime engineers, and VM architects. We will share our experiences as creators of both the JVM and programming languages for the JVM. We also welcome non-JVM developers of similar technologies to attend or speak on their runtime, VM, or language of choice. Presentations will be recorded and made available to the public. The summit will be immediately followed by the OpenJDK Committers' Workshop, August 1-2. This event is being organized by language and JVM engineers -- no marketers involved! So bring your slide rules and be prepared for some seriously geeky discussions. Format The summit is held in a single classroom-style room to support direct communication between participants. About 100-120 attendees are expected. The schedule consists of a single track of traditional presentations interspersed with less-formal multitrack "workshop" discussion groups and, possibly, impromptu "lightning talks." Workshops are open discussions with only a small amount of prepared material. We ask each registrant to suggest a few topics of interest. After choosing the most popular topics, we'll ask some registrants if they'd like to act as discussion leaders. To register: register.jvmlangsummit.com For further information: jvmlangsummit.com Questions: inquire2018 at jvmlangsummit.com _______________________________________________ mlvm-dev mailing list mlvm-dev at openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev From gromero at linux.vnet.ibm.com Mon May 21 16:14:35 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Mon, 21 May 2018 13:14:35 -0300 Subject: A question on walking x86 frame stack Message-ID: <51596e96-55bb-6db3-9869-0bb582c26258@linux.vnet.ibm.com> Hello, Does anybody know what's the reason behind the 5 "magic number" being add (to FP) and subtracted (from SP) in this SA code for walking the stack (lines 523 and 524): http://hg.openjdk.java.net/jdk/jdk/file/ec881a19d294/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/x86/X86Frame.java#l523 I think it's just for providing more context / information on the adjacent frames but I'm not sure... In that sense I'm wondering if the 5 fields up and down the frame have any special meaning or are indeed only other adjacent frames. Thanks. Best regards, Gustavo From aph at redhat.com Mon May 21 16:23:19 2018 From: aph at redhat.com (Andrew Haley) Date: Mon, 21 May 2018 17:23:19 +0100 Subject: RFR: JDK-8202016: Use obj+offset in interpreter array access In-Reply-To: References: <0cb46aaf-cc6f-5836-7278-7f3018be6d35@redhat.com> <041699e9-9c5b-0a8b-1b94-45a181018b47@redhat.com> <8b16fb83-25c4-8fcf-7ef6-13d3943f3fd7@redhat.com> <7aeaa2ef-12ca-db86-0698-ab5b3a5521ff@redhat.com> Message-ID: On 05/16/2018 04:20 PM, Roman Kennke wrote: > This would mean to also drag index and/or offset and/or disp through the > calls, some of which are optional/only used for specific types of > accesses. Seeing that all this stuff is already encapsulated in Address, > we can just as well use Address. Right? Hmm. I guess so. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From swatibits14 at gmail.com Mon May 21 16:44:20 2018 From: swatibits14 at gmail.com (Swati Sharma) Date: Mon, 21 May 2018 22:14:20 +0530 Subject: UseNUMA membind Issue in openJDK In-Reply-To: References: <9a0310b7-2880-db69-cfbc-7abba844ecbf@oracle.com> Message-ID: +Gustavo Hi David, Updated previous patch.Some issue was there with issinglenodeound() method on SUSE, so just swapped the condition, checking in same. Please review and let me know your comments. Derek : Thanks for the info :) I dint know that its already reported. Here is the updated patch,attached too. ======================PATCH============================== diff --git a/src/hotspot/os/linux/os_linux.cpp b/src/hotspot/os/linux/os_ linux.cpp --- a/src/hotspot/os/linux/os_linux.cpp +++ b/src/hotspot/os/linux/os_linux.cpp @@ -2832,14 +2832,42 @@ // Map all node ids in which is possible to allocate memory. Also nodes are // not always consecutively available, i.e. available from 0 to the highest // node number. + // If the nodes have been bound explicitly using numactl membind, then + // allocate memory from those nodes only. for (size_t node = 0; node <= highest_node_number; node++) { - if (Linux::isnode_in_configured_nodes(node)) { + if (Linux::isnode_in_bounded_nodes(node)) { ids[i++] = node; } } return i; } +extern "C" struct bitmask { + unsigned long size; /* number of bits in the map */ + unsigned long *maskp; +}; +// Check if single memory node bound. +// Returns true if single memory node bound. +bool os::Linux::issingle_node_bound() { + struct bitmask* bmp = _numa_get_membind != NULL ? _numa_get_membind() : NULL; + if(!(bmp != NULL && bmp->maskp != NULL)) return false; + int issingle = 0; + // System can have more than 64 nodes so check in all the elements of + // unsigned long array + for (unsigned long i = 0; i < (bmp->size / (8 * sizeof(unsigned long))); i++) { + if (bmp->maskp[i] == 0) { + continue; + } else if ((bmp->maskp[i] & (bmp->maskp[i] - 1)) == 0) { + issingle++; + } else { + return false; + } + } + if (issingle == 1) + return true; + return false; +} + bool os::get_page_info(char *start, page_info* info) { return false; } @@ -2930,6 +2958,10 @@ libnuma_dlsym(handle, "numa_bitmask_isbitset"))); set_numa_distance(CAST_TO_FN_PTR(numa_distance_func_t, libnuma_dlsym(handle, "numa_distance"))); + set_numa_set_membind(CAST_TO_FN_PTR(numa_set_membind_func_t, + libnuma_dlsym(handle, "numa_set_membind"))); + set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, + libnuma_v2_dlsym(handle, "numa_get_membind"))); if (numa_available() != -1) { set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, "numa_all_nodes")); @@ -3054,6 +3086,8 @@ os::Linux::numa_set_bind_policy_func_t os::Linux::_numa_set_bind_policy; os::Linux::numa_bitmask_isbitset_func_t os::Linux::_numa_bitmask_isbitset; os::Linux::numa_distance_func_t os::Linux::_numa_distance; +os::Linux::numa_set_membind_func_t os::Linux::_numa_set_membind; +os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; unsigned long* os::Linux::_numa_all_nodes; struct bitmask* os::Linux::_numa_all_nodes_ptr; struct bitmask* os::Linux::_numa_nodes_ptr; @@ -4962,8 +4996,9 @@ if (!Linux::libnuma_init()) { UseNUMA = false; } else { - if ((Linux::numa_max_node() < 1)) { - // There's only one node(they start from 0), disable NUMA. + if ((Linux::numa_max_node() < 1) || Linux::issingle_node_bound()) { + // If there's only one node(they start from 0) or if the process + // is bound explicitly to a single node using membind, disable NUMA. UseNUMA = false; } } diff --git a/src/hotspot/os/linux/os_linux.hpp b/src/hotspot/os/linux/os_ linux.hpp --- a/src/hotspot/os/linux/os_linux.hpp +++ b/src/hotspot/os/linux/os_linux.hpp @@ -228,6 +228,8 @@ typedef int (*numa_tonode_memory_func_t)(void *start, size_t size, int node); typedef void (*numa_interleave_memory_func_t)(void *start, size_t size, unsigned long *nodemask); typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t size, struct bitmask* mask); + typedef void (*numa_set_membind_func_t)(struct bitmask *mask); + typedef struct bitmask* (*numa_get_membind_func_t)(void); typedef void (*numa_set_bind_policy_func_t)(int policy); typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, unsigned int n); @@ -244,6 +246,8 @@ static numa_set_bind_policy_func_t _numa_set_bind_policy; static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; static numa_distance_func_t _numa_distance; + static numa_set_membind_func_t _numa_set_membind; + static numa_get_membind_func_t _numa_get_membind; static unsigned long* _numa_all_nodes; static struct bitmask* _numa_all_nodes_ptr; static struct bitmask* _numa_nodes_ptr; @@ -259,6 +263,8 @@ static void set_numa_set_bind_policy(numa_set_bind_policy_func_t func) { _numa_set_bind_policy = func; } static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t func) { _numa_bitmask_isbitset = func; } static void set_numa_distance(numa_distance_func_t func) { _numa_distance = func; } + static void set_numa_set_membind(numa_set_membind_func_t func) { _numa_set_membind = func; } + static void set_numa_get_membind(numa_get_membind_func_t func) { _numa_get_membind = func; } static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes = ptr; } static void set_numa_all_nodes_ptr(struct bitmask **ptr) { _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } static void set_numa_nodes_ptr(struct bitmask **ptr) { _numa_nodes_ptr = (ptr == NULL ? NULL : *ptr); } @@ -320,6 +326,15 @@ } else return 0; } + // Check if node in bounded nodes + static bool isnode_in_bounded_nodes(int node) { + struct bitmask* bmp = _numa_get_membind != NULL ? _numa_get_membind() : NULL; + if (bmp != NULL && _numa_bitmask_isbitset != NULL && _numa_bitmask_isbitset(bmp, node)) { + return true; + } else + return false; + } + static bool issingle_node_bound(); }; #endif // OS_LINUX_VM_OS_LINUX_HPP ========================================================== On Fri, May 4, 2018 at 2:56 AM, White, Derek wrote: > Hi Swati, > > I think this was reported as JDK-8189922 "UseNUMA memory interleaving > allocates heap for unavailable nodes". https://bugs.openjdk.java.net/ > browse/JDK-8189922 > > Thanks for working on a fix for this! > > - Derek > > > -----Original Message----- > > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > > Behalf Of Swati Sharma > > Sent: Wednesday, May 02, 2018 6:24 AM > > To: David Holmes > > Cc: hotspot-dev at openjdk.java.net; Prakash.Raghavendra at amd.com > > Subject: Re: UseNUMA membind Issue in openJDK > > > > Hi David, > > > > I have localized the struct bitmask declaration in os_linux.cpp. > > > > Here is the updated patch > > ===================================PATCH====================== > > ============================= > > diff --git a/src/hotspot/os/linux/os_linux.cpp > > b/src/hotspot/os/linux/os_linux.cpp > > --- a/src/hotspot/os/linux/os_linux.cpp > > +++ b/src/hotspot/os/linux/os_linux.cpp > > @@ -2832,14 +2832,42 @@ > > // Map all node ids in which is possible to allocate memory. Also > nodes are > > // not always consecutively available, i.e. available from 0 to the > highest > > // node number. > > + // If the nodes have been bound explicitly using numactl membind, > > + then // allocate memory from those nodes only. > > for (size_t node = 0; node <= highest_node_number; node++) { > > - if (Linux::isnode_in_configured_nodes(node)) { > > + if (Linux::isnode_in_bounded_nodes(node)) { > > ids[i++] = node; > > } > > } > > return i; > > } > > > > +extern "C" struct bitmask { > > + unsigned long size; /* number of bits in the map */ > > + unsigned long *maskp; > > +}; > > +// Check if single memory node bound. > > +// Returns true if single memory node bound. > > +bool os::Linux::issingle_node_bound() { > > + struct bitmask* bmp = _numa_get_membind != NULL ? > > _numa_get_membind() : > > NULL; > > + if(bmp == NULL) return false; > > + int issingle = 0; > > + // System can have more than 64 nodes so check in all the elements of > > + // unsigned long array for (unsigned long i = 0; i < (bmp->size / (8 > > + * sizeof(unsigned long))); > > i++) { > > + if (bmp->maskp != NULL && (((bmp->maskp[i]) & (((bmp->maskp[i])) - > > + 1)) > > == 0)) { > > + issingle++; > > + } else if (bmp->maskp[i] == 0) { > > + continue; > > + } else { > > + return false; > > + } > > + } > > + if (issingle == 1) > > + return true; > > + return false; > > +} > > + > > bool os::get_page_info(char *start, page_info* info) { > > return false; > > } > > @@ -2930,6 +2958,10 @@ > > libnuma_dlsym(handle, > > "numa_bitmask_isbitset"))); > > set_numa_distance(CAST_TO_FN_PTR(numa_distance_func_t, > > libnuma_dlsym(handle, > "numa_distance"))); > > + > > set_numa_set_membind(CAST_TO_FN_PTR(numa_set_membind_func_t, > > + libnuma_dlsym(handle, > > "numa_set_membind"))); > > + > > set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > > + libnuma_v2_dlsym(handle, > > "numa_get_membind"))); > > > > if (numa_available() != -1) { > > set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, > > "numa_all_nodes")); @@ -3054,6 +3086,8 @@ > > os::Linux::numa_set_bind_policy_func_t os::Linux::_numa_set_bind_policy; > > os::Linux::numa_bitmask_isbitset_func_t os::Linux::_numa_bitmask_isbit > set; > > os::Linux::numa_distance_func_t os::Linux::_numa_distance; > > +os::Linux::numa_set_membind_func_t os::Linux::_numa_set_membind; > > +os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; > > unsigned long* os::Linux::_numa_all_nodes; struct bitmask* > > os::Linux::_numa_all_nodes_ptr; struct bitmask* > > os::Linux::_numa_nodes_ptr; @@ -4962,8 +4996,9 @@ > > if (!Linux::libnuma_init()) { > > UseNUMA = false; > > } else { > > - if ((Linux::numa_max_node() < 1)) { > > - // There's only one node(they start from 0), disable NUMA. > > + if ((Linux::numa_max_node() < 1) || Linux::issingle_node_bound()) > { > > + // If there's only one node(they start from 0) or if the process > > + // is bound explicitly to a single node using membind, disable > > NUMA. > > UseNUMA = false; > > } > > } > > diff --git a/src/hotspot/os/linux/os_linux.hpp > > b/src/hotspot/os/linux/os_linux.hpp > > --- a/src/hotspot/os/linux/os_linux.hpp > > +++ b/src/hotspot/os/linux/os_linux.hpp > > @@ -228,6 +228,8 @@ > > typedef int (*numa_tonode_memory_func_t)(void *start, size_t size, > int > > node); > > typedef void (*numa_interleave_memory_func_t)(void *start, size_t > size, > > unsigned long *nodemask); > > typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t > > size, struct bitmask* mask); > > + typedef void (*numa_set_membind_func_t)(struct bitmask *mask); > > + typedef struct bitmask* (*numa_get_membind_func_t)(void); > > > > typedef void (*numa_set_bind_policy_func_t)(int policy); > > typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, > > unsigned int n); @@ -244,6 +246,8 @@ > > static numa_set_bind_policy_func_t _numa_set_bind_policy; > > static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; > > static numa_distance_func_t _numa_distance; > > + static numa_set_membind_func_t _numa_set_membind; static > > + numa_get_membind_func_t _numa_get_membind; > > static unsigned long* _numa_all_nodes; > > static struct bitmask* _numa_all_nodes_ptr; > > static struct bitmask* _numa_nodes_ptr; @@ -259,6 +263,8 @@ > > static void set_numa_set_bind_policy(numa_set_bind_policy_func_t > func) > > { _numa_set_bind_policy = func; } > > static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t > > func) { _numa_bitmask_isbitset = func; } > > static void set_numa_distance(numa_distance_func_t func) { > > _numa_distance = func; } > > + static void set_numa_set_membind(numa_set_membind_func_t func) { > > _numa_set_membind = func; } > > + static void set_numa_get_membind(numa_get_membind_func_t func) { > > _numa_get_membind = func; } > > static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes = > > ptr; } > > static void set_numa_all_nodes_ptr(struct bitmask **ptr) { > > _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > > static void set_numa_nodes_ptr(struct bitmask **ptr) { _numa_nodes_ptr > > = (ptr == NULL ? NULL : *ptr); } @@ -320,6 +326,15 @@ > > } else > > return 0; > > } > > + // Check if node in bounded nodes > > + static bool isnode_in_bounded_nodes(int node) { > > + struct bitmask* bmp = _numa_get_membind != NULL ? > > + _numa_get_membind() > > : NULL; > > + if (bmp != NULL && _numa_bitmask_isbitset != NULL && > > _numa_bitmask_isbitset(bmp, node)) { > > + return true; > > + } else > > + return false; > > + } > > + static bool issingle_node_bound(); > > }; > > > > #endif // OS_LINUX_VM_OS_LINUX_HPP > > > > =============================================================== > > ============================= > > > > Thanks, > > Swati > > > > On Thu, Apr 26, 2018 at 6:10 PM, David Holmes > > wrote: > > > > > > Hi Swati, > > > > > > On 26/04/2018 10:20 PM, Swati Sharma wrote: > > >> > > >> Hi Everyone, > > >> > > >> I work at AMD and this is my first patch as a new member of openJDK > > >> community. > > > > > > > > > Welcome! > > > > > > I can't comment on the actual NUMA details of the patch (though I can > > > see > > what you're doing), but the struct bitmask declaration in os.hpp should > be > > localized in os_linux.hpp as far as I can see, as it's only needed > internally in > > the Linux code. > > > > > > Thanks, > > > David > > > ----- > > > > > > > > >> I have found some issue while running specjbb2015 composite workload > > with > > >> the flag -XX:+UseNUMA. It seems that JVM does not allocate memory > > according > > >> to the explicit node binding done using "numactl --membind". > > >> > > >> E.g. If bound to a single memory node, JVM divides the whole heap > > >> based > > on > > >> the total number of numa nodes available on the system which creates > > >> more logical groups(lgrps) than required which cannot be used except > the > > one. > > >> > > >> The following examples will explain clearly : > > >> (Note : Collected GC logs with > > >> -Xlog:gc*=debug:file=gc.log:time,uptimemillis) > > >> 1) Allocating a heap of 22GB for single node divides the whole heap > > >> in 8 lgrp(Actual no of Nodes are 8) > > >> $numactl --cpunodebind=0 --membind=0 java -Xmx24g -Xms24g > > >> -Xmn22g -XX:+UseNUMA > > >> > > >> eden space 22511616K(22GB), 12% used > > >> lgrp 0 space 2813952K, 100% used lgrp 1 > space > > >> 2813952K, 0% used lgrp 2 space 2813952K, 0% > used > > >> lgrp 3 space 2813952K, 0% used lgrp 4 > > space > > >> 2813952K, 0% used lgrp 5 space 2813952K, 0% > used > > >> lgrp 6 space 2813952K, 0% used lgrp 7 > > space > > >> 2813952K, 0% used > > >> > > >> Observation : Instead of disabling UseNUMA for single node binding > > >> JVM divides the memory in 8 lgrps and allocates memory always on the > > >> bounded node hence in eden space allocation never happens more than > > 12%. > > >> > > >> 2) Another case of binding to node 0 and 7 results in dividing the > > >> heap > > in > > >> 8lgrp > > >> $numactl --cpunodebind=0,7 ?membind=0,7 java -Xms50g -Xmx50g - > > Xmn45g > > >> -XX:+UseNUMA > > >> > > >> eden space 46718976K, 6% used > > >> lgrp 0 space 5838848K, 14% used lgrp 1 space > > 5838848K, > > >> 0% used lgrp 2 space 5838848K, 0% used > > >> lgrp 3 space 5838848K, 0% used lgrp 4 space > > >> 5838848K, 0% used lgrp 5 space 5838848K, > 0% > > >> used > > >> lgrp 6 space 5838848K, 0% used lgrp 7 space > > >> 5847040K, 35% used > > >> > > >> Observation : Similar to first case allocation happens only on 0th > > >> and > > 7th > > >> node and rest of the lgrps never gets used. > > >> > > >> After applying the patch, JVM divides the given heap size according > > >> to > > the > > >> bounded memory nodes only. > > >> > > >> 1) Binding to single node disables UseNUMA > > >> eden space 46718976K(45GB), 99% used > > >> > > >> Observation : UseNUMA gets disabled hence no lgrp creation and the > > >> whole heap allocation happens on the bounded node. > > >> > > >> 2) Binding to node 0 and 7 > > >> $ numactl --cpunodebind=0,7 ?membind=0,7 java -Xms50g -Xmx50g > > -Xmn45g > > >> -XX:+UseNUMA > > >> eden space 46718976K(45GB), 99% used > > >> lgrp 0 space 23359488K(23.5GB), 100% used lgrp 7 > space > > >> 23359488K(23.5GB), 99% used > > >> > > >> Observation : Only two lgrps gets created and heap size gets divided > > >> equally in both nodes. > > >> > > >> If there is no binding, then JVM will divide the whole heap based on > > >> the number of NUMA nodes available on the system. > > >> > > >> The following patch fixes the issue(attached also). > > >> Please review and let me know your comments. > > >> > > >> Regression testing using jtreg (make -J=1 run-test-tier1 > > >> run-test-tier2) didn't show any new failures. > > >> > > >> > > ===============================PATCH========================== > > ============== > > >> diff --git a/src/hotspot/os/linux/os_linux.cpp > > >> b/src/hotspot/os/linux/os_linux.cpp > > >> --- a/src/hotspot/os/linux/os_linux.cpp > > >> +++ b/src/hotspot/os/linux/os_linux.cpp > > >> @@ -2832,8 +2832,10 @@ > > >> // Map all node ids in which is possible to allocate memory. Also > > nodes > > >> are > > >> // not always consecutively available, i.e. available from 0 to > > >> the highest > > >> // node number. > > >> + // If the nodes have been bound explicitly using numactl membind, > > >> + then // allocate memory from those nodes only. > > >> for (size_t node = 0; node <= highest_node_number; node++) { > > >> - if (Linux::isnode_in_configured_nodes(node)) { > > >> + if (Linux::isnode_in_bounded_nodes(node)) { > > >> ids[i++] = node; > > >> } > > >> } > > >> @@ -2930,6 +2932,10 @@ > > >> > > >> libnuma_dlsym(handle, "numa_bitmask_isbitset"))); > > >> set_numa_distance(CAST_TO_FN_PTR(numa_distance_func_t, > > >> libnuma_dlsym(handle, > > >> "numa_distance"))); > > >> + > > set_numa_set_membind(CAST_TO_FN_PTR(numa_set_membind_func_t, > > >> + libnuma_dlsym(handle, > > >> "numa_set_membind"))); > > >> + > > set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > > >> + libnuma_v2_dlsym(handle, > > >> "numa_get_membind"))); > > >> > > >> if (numa_available() != -1) { > > >> set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, > > >> "numa_all_nodes")); @@ -3054,6 +3060,8 @@ > > >> os::Linux::numa_set_bind_policy_func_t > > os::Linux::_numa_set_bind_policy; > > >> os::Linux::numa_bitmask_isbitset_func_t > > os::Linux::_numa_bitmask_isbitset; > > >> os::Linux::numa_distance_func_t os::Linux::_numa_distance; > > >> +os::Linux::numa_set_membind_func_t os::Linux::_numa_set_membind; > > >> +os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; > > >> unsigned long* os::Linux::_numa_all_nodes; > > >> struct bitmask* os::Linux::_numa_all_nodes_ptr; > > >> struct bitmask* os::Linux::_numa_nodes_ptr; @@ -4962,8 +4970,9 @@ > > >> if (!Linux::libnuma_init()) { > > >> UseNUMA = false; > > >> } else { > > >> - if ((Linux::numa_max_node() < 1)) { > > >> - // There's only one node(they start from 0), disable NUMA. > > >> + if ((Linux::numa_max_node() < 1) || > > >> + Linux::issingle_node_bound()) > > { > > >> + // If there's only one node(they start from 0) or if the > process > > >> + // is bound explicitly to a single node using membind, > > >> + disable > > >> NUMA. > > >> UseNUMA = false; > > >> } > > >> } > > >> diff --git a/src/hotspot/os/linux/os_linux.hpp > > >> b/src/hotspot/os/linux/os_linux.hpp > > >> --- a/src/hotspot/os/linux/os_linux.hpp > > >> +++ b/src/hotspot/os/linux/os_linux.hpp > > >> @@ -228,6 +228,8 @@ > > >> typedef int (*numa_tonode_memory_func_t)(void *start, size_t > > >> size, > > int > > >> node); > > >> typedef void (*numa_interleave_memory_func_t)(void *start, size_t > > size, > > >> unsigned long *nodemask); > > >> typedef void (*numa_interleave_memory_v2_func_t)(void *start, > > >> size_t size, struct bitmask* mask); > > >> + typedef void (*numa_set_membind_func_t)(struct bitmask *mask); > > >> + typedef struct bitmask* (*numa_get_membind_func_t)(void); > > >> > > >> typedef void (*numa_set_bind_policy_func_t)(int policy); > > >> typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, > > >> unsigned int n); @@ -244,6 +246,8 @@ > > >> static numa_set_bind_policy_func_t _numa_set_bind_policy; > > >> static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; > > >> static numa_distance_func_t _numa_distance; > > >> + static numa_set_membind_func_t _numa_set_membind; static > > >> + numa_get_membind_func_t _numa_get_membind; > > >> static unsigned long* _numa_all_nodes; > > >> static struct bitmask* _numa_all_nodes_ptr; > > >> static struct bitmask* _numa_nodes_ptr; @@ -259,6 +263,8 @@ > > >> static void set_numa_set_bind_policy(numa_set_bind_policy_func_t > > func) { > > >> _numa_set_bind_policy = func; } > > >> static void > > >> set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t > > func) > > >> { _numa_bitmask_isbitset = func; } > > >> static void set_numa_distance(numa_distance_func_t func) { > > >> _numa_distance = func; } > > >> + static void set_numa_set_membind(numa_set_membind_func_t func) > > { > > >> _numa_set_membind = func; } > > >> + static void set_numa_get_membind(numa_get_membind_func_t func) > > { > > >> _numa_get_membind = func; } > > >> static void set_numa_all_nodes(unsigned long* ptr) { > > >> _numa_all_nodes > > = > > >> ptr; } > > >> static void set_numa_all_nodes_ptr(struct bitmask **ptr) { > > >> _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > > >> static void set_numa_nodes_ptr(struct bitmask **ptr) { > > _numa_nodes_ptr = > > >> (ptr == NULL ? NULL : *ptr); } > > >> @@ -320,6 +326,34 @@ > > >> } else > > >> return 0; > > >> } > > >> + // Check if node in bounded nodes > > >> + static bool isnode_in_bounded_nodes(int node) { > > >> + struct bitmask* bmp = _numa_get_membind != NULL ? > > _numa_get_membind() > > >> : NULL; > > >> + if (bmp != NULL && _numa_bitmask_isbitset != NULL && > > >> _numa_bitmask_isbitset(bmp, node)) { > > >> + return true; > > >> + } else > > >> + return false; > > >> + } > > >> + // Check if a single node is bound static bool > > >> + issingle_node_bound() { > > >> + struct bitmask* bmp = _numa_get_membind != NULL ? > > _numa_get_membind() > > >> : NULL; > > >> + if(bmp == NULL) return false; > > >> + int issingle = 0; > > >> + // System can have more than 64 nodes so check in all the > > >> + elements > > of > > >> + // unsigned long array > > >> + for (unsigned long i = 0; i < (bmp->size / (8 * sizeof(unsigned > > >> long))); i++) { > > >> + if (bmp->maskp != NULL && (((bmp->maskp[i]) & > > >> + (((bmp->maskp[i])) > > - > > >> 1)) == 0)) { > > >> + issingle++; > > >> + } else if (bmp->maskp[i] == 0) { > > >> + continue; > > >> + } else { > > >> + return false; > > >> + } > > >> + } > > >> + if (issingle == 1) > > >> + return true; > > >> + return false; > > >> + } > > >> }; > > >> > > >> #endif // OS_LINUX_VM_OS_LINUX_HPP > > >> diff --git a/src/hotspot/share/runtime/os.hpp > > >> b/src/hotspot/share/runtime/os.hpp > > >> --- a/src/hotspot/share/runtime/os.hpp > > >> +++ b/src/hotspot/share/runtime/os.hpp > > >> @@ -81,6 +81,10 @@ > > >> CriticalPriority = 11 // Critical thread priority > > >> }; > > >> > > >> +extern "C" struct bitmask { > > >> + unsigned long size; /* number of bits in the map */ > > >> + unsigned long *maskp; > > >> +}; > > >> // Executable parameter flag for os::commit_memory() and > > >> // os::commit_memory_or_exit(). > > >> const bool ExecMem = true; > > >> > > >> > > =============================================================== > > ============== > > >> > > >> Thanks, > > >> Swati > > >> > From stefan.karlsson at oracle.com Mon May 21 18:44:05 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 21 May 2018 20:44:05 +0200 Subject: RFR: 8203490: StringTable::dump lacks a load barrier Message-ID: <418d1f6a-085c-ddda-aa4d-7784cbfac18c@oracle.com> Hi all, Please review this patch to add a missing load barrier to StringTable::dump. http://cr.openjdk.java.net/~stefank/8203490/webrev.01 Most of the loads where fixed with the following changeset: ? http://hg.openjdk.java.net/jdk/jdk/rev/688e5cbd0b91 ? 8192003: Refactor weak references in StringTable to use the Access API but the StringTable::dump function wasn't updated. This patch adds the missing load barrier. Thanks, StefanK From karen.kinnear at oracle.com Mon May 21 18:48:29 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Mon, 21 May 2018 14:48:29 -0400 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: Message-ID: <8B73333D-3D51-48FA-BB39-DE3A171F72C0@oracle.com> David, Many thanks for the extremely careful attention to detail. And thank you for the annotated master webrev. I still owe you a review of the tests. I had a number of questions/comments about the sources - mostly requests for clarifying comments: 1. LinkResolver::check_method_accessability: line 591: worth adding something like "e.g. linkage errors due to nest_host resolution 2. LinkResolver.cpp line 782 cleanup: FIXME: 3. JVM_AreNestMates If member is a primitive or an array - what happens? 4. reflection.cpp line 714: cleanup: FIXME - perhaps add assertions? 5. reflection.cpp lines 721-722: perhaps add a comment that it is the caller's responsibility to check HAS_PENDING_EXCEPTION if return is false - specifically to decide context-specific handling of nestmate resolution/validation 6. instanceKlass.cpp line 255: FIXME: an exception from this is perhaps impossible did you want an assertion here? 7. JVM_GetNestHost is the only caller of IK->nest_host that passes in a NULL. I would expect one of these to clear PENDING_EXCEPTION,it would be helpful for the comments to clarify whose responsibility this is. 8. templateTable_x86.cpp line 3789: I belive that rax is reference klass (from f1) if interface method OR if j.l.Object final/private method line 3794: I think this is more precisely: "First check for Object (not private/final) case, then private/final interface method, then regular interface method" and line 3810 "Check for private/final method invocation - indicated by vfinal" 9. cpCache.cpp line 191 set_f1(holder); // interface klass* Could you possibly add a comment that for private/final method invocations REFC == DECC also line 180: check for private/final interface method invocations 10. klassVtable.cpp Could please add a line to the comment in initialize_itable_for_interface before 1217: call to lookup_instance_method_in_klasses // Invokespecial does not perform selection based on the receiver, so it does // not used the cached itable 11. instanceKlass has_nest_member has this comment: // can't have two names the same, so we're done Where is that ensured? thanks, Karen > On May 14, 2018, at 8:52 PM, David Holmes wrote: > > This review is being spread across four groups: langtools, core-libs, hotspot and serviceability. This is the specific review thread for hotspot - webrev: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ > > See below for full details - including annotated full webrev guiding the review. > > The intent is to have JEP-181 targeted and integrated by the end of this month. > > Thanks, > David > ----- > > The nestmates project (JEP-181) introduces new classfile attributes to identify classes and interfaces in the same nest, so that the VM can perform access control based on those attributes and so allow direct private access between nestmates without requiring javac to generate synthetic accessor methods. These access control changes also extend to core reflection and the MethodHandle.Lookup contexts. > > Direct private calls between nestmates requires a more general calling context than is permitted by invokespecial, and so the JVMS is updated to allow, and javac updated to use, invokevirtual and invokeinterface for private class and interface method calls respectively. These changed semantics also extend to MethodHandle findXXX operations. > > At this time we are only concerned with static nest definitions, which map to a top-level class/interface as the nest-host and all its nested types as nest-members. > > Please see the JEP for further details. > > JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 > Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 > CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 > > All of the specification changes have been previously been worked out by the Valhalla Project Expert Group, and the implementation reviewed by the various contributors and discussed on the valhalla-dev mailing list. > > Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan > > Master webrev of all changes: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ > > Annotated master webrev index: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html > > Performance: this is expected to be performance neutral in a general sense. Benchmarking and performance runs are about to start. > > Testing Discussion: > ------------------ > > The testing for nestmates can be broken into four main groups: > > - New tests specifically related to nestmates and currently in the runtime/Nestmates directory > > - New tests to complement existing tests by adding in testcases not previously expressible. > - For example java/lang/invoke/SpecialInterfaceCall.java tests use of invokespecial for private interface methods and performing receiver typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do similar tests for invokeinterface. > > - New JVM TI tests to verify the spec changes related to nest attributes. > > - Existing tests significantly affected by the nestmates changes, primarily: > - runtime/SelectionResolution > > In most cases the nestmate changes makes certain invocations that were illegal, legal (e.g. not requiring invokespecial to invoke private interface methods; allowing access to private members via reflection/Methodhandles that were previously not allowed). > > - Existing tests incidentally affected by the nestmate changes > > This includes tests of things utilising class redefinition/retransformation to alter nested types but which unintentionally alter nest relationships (which is not permitted). > > There are still a number of tests problem-listed with issues filed against them to have them adapted to work with nestmates. Some of these are intended to be addressed in the short-term, while some (such as the runtime/SelectionResolution test changes) may not eventuate. > > - https://bugs.openjdk.java.net/browse/JDK-8203033 > - https://bugs.openjdk.java.net/browse/JDK-8199450 > - https://bugs.openjdk.java.net/browse/JDK-8196855 > - https://bugs.openjdk.java.net/browse/JDK-8194857 > - https://bugs.openjdk.java.net/browse/JDK-8187655 > > There is also further test work still to be completed (the JNI and JDI invocation tests): > - https://bugs.openjdk.java.net/browse/JDK-8191117 > which will continue in parallel with the main RFR. > > Pre-integration Testing: > - General: > - Mach5: hs/jdk tier1,2 > - Mach5: hs-nightly (tiers 1 -3) > - Targetted > - nashorn (for asm changes) > - hotspot: runtime/* > serviceability/* > compiler/* > vmTestbase/* > - jdk: java/lang/invoke/* > java/lang/reflect/* > java/lang/instrument/* > java/lang/Class/* > java/lang/management/* > - langtools: tools/javac > tools/javap > From per.liden at oracle.com Mon May 21 19:29:50 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 21 May 2018 21:29:50 +0200 Subject: RFR: 8203490: StringTable::dump lacks a load barrier In-Reply-To: <418d1f6a-085c-ddda-aa4d-7784cbfac18c@oracle.com> References: <418d1f6a-085c-ddda-aa4d-7784cbfac18c@oracle.com> Message-ID: <68f9a743-3db3-26d5-0a5a-ea8686dc2d48@oracle.com> On 2018-05-21 20:44, Stefan Karlsson wrote: > Hi all, > > Please review this patch to add a missing load barrier to > StringTable::dump. > > http://cr.openjdk.java.net/~stefank/8203490/webrev.01 Looks good! /Per > > Most of the loads where fixed with the following changeset: > > ? http://hg.openjdk.java.net/jdk/jdk/rev/688e5cbd0b91 > ? 8192003: Refactor weak references in StringTable to use the Access API > > but the StringTable::dump function wasn't updated. This patch adds the > missing load barrier. > > Thanks, > StefanK From coleen.phillimore at oracle.com Mon May 21 19:51:52 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 21 May 2018 15:51:52 -0400 Subject: RFR: 8203490: StringTable::dump lacks a load barrier In-Reply-To: <418d1f6a-085c-ddda-aa4d-7784cbfac18c@oracle.com> References: <418d1f6a-085c-ddda-aa4d-7784cbfac18c@oracle.com> Message-ID: This looks good. Coleen On 5/21/18 2:44 PM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to add a missing load barrier to > StringTable::dump. > > http://cr.openjdk.java.net/~stefank/8203490/webrev.01 > > Most of the loads where fixed with the following changeset: > > ? http://hg.openjdk.java.net/jdk/jdk/rev/688e5cbd0b91 > ? 8192003: Refactor weak references in StringTable to use the Access API > > but the StringTable::dump function wasn't updated. This patch adds the > missing load barrier. > > Thanks, > StefanK From yumin.qi at gmail.com Mon May 21 21:59:47 2018 From: yumin.qi at gmail.com (yumin qi) Date: Mon, 21 May 2018 14:59:47 -0700 Subject: A question on walking x86 frame stack In-Reply-To: <51596e96-55bb-6db3-9869-0bb582c26258@linux.vnet.ibm.com> References: <51596e96-55bb-6db3-9869-0bb582c26258@linux.vnet.ibm.com> Message-ID: Hi, Gustavo Yes, it displays the nearby stack values around the frame pointer. Since FP used by JIT code, it may not be valid stack address. It is decided by the author to show 5 up/down around FP, 20 up/down around SP (in case FP not valid), no special meanings I think. Thanks Yumin On Mon, May 21, 2018 at 9:14 AM, Gustavo Romero wrote: > Hello, > > Does anybody know what's the reason behind the 5 "magic number" being add > (to FP) and subtracted (from SP) in this SA code for walking the stack > (lines 523 and 524): > > http://hg.openjdk.java.net/jdk/jdk/file/ec881a19d294/src/jdk > .hotspot.agent/share/classes/sun/jvm/hotspot/runtime/x86/ > X86Frame.java#l523 > > I think it's just for providing more context / information on the adjacent > frames but I'm not sure... In that sense I'm wondering if the 5 fields up > and down the frame have any special meaning or are indeed only other > adjacent frames. > > > Thanks. > > Best regards, > Gustavo > > From gromero at linux.vnet.ibm.com Mon May 21 22:43:12 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Mon, 21 May 2018 19:43:12 -0300 Subject: A question on walking x86 frame stack In-Reply-To: References: <51596e96-55bb-6db3-9869-0bb582c26258@linux.vnet.ibm.com> Message-ID: <06d534b6-d645-3bbf-de95-4f66c791e7a0@linux.vnet.ibm.com> Hi Yumin, On 05/21/2018 06:59 PM, yumin qi wrote: > ? Yes, it displays the nearby stack values around the frame pointer. > ? Since FP used by JIT code, it may not be valid stack address. It is decided by the author to show 5 up/down around FP, 20 up/down around SP (in case FP not valid), no special meanings I think. Got it! Thanks a lot for your reply. Best regards, Gustavo > Thanks > Yumin > > > On Mon, May 21, 2018 at 9:14 AM, Gustavo Romero > wrote: > > Hello, > > Does anybody know what's the reason behind the 5 "magic number" being add > (to FP) and subtracted (from SP) in this SA code for walking the stack > (lines 523 and 524): > > http://hg.openjdk.java.net/jdk/jdk/file/ec881a19d294/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/runtime/x86/X86Frame.java#l523 > > I think it's just for providing more context / information on the adjacent > frames but I'm not sure... In that sense I'm wondering if the 5 fields up > and down the frame have any special meaning or are indeed only other > adjacent frames. > > > Thanks. > > Best regards, > Gustavo > > From david.holmes at oracle.com Tue May 22 01:19:49 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 22 May 2018 11:19:49 +1000 Subject: 8203341: Add a safepoint-aware Semaphore In-Reply-To: <8d23706b-8afe-3367-e3b7-fc8ea3cd9957@oracle.com> References: <2f8412d5-2fe4-2c8f-077c-cf96e8f9bebf@oracle.com> <5AFDA1E3.6010401@oracle.com> <385fc0f8-461d-5f0f-d678-1dcb69b173cf@oracle.com> <5AFEC537.4050304@oracle.com> <322b2a9c-4dfe-9790-1c1d-b1c7cfa4a8ff@oracle.com> <89f62752-c90f-02c0-ab5a-c3c1714cc3ae@oracle.com> <8d23706b-8afe-3367-e3b7-fc8ea3cd9957@oracle.com> Message-ID: <5c1dbe12-e6f4-6fba-de5a-a92b501c9572@oracle.com> Hi Stefan, On 22/05/2018 12:42 AM, Stefan Karlsson wrote: > Hi David, > > On 2018-05-21 02:35, David Holmes wrote: >> Hi Stefan, >> >> This updated webrev only came out late Friday night for me so I did >> not get a chance to review before you pushed it. > > Let's continue the discussion here, and push an update to the code as a > new RFE. > >> I would have argued for keeping consistency with existing similar >> API's and use the "bool safepoint_check" parameter instead of adding a >> new variant of the "wait" method. >> >> src/hotspot/share/runtime/semaphore.inline.hpp >> >> ?2? * Copyright (c) 2015, 2018, >> >> How does a new file have a 2015 copyright? > > This code comes from the ZGC repository, and was created 2015. > >> >> ? 41 inline void Semaphore::wait_with_safepoint_check() { >> ? 42?? Thread* thread = Thread::current(); >> ? 43?? if (thread->is_Java_thread()) { >> ? 44 wait_with_safepoint_check(static_cast(thread)); >> ? 45?? } else { >> ? 46???? // Wait for value >> ? 47???? _impl.wait(); >> ? 48?? } >> ? 49 } >> >> This seems misnamed - if not a JavaThread then there is no safepoint >> check. >> >> It seems confusing to me to now have three wait variants with no clear >> indicator as to whether specific types of threads should only call >> specific variants (and no assertions checking that). I would expect >> JavaThreads to always call wait_with_safepoint_check(JavaThread), >> while non-JavaThread's should only call wait(). The parameter-less >> wait_with_safepoint_check() doesn't seem desirable. I'm guessing it is >> there to allow the dispatching to be done by the semaphore code rather >> than having the callers call Thread::current() and dispatch themselves? > > Yes, this is what the code in ZGC looks like: > http://hg.openjdk.java.net/zgc/zgc/file/d99bcf9de8f3/src/hotspot/share/gc/z/zFuture.inline.hpp > > > This code can be called from JavaThreads and GC threads > (non-JavaThreads), and after this change this would start to use > wait_with_safepoint_check(). > >> >> But unless we may sometimes want JavaThreads to not perform safepoint >> checks (do we??), >> this seems overly complicated. We could have simply defined wait to >> take a Thread parameter (default NULL) and then decide whether to use >> ThreadBlockInVM or not based on whether it was a JavaThread. > > This would introduce a virtual call in the handshake code. > >> At most two variants seem needed (if you don't like potentially >> calling Thread::current() if you don't need the Thread) rather than >> three. > > We have these use-cases (maybe there are more): > 1) GC threads using Semaphores - using Semaphore::wait - no safepoint > checking > 2) Java threads executing handshake code - > Semaphore::wait_with_safepoint_check(JavaThread*) - with safepoint checking > 3) ZGC code that are executed by both GC threads and Java threads - > using Semaphore::wait_with_safepoint_check() -? dispatching to either > (1) or (2). > > We could easily move the code in Semaphore::wait_with_safepoint_check() > to ZGC, if you don't think it's usable by other parts of the VM. That > would also solve the naming problem. > > http://cr.openjdk.java.net/~stefank/8203341/webrev.04.delta/ > http://cr.openjdk.java.net/~stefank/8203341/webrev.04/ > > Is this OK, or do you still want a bool to mimic the Mutex API? I think > most usages of Semaphores want to use the wait() function (without > safepoint check), but the default for Mutex is to lock with safepoint > check. tl;dr The above looks okay. To be clear ... JavaThreads always want a safepoint check; non-JavaThreads never want a safepoint check, so it boils down to how you decide which variant to call. On the callee side you can either have two methods with different semantics (as proposed) or one method that takes a parameter to select the behaviour (as mutex does). Given the pre-existence of mutex et al I prefer to continue using a parameter to select, but can live with two methods. Then at the caller side it comes down to whether you know if you are a JavaThread or not. If you know you are and you have an existing thread reference, or you know you are not, then you can call the right variant directly. Otherwise you need to obtain the currentThread and/or check its type - that can either be done at the callsite or folded inside the argument-less dispatching wait_xxx() method as per the committed changes. If you go with the argument-less dispatching variant then accurate naming becomes awkward: wait_with_safepoint_check_if_needed(). Blegghh. :) So I prefer the callsites doing the dispatching themselves as an initial position - and if we find this becomes burdensome then consider changing it. So what you propose in webrev.04 seems fine to move this along. Thanks, David > > Thanks, > StefanK > >> >> Cheers, >> David >> ----- >> >> On 18/05/2018 10:45 PM, Stefan Karlsson wrote: >>> I got offline suggestions that I should rename the function to >>> wait_with_safepoint_check: >>> >>> http://cr.openjdk.java.net/~stefank/8203341/webrev.03/ >>> http://cr.openjdk.java.net/~stefank/8203341/webrev.03.delta/ >>> >>> Thanks, >>> StefanK >>> >>> On 2018-05-18 14:21, Erik ?sterlund wrote: >>>> Hi Stefan, >>>> >>>> Looks good. >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 2018-05-18 14:13, Stefan Karlsson wrote: >>>>> Updated webrevs: >>>>> ?http://cr.openjdk.java.net/~stefank/8203341/webrev.02/ >>>>> ?http://cr.openjdk.java.net/~stefank/8203341/webrev.02.delta/ >>>>> >>>>> I removed the SemaphoreSafepointAware class, as Erik requested, but >>>>> instead of adding a bool I added a new function for the safepoint >>>>> aware waits. >>>>> >>>>> Thanks, >>>>> StefanK >>>>> >>>>> On 2018-05-17 17:59, Stefan Karlsson wrote: >>>>>> On 2018-05-17 17:38, Erik ?sterlund wrote: >>>>>>> Hi Stefan, >>>>>>> >>>>>>> I wonder if it would make sense to let Semaphore::wait have a >>>>>>> bool safepoint_check, similar to Mutex (possibly with a default >>>>>>> value), instead of introducing a new class with the same API, >>>>>>> that only differs in this property. >>>>>> >>>>>> That sounds reasonable to me. >>>>>> >>>>>> I also need to change my patch so that the thread-local handshake >>>>>> code pass in the existing JavaThread, so that we don't have to >>>>>> call Thread::current()->is_JavaThread() unnecessarily. >>>>>> >>>>>> Thanks, >>>>>> StefanK >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>>>> >>>>>>> On 2018-05-17 09:46, Stefan Karlsson wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Please review this patch to add a safepoint-aware Semaphore (and >>>>>>>> refactor the semaphore usage in thread-local handshakes). >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~stefank/8203341/webrev.01/ >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8203341 >>>>>>>> >>>>>>>> Both ZGC and the thread-local handshakes need to move the >>>>>>>> JavaThread to a blocked state before waiting on a Semaphore. I >>>>>>>> propose that we introduce a new SemaphoreSafepointAware wrapper >>>>>>>> around Semaphore. >>>>>>>> >>>>>>>> There are two things to consider with this patch: >>>>>>>> >>>>>>>> 1) In ZGC, where this patch originates from, we set the >>>>>>>> JavaThread's osthread state with >>>>>>>> osthread->set_state(CONDVAR_WAIT) (using OSThreadWaitState). >>>>>>>> This means that we get the following printed when the threads >>>>>>>> are dumped: "waiting on condition ". I've kept this behavior for >>>>>>>> the SemaphoreSafepointAware class. This means that we now change >>>>>>>> the osthread state for JavaThreads blocking in >>>>>>>> HandshakeState::process_self_inner. >>>>>>>> >>>>>>>> 2) The thread-local handshakes uses trywait before entering wait: >>>>>>>> ?? if (!_semaphore.trywait()) { >>>>>>>> -??? ThreadBlockInVM tbivm(thread); >>>>>>>> ???? _semaphore.wait(); >>>>>>>> ?? } >>>>>>>> >>>>>>>> At the time when SemaphoreSafepointAware was written, we didn't >>>>>>>> have trywait on all platforms, now we do. Maybe we should always >>>>>>>> do the trywait dance inside SemaphoreSafepointAware::wait() ? >>>>>>>> >>>>>>>> inline void SemaphoreSafepointAware::wait() { >>>>>>>> ? if (Thread::current()->is_Java_thread()) { >>>>>>>> ??? if (!_semaphore.trywait) { >>>>>>>> ????? JavaThread* thread = JavaThread::current(); >>>>>>>> >>>>>>>> ????? // Prepare to block and allow safepoints while blocked >>>>>>>> ????? ThreadBlockInVM tbivm(thread); >>>>>>>> ????? OSThreadWaitState osts(thread->osthread(), false /* not in >>>>>>>> Object.wait() */); >>>>>>>> >>>>>>>> ????? // Wait for value >>>>>>>> ???? _semaphore.wait(); >>>>>>>> ?? } >>>>>>>> >>>>>>>> Thanks, >>>>>>>> StefanK >>>>>>> >>>>>> >>>> > From david.holmes at oracle.com Tue May 22 01:45:42 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 22 May 2018 11:45:42 +1000 Subject: UseNUMA membind Issue in openJDK In-Reply-To: References: <9a0310b7-2880-db69-cfbc-7abba844ecbf@oracle.com> Message-ID: Hi Swati, On 22/05/2018 2:44 AM, Swati Sharma wrote: > +Gustavo > > Hi David, > > Updated previous patch.Some issue was there with issinglenodeound() > method on SUSE, so just swapped the condition, checking in same. Please > review and let me know your comments. As I originally said "I can't comment on the actual NUMA details of the patch.", so won't be reviewing it. Sorry. David > Derek : Thanks for the info :) I dint know that its already reported. > > Here is the updated patch,attached too. > ======================PATCH============================== > diff --git a/src/hotspot/os/linux/os_linux.cpp > b/src/hotspot/os/linux/os_linux.cpp > --- a/src/hotspot/os/linux/os_linux.cpp > +++ b/src/hotspot/os/linux/os_linux.cpp > @@ -2832,14 +2832,42 @@ > ? ?// Map all node ids in which is possible to allocate memory. Also > nodes are > ? ?// not always consecutively available, i.e. available from 0 to the > highest > ? ?// node number. > +? // If the nodes have been bound explicitly using numactl membind, then > +? // allocate memory from those nodes only. > ? ?for (size_t node = 0; node <= highest_node_number; node++) { > -? ? if (Linux::isnode_in_configured_nodes(node)) { > +? ? if (Linux::isnode_in_bounded_nodes(node)) { > ? ? ? ?ids[i++] = node; > ? ? ?} > ? ?} > ? ?return i; > ?} > +extern "C"? struct bitmask { > +? unsigned long size; /* number of bits in the map */ > +? unsigned long *maskp; > +}; > +// Check if single memory node bound. > +// Returns true if single memory node bound. > +bool os::Linux::issingle_node_bound() { > +? struct bitmask* bmp = _numa_get_membind != NULL ? _numa_get_membind() > : NULL; > +? if(!(bmp != NULL && bmp->maskp != NULL)) return false; > +? int issingle = 0; > +? // System can have more than 64 nodes so check in all the elements of > +? // unsigned long array > +? for (unsigned long i = 0; i < (bmp->size / (8 * sizeof(unsigned > long))); i++) { > +? ? if (bmp->maskp[i] == 0) { > +? ? ? continue; > +? ? } else if ((bmp->maskp[i] & (bmp->maskp[i] - 1)) == 0) { > +? ? ? issingle++; > +? ? } else { > +? ? ? return false; > +? ? } > +? } > +? if (issingle == 1) > +? ? return true; > +? return false; > +} > + > ?bool os::get_page_info(char *start, page_info* info) { > ? ?return false; > ?} > @@ -2930,6 +2958,10 @@ > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? libnuma_dlsym(handle, > "numa_bitmask_isbitset"))); > ? ? ? ?set_numa_distance(CAST_TO_FN_PTR(numa_distance_func_t, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? libnuma_dlsym(handle, > "numa_distance"))); > +? ? ? set_numa_set_membind(CAST_TO_FN_PTR(numa_set_membind_func_t, > +? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? libnuma_dlsym(handle, > "numa_set_membind"))); > +? ? ? set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > +? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? libnuma_v2_dlsym(handle, > "numa_get_membind"))); > ? ? ? ?if (numa_available() != -1) { > ? ? ? ? ?set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, > "numa_all_nodes")); > @@ -3054,6 +3086,8 @@ > ?os::Linux::numa_set_bind_policy_func_t os::Linux::_numa_set_bind_policy; > ?os::Linux::numa_bitmask_isbitset_func_t os::Linux::_numa_bitmask_isbitset; > ?os::Linux::numa_distance_func_t os::Linux::_numa_distance; > +os::Linux::numa_set_membind_func_t os::Linux::_numa_set_membind; > +os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; > ?unsigned long* os::Linux::_numa_all_nodes; > ?struct bitmask* os::Linux::_numa_all_nodes_ptr; > ?struct bitmask* os::Linux::_numa_nodes_ptr; > @@ -4962,8 +4996,9 @@ > ? ? ?if (!Linux::libnuma_init()) { > ? ? ? ?UseNUMA = false; > ? ? ?} else { > -? ? ? if ((Linux::numa_max_node() < 1)) { > -? ? ? ? // There's only one node(they start from 0), disable NUMA. > +? ? ? if ((Linux::numa_max_node() < 1) || Linux::issingle_node_bound()) { > +? ? ? ? // If there's only one node(they start from 0) or if the process > +? ? ? ? // is bound explicitly to a single node using membind, disable > NUMA. > ? ? ? ? ?UseNUMA = false; > ? ? ? ?} > ? ? ?} > diff --git a/src/hotspot/os/linux/os_linux.hpp > b/src/hotspot/os/linux/os_linux.hpp > --- a/src/hotspot/os/linux/os_linux.hpp > +++ b/src/hotspot/os/linux/os_linux.hpp > @@ -228,6 +228,8 @@ > ? ?typedef int (*numa_tonode_memory_func_t)(void *start, size_t size, > int node); > ? ?typedef void (*numa_interleave_memory_func_t)(void *start, size_t > size, unsigned long *nodemask); > ? ?typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t > size, struct bitmask* mask); > +? typedef void (*numa_set_membind_func_t)(struct bitmask *mask); > +? typedef struct bitmask* (*numa_get_membind_func_t)(void); > ? ?typedef void (*numa_set_bind_policy_func_t)(int policy); > ? ?typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, > unsigned int n); > @@ -244,6 +246,8 @@ > ? ?static numa_set_bind_policy_func_t _numa_set_bind_policy; > ? ?static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; > ? ?static numa_distance_func_t _numa_distance; > +? static numa_set_membind_func_t _numa_set_membind; > +? static numa_get_membind_func_t _numa_get_membind; > ? ?static unsigned long* _numa_all_nodes; > ? ?static struct bitmask* _numa_all_nodes_ptr; > ? ?static struct bitmask* _numa_nodes_ptr; > @@ -259,6 +263,8 @@ > ? ?static void set_numa_set_bind_policy(numa_set_bind_policy_func_t > func) { _numa_set_bind_policy = func; } > ? ?static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t > func) { _numa_bitmask_isbitset = func; } > ? ?static void set_numa_distance(numa_distance_func_t func) { > _numa_distance = func; } > +? static void set_numa_set_membind(numa_set_membind_func_t func) { > _numa_set_membind = func; } > +? static void set_numa_get_membind(numa_get_membind_func_t func) { > _numa_get_membind = func; } > ? ?static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes > = ptr; } > ? ?static void set_numa_all_nodes_ptr(struct bitmask **ptr) { > _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > ? ?static void set_numa_nodes_ptr(struct bitmask **ptr) { > _numa_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > @@ -320,6 +326,15 @@ > ? ? ?} else > ? ? ? ?return 0; > ? ?} > +? // Check if node in bounded nodes > +? static bool isnode_in_bounded_nodes(int node) { > +? ? struct bitmask* bmp = _numa_get_membind != NULL ? > _numa_get_membind() : NULL; > +? ? if (bmp != NULL && _numa_bitmask_isbitset != NULL && > _numa_bitmask_isbitset(bmp, node)) { > +? ? ? return true; > +? ? } else > +? ? ? return false; > +? } > +? static bool issingle_node_bound(); > ?}; > ?#endif // OS_LINUX_VM_OS_LINUX_HPP > > ========================================================== > > On Fri, May 4, 2018 at 2:56 AM, White, Derek > wrote: > > Hi Swati, > > I think this was reported as JDK-8189922 "UseNUMA memory > interleaving allocates heap for unavailable nodes". > https://bugs.openjdk.java.net/browse/JDK-8189922 > > > Thanks for working on a fix for this! > > ?- Derek > > > -----Original Message----- > > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net > ] On > > Behalf Of Swati Sharma > > Sent: Wednesday, May 02, 2018 6:24 AM > > To: David Holmes > > > Cc: hotspot-dev at openjdk.java.net ; > Prakash.Raghavendra at amd.com > > Subject: Re: UseNUMA membind Issue in openJDK > > > > Hi David, > > > > I have localized the struct bitmask declaration in os_linux.cpp. > > > > Here is the updated patch > > ===================================PATCH====================== > > ============================= > > diff --git a/src/hotspot/os/linux/os_linux.cpp > > b/src/hotspot/os/linux/os_linux.cpp > > --- a/src/hotspot/os/linux/os_linux.cpp > > +++ b/src/hotspot/os/linux/os_linux.cpp > > @@ -2832,14 +2832,42 @@ > >? ? // Map all node ids in which is possible to allocate memory. Also nodes are > >? ? // not always consecutively available, i.e. available from 0 to the highest > >? ? // node number. > > +? // If the nodes have been bound explicitly using numactl membind, > > + then? // allocate memory from those nodes only. > >? ? for (size_t node = 0; node <= highest_node_number; node++) { > > -? ? if (Linux::isnode_in_configured_nodes(node)) { > > +? ? if (Linux::isnode_in_bounded_nodes(node)) { > >? ? ? ? ids[i++] = node; > >? ? ? } > >? ? } > >? ? return i; > >? } > > > > +extern "C"? struct bitmask { > > +? unsigned long size; /* number of bits in the map */ > > +? unsigned long *maskp; > > +}; > > +// Check if single memory node bound. > > +// Returns true if single memory node bound. > > +bool os::Linux::issingle_node_bound() { > > +? struct bitmask* bmp = _numa_get_membind != NULL ? > > _numa_get_membind() : > > NULL; > > +? if(bmp == NULL) return false; > > +? int issingle = 0; > > +? // System can have more than 64 nodes so check in all the elements of > > + // unsigned long array? for (unsigned long i = 0; i < > (bmp->size / (8 > > + * sizeof(unsigned long))); > > i++) { > > +? ? if (bmp->maskp != NULL && (((bmp->maskp[i]) & (((bmp->maskp[i])) - > > + 1)) > > == 0)) { > > +? ? ? issingle++; > > +? ? } else if (bmp->maskp[i] == 0) { > > +? ? ? continue; > > +? ? } else { > > +? ? ? return false; > > +? ? } > > +? } > > +? if (issingle == 1) > > +? ? return true; > > +? return false; > > +} > > + > >? bool os::get_page_info(char *start, page_info* info) { > >? ? return false; > >? } > > @@ -2930,6 +2958,10 @@ > >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?libnuma_dlsym(handle, > > "numa_bitmask_isbitset"))); > >? ? ? ? set_numa_distance(CAST_TO_FN_PTR(numa_distance_func_t, > >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?libnuma_dlsym(handle, > "numa_distance"))); > > + > > set_numa_set_membind(CAST_TO_FN_PTR(numa_set_membind_func_t, > > +? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? libnuma_dlsym(handle, > > "numa_set_membind"))); > > + > > set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > > +? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? libnuma_v2_dlsym(handle, > > "numa_get_membind"))); > > > >? ? ? ? if (numa_available() != -1) { > >? ? ? ? ? set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, > > "numa_all_nodes")); @@ -3054,6 +3086,8 @@ > > os::Linux::numa_set_bind_policy_func_t > os::Linux::_numa_set_bind_policy; > > os::Linux::numa_bitmask_isbitset_func_t > os::Linux::_numa_bitmask_isbitset; > >? os::Linux::numa_distance_func_t os::Linux::_numa_distance; > > +os::Linux::numa_set_membind_func_t os::Linux::_numa_set_membind; > > +os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; > >? unsigned long* os::Linux::_numa_all_nodes;? struct bitmask* > > os::Linux::_numa_all_nodes_ptr;? struct bitmask* > > os::Linux::_numa_nodes_ptr; @@ -4962,8 +4996,9 @@ > >? ? ? if (!Linux::libnuma_init()) { > >? ? ? ? UseNUMA = false; > >? ? ? } else { > > -? ? ? if ((Linux::numa_max_node() < 1)) { > > -? ? ? ? // There's only one node(they start from 0), disable NUMA. > > +? ? ? if ((Linux::numa_max_node() < 1) || > Linux::issingle_node_bound()) { > > +? ? ? ? // If there's only one node(they start from 0) or if the > process > > +? ? ? ? // is bound explicitly to a single node using membind, > disable > > NUMA. > >? ? ? ? ? UseNUMA = false; > >? ? ? ? } > >? ? ? } > > diff --git a/src/hotspot/os/linux/os_linux.hpp > > b/src/hotspot/os/linux/os_linux.hpp > > --- a/src/hotspot/os/linux/os_linux.hpp > > +++ b/src/hotspot/os/linux/os_linux.hpp > > @@ -228,6 +228,8 @@ > >? ? typedef int (*numa_tonode_memory_func_t)(void *start, size_t > size, int > > node); > >? ? typedef void (*numa_interleave_memory_func_t)(void *start, > size_t size, > > unsigned long *nodemask); > >? ? typedef void (*numa_interleave_memory_v2_func_t)(void *start, > size_t > > size, struct bitmask* mask); > > +? typedef void (*numa_set_membind_func_t)(struct bitmask *mask); > > + typedef struct bitmask* (*numa_get_membind_func_t)(void); > > > >? ? typedef void (*numa_set_bind_policy_func_t)(int policy); > >? ? typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, > > unsigned int n); @@ -244,6 +246,8 @@ > >? ? static numa_set_bind_policy_func_t _numa_set_bind_policy; > >? ? static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; > >? ? static numa_distance_func_t _numa_distance; > > +? static numa_set_membind_func_t _numa_set_membind;? static > > + numa_get_membind_func_t _numa_get_membind; > >? ? static unsigned long* _numa_all_nodes; > >? ? static struct bitmask* _numa_all_nodes_ptr; > >? ? static struct bitmask* _numa_nodes_ptr; @@ -259,6 +263,8 @@ > >? ? static void set_numa_set_bind_policy(numa_set_bind_policy_func_t func) > > { _numa_set_bind_policy = func; } > >? ? static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t > > func) { _numa_bitmask_isbitset = func; } > >? ? static void set_numa_distance(numa_distance_func_t func) { > > _numa_distance = func; } > > +? static void set_numa_set_membind(numa_set_membind_func_t func) { > > _numa_set_membind = func; } > > +? static void set_numa_get_membind(numa_get_membind_func_t func) { > > _numa_get_membind = func; } > >? ? static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes = > > ptr; } > >? ? static void set_numa_all_nodes_ptr(struct bitmask **ptr) { > > _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > >? ? static void set_numa_nodes_ptr(struct bitmask **ptr) { _numa_nodes_ptr > > = (ptr == NULL ? NULL : *ptr); } @@ -320,6 +326,15 @@ > >? ? ? } else > >? ? ? ? return 0; > >? ? } > > +? // Check if node in bounded nodes > > +? static bool isnode_in_bounded_nodes(int node) { > > +? ? struct bitmask* bmp = _numa_get_membind != NULL ? > > + _numa_get_membind() > > : NULL; > > +? ? if (bmp != NULL && _numa_bitmask_isbitset != NULL && > > _numa_bitmask_isbitset(bmp, node)) { > > +? ? ? return true; > > +? ? } else > > +? ? ? return false; > > +? } > > +? static bool issingle_node_bound(); > >? }; > > > >? #endif // OS_LINUX_VM_OS_LINUX_HPP > > > > =============================================================== > > ============================= > > > > Thanks, > > Swati > > > > On Thu, Apr 26, 2018 at 6:10 PM, David Holmes > > > > wrote: > > > > > > Hi Swati, > > > > > > On 26/04/2018 10:20 PM, Swati Sharma wrote: > > >> > > >> Hi Everyone, > > >> > > >> I work at AMD and this is my first patch as a new member of > openJDK > > >> community. > > > > > > > > > Welcome! > > > > > > I can't comment on the actual NUMA details of the patch (though > I can > > > see > > what you're doing), but the struct bitmask declaration in os.hpp > should be > > localized in os_linux.hpp as far as I can see, as it's only > needed internally in > > the Linux code. > > > > > > Thanks, > > > David > > > ----- > > > > > > > > >> I have found some issue while running? specjbb2015 composite > workload > > with > > >> the flag -XX:+UseNUMA. It seems that JVM does not allocate memory > > according > > >> to the explicit node binding done using "numactl --membind". > > >> > > >> E.g. If bound to a single memory node,? JVM divides the whole heap > > >> based > > on > > >> the total number of numa nodes available on the system which > creates > > >> more logical groups(lgrps) than required which cannot be used > except the > > one. > > >> > > >> The following examples will explain clearly : > > >> (Note : Collected GC logs with > > >> -Xlog:gc*=debug:file=gc.log:time,uptimemillis) > > >> 1) Allocating a heap of 22GB for single node divides the whole > heap > > >> in 8 lgrp(Actual no of Nodes are 8) > > >>? ? ? $numactl --cpunodebind=0 --membind=0 java -Xmx24g -Xms24g > > >> -Xmn22g -XX:+UseNUMA > > >> > > >>? ? ? eden space 22511616K(22GB), 12% used > > >>? ? ? lgrp 0 space 2813952K, 100% used > ?lgrp 1 space > > >> 2813952K, 0% used? ? ? ? ? ? ? ? ? ? ? ? ? lgrp 2 space > 2813952K, 0% used > > >>? ? ? lgrp 3 space 2813952K, 0% used > ?lgrp 4 > > space > > >> 2813952K, 0% used? ? ? ? ? ? ? ? ? ? ? ? ? lgrp 5 space > 2813952K, 0% used > > >>? ? ? lgrp 6 space 2813952K, 0% used > ?lgrp 7 > > space > > >> 2813952K, 0% used > > >> > > >> Observation : Instead of disabling UseNUMA for single node binding > > >> JVM divides the memory in 8 lgrps and allocates memory always > on the > > >> bounded node hence in eden space allocation never happens more > than > > 12%. > > >> > > >> 2) Another case of binding to node 0 and 7 results in dividing the > > >> heap > > in > > >> 8lgrp > > >>? ? ? $numactl --cpunodebind=0,7 ?membind=0,7 java -Xms50g > -Xmx50g - > > Xmn45g > > >>? ?-XX:+UseNUMA > > >> > > >>? ? ? eden space 46718976K, 6% used > > >>? ? ? lgrp 0 space 5838848K, 14% used? ? ? ? ? ? ? ? ? lgrp 1 space > > 5838848K, > > >> 0% used? ? ? ? ? ? ? ? ? ? ? ? ? ? ? lgrp 2 space 5838848K, 0% > used > > >>? ? ? lgrp 3 space 5838848K, 0% used? ? ? ? ? ? ? ? ? ? lgrp 4 > space > > >> 5838848K, 0% used? ? ? ? ? ? ? ? ? ? ? ? ? ? ? lgrp 5 space > 5838848K, 0% > > >> used > > >>? ? ? ?lgrp 6 space 5838848K, 0% used? ? ? ? ? ? ? ? ? ? lgrp 7 > space > > >> 5847040K, 35% used > > >> > > >> Observation : Similar to first case allocation happens only on 0th > > >> and > > 7th > > >> node and rest of the lgrps never gets used. > > >> > > >> After applying the patch, JVM divides the given heap size > according > > >> to > > the > > >> bounded memory nodes only. > > >> > > >> 1) Binding to single node disables UseNUMA > > >>? ? ? eden space 46718976K(45GB), 99% used > > >> > > >> Observation : UseNUMA gets disabled hence no lgrp creation and the > > >> whole heap allocation happens on the bounded node. > > >> > > >> 2) Binding to node 0 and 7 > > >>? ? ? ?$ numactl --cpunodebind=0,7 ?membind=0,7 java -Xms50g > -Xmx50g > > -Xmn45g > > >>? ?-XX:+UseNUMA > > >>? ? ? ?eden space 46718976K(45GB), 99% used > > >>? ? ? ?lgrp 0 space 23359488K(23.5GB), 100% used > lgrp 7 space > > >> 23359488K(23.5GB), 99% used > > >> > > >> Observation : Only two lgrps gets created and heap size gets > divided > > >> equally in both nodes. > > >> > > >> If there is no binding, then JVM will divide the whole heap > based on > > >> the number of NUMA nodes available on the system. > > >> > > >> The following patch fixes the issue(attached also). > > >> Please review and let me know your comments. > > >> > > >> Regression testing using jtreg (make -J=1 run-test-tier1 > > >> run-test-tier2) didn't show any new failures. > > >> > > >> > > ===============================PATCH========================== > > ============== > > >> diff --git a/src/hotspot/os/linux/os_linux.cpp > > >> b/src/hotspot/os/linux/os_linux.cpp > > >> --- a/src/hotspot/os/linux/os_linux.cpp > > >> +++ b/src/hotspot/os/linux/os_linux.cpp > > >> @@ -2832,8 +2832,10 @@ > > >>? ? ?// Map all node ids in which is possible to allocate > memory. Also > > nodes > > >> are > > >>? ? ?// not always consecutively available, i.e. available from > 0 to > > >> the highest > > >>? ? ?// node number. > > >> +? // If the nodes have been bound explicitly using numactl > membind, > > >> + then? // allocate memory from those nodes only. > > >>? ? ?for (size_t node = 0; node <= highest_node_number; node++) { > > >> -? ? if (Linux::isnode_in_configured_nodes(node)) { > > >> +? ? if (Linux::isnode_in_bounded_nodes(node)) { > > >>? ? ? ? ?ids[i++] = node; > > >>? ? ? ?} > > >>? ? ?} > > >> @@ -2930,6 +2932,10 @@ > > >> > > >> libnuma_dlsym(handle, "numa_bitmask_isbitset"))); > > >>? ? ? ? ?set_numa_distance(CAST_TO_FN_PTR(numa_distance_func_t, > > >>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? libnuma_dlsym(handle, > > >> "numa_distance"))); > > >> + > > set_numa_set_membind(CAST_TO_FN_PTR(numa_set_membind_func_t, > > >> +? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? libnuma_dlsym(handle, > > >> "numa_set_membind"))); > > >> + > > set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > > >> + > libnuma_v2_dlsym(handle, > > >> "numa_get_membind"))); > > >> > > >>? ? ? ? ?if (numa_available() != -1) { > > >>? ? ? ? ? ?set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, > > >> "numa_all_nodes")); @@ -3054,6 +3060,8 @@ > > >>? ?os::Linux::numa_set_bind_policy_func_t > > os::Linux::_numa_set_bind_policy; > > >>? ?os::Linux::numa_bitmask_isbitset_func_t > > os::Linux::_numa_bitmask_isbitset; > > >>? ?os::Linux::numa_distance_func_t os::Linux::_numa_distance; > > >> +os::Linux::numa_set_membind_func_t os::Linux::_numa_set_membind; > > >> +os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; > > >>? ?unsigned long* os::Linux::_numa_all_nodes; > > >>? ?struct bitmask* os::Linux::_numa_all_nodes_ptr; > > >>? ?struct bitmask* os::Linux::_numa_nodes_ptr; @@ -4962,8 > +4970,9 @@ > > >>? ? ? ?if (!Linux::libnuma_init()) { > > >>? ? ? ? ?UseNUMA = false; > > >>? ? ? ?} else { > > >> -? ? ? if ((Linux::numa_max_node() < 1)) { > > >> -? ? ? ? // There's only one node(they start from 0), disable > NUMA. > > >> +? ? ? if ((Linux::numa_max_node() < 1) || > > >> + Linux::issingle_node_bound()) > > { > > >> +? ? ? ? // If there's only one node(they start from 0) or if the process > > >> +? ? ? ? // is bound explicitly to a single node using membind, > > >> + disable > > >> NUMA. > > >>? ? ? ? ? ?UseNUMA = false; > > >>? ? ? ? ?} > > >>? ? ? ?} > > >> diff --git a/src/hotspot/os/linux/os_linux.hpp > > >> b/src/hotspot/os/linux/os_linux.hpp > > >> --- a/src/hotspot/os/linux/os_linux.hpp > > >> +++ b/src/hotspot/os/linux/os_linux.hpp > > >> @@ -228,6 +228,8 @@ > > >>? ? ?typedef int (*numa_tonode_memory_func_t)(void *start, size_t > > >> size, > > int > > >> node); > > >>? ? ?typedef void (*numa_interleave_memory_func_t)(void *start, size_t > > size, > > >> unsigned long *nodemask); > > >>? ? ?typedef void (*numa_interleave_memory_v2_func_t)(void *start, > > >> size_t size, struct bitmask* mask); > > >> +? typedef void (*numa_set_membind_func_t)(struct bitmask *mask); > > >> + typedef struct bitmask* (*numa_get_membind_func_t)(void); > > >> > > >>? ? ?typedef void (*numa_set_bind_policy_func_t)(int policy); > > >>? ? ?typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, > > >> unsigned int n); @@ -244,6 +246,8 @@ > > >>? ? ?static numa_set_bind_policy_func_t _numa_set_bind_policy; > > >>? ? ?static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; > > >>? ? ?static numa_distance_func_t _numa_distance; > > >> +? static numa_set_membind_func_t _numa_set_membind;? static > > >> + numa_get_membind_func_t _numa_get_membind; > > >>? ? ?static unsigned long* _numa_all_nodes; > > >>? ? ?static struct bitmask* _numa_all_nodes_ptr; > > >>? ? ?static struct bitmask* _numa_nodes_ptr; @@ -259,6 +263,8 @@ > > >>? ? ?static void > set_numa_set_bind_policy(numa_set_bind_policy_func_t > > func) { > > >> _numa_set_bind_policy = func; } > > >>? ? ?static void > > >> set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t > > func) > > >> { _numa_bitmask_isbitset = func; } > > >>? ? ?static void set_numa_distance(numa_distance_func_t func) { > > >> _numa_distance = func; } > > >> +? static void set_numa_set_membind(numa_set_membind_func_t func) > > { > > >> _numa_set_membind = func; } > > >> +? static void set_numa_get_membind(numa_get_membind_func_t func) > > { > > >> _numa_get_membind = func; } > > >>? ? ?static void set_numa_all_nodes(unsigned long* ptr) { > > >> _numa_all_nodes > > = > > >> ptr; } > > >>? ? ?static void set_numa_all_nodes_ptr(struct bitmask **ptr) { > > >> _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > > >>? ? ?static void set_numa_nodes_ptr(struct bitmask **ptr) { > > _numa_nodes_ptr = > > >> (ptr == NULL ? NULL : *ptr); } > > >> @@ -320,6 +326,34 @@ > > >>? ? ? ?} else > > >>? ? ? ? ?return 0; > > >>? ? ?} > > >> +? // Check if node in bounded nodes > > >> +? static bool isnode_in_bounded_nodes(int node) { > > >> +? ? struct bitmask* bmp = _numa_get_membind != NULL ? > > _numa_get_membind() > > >> : NULL; > > >> +? ? if (bmp != NULL && _numa_bitmask_isbitset != NULL && > > >> _numa_bitmask_isbitset(bmp, node)) { > > >> +? ? ? return true; > > >> +? ? } else > > >> +? ? ? return false; > > >> +? } > > >> +? // Check if a single node is bound? static bool > > >> + issingle_node_bound() { > > >> +? ? struct bitmask* bmp = _numa_get_membind != NULL ? > > _numa_get_membind() > > >> : NULL; > > >> +? ? if(bmp == NULL) return false; > > >> +? ? int issingle = 0; > > >> +? ? // System can have more than 64 nodes so check in all the > > >> + elements > > of > > >> +? ? // unsigned long array > > >> +? ? for (unsigned long i = 0; i < (bmp->size / (8 * sizeof(unsigned > > >> long))); i++) { > > >> +? ? ? ?if (bmp->maskp != NULL && (((bmp->maskp[i]) & > > >> + (((bmp->maskp[i])) > > - > > >> 1)) == 0)) { > > >> +? ? ? ? ?issingle++; > > >> +? ? ? ?} else if (bmp->maskp[i] == 0) { > > >> +? ? ? ? ?continue; > > >> +? ? ? ?} else { > > >> +? ? ? ? ?return false; > > >> +? ? ? ?} > > >> +? ? } > > >> +? ? if (issingle == 1) > > >> +? ? ? return true; > > >> +? ? return false; > > >> +? } > > >>? ?}; > > >> > > >>? ?#endif // OS_LINUX_VM_OS_LINUX_HPP > > >> diff --git a/src/hotspot/share/runtime/os.hpp > > >> b/src/hotspot/share/runtime/os.hpp > > >> --- a/src/hotspot/share/runtime/os.hpp > > >> +++ b/src/hotspot/share/runtime/os.hpp > > >> @@ -81,6 +81,10 @@ > > >>? ? ?CriticalPriority = 11? ? ? // Critical thread priority > > >>? ?}; > > >> > > >> +extern "C" struct bitmask { > > >> +? unsigned long size; /* number of bits in the map */ > > >> +? unsigned long *maskp; > > >> +}; > > >>? ?// Executable parameter flag for os::commit_memory() and > > >>? ?// os::commit_memory_or_exit(). > > >>? ?const bool ExecMem = true; > > >> > > >> > > =============================================================== > > ============== > > >> > > >> Thanks, > > >> Swati > > >> > > From mikhailo.seledtsov at oracle.com Tue May 22 02:45:56 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Mon, 21 May 2018 19:45:56 -0700 Subject: RFR(L) : 8199379 : [TESTBUG] Open source vm testbase JDB tests In-Reply-To: References: <6D060F7C-C265-48CF-BB21-532848461B2A@oracle.com> Message-ID: <5B038464.4080304@oracle.com> +1, Misha On 5/18/18, 6:42 PM, serguei.spitsyn at oracle.com wrote: > Hi Igor, > > This looks good to me. > Thank you a lot for taking care about this move from closed to open! > > Thanks, > Serguei > > > On 5/15/18 14:21, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8199379/webrev.00/index.html >>> 18000 lines changed: 17999 ins; 0 del; 1 mod; >> >> Hi all, >> >> could you please review this patch which open sources JDB tests from >> VM testbase? >> >> these tests were developed to test 'jdb' tool and JDI thru it. >> >> As usually w/ VM testbase code, these tests are old, they have been >> run in hotspot testing for a long period of time. Originally, these >> tests were run by a test harness different from jtreg and had >> different build and execution schemes, some parts couldn't be easily >> translated to jtreg, so tests might have actions or pieces of code >> which look weird. In a long term, we are planning to rework them. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8199379 >> webrev: >> http://cr.openjdk.java.net/~iignatyev//8199379/webrev.00/index.html >> testing: :vmTestbase_nsk_jdb test group >> >> Thanks, >> -- Igor > From serguei.spitsyn at oracle.com Tue May 22 05:33:00 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Mon, 21 May 2018 22:33:00 -0700 Subject: RFR(M) : 8202946 : [TESTBUG] Open source VM testbase OOM tests In-Reply-To: <2DD8F9C6-8471-4BF6-8573-0DA3F2B6C66B@oracle.com> References: <2DD8F9C6-8471-4BF6-8573-0DA3F2B6C66B@oracle.com> Message-ID: <23ea1dbe-f6bc-b76a-237a-ff6fc1a6dc58@oracle.com> Hi Igor, This looks good. This change has to be reviewed by someone from the GC team. Thanks, Serguei On 5/15/18 16:16, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8202946/webrev.00/index.html >> 1619 lines changed: 1619 ins; 0 del; 0 mod; > Hi all, > > could you please review this patch which open sources OOM tests from VM testbase? these tests test OutOfMemoryError throwing in different scenarios. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8202946 > webrev: http://cr.openjdk.java.net/~iignatyev//8202946/webrev.00/index.html > testing: :vmTestbase_vm_oom test group > > Thanks, > -- Igor From kim.barrett at oracle.com Tue May 22 05:41:47 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 22 May 2018 07:41:47 +0200 Subject: RFR: 8202863: Rename OopStorage inner collection classes In-Reply-To: References: Message-ID: > On May 19, 2018, at 1:14 AM, coleen.phillimore at oracle.com wrote: > > > Also, this looks trivial enough to push with one reviewer. Thanks, I was thinking that too, since it?s a purely mechanical renaming of non-public classes, but forgot to mention that possibility in the RFR email. Might push today. > Coleen > > On 5/17/18 5:34 PM, Kim Barrett wrote: >>> On May 17, 2018, at 4:44 PM, coleen.phillimore at oracle.com wrote: >>> >>> Looks good! >>> Coleen >> Thanks. >> >>> On 5/17/18 4:33 PM, Kim Barrett wrote: >>>> [Resending hotspot-dev too, so Coleen will see it, since she requested this change.] >>>> >>>> Please review this simple renaming of some private inner classes of >>>> OopStorage. The renamings were suggested during the review of >>>> JDK-8200557, but were deferred to a separate change to avoid adding >>>> complexity to that review. The following classes are being renamed: >>>> >>>> BlockArray => ActiveArray >>>> BlockList => AllocateList >>>> BlockEntry => AllocateEntry >>>> >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8202863 >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~kbarrett/8202863/open.00/ >>>> >>>> Testing: >>>> Mach5 CI equivalent. From stefan.karlsson at oracle.com Tue May 22 06:45:53 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 22 May 2018 08:45:53 +0200 Subject: 8203341: Add a safepoint-aware Semaphore In-Reply-To: <5c1dbe12-e6f4-6fba-de5a-a92b501c9572@oracle.com> References: <2f8412d5-2fe4-2c8f-077c-cf96e8f9bebf@oracle.com> <5AFDA1E3.6010401@oracle.com> <385fc0f8-461d-5f0f-d678-1dcb69b173cf@oracle.com> <5AFEC537.4050304@oracle.com> <322b2a9c-4dfe-9790-1c1d-b1c7cfa4a8ff@oracle.com> <89f62752-c90f-02c0-ab5a-c3c1714cc3ae@oracle.com> <8d23706b-8afe-3367-e3b7-fc8ea3cd9957@oracle.com> <5c1dbe12-e6f4-6fba-de5a-a92b501c9572@oracle.com> Message-ID: <7bfd5e07-6a7e-d33b-8057-dc3ba3d34c74@oracle.com> Hi David, Thanks for reviewing and for the help to get this sorted out. I'll push the proposed webrev.04.delta patch. StefanK On 2018-05-22 03:19, David Holmes wrote: > Hi Stefan, > > On 22/05/2018 12:42 AM, Stefan Karlsson wrote: >> Hi David, >> >> On 2018-05-21 02:35, David Holmes wrote: >>> Hi Stefan, >>> >>> This updated webrev only came out late Friday night for me so I did >>> not get a chance to review before you pushed it. >> >> Let's continue the discussion here, and push an update to the code as >> a new RFE. >> >>> I would have argued for keeping consistency with existing similar >>> API's and use the "bool safepoint_check" parameter instead of adding >>> a new variant of the "wait" method. >>> >>> src/hotspot/share/runtime/semaphore.inline.hpp >>> >>> ?2? * Copyright (c) 2015, 2018, >>> >>> How does a new file have a 2015 copyright? >> >> This code comes from the ZGC repository, and was created 2015. >> >>> >>> ? 41 inline void Semaphore::wait_with_safepoint_check() { >>> ? 42?? Thread* thread = Thread::current(); >>> ? 43?? if (thread->is_Java_thread()) { >>> ? 44 wait_with_safepoint_check(static_cast(thread)); >>> ? 45?? } else { >>> ? 46???? // Wait for value >>> ? 47???? _impl.wait(); >>> ? 48?? } >>> ? 49 } >>> >>> This seems misnamed - if not a JavaThread then there is no safepoint >>> check. >>> >>> It seems confusing to me to now have three wait variants with no >>> clear indicator as to whether specific types of threads should only >>> call specific variants (and no assertions checking that). I would >>> expect JavaThreads to always call >>> wait_with_safepoint_check(JavaThread), while non-JavaThread's should >>> only call wait(). The parameter-less wait_with_safepoint_check() >>> doesn't seem desirable. I'm guessing it is there to allow the >>> dispatching to be done by the semaphore code rather than having the >>> callers call Thread::current() and dispatch themselves? >> >> Yes, this is what the code in ZGC looks like: >> http://hg.openjdk.java.net/zgc/zgc/file/d99bcf9de8f3/src/hotspot/share/gc/z/zFuture.inline.hpp >> >> >> This code can be called from JavaThreads and GC threads >> (non-JavaThreads), and after this change this would start to use >> wait_with_safepoint_check(). >> >>> >>> But unless we may sometimes want JavaThreads to not perform safepoint >>> checks (do we??), >>> this seems overly complicated. We could have simply defined wait to >>> take a Thread parameter (default NULL) and then decide whether to use >>> ThreadBlockInVM or not based on whether it was a JavaThread. >> >> This would introduce a virtual call in the handshake code. >> >>> At most two variants seem needed (if you don't like potentially >>> calling Thread::current() if you don't need the Thread) rather than >>> three. >> >> We have these use-cases (maybe there are more): >> 1) GC threads using Semaphores - using Semaphore::wait - no safepoint >> checking >> 2) Java threads executing handshake code - >> Semaphore::wait_with_safepoint_check(JavaThread*) - with safepoint >> checking >> 3) ZGC code that are executed by both GC threads and Java threads - >> using Semaphore::wait_with_safepoint_check() -? dispatching to either >> (1) or (2). >> >> We could easily move the code in >> Semaphore::wait_with_safepoint_check() to ZGC, if you don't think it's >> usable by other parts of the VM. That would also solve the naming >> problem. >> >> http://cr.openjdk.java.net/~stefank/8203341/webrev.04.delta/ >> http://cr.openjdk.java.net/~stefank/8203341/webrev.04/ >> >> Is this OK, or do you still want a bool to mimic the Mutex API? I >> think most usages of Semaphores want to use the wait() function >> (without safepoint check), but the default for Mutex is to lock with >> safepoint check. > > tl;dr The above looks okay. > > To be clear ... JavaThreads always want a safepoint check; > non-JavaThreads never want a safepoint check, so it boils down to how > you decide which variant to call. > > On the callee side you can either have two methods with different > semantics (as proposed) or one method that takes a parameter to select > the behaviour (as mutex does). Given the pre-existence of mutex et al I > prefer to continue using a parameter to select, but can live with two > methods. > > Then at the caller side it comes down to whether you know if you are a > JavaThread or not. If you know you are and you have an existing thread > reference, or you know you are not, then you can call the right variant > directly. Otherwise you need to obtain the currentThread and/or check > its type - that can either be done at the callsite or folded inside the > argument-less dispatching wait_xxx() method as per the committed > changes. If you go with the argument-less dispatching variant then > accurate naming becomes awkward: wait_with_safepoint_check_if_needed(). > Blegghh. :) > > So I prefer the callsites doing the dispatching themselves as an initial > position - and if we find this becomes burdensome then consider changing > it. > > So what you propose in webrev.04 seems fine to move this along. > > Thanks, > David > >> >> Thanks, >> StefanK >> >>> >>> Cheers, >>> David >>> ----- >>> >>> On 18/05/2018 10:45 PM, Stefan Karlsson wrote: >>>> I got offline suggestions that I should rename the function to >>>> wait_with_safepoint_check: >>>> >>>> http://cr.openjdk.java.net/~stefank/8203341/webrev.03/ >>>> http://cr.openjdk.java.net/~stefank/8203341/webrev.03.delta/ >>>> >>>> Thanks, >>>> StefanK >>>> >>>> On 2018-05-18 14:21, Erik ?sterlund wrote: >>>>> Hi Stefan, >>>>> >>>>> Looks good. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 2018-05-18 14:13, Stefan Karlsson wrote: >>>>>> Updated webrevs: >>>>>> ?http://cr.openjdk.java.net/~stefank/8203341/webrev.02/ >>>>>> ?http://cr.openjdk.java.net/~stefank/8203341/webrev.02.delta/ >>>>>> >>>>>> I removed the SemaphoreSafepointAware class, as Erik requested, >>>>>> but instead of adding a bool I added a new function for the >>>>>> safepoint aware waits. >>>>>> >>>>>> Thanks, >>>>>> StefanK >>>>>> >>>>>> On 2018-05-17 17:59, Stefan Karlsson wrote: >>>>>>> On 2018-05-17 17:38, Erik ?sterlund wrote: >>>>>>>> Hi Stefan, >>>>>>>> >>>>>>>> I wonder if it would make sense to let Semaphore::wait have a >>>>>>>> bool safepoint_check, similar to Mutex (possibly with a default >>>>>>>> value), instead of introducing a new class with the same API, >>>>>>>> that only differs in this property. >>>>>>> >>>>>>> That sounds reasonable to me. >>>>>>> >>>>>>> I also need to change my patch so that the thread-local handshake >>>>>>> code pass in the existing JavaThread, so that we don't have to >>>>>>> call Thread::current()->is_JavaThread() unnecessarily. >>>>>>> >>>>>>> Thanks, >>>>>>> StefanK >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> /Erik >>>>>>>> >>>>>>>> On 2018-05-17 09:46, Stefan Karlsson wrote: >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> Please review this patch to add a safepoint-aware Semaphore >>>>>>>>> (and refactor the semaphore usage in thread-local handshakes). >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~stefank/8203341/webrev.01/ >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8203341 >>>>>>>>> >>>>>>>>> Both ZGC and the thread-local handshakes need to move the >>>>>>>>> JavaThread to a blocked state before waiting on a Semaphore. I >>>>>>>>> propose that we introduce a new SemaphoreSafepointAware wrapper >>>>>>>>> around Semaphore. >>>>>>>>> >>>>>>>>> There are two things to consider with this patch: >>>>>>>>> >>>>>>>>> 1) In ZGC, where this patch originates from, we set the >>>>>>>>> JavaThread's osthread state with >>>>>>>>> osthread->set_state(CONDVAR_WAIT) (using OSThreadWaitState). >>>>>>>>> This means that we get the following printed when the threads >>>>>>>>> are dumped: "waiting on condition ". I've kept this behavior >>>>>>>>> for the SemaphoreSafepointAware class. This means that we now >>>>>>>>> change the osthread state for JavaThreads blocking in >>>>>>>>> HandshakeState::process_self_inner. >>>>>>>>> >>>>>>>>> 2) The thread-local handshakes uses trywait before entering wait: >>>>>>>>> ?? if (!_semaphore.trywait()) { >>>>>>>>> -??? ThreadBlockInVM tbivm(thread); >>>>>>>>> ???? _semaphore.wait(); >>>>>>>>> ?? } >>>>>>>>> >>>>>>>>> At the time when SemaphoreSafepointAware was written, we didn't >>>>>>>>> have trywait on all platforms, now we do. Maybe we should >>>>>>>>> always do the trywait dance inside >>>>>>>>> SemaphoreSafepointAware::wait() ? >>>>>>>>> >>>>>>>>> inline void SemaphoreSafepointAware::wait() { >>>>>>>>> ? if (Thread::current()->is_Java_thread()) { >>>>>>>>> ??? if (!_semaphore.trywait) { >>>>>>>>> ????? JavaThread* thread = JavaThread::current(); >>>>>>>>> >>>>>>>>> ????? // Prepare to block and allow safepoints while blocked >>>>>>>>> ????? ThreadBlockInVM tbivm(thread); >>>>>>>>> ????? OSThreadWaitState osts(thread->osthread(), false /* not >>>>>>>>> in Object.wait() */); >>>>>>>>> >>>>>>>>> ????? // Wait for value >>>>>>>>> ???? _semaphore.wait(); >>>>>>>>> ?? } >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> StefanK >>>>>>>> >>>>>>> >>>>> >> From kim.barrett at oracle.com Tue May 22 08:09:28 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 22 May 2018 10:09:28 +0200 Subject: RFR: 8202813: Move vm_weak processing from SystemDictionary::do_unloading to WeakProcessor In-Reply-To: References: <10a99bf5-d3cc-9d5b-8a38-5ba1672f9f8b@oracle.com> <4E15250A-9872-4195-892B-C1E0078A80FD@oracle.com> <6dc7d3f8-dd9f-0990-860d-3b9423c1e271@oracle.com> Message-ID: > On May 17, 2018, at 9:17 PM, Kim Barrett wrote: > >> On May 17, 2018, at 3:13 AM, Stefan Johansson wrote: >> I agree that baby steps is often good, but now when JFR is in would it make sense to include more in this change? > > OK, I can do that. > >> >> I would like it to be added when needed, to make sure we only include the parts we really need. Right now it would be hard for me to review those parts since I can't verify that they work as intended. I'm mostly concerned about the parts around serial/parallel phases, which is currently unused. > > Since you and (privately) Coleen are both complaining about this, I?ll reconsider. > > Withdrawing this change for now. I've merged with jfr changes and updated per StefanJ's comments. In particular, SystemDictionary::oops_do is gone. This involved changing JFR's leak profiler to call always_strong_oops_do instead, which is what I think it should have been calling all along. However, SystemDictionary::roots_oops_do still conditionally processes SystemDictionary::vm_weak_oop_storage(). Coleen thinks that shouldn't be needed, but I'm not ready to get rid of it yet, and would like to defer that to a later change. No incremental webrev. It was kind of messy with the JFR merge conflict and the parameter change, so didn't seem worthwhile trying to generate. New webrev: http://cr.openjdk.java.net/~kbarrett/8202813/open.01/ From martin.doerr at sap.com Tue May 22 08:25:51 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 22 May 2018 08:25:51 +0000 Subject: RFR(XS) : 8203437 : 8199370 broke build on linux-ppc64le (w/ GCC 4.8.5.) In-Reply-To: References: <31AF805E-B9BE-4BCC-BE81-6111216B2D93@oracle.com> <9957f149-ac01-a069-cdc9-ac70c73b70eb@redhat.com> Message-ID: <886e70e3d322492d88c54783147ba0ce@sap.com> Hi Igor, sorry that I couldn't reply earlier. Thank you for fixing this issue. Best regards, Martin -----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Igor Ignatyev Sent: Freitag, 18. Mai 2018 21:38 To: Aleksey Shipilev Cc: hotspot-dev Source Developers Subject: Re: RFR(XS) : 8203437 : 8199370 broke build on linux-ppc64le (w/ GCC 4.8.5.) > On May 18, 2018, at 11:48 AM, Aleksey Shipilev wrote: > > On 05/18/2018 08:38 PM, Igor Ignatyev wrote: >> JBS: https://bugs.openjdk.java.net/browse/JDK-8203437 >> webrev: http://cr.openjdk.java.net/~iignatyev//8203437/webrev.00 > > Looks OK. Not immediately evident why only those two fields require initialization. > >> testing: 'make test-image' w/ gcc4.8.2. > > Interestingly, my ppc64le builds are fine without this fix. Do I need to add "test-image" to CI > scripts to compile the tests too? I've double-checked 'make images' doesn't build test image, actually it's just an alias for 'product-images', so yes 'test-image' is needed to compile native part of tests. alternatively, you can use 'all-images' target which builds product, test and doc images. -- Igor > > -Aleksey > From david.holmes at oracle.com Tue May 22 09:04:52 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 22 May 2018 19:04:52 +1000 Subject: RFR (trivial) 8203626: ProblemList compiler/runtime/TestFloatsOnStackDeopt.java Message-ID: This test is causing too many tier 1 and 2 failures. Bug: https://bugs.openjdk.java.net/browse/JDK-8203626 webrev: http://cr.openjdk.java.net/~dholmes/8203626/webrev/ Thanks, David From tobias.hartmann at oracle.com Tue May 22 09:09:27 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Tue, 22 May 2018 11:09:27 +0200 Subject: RFR (trivial) 8203626: ProblemList compiler/runtime/TestFloatsOnStackDeopt.java In-Reply-To: References: Message-ID: <4c298a0a-1a38-6d3d-7bf7-9f09cb01547b@oracle.com> Reviewed. Thanks, Tobias On 22.05.2018 11:04, David Holmes wrote: > This test is causing too many tier 1 and 2 failures. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203626 > webrev: http://cr.openjdk.java.net/~dholmes/8203626/webrev/ > > Thanks, > David From david.holmes at oracle.com Tue May 22 09:20:17 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 22 May 2018 19:20:17 +1000 Subject: RFR (trivial) 8203626: ProblemList compiler/runtime/TestFloatsOnStackDeopt.java In-Reply-To: <4c298a0a-1a38-6d3d-7bf7-9f09cb01547b@oracle.com> References: <4c298a0a-1a38-6d3d-7bf7-9f09cb01547b@oracle.com> Message-ID: Thanks Tobias. David On 22/05/2018 7:09 PM, Tobias Hartmann wrote: > Reviewed. > > Thanks, > Tobias > > On 22.05.2018 11:04, David Holmes wrote: >> This test is causing too many tier 1 and 2 failures. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203626 >> webrev: http://cr.openjdk.java.net/~dholmes/8203626/webrev/ >> >> Thanks, >> David From martin.doerr at sap.com Tue May 22 10:16:59 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 22 May 2018 10:16:59 +0000 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: References: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> Message-ID: <7e7fc484287e4da4926176e0b5ae1b64@sap.com> Hi Kim, I can't see how a new implicit consume is introduced by Michihiro's change. He just explained how the existing code works. If implicit consume has been rejected the current code is wrong: "new_obj = o->forwardee();" would need some kind of barrier before using the new_obj. But this issue is not related to what Michihiro wants to change AFAICS. Best regards, Martin -----Original Message----- From: ppc-aix-port-dev [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] On Behalf Of Kim Barrett Sent: Montag, 21. Mai 2018 06:00 To: Michihiro Horie Cc: hotspot-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; Gustavo Bueno Romero ; ppc-aix-port-dev at openjdk.java.net; david.holmes at oracle.com Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 > On May 18, 2018, at 5:12 PM, Michihiro Horie wrote: > > Dear all, > > I update the webrev: http://cr.openjdk.java.net/~mhorie/8154736/webrev.09/ > > With the release barrier before the CAS, new_obj can be observed from other threads. If the CAS succeeds, the current thread can use new_obj without barriers. If the CAS fails, "o->forwardee()" is ordered with respect to CAS by accessing the same memory location "_mark", so no barriers needed. The order of (1) access to the forwardee and (2) access to forwardee's fields is preserved due to Release-Consume ordering on supported platforms. (The ordering between "new_obj = o->forwardee();" and logging or other usages is not changed.) > > Regarding the maintainability, the requirement is CAS(memory_order_release) as Release-Consume to be consistent with C++11. This requirement is necessary when a weaker platform like DEC Alpha is to be supported. On currently supported platforms, code change can be safe if the code meats this requirement (and the order of (1) access to the forwardee and (2) access to forwardee's fields is the natural way of coding). Relying on implicit consume has been been discussed and rejected, in the earlier thread on this topic and I think elsewhere too. http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-October/021538.html From shade at redhat.com Tue May 22 15:35:19 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 22 May 2018 17:35:19 +0200 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: Message-ID: On 05/15/2018 02:52 AM, David Holmes wrote: > This review is being spread across four groups: langtools, core-libs, hotspot and serviceability. > This is the specific review thread for hotspot - webrev: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ I have not looked through the code, but built current valhalla/valhalla @ 41799b26a947 (nestmates) with x86_32 and aarch64, and ran these tests fastdebug builds on both platforms: test/hotspot/jtreg/runtime/Nestmates/ x86_32: OK aarch64: OK test/hotspot/jtreg/runtime/SelectionResolution/ x86_32: OK aarch64: OK test/jdk/java/lang/instrument/RedefineNestmateAttr/ x86_32: OK aarch64: OK test/jdk/java/lang/invoke/ x86_32: OK-ish -- these failures seem to manifest without nestmates too java/lang/invoke/condy/CondyReturnPrimitiveTest.java Failed. Unexpected exit from test [exit code: 134] java/lang/invoke/condy/CondyWrongType.java Failed. Execution failed: `main' threw exception: java.lang.Exception: failures: 5 java/lang/invoke/LFCaching/LFGarbageCollectedTest.java Error. Parse Exception: `@library' must appear before first action tag aarch64: FAILED java/lang/invoke/SpecialInterfaceCall.java http://cr.openjdk.java.net/~shade/nestmates/SpecialInterfaceCall-hs_err.log I have not dug deeper yet, AArch64 maintainers are welcome to step in! Thanks, -Aleksey P.S. It would be convenient to have a way to run all the tests that the particular feature had affected. Probably adding @bug with parent JBS ID would be a step towards this? From adinn at redhat.com Tue May 22 16:01:02 2018 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 22 May 2018 17:01:02 +0100 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: Message-ID: <0cf68c20-51d3-6754-8837-698d120ebf53@redhat.com> On 22/05/18 16:35, Aleksey Shipilev wrote: > I have not looked through the code, but built current valhalla/valhalla @ 41799b26a947 (nestmates) > with x86_32 and aarch64, and ran these tests fastdebug builds on both platforms: > > test/hotspot/jtreg/runtime/Nestmates/ > x86_32: OK > aarch64: OK > > test/hotspot/jtreg/runtime/SelectionResolution/ > x86_32: OK > aarch64: OK > > test/jdk/java/lang/instrument/RedefineNestmateAttr/ > x86_32: OK > aarch64: OK > > test/jdk/java/lang/invoke/ > x86_32: OK-ish -- these failures seem to manifest without nestmates too > java/lang/invoke/condy/CondyReturnPrimitiveTest.java > > Failed. Unexpected exit from test [exit code: 134] > java/lang/invoke/condy/CondyWrongType.java > Failed. Execution failed: `main' threw exception: java.lang.Exception: failures: 5 > java/lang/invoke/LFCaching/LFGarbageCollectedTest.java > > Error. Parse Exception: `@library' must appear before first action tag > > aarch64: FAILED > java/lang/invoke/SpecialInterfaceCall.java > http://cr.openjdk.java.net/~shade/nestmates/SpecialInterfaceCall-hs_err.log > > I have not dug deeper yet, AArch64 maintainers are welcome to step in! I'm not sure what is going on here. A few days ago I patched the jdk/jdk dev tree with David's megapatch, built on AArch64 and ran tier1 tests, obtaining results before and after patching. There were some tier1 test failures which did not seem to relate to the NestMates patch. All three of CondyReturnPrimitiveTest, CondyWrongType and SpecialInterfaceCall passed before and after. I will check to see the patch has been applied correctly and rerun these tests. Aleksey, perhaps you could try again using a patched jdkdev? regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From mikhailo.seledtsov at oracle.com Tue May 22 16:23:27 2018 From: mikhailo.seledtsov at oracle.com (mikhailo) Date: Tue, 22 May 2018 09:23:27 -0700 Subject: RFR: 8199064: Test applications/jcstress/other/Test.java#id1108 fails on Sparc In-Reply-To: <4A1B218C-66B9-433A-9BA9-BA495CDF4EA9@oracle.com> References: <4A1B218C-66B9-433A-9BA9-BA495CDF4EA9@oracle.com> Message-ID: Looks good to me, Misha On 05/16/2018 03:14 PM, Leonid Mesnik wrote: > Hi > > Could you please review following patch which increase timeout for jcstress test wrappers in jtreg. > > The default jtreg timeout is not enough for a lot of jcstress tests. The longest tests take more then 1800 seconds on typical VM used for testing (2 CPU/16GB of Memory) with default VM options. > These tests are not executed in tier1/2 testing so execution time in the case if test really hang is not very critical. I just set timeout to 2400 seconds for all test wrappers. > The timeoutfactor could be used to adjust timeouts for slow platforms and/or VM options. > > > webrev: http://cr.openjdk.java.net/~lmesnik/8199064/webrev.00/ > bug: https://bugs.openjdk.java.net/browse/JDK-8199064 > > Leonid From shade at redhat.com Tue May 22 16:35:40 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 22 May 2018 18:35:40 +0200 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <0cf68c20-51d3-6754-8837-698d120ebf53@redhat.com> References: <0cf68c20-51d3-6754-8837-698d120ebf53@redhat.com> Message-ID: <759bf969-1aed-852b-1e9d-5d96178086f9@redhat.com> On 05/22/2018 06:01 PM, Andrew Dinn wrote: >> aarch64: FAILED >> java/lang/invoke/SpecialInterfaceCall.java >> http://cr.openjdk.java.net/~shade/nestmates/SpecialInterfaceCall-hs_err.log >> >> I have not dug deeper yet, AArch64 maintainers are welcome to step in! > I'm not sure what is going on here. > > A few days ago I patched the jdk/jdk dev tree with David's megapatch, > built on AArch64 and ran tier1 tests, obtaining results before and after > patching. There were some tier1 test failures which did not seem to > relate to the NestMates patch. All three of CondyReturnPrimitiveTest, > CondyWrongType and SpecialInterfaceCall passed before and after. > > I will check to see the patch has been applied correctly and rerun these > tests. Aleksey, perhaps you could try again using a patched jdkdev? Applied David's webrev.hotspot.v1 over current jdk/jdk, and SpecialInterfaceCall fails the same way with linux-aarch64-fastdebug on my Raspberry Pi 3. -Aleksey From zgu at redhat.com Tue May 22 17:40:12 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 22 May 2018 13:40:12 -0400 Subject: RFR(XXS) 8203635: JFR sampler thread does not record stack info Message-ID: Please review this trivial change that added record_stack_base_and_size() call to JFR sampler thread, so NMT can count it correctly. Bug: https://bugs.openjdk.java.net/browse/JDK-8203635 Webrev: http://cr.openjdk.java.net/~zgu/8203635/webrev.00/ Test: tier1_runtime on Linux 64 Thanks, -Zhengyu From shade at redhat.com Tue May 22 17:45:02 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Tue, 22 May 2018 19:45:02 +0200 Subject: RFR(XXS) 8203635: JFR sampler thread does not record stack info In-Reply-To: References: Message-ID: <72cfcdcf-2545-26e5-a513-d91f375905b4@redhat.com> On 05/22/2018 07:40 PM, Zhengyu Gu wrote: > Please review this trivial change that added record_stack_base_and_size() call to JFR sampler > thread, so NMT can count it correctly. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203635 > Webrev: http://cr.openjdk.java.net/~zgu/8203635/webrev.00/ Looks good to me. I wish this was handled in superclass initialization, so no one would miss this. -Aleksey From leonid.mesnik at oracle.com Tue May 22 17:52:39 2018 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Tue, 22 May 2018 10:52:39 -0700 Subject: RFR: 8199064: Test applications/jcstress/other/Test.java#id1108 fails on Sparc In-Reply-To: References: <4A1B218C-66B9-433A-9BA9-BA495CDF4EA9@oracle.com> Message-ID: Thank you for review. Leonid > On May 22, 2018, at 9:23 AM, mikhailo wrote: > > Looks good to me, > > > Misha > > > On 05/16/2018 03:14 PM, Leonid Mesnik wrote: >> Hi >> >> Could you please review following patch which increase timeout for jcstress test wrappers in jtreg. >> >> The default jtreg timeout is not enough for a lot of jcstress tests. The longest tests take more then 1800 seconds on typical VM used for testing (2 CPU/16GB of Memory) with default VM options. >> These tests are not executed in tier1/2 testing so execution time in the case if test really hang is not very critical. I just set timeout to 2400 seconds for all test wrappers. >> The timeoutfactor could be used to adjust timeouts for slow platforms and/or VM options. >> >> >> webrev: http://cr.openjdk.java.net/~lmesnik/8199064/webrev.00/ >> bug: https://bugs.openjdk.java.net/browse/JDK-8199064 >> >> Leonid > From zgu at redhat.com Tue May 22 17:57:53 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 22 May 2018 13:57:53 -0400 Subject: RFR(XXS) 8203635: JFR sampler thread does not record stack info In-Reply-To: <72cfcdcf-2545-26e5-a513-d91f375905b4@redhat.com> References: <72cfcdcf-2545-26e5-a513-d91f375905b4@redhat.com> Message-ID: Thanks for the review, Aleksey. On 05/22/2018 01:45 PM, Aleksey Shipilev wrote: > On 05/22/2018 07:40 PM, Zhengyu Gu wrote: >> Please review this trivial change that added record_stack_base_and_size() call to JFR sampler >> thread, so NMT can count it correctly. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203635 >> Webrev: http://cr.openjdk.java.net/~zgu/8203635/webrev.00/ > > Looks good to me. I wish this was handled in superclass initialization, so no one would miss this. Agree! However, it is a bit tricky, as the call has to be made from *current* thread. Thanks, -Zhengyu > > -Aleksey > From coleen.phillimore at oracle.com Tue May 22 18:12:40 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 22 May 2018 14:12:40 -0400 Subject: RFR(XXS) 8203635: JFR sampler thread does not record stack info In-Reply-To: References: Message-ID: <8f2fae61-00e1-8272-5f2d-ec51eed554c3@oracle.com> This looks good (and can count as trivial I think). Coleen On 5/22/18 1:40 PM, Zhengyu Gu wrote: > Please review this trivial change that added > record_stack_base_and_size() call to JFR sampler thread, so NMT can > count it correctly. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203635 > Webrev: http://cr.openjdk.java.net/~zgu/8203635/webrev.00/ > > Test: > ? tier1_runtime on Linux 64 > > Thanks, > > -Zhengyu > > > From thomas.stuefe at gmail.com Tue May 22 18:16:07 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 22 May 2018 20:16:07 +0200 Subject: RFR(XXS) 8203635: JFR sampler thread does not record stack info In-Reply-To: References: <72cfcdcf-2545-26e5-a513-d91f375905b4@redhat.com> Message-ID: Hi Zhengyu, looks good. I wish it would be like this: a non-virtual Thread::run() which does common initialization, and then calls a virtual protected abstract Thread::run_impl(), which is implemented by child classes... Regards, Thomas On Tue, May 22, 2018 at 7:57 PM, Zhengyu Gu wrote: > Thanks for the review, Aleksey. > > On 05/22/2018 01:45 PM, Aleksey Shipilev wrote: >> >> On 05/22/2018 07:40 PM, Zhengyu Gu wrote: >>> >>> Please review this trivial change that added record_stack_base_and_size() >>> call to JFR sampler >>> thread, so NMT can count it correctly. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203635 >>> Webrev: http://cr.openjdk.java.net/~zgu/8203635/webrev.00/ >> >> >> Looks good to me. I wish this was handled in superclass initialization, so >> no one would miss this. > > > Agree! However, it is a bit tricky, as the call has to be made from > *current* thread. > > Thanks, > > -Zhengyu > >> >> -Aleksey >> > From zgu at redhat.com Tue May 22 18:27:03 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 22 May 2018 14:27:03 -0400 Subject: RFR(XXS) 8203635: JFR sampler thread does not record stack info In-Reply-To: <8f2fae61-00e1-8272-5f2d-ec51eed554c3@oracle.com> References: <8f2fae61-00e1-8272-5f2d-ec51eed554c3@oracle.com> Message-ID: <6dcd28b0-0063-b7d2-db7e-9b7058a0aa58@redhat.com> Thanks, Coleen. -Zhengyu On 05/22/2018 02:12 PM, coleen.phillimore at oracle.com wrote: > This looks good (and can count as trivial I think). > Coleen > > On 5/22/18 1:40 PM, Zhengyu Gu wrote: >> Please review this trivial change that added >> record_stack_base_and_size() call to JFR sampler thread, so NMT can >> count it correctly. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203635 >> Webrev: http://cr.openjdk.java.net/~zgu/8203635/webrev.00/ >> >> Test: >> ? tier1_runtime on Linux 64 >> >> Thanks, >> >> -Zhengyu >> >> >> > From kim.barrett at oracle.com Tue May 22 18:27:26 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 22 May 2018 20:27:26 +0200 Subject: RFR: 8202813: Move vm_weak processing from SystemDictionary::do_unloading to WeakProcessor In-Reply-To: References: <10a99bf5-d3cc-9d5b-8a38-5ba1672f9f8b@oracle.com> <4E15250A-9872-4195-892B-C1E0078A80FD@oracle.com> <6dc7d3f8-dd9f-0990-860d-3b9423c1e271@oracle.com> Message-ID: > On May 22, 2018, at 10:09 AM, Kim Barrett wrote: > New webrev: > http://cr.openjdk.java.net/~kbarrett/8202813/open.01/ I forgot to remove the no longer used is_alive argument for SystemDictionary::do_unloading. Updated webrevs: full: http://cr.openjdk.java.net/~kbarrett/8202813/open.02/ incr: http://cr.openjdk.java.net/~kbarrett/8202813/open.02.inc/ From zgu at redhat.com Tue May 22 18:39:32 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Tue, 22 May 2018 14:39:32 -0400 Subject: RFR(XXS) 8203635: JFR sampler thread does not record stack info In-Reply-To: References: <72cfcdcf-2545-26e5-a513-d91f375905b4@redhat.com> Message-ID: <612bcdad-f37d-14fa-de85-38adc8f408a2@redhat.com> Thanks, Thomas. On 05/22/2018 02:16 PM, Thomas St?fe wrote: > Hi Zhengyu, > > looks good. > > I wish it would be like this: > > a non-virtual Thread::run() which does common initialization, and then > calls a virtual protected abstract Thread::run_impl(), which is > implemented by child classes... Most of subclasses of Thread, e.g. JavaThread, WorkerThread and etc. handle in similar way. I am curious why this Sampler thread can not subclass one of them? -Zhengyu > > Regards, Thomas > > > > On Tue, May 22, 2018 at 7:57 PM, Zhengyu Gu wrote: >> Thanks for the review, Aleksey. >> >> On 05/22/2018 01:45 PM, Aleksey Shipilev wrote: >>> >>> On 05/22/2018 07:40 PM, Zhengyu Gu wrote: >>>> >>>> Please review this trivial change that added record_stack_base_and_size() >>>> call to JFR sampler >>>> thread, so NMT can count it correctly. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203635 >>>> Webrev: http://cr.openjdk.java.net/~zgu/8203635/webrev.00/ >>> >>> >>> Looks good to me. I wish this was handled in superclass initialization, so >>> no one would miss this. >> >> >> Agree! However, it is a bit tricky, as the call has to be made from >> *current* thread. >> >> Thanks, >> >> -Zhengyu >> >>> >>> -Aleksey >>> >> From david.holmes at oracle.com Tue May 22 21:44:22 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 23 May 2018 07:44:22 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <759bf969-1aed-852b-1e9d-5d96178086f9@redhat.com> References: <0cf68c20-51d3-6754-8837-698d120ebf53@redhat.com> <759bf969-1aed-852b-1e9d-5d96178086f9@redhat.com> Message-ID: Hi Aleksey, On 23/05/2018 2:35 AM, Aleksey Shipilev wrote: > On 05/22/2018 06:01 PM, Andrew Dinn wrote: >>> aarch64: FAILED >>> java/lang/invoke/SpecialInterfaceCall.java >>> http://cr.openjdk.java.net/~shade/nestmates/SpecialInterfaceCall-hs_err.log >>> >>> I have not dug deeper yet, AArch64 maintainers are welcome to step in! >> I'm not sure what is going on here. >> >> A few days ago I patched the jdk/jdk dev tree with David's megapatch, >> built on AArch64 and ran tier1 tests, obtaining results before and after >> patching. There were some tier1 test failures which did not seem to >> relate to the NestMates patch. All three of CondyReturnPrimitiveTest, >> CondyWrongType and SpecialInterfaceCall passed before and after. >> >> I will check to see the patch has been applied correctly and rerun these >> tests. Aleksey, perhaps you could try again using a patched jdkdev? > > Applied David's webrev.hotspot.v1 over current jdk/jdk, and SpecialInterfaceCall fails the same way > with linux-aarch64-fastdebug on my Raspberry Pi 3. You can't just apply the hotspot patch! You need the entire thing. David > -Aleksey > From shade at redhat.com Tue May 22 22:50:26 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 23 May 2018 00:50:26 +0200 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <0cf68c20-51d3-6754-8837-698d120ebf53@redhat.com> <759bf969-1aed-852b-1e9d-5d96178086f9@redhat.com> Message-ID: On 05/22/2018 11:44 PM, David Holmes wrote: > On 23/05/2018 2:35 AM, Aleksey Shipilev wrote: >> On 05/22/2018 06:01 PM, Andrew Dinn wrote: >>>> ???? aarch64: FAILED >>>> ??????? java/lang/invoke/SpecialInterfaceCall.java >>>> ?????????? http://cr.openjdk.java.net/~shade/nestmates/SpecialInterfaceCall-hs_err.log >>>> >>>> I have not dug deeper yet, AArch64 maintainers are welcome to step in! >>> I'm not sure what is going on here. >>> >>> A few days ago I patched the jdk/jdk dev tree with David's megapatch, >>> built on AArch64 and ran tier1 tests, obtaining results before and after >>> patching. There were some tier1 test failures which did not seem to >>> relate to the NestMates patch. All three of CondyReturnPrimitiveTest, >>> CondyWrongType and SpecialInterfaceCall passed before and after. >>> >>> I will check to see the patch has been applied correctly and rerun these >>> tests. Aleksey, perhaps you could try again using a patched jdkdev? >> >> Applied David's webrev.hotspot.v1 over current jdk/jdk, and SpecialInterfaceCall fails the same way >> with linux-aarch64-fastdebug on my Raspberry Pi 3. > > You can't just apply the hotspot patch! You need the entire thing. Duh, right. In this case, we were "saved" by the fact the failing test seems to explore hotspot side of things only, and it fails the same way the nestmates branch in valhalla repo. And anyhow, applying the full patch fails the same way :) -Aleksey From igor.ignatyev at oracle.com Tue May 22 23:35:09 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 22 May 2018 16:35:09 -0700 Subject: RFR(XL) : 8199383 : [TESTBUG] Open source VM testbase JVMTI tests Message-ID: http://cr.openjdk.java.net/~iignatyev//8199383/webrev.00/index.html > 308253 lines changed: 308253 ins; 0 del; 0 mod; Hi all, could you please review this patch which open sources JVMTI tests from VM testbase? As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. webrev: http://cr.openjdk.java.net/~iignatyev//8199383/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8199383 testing: :vmTestbase_nsk_jvmti test group Thanks, -- Igor From erik.joelsson at oracle.com Wed May 23 00:05:17 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 22 May 2018 17:05:17 -0700 Subject: RFR(XL) : 8199383 : [TESTBUG] Open source VM testbase JVMTI tests In-Reply-To: References: Message-ID: <8bfed6c4-d0e0-4324-5982-d4ef040d12c1@oracle.com> Looks like the line "# } nsk/jvmti" is a left over. Otherwise this looks ok, even if it's an enormous amount of duplication. Hopefully we can figure out a better way to express common parameters for tests soon. /Erik On 2018-05-22 16:35, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8199383/webrev.00/index.html >> 308253 lines changed: 308253 ins; 0 del; 0 mod; > Hi all, > > could you please review this patch which open sources JVMTI tests from VM testbase? > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > webrev: http://cr.openjdk.java.net/~iignatyev//8199383/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8199383 > testing: :vmTestbase_nsk_jvmti test group > > Thanks, > -- Igor From david.holmes at oracle.com Wed May 23 02:31:24 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 23 May 2018 12:31:24 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <0cf68c20-51d3-6754-8837-698d120ebf53@redhat.com> <759bf969-1aed-852b-1e9d-5d96178086f9@redhat.com> Message-ID: On 23/05/2018 8:50 AM, Aleksey Shipilev wrote: > On 05/22/2018 11:44 PM, David Holmes wrote: >> On 23/05/2018 2:35 AM, Aleksey Shipilev wrote: >>> On 05/22/2018 06:01 PM, Andrew Dinn wrote: >>>>> ???? aarch64: FAILED >>>>> ??????? java/lang/invoke/SpecialInterfaceCall.java >>>>> ?????????? http://cr.openjdk.java.net/~shade/nestmates/SpecialInterfaceCall-hs_err.log >>>>> >>>>> I have not dug deeper yet, AArch64 maintainers are welcome to step in! >>>> I'm not sure what is going on here. >>>> >>>> A few days ago I patched the jdk/jdk dev tree with David's megapatch, >>>> built on AArch64 and ran tier1 tests, obtaining results before and after >>>> patching. There were some tier1 test failures which did not seem to >>>> relate to the NestMates patch. All three of CondyReturnPrimitiveTest, >>>> CondyWrongType and SpecialInterfaceCall passed before and after. >>>> >>>> I will check to see the patch has been applied correctly and rerun these >>>> tests. Aleksey, perhaps you could try again using a patched jdkdev? >>> >>> Applied David's webrev.hotspot.v1 over current jdk/jdk, and SpecialInterfaceCall fails the same way >>> with linux-aarch64-fastdebug on my Raspberry Pi 3. >> >> You can't just apply the hotspot patch! You need the entire thing. > > Duh, right. In this case, we were "saved" by the fact the failing test seems to explore hotspot side > of things only, and it fails the same way the nestmates branch in valhalla repo. And anyhow, > applying the full patch fails the same way :) SpecialInterfaceCall shouldn't be failing! Can you confirm it only fails with nestmates patches applied? (It's not a nestmate test and should** be unaffected by nestmate changes.) ** There is a slight difference that shouldn't affect anything - and certainly shouldn't be platform specific! Thanks, David > -Aleksey > From david.holmes at oracle.com Wed May 23 04:07:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 23 May 2018 14:07:01 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <8fe8a029-04f8-415a-efc6-3adef875255c@oracle.com> References: <8fe8a029-04f8-415a-efc6-3adef875255c@oracle.com> Message-ID: Hi Coleen, Thanks for looking at this! On 19/05/2018 8:36 AM, coleen.phillimore at oracle.com wrote: > > Hi, I haven't been following along or read through all of this, but I > had a look through the code and have some questions about it. > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/src/hotspot/share/prims/jvmtiRedefineClasses.cpp.udiff.html > > > There is commented out code at line 806 in > compare_and_normalize_class_versions.?? Could you make the added code a > separate function defined above so this isn't the longest function in > the jvm? > > 739 JvmtiThreadState *state = > JvmtiThreadState::state_for((JavaThread*)thread); > 740 RedefineVerifyMark rvm(the_class, scratch_class, state); > > Why does this need RedefineVerifyMark?? It doesn't call the verifier. The above was moved to and answered on the serviceability-dev review thread. > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/src/hotspot/share/runtime/reflection.cpp.frames.html > > > 720 bool access = cur_ik->has_nestmate_access_to(field_ik, THREAD); > 721 if (HAS_PENDING_EXCEPTION) { > 722 return false; > 723 } > > This looks suspicious.?? Do you want to CLEAR_PENDING_EXCEPTION? This > looks like the combination of these functions drops exceptions, that > could show up in inconvenient places. There was a lot of changes to the way exceptions were handled as this feature developed. The resulting code may well need some clean up ... The exceptions have to propagate so I just updated with: bool access = cur_ik->has_nestmate_access_to(field_ik, CHECK_false); > Or add TRAPS to > reflection::verify_field_access so that the callers properly handle the > pending exception? As Reflection::verify_field_access is a boolean function it gets used in "if" expressions that make it unsuitable for use with TRAPS and CHECK macros. I can either rewrite all those if's to introduce a local variable assignment that is checked in the if, or else just do the pending exception check explicitly. I chose the latter for simplicity - and in one case (ciField) we need to clear the exception as well so CHECK_ doesn't apply. > This is CHECK_false. and also maybe has_nestmate_access_to, should pass > CHECK_false to its nest_host() calls, so it doesn't look so weird. So we currently have, for example: InstanceKlass* cur_host = nest_host(icce, THREAD); if (cur_host == NULL || HAS_PENDING_EXCEPTION) { return false; } and you're suggesting: InstanceKlass* cur_host = nest_host(icce, CHECK_false); if (cur_host == NULL) { return false; } I'm not really seeing a great difference in readability. Code-wise I think the current code is slightly more efficient, but I'm not sure about that. > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/src/hotspot/share/oops/instanceKlass.cpp.udiff.html > > > + // names match so check actual klass - this may trigger class loading if > + // it doesn't match (but that should be impossible) > + Klass* k2 = _constants->klass_at(cp_index, CHECK_false); > > > If throwing an exception is impossible, this should probably have a > CATCH.? This is code is odd about exceptions.? It looks like > has_nestmate_access() I _think_ it is impossible - it's a very odd situation and I could not see how to construct a test that would hit this. But I don't know for sure that it is impossible. So the code assumes classloading and thus exceptions are possible. > > + } > + else { > > > Oh please can you fix these?? The hotspot style is the? } else { all on > one line (OTBS). Okay - 339 cases in hotspot as above versus 12913 as style guide suggests :) I will fix my additions in instanceKlass.cpp. > Since has_nest_member is only used in InstanceKlass, it should have > private access. > > +? bool has_nest_member(InstanceKlass* k, TRAPS) const; True it could ... but I really prefer to have the nest functions grouped together, and switching back and forth between private and public would look messy. > Looks like raw_nest_host() is uncalled (and shouldn't be public). It was a public debugging hook. Now removed again. > That's all I can do for now.? I think you have good reviewers for the > feature itself, but if you want more, I'll spend more time with this. Thanks again! David > thanks, > Coleen > > On 5/14/18 8:52 PM, David Holmes wrote: >> This review is being spread across four groups: langtools, core-libs, >> hotspot and serviceability. This is the specific review thread for >> hotspot - webrev: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >> >> See below for full details - including annotated full webrev guiding >> the review. >> >> The intent is to have JEP-181 targeted and integrated by the end of >> this month. >> >> Thanks, >> David >> ----- >> >> The nestmates project (JEP-181) introduces new classfile attributes to >> identify classes and interfaces in the same nest, so that the VM can >> perform access control based on those attributes and so allow direct >> private access between nestmates without requiring javac to generate >> synthetic accessor methods. These access control changes also extend >> to core reflection and the MethodHandle.Lookup contexts. >> >> Direct private calls between nestmates requires a more general calling >> context than is permitted by invokespecial, and so the JVMS is updated >> to allow, and javac updated to use, invokevirtual and invokeinterface >> for private class and interface method calls respectively. These >> changed semantics also extend to MethodHandle findXXX operations. >> >> At this time we are only concerned with static nest definitions, which >> map to a top-level class/interface as the nest-host and all its nested >> types as nest-members. >> >> Please see the JEP for further details. >> >> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >> >> All of the specification changes have been previously been worked out >> by the Valhalla Project Expert Group, and the implementation reviewed >> by the various contributors and discussed on the valhalla-dev mailing >> list. >> >> Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, >> Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir >> Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan >> >> Master webrev of all changes: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >> >> Annotated master webrev index: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >> >> Performance: this is expected to be performance neutral in a general >> sense. Benchmarking and performance runs are about to start. >> >> Testing Discussion: >> ------------------ >> >> The testing for nestmates can be broken into four main groups: >> >> -? New tests specifically related to nestmates and currently in the >> runtime/Nestmates directory >> >> - New tests to complement existing tests by adding in testcases not >> previously expressible. >> ? -? For example java/lang/invoke/SpecialInterfaceCall.java tests use >> of invokespecial for private interface methods and performing receiver >> typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do >> similar tests for invokeinterface. >> >> -? New JVM TI tests to verify the spec changes related to nest >> attributes. >> >> -? Existing tests significantly affected by the nestmates changes, >> primarily: >> ?? -? runtime/SelectionResolution >> >> ?? In most cases the nestmate changes makes certain invocations that >> were illegal, legal (e.g. not requiring invokespecial to invoke >> private interface methods; allowing access to private members via >> reflection/Methodhandles that were previously not allowed). >> >> - Existing tests incidentally affected by the nestmate changes >> >> ? This includes tests of things utilising class >> redefinition/retransformation to alter nested types but which >> unintentionally alter nest relationships (which is not permitted). >> >> There are still a number of tests problem-listed with issues filed >> against them to have them adapted to work with nestmates. Some of >> these are intended to be addressed in the short-term, while some (such >> as the runtime/SelectionResolution test changes) may not eventuate. >> >> - https://bugs.openjdk.java.net/browse/JDK-8203033 >> - https://bugs.openjdk.java.net/browse/JDK-8199450 >> - https://bugs.openjdk.java.net/browse/JDK-8196855 >> - https://bugs.openjdk.java.net/browse/JDK-8194857 >> - https://bugs.openjdk.java.net/browse/JDK-8187655 >> >> There is also further test work still to be completed (the JNI and JDI >> invocation tests): >> - https://bugs.openjdk.java.net/browse/JDK-8191117 >> which will continue in parallel with the main RFR. >> >> Pre-integration Testing: >> ?- General: >> ??? - Mach5: hs/jdk tier1,2 >> ??? - Mach5: hs-nightly (tiers 1 -3) >> ?- Targetted >> ?? - nashorn (for asm changes) >> ?? - hotspot: runtime/* >> ????????????? serviceability/* >> ????????????? compiler/* >> ????????????? vmTestbase/* >> ?? - jdk: java/lang/invoke/* >> ????????? java/lang/reflect/* >> ????????? java/lang/instrument/* >> ????????? java/lang/Class/* >> ????????? java/lang/management/* >> ? - langtools: tools/javac >> ?????????????? tools/javap >> > From david.holmes at oracle.com Wed May 23 06:57:18 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 23 May 2018 16:57:18 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: Message-ID: <06529fc3-2eba-101b-9aee-2757893cb8fb@oracle.com> Here are the updates so far in response to all the review comments. Incremental webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2-incr/ Full webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2/ Change summary: test/runtime/Nestmates/reflectionAPI/* - moved to java/lang/reflect/Nestmates src/hotspot/cpu/arm/templateTable_arm.cpp - Fixed ARM invocation logic as provided by Boris. src/hotspot/share/interpreter/linkResolver.cpp - expanded comment regarding exceptions - Removed leftover debugging code src/hotspot/share/oops/instanceKlass.cpp - Removed FIXME comments - corrected incorrect comment - Fixed if/else formatting src/hotspot/share/oops/instanceKlass.hpp - removed unused debug method src/hotspot/share/oops/klassVtable.cpp - added comment by request of Karen src/hotspot/share/runtime/reflection.cpp - Removed FIXME comments - expanded comments in places - used CHECK_false Thanks, David On 15/05/2018 10:52 AM, David Holmes wrote: > This review is being spread across four groups: langtools, core-libs, > hotspot and serviceability. This is the specific review thread for > hotspot - webrev: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ > > See below for full details - including annotated full webrev guiding the > review. > > The intent is to have JEP-181 targeted and integrated by the end of this > month. > > Thanks, > David > ----- > > The nestmates project (JEP-181) introduces new classfile attributes to > identify classes and interfaces in the same nest, so that the VM can > perform access control based on those attributes and so allow direct > private access between nestmates without requiring javac to generate > synthetic accessor methods. These access control changes also extend to > core reflection and the MethodHandle.Lookup contexts. > > Direct private calls between nestmates requires a more general calling > context than is permitted by invokespecial, and so the JVMS is updated > to allow, and javac updated to use, invokevirtual and invokeinterface > for private class and interface method calls respectively. These changed > semantics also extend to MethodHandle findXXX operations. > > At this time we are only concerned with static nest definitions, which > map to a top-level class/interface as the nest-host and all its nested > types as nest-members. > > Please see the JEP for further details. > > JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 > Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 > CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 > > All of the specification changes have been previously been worked out by > the Valhalla Project Expert Group, and the implementation reviewed by > the various contributors and discussed on the valhalla-dev mailing list. > > Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, > Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir > Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan > > Master webrev of all changes: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ > > Annotated master webrev index: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html > > Performance: this is expected to be performance neutral in a general > sense. Benchmarking and performance runs are about to start. > > Testing Discussion: > ------------------ > > The testing for nestmates can be broken into four main groups: > > -? New tests specifically related to nestmates and currently in the > runtime/Nestmates directory > > - New tests to complement existing tests by adding in testcases not > previously expressible. > ? -? For example java/lang/invoke/SpecialInterfaceCall.java tests use > of invokespecial for private interface methods and performing receiver > typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do > similar tests for invokeinterface. > > -? New JVM TI tests to verify the spec changes related to nest attributes. > > -? Existing tests significantly affected by the nestmates changes, > primarily: > ?? -? runtime/SelectionResolution > > ?? In most cases the nestmate changes makes certain invocations that > were illegal, legal (e.g. not requiring invokespecial to invoke private > interface methods; allowing access to private members via > reflection/Methodhandles that were previously not allowed). > > - Existing tests incidentally affected by the nestmate changes > > ? This includes tests of things utilising class > redefinition/retransformation to alter nested types but which > unintentionally alter nest relationships (which is not permitted). > > There are still a number of tests problem-listed with issues filed > against them to have them adapted to work with nestmates. Some of these > are intended to be addressed in the short-term, while some (such as the > runtime/SelectionResolution test changes) may not eventuate. > > - https://bugs.openjdk.java.net/browse/JDK-8203033 > - https://bugs.openjdk.java.net/browse/JDK-8199450 > - https://bugs.openjdk.java.net/browse/JDK-8196855 > - https://bugs.openjdk.java.net/browse/JDK-8194857 > - https://bugs.openjdk.java.net/browse/JDK-8187655 > > There is also further test work still to be completed (the JNI and JDI > invocation tests): > - https://bugs.openjdk.java.net/browse/JDK-8191117 > which will continue in parallel with the main RFR. > > Pre-integration Testing: > ?- General: > ??? - Mach5: hs/jdk tier1,2 > ??? - Mach5: hs-nightly (tiers 1 -3) > ?- Targetted > ?? - nashorn (for asm changes) > ?? - hotspot: runtime/* > ????????????? serviceability/* > ????????????? compiler/* > ????????????? vmTestbase/* > ?? - jdk: java/lang/invoke/* > ????????? java/lang/reflect/* > ????????? java/lang/instrument/* > ????????? java/lang/Class/* > ????????? java/lang/management/* > ? - langtools: tools/javac > ?????????????? tools/javap > From david.holmes at oracle.com Wed May 23 06:57:54 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 23 May 2018 16:57:54 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <53feed71-0ab9-7743-6034-3c95bb3b20e8@bell-sw.com> Message-ID: <848b45e0-df51-7d74-e4eb-60819d101330@oracle.com> Hi Boris, On 17/05/2018 7:23 PM, Boris Ulasevich wrote: > Hi David, > > You are right! My bad, R2_tmp parameter conflicts with Rklass on > check_klass_subtype(..) call. See correct patch in attach. Now all > runtime/Nestmates tests passed! :) I went to aplpy your patch and found it's not a diff against my patch but against the original non-nestmate code, so I want to be clear on the changes. AFAICS the differences are: - const Register Rklass = R3_tmp; + const Register Rklass = R2_tmp; // Note! Same register with Rrecv This initially concerned me as we stomp on Rrecv when we do the load_klass, but then you moved the: __ load_klass(Rklass, Rrecv); to after the object case, which used Rrecv. I had assumed Rrecv was somehow used when we actually do the call, but I'm assuming jump_from_interpreted is done in such as way that the receiver is still available on the interpreter stack and is used from there. - __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); + __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R3_tmp, noreg, subtype); Okay - reworking of tmp regs. - __ jump_from_interpreted(Rindex); + __ jump_from_interpreted(Rmethod); I'm going to trust this is okay :) It's not at all clear to me how the f2 Method* gets passed through on different platforms - sometimes its in the "index" and sometimes the "method" registers. (I _really_ wish there was consistency in terminology across the different platforms - this code is awful for trying to compare the different platforms to figure out what to do on a new one.) Thanks, David > regards, > Boris > > On 17.05.2018 11:24, David Holmes wrote: >> On 17/05/2018 6:13 PM, Boris Ulasevich wrote: >>> Hi David, >>> >>> I see three tests failing: >>> ?> NullPointerException at >>> TestInterfaceMethodSelection.doInvoke(TestInterfaceMethodSelection.java:191) >>> >>> ?> NullPointerException at TestInvoke.access_priv(TestInvoke.java:54) >>> ?> InvocationTargetException at >>> TestReflection.access_priv(TestReflection.java:61) >>> >>> I will send you test details in a separate mail. >> >> Ok. This indicates a bug in the assembly code. The NPE's will likely >> be SEGVs caused by a zero register. >> >> Thanks, >> David >> >> >>> Boris >>> >>> On 17.05.2018 00:23, David Holmes wrote: >>>> Hi Boris, >>>> >>>> Many thanks for looking at this and working through the ARM code. >>>> >>>> I will study your patch in detail but am concerned by the "passes >>>> most of runtime/Nestmates tests Ok."! What tests are not passing? >>>> >>>> David >>>> >>>> On 17/05/2018 1:05 AM, Boris Ulasevich wrote: >>>>> Hi David, >>>>> >>>>> Let us look to the change in templateTable_arm.cpp. I have several >>>>> notes. >>>>> >>>>> 1. We have compilation error because check_klass_subtype call needs >>>>> one more temporary register parameter. I think correct change is: >>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >>>>> -> >>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); >>>>> >>>>> 2. R0_tmp holds mdp used for profilation several lines below, so we >>>>> can't spoil it. I think correct change is: >>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); >>>>> -> >>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R2_tmp, noreg, subtype); >>>>> >>>>> 3. We can't jump to Rindex. I believe it was supposed to jump to >>>>> Rmethod: >>>>> jump_from_interpreted(Rindex); >>>>> -> >>>>> jump_from_interpreted(Rmethod); >>>>> >>>>> 4. Please note than Rklass and Rflags reuses same register. Before >>>>> the change their life ranges had no intersection. I would propose >>>>> to use R2 for Rklass (same with Rrecv) and move load_klass call >>>>> after invokevirtual_helper call to avoid life range intersecton. >>>>> >>>>> See my patch against original templateTable_arm.cpp version in >>>>> attach - with this change ARM build compiles and passes most of >>>>> runtime/Nestmates tests Ok. >>>>> >>>>> regards, >>>>> Boris >>>>> >>>>> On 15.05.2018 03:52, David Holmes wrote: >>>>>> This review is being spread across four groups: langtools, >>>>>> core-libs, hotspot and serviceability. This is the specific review >>>>>> thread for hotspot - webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>>>> >>>>>> See below for full details - including annotated full webrev >>>>>> guiding the review. >>>>>> >>>>>> The intent is to have JEP-181 targeted and integrated by the end >>>>>> of this month. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>> The nestmates project (JEP-181) introduces new classfile >>>>>> attributes to identify classes and interfaces in the same nest, so >>>>>> that the VM can perform access control based on those attributes >>>>>> and so allow direct private access between nestmates without >>>>>> requiring javac to generate synthetic accessor methods. These >>>>>> access control changes also extend to core reflection and the >>>>>> MethodHandle.Lookup contexts. >>>>>> >>>>>> Direct private calls between nestmates requires a more general >>>>>> calling context than is permitted by invokespecial, and so the >>>>>> JVMS is updated to allow, and javac updated to use, invokevirtual >>>>>> and invokeinterface for private class and interface method calls >>>>>> respectively. These changed semantics also extend to MethodHandle >>>>>> findXXX operations. >>>>>> >>>>>> At this time we are only concerned with static nest definitions, >>>>>> which map to a top-level class/interface as the nest-host and all >>>>>> its nested types as nest-members. >>>>>> >>>>>> Please see the JEP for further details. >>>>>> >>>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>>>> >>>>>> All of the specification changes have been previously been worked >>>>>> out by the Valhalla Project Expert Group, and the implementation >>>>>> reviewed by the various contributors and discussed on the >>>>>> valhalla-dev mailing list. >>>>>> >>>>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen >>>>>> Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, >>>>>> Kumar Srinivasan >>>>>> >>>>>> Master webrev of all changes: >>>>>> >>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>>>> >>>>>> Annotated master webrev index: >>>>>> >>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>>>> >>>>>> Performance: this is expected to be performance neutral in a >>>>>> general sense. Benchmarking and performance runs are about to start. >>>>>> >>>>>> Testing Discussion: >>>>>> ------------------ >>>>>> >>>>>> The testing for nestmates can be broken into four main groups: >>>>>> >>>>>> -? New tests specifically related to nestmates and currently in >>>>>> the runtime/Nestmates directory >>>>>> >>>>>> - New tests to complement existing tests by adding in testcases >>>>>> not previously expressible. >>>>>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>>>>> use of invokespecial for private interface methods and performing >>>>>> receiver typechecks, so we add >>>>>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>>>>> invokeinterface. >>>>>> >>>>>> -? New JVM TI tests to verify the spec changes related to nest >>>>>> attributes. >>>>>> >>>>>> -? Existing tests significantly affected by the nestmates changes, >>>>>> primarily: >>>>>> ??? -? runtime/SelectionResolution >>>>>> >>>>>> ??? In most cases the nestmate changes makes certain invocations >>>>>> that were illegal, legal (e.g. not requiring invokespecial to >>>>>> invoke private interface methods; allowing access to private >>>>>> members via reflection/Methodhandles that were previously not >>>>>> allowed). >>>>>> >>>>>> - Existing tests incidentally affected by the nestmate changes >>>>>> >>>>>> ?? This includes tests of things utilising class >>>>>> redefinition/retransformation to alter nested types but which >>>>>> unintentionally alter nest relationships (which is not permitted). >>>>>> >>>>>> There are still a number of tests problem-listed with issues filed >>>>>> against them to have them adapted to work with nestmates. Some of >>>>>> these are intended to be addressed in the short-term, while some >>>>>> (such as the runtime/SelectionResolution test changes) may not >>>>>> eventuate. >>>>>> >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>>>> >>>>>> There is also further test work still to be completed (the JNI and >>>>>> JDI invocation tests): >>>>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>>>> which will continue in parallel with the main RFR. >>>>>> >>>>>> Pre-integration Testing: >>>>>> ??- General: >>>>>> ???? - Mach5: hs/jdk tier1,2 >>>>>> ???? - Mach5: hs-nightly (tiers 1 -3) >>>>>> ??- Targetted >>>>>> ??? - nashorn (for asm changes) >>>>>> ??? - hotspot: runtime/* >>>>>> ?????????????? serviceability/* >>>>>> ?????????????? compiler/* >>>>>> ?????????????? vmTestbase/* >>>>>> ??? - jdk: java/lang/invoke/* >>>>>> ?????????? java/lang/reflect/* >>>>>> ?????????? java/lang/instrument/* >>>>>> ?????????? java/lang/Class/* >>>>>> ?????????? java/lang/management/* >>>>>> ?? - langtools: tools/javac >>>>>> ??????????????? tools/javap >>>>>> From david.holmes at oracle.com Wed May 23 06:58:04 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 23 May 2018 16:58:04 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <8B73333D-3D51-48FA-BB39-DE3A171F72C0@oracle.com> References: <8B73333D-3D51-48FA-BB39-DE3A171F72C0@oracle.com> Message-ID: <2767373b-c761-833c-5303-19fffe68e1db@oracle.com> Hi Karen, Thanks for looking (yet again!) at this. On 22/05/2018 4:48 AM, Karen Kinnear wrote: > David, > > Many thanks for the extremely careful attention to detail. And thank you > for the annotated master webrev. > I still owe you a review of the tests. > > I had a number of questions/comments about the sources - mostly requests > for clarifying comments: > 1. LinkResolver::check_method_accessability: > line 591: worth adding something like "e.g. linkage errors due to > nest_host resolution Done > 2. LinkResolver.cpp line 782 > ? cleanup: > ? FIXME: Debugging code removed - thanks! > 3. JVM_AreNestMates > If member is a primitive or an array - what happens? We crash or hit asserts - only instanceKlasses support nestmate attributes. That's why they get filtered in the Java code. > 4. reflection.cpp line 714: > ? cleanup: FIXME - perhaps add assertions? I worry that there may be a valid code path that allows us to get there with non-instanceKlasses, in which case in product we would crash. Ideally I'd like to be able to prove it is impossible, but I don't know how to do that. I will remove the comment. > 5. reflection.cpp lines 721-722: > ? perhaps add a comment that it is the caller's responsibility to check > ? HAS_PENDING_EXCEPTION if return is false - specifically to decide > ? context-specific handling of nestmate resolution/validation That code is now less visible through use of CHECK_false on the has_nestmate_access_to call (suggested by Coleen). But I've expanded the comment before it. > 6. instanceKlass.cpp line 255: > FIXME: an exception from this is perhaps impossible > ? did you want an assertion here? Again this is a case where I _think_ it's not possible but I can't be sure and I would not want product code to crash. My choice with these kinds of FIXME is to either prove the answer to the question (which I'm unable to do) or else delete the comment. So comment deleted. > > 7. JVM_GetNestHost is the only caller of IK->nest_host that passes in a > NULL. > ? I would expect one of these to clear PENDING_EXCEPTION,it would be > helpful > ? for the comments to clarify whose responsibility this is. The exception does not get cleared but propagates to the Java code. The Java code then filters out LinkageErrors and allows runtime exceptions to propagate. > 8. templateTable_x86.cpp > line 3789: > ? I belive that rax is reference klass (from f1) if interface method OR if > ?? j.l.Object final/private method I can't see that in ConstantPoolCacheEntry::set_direct_or_vtable_call. Only for the interface method case do we explicitly call set_f1. rax is not passed to invokevirtual_helper, which is what processes the Object case - in fact rax gets stomped by that code. > line 3794: > ? I think this is more precisely: > ? "First check for Object (not private/final) case, then private/final > interface method, then regular interface method" No, invokevirtual_helper processes all the Object cases. My most recent changes in this area stopped the final Object methods from taking the new private interface method path unintentionally. > ?and line 3810 > ? "Check for private/final method invocation - indicated by vfinal" At this point final Object methods have already been handled, all that is left are private (interface) methods. > 9. cpCache.cpp line 191 > ? set_f1(holder); // interface klass* > Could you possibly add a comment that for private/final method invocations > ? REFC == DECC > > also line 180: > check for private/final interface method invocations I'm not quite sure what form of comment you are looking for here. holder() is the DEFC but there is no concept of REFC in this code that I can see. ?? > 10. klassVtable.cpp > Could please add a line to the comment in initialize_itable_for_interface > before 1217: call to lookup_instance_method_in_klasses > // Invokespecial does not perform selection based on the receiver, so it > does > // not used the cached itable I've added it but to be honest I don't understand it's relevance in the context of initializing the itable. ?? > 11. instanceKlass > ?has_nest_member > has this comment: > // can't have two names the same, so we're done > Where is that ensured? In walking through the explanation I realized the comment is wrong. This was one of several cleanups John suggested (though I added the comment): https://bugs.openjdk.java.net/browse/JDK-8199567 Resolving a CP entry will cause a class load to occur for "name", and if I have multiple CP entries with the same "name" (somehow) then they must all resolve to the same klass. So once I've found a matching name and examined the actual klass, searching for additional entries with the same name is pointless as they must resolve to the same klass. Hence the loop can terminate once we have a name match. The comment should read: // can't have different classes for the same name, so we're done BTW there is no test case for the case where the klass comparison fails. I could not see how to construct one. It may well be impossible as per the earlier comment in the code: // names match so check actual klass - this may trigger class loading if // it doesn't match (but that should be impossible) If has_nest_member underpinned a public API (which I initially thought it might) then it would be possible by simply having two same-named classes from different classloaders. But the way this API is used is for validation between a nest-member and it's nest-host as part of nest-host resolution, and I can't see any way to get two resolved klasses that somehow exist in different loaders with the same name. Thanks again! David ----- > > thanks, > Karen > >> On May 14, 2018, at 8:52 PM, David Holmes > > wrote: >> >> This review is being spread across four groups: langtools, core-libs, >> hotspot and serviceability. This is the specific review thread for >> hotspot - webrev: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >> >> See below for full details - including annotated full webrev guiding >> the review. >> >> The intent is to have JEP-181 targeted and integrated by the end of >> this month. >> >> Thanks, >> David >> ----- >> >> The nestmates project (JEP-181) introduces new classfile attributes to >> identify classes and interfaces in the same nest, so that the VM can >> perform access control based on those attributes and so allow direct >> private access between nestmates without requiring javac to generate >> synthetic accessor methods. These access control changes also extend >> to core reflection and the MethodHandle.Lookup contexts. >> >> Direct private calls between nestmates requires a more general calling >> context than is permitted by invokespecial, and so the JVMS is updated >> to allow, and javac updated to use, invokevirtual and invokeinterface >> for private class and interface method calls respectively. These >> changed semantics also extend to MethodHandle findXXX operations. >> >> At this time we are only concerned with static nest definitions, which >> map to a top-level class/interface as the nest-host and all its nested >> types as nest-members. >> >> Please see the JEP for further details. >> >> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >> >> All of the specification changes have been previously been worked out >> by the Valhalla Project Expert Group, and the implementation reviewed >> by the various contributors and discussed on the valhalla-dev mailing >> list. >> >> Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, >> Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir >> Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan >> >> Master webrev of all changes: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >> >> Annotated master webrev index: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >> >> Performance: this is expected to be performance neutral in a general >> sense. Benchmarking and performance runs are about to start. >> >> Testing Discussion: >> ------------------ >> >> The testing for nestmates can be broken into four main groups: >> >> - ?New tests specifically related to nestmates and currently in the >> runtime/Nestmates directory >> >> - New tests to complement existing tests by adding in testcases not >> previously expressible. >> ?- ?For example java/lang/invoke/SpecialInterfaceCall.java tests use >> of invokespecial for private interface methods and performing receiver >> typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do >> similar tests for invokeinterface. >> >> - ?New JVM TI tests to verify the spec changes related to nest attributes. >> >> - ?Existing tests significantly affected by the nestmates changes, >> primarily: >> ??- ?runtime/SelectionResolution >> >> ??In most cases the nestmate changes makes certain invocations that >> were illegal, legal (e.g. not requiring invokespecial to invoke >> private interface methods; allowing access to private members via >> reflection/Methodhandles that were previously not allowed). >> >> - Existing tests incidentally affected by the nestmate changes >> >> ?This includes tests of things utilising class >> redefinition/retransformation to alter nested types but which >> unintentionally alter nest relationships (which is not permitted). >> >> There are still a number of tests problem-listed with issues filed >> against them to have them adapted to work with nestmates. Some of >> these are intended to be addressed in the short-term, while some (such >> as the runtime/SelectionResolution test changes) may not eventuate. >> >> - https://bugs.openjdk.java.net/browse/JDK-8203033 >> - https://bugs.openjdk.java.net/browse/JDK-8199450 >> - https://bugs.openjdk.java.net/browse/JDK-8196855 >> - https://bugs.openjdk.java.net/browse/JDK-8194857 >> - https://bugs.openjdk.java.net/browse/JDK-8187655 >> >> There is also further test work still to be completed (the JNI and JDI >> invocation tests): >> - https://bugs.openjdk.java.net/browse/JDK-8191117 >> which will continue in parallel with the main RFR. >> >> Pre-integration Testing: >> - General: >> ???- Mach5: hs/jdk tier1,2 >> ???- Mach5: hs-nightly (tiers 1 -3) >> - Targetted >> ??- nashorn (for asm changes) >> ??- hotspot: runtime/* >> ?????????????serviceability/* >> ?????????????compiler/* >> ?????????????vmTestbase/* >> ??- jdk: java/lang/invoke/* >> ?????????java/lang/reflect/* >> ?????????java/lang/instrument/* >> ?????????java/lang/Class/* >> ?????????java/lang/management/* >> ?- langtools: tools/javac >> ??????????????tools/javap >> > From shade at redhat.com Wed May 23 07:27:58 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 23 May 2018 09:27:58 +0200 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <0cf68c20-51d3-6754-8837-698d120ebf53@redhat.com> <759bf969-1aed-852b-1e9d-5d96178086f9@redhat.com> Message-ID: <4661377e-a8ef-9844-18d5-40dfe607d616@redhat.com> On 05/23/2018 04:31 AM, David Holmes wrote: > On 23/05/2018 8:50 AM, Aleksey Shipilev wrote: >> On 05/22/2018 11:44 PM, David Holmes wrote: >>> On 23/05/2018 2:35 AM, Aleksey Shipilev wrote: >>>> On 05/22/2018 06:01 PM, Andrew Dinn wrote: >>>>>> ????? aarch64: FAILED >>>>>> ???????? java/lang/invoke/SpecialInterfaceCall.java >>>>>> ??????????? http://cr.openjdk.java.net/~shade/nestmates/SpecialInterfaceCall-hs_err.log >>>>>> >>>>>> I have not dug deeper yet, AArch64 maintainers are welcome to step in! >>>>> I'm not sure what is going on here. >>>>> >>>>> A few days ago I patched the jdk/jdk dev tree with David's megapatch, >>>>> built on AArch64 and ran tier1 tests, obtaining results before and after >>>>> patching. There were some tier1 test failures which did not seem to >>>>> relate to the NestMates patch. All three of CondyReturnPrimitiveTest, >>>>> CondyWrongType and SpecialInterfaceCall passed before and after. >>>>> >>>>> I will check to see the patch has been applied correctly and rerun these >>>>> tests. Aleksey, perhaps you could try again using a patched jdkdev? >>>> >>>> Applied David's webrev.hotspot.v1 over current jdk/jdk, and SpecialInterfaceCall fails the same way >>>> with linux-aarch64-fastdebug on my Raspberry Pi 3. >>> >>> You can't just apply the hotspot patch! You need the entire thing. >> >> Duh, right. In this case, we were "saved" by the fact the failing test seems to explore hotspot side >> of things only, and it fails the same way the nestmates branch in valhalla repo. And anyhow, >> applying the full patch fails the same way :) > > SpecialInterfaceCall shouldn't be failing! Can you confirm it only fails with nestmates patches > applied? (It's not a nestmate test and should** be unaffected by nestmate changes.) It seems to be a pre-existing issue, indeed. So, nestmates does not necessarily makes it worse. Let' me bisect. Nestmates looks good-ish then :) -Aleksey From david.holmes at oracle.com Wed May 23 07:55:07 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 23 May 2018 17:55:07 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <4661377e-a8ef-9844-18d5-40dfe607d616@redhat.com> References: <0cf68c20-51d3-6754-8837-698d120ebf53@redhat.com> <759bf969-1aed-852b-1e9d-5d96178086f9@redhat.com> <4661377e-a8ef-9844-18d5-40dfe607d616@redhat.com> Message-ID: Hi Aleksey, The problem will likely be: http://hg.openjdk.java.net/jdk/jdk/rev/2ace90aec488 Though there's no platform specific code there so it may be that it tickles some other bug. David On 23/05/2018 5:27 PM, Aleksey Shipilev wrote: > On 05/23/2018 04:31 AM, David Holmes wrote: >> On 23/05/2018 8:50 AM, Aleksey Shipilev wrote: >>> On 05/22/2018 11:44 PM, David Holmes wrote: >>>> On 23/05/2018 2:35 AM, Aleksey Shipilev wrote: >>>>> On 05/22/2018 06:01 PM, Andrew Dinn wrote: >>>>>>> ????? aarch64: FAILED >>>>>>> ???????? java/lang/invoke/SpecialInterfaceCall.java >>>>>>> ??????????? http://cr.openjdk.java.net/~shade/nestmates/SpecialInterfaceCall-hs_err.log >>>>>>> >>>>>>> I have not dug deeper yet, AArch64 maintainers are welcome to step in! >>>>>> I'm not sure what is going on here. >>>>>> >>>>>> A few days ago I patched the jdk/jdk dev tree with David's megapatch, >>>>>> built on AArch64 and ran tier1 tests, obtaining results before and after >>>>>> patching. There were some tier1 test failures which did not seem to >>>>>> relate to the NestMates patch. All three of CondyReturnPrimitiveTest, >>>>>> CondyWrongType and SpecialInterfaceCall passed before and after. >>>>>> >>>>>> I will check to see the patch has been applied correctly and rerun these >>>>>> tests. Aleksey, perhaps you could try again using a patched jdkdev? >>>>> >>>>> Applied David's webrev.hotspot.v1 over current jdk/jdk, and SpecialInterfaceCall fails the same way >>>>> with linux-aarch64-fastdebug on my Raspberry Pi 3. >>>> >>>> You can't just apply the hotspot patch! You need the entire thing. >>> >>> Duh, right. In this case, we were "saved" by the fact the failing test seems to explore hotspot side >>> of things only, and it fails the same way the nestmates branch in valhalla repo. And anyhow, >>> applying the full patch fails the same way :) >> >> SpecialInterfaceCall shouldn't be failing! Can you confirm it only fails with nestmates patches >> applied? (It's not a nestmate test and should** be unaffected by nestmate changes.) > > It seems to be a pre-existing issue, indeed. So, nestmates does not necessarily makes it worse. Let' > me bisect. Nestmates looks good-ish then :) > > -Aleksey > From david.holmes at oracle.com Wed May 23 08:35:53 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 23 May 2018 18:35:53 +1000 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: <7e7fc484287e4da4926176e0b5ae1b64@sap.com> References: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> <7e7fc484287e4da4926176e0b5ae1b64@sap.com> Message-ID: <7415d5f1-9add-bc75-4acd-cf167bf22486@oracle.com> On 22/05/2018 8:16 PM, Doerr, Martin wrote: > Hi Kim, > > I can't see how a new implicit consume is introduced by Michihiro's change. He just explained how the existing code works. > > If implicit consume has been rejected the current code is wrong: > "new_obj = o->forwardee();" would need some kind of barrier before using the new_obj. But if forwardee is set by a CAS with (default) full bi-directional fence we have that barrier. The argument is that such a strong barrier can be relaxed yet maintain correctness - isn't it? David > But this issue is not related to what Michihiro wants to change AFAICS. > > Best regards, > Martin > > > -----Original Message----- > From: ppc-aix-port-dev [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] On Behalf Of Kim Barrett > Sent: Montag, 21. Mai 2018 06:00 > To: Michihiro Horie > Cc: hotspot-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; Gustavo Bueno Romero ; ppc-aix-port-dev at openjdk.java.net; david.holmes at oracle.com > Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 > >> On May 18, 2018, at 5:12 PM, Michihiro Horie wrote: >> >> Dear all, >> >> I update the webrev: http://cr.openjdk.java.net/~mhorie/8154736/webrev.09/ >> >> With the release barrier before the CAS, new_obj can be observed from other threads. If the CAS succeeds, the current thread can use new_obj without barriers. If the CAS fails, "o->forwardee()" is ordered with respect to CAS by accessing the same memory location "_mark", so no barriers needed. The order of (1) access to the forwardee and (2) access to forwardee's fields is preserved due to Release-Consume ordering on supported platforms. (The ordering between "new_obj = o->forwardee();" and logging or other usages is not changed.) >> >> Regarding the maintainability, the requirement is CAS(memory_order_release) as Release-Consume to be consistent with C++11. This requirement is necessary when a weaker platform like DEC Alpha is to be supported. On currently supported platforms, code change can be safe if the code meats this requirement (and the order of (1) access to the forwardee and (2) access to forwardee's fields is the natural way of coding). > > Relying on implicit consume has been been discussed and rejected, in > the earlier thread on this topic and I think elsewhere too. > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-October/021538.html > From kirk.pepperdine at gmail.com Wed May 23 08:38:49 2018 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Wed, 23 May 2018 10:38:49 +0200 Subject: OpenJDK G1 Patch In-Reply-To: References: Message-ID: <50B3FD40-0909-40E0-A178-2D4175A276C5@gmail.com> > On May 23, 2018, at 9:36 AM, Michal Frajt wrote: > > Hi Kirk, > > I?m just on the back end of the Oracle Dev tour in Japan? Once I land I?ll work on the test case to hopefully sort out what is happening with references and G1 (if this is indeed the cause of the grief I?ve been trying to sort through) and I do plan on sharing the results. > - please share with us your finding, we would like to know how the G1 handles processing of weak references which are across all regions, we simply need to get them cleaned within a certain time at least (our CMSTriggerInterval) > - we have had similar issue with the Azul C4 where the weak references behave(d) very differently, when you just shortly access a weak reference referent, the weak reference won?t be cleaned in the next C4 major cycle (like minutes interval), so you have to stop accessing it which goes completely against the weak reference (or soft reference) idea Indeed, this feels like a bug.. certainly doesn?t meet WeakReference definitions as I understand it. > > Yes, I understand but the ParNew before the CMS cycle has typically proven to be a huge loss overall. > - our finding is exactly opposite, you should run ParNew before the initial marking is started > - I had to even patch Hotspot http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2012-August/001297.html, https://bugs.openjdk.java.net/browse/JDK-7189971 Right, still a loss in every prod system where I?ve seen it being used. > > If you need to run frequent CMS cycles to keep tenured clean then it?s very very likely that you?re suffering from a premature promotion problem. We have tuning strategies to manage this situation that doesn?t require excessive triggering of CMS. > - we run frequent CMS cycles to keep the weak references cleaned, tenured getting cleaned is just a side effect I can?t say that I understand your use case but it sounds as if you *need* to have weak references cleaned frequently then again I?m thinking there is something in the design of your framework that is bothering me. If you need weak references you need weak references. That said, if you need them cleaned quickly then I have to ask why? > > Right, I can typically have JVMs run for weeks or longer without suffering from concurrent mode failures. It?s about controlling frequency and promotion rates. And it?s not so easy as the aging process is affected by the size of Eden. > - right, in one of our distribution layers I was evening thinking about accessing the object age via the Unsafe to re-create it right before it gets promoted so the object stays very likely in the young generation > - now when thinking more about it, I could actually just try reduce the age via the Unsafe, the object should stay longer in the survivor spaces, need to try Messing with the internal structures in this way seems like a gross violation? it feels worse than the triggering of a full. Again I?d suggest there is a tuning strategy to manage this. Again, this isn?t to say your are facing real issues and that these fixes don?t work for you. The question is; is it appropriate to drop in patches to solve your specific problem for you domain without regard to other use cases? If so, what if I drop in a patch that is detrimental to your applications performance? IMO, what is really missing here is a general due diligence that confirms that this solves problems in the broader population. That we have different experiences with ParNew before CMS InitialMark/Remark isn?t surprising to me. And, it?s not that I don?t trust your benchmarks, it?s that I don?t trust anyones benchmarks including my own unless they have gone through a serious level of vetting. Even then?... > > Regards, > Michal > > > Od: "Kirk Pepperdine" kirk.pepperdine at gmail.com > Komu: "Michal Frajt" michal at frajt.eu > Kopie: > Datum: Tue, 22 May 2018 19:11:12 +0200 > P?edmet: Re: OpenJDK G1 Patch > > >> On May 22, 2018, at 9:28 AM, Michal Frajt > wrote: >> >> >> Hi Kirk, >> >> - weak references were in 2001 the only alternative for C++ smart pointers like design and I guess still there is no other solution available > > I?m just on the back end of the Oracle Dev tour in Japan? Once I land I?ll work on the test case to hopefully sort out what is happening with references and G1 (if this is indeed the cause of the grief I?ve been trying to sort through) and I do plan on sharing the results. > >> - CMSTriggerInterval is not invoking the full collection cycle, it invokes normal concurrent CMS cycle (1 ParNew + 2 STW) so that there is at least one concurrent CMS cycle per the trigger interval (considering CMS runs due to CMSTriggerRatio and another reasons) > > Yes, I understand but the ParNew before the CMS cycle has typically proven to be a huge loss overall. > >> - we are fine with running regular concurrent CMS cycle which next to releasing the weak references keeps our old gen always clean having much higher chance to survive sudden increase in the promotion rate > > If you need to run frequent CMS cycles to keep tenured clean then it?s very very likely that you?re suffering from a premature promotion problem. We have tuning strategies to manage this situation that doesn?t require excessive triggering of CMS. > > >> - we never run the full cycle, or better we have never seen an end of the full cycle as applications are killed and restarted before it can finish > > Right, I can typically have JVMs run for weeks or longer without suffering from concurrent mode failures. It?s about controlling frequency and promotion rates. And it?s not so easy as the aging process is affected by the size of Eden. > > G1 pause times should cluster to some value that is approximately constant.. or should mostly band around a narrow range of values. To achieve this with minimal pause times and not risk suffering a full requires that you allocate more memory and a larger pause time goal. Unfortunately the larger heap sizes runs counter to your goal. However G1 was designed for big iron and I find that if you under-resource it you?re asking for high GC overhead and other troubles. For smaller heaps Parallel and/or CMS is still by far a better choice. > > Kind regards, > Kirk > From david.holmes at oracle.com Wed May 23 08:43:18 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 23 May 2018 18:43:18 +1000 Subject: RFR(XXS) 8203635: JFR sampler thread does not record stack info In-Reply-To: References: <72cfcdcf-2545-26e5-a513-d91f375905b4@redhat.com> Message-ID: On 23/05/2018 4:16 AM, Thomas St?fe wrote: > Hi Zhengyu, > > looks good. > > I wish it would be like this: > > a non-virtual Thread::run() which does common initialization, and then > calls a virtual protected abstract Thread::run_impl(), which is > implemented by child classes... We already do that - thread_native_entry calls thread->run David > Regards, Thomas > > > > On Tue, May 22, 2018 at 7:57 PM, Zhengyu Gu wrote: >> Thanks for the review, Aleksey. >> >> On 05/22/2018 01:45 PM, Aleksey Shipilev wrote: >>> >>> On 05/22/2018 07:40 PM, Zhengyu Gu wrote: >>>> >>>> Please review this trivial change that added record_stack_base_and_size() >>>> call to JFR sampler >>>> thread, so NMT can count it correctly. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203635 >>>> Webrev: http://cr.openjdk.java.net/~zgu/8203635/webrev.00/ >>> >>> >>> Looks good to me. I wish this was handled in superclass initialization, so >>> no one would miss this. >> >> >> Agree! However, it is a bit tricky, as the call has to be made from >> *current* thread. >> >> Thanks, >> >> -Zhengyu >> >>> >>> -Aleksey >>> >> From martin.doerr at sap.com Wed May 23 09:13:38 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 23 May 2018 09:13:38 +0000 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: <7415d5f1-9add-bc75-4acd-cf167bf22486@oracle.com> References: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> <7e7fc484287e4da4926176e0b5ae1b64@sap.com> <7415d5f1-9add-bc75-4acd-cf167bf22486@oracle.com> Message-ID: <7df88fec76254789b83ee9a48ec2f54c@sap.com> Hi David, Michihiro's change does not rely on Consume AFAICS. Only existing code does. The CAS is not in the path between "new_obj = o->forwardee();" and the usage of new_obj. So the CAS' barrier won't help. There's just ordering by "Consume". The other thread's CAS (which has set the _mark field) currently has a post-barrier which won't help the reading thread, either. Michihiro's change relies on the ordering of the CAS with respect to "new_obj = o->forwardee();" by accessing the same memory location "_mark" which is ensured by memory coherency. And it relies on that compilers don't speculatively load o->forwardee() before the CAS which is ensured by the integrated compiler barriers (clobber "memory" in the volatile inline asm code). And it is also prevented because _mark is declared volatile. Best regards, Martin -----Original Message----- From: David Holmes [mailto:david.holmes at oracle.com] Sent: Mittwoch, 23. Mai 2018 10:36 To: Doerr, Martin ; Kim Barrett ; Michihiro Horie Cc: hotspot-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; Gustavo Bueno Romero ; ppc-aix-port-dev at openjdk.java.net Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 On 22/05/2018 8:16 PM, Doerr, Martin wrote: > Hi Kim, > > I can't see how a new implicit consume is introduced by Michihiro's change. He just explained how the existing code works. > > If implicit consume has been rejected the current code is wrong: > "new_obj = o->forwardee();" would need some kind of barrier before using the new_obj. But if forwardee is set by a CAS with (default) full bi-directional fence we have that barrier. The argument is that such a strong barrier can be relaxed yet maintain correctness - isn't it? David > But this issue is not related to what Michihiro wants to change AFAICS. > > Best regards, > Martin > > > -----Original Message----- > From: ppc-aix-port-dev [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] On Behalf Of Kim Barrett > Sent: Montag, 21. Mai 2018 06:00 > To: Michihiro Horie > Cc: hotspot-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; Gustavo Bueno Romero ; ppc-aix-port-dev at openjdk.java.net; david.holmes at oracle.com > Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 > >> On May 18, 2018, at 5:12 PM, Michihiro Horie wrote: >> >> Dear all, >> >> I update the webrev: http://cr.openjdk.java.net/~mhorie/8154736/webrev.09/ >> >> With the release barrier before the CAS, new_obj can be observed from other threads. If the CAS succeeds, the current thread can use new_obj without barriers. If the CAS fails, "o->forwardee()" is ordered with respect to CAS by accessing the same memory location "_mark", so no barriers needed. The order of (1) access to the forwardee and (2) access to forwardee's fields is preserved due to Release-Consume ordering on supported platforms. (The ordering between "new_obj = o->forwardee();" and logging or other usages is not changed.) >> >> Regarding the maintainability, the requirement is CAS(memory_order_release) as Release-Consume to be consistent with C++11. This requirement is necessary when a weaker platform like DEC Alpha is to be supported. On currently supported platforms, code change can be safe if the code meats this requirement (and the order of (1) access to the forwardee and (2) access to forwardee's fields is the natural way of coding). > > Relying on implicit consume has been been discussed and rejected, in > the earlier thread on this topic and I think elsewhere too. > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-October/021538.html > From gil at azul.com Wed May 23 11:32:11 2018 From: gil at azul.com (Gil Tene) Date: Wed, 23 May 2018 11:32:11 +0000 Subject: OpenJDK G1 Patch In-Reply-To: <50B3FD40-0909-40E0-A178-2D4175A276C5@gmail.com> References: <50B3FD40-0909-40E0-A178-2D4175A276C5@gmail.com> Message-ID: > On May 23, 2018, at 11:38 AM, Kirk Pepperdine wrote: > > >> On May 23, 2018, at 9:36 AM, Michal Frajt wrote: >> >> Hi Kirk, >> >> I?m just on the back end of the Oracle Dev tour in Japan? Once I land I?ll work on the test case to hopefully sort out what is happening with references and G1 (if this is indeed the cause of the grief I?ve been trying to sort through) and I do plan on sharing the results. >> - please share with us your finding, we would like to know how the G1 handles processing of weak references which are across all regions, we simply need to get them cleaned within a certain time at least (our CMSTriggerInterval) >> - we have had similar issue with the Azul C4 where the weak references behave(d) very differently, when you just shortly access a weak reference referent, the weak reference won?t be cleaned in the next C4 major cycle (like minutes interval), so you have to stop accessing it which goes completely against the weak reference (or soft reference) idea > > Indeed, this feels like a bug.. certainly doesn?t meet WeakReference definitions as I understand it. That *is* the WeakReference definition. Weak references are only cleared if their referent is determined to be weakly reachable. Doing a WeakReference.get() produces a strong reference, making the referent strongly reachable, and by-definition preventing the weak reference from being cleared until that strong reference is itself proven to be unreachable. Doing WeakReference.get() concurrently with collection activity will prevent that weak reference from being cleared in that cycle. Doing so continually and concurrently with all future collection activity has an obvious and well defined effect of preventing the reference from ever being cleared. Doing continual hot-gets on weak references and expecting them to be cleaned up anyway is an application logic bug, and a complete mis-reading of the spec. You do have options for making stuff like this work by relying on (unspecified and non-guaranteed) GC behavior, like STW pauses. But these options generally rely on *something* stopping your hot-get activity. You can bake-in relying on STW collections to stall your access to weak references so they can complete both reachability analysis and clearing without you having a chance to change things in the middle, but then you are stuck with STW pauses. When the collector doesn't stop you when determining reachability, you also lose that "benefit" of pausing your eager-getting behavior. You can try to bake-in some other assumptions (like "CMS will only see the strong reference if I write it to memory, or if it is still on my stack during it's shorter STW pause", but you are still building based on an assumption of a specific underlying implementation, and not on anything that the WeakReference documentation or anything else in the JavaDocs or Java spec allows you to assume. Doing that is no different than e.g. writing a cool concurrent algorithm that relies on an assumptions that machines have 2 cores or less, and then hitting the reality of 3-way races on machines with 3+ cores when those become commonplace. C4 and G1 (and ZGC and Shenandoah) all treat WeakRefs the same [well established and correct] way in this regard AFAIK. > > > >> >> Yes, I understand but the ParNew before the CMS cycle has typically proven to be a huge loss overall. >> - our finding is exactly opposite, you should run ParNew before the initial marking is started >> - I had to even patch Hotspot http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2012-August/001297.html, https://bugs.openjdk.java.net/browse/JDK-7189971 > Right, still a loss in every prod system where I?ve seen it being used. > >> >> If you need to run frequent CMS cycles to keep tenured clean then it?s very very likely that you?re suffering from a premature promotion problem. We have tuning strategies to manage this situation that doesn?t require excessive triggering of CMS. >> - we run frequent CMS cycles to keep the weak references cleaned, tenured getting cleaned is just a side effect > > I can?t say that I understand your use case but it sounds as if you *need* to have weak references cleaned frequently then again I?m thinking there is something in the design of your framework that is bothering me. If you need weak references you need weak references. That said, if you need them cleaned quickly then I have to ask why? > >> >> Right, I can typically have JVMs run for weeks or longer without suffering from concurrent mode failures. It?s about controlling frequency and promotion rates. And it?s not so easy as the aging process is affected by the size of Eden. >> - right, in one of our distribution layers I was evening thinking about accessing the object age via the Unsafe to re-create it right before it gets promoted so the object stays very likely in the young generation >> - now when thinking more about it, I could actually just try reduce the age via the Unsafe, the object should stay longer in the survivor spaces, need to try > > Messing with the internal structures in this way seems like a gross violation? it feels worse than the triggering of a full. Again I?d suggest there is a tuning strategy to manage this. > > Again, this isn?t to say your are facing real issues and that these fixes don?t work for you. The question is; is it appropriate to drop in patches to solve your specific problem for you domain without regard to other use cases? If so, what if I drop in a patch that is detrimental to your applications performance? IMO, what is really missing here is a general due diligence that confirms that this solves problems in the broader population. That we have different experiences with ParNew before CMS InitialMark/Remark isn?t surprising to me. And, it?s not that I don?t trust your benchmarks, it?s that I don?t trust anyones benchmarks including my own unless they have gone through a serious level of vetting. Even then?... >> >> Regards, >> Michal >> >> >> Od: "Kirk Pepperdine" kirk.pepperdine at gmail.com >> Komu: "Michal Frajt" michal at frajt.eu >> Kopie: >> Datum: Tue, 22 May 2018 19:11:12 +0200 >> P?edmet: Re: OpenJDK G1 Patch >> >> >>> On May 22, 2018, at 9:28 AM, Michal Frajt > wrote: >>> >>> >>> Hi Kirk, >>> >>> - weak references were in 2001 the only alternative for C++ smart pointers like design and I guess still there is no other solution available >> >> I?m just on the back end of the Oracle Dev tour in Japan? Once I land I?ll work on the test case to hopefully sort out what is happening with references and G1 (if this is indeed the cause of the grief I?ve been trying to sort through) and I do plan on sharing the results. >> >>> - CMSTriggerInterval is not invoking the full collection cycle, it invokes normal concurrent CMS cycle (1 ParNew + 2 STW) so that there is at least one concurrent CMS cycle per the trigger interval (considering CMS runs due to CMSTriggerRatio and another reasons) >> >> Yes, I understand but the ParNew before the CMS cycle has typically proven to be a huge loss overall. >> >>> - we are fine with running regular concurrent CMS cycle which next to releasing the weak references keeps our old gen always clean having much higher chance to survive sudden increase in the promotion rate >> >> If you need to run frequent CMS cycles to keep tenured clean then it?s very very likely that you?re suffering from a premature promotion problem. We have tuning strategies to manage this situation that doesn?t require excessive triggering of CMS. >> >> >>> - we never run the full cycle, or better we have never seen an end of the full cycle as applications are killed and restarted before it can finish >> >> Right, I can typically have JVMs run for weeks or longer without suffering from concurrent mode failures. It?s about controlling frequency and promotion rates. And it?s not so easy as the aging process is affected by the size of Eden. >> >> G1 pause times should cluster to some value that is approximately constant.. or should mostly band around a narrow range of values. To achieve this with minimal pause times and not risk suffering a full requires that you allocate more memory and a larger pause time goal. Unfortunately the larger heap sizes runs counter to your goal. However G1 was designed for big iron and I find that if you under-resource it you?re asking for high GC overhead and other troubles. For smaller heaps Parallel and/or CMS is still by far a better choice. >> >> Kind regards, >> Kirk >> From michal at frajt.eu Wed May 23 12:35:25 2018 From: michal at frajt.eu (Michal Frajt) Date: Wed, 23 May 2018 14:35:25 +0200 Subject: OpenJDK G1 Patch In-Reply-To: References: =?iso-8859-1?q?=3CP967SB=243CE5626D270836379A35B3AD2B1A0127=40frajt?= =?iso-8859-1?q?=2Eeu=3E_=3C50B3FD40=2D0909=2D40E0=2DA178=2D2D4175A276?= =?iso-8859-1?q?C5=40gmail=2Ecom=3E_=3CD9933EDE=2D5413=2D442B=2D976F?= =?iso-8859-1?q?=2DF5D5541F86AD=40azul=2Ecom=3E?= Message-ID: > Doing continual hot-gets on weak references and expecting them to be cleaned up anyway is an application logic bug, and a complete mis-reading of the spec. What means hot-get? The weak reference have the public method get to be called, what else? The new garbage collectors (especially C4) will never clear the weak reference just because the application code allowed itself to call the get method once within a GC major cycle (minutes interval). Do you think that calling the get method once per 5 minutes can be called hot?? Within the 5 minutes a Java application can access terabytes of heap data but should not touch any weak reference, is this what you say? ? > Weak references are only cleared if their referent is determined to be weakly reachable. ?Then do it! The 0.999999999% of the C4 major cycle time many weak reference referents are weakly reachable but still never cleared.? ? > But you are still building based on an assumption of a specific underlying implementation, and not on anything that the WeakReference documentation or anything else in the JavaDocs or Java spec allows you to assume. Indeed. The weak reference usage for the application framework came in 2001 when we were looking for the C++ smart pointer like structure in Java. Nearly 20 years later it still works fine with the CMS collector. It is only hard to accept that the latest G1 collector will only increase our STW times and complicate weak references handling. Where is the progress? Is there another smart pointer like handling in Java? No, only weak reference was very close to it. Michal? ? Od: "Gil Tene" gil at azul.com Komu: "Kirk Pepperdine" kirk.pepperdine at gmail.com Kopie: "Michal Frajt" michal at frajt.eu,"hotspot-dev at openjdk.java.net" hotspot-dev at openjdk.java.net Datum: Wed, 23 May 2018 11:32:11 +0000 P?edmet: Re: OpenJDK G1 Patch ? ? > On May 23, 2018, at 11:38 AM, Kirk Pepperdine wrote: > > >> On May 23, 2018, at 9:36 AM, Michal Frajt wrote: >> >> Hi Kirk, >> >> I?m just on the back end of the Oracle Dev tour in Japan? Once I land I?ll work on the test case to hopefully sort out what is happening with references and G1 (if this is indeed the cause of the grief I?ve been trying to sort through) and I do plan on sharing the results. >> - please share with us your finding, we would like to know how the G1 handles processing of weak references which are across all regions, we simply need to get them cleaned within a certain time at least (our CMSTriggerInterval) >> - we have had similar issue with the Azul C4 where the weak references behave(d) very differently, when you just shortly access a weak reference referent, the weak reference won?t be cleaned in the next C4 major cycle (like minutes interval), so you have to stop accessing it which goes completely against the weak reference (or soft reference) idea > > Indeed, this feels like a bug.. certainly doesn?t meet WeakReference definitions as I understand it. ? That *is* the WeakReference definition. Weak references are only cleared if their referent is determined to be weakly reachable. Doing a WeakReference.get() produces a strong reference, making the referent strongly reachable, and by-definition preventing the weak reference from being cleared until that strong reference is itself proven to be unreachable. Doing WeakReference.get() concurrently with collection activity will prevent that weak reference from being cleared in that cycle. Doing so continually and concurrently with all future collection activity has an obvious and well defined effect of preventing the reference from ever being cleared. ? Doing continual hot-gets on weak references and expecting them to be cleaned up anyway is an application logic bug, and a complete mis-reading of the spec. You do have options for making stuff like this work by relying on (unspecified and non-guaranteed) GC behavior, like STW pauses. But these options generally rely on *something* stopping your hot-get activity. You can bake-in relying on STW collections to stall your access to weak references so they can complete both reachability analysis and clearing without you having a chance to change things in the middle, but then you are stuck with STW pauses. When the collector doesn't stop you when determining reachability, you also lose that "benefit" of pausing your eager-getting behavior. You can try to bake-in some other assumptions (like "CMS will only see the strong reference if I write it to memory, or if it is still on my stack during it's shorter STW pause", but you are still building based on an assumption of a specific underlying implementation, and not on anything that the WeakReference documentation or anything else in the JavaDocs or Java spec allows you to assume. Doing that is no different than e.g. writing a cool concurrent algorithm that relies on an assumptions that machines have 2 cores or less, and then hitting the reality of 3-way races on machines with 3+ cores when those become commonplace. ? C4 and G1 (and ZGC and Shenandoah) all treat WeakRefs the same [well established and correct] way in this regard AFAIK. ? > > > >> >> Yes, I understand but the ParNew before the CMS cycle has typically proven to be a huge loss overall. >> - our finding is exactly opposite, you should run ParNew before the initial marking is started >> - I had to even patch Hotspot http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2012-August/001297.html, https://bugs.openjdk.java.net/browse/JDK-7189971 > Right, still a loss in every prod system where I?ve seen it being used. > >> >> If you need to run frequent CMS cycles to keep tenured clean then it?s very very likely that you?re suffering from a premature promotion problem. We have tuning strategies to manage this situation that doesn?t require excessive triggering of CMS. >> - we run frequent CMS cycles to keep the weak references cleaned, tenured getting cleaned is just a side effect > > I can?t say that I understand your use case but it sounds as if you *need* to have weak references cleaned frequently then again I?m thinking there is something in the design of your framework that is bothering me. If you need weak references you need weak references. That said, if you need them cleaned quickly then I have to ask why? > >> >> Right, I can typically have JVMs run for weeks or longer without suffering from concurrent mode failures. It?s about controlling frequency and promotion rates. And it?s not so easy as the aging process is affected by the size of Eden. >> - right, in one of our distribution layers I was evening thinking about accessing the object age via the Unsafe to re-create it right before it gets promoted so the object stays very likely in the young generation >> - now when thinking more about it, I could actually just try reduce the age via the Unsafe, the object should stay longer in the survivor spaces, need to try > > Messing with the internal structures in this way seems like a gross violation? it feels worse than the triggering of a full. Again I?d suggest there is a tuning strategy to manage this. > > Again, this isn?t to say your are facing real issues and that these fixes don?t work for you. The question is; is it appropriate to drop in patches to solve your specific problem for you domain without regard to other use cases? If so, what if I drop in a patch that is detrimental to your applications performance? IMO, what is really missing here is a general due diligence that confirms that this solves problems in the broader population. That we have different experiences with ParNew before CMS InitialMark/Remark isn?t surprising to me. And, it?s not that I don?t trust your benchmarks, it?s that I don?t trust anyones benchmarks including my own unless they have gone through a serious level of vetting. Even then?... >> >> Regards, >> Michal >> >> >> Od: "Kirk Pepperdine" kirk.pepperdine at gmail.com >> Komu: "Michal Frajt" michal at frajt.eu >> Kopie: >> Datum: Tue, 22 May 2018 19:11:12 +0200 >> P?edmet: Re: OpenJDK G1 Patch >> >> >>> On May 22, 2018, at 9:28 AM, Michal Frajt > wrote: >>> >>> >>> Hi Kirk, >>> >>> - weak references were in 2001 the only alternative for C++ smart pointers like design and I guess still there is no other solution available >> >> I?m just on the back end of the Oracle Dev tour in Japan? Once I land I?ll work on the test case to hopefully sort out what is happening with references and G1 (if this is indeed the cause of the grief I?ve been trying to sort through) and I do plan on sharing the results. >> >>> - CMSTriggerInterval is not invoking the full collection cycle, it invokes normal concurrent CMS cycle (1 ParNew + 2 STW) so that there is at least one concurrent CMS cycle per the trigger interval (considering CMS runs due to CMSTriggerRatio and another reasons) >> >> Yes, I understand but the ParNew before the CMS cycle has typically proven to be a huge loss overall. >> >>> - we are fine with running regular concurrent CMS cycle which next to releasing the weak references keeps our old gen always clean having much higher chance to survive sudden increase in the promotion rate >> >> If you need to run frequent CMS cycles to keep tenured clean then it?s very very likely that you?re suffering from a premature promotion problem. We have tuning strategies to manage this situation that doesn?t require excessive triggering of CMS. >> >> >>> - we never run the full cycle, or better we have never seen an end of the full cycle as applications are killed and restarted before it can finish >> >> Right, I can typically have JVMs run for weeks or longer without suffering from concurrent mode failures. It?s about controlling frequency and promotion rates. And it?s not so easy as the aging process is affected by the size of Eden. >> >> G1 pause times should cluster to some value that is approximately constant.. or should mostly band around a narrow range of values. To achieve this with minimal pause times and not risk suffering a full requires that you allocate more memory and a larger pause time goal. Unfortunately the larger heap sizes runs counter to your goal. However G1 was designed for big iron and I find that if you under-resource it you?re asking for high GC overhead and other troubles. For smaller heaps Parallel and/or CMS is still by far a better choice. >> >> Kind regards, >> Kirk >> ? From boris.ulasevich at bell-sw.com Wed May 23 13:40:59 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Wed, 23 May 2018 16:40:59 +0300 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <848b45e0-df51-7d74-e4eb-60819d101330@oracle.com> References: <53feed71-0ab9-7743-6034-3c95bb3b20e8@bell-sw.com> <848b45e0-df51-7d74-e4eb-60819d101330@oracle.com> Message-ID: <9a8eadf0-e745-ee38-fb37-b29d99fa96c0@bell-sw.com> Hi David, some minor comments below.. On 23.05.2018 09:57, David Holmes wrote: > Hi Boris, > > On 17/05/2018 7:23 PM, Boris Ulasevich wrote: >> Hi David, >> >> You are right! My bad, R2_tmp parameter conflicts with Rklass on >> check_klass_subtype(..) call. See correct patch in attach. Now all >> runtime/Nestmates tests passed! :) > > I went to aplpy your patch and found it's not a diff against my patch > but against the original non-nestmate code, Yes, sorry. For me it was natural to apply patch from your review to local repo, rework it and just send updated diff from my machine :) > so I want to be clear on the changes. AFAICS the differences are: > > -? const Register Rklass? = R3_tmp; > +? const Register Rklass? = R2_tmp; // Note! Same register with Rrecv Yes. > This initially concerned me as we stomp on Rrecv when we do the > load_klass, but then you moved? the: > > ?? __ load_klass(Rklass, Rrecv); > > to after the object case, which used Rrecv. I had assumed Rrecv was > somehow used when we actually do the call, but I'm assuming > jump_from_interpreted is done in such as way that the receiver is still > available on the interpreter stack and is used from there. Ok. I'm not sure I understand you well. When I moved load_klass call down my point was to skip unnecessary load for the notObjectMethod case (Rklass is not required in this case) and to make possible to reuse same R2 register by both Rklass and Rrecv register. > -? __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); > +? __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R3_tmp, noreg, subtype); > > Okay - reworking of tmp regs. Yes, ARM32 requires one more tmp reg, and we can't spoil R0. > -? __ jump_from_interpreted(Rindex); > +? __ jump_from_interpreted(Rmethod); > > I'm going to trust this is okay :) It's not at all clear to me how the > f2 Method* gets passed through on different platforms - sometimes its in > the "index" and sometimes the "method" registers. I see. On ARM/AARCH64 we have preallocated register Rmethod/rmethod to hold current method pointer and prepare_invoke call to setup registers properly. > (I _really_ wish there > was consistency in terminology across the different platforms - this > code is awful for trying to compare the different platforms to figure > out what to do on a new one.) > > Thanks, > David > >> regards, >> Boris >> >> On 17.05.2018 11:24, David Holmes wrote: >>> On 17/05/2018 6:13 PM, Boris Ulasevich wrote: >>>> Hi David, >>>> >>>> I see three tests failing: >>>> ?> NullPointerException at >>>> TestInterfaceMethodSelection.doInvoke(TestInterfaceMethodSelection.java:191) >>>> >>>> ?> NullPointerException at TestInvoke.access_priv(TestInvoke.java:54) >>>> ?> InvocationTargetException at >>>> TestReflection.access_priv(TestReflection.java:61) >>>> >>>> I will send you test details in a separate mail. >>> >>> Ok. This indicates a bug in the assembly code. The NPE's will likely >>> be SEGVs caused by a zero register. >>> >>> Thanks, >>> David >>> >>> >>>> Boris >>>> >>>> On 17.05.2018 00:23, David Holmes wrote: >>>>> Hi Boris, >>>>> >>>>> Many thanks for looking at this and working through the ARM code. >>>>> >>>>> I will study your patch in detail but am concerned by the "passes >>>>> most of runtime/Nestmates tests Ok."! What tests are not passing? >>>>> >>>>> David >>>>> >>>>> On 17/05/2018 1:05 AM, Boris Ulasevich wrote: >>>>>> Hi David, >>>>>> >>>>>> Let us look to the change in templateTable_arm.cpp. I have several >>>>>> notes. >>>>>> >>>>>> 1. We have compilation error because check_klass_subtype call >>>>>> needs one more temporary register parameter. I think correct >>>>>> change is: >>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >>>>>> -> >>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); >>>>>> >>>>>> 2. R0_tmp holds mdp used for profilation several lines below, so >>>>>> we can't spoil it. I think correct change is: >>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, subtype); >>>>>> -> >>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R2_tmp, noreg, subtype); >>>>>> >>>>>> 3. We can't jump to Rindex. I believe it was supposed to jump to >>>>>> Rmethod: >>>>>> jump_from_interpreted(Rindex); >>>>>> -> >>>>>> jump_from_interpreted(Rmethod); >>>>>> >>>>>> 4. Please note than Rklass and Rflags reuses same register. Before >>>>>> the change their life ranges had no intersection. I would propose >>>>>> to use R2 for Rklass (same with Rrecv) and move load_klass call >>>>>> after invokevirtual_helper call to avoid life range intersecton. >>>>>> >>>>>> See my patch against original templateTable_arm.cpp version in >>>>>> attach - with this change ARM build compiles and passes most of >>>>>> runtime/Nestmates tests Ok. >>>>>> >>>>>> regards, >>>>>> Boris >>>>>> >>>>>> On 15.05.2018 03:52, David Holmes wrote: >>>>>>> This review is being spread across four groups: langtools, >>>>>>> core-libs, hotspot and serviceability. This is the specific >>>>>>> review thread for hotspot - webrev: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>>>>> >>>>>>> >>>>>>> See below for full details - including annotated full webrev >>>>>>> guiding the review. >>>>>>> >>>>>>> The intent is to have JEP-181 targeted and integrated by the end >>>>>>> of this month. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>> The nestmates project (JEP-181) introduces new classfile >>>>>>> attributes to identify classes and interfaces in the same nest, >>>>>>> so that the VM can perform access control based on those >>>>>>> attributes and so allow direct private access between nestmates >>>>>>> without requiring javac to generate synthetic accessor methods. >>>>>>> These access control changes also extend to core reflection and >>>>>>> the MethodHandle.Lookup contexts. >>>>>>> >>>>>>> Direct private calls between nestmates requires a more general >>>>>>> calling context than is permitted by invokespecial, and so the >>>>>>> JVMS is updated to allow, and javac updated to use, invokevirtual >>>>>>> and invokeinterface for private class and interface method calls >>>>>>> respectively. These changed semantics also extend to MethodHandle >>>>>>> findXXX operations. >>>>>>> >>>>>>> At this time we are only concerned with static nest definitions, >>>>>>> which map to a top-level class/interface as the nest-host and all >>>>>>> its nested types as nest-members. >>>>>>> >>>>>>> Please see the JEP for further details. >>>>>>> >>>>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>>>>> >>>>>>> All of the specification changes have been previously been worked >>>>>>> out by the Valhalla Project Expert Group, and the implementation >>>>>>> reviewed by the various contributors and discussed on the >>>>>>> valhalla-dev mailing list. >>>>>>> >>>>>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>>>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen >>>>>>> Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, >>>>>>> Kumar Srinivasan >>>>>>> >>>>>>> Master webrev of all changes: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>>>>> >>>>>>> Annotated master webrev index: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>>>>> >>>>>>> >>>>>>> Performance: this is expected to be performance neutral in a >>>>>>> general sense. Benchmarking and performance runs are about to start. >>>>>>> >>>>>>> Testing Discussion: >>>>>>> ------------------ >>>>>>> >>>>>>> The testing for nestmates can be broken into four main groups: >>>>>>> >>>>>>> -? New tests specifically related to nestmates and currently in >>>>>>> the runtime/Nestmates directory >>>>>>> >>>>>>> - New tests to complement existing tests by adding in testcases >>>>>>> not previously expressible. >>>>>>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java >>>>>>> tests use of invokespecial for private interface methods and >>>>>>> performing receiver typechecks, so we add >>>>>>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests >>>>>>> for invokeinterface. >>>>>>> >>>>>>> -? New JVM TI tests to verify the spec changes related to nest >>>>>>> attributes. >>>>>>> >>>>>>> -? Existing tests significantly affected by the nestmates >>>>>>> changes, primarily: >>>>>>> ??? -? runtime/SelectionResolution >>>>>>> >>>>>>> ??? In most cases the nestmate changes makes certain invocations >>>>>>> that were illegal, legal (e.g. not requiring invokespecial to >>>>>>> invoke private interface methods; allowing access to private >>>>>>> members via reflection/Methodhandles that were previously not >>>>>>> allowed). >>>>>>> >>>>>>> - Existing tests incidentally affected by the nestmate changes >>>>>>> >>>>>>> ?? This includes tests of things utilising class >>>>>>> redefinition/retransformation to alter nested types but which >>>>>>> unintentionally alter nest relationships (which is not permitted). >>>>>>> >>>>>>> There are still a number of tests problem-listed with issues >>>>>>> filed against them to have them adapted to work with nestmates. >>>>>>> Some of these are intended to be addressed in the short-term, >>>>>>> while some (such as the runtime/SelectionResolution test changes) >>>>>>> may not eventuate. >>>>>>> >>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>>>>> >>>>>>> There is also further test work still to be completed (the JNI >>>>>>> and JDI invocation tests): >>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>>>>> which will continue in parallel with the main RFR. >>>>>>> >>>>>>> Pre-integration Testing: >>>>>>> ??- General: >>>>>>> ???? - Mach5: hs/jdk tier1,2 >>>>>>> ???? - Mach5: hs-nightly (tiers 1 -3) >>>>>>> ??- Targetted >>>>>>> ??? - nashorn (for asm changes) >>>>>>> ??? - hotspot: runtime/* >>>>>>> ?????????????? serviceability/* >>>>>>> ?????????????? compiler/* >>>>>>> ?????????????? vmTestbase/* >>>>>>> ??? - jdk: java/lang/invoke/* >>>>>>> ?????????? java/lang/reflect/* >>>>>>> ?????????? java/lang/instrument/* >>>>>>> ?????????? java/lang/Class/* >>>>>>> ?????????? java/lang/management/* >>>>>>> ?? - langtools: tools/javac >>>>>>> ??????????????? tools/javap >>>>>>> From glaubitz at physik.fu-berlin.de Wed May 23 14:02:13 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Wed, 23 May 2018 16:02:13 +0200 Subject: Another failure on linux-sparc Message-ID: <2b15f2ca-ea28-44bf-3f39-0d0f1c23a956@physik.fu-berlin.de> Hi! I just did a test build on an old Sun Fire 2000 because unfortunately, Debian's SPARC T5 has run into hardware issues with the system board and so far we have been unable to fix it - the replacement board is quite expensive :(. Anyway, trying to build Hotspot for linux-sparc fails with: === Output from failing command(s) repeated here === /usr/bin/printf "* For target hotspot_variant-server_libjvm_objs_library_call.o:\n" * For target hotspot_variant-server_libjvm_objs_library_call.o: (/bin/grep -v -e "^Note: including file:" < /srv/openjdk/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs/hotspot_variant-server_libjvm_objs_library_call.o.log || true) | /usr/bin/head -n 12 /srv/openjdk/jdk/src/hotspot/share/opto/library_call.cpp: In member function ?bool LibraryCallKit::inline_native_clone(bool)?: /srv/openjdk/jdk/src/hotspot/share/opto/library_call.cpp:4272:38: error: incomplete type ?BarrierSet? used in nested name specifier BarrierSetC2* bs = BarrierSet::barrier_set()->barrier_set_c2(); ^~~~~~~~~~~ if test `/usr/bin/wc -l < /srv/openjdk/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs/hotspot_variant-server_libjvm_objs_library_call.o.log` -gt 12; then /bin/echo " ... (rest of output omitted)" ; fi /usr/bin/printf "\n* All command lines available in /srv/openjdk/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs.\n" * All command lines available in /srv/openjdk/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs. /usr/bin/printf "=== End of repeated output ===\n" === End of repeated output === Anyone seen this already on other configurations? Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From kirk.pepperdine at gmail.com Wed May 23 14:07:40 2018 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Wed, 23 May 2018 16:07:40 +0200 Subject: OpenJDK G1 Patch In-Reply-To: References: <50B3FD40-0909-40E0-A178-2D4175A276C5@gmail.com> Message-ID: <3CB7D32D-14D1-433A-A084-89DDFED6740C@gmail.com> > On May 23, 2018, at 1:32 PM, Gil Tene wrote: > > > >> On May 23, 2018, at 11:38 AM, Kirk Pepperdine wrote: >> >> >>> On May 23, 2018, at 9:36 AM, Michal Frajt wrote: >>> >>> Hi Kirk, >>> >>> I?m just on the back end of the Oracle Dev tour in Japan? Once I land I?ll work on the test case to hopefully sort out what is happening with references and G1 (if this is indeed the cause of the grief I?ve been trying to sort through) and I do plan on sharing the results. >>> - please share with us your finding, we would like to know how the G1 handles processing of weak references which are across all regions, we simply need to get them cleaned within a certain time at least (our CMSTriggerInterval) >>> - we have had similar issue with the Azul C4 where the weak references behave(d) very differently, when you just shortly access a weak reference referent, the weak reference won?t be cleaned in the next C4 major cycle (like minutes interval), so you have to stop accessing it which goes completely against the weak reference (or soft reference) idea >> >> Indeed, this feels like a bug.. certainly doesn?t meet WeakReference definitions as I understand it. > > That *is* the WeakReference definition. Weak references are only cleared if their referent is determined to > be weakly reachable. Doing a WeakReference.get() produces a strong reference, making the referent > strongly reachable, and by-definition preventing the weak reference from being cleared until that strong > reference is itself proven to be unreachable. Doing WeakReference.get() concurrently with collection activity > will prevent that weak reference from being cleared in that cycle. Doing so continually and concurrently > with all future collection activity has an obvious and well defined effect of preventing the reference from > ever being cleared. This makes complete sense? This is a bit of a ?race? in that if the GC runs at the right time the weakly referenced object will be collected otherwise it will be ?accidentally" strong. My guess is this is even trickier when marking concurrently? If this truly is a hot-get then you most likely shouldn?t be using a weak reference. It feels like an accident waiting to happen?. A poor replacement for a proper eviction policy if this is used in a cache. > > C4 and G1 (and ZGC and Shenandoah) all treat WeakRefs the same [well established and correct] way > in this regard AFAIK. Not sure about G1.. well I?m sure it treats weak refs correctly but there is also something going on that isn?t allowing G1 to collect weak references as one would expect? maybe it?s like the time stamp issue in C4 that kept soft references live when they shouldn?t have been. Kind regards, Kirk From shade at redhat.com Wed May 23 14:07:54 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 23 May 2018 16:07:54 +0200 Subject: Another failure on linux-sparc In-Reply-To: <2b15f2ca-ea28-44bf-3f39-0d0f1c23a956@physik.fu-berlin.de> References: <2b15f2ca-ea28-44bf-3f39-0d0f1c23a956@physik.fu-berlin.de> Message-ID: On 05/23/2018 04:02 PM, John Paul Adrian Glaubitz wrote: > Anyway, trying to build Hotspot for linux-sparc fails with: > > === Output from failing command(s) repeated here === > /usr/bin/printf "* For target hotspot_variant-server_libjvm_objs_library_call.o:\n" > * For target hotspot_variant-server_libjvm_objs_library_call.o: > (/bin/grep -v -e "^Note: including file:" < /srv/openjdk/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs/hotspot_variant-server_libjvm_objs_library_call.o.log || true) | /usr/bin/head -n 12 > /srv/openjdk/jdk/src/hotspot/share/opto/library_call.cpp: In member function ?bool LibraryCallKit::inline_native_clone(bool)?: > /srv/openjdk/jdk/src/hotspot/share/opto/library_call.cpp:4272:38: error: incomplete type ?BarrierSet? used in nested name specifier > BarrierSetC2* bs = BarrierSet::barrier_set()->barrier_set_c2(); > ^~~~~~~~~~~ > if test `/usr/bin/wc -l < /srv/openjdk/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs/hotspot_variant-server_libjvm_objs_library_call.o.log` -gt 12; then /bin/echo " ... (rest of output omitted)" ; fi > /usr/bin/printf "\n* All command lines available in /srv/openjdk/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs.\n" > > * All command lines available in /srv/openjdk/jdk/build/linux-sparcv9-normal-server-release/make-support/failure-logs. > /usr/bin/printf "=== End of repeated output ===\n" > === End of repeated output === > > Anyone seen this already on other configurations? I saw this today on x86_64/zero, and it seem Zero-specific: https://builds.shipilev.net/openjdk-jdk/openjdk-jdk-b215-x86_64-zero-fastdebug.build.log ...but, curiously, it was not reproducible with my local builds. I decided to wait another nightly before diving in. FWIW, I am not sure why share/opto is even built when compiler2 is disabled. -Aleksey From kirk.pepperdine at gmail.com Wed May 23 14:10:33 2018 From: kirk.pepperdine at gmail.com (Kirk Pepperdine) Date: Wed, 23 May 2018 16:10:33 +0200 Subject: OpenJDK G1 Patch In-Reply-To: References: Message-ID: Hi Michal, So we?re talking about G1 in this case and as I?ve been eluding, I believe there is a bug w.r.t. how weak references are handled by the concurrent mark. This has been a very useful discussion in helping me understand some application/G1GC behaviors.. I?m going to disengage from the conversation and focus on getting a test case together. Kind regards, Kirk > On May 23, 2018, at 2:35 PM, Michal Frajt wrote: > > > > Doing continual hot-gets on weak references and expecting them to be cleaned up anyway is an application logic bug, and a complete mis-reading of the spec. > What means hot-get? The weak reference have the public method get to be called, what else? The new garbage collectors (especially C4) will never clear the weak reference just because the application code allowed itself to call the get method once within a GC major cycle (minutes interval). Do you think that calling the get method once per 5 minutes can be called hot? Within the 5 minutes a Java application can access terabytes of heap data but should not touch any weak reference, is this what you say? > > > Weak references are only cleared if their referent is determined to be weakly reachable. > Then do it! The 0.999999999% of the C4 major cycle time many weak reference referents are weakly reachable but still never cleared. > > > But you are still building based on an assumption of a specific underlying implementation, and not on anything that the WeakReference documentation or anything else in the JavaDocs or Java spec allows you to assume. > Indeed. The weak reference usage for the application framework came in 2001 when we were looking for the C++ smart pointer like structure in Java. Nearly 20 years later it still works fine with the CMS collector. It is only hard to accept that the latest G1 collector will only increase our STW times and complicate weak references handling. Where is the progress? Is there another smart pointer like handling in Java? No, only weak reference was very close to it. > > Michal > > Od: "Gil Tene" gil at azul.com > Komu: "Kirk Pepperdine" kirk.pepperdine at gmail.com > Kopie: "Michal Frajt" michal at frajt.eu,"hotspot-dev at openjdk.java.net" hotspot-dev at openjdk.java.net > Datum: Wed, 23 May 2018 11:32:11 +0000 > P?edmet: Re: OpenJDK G1 Patch > > > > > On May 23, 2018, at 11:38 AM, Kirk Pepperdine wrote: > > > > > >> On May 23, 2018, at 9:36 AM, Michal Frajt wrote: > >> > >> Hi Kirk, > >> > >> I?m just on the back end of the Oracle Dev tour in Japan? Once I land I?ll work on the test case to hopefully sort out what is happening with references and G1 (if this is indeed the cause of the grief I?ve been trying to sort through) and I do plan on sharing the results. > >> - please share with us your finding, we would like to know how the G1 handles processing of weak references which are across all regions, we simply need to get them cleaned within a certain time at least (our CMSTriggerInterval) > >> - we have had similar issue with the Azul C4 where the weak references behave(d) very differently, when you just shortly access a weak reference referent, the weak reference won?t be cleaned in the next C4 major cycle (like minutes interval), so you have to stop accessing it which goes completely against the weak reference (or soft reference) idea > > > > Indeed, this feels like a bug.. certainly doesn?t meet WeakReference definitions as I understand it. > > That *is* the WeakReference definition. Weak references are only cleared if their referent is determined to > be weakly reachable. Doing a WeakReference.get() produces a strong reference, making the referent > strongly reachable, and by-definition preventing the weak reference from being cleared until that strong > reference is itself proven to be unreachable. Doing WeakReference.get() concurrently with collection activity > will prevent that weak reference from being cleared in that cycle. Doing so continually and concurrently > with all future collection activity has an obvious and well defined effect of preventing the reference from > ever being cleared. > > Doing continual hot-gets on weak references and expecting them to be cleaned up anyway is an application > logic bug, and a complete mis-reading of the spec. You do have options for making stuff like this work by > relying on (unspecified and non-guaranteed) GC behavior, like STW pauses. But these options generally rely > on *something* stopping your hot-get activity. You can bake-in relying on STW collections to stall your access > to weak references so they can complete both reachability analysis and clearing without you having a > chance to change things in the middle, but then you are stuck with STW pauses. When the collector doesn't > stop you when determining reachability, you also lose that "benefit" of pausing your eager-getting behavior. > You can try to bake-in some other assumptions (like "CMS will only see the strong reference if I write it to > memory, or if it is still on my stack during it's shorter STW pause", but you are still building based on an > assumption of a specific underlying implementation, and not on anything that the WeakReference > documentation or anything else in the JavaDocs or Java spec allows you to assume. Doing that is no > different than e.g. writing a cool concurrent algorithm that relies on an assumptions that machines have > 2 cores or less, and then hitting the reality of 3-way races on machines with 3+ cores when those become > commonplace. > > C4 and G1 (and ZGC and Shenandoah) all treat WeakRefs the same [well established and correct] way > in this regard AFAIK. > > > > > > > > >> > >> Yes, I understand but the ParNew before the CMS cycle has typically proven to be a huge loss overall. > >> - our finding is exactly opposite, you should run ParNew before the initial marking is started > >> - I had to even patch Hotspot http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2012-August/001297.html, https://bugs.openjdk.java.net/browse/JDK-7189971 > > Right, still a loss in every prod system where I?ve seen it being used. > > > >> > >> If you need to run frequent CMS cycles to keep tenured clean then it?s very very likely that you?re suffering from a premature promotion problem. We have tuning strategies to manage this situation that doesn?t require excessive triggering of CMS. > >> - we run frequent CMS cycles to keep the weak references cleaned, tenured getting cleaned is just a side effect > > > > I can?t say that I understand your use case but it sounds as if you *need* to have weak references cleaned frequently then again I?m thinking there is something in the design of your framework that is bothering me. If you need weak references you need weak references. That said, if you need them cleaned quickly then I have to ask why? > > > >> > >> Right, I can typically have JVMs run for weeks or longer without suffering from concurrent mode failures. It?s about controlling frequency and promotion rates. And it?s not so easy as the aging process is affected by the size of Eden. > >> - right, in one of our distribution layers I was evening thinking about accessing the object age via the Unsafe to re-create it right before it gets promoted so the object stays very likely in the young generation > >> - now when thinking more about it, I could actually just try reduce the age via the Unsafe, the object should stay longer in the survivor spaces, need to try > > > > Messing with the internal structures in this way seems like a gross violation? it feels worse than the triggering of a full. Again I?d suggest there is a tuning strategy to manage this. > > > > Again, this isn?t to say your are facing real issues and that these fixes don?t work for you. The question is; is it appropriate to drop in patches to solve your specific problem for you domain without regard to other use cases? If so, what if I drop in a patch that is detrimental to your applications performance? IMO, what is really missing here is a general due diligence that confirms that this solves problems in the broader population. That we have different experiences with ParNew before CMS InitialMark/Remark isn?t surprising to me. And, it?s not that I don?t trust your benchmarks, it?s that I don?t trust anyones benchmarks including my own unless they have gone through a serious level of vetting. Even then?... > >> > >> Regards, > >> Michal > >> > >> > >> Od: "Kirk Pepperdine" kirk.pepperdine at gmail.com > >> Komu: "Michal Frajt" michal at frajt.eu > >> Kopie: > >> Datum: Tue, 22 May 2018 19:11:12 +0200 > >> P?edmet: Re: OpenJDK G1 Patch > >> > >> > >>> On May 22, 2018, at 9:28 AM, Michal Frajt > wrote: > >>> > >>> > >>> Hi Kirk, > >>> > >>> - weak references were in 2001 the only alternative for C++ smart pointers like design and I guess still there is no other solution available > >> > >> I?m just on the back end of the Oracle Dev tour in Japan? Once I land I?ll work on the test case to hopefully sort out what is happening with references and G1 (if this is indeed the cause of the grief I?ve been trying to sort through) and I do plan on sharing the results. > >> > >>> - CMSTriggerInterval is not invoking the full collection cycle, it invokes normal concurrent CMS cycle (1 ParNew + 2 STW) so that there is at least one concurrent CMS cycle per the trigger interval (considering CMS runs due to CMSTriggerRatio and another reasons) > >> > >> Yes, I understand but the ParNew before the CMS cycle has typically proven to be a huge loss overall. > >> > >>> - we are fine with running regular concurrent CMS cycle which next to releasing the weak references keeps our old gen always clean having much higher chance to survive sudden increase in the promotion rate > >> > >> If you need to run frequent CMS cycles to keep tenured clean then it?s very very likely that you?re suffering from a premature promotion problem. We have tuning strategies to manage this situation that doesn?t require excessive triggering of CMS. > >> > >> > >>> - we never run the full cycle, or better we have never seen an end of the full cycle as applications are killed and restarted before it can finish > >> > >> Right, I can typically have JVMs run for weeks or longer without suffering from concurrent mode failures. It?s about controlling frequency and promotion rates. And it?s not so easy as the aging process is affected by the size of Eden. > >> > >> G1 pause times should cluster to some value that is approximately constant.. or should mostly band around a narrow range of values. To achieve this with minimal pause times and not risk suffering a full requires that you allocate more memory and a larger pause time goal. Unfortunately the larger heap sizes runs counter to your goal. However G1 was designed for big iron and I find that if you under-resource it you?re asking for high GC overhead and other troubles. For smaller heaps Parallel and/or CMS is still by far a better choice. > >> > >> Kind regards, > >> Kirk > >> > From glaubitz at physik.fu-berlin.de Wed May 23 14:15:36 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Wed, 23 May 2018 16:15:36 +0200 Subject: Another failure on linux-sparc In-Reply-To: References: <2b15f2ca-ea28-44bf-3f39-0d0f1c23a956@physik.fu-berlin.de> Message-ID: <49d0ec47-324c-1ea4-f23b-e25770667b59@physik.fu-berlin.de> On 05/23/2018 04:07 PM, Aleksey Shipilev wrote: > On 05/23/2018 04:02 PM, John Paul Adrian Glaubitz wrote: >> Anyone seen this already on other configurations? > > I saw this today on x86_64/zero, and it seem Zero-specific: > https://builds.shipilev.net/openjdk-jdk/openjdk-jdk-b215-x86_64-zero-fastdebug.build.log > > ...but, curiously, it was not reproducible with my local builds. I decided to wait another nightly > before diving in. FWIW, I am not sure why share/opto is even built when compiler2 is disabled. Shall I file a bug report for that? Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From shade at redhat.com Wed May 23 14:17:49 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 23 May 2018 16:17:49 +0200 Subject: Another failure on linux-sparc In-Reply-To: <49d0ec47-324c-1ea4-f23b-e25770667b59@physik.fu-berlin.de> References: <2b15f2ca-ea28-44bf-3f39-0d0f1c23a956@physik.fu-berlin.de> <49d0ec47-324c-1ea4-f23b-e25770667b59@physik.fu-berlin.de> Message-ID: <9f77c687-ed22-bf21-07a6-027d1c700ce5@redhat.com> On 05/23/2018 04:15 PM, John Paul Adrian Glaubitz wrote: > On 05/23/2018 04:07 PM, Aleksey Shipilev wrote: >> On 05/23/2018 04:02 PM, John Paul Adrian Glaubitz wrote: >>> Anyone seen this already on other configurations? >> >> I saw this today on x86_64/zero, and it seem Zero-specific: >> https://builds.shipilev.net/openjdk-jdk/openjdk-jdk-b215-x86_64-zero-fastdebug.build.log >> >> ...but, curiously, it was not reproducible with my local builds. I decided to wait another nightly >> before diving in. FWIW, I am not sure why share/opto is even built when compiler2 is disabled. > > Shall I file a bug report for that? Yeah, if it reproduces. But I would probably tried for a fix before doing this, because maybe the real fix is in build system, not in the C2 code. -Aleksey From glaubitz at physik.fu-berlin.de Wed May 23 14:20:56 2018 From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz) Date: Wed, 23 May 2018 16:20:56 +0200 Subject: Another failure on linux-sparc In-Reply-To: <9f77c687-ed22-bf21-07a6-027d1c700ce5@redhat.com> References: <2b15f2ca-ea28-44bf-3f39-0d0f1c23a956@physik.fu-berlin.de> <49d0ec47-324c-1ea4-f23b-e25770667b59@physik.fu-berlin.de> <9f77c687-ed22-bf21-07a6-027d1c700ce5@redhat.com> Message-ID: On 05/23/2018 04:17 PM, Aleksey Shipilev wrote: > On 05/23/2018 04:15 PM, John Paul Adrian Glaubitz wrote: >> Shall I file a bug report for that? > > Yeah, if it reproduces. But I would probably tried for a fix before doing this, because maybe the > real fix is in build system, not in the C2 code. Ok. I can wait a little but I cannot do any work myself at the moment, especially with the SPARC T5 being down ... -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaubitz at debian.org `. `' Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 From adinn at redhat.com Wed May 23 14:32:32 2018 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 23 May 2018 15:32:32 +0100 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <0cf68c20-51d3-6754-8837-698d120ebf53@redhat.com> <759bf969-1aed-852b-1e9d-5d96178086f9@redhat.com> <4661377e-a8ef-9844-18d5-40dfe607d616@redhat.com> Message-ID: <375ed44d-ad35-4074-75b6-c881ed5208fe@redhat.com> Hi David, There is no need to worry about nestmates here -- the problem is (very) real but lies elsewhere. First things first: the disparity between my test results and Aleksey's is because I tested a release build and he tested a fastdebug build. Now as to that: arguably, Aleksey is right to test with debug enabled so as to catch conditions innocuous to the test but still unexpected arguably, I am right to test what will end up being released because, err ... well, that's what actually needs to work I guess we really need to test both. Secondly, test SpecialInterfaceCall only failed because debug code was exercised (in code guarded by "if (VerifyfMethodHandles)") yet the failure was actually in non-debug code called under that branch. So, it is quite an important error to have found. To be more sepcific: The AArch64 version of MacroAssembler::check_klass_subtype_slow_path has had this error in it from when it was created. The default assumption is that r0 holds a pointer to the super class -- on x86 the equivalent register is rax. If a different register is passed in then it needs to be moved into r0, after saving the value stored there. The x86 implementation plants a push of rax onto followed by a mov from the input register to rax. On AArch64 the equivalent code always did the push but also always omitted the mov. It is probably worth providing a post mortem to understand why has this not been caught up until now. Firstly, subclass checks planted in other stubs seem always to have the super in r0 so the error does't show elsewhere. The method handle invocation stub uses r11 for the super but the debug code is only planted there and only executes the super check if VerifyfMethodHandles is set. So, it won't show in a release build. Also, the error only trips you up if the fast super check fails and a super search is needed. So, it won't show unless you can foil the super caching code by trying an invoke via one interface and then another. I guess SpecialInterfaceCall is the first indy test where this combination of events has arisen in a fastdbeug build. I am left wondering how many other 5 year old bugs we still have in the AArch64 code base? regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From adinn at redhat.com Wed May 23 16:54:03 2018 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 23 May 2018 17:54:03 +0100 Subject: RFR: AArch64: 8203699 java/lang/invoke/SpecialInterfaceCall fails with SIGILL on aarch64 Message-ID: Patch: The patch below against the latest jdk dev tree fixes an AArch64-only problem that currently only manifests in debug builds, specifically for jdk test java/lang/invoke/SpecialInterfaceCall. It corrects a long-standing error in the (generated) subclass check slow path code where a super presented in a register other than r0 fails to get copied into r0 after pushing the current value. n.b. the slight difference to x86 code is because of the need to collect registers for pushing to stack as pairs (the AArch64 stack must remain 16-byte aligned so an auxiliary slot storing zr may be pushed and popped). The need to cater for that requirement probably accounts for why the register-register copy got overlooked (despite it being pointed out as necessary in the immediately preceding comment :-). JIRA: http://bugs.openjdk.java.net/browse/JDK-8203699 webrev: http://cr.openjdk.java.net/~adinn/8203699/webrev.00/ Reviews would be welcome. n.b.b. I am on holiday from noon tomorrow. So, if this does not get reviewed in time to commit perhaps someone else from Red Hat could pick it up and see it through review to commit (yes, Aleksey, I guess that means you :-). Testing: Run test/java/lang/invoke/SpecialInterfaceCall before the patch and it fails. Run it after the patch and it works. This is an AArch64-only change so it cannot and need not be tested via the submit repo. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From shade at redhat.com Wed May 23 17:28:54 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 23 May 2018 19:28:54 +0200 Subject: RFR: AArch64: 8203699 java/lang/invoke/SpecialInterfaceCall fails with SIGILL on aarch64 In-Reply-To: References: Message-ID: On 05/23/2018 06:54 PM, Andrew Dinn wrote: > JIRA: http://bugs.openjdk.java.net/browse/JDK-8203699 > > webrev: http://cr.openjdk.java.net/~adinn/8203699/webrev.00/ Looks good. It passes the test for me, on the platform where it originally failed. > n.b.b. I am on holiday from noon tomorrow. So, if this does not get > reviewed in time to commit perhaps someone else from Red Hat could pick > it up and see it through review to commit (yes, Aleksey, I guess that > means you :-). Sure, I can push if you wouldn't. -Aleksey From adinn at redhat.com Wed May 23 19:34:53 2018 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 23 May 2018 20:34:53 +0100 Subject: RFR: AArch64: 8203699 java/lang/invoke/SpecialInterfaceCall fails with SIGILL on aarch64 In-Reply-To: References: Message-ID: <887ea79a-86d6-8763-05c8-9e4cce13310c@redhat.com> On 23/05/18 18:28, Aleksey Shipilev wrote: > On 05/23/2018 06:54 PM, Andrew Dinn wrote: >> JIRA: http://bugs.openjdk.java.net/browse/JDK-8203699 >> >> webrev: http://cr.openjdk.java.net/~adinn/8203699/webrev.00/ > > Looks good. > > It passes the test for me, on the platform where it originally failed. Thanks for the review Aleksey. Anyone else? >> n.b.b. I am on holiday from noon tomorrow. So, if this does not get >> reviewed in time to commit perhaps someone else from Red Hat could pick >> it up and see it through review to commit (yes, Aleksey, I guess that >> means you :-). > > Sure, I can push if you wouldn't. Excellent, thanks. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From igor.ignatyev at oracle.com Wed May 23 19:44:32 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 23 May 2018 12:44:32 -0700 Subject: RFR(M) : 8199380 : [TESTBUG] Open source VM testbase AOD tests Message-ID: <5E87B35F-FC4E-4814-A4A6-CE1E24D3C479@oracle.com> http://cr.openjdk.java.net/~iignatyev/8199380/webrev.00/ > 2370 lines changed: 2370 ins; 0 del; 0 mod; Hi all, could you please review this patch which open sources AOD tests from VM testbase? these tests test hotspot's attach-on-demand. As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. JBS: https://bugs.openjdk.java.net/browse/JDK-8199380 webrev: http://cr.openjdk.java.net/~iignatyev//8199380/webrev.00/index.html testing: :vmTestbase_vm_aod test group Thanks, -- Igor From erik.joelsson at oracle.com Wed May 23 19:53:31 2018 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 23 May 2018 12:53:31 -0700 Subject: RFR(M) : 8199380 : [TESTBUG] Open source VM testbase AOD tests In-Reply-To: <5E87B35F-FC4E-4814-A4A6-CE1E24D3C479@oracle.com> References: <5E87B35F-FC4E-4814-A4A6-CE1E24D3C479@oracle.com> Message-ID: Build changes look good. /Erik On 2018-05-23 12:44, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8199380/webrev.00/ >> 2370 lines changed: 2370 ins; 0 del; 0 mod; > Hi all, > > could you please review this patch which open sources AOD tests from VM testbase? these tests test hotspot's attach-on-demand. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199380 > webrev: http://cr.openjdk.java.net/~iignatyev//8199380/webrev.00/index.html > testing: :vmTestbase_vm_aod test group > > Thanks, > -- Igor From david.holmes at oracle.com Wed May 23 21:39:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 24 May 2018 07:39:15 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <9a8eadf0-e745-ee38-fb37-b29d99fa96c0@bell-sw.com> References: <53feed71-0ab9-7743-6034-3c95bb3b20e8@bell-sw.com> <848b45e0-df51-7d74-e4eb-60819d101330@oracle.com> <9a8eadf0-e745-ee38-fb37-b29d99fa96c0@bell-sw.com> Message-ID: <71dbee89-451e-a151-474a-74326a2fa1c6@oracle.com> Hi Boris, Thanks for verifying my changes. In case you didn't see you can apply the updated patch to my v1 changes from: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2/ One follow up below. On 23/05/2018 11:40 PM, Boris Ulasevich wrote: > Hi David, > > some minor comments below.. > > On 23.05.2018 09:57, David Holmes wrote: >> Hi Boris, >> >> On 17/05/2018 7:23 PM, Boris Ulasevich wrote: >>> Hi David, >>> >>> You are right! My bad, R2_tmp parameter conflicts with Rklass on >>> check_klass_subtype(..) call. See correct patch in attach. Now all >>> runtime/Nestmates tests passed! :) >> >> I went to aplpy your patch and found it's not a diff against my patch >> but against the original non-nestmate code, > > Yes, sorry. For me it was natural to apply patch from your review to > local repo, rework it and just send updated diff from my machine :) > >> so I want to be clear on the changes. AFAICS the differences are: >> >> -? const Register Rklass? = R3_tmp; >> +? const Register Rklass? = R2_tmp; // Note! Same register with Rrecv > > Yes. > >> This initially concerned me as we stomp on Rrecv when we do the >> load_klass, but then you moved? the: >> >> ??? __ load_klass(Rklass, Rrecv); >> >> to after the object case, which used Rrecv. I had assumed Rrecv was >> somehow used when we actually do the call, but I'm assuming >> jump_from_interpreted is done in such as way that the receiver is >> still available on the interpreter stack and is used from there. > > Ok. I'm not sure I understand you well. When I moved load_klass call > down my point was to skip unnecessary load for the notObjectMethod case > (Rklass is not required in this case) and to make possible to reuse same > R2 register by both Rklass and Rrecv register. Right. My concern was an assumption that as Rrecv was the receiver object that we had to leave it intact ready for the actual method invocation. But that isn't the case. The real method invocation happens elsewhere and the receiver is obtained by other means. Thanks, David >> -? __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >> +? __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R3_tmp, noreg, >> subtype); >> >> Okay - reworking of tmp regs. > > Yes, ARM32 requires one more tmp reg, and we can't spoil R0. > >> -? __ jump_from_interpreted(Rindex); >> +? __ jump_from_interpreted(Rmethod); >> >> I'm going to trust this is okay :) It's not at all clear to me how the >> f2 Method* gets passed through on different platforms - sometimes its >> in the "index" and sometimes the "method" registers. > > I see. On ARM/AARCH64 we have preallocated register Rmethod/rmethod to > hold current method pointer and prepare_invoke call to setup registers > properly. > >> (I _really_ wish there was consistency in terminology across the >> different platforms - this code is awful for trying to compare the >> different platforms to figure out what to do on a new one.) >> >> Thanks, >> David >> >>> regards, >>> Boris >>> >>> On 17.05.2018 11:24, David Holmes wrote: >>>> On 17/05/2018 6:13 PM, Boris Ulasevich wrote: >>>>> Hi David, >>>>> >>>>> I see three tests failing: >>>>> ?> NullPointerException at >>>>> TestInterfaceMethodSelection.doInvoke(TestInterfaceMethodSelection.java:191) >>>>> >>>>> ?> NullPointerException at TestInvoke.access_priv(TestInvoke.java:54) >>>>> ?> InvocationTargetException at >>>>> TestReflection.access_priv(TestReflection.java:61) >>>>> >>>>> I will send you test details in a separate mail. >>>> >>>> Ok. This indicates a bug in the assembly code. The NPE's will likely >>>> be SEGVs caused by a zero register. >>>> >>>> Thanks, >>>> David >>>> >>>> >>>>> Boris >>>>> >>>>> On 17.05.2018 00:23, David Holmes wrote: >>>>>> Hi Boris, >>>>>> >>>>>> Many thanks for looking at this and working through the ARM code. >>>>>> >>>>>> I will study your patch in detail but am concerned by the "passes >>>>>> most of runtime/Nestmates tests Ok."! What tests are not passing? >>>>>> >>>>>> David >>>>>> >>>>>> On 17/05/2018 1:05 AM, Boris Ulasevich wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> Let us look to the change in templateTable_arm.cpp. I have >>>>>>> several notes. >>>>>>> >>>>>>> 1. We have compilation error because check_klass_subtype call >>>>>>> needs one more temporary register parameter. I think correct >>>>>>> change is: >>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >>>>>>> -> >>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, >>>>>>> subtype); >>>>>>> >>>>>>> 2. R0_tmp holds mdp used for profilation several lines below, so >>>>>>> we can't spoil it. I think correct change is: >>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, >>>>>>> subtype); >>>>>>> -> >>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R2_tmp, noreg, >>>>>>> subtype); >>>>>>> >>>>>>> 3. We can't jump to Rindex. I believe it was supposed to jump to >>>>>>> Rmethod: >>>>>>> jump_from_interpreted(Rindex); >>>>>>> -> >>>>>>> jump_from_interpreted(Rmethod); >>>>>>> >>>>>>> 4. Please note than Rklass and Rflags reuses same register. >>>>>>> Before the change their life ranges had no intersection. I would >>>>>>> propose to use R2 for Rklass (same with Rrecv) and move >>>>>>> load_klass call after invokevirtual_helper call to avoid life >>>>>>> range intersecton. >>>>>>> >>>>>>> See my patch against original templateTable_arm.cpp version in >>>>>>> attach - with this change ARM build compiles and passes most of >>>>>>> runtime/Nestmates tests Ok. >>>>>>> >>>>>>> regards, >>>>>>> Boris >>>>>>> >>>>>>> On 15.05.2018 03:52, David Holmes wrote: >>>>>>>> This review is being spread across four groups: langtools, >>>>>>>> core-libs, hotspot and serviceability. This is the specific >>>>>>>> review thread for hotspot - webrev: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>>>>>> >>>>>>>> >>>>>>>> See below for full details - including annotated full webrev >>>>>>>> guiding the review. >>>>>>>> >>>>>>>> The intent is to have JEP-181 targeted and integrated by the end >>>>>>>> of this month. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>> The nestmates project (JEP-181) introduces new classfile >>>>>>>> attributes to identify classes and interfaces in the same nest, >>>>>>>> so that the VM can perform access control based on those >>>>>>>> attributes and so allow direct private access between nestmates >>>>>>>> without requiring javac to generate synthetic accessor methods. >>>>>>>> These access control changes also extend to core reflection and >>>>>>>> the MethodHandle.Lookup contexts. >>>>>>>> >>>>>>>> Direct private calls between nestmates requires a more general >>>>>>>> calling context than is permitted by invokespecial, and so the >>>>>>>> JVMS is updated to allow, and javac updated to use, >>>>>>>> invokevirtual and invokeinterface for private class and >>>>>>>> interface method calls respectively. These changed semantics >>>>>>>> also extend to MethodHandle findXXX operations. >>>>>>>> >>>>>>>> At this time we are only concerned with static nest definitions, >>>>>>>> which map to a top-level class/interface as the nest-host and >>>>>>>> all its nested types as nest-members. >>>>>>>> >>>>>>>> Please see the JEP for further details. >>>>>>>> >>>>>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>>>>>> >>>>>>>> All of the specification changes have been previously been >>>>>>>> worked out by the Valhalla Project Expert Group, and the >>>>>>>> implementation reviewed by the various contributors and >>>>>>>> discussed on the valhalla-dev mailing list. >>>>>>>> >>>>>>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>>>>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen >>>>>>>> Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, >>>>>>>> Kumar Srinivasan >>>>>>>> >>>>>>>> Master webrev of all changes: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>>>>>> >>>>>>>> Annotated master webrev index: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>>>>>> >>>>>>>> >>>>>>>> Performance: this is expected to be performance neutral in a >>>>>>>> general sense. Benchmarking and performance runs are about to >>>>>>>> start. >>>>>>>> >>>>>>>> Testing Discussion: >>>>>>>> ------------------ >>>>>>>> >>>>>>>> The testing for nestmates can be broken into four main groups: >>>>>>>> >>>>>>>> -? New tests specifically related to nestmates and currently in >>>>>>>> the runtime/Nestmates directory >>>>>>>> >>>>>>>> - New tests to complement existing tests by adding in testcases >>>>>>>> not previously expressible. >>>>>>>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java >>>>>>>> tests use of invokespecial for private interface methods and >>>>>>>> performing receiver typechecks, so we add >>>>>>>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests >>>>>>>> for invokeinterface. >>>>>>>> >>>>>>>> -? New JVM TI tests to verify the spec changes related to nest >>>>>>>> attributes. >>>>>>>> >>>>>>>> -? Existing tests significantly affected by the nestmates >>>>>>>> changes, primarily: >>>>>>>> ??? -? runtime/SelectionResolution >>>>>>>> >>>>>>>> ??? In most cases the nestmate changes makes certain invocations >>>>>>>> that were illegal, legal (e.g. not requiring invokespecial to >>>>>>>> invoke private interface methods; allowing access to private >>>>>>>> members via reflection/Methodhandles that were previously not >>>>>>>> allowed). >>>>>>>> >>>>>>>> - Existing tests incidentally affected by the nestmate changes >>>>>>>> >>>>>>>> ?? This includes tests of things utilising class >>>>>>>> redefinition/retransformation to alter nested types but which >>>>>>>> unintentionally alter nest relationships (which is not permitted). >>>>>>>> >>>>>>>> There are still a number of tests problem-listed with issues >>>>>>>> filed against them to have them adapted to work with nestmates. >>>>>>>> Some of these are intended to be addressed in the short-term, >>>>>>>> while some (such as the runtime/SelectionResolution test >>>>>>>> changes) may not eventuate. >>>>>>>> >>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>>>>>> >>>>>>>> There is also further test work still to be completed (the JNI >>>>>>>> and JDI invocation tests): >>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>>>>>> which will continue in parallel with the main RFR. >>>>>>>> >>>>>>>> Pre-integration Testing: >>>>>>>> ??- General: >>>>>>>> ???? - Mach5: hs/jdk tier1,2 >>>>>>>> ???? - Mach5: hs-nightly (tiers 1 -3) >>>>>>>> ??- Targetted >>>>>>>> ??? - nashorn (for asm changes) >>>>>>>> ??? - hotspot: runtime/* >>>>>>>> ?????????????? serviceability/* >>>>>>>> ?????????????? compiler/* >>>>>>>> ?????????????? vmTestbase/* >>>>>>>> ??? - jdk: java/lang/invoke/* >>>>>>>> ?????????? java/lang/reflect/* >>>>>>>> ?????????? java/lang/instrument/* >>>>>>>> ?????????? java/lang/Class/* >>>>>>>> ?????????? java/lang/management/* >>>>>>>> ?? - langtools: tools/javac >>>>>>>> ??????????????? tools/javap >>>>>>>> From david.holmes at oracle.com Wed May 23 21:56:38 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 24 May 2018 07:56:38 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <375ed44d-ad35-4074-75b6-c881ed5208fe@redhat.com> References: <0cf68c20-51d3-6754-8837-698d120ebf53@redhat.com> <759bf969-1aed-852b-1e9d-5d96178086f9@redhat.com> <4661377e-a8ef-9844-18d5-40dfe607d616@redhat.com> <375ed44d-ad35-4074-75b6-c881ed5208fe@redhat.com> Message-ID: Hi Andrew, On 24/05/2018 12:32 AM, Andrew Dinn wrote: > Hi David, > > There is no need to worry about nestmates here -- the problem is (very) > real but lies elsewhere. Thanks - that's a relief for me. :) I'm curious though why Aarch64 testing has not caught this before now? The update to SpecialInterfaceCall.java (which I believe caught this) was pushed over 3 weeks ago. David > First things first: the disparity between my test results and Aleksey's > is because I tested a release build and he tested a fastdebug build. Now > as to that: > > arguably, Aleksey is right to test with debug enabled so as to catch > conditions innocuous to the test but still unexpected > > arguably, I am right to test what will end up being released because, > err ... well, that's what actually needs to work > > I guess we really need to test both. > > Secondly, test SpecialInterfaceCall only failed because debug code was > exercised (in code guarded by "if (VerifyfMethodHandles)") yet the > failure was actually in non-debug code called under that branch. So, it > is quite an important error to have found. To be more sepcific: > > The AArch64 version of MacroAssembler::check_klass_subtype_slow_path > has had this error in it from when it was created. The default > assumption is that r0 holds a pointer to the super class -- on x86 the > equivalent register is rax. If a different register is passed in then it > needs to be moved into r0, after saving the value stored there. The x86 > implementation plants a push of rax onto followed by a mov from the > input register to rax. On AArch64 the equivalent code always did the > push but also always omitted the mov. > > It is probably worth providing a post mortem to understand why has this > not been caught up until now. Firstly, subclass checks planted in other > stubs seem always to have the super in r0 so the error does't show > elsewhere. The method handle invocation stub uses r11 for the super but > the debug code is only planted there and only executes the super check > if VerifyfMethodHandles is set. So, it won't show in a release build. > Also, the error only trips you up if the fast super check fails and a > super search is needed. So, it won't show unless you can foil the super > caching code by trying an invoke via one interface and then another. > > I guess SpecialInterfaceCall is the first indy test where this > combination of events has arisen in a fastdbeug build. I am left > wondering how many other 5 year old bugs we still have in the AArch64 > code base? > > regards, > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander > From gil at azul.com Wed May 23 23:00:49 2018 From: gil at azul.com (Gil Tene) Date: Wed, 23 May 2018 23:00:49 +0000 Subject: OpenJDK G1 Patch In-Reply-To: References: Message-ID: <94139F97-F807-47AB-99CD-4B6C74C68BDC@azul.com> > On May 23, 2018, at 3:35 PM, Michal Frajt wrote: > > > > Doing continual hot-gets on weak references and expecting them to be cleaned up anyway is an application logic bug, and a complete mis-reading of the spec. > What means hot-get? The weak reference have the public method get to be called, what else? The new garbage collectors (especially C4) will never clear the weak reference just because the application code allowed itself to call the get method once within a GC major cycle (minutes interval). Do you think that calling the get method once per 5 minutes can be called hot? Within the 5 minutes a Java application can access terabytes of heap data but should not touch any weak reference, is this what you say? Not sure why you think concurrent oldgen collections take 5 minutes. It's more like seconds for most large-heap applications. Maybe a few tens of seconds if you have a 500GB live set. A few minutes happens only for TBs of live set. "hot-get" in this context will literally mean "access the every weak reference at least once during any N milliseconds period, forever" where N is the length, in milliseconds, of a concurrent mark of your live set. Each weak reference that [individually] has that behavior is being explicitly kept strong by the application. And if you do it to ALL your weak references (meaning , you are keeping them all alive by accessing every single one of them at least once within an N millisecond period), none of them will be cleared. Any reference for which "you give it a rest" for even one GC cycle will be cleared. In the tests I am aware of where you were highlighting this issue in the past, each and every weakref involved was being accesses several times a second. > > > Weak references are only cleared if their referent is determined to be weakly reachable. > Then do it! The 0.999999999% of the C4 major cycle time many weak reference referents are weakly reachable but still never cleared. Snapshot-At-The-Beginning (SATB) markers, like G1 and Shenandoah use, define "live" as any object that was reachable at the start of the marking, or became reachable (through e.g. allocation or reachability strengthening) during the marking. Precise-wavefront markers, like C4 and ZGC use, define "live" as a smaller, more precise set, but still (by definition) treat any object that was observed to be strongly reachable at any point during the mark as "live". Both definitions fully fit with the Java WeakReference and SoftReference behavior definitions. Those fundamental definitions are inherent to the way the collection algorithms work, and strongly relied on in many places. E.g. Precise-wavefront collectors leverage this definition to provably eliminate the potential for propagating strong references to not-yet-marked objects, and deliver guaranteed-to-complete single pass marking at a result. E.g. SATB markers guarantee convergence of the marker working set and the eventual completion of the marking phase, with no need for a potentially-huge STW drain step (like CMS has). The collectors further use these qualities throughout their design to support various other invariants that are used in implementation. You are asking that all collectors created after 2001 change their basic algorithmic assumptions to match your 2001 design, which was based on the assumptions that WekaReference will display behaviors that it never said it would, and just happened to exist on the collectors at the time? If you really want that, you should propose a new type of Reference that guarantees the semantics you want. You could start by defining the required semantics for this new Reference type in JavaDoc form. We can then all debate the merit and implementability of such a thing. > > > But you are still building based on an assumption of a specific underlying implementation, and not on anything that the WeakReference documentation or anything else in the JavaDocs or Java spec allows you to assume. > Indeed. The weak reference usage for the application framework came in 2001 when we were looking for the C++ smart pointer like structure in Java. Nearly 20 years later it still works fine with the CMS collector. I've seen plenty of 2001 code that made all sorts of assumptions that were not valid, and broke over the years as new features, optimizations, and CPUs came out that made those invalid assumptions surface as latent application logic bugs. This is just one more example of that. I've seen code that assumed biased locking or lock coarsening would not happen. And code that assumed that non-volatile loads or stores could not be hoisted out of or sunk past loops. And code that assumed that classes would never load in parallel. And code that assumed that object headers and reference fields take up a specific number of bytes. I've seen code that assumes that "this" is reachable through the end of an instance method. And code that assumes that escape analysis cannot eliminate the ordering they thought a synchronized block implied. All that code may still work on 2001 JVMs running on 2001 OSs using 2001 CPUs. And it might even work if you run it with -Xint on modern hardware and Java 8. But most of it breaks on current JVMs run with default flags. All we can say to the complaints about the new stuff "breaking" patterns that rely on wrong assumptions is: RTFM. > It is only hard to accept that the latest G1 collector will only increase our STW times and complicate weak references handling. It's not the latest version that "adds" this WeakRef-get-prevents-clearing-in-this-cycle behavior. It's been in all versions of G1 that I know of. > Where is the progress? Is there another smart pointer like handling in Java? No, only weak reference was very close to it. Weak references are still as powerful as they ever were. They do exactly what they say they would, and there are tons of useful patterns that use them correctly. Note that AFAICT, there is no live-set or OOM problem here per-se. Non of the JVMs and collectors I know of should crash in the presence of such hot-get access patterns on all weakrefs. They would just observe an increased live set to the point where empty memory after collections get small enough and oldgen GC cycles grow long enough that concurrency can no longer be maintained (e.g. the rate of garbage collected cannot keep up with the rate of allocation or promotion, or some other earlier trigger in e.g. G1). At that point, your application would simply experience a pause, which would "helpfully" stall it's forward progress until memory was freed. That stall will then prevent you from hot-getting the weakrefs, which will in turn allow the next cycle to clear them, and free up all that memory you were keeping alive by hot-getting the weakrefs. You'd then get back to the level of empty memory you are probably wishing for, at the cost of a STW pause, which is what you had in 2001 to begin with. No harm no foul. Now if someone was hoping for some other semantic benefits from non-guaranteed early-death, e.g. like enqueuing the cleared references "quickly" so they could act on them as events in a timely manner, they were building an even deeper logic chain on a wrong assumption. > > Michal > > Od: "Gil Tene" gil at azul.com > Komu: "Kirk Pepperdine" kirk.pepperdine at gmail.com > Kopie: "Michal Frajt" michal at frajt.eu,"hotspot-dev at openjdk.java.net" hotspot-dev at openjdk.java.net > Datum: Wed, 23 May 2018 11:32:11 +0000 > P?edmet: Re: OpenJDK G1 Patch > > > > > On May 23, 2018, at 11:38 AM, Kirk Pepperdine wrote: > > > > > >> On May 23, 2018, at 9:36 AM, Michal Frajt wrote: > >> > >> Hi Kirk, > >> > >> I?m just on the back end of the Oracle Dev tour in Japan? Once I land I?ll work on the test case to hopefully sort out what is happening with references and G1 (if this is indeed the cause of the grief I?ve been trying to sort through) and I do plan on sharing the results. > >> - please share with us your finding, we would like to know how the G1 handles processing of weak references which are across all regions, we simply need to get them cleaned within a certain time at least (our CMSTriggerInterval) > >> - we have had similar issue with the Azul C4 where the weak references behave(d) very differently, when you just shortly access a weak reference referent, the weak reference won?t be cleaned in the next C4 major cycle (like minutes interval), so you have to stop accessing it which goes completely against the weak reference (or soft reference) idea > > > > Indeed, this feels like a bug.. certainly doesn?t meet WeakReference definitions as I understand it. > > That *is* the WeakReference definition. Weak references are only cleared if their referent is determined to > be weakly reachable. Doing a WeakReference.get() produces a strong reference, making the referent > strongly reachable, and by-definition preventing the weak reference from being cleared until that strong > reference is itself proven to be unreachable. Doing WeakReference.get() concurrently with collection activity > will prevent that weak reference from being cleared in that cycle. Doing so continually and concurrently > with all future collection activity has an obvious and well defined effect of preventing the reference from > ever being cleared. > > Doing continual hot-gets on weak references and expecting them to be cleaned up anyway is an application > logic bug, and a complete mis-reading of the spec. You do have options for making stuff like this work by > relying on (unspecified and non-guaranteed) GC behavior, like STW pauses. But these options generally rely > on *something* stopping your hot-get activity. You can bake-in relying on STW collections to stall your access > to weak references so they can complete both reachability analysis and clearing without you having a > chance to change things in the middle, but then you are stuck with STW pauses. When the collector doesn't > stop you when determining reachability, you also lose that "benefit" of pausing your eager-getting behavior. > You can try to bake-in some other assumptions (like "CMS will only see the strong reference if I write it to > memory, or if it is still on my stack during it's shorter STW pause", but you are still building based on an > assumption of a specific underlying implementation, and not on anything that the WeakReference > documentation or anything else in the JavaDocs or Java spec allows you to assume. Doing that is no > different than e.g. writing a cool concurrent algorithm that relies on an assumptions that machines have > 2 cores or less, and then hitting the reality of 3-way races on machines with 3+ cores when those become > commonplace. > > C4 and G1 (and ZGC and Shenandoah) all treat WeakRefs the same [well established and correct] way > in this regard AFAIK. > > > > > > > > >> > >> Yes, I understand but the ParNew before the CMS cycle has typically proven to be a huge loss overall. > >> - our finding is exactly opposite, you should run ParNew before the initial marking is started > >> - I had to even patch Hotspot http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2012-August/001297.html, https://bugs.openjdk.java.net/browse/JDK-7189971 > > Right, still a loss in every prod system where I?ve seen it being used. > > > >> > >> If you need to run frequent CMS cycles to keep tenured clean then it?s very very likely that you?re suffering from a premature promotion problem. We have tuning strategies to manage this situation that doesn?t require excessive triggering of CMS. > >> - we run frequent CMS cycles to keep the weak references cleaned, tenured getting cleaned is just a side effect > > > > I can?t say that I understand your use case but it sounds as if you *need* to have weak references cleaned frequently then again I?m thinking there is something in the design of your framework that is bothering me. If you need weak references you need weak references. That said, if you need them cleaned quickly then I have to ask why? > > > >> > >> Right, I can typically have JVMs run for weeks or longer without suffering from concurrent mode failures. It?s about controlling frequency and promotion rates. And it?s not so easy as the aging process is affected by the size of Eden. > >> - right, in one of our distribution layers I was evening thinking about accessing the object age via the Unsafe to re-create it right before it gets promoted so the object stays very likely in the young generation > >> - now when thinking more about it, I could actually just try reduce the age via the Unsafe, the object should stay longer in the survivor spaces, need to try > > > > Messing with the internal structures in this way seems like a gross violation? it feels worse than the triggering of a full. Again I?d suggest there is a tuning strategy to manage this. > > > > Again, this isn?t to say your are facing real issues and that these fixes don?t work for you. The question is; is it appropriate to drop in patches to solve your specific problem for you domain without regard to other use cases? If so, what if I drop in a patch that is detrimental to your applications performance? IMO, what is really missing here is a general due diligence that confirms that this solves problems in the broader population. That we have different experiences with ParNew before CMS InitialMark/Remark isn?t surprising to me. And, it?s not that I don?t trust your benchmarks, it?s that I don?t trust anyones benchmarks including my own unless they have gone through a serious level of vetting. Even then?... > >> > >> Regards, > >> Michal > >> > >> > >> Od: "Kirk Pepperdine" kirk.pepperdine at gmail.com > >> Komu: "Michal Frajt" michal at frajt.eu > >> Kopie: > >> Datum: Tue, 22 May 2018 19:11:12 +0200 > >> P?edmet: Re: OpenJDK G1 Patch > >> > >> > >>> On May 22, 2018, at 9:28 AM, Michal Frajt > wrote: > >>> > >>> > >>> Hi Kirk, > >>> > >>> - weak references were in 2001 the only alternative for C++ smart pointers like design and I guess still there is no other solution available > >> > >> I?m just on the back end of the Oracle Dev tour in Japan? Once I land I?ll work on the test case to hopefully sort out what is happening with references and G1 (if this is indeed the cause of the grief I?ve been trying to sort through) and I do plan on sharing the results. > >> > >>> - CMSTriggerInterval is not invoking the full collection cycle, it invokes normal concurrent CMS cycle (1 ParNew + 2 STW) so that there is at least one concurrent CMS cycle per the trigger interval (considering CMS runs due to CMSTriggerRatio and another reasons) > >> > >> Yes, I understand but the ParNew before the CMS cycle has typically proven to be a huge loss overall. > >> > >>> - we are fine with running regular concurrent CMS cycle which next to releasing the weak references keeps our old gen always clean having much higher chance to survive sudden increase in the promotion rate > >> > >> If you need to run frequent CMS cycles to keep tenured clean then it?s very very likely that you?re suffering from a premature promotion problem. We have tuning strategies to manage this situation that doesn?t require excessive triggering of CMS. > >> > >> > >>> - we never run the full cycle, or better we have never seen an end of the full cycle as applications are killed and restarted before it can finish > >> > >> Right, I can typically have JVMs run for weeks or longer without suffering from concurrent mode failures. It?s about controlling frequency and promotion rates. And it?s not so easy as the aging process is affected by the size of Eden. > >> > >> G1 pause times should cluster to some value that is approximately constant.. or should mostly band around a narrow range of values. To achieve this with minimal pause times and not risk suffering a full requires that you allocate more memory and a larger pause time goal. Unfortunately the larger heap sizes runs counter to your goal. However G1 was designed for big iron and I find that if you under-resource it you?re asking for high GC overhead and other troubles. For smaller heaps Parallel and/or CMS is still by far a better choice. > >> > >> Kind regards, > >> Kirk > >> > From igor.ignatyev at oracle.com Wed May 23 23:23:00 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 23 May 2018 16:23:00 -0700 Subject: RFR(L) : 8202812 : [TESTBUG] Open source VM testbase compiler tests In-Reply-To: References: Message-ID: <086B648F-7CBE-4AC6-9B3D-38B60518B143@oracle.com> a friendly reminder -- Igor > On May 18, 2018, at 10:50 AM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8202812/webrev.00/index.html >> 56733 lines changed: 56732 ins; 0 del; 1 mod; 1 > > Hi all, > > could you please review the patch which open sources compiler tests from vm testbase? these tests were developed in different time to cover different parts of JITs. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8202812 > webrev: http://cr.openjdk.java.net/~iignatyev/8202812/webrev.00/index.html > testing: :vmTestbase_vm_compiler test group > > Thanks, > -- Igor From coleen.phillimore at oracle.com Thu May 24 02:00:59 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 23 May 2018 22:00:59 -0400 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <8fe8a029-04f8-415a-efc6-3adef875255c@oracle.com> Message-ID: Hi David, replies below. On 5/23/18 12:07 AM, David Holmes wrote: > Hi Coleen, > > Thanks for looking at this! > > On 19/05/2018 8:36 AM, coleen.phillimore at oracle.com wrote: >> >> Hi, I haven't been following along or read through all of this, but I >> had a look through the code and have some questions about it. >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/src/hotspot/share/prims/jvmtiRedefineClasses.cpp.udiff.html >> >> >> There is commented out code at line 806 in >> compare_and_normalize_class_versions.?? Could you make the added code >> a separate function defined above so this isn't the longest function >> in the jvm? >> >> 739 JvmtiThreadState *state = >> JvmtiThreadState::state_for((JavaThread*)thread); >> 740 RedefineVerifyMark rvm(the_class, scratch_class, state); >> >> Why does this need RedefineVerifyMark?? It doesn't call the verifier. > > The above was moved to and answered on the serviceability-dev review > thread. > >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/src/hotspot/share/runtime/reflection.cpp.frames.html >> >> >> 720 bool access = cur_ik->has_nestmate_access_to(field_ik, THREAD); >> 721 if (HAS_PENDING_EXCEPTION) { >> 722 return false; >> 723 } >> >> This looks suspicious.?? Do you want to CLEAR_PENDING_EXCEPTION? This >> looks like the combination of these functions drops exceptions, that >> could show up in inconvenient places. > > There was a lot of changes to the way exceptions were handled as this > feature developed. The resulting code may well need some clean up ... > > The exceptions have to propagate so I just updated with: > > ?bool access = cur_ik->has_nestmate_access_to(field_ik, CHECK_false); > > Thanks. >> Or add TRAPS to reflection::verify_field_access so that the callers >> properly handle the pending exception? > > As Reflection::verify_field_access is a boolean function it gets used > in "if" expressions that make it unsuitable for use with TRAPS and > CHECK macros. I can either rewrite all those if's to introduce a local > variable assignment that is checked in the if, or else just do the > pending exception check explicitly. I chose the latter for simplicity > - and in one case (ciField) we need to clear the exception as well so > CHECK_ doesn't apply. Yes, I think this might be something that could be cleaned up. I didn't know if all the callers of verify_field_acess could handle a pending exception and it took a while to look at these to convince myself they weren't broken. > >> This is CHECK_false. and also maybe has_nestmate_access_to, should >> pass CHECK_false to its nest_host() calls, so it doesn't look so weird. > > So we currently have, for example: > > ? InstanceKlass* cur_host = nest_host(icce, THREAD); > ? if (cur_host == NULL || HAS_PENDING_EXCEPTION) { > ??? return false; > ? } > > and you're suggesting: > > ? InstanceKlass* cur_host = nest_host(icce, CHECK_false); > ? if (cur_host == NULL) { > ??? return false; > ? } > > I'm not really seeing a great difference in readability. Code-wise I > think the current code is slightly more efficient, but I'm not sure > about that. > I think a decent C++ optimizer could handle the two conditional statements.? This is another case that I had to follow because it looked suspicious. >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/src/hotspot/share/oops/instanceKlass.cpp.udiff.html >> >> >> + // names match so check actual klass - this may trigger class >> loading if >> + // it doesn't match (but that should be impossible) >> + Klass* k2 = _constants->klass_at(cp_index, CHECK_false); >> >> >> If throwing an exception is impossible, this should probably have a >> CATCH.? This is code is odd about exceptions.? It looks like >> has_nestmate_access() > > I _think_ it is impossible - it's a very odd situation and I could not > see how to construct a test that would hit this. But I don't know for > sure that it is impossible. So the code assumes classloading and thus > exceptions are possible. > Ok. >> >> + } >> + else { >> >> >> Oh please can you fix these?? The hotspot style is the? } else { all >> on one line (OTBS). > > Okay - 339 cases in hotspot as above versus 12913 as style guide > suggests :) I will fix my additions in instanceKlass.cpp. > Is that just this one file??? There must be more else statements in hotspot.? Thanks for fixing these. >> Since has_nest_member is only used in InstanceKlass, it should have >> private access. >> >> +? bool has_nest_member(InstanceKlass* k, TRAPS) const; > > True it could ... but I really prefer to have the nest functions > grouped together, and switching back and forth between private and > public would look messy. This is the one thing I like about Java.?? I've been adding public: private: around the functions that are private but still keeping them together.? If something is marked private, we know nothing should be calling it so it's good to have. > >> Looks like raw_nest_host() is uncalled (and shouldn't be public). > > It was a public debugging hook. Now removed again. > >> That's all I can do for now.? I think you have good reviewers for the >> feature itself, but if you want more, I'll spend more time with this. > Thanks, my review is complete. Coleen > Thanks again! > > David > >> thanks, >> Coleen >> >> On 5/14/18 8:52 PM, David Holmes wrote: >>> This review is being spread across four groups: langtools, >>> core-libs, hotspot and serviceability. This is the specific review >>> thread for hotspot - webrev: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>> >>> See below for full details - including annotated full webrev guiding >>> the review. >>> >>> The intent is to have JEP-181 targeted and integrated by the end of >>> this month. >>> >>> Thanks, >>> David >>> ----- >>> >>> The nestmates project (JEP-181) introduces new classfile attributes >>> to identify classes and interfaces in the same nest, so that the VM >>> can perform access control based on those attributes and so allow >>> direct private access between nestmates without requiring javac to >>> generate synthetic accessor methods. These access control changes >>> also extend to core reflection and the MethodHandle.Lookup contexts. >>> >>> Direct private calls between nestmates requires a more general >>> calling context than is permitted by invokespecial, and so the JVMS >>> is updated to allow, and javac updated to use, invokevirtual and >>> invokeinterface for private class and interface method calls >>> respectively. These changed semantics also extend to MethodHandle >>> findXXX operations. >>> >>> At this time we are only concerned with static nest definitions, >>> which map to a top-level class/interface as the nest-host and all >>> its nested types as nest-members. >>> >>> Please see the JEP for further details. >>> >>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>> >>> All of the specification changes have been previously been worked >>> out by the Valhalla Project Expert Group, and the implementation >>> reviewed by the various contributors and discussed on the >>> valhalla-dev mailing list. >>> >>> Acknowledgments and contributions: Alex Buckley, Maurizio >>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen >>> Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, >>> Kumar Srinivasan >>> >>> Master webrev of all changes: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>> >>> Annotated master webrev index: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>> >>> Performance: this is expected to be performance neutral in a general >>> sense. Benchmarking and performance runs are about to start. >>> >>> Testing Discussion: >>> ------------------ >>> >>> The testing for nestmates can be broken into four main groups: >>> >>> -? New tests specifically related to nestmates and currently in the >>> runtime/Nestmates directory >>> >>> - New tests to complement existing tests by adding in testcases not >>> previously expressible. >>> ? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>> use of invokespecial for private interface methods and performing >>> receiver typechecks, so we add >>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>> invokeinterface. >>> >>> -? New JVM TI tests to verify the spec changes related to nest >>> attributes. >>> >>> -? Existing tests significantly affected by the nestmates changes, >>> primarily: >>> ?? -? runtime/SelectionResolution >>> >>> ?? In most cases the nestmate changes makes certain invocations that >>> were illegal, legal (e.g. not requiring invokespecial to invoke >>> private interface methods; allowing access to private members via >>> reflection/Methodhandles that were previously not allowed). >>> >>> - Existing tests incidentally affected by the nestmate changes >>> >>> ? This includes tests of things utilising class >>> redefinition/retransformation to alter nested types but which >>> unintentionally alter nest relationships (which is not permitted). >>> >>> There are still a number of tests problem-listed with issues filed >>> against them to have them adapted to work with nestmates. Some of >>> these are intended to be addressed in the short-term, while some >>> (such as the runtime/SelectionResolution test changes) may not >>> eventuate. >>> >>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>> >>> There is also further test work still to be completed (the JNI and >>> JDI invocation tests): >>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>> which will continue in parallel with the main RFR. >>> >>> Pre-integration Testing: >>> ?- General: >>> ??? - Mach5: hs/jdk tier1,2 >>> ??? - Mach5: hs-nightly (tiers 1 -3) >>> ?- Targetted >>> ?? - nashorn (for asm changes) >>> ?? - hotspot: runtime/* >>> ????????????? serviceability/* >>> ????????????? compiler/* >>> ????????????? vmTestbase/* >>> ?? - jdk: java/lang/invoke/* >>> ????????? java/lang/reflect/* >>> ????????? java/lang/instrument/* >>> ????????? java/lang/Class/* >>> ????????? java/lang/management/* >>> ? - langtools: tools/javac >>> ?????????????? tools/javap >>> >> From david.holmes at oracle.com Thu May 24 03:01:43 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 24 May 2018 13:01:43 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <8fe8a029-04f8-415a-efc6-3adef875255c@oracle.com> Message-ID: <7d9b2882-aeaf-160c-2e5d-f2fd7bd53287@oracle.com> Hi Coleen, Okay I will make the changes you suggest. Once done I'll post a v3 RFR update (well once all v3 updates are in place). I'll need a v3 anyway to add in some test changes for tests that just got moved to open. Thanks, David On 24/05/2018 12:00 PM, coleen.phillimore at oracle.com wrote: > > Hi David, replies below. > > On 5/23/18 12:07 AM, David Holmes wrote: >> Hi Coleen, >> >> Thanks for looking at this! >> >> On 19/05/2018 8:36 AM, coleen.phillimore at oracle.com wrote: >>> >>> Hi, I haven't been following along or read through all of this, but I >>> had a look through the code and have some questions about it. >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/src/hotspot/share/prims/jvmtiRedefineClasses.cpp.udiff.html >>> >>> >>> There is commented out code at line 806 in >>> compare_and_normalize_class_versions.?? Could you make the added code >>> a separate function defined above so this isn't the longest function >>> in the jvm? >>> >>> 739 JvmtiThreadState *state = >>> JvmtiThreadState::state_for((JavaThread*)thread); >>> 740 RedefineVerifyMark rvm(the_class, scratch_class, state); >>> >>> Why does this need RedefineVerifyMark?? It doesn't call the verifier. >> >> The above was moved to and answered on the serviceability-dev review >> thread. >> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/src/hotspot/share/runtime/reflection.cpp.frames.html >>> >>> >>> 720 bool access = cur_ik->has_nestmate_access_to(field_ik, THREAD); >>> 721 if (HAS_PENDING_EXCEPTION) { >>> 722 return false; >>> 723 } >>> >>> This looks suspicious.?? Do you want to CLEAR_PENDING_EXCEPTION? This >>> looks like the combination of these functions drops exceptions, that >>> could show up in inconvenient places. >> >> There was a lot of changes to the way exceptions were handled as this >> feature developed. The resulting code may well need some clean up ... >> >> The exceptions have to propagate so I just updated with: >> >> ?bool access = cur_ik->has_nestmate_access_to(field_ik, CHECK_false); >> >> > Thanks. > > >>> Or add TRAPS to reflection::verify_field_access so that the callers >>> properly handle the pending exception? >> >> As Reflection::verify_field_access is a boolean function it gets used >> in "if" expressions that make it unsuitable for use with TRAPS and >> CHECK macros. I can either rewrite all those if's to introduce a local >> variable assignment that is checked in the if, or else just do the >> pending exception check explicitly. I chose the latter for simplicity >> - and in one case (ciField) we need to clear the exception as well so >> CHECK_ doesn't apply. > > Yes, I think this might be something that could be cleaned up. I didn't > know if all the callers of verify_field_acess could handle a pending > exception and it took a while to look at these to convince myself they > weren't broken. > >> >>> This is CHECK_false. and also maybe has_nestmate_access_to, should >>> pass CHECK_false to its nest_host() calls, so it doesn't look so weird. >> >> So we currently have, for example: >> >> ? InstanceKlass* cur_host = nest_host(icce, THREAD); >> ? if (cur_host == NULL || HAS_PENDING_EXCEPTION) { >> ??? return false; >> ? } >> >> and you're suggesting: >> >> ? InstanceKlass* cur_host = nest_host(icce, CHECK_false); >> ? if (cur_host == NULL) { >> ??? return false; >> ? } >> >> I'm not really seeing a great difference in readability. Code-wise I >> think the current code is slightly more efficient, but I'm not sure >> about that. >> > > I think a decent C++ optimizer could handle the two conditional > statements.? This is another case that I had to follow because it looked > suspicious. >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/src/hotspot/share/oops/instanceKlass.cpp.udiff.html >>> >>> >>> + // names match so check actual klass - this may trigger class >>> loading if >>> + // it doesn't match (but that should be impossible) >>> + Klass* k2 = _constants->klass_at(cp_index, CHECK_false); >>> >>> >>> If throwing an exception is impossible, this should probably have a >>> CATCH.? This is code is odd about exceptions.? It looks like >>> has_nestmate_access() >> >> I _think_ it is impossible - it's a very odd situation and I could not >> see how to construct a test that would hit this. But I don't know for >> sure that it is impossible. So the code assumes classloading and thus >> exceptions are possible. >> > > Ok. > >>> >>> + } >>> + else { >>> >>> >>> Oh please can you fix these?? The hotspot style is the? } else { all >>> on one line (OTBS). >> >> Okay - 339 cases in hotspot as above versus 12913 as style guide >> suggests :) I will fix my additions in instanceKlass.cpp. >> > > Is that just this one file??? There must be more else statements in > hotspot.? Thanks for fixing these. > >>> Since has_nest_member is only used in InstanceKlass, it should have >>> private access. >>> >>> +? bool has_nest_member(InstanceKlass* k, TRAPS) const; >> >> True it could ... but I really prefer to have the nest functions >> grouped together, and switching back and forth between private and >> public would look messy. > > This is the one thing I like about Java.?? I've been adding public: > private: around the functions that are private but still keeping them > together.? If something is marked private, we know nothing should be > calling it so it's good to have. > >> >>> Looks like raw_nest_host() is uncalled (and shouldn't be public). >> >> It was a public debugging hook. Now removed again. >> >>> That's all I can do for now.? I think you have good reviewers for the >>> feature itself, but if you want more, I'll spend more time with this. >> > > Thanks, my review is complete. > Coleen > >> Thanks again! >> >> David >> >>> thanks, >>> Coleen >>> >>> On 5/14/18 8:52 PM, David Holmes wrote: >>>> This review is being spread across four groups: langtools, >>>> core-libs, hotspot and serviceability. This is the specific review >>>> thread for hotspot - webrev: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>> >>>> See below for full details - including annotated full webrev guiding >>>> the review. >>>> >>>> The intent is to have JEP-181 targeted and integrated by the end of >>>> this month. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> The nestmates project (JEP-181) introduces new classfile attributes >>>> to identify classes and interfaces in the same nest, so that the VM >>>> can perform access control based on those attributes and so allow >>>> direct private access between nestmates without requiring javac to >>>> generate synthetic accessor methods. These access control changes >>>> also extend to core reflection and the MethodHandle.Lookup contexts. >>>> >>>> Direct private calls between nestmates requires a more general >>>> calling context than is permitted by invokespecial, and so the JVMS >>>> is updated to allow, and javac updated to use, invokevirtual and >>>> invokeinterface for private class and interface method calls >>>> respectively. These changed semantics also extend to MethodHandle >>>> findXXX operations. >>>> >>>> At this time we are only concerned with static nest definitions, >>>> which map to a top-level class/interface as the nest-host and all >>>> its nested types as nest-members. >>>> >>>> Please see the JEP for further details. >>>> >>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>> >>>> All of the specification changes have been previously been worked >>>> out by the Valhalla Project Expert Group, and the implementation >>>> reviewed by the various contributors and discussed on the >>>> valhalla-dev mailing list. >>>> >>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen >>>> Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, >>>> Kumar Srinivasan >>>> >>>> Master webrev of all changes: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>> >>>> Annotated master webrev index: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>> >>>> Performance: this is expected to be performance neutral in a general >>>> sense. Benchmarking and performance runs are about to start. >>>> >>>> Testing Discussion: >>>> ------------------ >>>> >>>> The testing for nestmates can be broken into four main groups: >>>> >>>> -? New tests specifically related to nestmates and currently in the >>>> runtime/Nestmates directory >>>> >>>> - New tests to complement existing tests by adding in testcases not >>>> previously expressible. >>>> ? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>>> use of invokespecial for private interface methods and performing >>>> receiver typechecks, so we add >>>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>>> invokeinterface. >>>> >>>> -? New JVM TI tests to verify the spec changes related to nest >>>> attributes. >>>> >>>> -? Existing tests significantly affected by the nestmates changes, >>>> primarily: >>>> ?? -? runtime/SelectionResolution >>>> >>>> ?? In most cases the nestmate changes makes certain invocations that >>>> were illegal, legal (e.g. not requiring invokespecial to invoke >>>> private interface methods; allowing access to private members via >>>> reflection/Methodhandles that were previously not allowed). >>>> >>>> - Existing tests incidentally affected by the nestmate changes >>>> >>>> ? This includes tests of things utilising class >>>> redefinition/retransformation to alter nested types but which >>>> unintentionally alter nest relationships (which is not permitted). >>>> >>>> There are still a number of tests problem-listed with issues filed >>>> against them to have them adapted to work with nestmates. Some of >>>> these are intended to be addressed in the short-term, while some >>>> (such as the runtime/SelectionResolution test changes) may not >>>> eventuate. >>>> >>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>> >>>> There is also further test work still to be completed (the JNI and >>>> JDI invocation tests): >>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>> which will continue in parallel with the main RFR. >>>> >>>> Pre-integration Testing: >>>> ?- General: >>>> ??? - Mach5: hs/jdk tier1,2 >>>> ??? - Mach5: hs-nightly (tiers 1 -3) >>>> ?- Targetted >>>> ?? - nashorn (for asm changes) >>>> ?? - hotspot: runtime/* >>>> ????????????? serviceability/* >>>> ????????????? compiler/* >>>> ????????????? vmTestbase/* >>>> ?? - jdk: java/lang/invoke/* >>>> ????????? java/lang/reflect/* >>>> ????????? java/lang/instrument/* >>>> ????????? java/lang/Class/* >>>> ????????? java/lang/management/* >>>> ? - langtools: tools/javac >>>> ?????????????? tools/javap >>>> >>> > From david.holmes at oracle.com Thu May 24 09:48:52 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 24 May 2018 19:48:52 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <06529fc3-2eba-101b-9aee-2757893cb8fb@oracle.com> References: <06529fc3-2eba-101b-9aee-2757893cb8fb@oracle.com> Message-ID: Here are the further updates based on review comments and rebasing to get the vmTestbase updates for which some closed test changes now have to be applied to the open versions. Incremental hotspot webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3-incr/ Full hotspot webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3/ Change summary: test/hotspot/jtreg/ProblemList.txt - Exclude vmTestbase/nsk/stress/except/except004.java under 8203046 test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/BasicTest.java test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/PrivateMethodsTest.java - updated to work with new invokeinterface rules and nestmate changes - misc cleanups src/hotspot/share/runtime/reflection.?pp - rename verify_field_access to verify_member_access (it's always been mis-named and I nearly forgot to do this cleanup!) and rename field_class to member_class - add TRAPS to verify_member_access to allow use with CHECK macros src/hotspot/share/ci/ciField.cpp src/hotspot/share/classfile/classFileParser.cpp src/hotspot/share/interpreter/linkResolver.cpp - updated to use THREAD/CHECK with verify_member_access - for ciField rename thread to THREAD so it can be used with HAS_PENDING_EXCEPTION src/hotspot/share/oops/instanceKlass.cpp - use CHECK_false when calling nest_host() - fix indent near nestmate code src/hotspot/share/oops/instanceKlass.hpp - make has_nest_member private Thanks, David ----- On 23/05/2018 4:57 PM, David Holmes wrote: > Here are the updates so far in response to all the review comments. > > Incremental webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2-incr/ > > Full webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2/ > > Change summary: > > test/runtime/Nestmates/reflectionAPI/* > - moved to java/lang/reflect/Nestmates > > src/hotspot/cpu/arm/templateTable_arm.cpp > - Fixed ARM invocation logic as provided by Boris. > > src/hotspot/share/interpreter/linkResolver.cpp > - expanded comment regarding exceptions > - Removed leftover debugging code > > src/hotspot/share/oops/instanceKlass.cpp > - Removed FIXME comments > - corrected incorrect comment > - Fixed if/else formatting > > src/hotspot/share/oops/instanceKlass.hpp > - removed unused debug method > > src/hotspot/share/oops/klassVtable.cpp > - added comment by request of Karen > > src/hotspot/share/runtime/reflection.cpp > - Removed FIXME comments > - expanded comments in places > - used CHECK_false > > Thanks, > David > > On 15/05/2018 10:52 AM, David Holmes wrote: >> This review is being spread across four groups: langtools, core-libs, >> hotspot and serviceability. This is the specific review thread for >> hotspot - webrev: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >> >> See below for full details - including annotated full webrev guiding >> the review. >> >> The intent is to have JEP-181 targeted and integrated by the end of >> this month. >> >> Thanks, >> David >> ----- >> >> The nestmates project (JEP-181) introduces new classfile attributes to >> identify classes and interfaces in the same nest, so that the VM can >> perform access control based on those attributes and so allow direct >> private access between nestmates without requiring javac to generate >> synthetic accessor methods. These access control changes also extend >> to core reflection and the MethodHandle.Lookup contexts. >> >> Direct private calls between nestmates requires a more general calling >> context than is permitted by invokespecial, and so the JVMS is updated >> to allow, and javac updated to use, invokevirtual and invokeinterface >> for private class and interface method calls respectively. These >> changed semantics also extend to MethodHandle findXXX operations. >> >> At this time we are only concerned with static nest definitions, which >> map to a top-level class/interface as the nest-host and all its nested >> types as nest-members. >> >> Please see the JEP for further details. >> >> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >> >> All of the specification changes have been previously been worked out >> by the Valhalla Project Expert Group, and the implementation reviewed >> by the various contributors and discussed on the valhalla-dev mailing >> list. >> >> Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, >> Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir >> Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan >> >> Master webrev of all changes: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >> >> Annotated master webrev index: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >> >> Performance: this is expected to be performance neutral in a general >> sense. Benchmarking and performance runs are about to start. >> >> Testing Discussion: >> ------------------ >> >> The testing for nestmates can be broken into four main groups: >> >> -? New tests specifically related to nestmates and currently in the >> runtime/Nestmates directory >> >> - New tests to complement existing tests by adding in testcases not >> previously expressible. >> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests use >> of invokespecial for private interface methods and performing receiver >> typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do >> similar tests for invokeinterface. >> >> -? New JVM TI tests to verify the spec changes related to nest >> attributes. >> >> -? Existing tests significantly affected by the nestmates changes, >> primarily: >> ??? -? runtime/SelectionResolution >> >> ??? In most cases the nestmate changes makes certain invocations that >> were illegal, legal (e.g. not requiring invokespecial to invoke >> private interface methods; allowing access to private members via >> reflection/Methodhandles that were previously not allowed). >> >> - Existing tests incidentally affected by the nestmate changes >> >> ?? This includes tests of things utilising class >> redefinition/retransformation to alter nested types but which >> unintentionally alter nest relationships (which is not permitted). >> >> There are still a number of tests problem-listed with issues filed >> against them to have them adapted to work with nestmates. Some of >> these are intended to be addressed in the short-term, while some (such >> as the runtime/SelectionResolution test changes) may not eventuate. >> >> - https://bugs.openjdk.java.net/browse/JDK-8203033 >> - https://bugs.openjdk.java.net/browse/JDK-8199450 >> - https://bugs.openjdk.java.net/browse/JDK-8196855 >> - https://bugs.openjdk.java.net/browse/JDK-8194857 >> - https://bugs.openjdk.java.net/browse/JDK-8187655 >> >> There is also further test work still to be completed (the JNI and JDI >> invocation tests): >> - https://bugs.openjdk.java.net/browse/JDK-8191117 >> which will continue in parallel with the main RFR. >> >> Pre-integration Testing: >> ??- General: >> ???? - Mach5: hs/jdk tier1,2 >> ???? - Mach5: hs-nightly (tiers 1 -3) >> ??- Targetted >> ??? - nashorn (for asm changes) >> ??? - hotspot: runtime/* >> ?????????????? serviceability/* >> ?????????????? compiler/* >> ?????????????? vmTestbase/* >> ??? - jdk: java/lang/invoke/* >> ?????????? java/lang/reflect/* >> ?????????? java/lang/instrument/* >> ?????????? java/lang/Class/* >> ?????????? java/lang/management/* >> ?? - langtools: tools/javac >> ??????????????? tools/javap >> From adinn at redhat.com Thu May 24 10:07:28 2018 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 24 May 2018 11:07:28 +0100 Subject: RFR: AArch64: 8203699 java/lang/invoke/SpecialInterfaceCall fails with SIGILL on aarch64 In-Reply-To: <887ea79a-86d6-8763-05c8-9e4cce13310c@redhat.com> References: <887ea79a-86d6-8763-05c8-9e4cce13310c@redhat.com> Message-ID: <33316793-e031-3bbb-d8a1-b0e0c29e0e03@redhat.com> On 23/05/18 20:34, Andrew Dinn wrote: > On 23/05/18 18:28, Aleksey Shipilev wrote: >> On 05/23/2018 06:54 PM, Andrew Dinn wrote: >>> JIRA: http://bugs.openjdk.java.net/browse/JDK-8203699 >>> >>> webrev: http://cr.openjdk.java.net/~adinn/8203699/webrev.00/ >> >> Looks good. >> >> It passes the test for me, on the platform where it originally failed. > > Thanks for the review Aleksey. Anyone else? Well, in the absence of anyone else to review this patch I guess I can always provide my own :-) I followed the logic of x86 in adding this conditional call to mov push(pushed_registers, sp); + if (super_klass != r0 || UseCompressedOops) { + mov(r0, super_klass); + } Now, the x86 code munges these two cases together and plants a move when UseCompressedOops is true because it does the move in the same conditional branch as the push. if (super_klass != rax || UseCompressedOops) { if (!IS_A_TEMP(rax)) { push(rax); pushed_rax = true; } mov(rax, super_klass); } A push /is/ needed for the UseCompressedOops case because r0/rax gets modified by an uncompress. The move does no harm on x86 when super_klass = rax because mov(rax, rax) gets elided. So, in that case calling mov is merely wasted generation time not wasted execution time. In the above AArch64 code it looks odd to include the move under the OR with UseCompressedOops. It only makes sense to do the move either when super_klass != r0 or unconditionally -- you either rely on the mov getting elided or you only call mov when the registers are different. So, I propose correcting patch to push(pushed_registers, sp); + if (super_klass != r0) { + mov(r0, super_klass); + } Revised webrev: http://cr.openjdk.java.net/~adinn/8203699/webrev.01/ regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander From shade at redhat.com Thu May 24 10:51:42 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 24 May 2018 12:51:42 +0200 Subject: RFR: AArch64: 8203699 java/lang/invoke/SpecialInterfaceCall fails with SIGILL on aarch64 In-Reply-To: <33316793-e031-3bbb-d8a1-b0e0c29e0e03@redhat.com> References: <887ea79a-86d6-8763-05c8-9e4cce13310c@redhat.com> <33316793-e031-3bbb-d8a1-b0e0c29e0e03@redhat.com> Message-ID: <622ae8b0-73b1-c992-ce7a-9f405c649b31@redhat.com> On 05/24/2018 12:07 PM, Andrew Dinn wrote: > On 23/05/18 20:34, Andrew Dinn wrote: > So, I propose correcting patch to > > push(pushed_registers, sp); > > + if (super_klass != r0) { > + mov(r0, super_klass); > + } > > Revised webrev: http://cr.openjdk.java.net/~adinn/8203699/webrev.01/ OK, fine. Although I see no pragmatic reason to spend more time on polishing this code, and forcing reviewers to re-review and re-test. I assume you have tested the new change? -Aleksey From karen.kinnear at oracle.com Thu May 24 22:56:12 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Thu, 24 May 2018 18:56:12 -0400 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <06529fc3-2eba-101b-9aee-2757893cb8fb@oracle.com> Message-ID: <5DCF1BE9-F04F-4A9B-8E96-B95BB4C0F4FD@oracle.com> Code looks good. Many thanks for the cleanups. Tests look excellent - thank you for e.g. the defmeth cleanups, the .jcod descriptions, and the many test cases. You have set a high bar for thoroughness - and we will all benefit from this. thanks, Karen > On May 24, 2018, at 5:48 AM, David Holmes wrote: > > Here are the further updates based on review comments and rebasing to get the vmTestbase updates for which some closed test changes now have to be applied to the open versions. > > Incremental hotspot webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3-incr/ > > Full hotspot webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3/ > > Change summary: > > test/hotspot/jtreg/ProblemList.txt > - Exclude vmTestbase/nsk/stress/except/except004.java under 8203046 > > test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/BasicTest.java > test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/PrivateMethodsTest.java > - updated to work with new invokeinterface rules and nestmate changes > - misc cleanups > > src/hotspot/share/runtime/reflection.?pp > - rename verify_field_access to verify_member_access (it's always been mis-named and I nearly forgot to do this cleanup!) and rename field_class to member_class > - add TRAPS to verify_member_access to allow use with CHECK macros > > src/hotspot/share/ci/ciField.cpp > src/hotspot/share/classfile/classFileParser.cpp > src/hotspot/share/interpreter/linkResolver.cpp > - updated to use THREAD/CHECK with verify_member_access > - for ciField rename thread to THREAD so it can be used with HAS_PENDING_EXCEPTION > > src/hotspot/share/oops/instanceKlass.cpp > - use CHECK_false when calling nest_host() > - fix indent near nestmate code > > src/hotspot/share/oops/instanceKlass.hpp > - make has_nest_member private > > Thanks, > David > ----- > > On 23/05/2018 4:57 PM, David Holmes wrote: >> Here are the updates so far in response to all the review comments. >> Incremental webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2-incr/ >> Full webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2/ >> Change summary: >> test/runtime/Nestmates/reflectionAPI/* >> - moved to java/lang/reflect/Nestmates >> src/hotspot/cpu/arm/templateTable_arm.cpp >> - Fixed ARM invocation logic as provided by Boris. >> src/hotspot/share/interpreter/linkResolver.cpp >> - expanded comment regarding exceptions >> - Removed leftover debugging code >> src/hotspot/share/oops/instanceKlass.cpp >> - Removed FIXME comments >> - corrected incorrect comment >> - Fixed if/else formatting >> src/hotspot/share/oops/instanceKlass.hpp >> - removed unused debug method >> src/hotspot/share/oops/klassVtable.cpp >> - added comment by request of Karen >> src/hotspot/share/runtime/reflection.cpp >> - Removed FIXME comments >> - expanded comments in places >> - used CHECK_false >> Thanks, >> David >> On 15/05/2018 10:52 AM, David Holmes wrote: >>> This review is being spread across four groups: langtools, core-libs, hotspot and serviceability. This is the specific review thread for hotspot - webrev: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>> >>> See below for full details - including annotated full webrev guiding the review. >>> >>> The intent is to have JEP-181 targeted and integrated by the end of this month. >>> >>> Thanks, >>> David >>> ----- >>> >>> The nestmates project (JEP-181) introduces new classfile attributes to identify classes and interfaces in the same nest, so that the VM can perform access control based on those attributes and so allow direct private access between nestmates without requiring javac to generate synthetic accessor methods. These access control changes also extend to core reflection and the MethodHandle.Lookup contexts. >>> >>> Direct private calls between nestmates requires a more general calling context than is permitted by invokespecial, and so the JVMS is updated to allow, and javac updated to use, invokevirtual and invokeinterface for private class and interface method calls respectively. These changed semantics also extend to MethodHandle findXXX operations. >>> >>> At this time we are only concerned with static nest definitions, which map to a top-level class/interface as the nest-host and all its nested types as nest-members. >>> >>> Please see the JEP for further details. >>> >>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>> >>> All of the specification changes have been previously been worked out by the Valhalla Project Expert Group, and the implementation reviewed by the various contributors and discussed on the valhalla-dev mailing list. >>> >>> Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan >>> >>> Master webrev of all changes: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>> >>> Annotated master webrev index: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>> >>> Performance: this is expected to be performance neutral in a general sense. Benchmarking and performance runs are about to start. >>> >>> Testing Discussion: >>> ------------------ >>> >>> The testing for nestmates can be broken into four main groups: >>> >>> - New tests specifically related to nestmates and currently in the runtime/Nestmates directory >>> >>> - New tests to complement existing tests by adding in testcases not previously expressible. >>> - For example java/lang/invoke/SpecialInterfaceCall.java tests use of invokespecial for private interface methods and performing receiver typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do similar tests for invokeinterface. >>> >>> - New JVM TI tests to verify the spec changes related to nest attributes. >>> >>> - Existing tests significantly affected by the nestmates changes, primarily: >>> - runtime/SelectionResolution >>> >>> In most cases the nestmate changes makes certain invocations that were illegal, legal (e.g. not requiring invokespecial to invoke private interface methods; allowing access to private members via reflection/Methodhandles that were previously not allowed). >>> >>> - Existing tests incidentally affected by the nestmate changes >>> >>> This includes tests of things utilising class redefinition/retransformation to alter nested types but which unintentionally alter nest relationships (which is not permitted). >>> >>> There are still a number of tests problem-listed with issues filed against them to have them adapted to work with nestmates. Some of these are intended to be addressed in the short-term, while some (such as the runtime/SelectionResolution test changes) may not eventuate. >>> >>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>> >>> There is also further test work still to be completed (the JNI and JDI invocation tests): >>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>> which will continue in parallel with the main RFR. >>> >>> Pre-integration Testing: >>> - General: >>> - Mach5: hs/jdk tier1,2 >>> - Mach5: hs-nightly (tiers 1 -3) >>> - Targetted >>> - nashorn (for asm changes) >>> - hotspot: runtime/* >>> serviceability/* >>> compiler/* >>> vmTestbase/* >>> - jdk: java/lang/invoke/* >>> java/lang/reflect/* >>> java/lang/instrument/* >>> java/lang/Class/* >>> java/lang/management/* >>> - langtools: tools/javac >>> tools/javap >>> From gromero at linux.vnet.ibm.com Thu May 24 23:24:23 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Thu, 24 May 2018 20:24:23 -0300 Subject: UseNUMA membind Issue in openJDK In-Reply-To: References: <9a0310b7-2880-db69-cfbc-7abba844ecbf@oracle.com> Message-ID: <187f278f-5069-3afa-c61f-f49a1fc0d790@linux.vnet.ibm.com> Hi Swati, Thanks for CC:ing me. Sorry for the delay replying it, I had to reserve a few specific machines before trying your patch :-) I think that UseNUMA's original task was to figure out the best binding setup for the JVM automatically but I understand that it also has to be aware that sometimes, for some (new) particular reasons, its binding task is "modulated" by other external agents. Thanks for proposing a fix. I have just a question/concern on the proposal: how the JVM should behave if CPUs are not bound in accordance to the bound memory nodes? For instance, what happens if no '--cpunodebind' is passed and '--membind=0,1,16' is passed at the same time on this numa topology: brianh at p215n12:~$ numactl -H available: 4 nodes (0-1,16-17) node 0 cpus: 0 1 2 3 8 9 10 11 16 17 18 19 24 25 26 27 32 33 34 35 node 0 size: 65342 MB node 0 free: 56902 MB node 1 cpus: 40 41 42 43 48 49 50 51 56 57 58 59 64 65 66 67 72 73 74 75 node 1 size: 65447 MB node 1 free: 58322 MB node 16 cpus: 80 81 82 83 88 89 90 91 96 97 98 99 104 105 106 107 112 113 114 115 node 16 size: 65448 MB node 16 free: 63096 MB node 17 cpus: 120 121 122 123 128 129 130 131 136 137 138 139 144 145 146 147 152 153 154 155 node 17 size: 65175 MB node 17 free: 61522 MB node distances: node 0 1 16 17 0: 10 20 40 40 1: 20 10 40 40 16: 40 40 10 20 17: 40 40 20 10 In that case JVM will spawn threads that will run on all CPUs, including those CPUs in numa node 17. Then once in src/hotspot/share/gc/parallel/mutableNUMASpace.cpp, in cas_allocate(): 834 // This version is lock-free. 835 HeapWord* MutableNUMASpace::cas_allocate(size_t size) { 836 Thread* thr = Thread::current(); 837 int lgrp_id = thr->lgrp_id(); 838 if (lgrp_id == -1 || !os::numa_has_group_homing()) { 839 lgrp_id = os::numa_get_group_id(); 840 thr->set_lgrp_id(lgrp_id); 841 } a newly created thread will try to be mapped to a numa node given your CPU ID. So if that CPU is in numa node 17 it will then not find it in: 843 int i = lgrp_spaces()->find(&lgrp_id, LGRPSpace::equals); and will fallback to a random map, picking up a random numa node among nodes 0, 1, and 16: 846 if (i == -1) { 847 i = os::random() % lgrp_spaces()->length(); 848 } If it picks up node 16, not so bad, but what if it picks up node 0 or 1? I see that if one binds mem but leaves CPU unbound one has to know exactly what she/he is doing, because it can be likely suboptimal. On the other hand, letting the node being picked up randomly when there are memory nodes bound but no CPUs seems even more suboptimal in some scenarios. Thus, should the JVM deal with it? @Zhengyu, do you have any opinion on that? Please find a few nits / comments inline. Note that I'm not a (R)eviewer so you still need two official reviews. Best regards, Gustavo On 05/21/2018 01:44 PM, Swati Sharma wrote: > ======================PATCH============================== > diff --git a/src/hotspot/os/linux/os_linux.cpp b/src/hotspot/os/linux/os_linux.cpp > --- a/src/hotspot/os/linux/os_linux.cpp > +++ b/src/hotspot/os/linux/os_linux.cpp > @@ -2832,14 +2832,42 @@ > // Map all node ids in which is possible to allocate memory. Also nodes are > // not always consecutively available, i.e. available from 0 to the highest > // node number. > + // If the nodes have been bound explicitly using numactl membind, then > + // allocate memory from those nodes only. I think ok to place that comment on the same existing line, like: - // node number. + // node number. If the nodes have been bound explicitly using numactl membind, + // then allocate memory from these nodes only. > for (size_t node = 0; node <= highest_node_number; node++) { > - if (Linux::isnode_in_configured_nodes(node)) { > + if (Linux::isnode_in_bounded_nodes(node)) { ---------------------------------^ s/bounded/bound/ > ids[i++] = node; > } > } > return i; > } > +extern "C" struct bitmask { > + unsigned long size; /* number of bits in the map */ > + unsigned long *maskp; > +}; I think it's possible to move the function below to os_linux.hpp with its friends and cope with the forward declaration of 'struct bitmask*` by using the functions from numa API, notably numa_bitmask_nbytes() and numa_bitmask_isbitset() only, avoiding the member dereferecing issue and the need to add the above struct explicitly. > +// Check if single memory node bound. > +// Returns true if single memory node bound. I suggest a minuscule improvement, something like: +// Check if bound to only one numa node. +// Returns true if bound to a single numa node, otherwise returns false. > +bool os::Linux::issingle_node_bound() { What about s/issingle_node_bound/isbound_to_single_node/ ? > + struct bitmask* bmp = _numa_get_membind != NULL ? _numa_get_membind() : NULL; > + if(!(bmp != NULL && bmp->maskp != NULL)) return false; -----^ Are you sure this checking is necessary? I think if numa_get_membind succeed bmp->maskp is always != NULL. Indentation here is odd. No space before 'if' and return on the same line. I would try to avoid lines over 80 chars. > + int issingle = 0; > + // System can have more than 64 nodes so check in all the elements of > + // unsigned long array > + for (unsigned long i = 0; i < (bmp->size / (8 * sizeof(unsigned long))); i++) { > + if (bmp->maskp[i] == 0) { > + continue; > + } else if ((bmp->maskp[i] & (bmp->maskp[i] - 1)) == 0) { > + issingle++; > + } else { > + return false; > + } > + } > + if (issingle == 1) > + return true; > + return false; > +} > + As I mentioned, I think it could be moved to os_linux.hpp instead. Also, it could be something like: +bool os::Linux::isbound_to_single_node(void) { + struct bitmask* bmp; + unsigned long mask; // a mask element in the mask array + unsigned long max_num_masks; + int single_node = 0; + + if (_numa_get_membind != NULL) { + bmp = _numa_get_membind(); + } else { + return false; + } + + max_num_masks = bmp->size / (8 * sizeof(unsigned long)); + + for (mask = 0; mask < max_num_masks; mask++) { + if (bmp->maskp[mask] != 0) { // at least one numa node in the mask + if (bmp->maskp[mask] & (bmp->maskp[mask] - 1) == 0) { + single_node++; // a single numa node in the mask + } else { + return false; + } + } + } + + if (single_node == 1) { + return true; // only a single mask with a single numa node + } else { + return false; + } +} > bool os::get_page_info(char *start, page_info* info) { > return false; > } > @@ -2930,6 +2958,10 @@ > libnuma_dlsym(handle, "numa_bitmask_isbitset"))); > set_numa_distance(CAST_TO_FN_PTR(numa_distance_func_t, > libnuma_dlsym(handle, "numa_distance"))); > + set_numa_set_membind(CAST_TO_FN_PTR(numa_set_membind_func_t, > + libnuma_dlsym(handle, "numa_set_membind"))); > + set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > + libnuma_v2_dlsym(handle, "numa_get_membind"))); > if (numa_available() != -1) { > set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, "numa_all_nodes")); > @@ -3054,6 +3086,8 @@ > os::Linux::numa_set_bind_policy_func_t os::Linux::_numa_set_bind_policy; > os::Linux::numa_bitmask_isbitset_func_t os::Linux::_numa_bitmask_isbitset; > os::Linux::numa_distance_func_t os::Linux::_numa_distance; > +os::Linux::numa_set_membind_func_t os::Linux::_numa_set_membind; > +os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; > unsigned long* os::Linux::_numa_all_nodes; > struct bitmask* os::Linux::_numa_all_nodes_ptr; > struct bitmask* os::Linux::_numa_nodes_ptr; > @@ -4962,8 +4996,9 @@ > if (!Linux::libnuma_init()) { > UseNUMA = false; > } else { > - if ((Linux::numa_max_node() < 1)) { > - // There's only one node(they start from 0), disable NUMA. > + if ((Linux::numa_max_node() < 1) || Linux::issingle_node_bound()) { > + // If there's only one node(they start from 0) or if the process > + // is bound explicitly to a single node using membind, disable NUMA. > UseNUMA = false; > } > } > diff --git a/src/hotspot/os/linux/os_linux.hpp b/src/hotspot/os/linux/os_linux.hpp > --- a/src/hotspot/os/linux/os_linux.hpp > +++ b/src/hotspot/os/linux/os_linux.hpp > @@ -228,6 +228,8 @@ > typedef int (*numa_tonode_memory_func_t)(void *start, size_t size, int node); > typedef void (*numa_interleave_memory_func_t)(void *start, size_t size, unsigned long *nodemask); > typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t size, struct bitmask* mask); > + typedef void (*numa_set_membind_func_t)(struct bitmask *mask); > + typedef struct bitmask* (*numa_get_membind_func_t)(void); > typedef void (*numa_set_bind_policy_func_t)(int policy); > typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, unsigned int n); > @@ -244,6 +246,8 @@ > static numa_set_bind_policy_func_t _numa_set_bind_policy; > static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; > static numa_distance_func_t _numa_distance; > + static numa_set_membind_func_t _numa_set_membind; > + static numa_get_membind_func_t _numa_get_membind; > static unsigned long* _numa_all_nodes; > static struct bitmask* _numa_all_nodes_ptr; > static struct bitmask* _numa_nodes_ptr; > @@ -259,6 +263,8 @@ > static void set_numa_set_bind_policy(numa_set_bind_policy_func_t func) { _numa_set_bind_policy = func; } > static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t func) { _numa_bitmask_isbitset = func; } > static void set_numa_distance(numa_distance_func_t func) { _numa_distance = func; } > + static void set_numa_set_membind(numa_set_membind_func_t func) { _numa_set_membind = func; } > + static void set_numa_get_membind(numa_get_membind_func_t func) { _numa_get_membind = func; } > static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes = ptr; } > static void set_numa_all_nodes_ptr(struct bitmask **ptr) { _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > static void set_numa_nodes_ptr(struct bitmask **ptr) { _numa_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > @@ -320,6 +326,15 @@ > } else > return 0; > } > + // Check if node in bounded nodes + // Check if node is in bound node set. Maybe? > + static bool isnode_in_bounded_nodes(int node) { > + struct bitmask* bmp = _numa_get_membind != NULL ? _numa_get_membind() : NULL; > + if (bmp != NULL && _numa_bitmask_isbitset != NULL && _numa_bitmask_isbitset(bmp, node)) { > + return true; > + } else > + return false; > + } > + static bool issingle_node_bound(); Looks like it can be re-written like: + static bool isnode_in_bound_nodes(int node) { + if (_numa_get_membind != NULL && _numa_bitmask_isbitset != NULL) { + return _numa_bitmask_isbitset(_numa_get_membind(), node); + } else { + return false; + } + } ? > }; > #endif // OS_LINUX_VM_OS_LINUX_HPP From serguei.spitsyn at oracle.com Fri May 25 00:00:00 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Thu, 24 May 2018 17:00:00 -0700 Subject: RFR(XL) : 8199383 : [TESTBUG] Open source VM testbase JVMTI tests In-Reply-To: References: Message-ID: Hi Igor, It looks good to me. Thanks, Serguei On 5/22/18 16:35, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8199383/webrev.00/index.html >> 308253 lines changed: 308253 ins; 0 del; 0 mod; > Hi all, > > could you please review this patch which open sources JVMTI tests from VM testbase? > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > webrev: http://cr.openjdk.java.net/~iignatyev//8199383/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8199383 > testing: :vmTestbase_nsk_jvmti test group > > Thanks, > -- Igor From david.holmes at oracle.com Fri May 25 00:07:52 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 25 May 2018 10:07:52 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <5DCF1BE9-F04F-4A9B-8E96-B95BB4C0F4FD@oracle.com> References: <06529fc3-2eba-101b-9aee-2757893cb8fb@oracle.com> <5DCF1BE9-F04F-4A9B-8E96-B95BB4C0F4FD@oracle.com> Message-ID: <984a273d-c1c0-975e-8f7a-8997c812e9b2@oracle.com> Thanks Karen! David On 25/05/2018 8:56 AM, Karen Kinnear wrote: > Code looks good. Many thanks for the cleanups. > Tests look excellent - thank you for e.g. the defmeth cleanups, the .jcod descriptions, and the many test cases. > > You have set a high bar for thoroughness - and we will all benefit from this. > > thanks, > Karen > >> On May 24, 2018, at 5:48 AM, David Holmes wrote: >> >> Here are the further updates based on review comments and rebasing to get the vmTestbase updates for which some closed test changes now have to be applied to the open versions. >> >> Incremental hotspot webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3-incr/ >> >> Full hotspot webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3/ >> >> Change summary: >> >> test/hotspot/jtreg/ProblemList.txt >> - Exclude vmTestbase/nsk/stress/except/except004.java under 8203046 >> >> test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/BasicTest.java >> test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/PrivateMethodsTest.java >> - updated to work with new invokeinterface rules and nestmate changes >> - misc cleanups >> >> src/hotspot/share/runtime/reflection.?pp >> - rename verify_field_access to verify_member_access (it's always been mis-named and I nearly forgot to do this cleanup!) and rename field_class to member_class >> - add TRAPS to verify_member_access to allow use with CHECK macros >> >> src/hotspot/share/ci/ciField.cpp >> src/hotspot/share/classfile/classFileParser.cpp >> src/hotspot/share/interpreter/linkResolver.cpp >> - updated to use THREAD/CHECK with verify_member_access >> - for ciField rename thread to THREAD so it can be used with HAS_PENDING_EXCEPTION >> >> src/hotspot/share/oops/instanceKlass.cpp >> - use CHECK_false when calling nest_host() >> - fix indent near nestmate code >> >> src/hotspot/share/oops/instanceKlass.hpp >> - make has_nest_member private >> >> Thanks, >> David >> ----- >> >> On 23/05/2018 4:57 PM, David Holmes wrote: >>> Here are the updates so far in response to all the review comments. >>> Incremental webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2-incr/ >>> Full webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2/ >>> Change summary: >>> test/runtime/Nestmates/reflectionAPI/* >>> - moved to java/lang/reflect/Nestmates >>> src/hotspot/cpu/arm/templateTable_arm.cpp >>> - Fixed ARM invocation logic as provided by Boris. >>> src/hotspot/share/interpreter/linkResolver.cpp >>> - expanded comment regarding exceptions >>> - Removed leftover debugging code >>> src/hotspot/share/oops/instanceKlass.cpp >>> - Removed FIXME comments >>> - corrected incorrect comment >>> - Fixed if/else formatting >>> src/hotspot/share/oops/instanceKlass.hpp >>> - removed unused debug method >>> src/hotspot/share/oops/klassVtable.cpp >>> - added comment by request of Karen >>> src/hotspot/share/runtime/reflection.cpp >>> - Removed FIXME comments >>> - expanded comments in places >>> - used CHECK_false >>> Thanks, >>> David >>> On 15/05/2018 10:52 AM, David Holmes wrote: >>>> This review is being spread across four groups: langtools, core-libs, hotspot and serviceability. This is the specific review thread for hotspot - webrev: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>> >>>> See below for full details - including annotated full webrev guiding the review. >>>> >>>> The intent is to have JEP-181 targeted and integrated by the end of this month. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> The nestmates project (JEP-181) introduces new classfile attributes to identify classes and interfaces in the same nest, so that the VM can perform access control based on those attributes and so allow direct private access between nestmates without requiring javac to generate synthetic accessor methods. These access control changes also extend to core reflection and the MethodHandle.Lookup contexts. >>>> >>>> Direct private calls between nestmates requires a more general calling context than is permitted by invokespecial, and so the JVMS is updated to allow, and javac updated to use, invokevirtual and invokeinterface for private class and interface method calls respectively. These changed semantics also extend to MethodHandle findXXX operations. >>>> >>>> At this time we are only concerned with static nest definitions, which map to a top-level class/interface as the nest-host and all its nested types as nest-members. >>>> >>>> Please see the JEP for further details. >>>> >>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>> >>>> All of the specification changes have been previously been worked out by the Valhalla Project Expert Group, and the implementation reviewed by the various contributors and discussed on the valhalla-dev mailing list. >>>> >>>> Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan >>>> >>>> Master webrev of all changes: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>> >>>> Annotated master webrev index: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>> >>>> Performance: this is expected to be performance neutral in a general sense. Benchmarking and performance runs are about to start. >>>> >>>> Testing Discussion: >>>> ------------------ >>>> >>>> The testing for nestmates can be broken into four main groups: >>>> >>>> - New tests specifically related to nestmates and currently in the runtime/Nestmates directory >>>> >>>> - New tests to complement existing tests by adding in testcases not previously expressible. >>>> - For example java/lang/invoke/SpecialInterfaceCall.java tests use of invokespecial for private interface methods and performing receiver typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do similar tests for invokeinterface. >>>> >>>> - New JVM TI tests to verify the spec changes related to nest attributes. >>>> >>>> - Existing tests significantly affected by the nestmates changes, primarily: >>>> - runtime/SelectionResolution >>>> >>>> In most cases the nestmate changes makes certain invocations that were illegal, legal (e.g. not requiring invokespecial to invoke private interface methods; allowing access to private members via reflection/Methodhandles that were previously not allowed). >>>> >>>> - Existing tests incidentally affected by the nestmate changes >>>> >>>> This includes tests of things utilising class redefinition/retransformation to alter nested types but which unintentionally alter nest relationships (which is not permitted). >>>> >>>> There are still a number of tests problem-listed with issues filed against them to have them adapted to work with nestmates. Some of these are intended to be addressed in the short-term, while some (such as the runtime/SelectionResolution test changes) may not eventuate. >>>> >>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>> >>>> There is also further test work still to be completed (the JNI and JDI invocation tests): >>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>> which will continue in parallel with the main RFR. >>>> >>>> Pre-integration Testing: >>>> - General: >>>> - Mach5: hs/jdk tier1,2 >>>> - Mach5: hs-nightly (tiers 1 -3) >>>> - Targetted >>>> - nashorn (for asm changes) >>>> - hotspot: runtime/* >>>> serviceability/* >>>> compiler/* >>>> vmTestbase/* >>>> - jdk: java/lang/invoke/* >>>> java/lang/reflect/* >>>> java/lang/instrument/* >>>> java/lang/Class/* >>>> java/lang/management/* >>>> - langtools: tools/javac >>>> tools/javap >>>> > From Derek.White at cavium.com Fri May 25 00:40:29 2018 From: Derek.White at cavium.com (White, Derek) Date: Fri, 25 May 2018 00:40:29 +0000 Subject: RFR: AArch64: 8203699 java/lang/invoke/SpecialInterfaceCall fails with SIGILL on aarch64 In-Reply-To: <622ae8b0-73b1-c992-ce7a-9f405c649b31@redhat.com> References: <887ea79a-86d6-8763-05c8-9e4cce13310c@redhat.com> <33316793-e031-3bbb-d8a1-b0e0c29e0e03@redhat.com> <622ae8b0-73b1-c992-ce7a-9f405c649b31@redhat.com> Message-ID: Hi Andrew, Both versions look good to me. - Derek > -----Original Message----- > From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On > Behalf Of Aleksey Shipilev > Sent: Thursday, May 24, 2018 6:52 AM > To: Andrew Dinn ; hotspot-dev Source Developers > > Subject: Re: RFR: AArch64: 8203699 java/lang/invoke/SpecialInterfaceCall > fails with SIGILL on aarch64 > > On 05/24/2018 12:07 PM, Andrew Dinn wrote: > > On 23/05/18 20:34, Andrew Dinn wrote: > > So, I propose correcting patch to > > > > push(pushed_registers, sp); > > > > + if (super_klass != r0) { > > + mov(r0, super_klass); > > + } > > > > Revised webrev: http://cr.openjdk.java.net/~adinn/8203699/webrev.01/ > > OK, fine. Although I see no pragmatic reason to spend more time on > polishing this code, and forcing reviewers to re-review and re-test. I assume > you have tested the new change? > > -Aleksey > From coleen.phillimore at oracle.com Fri May 25 01:48:41 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 24 May 2018 21:48:41 -0400 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <06529fc3-2eba-101b-9aee-2757893cb8fb@oracle.com> Message-ID: <29158d1a-7804-4c81-985b-c95ac0103493@oracle.com> This looks great.? Thank you for making the exception handling changes to now verify_member_access. Coleen On 5/24/18 5:48 AM, David Holmes wrote: > Here are the further updates based on review comments and rebasing to > get the vmTestbase updates for which some closed test changes now have > to be applied to the open versions. > > Incremental hotspot webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3-incr/ > > Full hotspot webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3/ > > Change summary: > > test/hotspot/jtreg/ProblemList.txt > - Exclude vmTestbase/nsk/stress/except/except004.java under 8203046 > > test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/BasicTest.java > test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/PrivateMethodsTest.java > - updated to work with new invokeinterface rules and nestmate changes > - misc cleanups > > src/hotspot/share/runtime/reflection.?pp > - rename verify_field_access to verify_member_access (it's always been > mis-named and I nearly forgot to do this cleanup!) and rename > field_class to member_class > - add TRAPS to verify_member_access to allow use with CHECK macros > > src/hotspot/share/ci/ciField.cpp > src/hotspot/share/classfile/classFileParser.cpp > src/hotspot/share/interpreter/linkResolver.cpp > - updated to use THREAD/CHECK with verify_member_access > - for ciField rename thread to THREAD so it can be used with > HAS_PENDING_EXCEPTION > > src/hotspot/share/oops/instanceKlass.cpp > - use CHECK_false when calling nest_host() > - fix indent near nestmate code > > src/hotspot/share/oops/instanceKlass.hpp > - make has_nest_member private > > Thanks, > David > ----- > > On 23/05/2018 4:57 PM, David Holmes wrote: >> Here are the updates so far in response to all the review comments. >> >> Incremental webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2-incr/ >> >> Full webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2/ >> >> Change summary: >> >> test/runtime/Nestmates/reflectionAPI/* >> - moved to java/lang/reflect/Nestmates >> >> src/hotspot/cpu/arm/templateTable_arm.cpp >> - Fixed ARM invocation logic as provided by Boris. >> >> src/hotspot/share/interpreter/linkResolver.cpp >> - expanded comment regarding exceptions >> - Removed leftover debugging code >> >> src/hotspot/share/oops/instanceKlass.cpp >> - Removed FIXME comments >> - corrected incorrect comment >> - Fixed if/else formatting >> >> src/hotspot/share/oops/instanceKlass.hpp >> - removed unused debug method >> >> src/hotspot/share/oops/klassVtable.cpp >> - added comment by request of Karen >> >> src/hotspot/share/runtime/reflection.cpp >> - Removed FIXME comments >> - expanded comments in places >> - used CHECK_false >> >> Thanks, >> David >> >> On 15/05/2018 10:52 AM, David Holmes wrote: >>> This review is being spread across four groups: langtools, >>> core-libs, hotspot and serviceability. This is the specific review >>> thread for hotspot - webrev: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>> >>> See below for full details - including annotated full webrev guiding >>> the review. >>> >>> The intent is to have JEP-181 targeted and integrated by the end of >>> this month. >>> >>> Thanks, >>> David >>> ----- >>> >>> The nestmates project (JEP-181) introduces new classfile attributes >>> to identify classes and interfaces in the same nest, so that the VM >>> can perform access control based on those attributes and so allow >>> direct private access between nestmates without requiring javac to >>> generate synthetic accessor methods. These access control changes >>> also extend to core reflection and the MethodHandle.Lookup contexts. >>> >>> Direct private calls between nestmates requires a more general >>> calling context than is permitted by invokespecial, and so the JVMS >>> is updated to allow, and javac updated to use, invokevirtual and >>> invokeinterface for private class and interface method calls >>> respectively. These changed semantics also extend to MethodHandle >>> findXXX operations. >>> >>> At this time we are only concerned with static nest definitions, >>> which map to a top-level class/interface as the nest-host and all >>> its nested types as nest-members. >>> >>> Please see the JEP for further details. >>> >>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>> >>> All of the specification changes have been previously been worked >>> out by the Valhalla Project Expert Group, and the implementation >>> reviewed by the various contributors and discussed on the >>> valhalla-dev mailing list. >>> >>> Acknowledgments and contributions: Alex Buckley, Maurizio >>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen >>> Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, >>> Kumar Srinivasan >>> >>> Master webrev of all changes: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>> >>> Annotated master webrev index: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>> >>> Performance: this is expected to be performance neutral in a general >>> sense. Benchmarking and performance runs are about to start. >>> >>> Testing Discussion: >>> ------------------ >>> >>> The testing for nestmates can be broken into four main groups: >>> >>> -? New tests specifically related to nestmates and currently in the >>> runtime/Nestmates directory >>> >>> - New tests to complement existing tests by adding in testcases not >>> previously expressible. >>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>> use of invokespecial for private interface methods and performing >>> receiver typechecks, so we add >>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>> invokeinterface. >>> >>> -? New JVM TI tests to verify the spec changes related to nest >>> attributes. >>> >>> -? Existing tests significantly affected by the nestmates changes, >>> primarily: >>> ??? -? runtime/SelectionResolution >>> >>> ??? In most cases the nestmate changes makes certain invocations >>> that were illegal, legal (e.g. not requiring invokespecial to invoke >>> private interface methods; allowing access to private members via >>> reflection/Methodhandles that were previously not allowed). >>> >>> - Existing tests incidentally affected by the nestmate changes >>> >>> ?? This includes tests of things utilising class >>> redefinition/retransformation to alter nested types but which >>> unintentionally alter nest relationships (which is not permitted). >>> >>> There are still a number of tests problem-listed with issues filed >>> against them to have them adapted to work with nestmates. Some of >>> these are intended to be addressed in the short-term, while some >>> (such as the runtime/SelectionResolution test changes) may not >>> eventuate. >>> >>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>> >>> There is also further test work still to be completed (the JNI and >>> JDI invocation tests): >>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>> which will continue in parallel with the main RFR. >>> >>> Pre-integration Testing: >>> ??- General: >>> ???? - Mach5: hs/jdk tier1,2 >>> ???? - Mach5: hs-nightly (tiers 1 -3) >>> ??- Targetted >>> ??? - nashorn (for asm changes) >>> ??? - hotspot: runtime/* >>> ?????????????? serviceability/* >>> ?????????????? compiler/* >>> ?????????????? vmTestbase/* >>> ??? - jdk: java/lang/invoke/* >>> ?????????? java/lang/reflect/* >>> ?????????? java/lang/instrument/* >>> ?????????? java/lang/Class/* >>> ?????????? java/lang/management/* >>> ?? - langtools: tools/javac >>> ??????????????? tools/javap >>> From john.r.rose at oracle.com Fri May 25 04:18:49 2018 From: john.r.rose at oracle.com (John Rose) Date: Thu, 24 May 2018 21:18:49 -0700 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <29158d1a-7804-4c81-985b-c95ac0103493@oracle.com> References: <06529fc3-2eba-101b-9aee-2757893cb8fb@oracle.com> <29158d1a-7804-4c81-985b-c95ac0103493@oracle.com> Message-ID: <5CCE67F3-F061-4350-930F-5824042F6740@oracle.com> Thanks to both Coleen and David for thrashing out this code until it makes sense. I also have been annoyed at verify_field_access. Good to see us make stuff better when we touch it. > On May 24, 2018, at 6:48 PM, coleen.phillimore at oracle.com wrote: > > > This looks great. Thank you for making the exception handling changes to now verify_member_access. From david.holmes at oracle.com Fri May 25 04:39:11 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 25 May 2018 14:39:11 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <29158d1a-7804-4c81-985b-c95ac0103493@oracle.com> References: <06529fc3-2eba-101b-9aee-2757893cb8fb@oracle.com> <29158d1a-7804-4c81-985b-c95ac0103493@oracle.com> Message-ID: <5dd3cefe-41ac-fd74-7194-340f10e71080@oracle.com> Thanks Coleen! And I changed canAccess to can_access (webrev updated in-place for that adjustment). David On 25/05/2018 11:48 AM, coleen.phillimore at oracle.com wrote: > > This looks great.? Thank you for making the exception handling changes > to now verify_member_access. > Coleen > > On 5/24/18 5:48 AM, David Holmes wrote: >> Here are the further updates based on review comments and rebasing to >> get the vmTestbase updates for which some closed test changes now have >> to be applied to the open versions. >> >> Incremental hotspot webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3-incr/ >> >> >> Full hotspot webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3/ >> >> Change summary: >> >> test/hotspot/jtreg/ProblemList.txt >> - Exclude vmTestbase/nsk/stress/except/except004.java under 8203046 >> >> test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/BasicTest.java >> test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/PrivateMethodsTest.java >> - updated to work with new invokeinterface rules and nestmate changes >> - misc cleanups >> >> src/hotspot/share/runtime/reflection.?pp >> - rename verify_field_access to verify_member_access (it's always been >> mis-named and I nearly forgot to do this cleanup!) and rename >> field_class to member_class >> - add TRAPS to verify_member_access to allow use with CHECK macros >> >> src/hotspot/share/ci/ciField.cpp >> src/hotspot/share/classfile/classFileParser.cpp >> src/hotspot/share/interpreter/linkResolver.cpp >> - updated to use THREAD/CHECK with verify_member_access >> - for ciField rename thread to THREAD so it can be used with >> HAS_PENDING_EXCEPTION >> >> src/hotspot/share/oops/instanceKlass.cpp >> - use CHECK_false when calling nest_host() >> - fix indent near nestmate code >> >> src/hotspot/share/oops/instanceKlass.hpp >> - make has_nest_member private >> >> Thanks, >> David >> ----- >> >> On 23/05/2018 4:57 PM, David Holmes wrote: >>> Here are the updates so far in response to all the review comments. >>> >>> Incremental webrev: >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2-incr/ >>> >>> >>> Full webrev: >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2/ >>> >>> Change summary: >>> >>> test/runtime/Nestmates/reflectionAPI/* >>> - moved to java/lang/reflect/Nestmates >>> >>> src/hotspot/cpu/arm/templateTable_arm.cpp >>> - Fixed ARM invocation logic as provided by Boris. >>> >>> src/hotspot/share/interpreter/linkResolver.cpp >>> - expanded comment regarding exceptions >>> - Removed leftover debugging code >>> >>> src/hotspot/share/oops/instanceKlass.cpp >>> - Removed FIXME comments >>> - corrected incorrect comment >>> - Fixed if/else formatting >>> >>> src/hotspot/share/oops/instanceKlass.hpp >>> - removed unused debug method >>> >>> src/hotspot/share/oops/klassVtable.cpp >>> - added comment by request of Karen >>> >>> src/hotspot/share/runtime/reflection.cpp >>> - Removed FIXME comments >>> - expanded comments in places >>> - used CHECK_false >>> >>> Thanks, >>> David >>> >>> On 15/05/2018 10:52 AM, David Holmes wrote: >>>> This review is being spread across four groups: langtools, >>>> core-libs, hotspot and serviceability. This is the specific review >>>> thread for hotspot - webrev: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>> >>>> See below for full details - including annotated full webrev guiding >>>> the review. >>>> >>>> The intent is to have JEP-181 targeted and integrated by the end of >>>> this month. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> The nestmates project (JEP-181) introduces new classfile attributes >>>> to identify classes and interfaces in the same nest, so that the VM >>>> can perform access control based on those attributes and so allow >>>> direct private access between nestmates without requiring javac to >>>> generate synthetic accessor methods. These access control changes >>>> also extend to core reflection and the MethodHandle.Lookup contexts. >>>> >>>> Direct private calls between nestmates requires a more general >>>> calling context than is permitted by invokespecial, and so the JVMS >>>> is updated to allow, and javac updated to use, invokevirtual and >>>> invokeinterface for private class and interface method calls >>>> respectively. These changed semantics also extend to MethodHandle >>>> findXXX operations. >>>> >>>> At this time we are only concerned with static nest definitions, >>>> which map to a top-level class/interface as the nest-host and all >>>> its nested types as nest-members. >>>> >>>> Please see the JEP for further details. >>>> >>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>> >>>> All of the specification changes have been previously been worked >>>> out by the Valhalla Project Expert Group, and the implementation >>>> reviewed by the various contributors and discussed on the >>>> valhalla-dev mailing list. >>>> >>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen >>>> Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, >>>> Kumar Srinivasan >>>> >>>> Master webrev of all changes: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>> >>>> Annotated master webrev index: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>> >>>> Performance: this is expected to be performance neutral in a general >>>> sense. Benchmarking and performance runs are about to start. >>>> >>>> Testing Discussion: >>>> ------------------ >>>> >>>> The testing for nestmates can be broken into four main groups: >>>> >>>> -? New tests specifically related to nestmates and currently in the >>>> runtime/Nestmates directory >>>> >>>> - New tests to complement existing tests by adding in testcases not >>>> previously expressible. >>>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>>> use of invokespecial for private interface methods and performing >>>> receiver typechecks, so we add >>>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>>> invokeinterface. >>>> >>>> -? New JVM TI tests to verify the spec changes related to nest >>>> attributes. >>>> >>>> -? Existing tests significantly affected by the nestmates changes, >>>> primarily: >>>> ??? -? runtime/SelectionResolution >>>> >>>> ??? In most cases the nestmate changes makes certain invocations >>>> that were illegal, legal (e.g. not requiring invokespecial to invoke >>>> private interface methods; allowing access to private members via >>>> reflection/Methodhandles that were previously not allowed). >>>> >>>> - Existing tests incidentally affected by the nestmate changes >>>> >>>> ?? This includes tests of things utilising class >>>> redefinition/retransformation to alter nested types but which >>>> unintentionally alter nest relationships (which is not permitted). >>>> >>>> There are still a number of tests problem-listed with issues filed >>>> against them to have them adapted to work with nestmates. Some of >>>> these are intended to be addressed in the short-term, while some >>>> (such as the runtime/SelectionResolution test changes) may not >>>> eventuate. >>>> >>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>> >>>> There is also further test work still to be completed (the JNI and >>>> JDI invocation tests): >>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>> which will continue in parallel with the main RFR. >>>> >>>> Pre-integration Testing: >>>> ??- General: >>>> ???? - Mach5: hs/jdk tier1,2 >>>> ???? - Mach5: hs-nightly (tiers 1 -3) >>>> ??- Targetted >>>> ??? - nashorn (for asm changes) >>>> ??? - hotspot: runtime/* >>>> ?????????????? serviceability/* >>>> ?????????????? compiler/* >>>> ?????????????? vmTestbase/* >>>> ??? - jdk: java/lang/invoke/* >>>> ?????????? java/lang/reflect/* >>>> ?????????? java/lang/instrument/* >>>> ?????????? java/lang/Class/* >>>> ?????????? java/lang/management/* >>>> ?? - langtools: tools/javac >>>> ??????????????? tools/javap >>>> > From per.liden at oracle.com Fri May 25 06:39:40 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 25 May 2018 08:39:40 +0200 Subject: RFR: 8203817: Monitor::try_lock() should not call check_prelock_state() Message-ID: <7182fe2f-bd59-c677-96fc-07cd82e2ec6c@oracle.com> In debug builds, Monitor::try_lock() calls check_prelock_state() to check the thread state, etc. The intention is to verify that the call is made from the correct context, a context where we're allowed to block and potentially safepoint. Unlike Monitor::lock(), Monitor::try_lock() will never block, hence the call to check_prelock_state() is overly strict and we should remove it. Removing it would match the behavior of all other non-blocking functions, like Monitor::lock_without_safepoint_check(), which doesn't call check_prelock_state() either (for a good reason). The specific problem I've run into with this is related to JFR. Monitor::try_lock() is by JFR to allow non-blocking event generation, so that you can generate JFR events from "any" context without risk blocking/safepointing (the logic is doing something like, if try_lock() fails then put the event on a different queue and let the next owner of the lock handle it). The overly strict checks done by check_prelock_state() in try_lock() breaks this logic, which in turn means that you can't generate JFR event from "any" context as was intended. The patch to fix this is a one-liner, just remove the call to check_prelock_state(). Bug: https://bugs.openjdk.java.net/browse/JDK-8203817 Webrev: http://cr.openjdk.java.net/~pliden/8203817/webrev.0 /Per From david.holmes at oracle.com Fri May 25 06:53:36 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 25 May 2018 16:53:36 +1000 Subject: RFR: 8203817: Monitor::try_lock() should not call check_prelock_state() In-Reply-To: <7182fe2f-bd59-c677-96fc-07cd82e2ec6c@oracle.com> References: <7182fe2f-bd59-c677-96fc-07cd82e2ec6c@oracle.com> Message-ID: <575cbdbd-405c-161d-3582-cf88bf53feea@oracle.com> Hi Per, Exactly what condition(s) does JFR violate? This is throwing away all the checks that guard against incorrect monitor use. It's not just about whether you'd block trying to acquire the Monitor, it's also about whether it is safe to acquire it from that code/thread in the first place. (Though I think some of the checks in there should also be considering the value of _safepoint_check_required.) Thanks, David On 25/05/2018 4:39 PM, Per Liden wrote: > In debug builds, Monitor::try_lock() calls check_prelock_state() to > check the thread state, etc. The intention is to verify that the call is > made from the correct context, a context where we're allowed to block > and potentially safepoint. Unlike Monitor::lock(), Monitor::try_lock() > will never block, hence the call to check_prelock_state() is overly > strict and we should remove it. Removing it would match the behavior of > all other non-blocking functions, like > Monitor::lock_without_safepoint_check(), which doesn't call > check_prelock_state() either (for a good reason). > > The specific problem I've run into with this is related to JFR. > Monitor::try_lock() is by JFR to allow non-blocking event generation, so > that you can generate JFR events from "any" context without risk > blocking/safepointing (the logic is doing something like, if try_lock() > fails then put the event on a different queue and let the next owner of > the lock handle it). The overly strict checks done by > check_prelock_state() in try_lock() breaks this logic, which in turn > means that you can't generate JFR event from "any" context as was intended. > > The patch to fix this is a one-liner, just remove the call to > check_prelock_state(). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203817 > Webrev: http://cr.openjdk.java.net/~pliden/8203817/webrev.0 > > /Per From erik.osterlund at oracle.com Fri May 25 07:17:30 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Fri, 25 May 2018 09:17:30 +0200 Subject: RFR: 8203817: Monitor::try_lock() should not call check_prelock_state() In-Reply-To: <575cbdbd-405c-161d-3582-cf88bf53feea@oracle.com> References: <7182fe2f-bd59-c677-96fc-07cd82e2ec6c@oracle.com> <575cbdbd-405c-161d-3582-cf88bf53feea@oracle.com> Message-ID: Hi David, The change Per is proposing would make try_lock perform the same checks that locking without safepoint checks does. Perhaps locking without safepont checks could perform more sanity checks, but that is a separate issue. I think try_lock should perform the same checks that locking without safepoint checks does. The alternatives are then to 1) Remove checking the prelock state like Per suggests for try_lock (then they do the same checks), or 2) Overhaul the safepoint checking refactoring out the bits that check the safepointing sanity chrcka from other deadlock checks (and correct those to check for rank <= special, and not == special), remove the safepoint checking part from try_lock and adding the deadlock checking parts to lock without safepoints. Doing #2 seems like a different RFE. In fact I believe that would be https://bugs.openjdk.java.net/browse/JDK-8184732 that I filed a while back. In summary, there is a whole bunch of problems in the deadlock detection system, and #2 makes it hard to not get dragged down in the rabbit hole. #1 is sufficient to make try_lock check as much (or little) as locking without safepoint checking. And I think that is enough for the scope of this change. Looks good to me. Thanks, /Erik > On 25 May 2018, at 08:53, David Holmes wrote: > > Hi Per, > > Exactly what condition(s) does JFR violate? This is throwing away all the checks that guard against incorrect monitor use. It's not just about whether you'd block trying to acquire the Monitor, it's also about whether it is safe to acquire it from that code/thread in the first place. (Though I think some of the checks in there should also be considering the value of _safepoint_check_required.) > > Thanks, > David > > >> On 25/05/2018 4:39 PM, Per Liden wrote: >> In debug builds, Monitor::try_lock() calls check_prelock_state() to check the thread state, etc. The intention is to verify that the call is made from the correct context, a context where we're allowed to block and potentially safepoint. Unlike Monitor::lock(), Monitor::try_lock() will never block, hence the call to check_prelock_state() is overly strict and we should remove it. Removing it would match the behavior of all other non-blocking functions, like Monitor::lock_without_safepoint_check(), which doesn't call check_prelock_state() either (for a good reason). >> The specific problem I've run into with this is related to JFR. Monitor::try_lock() is by JFR to allow non-blocking event generation, so that you can generate JFR events from "any" context without risk blocking/safepointing (the logic is doing something like, if try_lock() fails then put the event on a different queue and let the next owner of the lock handle it). The overly strict checks done by check_prelock_state() in try_lock() breaks this logic, which in turn means that you can't generate JFR event from "any" context as was intended. >> The patch to fix this is a one-liner, just remove the call to check_prelock_state(). >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203817 >> Webrev: http://cr.openjdk.java.net/~pliden/8203817/webrev.0 >> /Per From per.liden at oracle.com Fri May 25 07:22:46 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 25 May 2018 09:22:46 +0200 Subject: RFR: 8203817: Monitor::try_lock() should not call check_prelock_state() In-Reply-To: <575cbdbd-405c-161d-3582-cf88bf53feea@oracle.com> References: <7182fe2f-bd59-c677-96fc-07cd82e2ec6c@oracle.com> <575cbdbd-405c-161d-3582-cf88bf53feea@oracle.com> Message-ID: <12ffaa55-6ab1-de5f-66b0-81bb622e9539@oracle.com> Hi David, On 05/25/2018 08:53 AM, David Holmes wrote: > Hi Per, > > Exactly what condition(s) does JFR violate? This is throwing away all I know of two issues: This: assert((!thread->is_Java_thread() || ((JavaThread *)thread)->thread_state() == _thread_in_vm) || rank() == Mutex::special, "wrong thread state for using locks"); Because we're a Java thread, in a VM-leaf call context, grabbing a Mutex::leaf lock. And this: debug_only(if (rank() != Mutex::special) \ thread->check_for_valid_safepoint_state(false);) Which ends up doing a similar check inside check_for_valid_safepoint_state(). /Per > the checks that guard against incorrect monitor use. It's not just about > whether you'd block trying to acquire the Monitor, it's also about > whether it is safe to acquire it from that code/thread in the first > place. (Though I think some of the checks in there should also be > considering the value of _safepoint_check_required.) > > Thanks, > David > > > On 25/05/2018 4:39 PM, Per Liden wrote: >> In debug builds, Monitor::try_lock() calls check_prelock_state() to >> check the thread state, etc. The intention is to verify that the call >> is made from the correct context, a context where we're allowed to >> block and potentially safepoint. Unlike Monitor::lock(), >> Monitor::try_lock() will never block, hence the call to >> check_prelock_state() is overly strict and we should remove it. >> Removing it would match the behavior of all other non-blocking >> functions, like Monitor::lock_without_safepoint_check(), which doesn't >> call check_prelock_state() either (for a good reason). >> >> The specific problem I've run into with this is related to JFR. >> Monitor::try_lock() is by JFR to allow non-blocking event generation, >> so that you can generate JFR events from "any" context without risk >> blocking/safepointing (the logic is doing something like, if >> try_lock() fails then put the event on a different queue and let the >> next owner of the lock handle it). The overly strict checks done by >> check_prelock_state() in try_lock() breaks this logic, which in turn >> means that you can't generate JFR event from "any" context as was >> intended. >> >> The patch to fix this is a one-liner, just remove the call to >> check_prelock_state(). >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203817 >> Webrev: http://cr.openjdk.java.net/~pliden/8203817/webrev.0 >> >> /Per From magnus.ihse.bursie at oracle.com Fri May 25 07:40:51 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Fri, 25 May 2018 09:40:51 +0200 Subject: RFR(XL) : 8199383 : [TESTBUG] Open source VM testbase JVMTI tests In-Reply-To: <8bfed6c4-d0e0-4324-5982-d4ef040d12c1@oracle.com> References: <8bfed6c4-d0e0-4324-5982-d4ef040d12c1@oracle.com> Message-ID: On 2018-05-23 02:05, Erik Joelsson wrote: > Looks like the line "# } nsk/jvmti" is a left over. Otherwise this > looks ok, even if it's an enormous amount of duplication. Hopefully we > can figure out a better way to express common parameters for tests soon. Argee, the current solution is not scaling to this kind of multiple test with additions. There is already an issue on JBS for this: https://bugs.openjdk.java.net/browse/JDK-8201582 /Magnus > > /Erik > > > On 2018-05-22 16:35, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8199383/webrev.00/index.html >>> 308253 lines changed: 308253 ins; 0 del; 0 mod; >> Hi all, >> >> could you please review this patch which open sources JVMTI tests >> from VM testbase? >> >> As usually w/ VM testbase code, these tests are old, they have been >> run in hotspot testing for a long period of time. Originally, these >> tests were run by a test harness different from jtreg and had >> different build and execution schemes, some parts couldn't be easily >> translated to jtreg, so tests might have actions or pieces of code >> which look weird. In a long term, we are planning to rework them. >> >> webrev: >> http://cr.openjdk.java.net/~iignatyev//8199383/webrev.00/index.html >> JBS: https://bugs.openjdk.java.net/browse/JDK-8199383 >> testing: :vmTestbase_nsk_jvmti test group >> >> Thanks, >> -- Igor > From david.holmes at oracle.com Fri May 25 07:51:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 25 May 2018 17:51:15 +1000 Subject: RFR: 8203817: Monitor::try_lock() should not call check_prelock_state() In-Reply-To: References: <7182fe2f-bd59-c677-96fc-07cd82e2ec6c@oracle.com> <575cbdbd-405c-161d-3582-cf88bf53feea@oracle.com> Message-ID: <4efa9c7d-64da-a8c2-f3db-aa3a13a59d73@oracle.com> Hi Erik, On 25/05/2018 5:17 PM, Erik Osterlund wrote: > Hi David, > > The change Per is proposing would make try_lock perform the same checks > that locking without safepoint checks does. Yet locking without a safepoint check also ensures _safepoint_check_required != Monitor::_safepoint_check_always. But try_lock does not and can be applied to both kinds of Monitors/Mutexes. Further it also has a stated, but not checked, precondition that the thread is not _thread_in_vm - though that may only be an issue if we were to block. I'm looking at the checks in check_prelock_state wondering which are only truly needed if we risk blocking? I think most of them, with the exception of is_crash_protected. Are we guaranteed never to run this JFR code on the WatcherThread? Though I'm also concerned about lock-ranking and deadlocks if we try_lock multiple locks. Notwithstanding the rank/deadlock code is far from ideal already. I don't mind removing checks/guards that really don't apply in the try_lock case, but I'm wary of removing all guards without being more sure of that. Thanks, David ----- > Perhaps locking without > safepont checks could perform more sanity checks, but that is a separate > issue. I think try_lock should perform the same checks that locking > without safepoint checks does. The alternatives are then to > > 1) Remove checking the prelock state like Per suggests for try_lock > (then they do the same checks), or > 2) Overhaul the safepoint checking refactoring out the bits that check > the safepointing sanity chrcka from other deadlock checks (and correct > those to check for rank <= special, and not == special), remove the > safepoint checking part from try_lock and adding the deadlock checking > parts to lock without safepoints. > > Doing #2 seems like a different RFE. In fact I believe that would be > https://bugs.openjdk.java.net/browse/JDK-8184732 > ?that > I filed a while back. > > In summary, there is a whole bunch of problems in the deadlock detection > system, and #2 makes it hard to not get dragged down in the rabbit hole. > #1 is sufficient to ?make try_lock check as much (or little) as locking > without safepoint checking. And I think that is enough for the scope of > this change. > > Looks good to me. > > Thanks, > /Erik > > On 25 May 2018, at 08:53, David Holmes > wrote: > >> Hi Per, >> >> Exactly what condition(s) does JFR violate? This is throwing away all >> the checks that guard against incorrect monitor use. It's not just >> about whether you'd block trying to acquire the Monitor, it's also >> about whether it is safe to acquire it from that code/thread in the >> first place. (Though I think some of the checks in there should also >> be considering the value of _safepoint_check_required.) >> >> Thanks, >> David >> >> >> On 25/05/2018 4:39 PM, Per Liden wrote: >>> In debug builds, Monitor::try_lock() calls check_prelock_state() to >>> check the thread state, etc. The intention is to verify that the call >>> is made from the correct context, a context where we're allowed to >>> block and potentially safepoint. Unlike Monitor::lock(), >>> Monitor::try_lock() will never block, hence the call to >>> check_prelock_state() is overly strict and we should remove it. >>> Removing it would match the behavior of all other non-blocking >>> functions, like Monitor::lock_without_safepoint_check(), which >>> doesn't call check_prelock_state() either (for a good reason). >>> The specific problem I've run into with this is related to JFR. >>> Monitor::try_lock() is by JFR to allow non-blocking event generation, >>> so that you can generate JFR events from "any" context without risk >>> blocking/safepointing (the logic is doing something like, if >>> try_lock() fails then put the event on a different queue and let the >>> next owner of the lock handle it). The overly strict checks done by >>> check_prelock_state() in try_lock() breaks this logic, which in turn >>> means that you can't generate JFR event from "any" context as was >>> intended. >>> The patch to fix this is a one-liner, just remove the call to >>> check_prelock_state(). >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203817 >>> Webrev: http://cr.openjdk.java.net/~pliden/8203817/webrev.0 >>> /Per From aph at redhat.com Fri May 25 09:01:44 2018 From: aph at redhat.com (Andrew Haley) Date: Fri, 25 May 2018 10:01:44 +0100 Subject: RFR: AArch64: 8203699 java/lang/invoke/SpecialInterfaceCall fails with SIGILL on aarch64 In-Reply-To: References: <887ea79a-86d6-8763-05c8-9e4cce13310c@redhat.com> <33316793-e031-3bbb-d8a1-b0e0c29e0e03@redhat.com> <622ae8b0-73b1-c992-ce7a-9f405c649b31@redhat.com> Message-ID: <3ca651dc-67c1-d5b8-dd87-9e9a7c2d20e8@redhat.com> On 05/25/2018 01:40 AM, White, Derek wrote: > OK, fine. Although I see no pragmatic reason to spend more time on > polishing this code, and forcing reviewers to re-review and re-test. I assume > you have tested the new change? OK, but if you've not committed it yet please move this comment next to the instruction: // Get super_klass value into r0 (even if it was in r5 or r2). -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From per.liden at oracle.com Fri May 25 09:33:38 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 25 May 2018 11:33:38 +0200 Subject: RFR: 8203817: Monitor::try_lock() should not call check_prelock_state() In-Reply-To: <4efa9c7d-64da-a8c2-f3db-aa3a13a59d73@oracle.com> References: <7182fe2f-bd59-c677-96fc-07cd82e2ec6c@oracle.com> <575cbdbd-405c-161d-3582-cf88bf53feea@oracle.com> <4efa9c7d-64da-a8c2-f3db-aa3a13a59d73@oracle.com> Message-ID: <585ac20e-a74f-7ba8-f5ce-e642df8e98d0@oracle.com> Hi David, On 05/25/2018 09:51 AM, David Holmes wrote: > Hi Erik, > > On 25/05/2018 5:17 PM, Erik Osterlund wrote: >> Hi David, >> >> The change Per is proposing would make try_lock perform the same >> checks that locking without safepoint checks does. > > Yet locking without a safepoint check also ensures > _safepoint_check_required != Monitor::_safepoint_check_always. But > try_lock does not and can be applied to both kinds of Monitors/Mutexes. > Further it also has a stated, but not checked, precondition that the > thread is not _thread_in_vm - though that may only be an issue if we > were to block. > > I'm looking at the checks in check_prelock_state wondering which are > only truly needed if we risk blocking? I think most of them, with the > exception of is_crash_protected. Are we guaranteed never to run this JFR > code on the WatcherThread? > > Though I'm also concerned about lock-ranking and deadlocks if we > try_lock multiple locks. Notwithstanding the rank/deadlock code is far > from ideal already. > > I don't mind removing checks/guards that really don't apply in the > try_lock case, but I'm wary of removing all guards without being more > sure of that. Ok, so how about we just avoid the bad parts, by doing this: http://cr.openjdk.java.net/~pliden/8203817/webrev.1 /Per > > Thanks, > David > ----- > >> Perhaps locking without safepont checks could perform more sanity >> checks, but that is a separate issue. I think try_lock should perform >> the same checks that locking without safepoint checks does. The >> alternatives are then to >> >> 1) Remove checking the prelock state like Per suggests for try_lock >> (then they do the same checks), or >> 2) Overhaul the safepoint checking refactoring out the bits that check >> the safepointing sanity chrcka from other deadlock checks (and correct >> those to check for rank <= special, and not == special), remove the >> safepoint checking part from try_lock and adding the deadlock checking >> parts to lock without safepoints. >> >> Doing #2 seems like a different RFE. In fact I believe that would be >> https://bugs.openjdk.java.net/browse/JDK-8184732 >> ?that >> I filed a while back. >> >> In summary, there is a whole bunch of problems in the deadlock >> detection system, and #2 makes it hard to not get dragged down in the >> rabbit hole. #1 is sufficient to ?make try_lock check as much (or >> little) as locking without safepoint checking. And I think that is >> enough for the scope of this change. >> >> Looks good to me. >> >> Thanks, >> /Erik >> >> On 25 May 2018, at 08:53, David Holmes > > wrote: >> >>> Hi Per, >>> >>> Exactly what condition(s) does JFR violate? This is throwing away all >>> the checks that guard against incorrect monitor use. It's not just >>> about whether you'd block trying to acquire the Monitor, it's also >>> about whether it is safe to acquire it from that code/thread in the >>> first place. (Though I think some of the checks in there should also >>> be considering the value of _safepoint_check_required.) >>> >>> Thanks, >>> David >>> >>> >>> On 25/05/2018 4:39 PM, Per Liden wrote: >>>> In debug builds, Monitor::try_lock() calls check_prelock_state() to >>>> check the thread state, etc. The intention is to verify that the >>>> call is made from the correct context, a context where we're allowed >>>> to block and potentially safepoint. Unlike Monitor::lock(), >>>> Monitor::try_lock() will never block, hence the call to >>>> check_prelock_state() is overly strict and we should remove it. >>>> Removing it would match the behavior of all other non-blocking >>>> functions, like Monitor::lock_without_safepoint_check(), which >>>> doesn't call check_prelock_state() either (for a good reason). >>>> The specific problem I've run into with this is related to JFR. >>>> Monitor::try_lock() is by JFR to allow non-blocking event >>>> generation, so that you can generate JFR events from "any" context >>>> without risk blocking/safepointing (the logic is doing something >>>> like, if try_lock() fails then put the event on a different queue >>>> and let the next owner of the lock handle it). The overly strict >>>> checks done by check_prelock_state() in try_lock() breaks this >>>> logic, which in turn means that you can't generate JFR event from >>>> "any" context as was intended. >>>> The patch to fix this is a one-liner, just remove the call to >>>> check_prelock_state(). >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203817 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8203817/webrev.0 >>>> /Per From david.holmes at oracle.com Fri May 25 09:45:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 25 May 2018 19:45:12 +1000 Subject: RFR: 8203817: Monitor::try_lock() should not call check_prelock_state() In-Reply-To: <585ac20e-a74f-7ba8-f5ce-e642df8e98d0@oracle.com> References: <7182fe2f-bd59-c677-96fc-07cd82e2ec6c@oracle.com> <575cbdbd-405c-161d-3582-cf88bf53feea@oracle.com> <4efa9c7d-64da-a8c2-f3db-aa3a13a59d73@oracle.com> <585ac20e-a74f-7ba8-f5ce-e642df8e98d0@oracle.com> Message-ID: <3aa42dad-8ab1-09fc-fe99-ef156072af4c@oracle.com> On 25/05/2018 7:33 PM, Per Liden wrote: > Hi David, > > On 05/25/2018 09:51 AM, David Holmes wrote: >> Hi Erik, >> >> On 25/05/2018 5:17 PM, Erik Osterlund wrote: >>> Hi David, >>> >>> The change Per is proposing would make try_lock perform the same >>> checks that locking without safepoint checks does. >> >> Yet locking without a safepoint check also ensures >> _safepoint_check_required != Monitor::_safepoint_check_always. But >> try_lock does not and can be applied to both kinds of >> Monitors/Mutexes. Further it also has a stated, but not checked, >> precondition that the thread is not _thread_in_vm - though that may >> only be an issue if we were to block. >> >> I'm looking at the checks in check_prelock_state wondering which are >> only truly needed if we risk blocking? I think most of them, with the >> exception of is_crash_protected. Are we guaranteed never to run this >> JFR code on the WatcherThread? >> >> Though I'm also concerned about lock-ranking and deadlocks if we >> try_lock multiple locks. Notwithstanding the rank/deadlock code is far >> from ideal already. >> >> I don't mind removing checks/guards that really don't apply in the >> try_lock case, but I'm wary of removing all guards without being more >> sure of that. > > Ok, so how about we just avoid the bad parts, by doing this: > > http://cr.openjdk.java.net/~pliden/8203817/webrev.1 That seems a reasonable compromise. There may still be possible deadlock issues if we use try-lock to acquire multiple locks, but we probably don't do that ... and as Erik noted there are issues with the ranking/deadlock detection anyway. Thanks, David > /Per > >> >> Thanks, >> David >> ----- >> >>> Perhaps locking without safepont checks could perform more sanity >>> checks, but that is a separate issue. I think try_lock should perform >>> the same checks that locking without safepoint checks does. The >>> alternatives are then to >>> >>> 1) Remove checking the prelock state like Per suggests for try_lock >>> (then they do the same checks), or >>> 2) Overhaul the safepoint checking refactoring out the bits that >>> check the safepointing sanity chrcka from other deadlock checks (and >>> correct those to check for rank <= special, and not == special), >>> remove the safepoint checking part from try_lock and adding the >>> deadlock checking parts to lock without safepoints. >>> >>> Doing #2 seems like a different RFE. In fact I believe that would be >>> https://bugs.openjdk.java.net/browse/JDK-8184732 >>> ?that >>> I filed a while back. >>> >>> In summary, there is a whole bunch of problems in the deadlock >>> detection system, and #2 makes it hard to not get dragged down in the >>> rabbit hole. #1 is sufficient to ?make try_lock check as much (or >>> little) as locking without safepoint checking. And I think that is >>> enough for the scope of this change. >>> >>> Looks good to me. >>> >>> Thanks, >>> /Erik >>> >>> On 25 May 2018, at 08:53, David Holmes >> > wrote: >>> >>>> Hi Per, >>>> >>>> Exactly what condition(s) does JFR violate? This is throwing away >>>> all the checks that guard against incorrect monitor use. It's not >>>> just about whether you'd block trying to acquire the Monitor, it's >>>> also about whether it is safe to acquire it from that code/thread in >>>> the first place. (Though I think some of the checks in there should >>>> also be considering the value of _safepoint_check_required.) >>>> >>>> Thanks, >>>> David >>>> >>>> >>>> On 25/05/2018 4:39 PM, Per Liden wrote: >>>>> In debug builds, Monitor::try_lock() calls check_prelock_state() to >>>>> check the thread state, etc. The intention is to verify that the >>>>> call is made from the correct context, a context where we're >>>>> allowed to block and potentially safepoint. Unlike Monitor::lock(), >>>>> Monitor::try_lock() will never block, hence the call to >>>>> check_prelock_state() is overly strict and we should remove it. >>>>> Removing it would match the behavior of all other non-blocking >>>>> functions, like Monitor::lock_without_safepoint_check(), which >>>>> doesn't call check_prelock_state() either (for a good reason). >>>>> The specific problem I've run into with this is related to JFR. >>>>> Monitor::try_lock() is by JFR to allow non-blocking event >>>>> generation, so that you can generate JFR events from "any" context >>>>> without risk blocking/safepointing (the logic is doing something >>>>> like, if try_lock() fails then put the event on a different queue >>>>> and let the next owner of the lock handle it). The overly strict >>>>> checks done by check_prelock_state() in try_lock() breaks this >>>>> logic, which in turn means that you can't generate JFR event from >>>>> "any" context as was intended. >>>>> The patch to fix this is a one-liner, just remove the call to >>>>> check_prelock_state(). >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203817 >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8203817/webrev.0 >>>>> /Per From per.liden at oracle.com Fri May 25 11:47:03 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 25 May 2018 13:47:03 +0200 Subject: RFR: 8203817: Monitor::try_lock() should not call check_prelock_state() In-Reply-To: <3aa42dad-8ab1-09fc-fe99-ef156072af4c@oracle.com> References: <7182fe2f-bd59-c677-96fc-07cd82e2ec6c@oracle.com> <575cbdbd-405c-161d-3582-cf88bf53feea@oracle.com> <4efa9c7d-64da-a8c2-f3db-aa3a13a59d73@oracle.com> <585ac20e-a74f-7ba8-f5ce-e642df8e98d0@oracle.com> <3aa42dad-8ab1-09fc-fe99-ef156072af4c@oracle.com> Message-ID: <287080f3-79b0-06d7-6eb2-ae8932e9d0a2@oracle.com> On 05/25/2018 11:45 AM, David Holmes wrote: > On 25/05/2018 7:33 PM, Per Liden wrote: >> Hi David, >> >> On 05/25/2018 09:51 AM, David Holmes wrote: >>> Hi Erik, >>> >>> On 25/05/2018 5:17 PM, Erik Osterlund wrote: >>>> Hi David, >>>> >>>> The change Per is proposing would make try_lock perform the same >>>> checks that locking without safepoint checks does. >>> >>> Yet locking without a safepoint check also ensures >>> _safepoint_check_required != Monitor::_safepoint_check_always. But >>> try_lock does not and can be applied to both kinds of >>> Monitors/Mutexes. Further it also has a stated, but not checked, >>> precondition that the thread is not _thread_in_vm - though that may >>> only be an issue if we were to block. >>> >>> I'm looking at the checks in check_prelock_state wondering which are >>> only truly needed if we risk blocking? I think most of them, with the >>> exception of is_crash_protected. Are we guaranteed never to run this >>> JFR code on the WatcherThread? >>> >>> Though I'm also concerned about lock-ranking and deadlocks if we >>> try_lock multiple locks. Notwithstanding the rank/deadlock code is >>> far from ideal already. >>> >>> I don't mind removing checks/guards that really don't apply in the >>> try_lock case, but I'm wary of removing all guards without being more >>> sure of that. >> >> Ok, so how about we just avoid the bad parts, by doing this: >> >> http://cr.openjdk.java.net/~pliden/8203817/webrev.1 > > That seems a reasonable compromise. Good! Thanks for looking at this David! /Per > > There may still be possible deadlock issues if we use try-lock to > acquire multiple locks, but we probably don't do that ... and as Erik > noted there are issues with the ranking/deadlock detection anyway. > > Thanks, > David > >> /Per >> >>> >>> Thanks, >>> David >>> ----- >>> >>>> Perhaps locking without safepont checks could perform more sanity >>>> checks, but that is a separate issue. I think try_lock should >>>> perform the same checks that locking without safepoint checks does. >>>> The alternatives are then to >>>> >>>> 1) Remove checking the prelock state like Per suggests for try_lock >>>> (then they do the same checks), or >>>> 2) Overhaul the safepoint checking refactoring out the bits that >>>> check the safepointing sanity chrcka from other deadlock checks (and >>>> correct those to check for rank <= special, and not == special), >>>> remove the safepoint checking part from try_lock and adding the >>>> deadlock checking parts to lock without safepoints. >>>> >>>> Doing #2 seems like a different RFE. In fact I believe that would be >>>> https://bugs.openjdk.java.net/browse/JDK-8184732 >>>> ?that >>>> I filed a while back. >>>> >>>> In summary, there is a whole bunch of problems in the deadlock >>>> detection system, and #2 makes it hard to not get dragged down in >>>> the rabbit hole. #1 is sufficient to ?make try_lock check as much >>>> (or little) as locking without safepoint checking. And I think that >>>> is enough for the scope of this change. >>>> >>>> Looks good to me. >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 25 May 2018, at 08:53, David Holmes >>> > wrote: >>>> >>>>> Hi Per, >>>>> >>>>> Exactly what condition(s) does JFR violate? This is throwing away >>>>> all the checks that guard against incorrect monitor use. It's not >>>>> just about whether you'd block trying to acquire the Monitor, it's >>>>> also about whether it is safe to acquire it from that code/thread >>>>> in the first place. (Though I think some of the checks in there >>>>> should also be considering the value of _safepoint_check_required.) >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> >>>>> On 25/05/2018 4:39 PM, Per Liden wrote: >>>>>> In debug builds, Monitor::try_lock() calls check_prelock_state() >>>>>> to check the thread state, etc. The intention is to verify that >>>>>> the call is made from the correct context, a context where we're >>>>>> allowed to block and potentially safepoint. Unlike >>>>>> Monitor::lock(), Monitor::try_lock() will never block, hence the >>>>>> call to check_prelock_state() is overly strict and we should >>>>>> remove it. Removing it would match the behavior of all other >>>>>> non-blocking functions, like >>>>>> Monitor::lock_without_safepoint_check(), which doesn't call >>>>>> check_prelock_state() either (for a good reason). >>>>>> The specific problem I've run into with this is related to JFR. >>>>>> Monitor::try_lock() is by JFR to allow non-blocking event >>>>>> generation, so that you can generate JFR events from "any" context >>>>>> without risk blocking/safepointing (the logic is doing something >>>>>> like, if try_lock() fails then put the event on a different queue >>>>>> and let the next owner of the lock handle it). The overly strict >>>>>> checks done by check_prelock_state() in try_lock() breaks this >>>>>> logic, which in turn means that you can't generate JFR event from >>>>>> "any" context as was intended. >>>>>> The patch to fix this is a one-liner, just remove the call to >>>>>> check_prelock_state(). >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203817 >>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8203817/webrev.0 >>>>>> /Per From robbin.ehn at oracle.com Fri May 25 11:52:18 2018 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 25 May 2018 13:52:18 +0200 Subject: RFR: 8203817: Monitor::try_lock() should not call check_prelock_state() In-Reply-To: <3aa42dad-8ab1-09fc-fe99-ef156072af4c@oracle.com> References: <7182fe2f-bd59-c677-96fc-07cd82e2ec6c@oracle.com> <575cbdbd-405c-161d-3582-cf88bf53feea@oracle.com> <4efa9c7d-64da-a8c2-f3db-aa3a13a59d73@oracle.com> <585ac20e-a74f-7ba8-f5ce-e642df8e98d0@oracle.com> <3aa42dad-8ab1-09fc-fe99-ef156072af4c@oracle.com> Message-ID: >> http://cr.openjdk.java.net/~pliden/8203817/webrev.1 Thank you Per for this awesome patch, it solves a lot of special cases for me! Ship it! /Robbin > > That seems a reasonable compromise. > > There may still be possible deadlock issues if we use try-lock to acquire > multiple locks, but we probably don't do that ... and as Erik noted there are > issues with the ranking/deadlock detection anyway. > > Thanks, > David > >> /Per >> >>> >>> Thanks, >>> David >>> ----- >>> >>>> Perhaps locking without safepont checks could perform more sanity checks, >>>> but that is a separate issue. I think try_lock should perform the same >>>> checks that locking without safepoint checks does. The alternatives are then to >>>> >>>> 1) Remove checking the prelock state like Per suggests for try_lock (then >>>> they do the same checks), or >>>> 2) Overhaul the safepoint checking refactoring out the bits that check the >>>> safepointing sanity chrcka from other deadlock checks (and correct those to >>>> check for rank <= special, and not == special), remove the safepoint >>>> checking part from try_lock and adding the deadlock checking parts to lock >>>> without safepoints. >>>> >>>> Doing #2 seems like a different RFE. In fact I believe that would be >>>> https://bugs.openjdk.java.net/browse/JDK-8184732 >>>> ?that >>>> I filed a while back. >>>> >>>> In summary, there is a whole bunch of problems in the deadlock detection >>>> system, and #2 makes it hard to not get dragged down in the rabbit hole. #1 >>>> is sufficient to ?make try_lock check as much (or little) as locking without >>>> safepoint checking. And I think that is enough for the scope of this change. >>>> >>>> Looks good to me. >>>> >>>> Thanks, >>>> /Erik >>>> >>>> On 25 May 2018, at 08:53, David Holmes >>> > wrote: >>>> >>>>> Hi Per, >>>>> >>>>> Exactly what condition(s) does JFR violate? This is throwing away all the >>>>> checks that guard against incorrect monitor use. It's not just about >>>>> whether you'd block trying to acquire the Monitor, it's also about whether >>>>> it is safe to acquire it from that code/thread in the first place. (Though >>>>> I think some of the checks in there should also be considering the value of >>>>> _safepoint_check_required.) >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> >>>>> On 25/05/2018 4:39 PM, Per Liden wrote: >>>>>> In debug builds, Monitor::try_lock() calls check_prelock_state() to check >>>>>> the thread state, etc. The intention is to verify that the call is made >>>>>> from the correct context, a context where we're allowed to block and >>>>>> potentially safepoint. Unlike Monitor::lock(), Monitor::try_lock() will >>>>>> never block, hence the call to check_prelock_state() is overly strict and >>>>>> we should remove it. Removing it would match the behavior of all other >>>>>> non-blocking functions, like Monitor::lock_without_safepoint_check(), >>>>>> which doesn't call check_prelock_state() either (for a good reason). >>>>>> The specific problem I've run into with this is related to JFR. >>>>>> Monitor::try_lock() is by JFR to allow non-blocking event generation, so >>>>>> that you can generate JFR events from "any" context without risk >>>>>> blocking/safepointing (the logic is doing something like, if try_lock() >>>>>> fails then put the event on a different queue and let the next owner of >>>>>> the lock handle it). The overly strict checks done by >>>>>> check_prelock_state() in try_lock() breaks this logic, which in turn means >>>>>> that you can't generate JFR event from "any" context as was intended. >>>>>> The patch to fix this is a one-liner, just remove the call to >>>>>> check_prelock_state(). >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203817 >>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8203817/webrev.0 >>>>>> /Per From per.liden at oracle.com Fri May 25 11:59:15 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 25 May 2018 13:59:15 +0200 Subject: RFR: 8203817: Monitor::try_lock() should not call check_prelock_state() In-Reply-To: References: <7182fe2f-bd59-c677-96fc-07cd82e2ec6c@oracle.com> <575cbdbd-405c-161d-3582-cf88bf53feea@oracle.com> <4efa9c7d-64da-a8c2-f3db-aa3a13a59d73@oracle.com> <585ac20e-a74f-7ba8-f5ce-e642df8e98d0@oracle.com> <3aa42dad-8ab1-09fc-fe99-ef156072af4c@oracle.com> Message-ID: On 05/25/2018 01:52 PM, Robbin Ehn wrote: >>> http://cr.openjdk.java.net/~pliden/8203817/webrev.1 > > Thank you Per for this awesome patch, it solves a lot of special cases > for me! > > Ship it! Thanks for reviewing, Robbin! /Per > > /Robbin > >> >> That seems a reasonable compromise. >> >> There may still be possible deadlock issues if we use try-lock to >> acquire multiple locks, but we probably don't do that ... and as Erik >> noted there are issues with the ranking/deadlock detection anyway. >> >> Thanks, >> David >> >>> /Per >>> >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> Perhaps locking without safepont checks could perform more sanity >>>>> checks, but that is a separate issue. I think try_lock should >>>>> perform the same checks that locking without safepoint checks does. >>>>> The alternatives are then to >>>>> >>>>> 1) Remove checking the prelock state like Per suggests for try_lock >>>>> (then they do the same checks), or >>>>> 2) Overhaul the safepoint checking refactoring out the bits that >>>>> check the safepointing sanity chrcka from other deadlock checks >>>>> (and correct those to check for rank <= special, and not == >>>>> special), remove the safepoint checking part from try_lock and >>>>> adding the deadlock checking parts to lock without safepoints. >>>>> >>>>> Doing #2 seems like a different RFE. In fact I believe that would >>>>> be https://bugs.openjdk.java.net/browse/JDK-8184732 >>>>> ?that >>>>> I filed a while back. >>>>> >>>>> In summary, there is a whole bunch of problems in the deadlock >>>>> detection system, and #2 makes it hard to not get dragged down in >>>>> the rabbit hole. #1 is sufficient to ?make try_lock check as much >>>>> (or little) as locking without safepoint checking. And I think that >>>>> is enough for the scope of this change. >>>>> >>>>> Looks good to me. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 25 May 2018, at 08:53, David Holmes >>>> > wrote: >>>>> >>>>>> Hi Per, >>>>>> >>>>>> Exactly what condition(s) does JFR violate? This is throwing away >>>>>> all the checks that guard against incorrect monitor use. It's not >>>>>> just about whether you'd block trying to acquire the Monitor, it's >>>>>> also about whether it is safe to acquire it from that code/thread >>>>>> in the first place. (Though I think some of the checks in there >>>>>> should also be considering the value of _safepoint_check_required.) >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>> >>>>>> On 25/05/2018 4:39 PM, Per Liden wrote: >>>>>>> In debug builds, Monitor::try_lock() calls check_prelock_state() >>>>>>> to check the thread state, etc. The intention is to verify that >>>>>>> the call is made from the correct context, a context where we're >>>>>>> allowed to block and potentially safepoint. Unlike >>>>>>> Monitor::lock(), Monitor::try_lock() will never block, hence the >>>>>>> call to check_prelock_state() is overly strict and we should >>>>>>> remove it. Removing it would match the behavior of all other >>>>>>> non-blocking functions, like >>>>>>> Monitor::lock_without_safepoint_check(), which doesn't call >>>>>>> check_prelock_state() either (for a good reason). >>>>>>> The specific problem I've run into with this is related to JFR. >>>>>>> Monitor::try_lock() is by JFR to allow non-blocking event >>>>>>> generation, so that you can generate JFR events from "any" >>>>>>> context without risk blocking/safepointing (the logic is doing >>>>>>> something like, if try_lock() fails then put the event on a >>>>>>> different queue and let the next owner of the lock handle it). >>>>>>> The overly strict checks done by check_prelock_state() in >>>>>>> try_lock() breaks this logic, which in turn means that you can't >>>>>>> generate JFR event from "any" context as was intended. >>>>>>> The patch to fix this is a one-liner, just remove the call to >>>>>>> check_prelock_state(). >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203817 >>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8203817/webrev.0 >>>>>>> /Per From per.liden at oracle.com Fri May 25 13:23:09 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 25 May 2018 15:23:09 +0200 Subject: RFR: 8203817: Monitor::try_lock() should not call check_prelock_state() In-Reply-To: <287080f3-79b0-06d7-6eb2-ae8932e9d0a2@oracle.com> References: <7182fe2f-bd59-c677-96fc-07cd82e2ec6c@oracle.com> <575cbdbd-405c-161d-3582-cf88bf53feea@oracle.com> <4efa9c7d-64da-a8c2-f3db-aa3a13a59d73@oracle.com> <585ac20e-a74f-7ba8-f5ce-e642df8e98d0@oracle.com> <3aa42dad-8ab1-09fc-fe99-ef156072af4c@oracle.com> <287080f3-79b0-06d7-6eb2-ae8932e9d0a2@oracle.com> Message-ID: <6253bee2-44a5-c190-ccbb-5de7435aaa59@oracle.com> On 05/25/2018 01:47 PM, Per Liden wrote: > On 05/25/2018 11:45 AM, David Holmes wrote: >> On 25/05/2018 7:33 PM, Per Liden wrote: >>> Hi David, >>> >>> On 05/25/2018 09:51 AM, David Holmes wrote: >>>> Hi Erik, >>>> >>>> On 25/05/2018 5:17 PM, Erik Osterlund wrote: >>>>> Hi David, >>>>> >>>>> The change Per is proposing would make try_lock perform the same >>>>> checks that locking without safepoint checks does. >>>> >>>> Yet locking without a safepoint check also ensures >>>> _safepoint_check_required != Monitor::_safepoint_check_always. But >>>> try_lock does not and can be applied to both kinds of >>>> Monitors/Mutexes. Further it also has a stated, but not checked, >>>> precondition that the thread is not _thread_in_vm - though that may >>>> only be an issue if we were to block. >>>> >>>> I'm looking at the checks in check_prelock_state wondering which are >>>> only truly needed if we risk blocking? I think most of them, with >>>> the exception of is_crash_protected. Are we guaranteed never to run >>>> this JFR code on the WatcherThread? >>>> >>>> Though I'm also concerned about lock-ranking and deadlocks if we >>>> try_lock multiple locks. Notwithstanding the rank/deadlock code is >>>> far from ideal already. >>>> >>>> I don't mind removing checks/guards that really don't apply in the >>>> try_lock case, but I'm wary of removing all guards without being >>>> more sure of that. >>> >>> Ok, so how about we just avoid the bad parts, by doing this: >>> >>> http://cr.openjdk.java.net/~pliden/8203817/webrev.1 >> >> That seems a reasonable compromise. > > Good! Thanks for looking at this David! Erik, are you also happy with this compromise? /Per > > /Per > >> >> There may still be possible deadlock issues if we use try-lock to >> acquire multiple locks, but we probably don't do that ... and as Erik >> noted there are issues with the ranking/deadlock detection anyway. >> >> Thanks, >> David >> >>> /Per >>> >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> Perhaps locking without safepont checks could perform more sanity >>>>> checks, but that is a separate issue. I think try_lock should >>>>> perform the same checks that locking without safepoint checks does. >>>>> The alternatives are then to >>>>> >>>>> 1) Remove checking the prelock state like Per suggests for try_lock >>>>> (then they do the same checks), or >>>>> 2) Overhaul the safepoint checking refactoring out the bits that >>>>> check the safepointing sanity chrcka from other deadlock checks >>>>> (and correct those to check for rank <= special, and not == >>>>> special), remove the safepoint checking part from try_lock and >>>>> adding the deadlock checking parts to lock without safepoints. >>>>> >>>>> Doing #2 seems like a different RFE. In fact I believe that would >>>>> be https://bugs.openjdk.java.net/browse/JDK-8184732 >>>>> ?that >>>>> I filed a while back. >>>>> >>>>> In summary, there is a whole bunch of problems in the deadlock >>>>> detection system, and #2 makes it hard to not get dragged down in >>>>> the rabbit hole. #1 is sufficient to ?make try_lock check as much >>>>> (or little) as locking without safepoint checking. And I think that >>>>> is enough for the scope of this change. >>>>> >>>>> Looks good to me. >>>>> >>>>> Thanks, >>>>> /Erik >>>>> >>>>> On 25 May 2018, at 08:53, David Holmes >>>> > wrote: >>>>> >>>>>> Hi Per, >>>>>> >>>>>> Exactly what condition(s) does JFR violate? This is throwing away >>>>>> all the checks that guard against incorrect monitor use. It's not >>>>>> just about whether you'd block trying to acquire the Monitor, it's >>>>>> also about whether it is safe to acquire it from that code/thread >>>>>> in the first place. (Though I think some of the checks in there >>>>>> should also be considering the value of _safepoint_check_required.) >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>> >>>>>> On 25/05/2018 4:39 PM, Per Liden wrote: >>>>>>> In debug builds, Monitor::try_lock() calls check_prelock_state() >>>>>>> to check the thread state, etc. The intention is to verify that >>>>>>> the call is made from the correct context, a context where we're >>>>>>> allowed to block and potentially safepoint. Unlike >>>>>>> Monitor::lock(), Monitor::try_lock() will never block, hence the >>>>>>> call to check_prelock_state() is overly strict and we should >>>>>>> remove it. Removing it would match the behavior of all other >>>>>>> non-blocking functions, like >>>>>>> Monitor::lock_without_safepoint_check(), which doesn't call >>>>>>> check_prelock_state() either (for a good reason). >>>>>>> The specific problem I've run into with this is related to JFR. >>>>>>> Monitor::try_lock() is by JFR to allow non-blocking event >>>>>>> generation, so that you can generate JFR events from "any" >>>>>>> context without risk blocking/safepointing (the logic is doing >>>>>>> something like, if try_lock() fails then put the event on a >>>>>>> different queue and let the next owner of the lock handle it). >>>>>>> The overly strict checks done by check_prelock_state() in >>>>>>> try_lock() breaks this logic, which in turn means that you can't >>>>>>> generate JFR event from "any" context as was intended. >>>>>>> The patch to fix this is a one-liner, just remove the call to >>>>>>> check_prelock_state(). >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203817 >>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8203817/webrev.0 >>>>>>> /Per From kim.barrett at oracle.com Fri May 25 13:55:10 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 25 May 2018 15:55:10 +0200 Subject: RFR: 8202813: Move vm_weak processing from SystemDictionary::do_unloading to WeakProcessor In-Reply-To: References: <10a99bf5-d3cc-9d5b-8a38-5ba1672f9f8b@oracle.com> <4E15250A-9872-4195-892B-C1E0078A80FD@oracle.com> <6dc7d3f8-dd9f-0990-860d-3b9423c1e271@oracle.com> Message-ID: > On May 22, 2018, at 8:27 PM, Kim Barrett wrote: > >> On May 22, 2018, at 10:09 AM, Kim Barrett wrote: >> New webrev: >> http://cr.openjdk.java.net/~kbarrett/8202813/open.01/ > > I forgot to remove the no longer used is_alive argument for SystemDictionary::do_unloading. > > Updated webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8202813/open.02/ > incr: http://cr.openjdk.java.net/~kbarrett/8202813/open.02.inc/ After some offline discussion, Coleen wants to take this in a somewhat different direction, so I?m passing the baton off to her. From erik.osterlund at oracle.com Fri May 25 14:02:15 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Fri, 25 May 2018 16:02:15 +0200 Subject: RFR: 8203817: Monitor::try_lock() should not call check_prelock_state() In-Reply-To: <6253bee2-44a5-c190-ccbb-5de7435aaa59@oracle.com> References: <7182fe2f-bd59-c677-96fc-07cd82e2ec6c@oracle.com> <575cbdbd-405c-161d-3582-cf88bf53feea@oracle.com> <4efa9c7d-64da-a8c2-f3db-aa3a13a59d73@oracle.com> <585ac20e-a74f-7ba8-f5ce-e642df8e98d0@oracle.com> <3aa42dad-8ab1-09fc-fe99-ef156072af4c@oracle.com> <287080f3-79b0-06d7-6eb2-ae8932e9d0a2@oracle.com> <6253bee2-44a5-c190-ccbb-5de7435aaa59@oracle.com> Message-ID: <5B081767.7070201@oracle.com> Hi Per, Yes. Looks good to me. Thanks, /Erik On 2018-05-25 15:23, Per Liden wrote: > On 05/25/2018 01:47 PM, Per Liden wrote: >> On 05/25/2018 11:45 AM, David Holmes wrote: >>> On 25/05/2018 7:33 PM, Per Liden wrote: >>>> Hi David, >>>> >>>> On 05/25/2018 09:51 AM, David Holmes wrote: >>>>> Hi Erik, >>>>> >>>>> On 25/05/2018 5:17 PM, Erik Osterlund wrote: >>>>>> Hi David, >>>>>> >>>>>> The change Per is proposing would make try_lock perform the same >>>>>> checks that locking without safepoint checks does. >>>>> >>>>> Yet locking without a safepoint check also ensures >>>>> _safepoint_check_required != Monitor::_safepoint_check_always. But >>>>> try_lock does not and can be applied to both kinds of >>>>> Monitors/Mutexes. Further it also has a stated, but not checked, >>>>> precondition that the thread is not _thread_in_vm - though that >>>>> may only be an issue if we were to block. >>>>> >>>>> I'm looking at the checks in check_prelock_state wondering which >>>>> are only truly needed if we risk blocking? I think most of them, >>>>> with the exception of is_crash_protected. Are we guaranteed never >>>>> to run this JFR code on the WatcherThread? >>>>> >>>>> Though I'm also concerned about lock-ranking and deadlocks if we >>>>> try_lock multiple locks. Notwithstanding the rank/deadlock code is >>>>> far from ideal already. >>>>> >>>>> I don't mind removing checks/guards that really don't apply in the >>>>> try_lock case, but I'm wary of removing all guards without being >>>>> more sure of that. >>>> >>>> Ok, so how about we just avoid the bad parts, by doing this: >>>> >>>> http://cr.openjdk.java.net/~pliden/8203817/webrev.1 >>> >>> That seems a reasonable compromise. >> >> Good! Thanks for looking at this David! > > Erik, are you also happy with this compromise? > > /Per > >> >> /Per >> >>> >>> There may still be possible deadlock issues if we use try-lock to >>> acquire multiple locks, but we probably don't do that ... and as >>> Erik noted there are issues with the ranking/deadlock detection anyway. >>> >>> Thanks, >>> David >>> >>>> /Per >>>> >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> Perhaps locking without safepont checks could perform more sanity >>>>>> checks, but that is a separate issue. I think try_lock should >>>>>> perform the same checks that locking without safepoint checks >>>>>> does. The alternatives are then to >>>>>> >>>>>> 1) Remove checking the prelock state like Per suggests for >>>>>> try_lock (then they do the same checks), or >>>>>> 2) Overhaul the safepoint checking refactoring out the bits that >>>>>> check the safepointing sanity chrcka from other deadlock checks >>>>>> (and correct those to check for rank <= special, and not == >>>>>> special), remove the safepoint checking part from try_lock and >>>>>> adding the deadlock checking parts to lock without safepoints. >>>>>> >>>>>> Doing #2 seems like a different RFE. In fact I believe that would >>>>>> be https://bugs.openjdk.java.net/browse/JDK-8184732 >>>>>> that >>>>>> I filed a while back. >>>>>> >>>>>> In summary, there is a whole bunch of problems in the deadlock >>>>>> detection system, and #2 makes it hard to not get dragged down in >>>>>> the rabbit hole. #1 is sufficient to make try_lock check as much >>>>>> (or little) as locking without safepoint checking. And I think >>>>>> that is enough for the scope of this change. >>>>>> >>>>>> Looks good to me. >>>>>> >>>>>> Thanks, >>>>>> /Erik >>>>>> >>>>>> On 25 May 2018, at 08:53, David Holmes >>>>> > wrote: >>>>>> >>>>>>> Hi Per, >>>>>>> >>>>>>> Exactly what condition(s) does JFR violate? This is throwing >>>>>>> away all the checks that guard against incorrect monitor use. >>>>>>> It's not just about whether you'd block trying to acquire the >>>>>>> Monitor, it's also about whether it is safe to acquire it from >>>>>>> that code/thread in the first place. (Though I think some of the >>>>>>> checks in there should also be considering the value of >>>>>>> _safepoint_check_required.) >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>> >>>>>>> On 25/05/2018 4:39 PM, Per Liden wrote: >>>>>>>> In debug builds, Monitor::try_lock() calls >>>>>>>> check_prelock_state() to check the thread state, etc. The >>>>>>>> intention is to verify that the call is made from the correct >>>>>>>> context, a context where we're allowed to block and potentially >>>>>>>> safepoint. Unlike Monitor::lock(), Monitor::try_lock() will >>>>>>>> never block, hence the call to check_prelock_state() is overly >>>>>>>> strict and we should remove it. Removing it would match the >>>>>>>> behavior of all other non-blocking functions, like >>>>>>>> Monitor::lock_without_safepoint_check(), which doesn't call >>>>>>>> check_prelock_state() either (for a good reason). >>>>>>>> The specific problem I've run into with this is related to JFR. >>>>>>>> Monitor::try_lock() is by JFR to allow non-blocking event >>>>>>>> generation, so that you can generate JFR events from "any" >>>>>>>> context without risk blocking/safepointing (the logic is doing >>>>>>>> something like, if try_lock() fails then put the event on a >>>>>>>> different queue and let the next owner of the lock handle it). >>>>>>>> The overly strict checks done by check_prelock_state() in >>>>>>>> try_lock() breaks this logic, which in turn means that you >>>>>>>> can't generate JFR event from "any" context as was intended. >>>>>>>> The patch to fix this is a one-liner, just remove the call to >>>>>>>> check_prelock_state(). >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203817 >>>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8203817/webrev.0 >>>>>>>> /Per From per.liden at oracle.com Fri May 25 14:31:35 2018 From: per.liden at oracle.com (Per Liden) Date: Fri, 25 May 2018 16:31:35 +0200 Subject: RFR: 8203817: Monitor::try_lock() should not call check_prelock_state() In-Reply-To: <5B081767.7070201@oracle.com> References: <7182fe2f-bd59-c677-96fc-07cd82e2ec6c@oracle.com> <575cbdbd-405c-161d-3582-cf88bf53feea@oracle.com> <4efa9c7d-64da-a8c2-f3db-aa3a13a59d73@oracle.com> <585ac20e-a74f-7ba8-f5ce-e642df8e98d0@oracle.com> <3aa42dad-8ab1-09fc-fe99-ef156072af4c@oracle.com> <287080f3-79b0-06d7-6eb2-ae8932e9d0a2@oracle.com> <6253bee2-44a5-c190-ccbb-5de7435aaa59@oracle.com> <5B081767.7070201@oracle.com> Message-ID: <0a1f8fae-eca8-d823-b872-b405a8490411@oracle.com> Thanks Erik! /Per On 05/25/2018 04:02 PM, Erik ?sterlund wrote: > Hi Per, > > Yes. Looks good to me. > > Thanks, > /Erik > > On 2018-05-25 15:23, Per Liden wrote: >> On 05/25/2018 01:47 PM, Per Liden wrote: >>> On 05/25/2018 11:45 AM, David Holmes wrote: >>>> On 25/05/2018 7:33 PM, Per Liden wrote: >>>>> Hi David, >>>>> >>>>> On 05/25/2018 09:51 AM, David Holmes wrote: >>>>>> Hi Erik, >>>>>> >>>>>> On 25/05/2018 5:17 PM, Erik Osterlund wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> The change Per is proposing would make try_lock perform the same >>>>>>> checks that locking without safepoint checks does. >>>>>> >>>>>> Yet locking without a safepoint check also ensures >>>>>> _safepoint_check_required != Monitor::_safepoint_check_always. But >>>>>> try_lock does not and can be applied to both kinds of >>>>>> Monitors/Mutexes. Further it also has a stated, but not checked, >>>>>> precondition that the thread is not _thread_in_vm - though that >>>>>> may only be an issue if we were to block. >>>>>> >>>>>> I'm looking at the checks in check_prelock_state wondering which >>>>>> are only truly needed if we risk blocking? I think most of them, >>>>>> with the exception of is_crash_protected. Are we guaranteed never >>>>>> to run this JFR code on the WatcherThread? >>>>>> >>>>>> Though I'm also concerned about lock-ranking and deadlocks if we >>>>>> try_lock multiple locks. Notwithstanding the rank/deadlock code is >>>>>> far from ideal already. >>>>>> >>>>>> I don't mind removing checks/guards that really don't apply in the >>>>>> try_lock case, but I'm wary of removing all guards without being >>>>>> more sure of that. >>>>> >>>>> Ok, so how about we just avoid the bad parts, by doing this: >>>>> >>>>> http://cr.openjdk.java.net/~pliden/8203817/webrev.1 >>>> >>>> That seems a reasonable compromise. >>> >>> Good! Thanks for looking at this David! >> >> Erik, are you also happy with this compromise? >> >> /Per >> >>> >>> /Per >>> >>>> >>>> There may still be possible deadlock issues if we use try-lock to >>>> acquire multiple locks, but we probably don't do that ... and as >>>> Erik noted there are issues with the ranking/deadlock detection anyway. >>>> >>>> Thanks, >>>> David >>>> >>>>> /Per >>>>> >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> Perhaps locking without safepont checks could perform more sanity >>>>>>> checks, but that is a separate issue. I think try_lock should >>>>>>> perform the same checks that locking without safepoint checks >>>>>>> does. The alternatives are then to >>>>>>> >>>>>>> 1) Remove checking the prelock state like Per suggests for >>>>>>> try_lock (then they do the same checks), or >>>>>>> 2) Overhaul the safepoint checking refactoring out the bits that >>>>>>> check the safepointing sanity chrcka from other deadlock checks >>>>>>> (and correct those to check for rank <= special, and not == >>>>>>> special), remove the safepoint checking part from try_lock and >>>>>>> adding the deadlock checking parts to lock without safepoints. >>>>>>> >>>>>>> Doing #2 seems like a different RFE. In fact I believe that would >>>>>>> be https://bugs.openjdk.java.net/browse/JDK-8184732 >>>>>>> >>>>>>> that I filed a while back. >>>>>>> >>>>>>> In summary, there is a whole bunch of problems in the deadlock >>>>>>> detection system, and #2 makes it hard to not get dragged down in >>>>>>> the rabbit hole. #1 is sufficient to? make try_lock check as much >>>>>>> (or little) as locking without safepoint checking. And I think >>>>>>> that is enough for the scope of this change. >>>>>>> >>>>>>> Looks good to me. >>>>>>> >>>>>>> Thanks, >>>>>>> /Erik >>>>>>> >>>>>>> On 25 May 2018, at 08:53, David Holmes >>>>>> > wrote: >>>>>>> >>>>>>>> Hi Per, >>>>>>>> >>>>>>>> Exactly what condition(s) does JFR violate? This is throwing >>>>>>>> away all the checks that guard against incorrect monitor use. >>>>>>>> It's not just about whether you'd block trying to acquire the >>>>>>>> Monitor, it's also about whether it is safe to acquire it from >>>>>>>> that code/thread in the first place. (Though I think some of the >>>>>>>> checks in there should also be considering the value of >>>>>>>> _safepoint_check_required.) >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>> >>>>>>>> On 25/05/2018 4:39 PM, Per Liden wrote: >>>>>>>>> In debug builds, Monitor::try_lock() calls >>>>>>>>> check_prelock_state() to check the thread state, etc. The >>>>>>>>> intention is to verify that the call is made from the correct >>>>>>>>> context, a context where we're allowed to block and potentially >>>>>>>>> safepoint. Unlike Monitor::lock(), Monitor::try_lock() will >>>>>>>>> never block, hence the call to check_prelock_state() is overly >>>>>>>>> strict and we should remove it. Removing it would match the >>>>>>>>> behavior of all other non-blocking functions, like >>>>>>>>> Monitor::lock_without_safepoint_check(), which doesn't call >>>>>>>>> check_prelock_state() either (for a good reason). >>>>>>>>> The specific problem I've run into with this is related to JFR. >>>>>>>>> Monitor::try_lock() is by JFR to allow non-blocking event >>>>>>>>> generation, so that you can generate JFR events from "any" >>>>>>>>> context without risk blocking/safepointing (the logic is doing >>>>>>>>> something like, if try_lock() fails then put the event on a >>>>>>>>> different queue and let the next owner of the lock handle it). >>>>>>>>> The overly strict checks done by check_prelock_state() in >>>>>>>>> try_lock() breaks this logic, which in turn means that you >>>>>>>>> can't generate JFR event from "any" context as was intended. >>>>>>>>> The patch to fix this is a one-liner, just remove the call to >>>>>>>>> check_prelock_state(). >>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203817 >>>>>>>>> Webrev: http://cr.openjdk.java.net/~pliden/8203817/webrev.0 >>>>>>>>> /Per > From shade at redhat.com Fri May 25 14:34:21 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 25 May 2018 16:34:21 +0200 Subject: RFR: AArch64: 8203699 java/lang/invoke/SpecialInterfaceCall fails with SIGILL on aarch64 In-Reply-To: <3ca651dc-67c1-d5b8-dd87-9e9a7c2d20e8@redhat.com> References: <887ea79a-86d6-8763-05c8-9e4cce13310c@redhat.com> <33316793-e031-3bbb-d8a1-b0e0c29e0e03@redhat.com> <622ae8b0-73b1-c992-ce7a-9f405c649b31@redhat.com> <3ca651dc-67c1-d5b8-dd87-9e9a7c2d20e8@redhat.com> Message-ID: On 05/25/2018 11:01 AM, Andrew Haley wrote: > On 05/25/2018 01:40 AM, White, Derek wrote: >> OK, fine. Although I see no pragmatic reason to spend more time on >> polishing this code, and forcing reviewers to re-review and re-test. I assume >> you have tested the new change? > > OK, but if you've not committed it yet please move this comment next > to the instruction: > > // Get super_klass value into r0 (even if it was in r5 or r2). Haven't pushed yet, just retested. Is this what you want? diff -r 5f4f5b52ee39 src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp --- a/src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp Fri May 25 15:34:45 2018 +0530 +++ b/src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp Fri May 25 16:34:13 2018 +0200 @@ -1224,7 +1224,6 @@ assert(sub_klass != r0, "killed reg"); // killed by mov(r0, super) assert(sub_klass != r2, "killed reg"); // killed by lea(r2, &pst_counter) - // Get super_klass value into r0 (even if it was in r5 or r2). RegSet pushed_registers; if (!IS_A_TEMP(r2)) pushed_registers += r2; if (!IS_A_TEMP(r5)) pushed_registers += r5; @@ -1235,6 +1234,11 @@ push(pushed_registers, sp); + // Get super_klass value into r0 (even if it was in r5 or r2). + if (super_klass != r0) { + mov(r0, super_klass); + } + #ifndef PRODUCT mov(rscratch2, (address)&SharedRuntime::_partial_subtype_ctr); Address pst_counter_addr(rscratch2); -Aleksey From coleen.phillimore at oracle.com Fri May 25 15:32:54 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Fri, 25 May 2018 11:32:54 -0400 Subject: RFR (M) 8202813: Move vm_weak processing from SystemDictionary to WeakProcessor Message-ID: <4e7c8f9f-e1fa-0a29-ac22-5215b55dee69@oracle.com> Summary: SystemDictionary has all strong roots.? The weak oop_storage is processed by the WeakProcessor so it can be scanned and cleared concurrently and/or by parallel threads. Please see bug for explanation. open webrev at http://cr.openjdk.java.net/~coleenp/8202813.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8202813 There might be more GC code that can be cleaned out since G1RootClosures->weak_oops() seems unused now, but I'd like to defer that to a smaller cleanup.? Also ClassLoaderData is still processed inconsistently by the GCs, which would be nice to clean up. Tested with mach5 hs-tier1-5 with no failures.? Per's gc-test-suite, dev-submit for performance testing, and runThese with all collectors, with and without -XX:-ClassUnloading.? Also tested with class unloading gc tests (which are also in hs-tier number tests). Thanks, Coleen From kim.barrett at oracle.com Fri May 25 16:01:01 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 25 May 2018 18:01:01 +0200 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: <7e7fc484287e4da4926176e0b5ae1b64@sap.com> References: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> <7e7fc484287e4da4926176e0b5ae1b64@sap.com> Message-ID: <339D50B6-09D5-4342-A687-918A9A096B39@oracle.com> > On May 22, 2018, at 12:16 PM, Doerr, Martin wrote: > > Hi Kim, > > I can't see how a new implicit consume is introduced by Michihiro's change. He just explained how the existing code works. > > If implicit consume has been rejected the current code is wrong: > "new_obj = o->forwardee();" would need some kind of barrier before using the new_obj. > > But this issue is not related to what Michihiro wants to change AFAICS. The current full-fence CAS guarantees the stores into the new forwardee installed by the CAS happen before loads from that object after the CAS. Algorithmically, o->forwardee() is guaranteed to be the same object as was returned by the CAS. Hence, loads from the forwardee are being ordered by the fenced CAS. I've discussed this with others on the GC team; we think the minimal required barriers are CAS with memory_order_acq_rel, plus an acquire barrier on the else branch of 122 if (!test_mark->is_marked()) { ... 261 } else { 262 assert(o->is_forwarded(), "Sanity"); 263 new_obj = o->forwardee(); 264 } We've not done enough analysis to show this is sufficient, but we think anything weaker is not sufficient for shared code. > > Best regards, > Martin > > > -----Original Message----- > From: ppc-aix-port-dev [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] On Behalf Of Kim Barrett > Sent: Montag, 21. Mai 2018 06:00 > To: Michihiro Horie > Cc: hotspot-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; Gustavo Bueno Romero ; ppc-aix-port-dev at openjdk.java.net; david.holmes at oracle.com > Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 > >> On May 18, 2018, at 5:12 PM, Michihiro Horie wrote: >> >> Dear all, >> >> I update the webrev: http://cr.openjdk.java.net/~mhorie/8154736/webrev.09/ >> >> With the release barrier before the CAS, new_obj can be observed from other threads. If the CAS succeeds, the current thread can use new_obj without barriers. If the CAS fails, "o->forwardee()" is ordered with respect to CAS by accessing the same memory location "_mark", so no barriers needed. The order of (1) access to the forwardee and (2) access to forwardee's fields is preserved due to Release-Consume ordering on supported platforms. (The ordering between "new_obj = o->forwardee();" and logging or other usages is not changed.) >> >> Regarding the maintainability, the requirement is CAS(memory_order_release) as Release-Consume to be consistent with C++11. This requirement is necessary when a weaker platform like DEC Alpha is to be supported. On currently supported platforms, code change can be safe if the code meats this requirement (and the order of (1) access to the forwardee and (2) access to forwardee's fields is the natural way of coding). > > Relying on implicit consume has been been discussed and rejected, in > the earlier thread on this topic and I think elsewhere too. > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-October/021538.html From aph at redhat.com Fri May 25 16:12:21 2018 From: aph at redhat.com (Andrew Haley) Date: Fri, 25 May 2018 17:12:21 +0100 Subject: RFR: AArch64: 8203699 java/lang/invoke/SpecialInterfaceCall fails with SIGILL on aarch64 In-Reply-To: References: <887ea79a-86d6-8763-05c8-9e4cce13310c@redhat.com> <33316793-e031-3bbb-d8a1-b0e0c29e0e03@redhat.com> <622ae8b0-73b1-c992-ce7a-9f405c649b31@redhat.com> <3ca651dc-67c1-d5b8-dd87-9e9a7c2d20e8@redhat.com> Message-ID: <98048e5a-27fd-6173-e39b-9f63b3d6ff26@redhat.com> On 05/25/2018 03:34 PM, Aleksey Shipilev wrote: > Haven't pushed yet, just retested. Is this what you want? Yes. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From frank.yuan at oracle.com Mon May 28 01:12:42 2018 From: frank.yuan at oracle.com (Frank Yuan) Date: Mon, 28 May 2018 09:12:42 +0800 Subject: JEP: JWarmup precompile java hot methods at application startup In-Reply-To: <20180525132251.552041504@eggemoggin.niobe.net> References: <20180525132251.552041504@eggemoggin.niobe.net> Message-ID: <007201d3f620$fce0be90$f6a23bb0$@oracle.com> > 2018/5/25 10:00:17 -0700, yumin.qi at gmail.com: > > Hi, Mark > > > > Proposed a draft of new JEP: > > https://bugs.openjdk.java.net/browse/JDK-8203832 > > > > We (Alibaba Group Inc) have an implementation of warmup performance > > solution and want to contribute to OpenJDK, wish it can help java > > developers to resolve the same problems we have encountered. > > > > Please take a look and wait for a discussion/evaluation on it. > > I suggest that you ask hotspot-dev for feedback. > John Rose may be the right person :) Frank > - Mark From erik.osterlund at oracle.com Mon May 28 06:48:19 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Mon, 28 May 2018 08:48:19 +0200 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: References: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> <7e7fc484287e4da4926176e0b5ae1b64@sap.com> <339D50B6-09D5-4342-A687-918A9A096B39@oracle.com> Message-ID: <6D2268EC-C1E7-4C90-BCD3-90D02D21FA08@oracle.com> Hi Michihiro, In your analysis, you state that the failing CAS path today already relies on implicit consume ordering as reading forwardee() after the failed CAS is missing acquire and hence accesses into the new reloaded forwardee would rely on (implicit) data dependencies to the reloaded forwardee. That part of the analysis seems wrong to me. Since today even a failed CAS has acquire semantics (and stronger), and the reloaded forwardee always has the same value as was observed in the failed cas (in this context), all data dependency requirements to the reloaded forwardee are therefore no longer needed or relied upon. We do not use implicit consume in the shared C++ code. If you find any instances of that, it is a bug and should be purged with fire. Even explicit consume is currently strongly discouraged. Implicit consume is unreliable, especially in a project with many platforms. If you insist on using more fragile semantics that are known to be unreliable, I would like to at least know what measurable performance difference you observe between the semantics Kim proposed, compared to the elided acquire variant you insist on. My gut feeling tells me that double sync is very intrusive, but an isync scheduled almost immediately after an lwsync, should be significantly less intrusive. Thanks, /Erik > On 28 May 2018, at 03:28, Michihiro Horie wrote: > > Hi Kim, > > >I've discussed this with others on the GC team; we think the minimal > >required barriers are CAS with memory_order_acq_rel, plus an acquire > >barrier on the else branch of > > > > 122 if (!test_mark->is_marked()) { > >... > > 261 } else { > > 262 assert(o->is_forwarded(), "Sanity"); > > 263 new_obj = o->forwardee(); > > 264 } > > > >We've not done enough analysis to show this is sufficient, but we > >think anything weaker is not sufficient for shared code. > > Thank you for the discussions on your side with the GC team. > I summarized the point on why my change works as follows. Hope we are on the same page with this. > > > 1. Current implementation > > PSPromotionManager::copy_to_survivor_space is used to move live > objects to a different location. It uses a forwarding technique and > allows multiple threads to compete for performing the copy step. > > The first thread succeeds in installing its copy in the old object as > forwardee. Other threads may need to discard their copy and use the > one generated by the first thread which has won the race. > > Written program order: > (1) create new_obj as copy of obj > (2) full fence > (3) CAS to set the forwardee with new_obj > (4) full fence > (5) access to the new_obj's field if CAS succeeds > (6) access to the forwardee with "o->forwardee()" if CAS fails > (7) access to the forwardee's field if debugging is on > > When thread0 succeeds in CAS at (3), the copied new_obj by thread0 > must be accessible from thread1 at (6). (2) guarantees the order of > (1) and (3), although it is stronger than needed for the purpose of > ensuring a consistent view of copied new_obj from thread1. > > (5), (6), and (7) must be executed after (3). Apparently, (4) looks > guranteeing the order, although it is redundant. > > The order of (6) and (7) is guaranteed by consume. > (5) and (6) are on different control paths. > (5) and (7): Thread0 owns new_obj when CAS succeeded and can access it > without barrier. > > > 2. Proposed change > > Written program order: > (1) create new_obj as copy of obj > (2) release fence > (3) CAS to set the forwardee with new_obj > (4) no fence > (5) access to the new_obj's field if CAS succeeds > (6) access to the forwardee with "o->forwardee()" if CAS fails > (7) access to the forwardee's field if debugging is on > > Release fence at (2) is sufficient to make the copied new_obj > accessible from a thread that fails in CAS. > > No fence at (4) is acceptable because it is redundant. > > The order of (5), (6), and (7) is the same as the current > implementation. It is not affected by the proposed change. > > > 3. Reason why this is sufficient > > Memory coherence guarantees that all the threads share a consistent > view on the access to the same memory location, which is "_mark" in > the target code. Thread0 writes the "_mark" when it succeeds in > CAS at (3) and thread1 reads the "_mark" when it failes in CAS at (3). > Thread1 also reads the "_mark" by invoking "o->forwardee()" at (6). > (See CoRR1 in Section 8 of > https://www.cl.cam.ac.uk/~pes20/ppc-supplemental/test7.pdf) > > Also, compilers do not speculatively load "o->forwardee()" at (6) > before the CAS at (3). This is ensured by the integrated compiler > barriers (clobber "memory" in the volatile inline asm code). And it is > also prevented because "_mark" is declared volatile. > > > > Best regards, > -- > Michihiro, > IBM Research - Tokyo > > Kim Barrett ---2018/05/26 01:01:40---> On May 22, 2018, at 12:16 PM, Doerr, Martin wrote: > > > From: Kim Barrett > To: "Doerr, Martin" > Cc: Michihiro Horie , "hotspot-dev at openjdk.java.net" , "hotspot-gc-dev at openjdk.java.net" , Gustavo Bueno Romero , "ppc-aix-port-dev at openjdk.java.net" , "david.holmes at oracle.com" > Date: 2018/05/26 01:01 > Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 > > > > > > On May 22, 2018, at 12:16 PM, Doerr, Martin wrote: > > > > Hi Kim, > > > > I can't see how a new implicit consume is introduced by Michihiro's change. He just explained how the existing code works. > > > > If implicit consume has been rejected the current code is wrong: > > "new_obj = o->forwardee();" would need some kind of barrier before using the new_obj. > > > > But this issue is not related to what Michihiro wants to change AFAICS. > > The current full-fence CAS guarantees the stores into the new > forwardee installed by the CAS happen before loads from that object > after the CAS. Algorithmically, o->forwardee() is guaranteed to be > the same object as was returned by the CAS. Hence, loads from the > forwardee are being ordered by the fenced CAS. > > I've discussed this with others on the GC team; we think the minimal > required barriers are CAS with memory_order_acq_rel, plus an acquire > barrier on the else branch of > > 122 if (!test_mark->is_marked()) { > ... > 261 } else { > 262 assert(o->is_forwarded(), "Sanity"); > 263 new_obj = o->forwardee(); > 264 } > > We've not done enough analysis to show this is sufficient, but we > think anything weaker is not sufficient for shared code. > > > > > > Best regards, > > Martin > > > > > > -----Original Message----- > > From: ppc-aix-port-dev [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] On Behalf Of Kim Barrett > > Sent: Montag, 21. Mai 2018 06:00 > > To: Michihiro Horie > > Cc: hotspot-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; Gustavo Bueno Romero ; ppc-aix-port-dev at openjdk.java.net; david.holmes at oracle.com > > Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 > > > >> On May 18, 2018, at 5:12 PM, Michihiro Horie wrote: > >> > >> Dear all, > >> > >> I update the webrev: http://cr.openjdk.java.net/~mhorie/8154736/webrev.09/ > >> > >> With the release barrier before the CAS, new_obj can be observed from other threads. If the CAS succeeds, the current thread can use new_obj without barriers. If the CAS fails, "o->forwardee()" is ordered with respect to CAS by accessing the same memory location "_mark", so no barriers needed. The order of (1) access to the forwardee and (2) access to forwardee's fields is preserved due to Release-Consume ordering on supported platforms. (The ordering between "new_obj = o->forwardee();" and logging or other usages is not changed.) > >> > >> Regarding the maintainability, the requirement is CAS(memory_order_release) as Release-Consume to be consistent with C++11. This requirement is necessary when a weaker platform like DEC Alpha is to be supported. On currently supported platforms, code change can be safe if the code meats this requirement (and the order of (1) access to the forwardee and (2) access to forwardee's fields is the natural way of coding). > > > > Relying on implicit consume has been been discussed and rejected, in > > the earlier thread on this topic and I think elsewhere too. > > > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-October/021538.html > > > > > From martin.doerr at sap.com Mon May 28 09:05:51 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 28 May 2018 09:05:51 +0000 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: <6D2268EC-C1E7-4C90-BCD3-90D02D21FA08@oracle.com> References: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> <7e7fc484287e4da4926176e0b5ae1b64@sap.com> <339D50B6-09D5-4342-A687-918A9A096B39@oracle.com> <6D2268EC-C1E7-4C90-BCD3-90D02D21FA08@oracle.com> Message-ID: <6a2e6c46299a4641b79b953952ec8c6f@sap.com> Hi everybody, thank you very much for discussing this issue and helping to fix the PPC64 performance bottleneck. Thanks, Kim and Erik, for explaining by which trick we?re using the CAS? acquire barrier in current code. Thanks, Michihiro, for explaining your current proposal which relies on consume. I?m convinced that it works for PPC and ARM (and of course for the strong memory model platforms). If I understand it correctly, acquire is desired to help compilers and the hardware barrier is not needed. The current implementation just uses our handmade inline asm code, so it?s pointless for the compilers if we use acquire or not. However, if we want to use C++11 atomics instead of our inline asm code in the future, I think memory_order_acq_rel will be recommended to avoid compiler problems. Was this the reason for the acquire or did I miss anything? From performance point of view, I think we can live with an unnecessary acquire barrier. It?s so much cheaper than a full fence. So if this is the only remaining issue, I think we could just add it. Btw. there are places in shared code where we rely on consume. I had found one and added a comment some time ago (compiledMethod.hpp): ?Note: _exception_cache may be read concurrently. We rely on memory_order_consume here.? Seems to work correctly with volatile fields. Best regards, Martin From: ppc-aix-port-dev [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] On Behalf Of Erik Osterlund Sent: Montag, 28. Mai 2018 08:48 To: Michihiro Horie Cc: hotspot-dev at openjdk.java.net; Kim Barrett ; Gustavo Bueno Romero ; ppc-aix-port-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; david.holmes at oracle.com Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 Hi Michihiro, In your analysis, you state that the failing CAS path today already relies on implicit consume ordering as reading forwardee() after the failed CAS is missing acquire and hence accesses into the new reloaded forwardee would rely on (implicit) data dependencies to the reloaded forwardee. That part of the analysis seems wrong to me. Since today even a failed CAS has acquire semantics (and stronger), and the reloaded forwardee always has the same value as was observed in the failed cas (in this context), all data dependency requirements to the reloaded forwardee are therefore no longer needed or relied upon. We do not use implicit consume in the shared C++ code. If you find any instances of that, it is a bug and should be purged with fire. Even explicit consume is currently strongly discouraged. Implicit consume is unreliable, especially in a project with many platforms. If you insist on using more fragile semantics that are known to be unreliable, I would like to at least know what measurable performance difference you observe between the semantics Kim proposed, compared to the elided acquire variant you insist on. My gut feeling tells me that double sync is very intrusive, but an isync scheduled almost immediately after an lwsync, should be significantly less intrusive. Thanks, /Erik On 28 May 2018, at 03:28, Michihiro Horie > wrote: Hi Kim, >I've discussed this with others on the GC team; we think the minimal >required barriers are CAS with memory_order_acq_rel, plus an acquire >barrier on the else branch of > > 122 if (!test_mark->is_marked()) { >... > 261 } else { > 262 assert(o->is_forwarded(), "Sanity"); > 263 new_obj = o->forwardee(); > 264 } > >We've not done enough analysis to show this is sufficient, but we >think anything weaker is not sufficient for shared code. Thank you for the discussions on your side with the GC team. I summarized the point on why my change works as follows. Hope we are on the same page with this. 1. Current implementation PSPromotionManager::copy_to_survivor_space is used to move live objects to a different location. It uses a forwarding technique and allows multiple threads to compete for performing the copy step. The first thread succeeds in installing its copy in the old object as forwardee. Other threads may need to discard their copy and use the one generated by the first thread which has won the race. Written program order: (1) create new_obj as copy of obj (2) full fence (3) CAS to set the forwardee with new_obj (4) full fence (5) access to the new_obj's field if CAS succeeds (6) access to the forwardee with "o->forwardee()" if CAS fails (7) access to the forwardee's field if debugging is on When thread0 succeeds in CAS at (3), the copied new_obj by thread0 must be accessible from thread1 at (6). (2) guarantees the order of (1) and (3), although it is stronger than needed for the purpose of ensuring a consistent view of copied new_obj from thread1. (5), (6), and (7) must be executed after (3). Apparently, (4) looks guranteeing the order, although it is redundant. The order of (6) and (7) is guaranteed by consume. (5) and (6) are on different control paths. (5) and (7): Thread0 owns new_obj when CAS succeeded and can access it without barrier. 2. Proposed change Written program order: (1) create new_obj as copy of obj (2) release fence (3) CAS to set the forwardee with new_obj (4) no fence (5) access to the new_obj's field if CAS succeeds (6) access to the forwardee with "o->forwardee()" if CAS fails (7) access to the forwardee's field if debugging is on Release fence at (2) is sufficient to make the copied new_obj accessible from a thread that fails in CAS. No fence at (4) is acceptable because it is redundant. The order of (5), (6), and (7) is the same as the current implementation. It is not affected by the proposed change. 3. Reason why this is sufficient Memory coherence guarantees that all the threads share a consistent view on the access to the same memory location, which is "_mark" in the target code. Thread0 writes the "_mark" when it succeeds in CAS at (3) and thread1 reads the "_mark" when it failes in CAS at (3). Thread1 also reads the "_mark" by invoking "o->forwardee()" at (6). (See CoRR1 in Section 8 of https://www.cl.cam.ac.uk/~pes20/ppc-supplemental/test7.pdf) Also, compilers do not speculatively load "o->forwardee()" at (6) before the CAS at (3). This is ensured by the integrated compiler barriers (clobber "memory" in the volatile inline asm code). And it is also prevented because "_mark" is declared volatile. Best regards, -- Michihiro, IBM Research - Tokyo Kim Barrett ---2018/05/26 01:01:40---> On May 22, 2018, at 12:16 PM, Doerr, Martin > wrote: > From: Kim Barrett > To: "Doerr, Martin" > Cc: Michihiro Horie >, "hotspot-dev at openjdk.java.net" >, "hotspot-gc-dev at openjdk.java.net" >, Gustavo Bueno Romero >, "ppc-aix-port-dev at openjdk.java.net" >, "david.holmes at oracle.com" > Date: 2018/05/26 01:01 Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 ________________________________ > On May 22, 2018, at 12:16 PM, Doerr, Martin > wrote: > > Hi Kim, > > I can't see how a new implicit consume is introduced by Michihiro's change. He just explained how the existing code works. > > If implicit consume has been rejected the current code is wrong: > "new_obj = o->forwardee();" would need some kind of barrier before using the new_obj. > > But this issue is not related to what Michihiro wants to change AFAICS. The current full-fence CAS guarantees the stores into the new forwardee installed by the CAS happen before loads from that object after the CAS. Algorithmically, o->forwardee() is guaranteed to be the same object as was returned by the CAS. Hence, loads from the forwardee are being ordered by the fenced CAS. I've discussed this with others on the GC team; we think the minimal required barriers are CAS with memory_order_acq_rel, plus an acquire barrier on the else branch of 122 if (!test_mark->is_marked()) { ... 261 } else { 262 assert(o->is_forwarded(), "Sanity"); 263 new_obj = o->forwardee(); 264 } We've not done enough analysis to show this is sufficient, but we think anything weaker is not sufficient for shared code. > > Best regards, > Martin > > > -----Original Message----- > From: ppc-aix-port-dev [mailto:ppc-aix-port-dev-bounces at openjdk.java.net] On Behalf Of Kim Barrett > Sent: Montag, 21. Mai 2018 06:00 > To: Michihiro Horie > > Cc: hotspot-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; Gustavo Bueno Romero >; ppc-aix-port-dev at openjdk.java.net; david.holmes at oracle.com > Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 > >> On May 18, 2018, at 5:12 PM, Michihiro Horie > wrote: >> >> Dear all, >> >> I update the webrev: http://cr.openjdk.java.net/~mhorie/8154736/webrev.09/ >> >> With the release barrier before the CAS, new_obj can be observed from other threads. If the CAS succeeds, the current thread can use new_obj without barriers. If the CAS fails, "o->forwardee()" is ordered with respect to CAS by accessing the same memory location "_mark", so no barriers needed. The order of (1) access to the forwardee and (2) access to forwardee's fields is preserved due to Release-Consume ordering on supported platforms. (The ordering between "new_obj = o->forwardee();" and logging or other usages is not changed.) >> >> Regarding the maintainability, the requirement is CAS(memory_order_release) as Release-Consume to be consistent with C++11. This requirement is necessary when a weaker platform like DEC Alpha is to be supported. On currently supported platforms, code change can be safe if the code meats this requirement (and the order of (1) access to the forwardee and (2) access to forwardee's fields is the natural way of coding). > > Relying on implicit consume has been been discussed and rejected, in > the earlier thread on this topic and I think elsewhere too. > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2016-October/021538.html From david.holmes at oracle.com Mon May 28 11:20:36 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 28 May 2018 21:20:36 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <06529fc3-2eba-101b-9aee-2757893cb8fb@oracle.com> Message-ID: <97f8cedf-4ebc-610f-0528-e1b91f35eece@oracle.com> I've added some missing JNI tests for the basic access checks. Given JNI ignores access it should go without saying that JNI access to nestmates will work fine, but it doesn't hurt to verify that. http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v4-incr/ Thanks, David On 24/05/2018 7:48 PM, David Holmes wrote: > Here are the further updates based on review comments and rebasing to > get the vmTestbase updates for which some closed test changes now have > to be applied to the open versions. > > Incremental hotspot webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3-incr/ > > Full hotspot webrev: > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3/ > > Change summary: > > test/hotspot/jtreg/ProblemList.txt > - Exclude vmTestbase/nsk/stress/except/except004.java under 8203046 > > test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/BasicTest.java > test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/PrivateMethodsTest.java > - updated to work with new invokeinterface rules and nestmate changes > - misc cleanups > > src/hotspot/share/runtime/reflection.?pp > - rename verify_field_access to verify_member_access (it's always been > mis-named and I nearly forgot to do this cleanup!) and rename > field_class to member_class > - add TRAPS to verify_member_access to allow use with CHECK macros > > src/hotspot/share/ci/ciField.cpp > src/hotspot/share/classfile/classFileParser.cpp > src/hotspot/share/interpreter/linkResolver.cpp > - updated to use THREAD/CHECK with verify_member_access > - for ciField rename thread to THREAD so it can be used with > HAS_PENDING_EXCEPTION > > src/hotspot/share/oops/instanceKlass.cpp > - use CHECK_false when calling nest_host() > - fix indent near nestmate code > > src/hotspot/share/oops/instanceKlass.hpp > - make has_nest_member private > > Thanks, > David > ----- > > On 23/05/2018 4:57 PM, David Holmes wrote: >> Here are the updates so far in response to all the review comments. >> >> Incremental webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2-incr/ >> >> >> Full webrev: >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2/ >> >> Change summary: >> >> test/runtime/Nestmates/reflectionAPI/* >> - moved to java/lang/reflect/Nestmates >> >> src/hotspot/cpu/arm/templateTable_arm.cpp >> - Fixed ARM invocation logic as provided by Boris. >> >> src/hotspot/share/interpreter/linkResolver.cpp >> - expanded comment regarding exceptions >> - Removed leftover debugging code >> >> src/hotspot/share/oops/instanceKlass.cpp >> - Removed FIXME comments >> - corrected incorrect comment >> - Fixed if/else formatting >> >> src/hotspot/share/oops/instanceKlass.hpp >> - removed unused debug method >> >> src/hotspot/share/oops/klassVtable.cpp >> - added comment by request of Karen >> >> src/hotspot/share/runtime/reflection.cpp >> - Removed FIXME comments >> - expanded comments in places >> - used CHECK_false >> >> Thanks, >> David >> >> On 15/05/2018 10:52 AM, David Holmes wrote: >>> This review is being spread across four groups: langtools, core-libs, >>> hotspot and serviceability. This is the specific review thread for >>> hotspot - webrev: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>> >>> See below for full details - including annotated full webrev guiding >>> the review. >>> >>> The intent is to have JEP-181 targeted and integrated by the end of >>> this month. >>> >>> Thanks, >>> David >>> ----- >>> >>> The nestmates project (JEP-181) introduces new classfile attributes >>> to identify classes and interfaces in the same nest, so that the VM >>> can perform access control based on those attributes and so allow >>> direct private access between nestmates without requiring javac to >>> generate synthetic accessor methods. These access control changes >>> also extend to core reflection and the MethodHandle.Lookup contexts. >>> >>> Direct private calls between nestmates requires a more general >>> calling context than is permitted by invokespecial, and so the JVMS >>> is updated to allow, and javac updated to use, invokevirtual and >>> invokeinterface for private class and interface method calls >>> respectively. These changed semantics also extend to MethodHandle >>> findXXX operations. >>> >>> At this time we are only concerned with static nest definitions, >>> which map to a top-level class/interface as the nest-host and all its >>> nested types as nest-members. >>> >>> Please see the JEP for further details. >>> >>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>> >>> All of the specification changes have been previously been worked out >>> by the Valhalla Project Expert Group, and the implementation reviewed >>> by the various contributors and discussed on the valhalla-dev mailing >>> list. >>> >>> Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, >>> Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, >>> Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan >>> >>> Master webrev of all changes: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>> >>> Annotated master webrev index: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>> >>> Performance: this is expected to be performance neutral in a general >>> sense. Benchmarking and performance runs are about to start. >>> >>> Testing Discussion: >>> ------------------ >>> >>> The testing for nestmates can be broken into four main groups: >>> >>> -? New tests specifically related to nestmates and currently in the >>> runtime/Nestmates directory >>> >>> - New tests to complement existing tests by adding in testcases not >>> previously expressible. >>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java tests >>> use of invokespecial for private interface methods and performing >>> receiver typechecks, so we add >>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests for >>> invokeinterface. >>> >>> -? New JVM TI tests to verify the spec changes related to nest >>> attributes. >>> >>> -? Existing tests significantly affected by the nestmates changes, >>> primarily: >>> ??? -? runtime/SelectionResolution >>> >>> ??? In most cases the nestmate changes makes certain invocations that >>> were illegal, legal (e.g. not requiring invokespecial to invoke >>> private interface methods; allowing access to private members via >>> reflection/Methodhandles that were previously not allowed). >>> >>> - Existing tests incidentally affected by the nestmate changes >>> >>> ?? This includes tests of things utilising class >>> redefinition/retransformation to alter nested types but which >>> unintentionally alter nest relationships (which is not permitted). >>> >>> There are still a number of tests problem-listed with issues filed >>> against them to have them adapted to work with nestmates. Some of >>> these are intended to be addressed in the short-term, while some >>> (such as the runtime/SelectionResolution test changes) may not >>> eventuate. >>> >>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>> >>> There is also further test work still to be completed (the JNI and >>> JDI invocation tests): >>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>> which will continue in parallel with the main RFR. >>> >>> Pre-integration Testing: >>> ??- General: >>> ???? - Mach5: hs/jdk tier1,2 >>> ???? - Mach5: hs-nightly (tiers 1 -3) >>> ??- Targetted >>> ??? - nashorn (for asm changes) >>> ??? - hotspot: runtime/* >>> ?????????????? serviceability/* >>> ?????????????? compiler/* >>> ?????????????? vmTestbase/* >>> ??? - jdk: java/lang/invoke/* >>> ?????????? java/lang/reflect/* >>> ?????????? java/lang/instrument/* >>> ?????????? java/lang/Class/* >>> ?????????? java/lang/management/* >>> ?? - langtools: tools/javac >>> ??????????????? tools/javap >>> From per.liden at oracle.com Mon May 28 12:16:52 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 28 May 2018 14:16:52 +0200 Subject: RFR: 8203885: ConcurrentLocksDump::dump_at_safepoint() should not allocate array in resource area Message-ID: <99967815-2c84-7e09-7934-1b5c22639425@oracle.com> ConcurrentLocksDump::dump_at_safepoint() creates a GrowableArray, which gets allocated in a resource area. This array is than passed down a call chain, where it can't control that another ResourceMark isn't created. In the leaf of this call chain, a closure (FindInstanceClosure) is executed, which appends to the array, which means it might need to be resized. This doesn't work if a new ResourceMark has been created, since the array resize will happen in a nested ResourceArea context. As a result, the append operation fails in GenericGrowableArray::check_nesting(). This has so far gone unnoticed because CollectedHeap::object_iterate() in existing collectors typically don't create new ResourceMarks. This is not true for ZGC (and potentially other concurrent collectors), which needs to walk thread stacks, which in turn requires a ResourceMark. The proposed fix is to make this array C Heap allocated. Bug: https://bugs.openjdk.java.net/browse/JDK-8203885 Webrev: http://cr.openjdk.java.net/~pliden/8203885/webrev.0 Testing: hs-tier{1,3} /Per From david.holmes at oracle.com Mon May 28 12:47:03 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 28 May 2018 22:47:03 +1000 Subject: RFR: 8203885: ConcurrentLocksDump::dump_at_safepoint() should not allocate array in resource area In-Reply-To: <99967815-2c84-7e09-7934-1b5c22639425@oracle.com> References: <99967815-2c84-7e09-7934-1b5c22639425@oracle.com> Message-ID: <0c772b64-8649-6a17-b581-03f314b4a967@oracle.com> Hi Per, On 28/05/2018 10:16 PM, Per Liden wrote: > ConcurrentLocksDump::dump_at_safepoint() creates a GrowableArray, which > gets allocated in a resource area. This array is than passed down a call > chain, where it can't control that another ResourceMark isn't created. > In the leaf of this call chain, a closure (FindInstanceClosure) is > executed, which appends to the array, which means it might need to be > resized. This doesn't work if a new ResourceMark has been created, since > the array resize will happen in a nested ResourceArea context. As a > result, the append operation fails in > GenericGrowableArray::check_nesting(). > > This has so far gone unnoticed because CollectedHeap::object_iterate() > in existing collectors typically don't create new ResourceMarks. This is > not true for ZGC (and potentially other concurrent collectors), which > needs to walk thread stacks, which in turn requires a ResourceMark. > > The proposed fix is to make this array C Heap allocated. That seems fine in itself but I'm not clear what the OOM behaviour of either the old or the new code is here?? Thanks, David > Bug: https://bugs.openjdk.java.net/browse/JDK-8203885 > Webrev: http://cr.openjdk.java.net/~pliden/8203885/webrev.0 > > Testing: hs-tier{1,3} > > /Per From per.liden at oracle.com Mon May 28 13:05:30 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 28 May 2018 15:05:30 +0200 Subject: RFR: 8203885: ConcurrentLocksDump::dump_at_safepoint() should not allocate array in resource area In-Reply-To: <0c772b64-8649-6a17-b581-03f314b4a967@oracle.com> References: <99967815-2c84-7e09-7934-1b5c22639425@oracle.com> <0c772b64-8649-6a17-b581-03f314b4a967@oracle.com> Message-ID: <6f1dc53d-4c40-dd9b-4bce-524742b78702@oracle.com> On 05/28/2018 02:47 PM, David Holmes wrote: > Hi Per, > > On 28/05/2018 10:16 PM, Per Liden wrote: >> ConcurrentLocksDump::dump_at_safepoint() creates a GrowableArray, >> which gets allocated in a resource area. This array is than passed >> down a call chain, where it can't control that another ResourceMark >> isn't created. In the leaf of this call chain, a closure >> (FindInstanceClosure) is executed, which appends to the array, which >> means it might need to be resized. This doesn't work if a new >> ResourceMark has been created, since the array resize will happen in a >> nested ResourceArea context. As a result, the append operation fails >> in GenericGrowableArray::check_nesting(). >> >> This has so far gone unnoticed because CollectedHeap::object_iterate() >> in existing collectors typically don't create new ResourceMarks. This >> is not true for ZGC (and potentially other concurrent collectors), >> which needs to walk thread stacks, which in turn requires a ResourceMark. >> >> The proposed fix is to make this array C Heap allocated. > > That seems fine in itself but I'm not clear what the OOM behaviour of > either the old or the new code is here?? Thanks for looking at this David. On OOM, both the old and the new code will eventually end up in vm_exit_out_of_memory(). /Per > > Thanks, > David > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203885 >> Webrev: http://cr.openjdk.java.net/~pliden/8203885/webrev.0 >> >> Testing: hs-tier{1,3} >> >> /Per From david.holmes at oracle.com Mon May 28 13:20:11 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 28 May 2018 23:20:11 +1000 Subject: RFR: 8203885: ConcurrentLocksDump::dump_at_safepoint() should not allocate array in resource area In-Reply-To: <6f1dc53d-4c40-dd9b-4bce-524742b78702@oracle.com> References: <99967815-2c84-7e09-7934-1b5c22639425@oracle.com> <0c772b64-8649-6a17-b581-03f314b4a967@oracle.com> <6f1dc53d-4c40-dd9b-4bce-524742b78702@oracle.com> Message-ID: <16f525e1-e18e-5741-3d25-fd0f728ea666@oracle.com> On 28/05/2018 11:05 PM, Per Liden wrote: > On 05/28/2018 02:47 PM, David Holmes wrote: >> Hi Per, >> >> On 28/05/2018 10:16 PM, Per Liden wrote: >>> ConcurrentLocksDump::dump_at_safepoint() creates a GrowableArray, >>> which gets allocated in a resource area. This array is than passed >>> down a call chain, where it can't control that another ResourceMark >>> isn't created. In the leaf of this call chain, a closure >>> (FindInstanceClosure) is executed, which appends to the array, which >>> means it might need to be resized. This doesn't work if a new >>> ResourceMark has been created, since the array resize will happen in >>> a nested ResourceArea context. As a result, the append operation >>> fails in GenericGrowableArray::check_nesting(). >>> >>> This has so far gone unnoticed because >>> CollectedHeap::object_iterate() in existing collectors typically >>> don't create new ResourceMarks. This is not true for ZGC (and >>> potentially other concurrent collectors), which needs to walk thread >>> stacks, which in turn requires a ResourceMark. >>> >>> The proposed fix is to make this array C Heap allocated. >> >> That seems fine in itself but I'm not clear what the OOM behaviour of >> either the old or the new code is here?? > > Thanks for looking at this David. > > On OOM, both the old and the new code will eventually end up in > vm_exit_out_of_memory(). Okay - that's not good. Probably non trivial to fix as the dump has to allow for failure - and can't directly propagate an exception. I'll file a bug. Thanks, David > /Per > >> >> Thanks, >> David >> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203885 >>> Webrev: http://cr.openjdk.java.net/~pliden/8203885/webrev.0 >>> >>> Testing: hs-tier{1,3} >>> >>> /Per From per.liden at oracle.com Mon May 28 15:38:05 2018 From: per.liden at oracle.com (Per Liden) Date: Mon, 28 May 2018 17:38:05 +0200 Subject: RFR: 8203885: ConcurrentLocksDump::dump_at_safepoint() should not allocate array in resource area In-Reply-To: <16f525e1-e18e-5741-3d25-fd0f728ea666@oracle.com> References: <99967815-2c84-7e09-7934-1b5c22639425@oracle.com> <0c772b64-8649-6a17-b581-03f314b4a967@oracle.com> <6f1dc53d-4c40-dd9b-4bce-524742b78702@oracle.com> <16f525e1-e18e-5741-3d25-fd0f728ea666@oracle.com> Message-ID: Hi David, On 05/28/2018 03:20 PM, David Holmes wrote: > On 28/05/2018 11:05 PM, Per Liden wrote: >> On 05/28/2018 02:47 PM, David Holmes wrote: >>> Hi Per, >>> >>> On 28/05/2018 10:16 PM, Per Liden wrote: >>>> ConcurrentLocksDump::dump_at_safepoint() creates a GrowableArray, >>>> which gets allocated in a resource area. This array is than passed >>>> down a call chain, where it can't control that another ResourceMark >>>> isn't created. In the leaf of this call chain, a closure >>>> (FindInstanceClosure) is executed, which appends to the array, which >>>> means it might need to be resized. This doesn't work if a new >>>> ResourceMark has been created, since the array resize will happen in >>>> a nested ResourceArea context. As a result, the append operation >>>> fails in GenericGrowableArray::check_nesting(). >>>> >>>> This has so far gone unnoticed because >>>> CollectedHeap::object_iterate() in existing collectors typically >>>> don't create new ResourceMarks. This is not true for ZGC (and >>>> potentially other concurrent collectors), which needs to walk thread >>>> stacks, which in turn requires a ResourceMark. >>>> >>>> The proposed fix is to make this array C Heap allocated. >>> >>> That seems fine in itself but I'm not clear what the OOM behaviour of >>> either the old or the new code is here?? >> >> Thanks for looking at this David. >> >> On OOM, both the old and the new code will eventually end up in >> vm_exit_out_of_memory(). > > Okay - that's not good. Probably non trivial to fix as the dump has to > allow for failure - and can't directly propagate an exception. I'll file > a bug. Check! Since my patch doesn't make this better or worse, I assume you're ok with me going ahead with my change? /Per > > Thanks, > David > >> /Per >> >>> >>> Thanks, >>> David >>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203885 >>>> Webrev: http://cr.openjdk.java.net/~pliden/8203885/webrev.0 >>>> >>>> Testing: hs-tier{1,3} >>>> >>>> /Per From kim.barrett at oracle.com Mon May 28 21:03:35 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 28 May 2018 17:03:35 -0400 Subject: RFR (M) 8202813: Move vm_weak processing from SystemDictionary to WeakProcessor In-Reply-To: <4e7c8f9f-e1fa-0a29-ac22-5215b55dee69@oracle.com> References: <4e7c8f9f-e1fa-0a29-ac22-5215b55dee69@oracle.com> Message-ID: <98F4F244-C5C7-4F2B-AEB3-40E8E44F3708@oracle.com> > On May 25, 2018, at 11:32 AM, coleen.phillimore at oracle.com wrote: > > Summary: SystemDictionary has all strong roots. The weak oop_storage is processed by the WeakProcessor so it can be scanned and cleared concurrently and/or by parallel threads. > > Please see bug for explanation. > > open webrev at http://cr.openjdk.java.net/~coleenp/8202813.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8202813 > > There might be more GC code that can be cleaned out since G1RootClosures->weak_oops() seems unused now, but I'd like to defer that to a smaller cleanup. Also ClassLoaderData is still processed inconsistently by the GCs, which would be nice to clean up. > > Tested with mach5 hs-tier1-5 with no failures. Per's gc-test-suite, dev-submit for performance testing, and runThese with all collectors, with and without -XX:-ClassUnloading. Also tested with class unloading gc tests (which are also in hs-tier number tests). > > Thanks, > Coleen Looks good. From zgu at redhat.com Mon May 28 21:11:06 2018 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 28 May 2018 17:11:06 -0400 Subject: RFR(M) 8203641: Refactor String Deduplication into shared Message-ID: Hi, Please review this refactoring of G1 string deduplication into shared directory, so that other GCs (such as Shenandoah) can advantage of existing infrastructure and plugin their own implementation. This refactoring preserves G1's String Deduplication infrastructure (please see the comments in stringDedup.hpp for details), so that there is no change to G1 outside of string deduplication code. Following changes are made to support different GCs: 1. Allows plugin new dedup queue implementation. While it keeps G1's dedup queue static interface, queue itself now is a pure virtual class. Different GC can provide different implementation to fit its own enqueuing mechanism. For example, G1 enqueues deduplication candidates during STW evacuate/mark pause, while Shenandoah implementation does it during concurrent mark. 2. Abstracted out generation related statistics out of StringDedupStat base class, cause not all GCs are generational. G1StringDedupStat simply extends the base to add generational statistics. 3. Moved table and queue's parallel processing logic from closure (StringDedupUnlinkOrOopsDoClosure) to corresponding table and queue. This gives flexibility to construct closure to share among the workers (as G1 does), as well as private closure for each worker (as Shenandoah does). Bug: https://bugs.openjdk.java.net/browse/JDK-8203641 Webrev: http://cr.openjdk.java.net/~zgu/8203641/webrev.00/index.html Test: Submit test came back clean. Thanks, -Zhengyu From david.holmes at oracle.com Mon May 28 21:28:51 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 29 May 2018 07:28:51 +1000 Subject: RFR: 8203885: ConcurrentLocksDump::dump_at_safepoint() should not allocate array in resource area In-Reply-To: References: <99967815-2c84-7e09-7934-1b5c22639425@oracle.com> <0c772b64-8649-6a17-b581-03f314b4a967@oracle.com> <6f1dc53d-4c40-dd9b-4bce-524742b78702@oracle.com> <16f525e1-e18e-5741-3d25-fd0f728ea666@oracle.com> Message-ID: <2f430628-cac6-4180-87a7-a9f226385b6a@oracle.com> On 29/05/2018 1:38 AM, Per Liden wrote: > Hi David, > > On 05/28/2018 03:20 PM, David Holmes wrote: >> On 28/05/2018 11:05 PM, Per Liden wrote: >>> On 05/28/2018 02:47 PM, David Holmes wrote: >>>> Hi Per, >>>> >>>> On 28/05/2018 10:16 PM, Per Liden wrote: >>>>> ConcurrentLocksDump::dump_at_safepoint() creates a GrowableArray, >>>>> which gets allocated in a resource area. This array is than passed >>>>> down a call chain, where it can't control that another ResourceMark >>>>> isn't created. In the leaf of this call chain, a closure >>>>> (FindInstanceClosure) is executed, which appends to the array, >>>>> which means it might need to be resized. This doesn't work if a new >>>>> ResourceMark has been created, since the array resize will happen >>>>> in a nested ResourceArea context. As a result, the append operation >>>>> fails in GenericGrowableArray::check_nesting(). >>>>> >>>>> This has so far gone unnoticed because >>>>> CollectedHeap::object_iterate() in existing collectors typically >>>>> don't create new ResourceMarks. This is not true for ZGC (and >>>>> potentially other concurrent collectors), which needs to walk >>>>> thread stacks, which in turn requires a ResourceMark. >>>>> >>>>> The proposed fix is to make this array C Heap allocated. >>>> >>>> That seems fine in itself but I'm not clear what the OOM behaviour >>>> of either the old or the new code is here?? >>> >>> Thanks for looking at this David. >>> >>> On OOM, both the old and the new code will eventually end up in >>> vm_exit_out_of_memory(). >> >> Okay - that's not good. Probably non trivial to fix as the dump has to >> allow for failure - and can't directly propagate an exception. I'll >> file a bug. > > Check! Since my patch doesn't make this better or worse, I assume you're > ok with me going ahead with my change? Absolutely. David > /Per > >> >> Thanks, >> David >> >>> /Per >>> >>>> >>>> Thanks, >>>> David >>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203885 >>>>> Webrev: http://cr.openjdk.java.net/~pliden/8203885/webrev.0 >>>>> >>>>> Testing: hs-tier{1,3} >>>>> >>>>> /Per From john.r.rose at oracle.com Mon May 28 22:09:44 2018 From: john.r.rose at oracle.com (John Rose) Date: Mon, 28 May 2018 15:09:44 -0700 Subject: JEP: JWarmup precompile java hot methods at application startup In-Reply-To: <007201d3f620$fce0be90$f6a23bb0$@oracle.com> References: <20180525132251.552041504@eggemoggin.niobe.net> <007201d3f620$fce0be90$f6a23bb0$@oracle.com> Message-ID: <8A5120DA-E837-4FA1-AC3A-E0C5973AFA43@oracle.com> Mark is saying you should propose it on hotspot-dev at openjdk.java.net. That way all interested parties can comment. I will be listening, of course. ? John On May 27, 2018, at 6:12 PM, Frank Yuan wrote: > >> >> 2018/5/25 10:00:17 -0700, yumin.qi at gmail.com: >>> Hi, Mark >>> >>> Proposed a draft of new JEP: >>> https://bugs.openjdk.java.net/browse/JDK-8203832 >>> >>> We (Alibaba Group Inc) have an implementation of warmup performance >>> solution and want to contribute to OpenJDK, wish it can help java >>> developers to resolve the same problems we have encountered. >>> >>> Please take a look and wait for a discussion/evaluation on it. >> >> I suggest that you ask hotspot-dev for feedback. >> > > John Rose may be the right person :) > > Frank > >> - Mark From kim.barrett at oracle.com Mon May 28 23:53:52 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 28 May 2018 19:53:52 -0400 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: References: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> <7e7fc484287e4da4926176e0b5ae1b64@sap.com> <339D50B6-09D5-4342-A687-918A9A096B39@oracle.com> <6D2268EC-C1E7-4C90-BCD3-90D02D21FA08@oracle.com> Message-ID: > On May 28, 2018, at 4:12 AM, Michihiro Horie wrote: > > Hi Erik, > > Thank you very much for your review. > > I understood that implicit consume should not be used in the shared code. Also, I believe performance degradation would be negligible even if we use acquire. > > New webrev uses memory_order_acq_rel: http://cr.openjdk.java.net/~mhorie/8154736/webrev.10 This is missing the acquire barrier on the else branch for the initial test, so fails to meet the previously described minimal requirements for even possibly being sufficient. Any analysis of weakening the CAS barriers must consider that test and successor code. In the analysis, it?s not just the lexically nearby debugging / logging code that needs to be considered; the forwardee is being returned to caller(s) that will presumably do something with that object. Since the whole point of this discussion is performance, any proposed change should come with performance information. From yumin.qi at gmail.com Tue May 29 04:09:44 2018 From: yumin.qi at gmail.com (yumin qi) Date: Mon, 28 May 2018 21:09:44 -0700 Subject: JEP: https://bugs.openjdk.java.net/browse/JDK-8203832 Message-ID: Hi? Experts This is a newly filed JEP (JWarmup) for working on resolving java performance issue caused by both application load peaking up and JIT threads compiling java hot methods happen at same time. https://bugs.openjdk.java.net/browse/JDK-8203832 For a large java application, the load comes in short period of time, like the 'Single Day' sale on Alibaba's e-commerce application, this massive load comes in and makes many java methods ready for JIT compilation to convert them into native methods. The compiler threads will kick in to do the complication work and take system resource from mutator java threads which are busy on processing requests thus lead to peak time performance degradation. The JWarmup technique was proposed to avoid such issue by precompiling the hot methods at application startup and it has been successfully applied to Alibaba's e-commerce applications. We would like to contribute it to OpenJDK and wish it can help java developers overcome the same issue. Please review and give your feedback. Thanks Yumin (Alibaba Group Inc) From yumin.qi at gmail.com Tue May 29 04:10:52 2018 From: yumin.qi at gmail.com (yumin qi) Date: Mon, 28 May 2018 21:10:52 -0700 Subject: JEP: JWarmup precompile java hot methods at application startup In-Reply-To: <8A5120DA-E837-4FA1-AC3A-E0C5973AFA43@oracle.com> References: <20180525132251.552041504@eggemoggin.niobe.net> <007201d3f620$fce0be90$f6a23bb0$@oracle.com> <8A5120DA-E837-4FA1-AC3A-E0C5973AFA43@oracle.com> Message-ID: Thanks John, Frank I have post it on hotspot-dev at openjdk.java.net. Yumin On Mon, May 28, 2018 at 3:09 PM, John Rose wrote: > Mark is saying you should propose it on hotspot-dev at openjdk.java.net. > That way all interested parties can comment. I will be listening, of > course. > ? John > > On May 27, 2018, at 6:12 PM, Frank Yuan wrote: > > > > 2018/5/25 10:00:17 -0700, yumin.qi at gmail.com: > > Hi, Mark > > Proposed a draft of new JEP: > https://bugs.openjdk.java.net/browse/JDK-8203832 > > We (Alibaba Group Inc) have an implementation of warmup performance > solution and want to contribute to OpenJDK, wish it can help java > developers to resolve the same problems we have encountered. > > Please take a look and wait for a discussion/evaluation on it. > > > I suggest that you ask hotspot-dev for feedback. > > > John Rose may be the right person :) > > Frank > > - Mark > > > From thomas.stuefe at gmail.com Tue May 29 05:59:25 2018 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 29 May 2018 07:59:25 +0200 Subject: RFR: 8203885: ConcurrentLocksDump::dump_at_safepoint() should not allocate array in resource area In-Reply-To: <99967815-2c84-7e09-7934-1b5c22639425@oracle.com> References: <99967815-2c84-7e09-7934-1b5c22639425@oracle.com> Message-ID: Hi Per, this looks good. Best Regards, Thomas On Mon, May 28, 2018 at 2:16 PM, Per Liden wrote: > ConcurrentLocksDump::dump_at_safepoint() creates a GrowableArray, which gets > allocated in a resource area. This array is than passed down a call chain, > where it can't control that another ResourceMark isn't created. In the leaf > of this call chain, a closure (FindInstanceClosure) is executed, which > appends to the array, which means it might need to be resized. This doesn't > work if a new ResourceMark has been created, since the array resize will > happen in a nested ResourceArea context. As a result, the append operation > fails in GenericGrowableArray::check_nesting(). > > This has so far gone unnoticed because CollectedHeap::object_iterate() in > existing collectors typically don't create new ResourceMarks. This is not > true for ZGC (and potentially other concurrent collectors), which needs to > walk thread stacks, which in turn requires a ResourceMark. > > The proposed fix is to make this array C Heap allocated. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203885 > Webrev: http://cr.openjdk.java.net/~pliden/8203885/webrev.0 > > Testing: hs-tier{1,3} > > /Per From kim.barrett at oracle.com Tue May 29 06:09:20 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 29 May 2018 02:09:20 -0400 Subject: RFR: 8203885: ConcurrentLocksDump::dump_at_safepoint() should not allocate array in resource area In-Reply-To: <99967815-2c84-7e09-7934-1b5c22639425@oracle.com> References: <99967815-2c84-7e09-7934-1b5c22639425@oracle.com> Message-ID: <08E2B22F-DDE0-4F20-9F87-B0B803721245@oracle.com> > On May 28, 2018, at 8:16 AM, Per Liden wrote: > > ConcurrentLocksDump::dump_at_safepoint() creates a GrowableArray, which gets allocated in a resource area. This array is than passed down a call chain, where it can't control that another ResourceMark isn't created. In the leaf of this call chain, a closure (FindInstanceClosure) is executed, which appends to the array, which means it might need to be resized. This doesn't work if a new ResourceMark has been created, since the array resize will happen in a nested ResourceArea context. As a result, the append operation fails in GenericGrowableArray::check_nesting(). > > This has so far gone unnoticed because CollectedHeap::object_iterate() in existing collectors typically don't create new ResourceMarks. This is not true for ZGC (and potentially other concurrent collectors), which needs to walk thread stacks, which in turn requires a ResourceMark. > > The proposed fix is to make this array C Heap allocated. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8203885 > Webrev: http://cr.openjdk.java.net/~pliden/8203885/webrev.0 > > Testing: hs-tier{1,3} > > /Per Looks good. I was going to ask why the GrowableArray needs to be heap allocated? Why not just GrowableArray aos_objects(INITIAL_ARRAY_SIZE, true); ... s/aos_objects/&aos_objects/ ... // delete aos_objects no longer needed After some digging, probably because of this, from growableArray.hpp: 119 assert(!on_C_heap() || allocated_on_C_heap(), "growable array must be on C heap if elements are"); Conflating the allocation of the GrowableArray object with the allocation of the underlying array. That seems wrong, but not a problem to be solved as part of this change. From per.liden at oracle.com Tue May 29 06:12:39 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 29 May 2018 08:12:39 +0200 Subject: RFR: 8203885: ConcurrentLocksDump::dump_at_safepoint() should not allocate array in resource area In-Reply-To: References: <99967815-2c84-7e09-7934-1b5c22639425@oracle.com> Message-ID: <56e01388-8932-3d2f-211d-470a5dbcbb6d@oracle.com> Thanks for reviewing, Thomas! /Per On 05/29/2018 07:59 AM, Thomas St?fe wrote: > Hi Per, > > this looks good. > > Best Regards, Thomas > > On Mon, May 28, 2018 at 2:16 PM, Per Liden wrote: >> ConcurrentLocksDump::dump_at_safepoint() creates a GrowableArray, which gets >> allocated in a resource area. This array is than passed down a call chain, >> where it can't control that another ResourceMark isn't created. In the leaf >> of this call chain, a closure (FindInstanceClosure) is executed, which >> appends to the array, which means it might need to be resized. This doesn't >> work if a new ResourceMark has been created, since the array resize will >> happen in a nested ResourceArea context. As a result, the append operation >> fails in GenericGrowableArray::check_nesting(). >> >> This has so far gone unnoticed because CollectedHeap::object_iterate() in >> existing collectors typically don't create new ResourceMarks. This is not >> true for ZGC (and potentially other concurrent collectors), which needs to >> walk thread stacks, which in turn requires a ResourceMark. >> >> The proposed fix is to make this array C Heap allocated. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203885 >> Webrev: http://cr.openjdk.java.net/~pliden/8203885/webrev.0 >> >> Testing: hs-tier{1,3} >> >> /Per From per.liden at oracle.com Tue May 29 06:20:55 2018 From: per.liden at oracle.com (Per Liden) Date: Tue, 29 May 2018 08:20:55 +0200 Subject: RFR: 8203885: ConcurrentLocksDump::dump_at_safepoint() should not allocate array in resource area In-Reply-To: <08E2B22F-DDE0-4F20-9F87-B0B803721245@oracle.com> References: <99967815-2c84-7e09-7934-1b5c22639425@oracle.com> <08E2B22F-DDE0-4F20-9F87-B0B803721245@oracle.com> Message-ID: <1d42feb8-6e91-9b4b-ba6d-47dd82739cda@oracle.com> Hi Kim, On 05/29/2018 08:09 AM, Kim Barrett wrote: >> On May 28, 2018, at 8:16 AM, Per Liden wrote: >> >> ConcurrentLocksDump::dump_at_safepoint() creates a GrowableArray, which gets allocated in a resource area. This array is than passed down a call chain, where it can't control that another ResourceMark isn't created. In the leaf of this call chain, a closure (FindInstanceClosure) is executed, which appends to the array, which means it might need to be resized. This doesn't work if a new ResourceMark has been created, since the array resize will happen in a nested ResourceArea context. As a result, the append operation fails in GenericGrowableArray::check_nesting(). >> >> This has so far gone unnoticed because CollectedHeap::object_iterate() in existing collectors typically don't create new ResourceMarks. This is not true for ZGC (and potentially other concurrent collectors), which needs to walk thread stacks, which in turn requires a ResourceMark. >> >> The proposed fix is to make this array C Heap allocated. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8203885 >> Webrev: http://cr.openjdk.java.net/~pliden/8203885/webrev.0 >> >> Testing: hs-tier{1,3} >> >> /Per > > Looks good. Thanks for reviewing! > > I was going to ask why the GrowableArray needs to be heap allocated? > Why not just > > GrowableArray aos_objects(INITIAL_ARRAY_SIZE, true); > ... s/aos_objects/&aos_objects/ ... > // delete aos_objects no longer needed > > After some digging, probably because of this, from growableArray.hpp: > > 119 assert(!on_C_heap() || allocated_on_C_heap(), "growable array must be on C heap if elements are"); I had the same reaction and initially made it stack allocated, but noticed that GrowableArray doesn't allow that :( > > Conflating the allocation of the GrowableArray object with the > allocation of the underlying array. That seems wrong, but not a > problem to be solved as part of this change. > I agree. I suspect someone wanted to protect against potential problems with having different life cycles of the GrowableArray and the backing array. But this seems overly strict. /Per From david.holmes at oracle.com Tue May 29 07:06:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 29 May 2018 17:06:15 +1000 Subject: RFR (XS) 8189766: whitebox failure with -Xcheck:jni Message-ID: bug: https://bugs.openjdk.java.net/browse/JDK-8189766 webrev: http://cr.openjdk.java.net/~dholmes/8189766/webrev/ A WB_ENTRY already handles the pending JNI exception check by explicitly clearing it via a helper: #define WB_ENTRY(result_type, header) JNI_ENTRY(result_type, header) \ ClearPendingJniExcCheck _clearCheck(env); #define WB_END JNI_END But the whitebox function then does e.g: WB_ENTRY(jobject, WB_GetBooleanVMFlag(JNIEnv* env, jobject o, jstring name)) bool result; if (GetVMFlag (thread, env, name, &result, &JVMFlag::boolAt)) { ThreadToNativeFromVM ttnfv(thread); // can't be in VM when we call JNI return booleanBox(thread, env, result); } return NULL; WB_END where booleanBox is defined eventually as: template static jobject box(JavaThread* thread, JNIEnv* env, Symbol* name, Symbol* sig, T value) { ResourceMark rm(thread); jclass clazz = env->FindClass(name->as_C_string()); CHECK_JNI_EXCEPTION_(env, NULL); jmethodID methodID = env->GetStaticMethodID(clazz, vmSymbols::valueOf_name()->as_C_string(), sig->as_C_string()); CHECK_JNI_EXCEPTION_(env, NULL); jobject result = env->CallStaticObjectMethod(clazz, methodID, value); CHECK_JNI_EXCEPTION_(env, NULL); return result; } so we call JNI methods that will set the pending_jni_exception_check flag, but CHECK_JNI_EXCEPTION is defined as: #define CHECK_JNI_EXCEPTION_(env, value) \ do { \ JavaThread* THREAD = JavaThread::thread_from_jni_environment(env); \ if (HAS_PENDING_EXCEPTION) { \ return(value); \ } \ } while (0) which means it is not clearing the pending_jni_exception_check flag even though it is checking for exceptions! So we explicitly clear the flag. Trying to use the JNI ExceptionOccurred function fails as not all the methods that call CHECK_JNI_EXCEPTION are _thread_in_native. Testing (in progress): hotspot tier1 and tier2 with -Xcheck:jni regular CI testing Thanks, David From swatibits14 at gmail.com Tue May 29 09:12:22 2018 From: swatibits14 at gmail.com (Swati Sharma) Date: Tue, 29 May 2018 14:42:22 +0530 Subject: UseNUMA membind Issue in openJDK In-Reply-To: <187f278f-5069-3afa-c61f-f49a1fc0d790@linux.vnet.ibm.com> References: <9a0310b7-2880-db69-cfbc-7abba844ecbf@oracle.com> <187f278f-5069-3afa-c61f-f49a1fc0d790@linux.vnet.ibm.com> Message-ID: Hi Gustavo, Thanks for replying :) I have incorporated some changes suggested by you. The use of struct bitmask's maskp for checking 64 bit in single iteration is more optimized compared to numa_bitmask_isbitset() as by using this we need to check each bit for 1024 times(SUSE case) and 64 times(Ubuntu Case). If its fine to iterate at initialization time then I can change. For the answer to your question: If it picks up node 16, not so bad, but what if it picks up node 0 or 1? It can be checked based on numa_distance instead of picking up the lgrps randomly. Thanks, Swati On Fri, May 25, 2018 at 4:54 AM, Gustavo Romero wrote: > Hi Swati, > > > Thanks for CC:ing me. Sorry for the delay replying it, I had to reserve a > few > specific machines before trying your patch :-) > > I think that UseNUMA's original task was to figure out the best binding > setup for the JVM automatically but I understand that it also has to be > aware > that sometimes, for some (new) particular reasons, its binding task is > "modulated" by other external agents. Thanks for proposing a fix. > > I have just a question/concern on the proposal: how the JVM should behave > if > CPUs are not bound in accordance to the bound memory nodes? For instance, > what > happens if no '--cpunodebind' is passed and '--membind=0,1,16' is passed at > the same time on this numa topology: > > brianh at p215n12:~$ numactl -H > available: 4 nodes (0-1,16-17) > node 0 cpus: 0 1 2 3 8 9 10 11 16 17 18 19 24 25 26 27 32 33 34 35 > node 0 size: 65342 MB > node 0 free: 56902 MB > node 1 cpus: 40 41 42 43 48 49 50 51 56 57 58 59 64 65 66 67 72 73 74 75 > node 1 size: 65447 MB > node 1 free: 58322 MB > node 16 cpus: 80 81 82 83 88 89 90 91 96 97 98 99 104 105 106 107 112 113 > 114 115 > node 16 size: 65448 MB > node 16 free: 63096 MB > node 17 cpus: 120 121 122 123 128 129 130 131 136 137 138 139 144 145 146 > 147 152 153 154 155 > node 17 size: 65175 MB > node 17 free: 61522 MB > node distances: > node 0 1 16 17 > 0: 10 20 40 40 > 1: 20 10 40 40 > 16: 40 40 10 20 > 17: 40 40 20 10 > > > In that case JVM will spawn threads that will run on all CPUs, including > those > CPUs in numa node 17. Then once in > src/hotspot/share/gc/parallel/mutableNUMASpace.cpp, in cas_allocate(): > > 834 // This version is lock-free. > 835 HeapWord* MutableNUMASpace::cas_allocate(size_t size) { > 836 Thread* thr = Thread::current(); > 837 int lgrp_id = thr->lgrp_id(); > 838 if (lgrp_id == -1 || !os::numa_has_group_homing()) { > 839 lgrp_id = os::numa_get_group_id(); > 840 thr->set_lgrp_id(lgrp_id); > 841 } > > a newly created thread will try to be mapped to a numa node given your CPU > ID. > So if that CPU is in numa node 17 it will then not find it in: > > 843 int i = lgrp_spaces()->find(&lgrp_id, LGRPSpace::equals); > > and will fallback to a random map, picking up a random numa node among > nodes > 0, 1, and 16: > > 846 if (i == -1) { > 847 i = os::random() % lgrp_spaces()->length(); > 848 } > > If it picks up node 16, not so bad, but what if it picks up node 0 or 1? > > I see that if one binds mem but leaves CPU unbound one has to know exactly > what > she/he is doing, because it can be likely suboptimal. On the other hand, > letting > the node being picked up randomly when there are memory nodes bound but no > CPUs > seems even more suboptimal in some scenarios. Thus, should the JVM deal > with it? > > @Zhengyu, do you have any opinion on that? > > Please find a few nits / comments inline. > > Note that I'm not a (R)eviewer so you still need two official reviews. > > > Best regards, > Gustavo > > On 05/21/2018 01:44 PM, Swati Sharma wrote: > >> ======================PATCH============================== >> diff --git a/src/hotspot/os/linux/os_linux.cpp >> b/src/hotspot/os/linux/os_linux.cpp >> --- a/src/hotspot/os/linux/os_linux.cpp >> +++ b/src/hotspot/os/linux/os_linux.cpp >> @@ -2832,14 +2832,42 @@ >> // Map all node ids in which is possible to allocate memory. Also >> nodes are >> // not always consecutively available, i.e. available from 0 to the >> highest >> // node number. >> + // If the nodes have been bound explicitly using numactl membind, then >> + // allocate memory from those nodes only. >> > > I think ok to place that comment on the same existing line, like: > > - // node number. > + // node number. If the nodes have been bound explicitly using numactl > membind, > + // then allocate memory from these nodes only. > > > for (size_t node = 0; node <= highest_node_number; node++) { >> - if (Linux::isnode_in_configured_nodes(node)) { >> + if (Linux::isnode_in_bounded_nodes(node)) { >> > ---------------------------------^ s/bounded/bound/ > > > ids[i++] = node; >> } >> } >> return i; >> } >> +extern "C" struct bitmask { >> + unsigned long size; /* number of bits in the map */ >> + unsigned long *maskp; >> +}; >> > > I think it's possible to move the function below to os_linux.hpp with its > friends and cope with the forward declaration of 'struct bitmask*` by > using the > functions from numa API, notably numa_bitmask_nbytes() and > numa_bitmask_isbitset() only, avoiding the member dereferecing issue and > the > need to add the above struct explicitly. > > > +// Check if single memory node bound. >> +// Returns true if single memory node bound. >> > > I suggest a minuscule improvement, something like: > > +// Check if bound to only one numa node. > +// Returns true if bound to a single numa node, otherwise returns false. > > > +bool os::Linux::issingle_node_bound() { >> > > What about s/issingle_node_bound/isbound_to_single_node/ ? > > > + struct bitmask* bmp = _numa_get_membind != NULL ? _numa_get_membind() : >> NULL; >> + if(!(bmp != NULL && bmp->maskp != NULL)) return false; >> > -----^ > Are you sure this checking is necessary? I think if numa_get_membind > succeed > bmp->maskp is always != NULL. > > Indentation here is odd. No space before 'if' and return on the same line. > > I would try to avoid lines over 80 chars. > > > + int issingle = 0; >> + // System can have more than 64 nodes so check in all the elements of >> + // unsigned long array >> + for (unsigned long i = 0; i < (bmp->size / (8 * sizeof(unsigned >> long))); i++) { >> + if (bmp->maskp[i] == 0) { >> + continue; >> + } else if ((bmp->maskp[i] & (bmp->maskp[i] - 1)) == 0) { >> + issingle++; >> + } else { >> + return false; >> + } >> + } >> + if (issingle == 1) >> + return true; >> + return false; >> +} >> + >> > > As I mentioned, I think it could be moved to os_linux.hpp instead. Also, it > could be something like: > > +bool os::Linux::isbound_to_single_node(void) { > + struct bitmask* bmp; > + unsigned long mask; // a mask element in the mask array > + unsigned long max_num_masks; > + int single_node = 0; > + > + if (_numa_get_membind != NULL) { > + bmp = _numa_get_membind(); > + } else { > + return false; > + } > + > + max_num_masks = bmp->size / (8 * sizeof(unsigned long)); > + > + for (mask = 0; mask < max_num_masks; mask++) { > + if (bmp->maskp[mask] != 0) { // at least one numa node in the mask > + if (bmp->maskp[mask] & (bmp->maskp[mask] - 1) == 0) { > + single_node++; // a single numa node in the mask > + } else { > + return false; > + } > + } > + } > + > + if (single_node == 1) { > + return true; // only a single mask with a single numa node > + } else { > + return false; > + } > +} > > > bool os::get_page_info(char *start, page_info* info) { >> return false; >> } >> @@ -2930,6 +2958,10 @@ >> libnuma_dlsym(handle, >> "numa_bitmask_isbitset"))); >> set_numa_distance(CAST_TO_FN_PTR(numa_distance_func_t, >> libnuma_dlsym(handle, >> "numa_distance"))); >> + set_numa_set_membind(CAST_TO_FN_PTR(numa_set_membind_func_t, >> + libnuma_dlsym(handle, >> "numa_set_membind"))); >> + set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, >> + libnuma_v2_dlsym(handle, >> "numa_get_membind"))); >> if (numa_available() != -1) { >> set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, >> "numa_all_nodes")); >> @@ -3054,6 +3086,8 @@ >> os::Linux::numa_set_bind_policy_func_t os::Linux::_numa_set_bind_poli >> cy; >> os::Linux::numa_bitmask_isbitset_func_t os::Linux::_numa_bitmask_isbit >> set; >> os::Linux::numa_distance_func_t os::Linux::_numa_distance; >> +os::Linux::numa_set_membind_func_t os::Linux::_numa_set_membind; >> +os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; >> unsigned long* os::Linux::_numa_all_nodes; >> struct bitmask* os::Linux::_numa_all_nodes_ptr; >> struct bitmask* os::Linux::_numa_nodes_ptr; >> @@ -4962,8 +4996,9 @@ >> if (!Linux::libnuma_init()) { >> UseNUMA = false; >> } else { >> - if ((Linux::numa_max_node() < 1)) { >> - // There's only one node(they start from 0), disable NUMA. >> + if ((Linux::numa_max_node() < 1) || Linux::issingle_node_bound()) { >> + // If there's only one node(they start from 0) or if the process >> + // is bound explicitly to a single node using membind, disable >> NUMA. >> UseNUMA = false; >> } >> } >> diff --git a/src/hotspot/os/linux/os_linux.hpp >> b/src/hotspot/os/linux/os_linux.hpp >> --- a/src/hotspot/os/linux/os_linux.hpp >> +++ b/src/hotspot/os/linux/os_linux.hpp >> @@ -228,6 +228,8 @@ >> typedef int (*numa_tonode_memory_func_t)(void *start, size_t size, >> int node); >> typedef void (*numa_interleave_memory_func_t)(void *start, size_t >> size, unsigned long *nodemask); >> typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t >> size, struct bitmask* mask); >> + typedef void (*numa_set_membind_func_t)(struct bitmask *mask); >> + typedef struct bitmask* (*numa_get_membind_func_t)(void); >> typedef void (*numa_set_bind_policy_func_t)(int policy); >> typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, >> unsigned int n); >> @@ -244,6 +246,8 @@ >> static numa_set_bind_policy_func_t _numa_set_bind_policy; >> static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; >> static numa_distance_func_t _numa_distance; >> + static numa_set_membind_func_t _numa_set_membind; >> + static numa_get_membind_func_t _numa_get_membind; >> static unsigned long* _numa_all_nodes; >> static struct bitmask* _numa_all_nodes_ptr; >> static struct bitmask* _numa_nodes_ptr; >> @@ -259,6 +263,8 @@ >> static void set_numa_set_bind_policy(numa_set_bind_policy_func_t >> func) { _numa_set_bind_policy = func; } >> static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t >> func) { _numa_bitmask_isbitset = func; } >> static void set_numa_distance(numa_distance_func_t func) { >> _numa_distance = func; } >> + static void set_numa_set_membind(numa_set_membind_func_t func) { >> _numa_set_membind = func; } >> + static void set_numa_get_membind(numa_get_membind_func_t func) { >> _numa_get_membind = func; } >> static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes >> = ptr; } >> static void set_numa_all_nodes_ptr(struct bitmask **ptr) { >> _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } >> static void set_numa_nodes_ptr(struct bitmask **ptr) { >> _numa_nodes_ptr = (ptr == NULL ? NULL : *ptr); } >> @@ -320,6 +326,15 @@ >> } else >> return 0; >> } >> + // Check if node in bounded nodes >> > > + // Check if node is in bound node set. Maybe? > > > + static bool isnode_in_bounded_nodes(int node) { >> + struct bitmask* bmp = _numa_get_membind != NULL ? >> _numa_get_membind() : NULL; >> + if (bmp != NULL && _numa_bitmask_isbitset != NULL && >> _numa_bitmask_isbitset(bmp, node)) { >> + return true; >> + } else >> + return false; >> + } >> + static bool issingle_node_bound(); >> > > Looks like it can be re-written like: > > + static bool isnode_in_bound_nodes(int node) { > + if (_numa_get_membind != NULL && _numa_bitmask_isbitset != NULL) { > + return _numa_bitmask_isbitset(_numa_get_membind(), node); > + } else { > + return false; > + } > + } > > ? > > > }; >> #endif // OS_LINUX_VM_OS_LINUX_HPP >> > > From stefan.johansson at oracle.com Tue May 29 10:27:49 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 29 May 2018 12:27:49 +0200 Subject: RFR (M) 8202813: Move vm_weak processing from SystemDictionary to WeakProcessor In-Reply-To: <4e7c8f9f-e1fa-0a29-ac22-5215b55dee69@oracle.com> References: <4e7c8f9f-e1fa-0a29-ac22-5215b55dee69@oracle.com> Message-ID: <727eb265-3a7d-6e91-6677-b630de24274a@oracle.com> Hi Coleen, On 2018-05-25 17:32, coleen.phillimore at oracle.com wrote: > Summary: SystemDictionary has all strong roots.? The weak oop_storage is > processed by the WeakProcessor so it can be scanned and cleared > concurrently and/or by parallel threads. > > Please see bug for explanation. > > open webrev at http://cr.openjdk.java.net/~coleenp/8202813.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8202813 > > There might be more GC code that can be cleaned out since > G1RootClosures->weak_oops() seems unused now, but I'd like to defer that > to a smaller cleanup.? Also ClassLoaderData is still processed > inconsistently by the GCs, which would be nice to clean up. > > Tested with mach5 hs-tier1-5 with no failures.? Per's gc-test-suite, > dev-submit for performance testing, and runThese with all collectors, > with and without -XX:-ClassUnloading.? Also tested with class unloading > gc tests (which are also in hs-tier number tests). The changes looks good, just a couple of questions: * In your performance runs did you check the pause times? From what I can see some of the processing have moved from a parallel phase to a serial one, which could be problematic if the vm_weak_oop_storage is large. * When turning of ClassUnloading we treated the vm_weak_oop_storage as strong roots in the past, was that not needed? I don't see this being done after this change. Thanks, Stefan > > Thanks, > Coleen From martin.doerr at sap.com Tue May 29 10:30:55 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 29 May 2018 10:30:55 +0000 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: References: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> <7e7fc484287e4da4926176e0b5ae1b64@sap.com> <339D50B6-09D5-4342-A687-918A9A096B39@oracle.com> <6D2268EC-C1E7-4C90-BCD3-90D02D21FA08@oracle.com> Message-ID: <3e05cc86c9d4406d8a9875b705fbf1fc@sap.com> Hi Kim, I'm trying to understand how this is related to Michihiro's change. The else path of the initial test is not affected by it AFAICS. So it sounds like a request to fix the current implementation in addition to what his original intend was. Anyway, I agree with that implicit consume is not good. And I think it would be good to treat both o->forwardee() the same way. What about keeping memory_order_release for the CAS and using acquire for both o->forwardee()? The case in which the CAS succeeds is safe because the current thread has created new_obj so it doesn't need memory barriers to access it. Thanks and best regards, Martin -----Original Message----- From: Kim Barrett [mailto:kim.barrett at oracle.com] Sent: Dienstag, 29. Mai 2018 01:54 To: Michihiro Horie Cc: Erik Osterlund ; david.holmes at oracle.com; Gustavo Bueno Romero ; hotspot-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; Doerr, Martin Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 > On May 28, 2018, at 4:12 AM, Michihiro Horie wrote: > > Hi Erik, > > Thank you very much for your review. > > I understood that implicit consume should not be used in the shared code. Also, I believe performance degradation would be negligible even if we use acquire. > > New webrev uses memory_order_acq_rel: http://cr.openjdk.java.net/~mhorie/8154736/webrev.10 This is missing the acquire barrier on the else branch for the initial test, so fails to meet the previously described minimal requirements for even possibly being sufficient. Any analysis of weakening the CAS barriers must consider that test and successor code. In the analysis, it?s not just the lexically nearby debugging / logging code that needs to be considered; the forwardee is being returned to caller(s) that will presumably do something with that object. Since the whole point of this discussion is performance, any proposed change should come with performance information. From coleen.phillimore at oracle.com Tue May 29 11:46:01 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 29 May 2018 07:46:01 -0400 Subject: RFR (M) 8202813: Move vm_weak processing from SystemDictionary to WeakProcessor In-Reply-To: <727eb265-3a7d-6e91-6677-b630de24274a@oracle.com> References: <4e7c8f9f-e1fa-0a29-ac22-5215b55dee69@oracle.com> <727eb265-3a7d-6e91-6677-b630de24274a@oracle.com> Message-ID: On 5/29/18 6:27 AM, Stefan Johansson wrote: > Hi Coleen, > > On 2018-05-25 17:32, coleen.phillimore at oracle.com wrote: >> Summary: SystemDictionary has all strong roots.? The weak oop_storage >> is processed by the WeakProcessor so it can be scanned and cleared >> concurrently and/or by parallel threads. >> >> Please see bug for explanation. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8202813.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8202813 >> >> There might be more GC code that can be cleaned out since >> G1RootClosures->weak_oops() seems unused now, but I'd like to defer >> that to a smaller cleanup.? Also ClassLoaderData is still processed >> inconsistently by the GCs, which would be nice to clean up. >> >> Tested with mach5 hs-tier1-5 with no failures.? Per's gc-test-suite, >> dev-submit for performance testing, and runThese with all collectors, >> with and without -XX:-ClassUnloading.? Also tested with class >> unloading gc tests (which are also in hs-tier number tests). > > The changes looks good, just a couple of questions: > * In your performance runs did you check the pause times? From what I > can see some of the processing have moved from a parallel phase to a > serial one, which could be problematic if the vm_weak_oop_storage is > large. Thank you for looking at this Stefan.? The SystemDictionary::vm_weak_oop_storage is not large (~500 entries avg in my measurements with SPECjbb2015).? I don't think it benefited from parallelism.? That said, I don't think processing the few strong roots in SystemDictionary are worth processing in their own thread anymore either (it used to be a lot larger when this was all set up).?? It might make sense to move all of these strong roots together into Universe. The goal is to make the oop_storage processing parallel, which Kim is working on, and not have these separate. I did generate all of the gc.log files from a performance run but haven't dug them out yet.? Eric Caspole's been helping me graph the pause times so can we talk offline for what to look for in these files? > > * When turning of ClassUnloading we treated the vm_weak_oop_storage as > strong roots in the past, was that not needed? I don't see this being > done after this change. It's not needed because the oops in the vm_weak_oop_storage relating to ClassUnloading will be marked when we process the ClassLoaderDataGraph so this extra walk only marked the additional ResolvedMethodTable oops.? This table has appropriate NULL checks and should probably be removed from SystemDictionary.? It was added there for convenience. Thank you! Coleen > > Thanks, > Stefan > >> >> Thanks, >> Coleen From coleen.phillimore at oracle.com Tue May 29 11:46:14 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 29 May 2018 07:46:14 -0400 Subject: RFR (M) 8202813: Move vm_weak processing from SystemDictionary to WeakProcessor In-Reply-To: <98F4F244-C5C7-4F2B-AEB3-40E8E44F3708@oracle.com> References: <4e7c8f9f-e1fa-0a29-ac22-5215b55dee69@oracle.com> <98F4F244-C5C7-4F2B-AEB3-40E8E44F3708@oracle.com> Message-ID: Thanks Kim! Coleen On 5/28/18 5:03 PM, Kim Barrett wrote: >> On May 25, 2018, at 11:32 AM, coleen.phillimore at oracle.com wrote: >> >> Summary: SystemDictionary has all strong roots. The weak oop_storage is processed by the WeakProcessor so it can be scanned and cleared concurrently and/or by parallel threads. >> >> Please see bug for explanation. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8202813.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8202813 >> >> There might be more GC code that can be cleaned out since G1RootClosures->weak_oops() seems unused now, but I'd like to defer that to a smaller cleanup. Also ClassLoaderData is still processed inconsistently by the GCs, which would be nice to clean up. >> >> Tested with mach5 hs-tier1-5 with no failures. Per's gc-test-suite, dev-submit for performance testing, and runThese with all collectors, with and without -XX:-ClassUnloading. Also tested with class unloading gc tests (which are also in hs-tier number tests). >> >> Thanks, >> Coleen > Looks good. > From coleen.phillimore at oracle.com Tue May 29 11:48:26 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 29 May 2018 07:48:26 -0400 Subject: RFR: 8203885: ConcurrentLocksDump::dump_at_safepoint() should not allocate array in resource area In-Reply-To: <1d42feb8-6e91-9b4b-ba6d-47dd82739cda@oracle.com> References: <99967815-2c84-7e09-7934-1b5c22639425@oracle.com> <08E2B22F-DDE0-4F20-9F87-B0B803721245@oracle.com> <1d42feb8-6e91-9b4b-ba6d-47dd82739cda@oracle.com> Message-ID: <23bdf1ee-0e95-df0a-09e9-322195673f50@oracle.com> On 5/29/18 2:20 AM, Per Liden wrote: > Hi Kim, > > On 05/29/2018 08:09 AM, Kim Barrett wrote: >>> On May 28, 2018, at 8:16 AM, Per Liden wrote: >>> >>> ConcurrentLocksDump::dump_at_safepoint() creates a GrowableArray, >>> which gets allocated in a resource area. This array is than passed >>> down a call chain, where it can't control that another ResourceMark >>> isn't created. In the leaf of this call chain, a closure >>> (FindInstanceClosure) is executed, which appends to the array, which >>> means it might need to be resized. This doesn't work if a new >>> ResourceMark has been created, since the array resize will happen in >>> a nested ResourceArea context. As a result, the append operation >>> fails in GenericGrowableArray::check_nesting(). >>> >>> This has so far gone unnoticed because >>> CollectedHeap::object_iterate() in existing collectors typically >>> don't create new ResourceMarks. This is not true for ZGC (and >>> potentially other concurrent collectors), which needs to walk thread >>> stacks, which in turn requires a ResourceMark. >>> >>> The proposed fix is to make this array C Heap allocated. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8203885 >>> Webrev: http://cr.openjdk.java.net/~pliden/8203885/webrev.0 >>> >>> Testing: hs-tier{1,3} >>> >>> /Per >> >> Looks good. > > Thanks for reviewing! > >> >> I was going to ask why the GrowableArray needs to be heap allocated? >> Why not just >> >> ?? GrowableArray aos_objects(INITIAL_ARRAY_SIZE, true); >> ?? ... s/aos_objects/&aos_objects/ ... >> ?? // delete aos_objects no longer needed >> >> After some digging, probably because of this, from growableArray.hpp: >> >> 119??? assert(!on_C_heap() || allocated_on_C_heap(), "growable array >> must be on C heap if elements are"); > > I had the same reaction and initially made it stack allocated, but > noticed that GrowableArray doesn't allow that :( > >> >> Conflating the allocation of the GrowableArray object with the >> allocation of the underlying array.? That seems wrong, but not a >> problem to be solved as part of this change. >> > > I agree. I suspect someone wanted to protect against potential > problems with having different life cycles of the GrowableArray and > the backing array. But this seems overly strict. Yes, this is exactly the reason, and we did have bugs.? That's why we have this check. Coleen > > /Per From erik.osterlund at oracle.com Tue May 29 12:00:55 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Tue, 29 May 2018 14:00:55 +0200 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: <3e05cc86c9d4406d8a9875b705fbf1fc@sap.com> References: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> <7e7fc484287e4da4926176e0b5ae1b64@sap.com> <339D50B6-09D5-4342-A687-918A9A096B39@oracle.com> <6D2268EC-C1E7-4C90-BCD3-90D02D21FA08@oracle.com> <3e05cc86c9d4406d8a9875b705fbf1fc@sap.com> Message-ID: <5B0D40F7.5050807@oracle.com> Hi Martin and Michihiro, On 2018-05-29 12:30, Doerr, Martin wrote: > Hi Kim, > > I'm trying to understand how this is related to Michihiro's change. The else path of the initial test is not affected by it AFAICS. > So it sounds like a request to fix the current implementation in addition to what his original intend was. I think we are just trying to nail down the correct fencing and just go for that. And yes, this is arguably a pre-existing problem, but in a race involving the very same accesses that we are changing the fencing for. So it is not completely unrelated I suppose. In particular, hotspot has code that assumes that if you on the writer side issue a full fence before publishing a pointer to newly initialized data, then the initializing stores and their side effects should be globally "visible" across the system before the pointer to it is published, and hence elide the need for acquire on the loading side, without relying on retained data dependencies on the loader side. I believe this code falls under that category. It is assumed that the leading fence of the CAS publishing the forwarding pointer makes the initializing stores globally observable before publishing a pointer to the initialized data, hence assuming that any loads able to observe the new pointer would not rely on acquire or data dependent loads to correctly read the initialized data. Unfortunately, this is not reliable in the IRIW case, as per the litmus test "MP+sync+ctrl" as described in "Understanding POWER multiprocessors" (https://dl.acm.org/citation.cfm?id=1993520), as opposed to "MP+sync+addr" that gets away with it because of the data dependency (not IRIW). Similarly, an isync does the job too on the reader side as shown in MP+sync+ctrlisync. So while what I believe was the previous reasoning that the leading sync of the CAS would elide the necessity for acquire on the reader side without relying on data dependent loads (implicit consume), I think that assumption was wrong in the first place and that we do indeed need explicit acquire (even with the precious conservative CAS fencing) in this context to not rely on implicit consume semantics generating the required data dependent loads on the reader side. In practice though, the leading sync of the CAS has been enough to generate the correct machine code. Now, with the leading sync removed, we are increasing the possible holes in the generated machine code due to this flawed reasoning. So it would be nice to do something more sound instead that does not make such assumptions. > Anyway, I agree with that implicit consume is not good. And I think it would be good to treat both o->forwardee() the same way. > What about keeping memory_order_release for the CAS and using acquire for both o->forwardee()? > The case in which the CAS succeeds is safe because the current thread has created new_obj so it doesn't need memory barriers to access it. Sure, that sounds good to me. Thanks, /Erik > Thanks and best regards, > Martin > > > -----Original Message----- > From: Kim Barrett [mailto:kim.barrett at oracle.com] > Sent: Dienstag, 29. Mai 2018 01:54 > To: Michihiro Horie > Cc: Erik Osterlund ; david.holmes at oracle.com; Gustavo Bueno Romero ; hotspot-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; Doerr, Martin > Subject: Re: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 > >> On May 28, 2018, at 4:12 AM, Michihiro Horie wrote: >> >> Hi Erik, >> >> Thank you very much for your review. >> >> I understood that implicit consume should not be used in the shared code. Also, I believe performance degradation would be negligible even if we use acquire. >> >> New webrev uses memory_order_acq_rel: http://cr.openjdk.java.net/~mhorie/8154736/webrev.10 > This is missing the acquire barrier on the else branch for the initial test, so fails to meet > the previously described minimal requirements for even possibly being sufficient. Any > analysis of weakening the CAS barriers must consider that test and successor code. > > In the analysis, it?s not just the lexically nearby debugging / logging code that needs to be > considered; the forwardee is being returned to caller(s) that will presumably do something > with that object. > > Since the whole point of this discussion is performance, any proposed change should come > with performance information. > From stefan.johansson at oracle.com Tue May 29 12:38:03 2018 From: stefan.johansson at oracle.com (Stefan Johansson) Date: Tue, 29 May 2018 14:38:03 +0200 Subject: RFR (M) 8202813: Move vm_weak processing from SystemDictionary to WeakProcessor In-Reply-To: References: <4e7c8f9f-e1fa-0a29-ac22-5215b55dee69@oracle.com> <727eb265-3a7d-6e91-6677-b630de24274a@oracle.com> Message-ID: <959bffd0-d4c0-1c32-6e85-f488a726e3e2@oracle.com> On 2018-05-29 13:46, coleen.phillimore at oracle.com wrote: > > > On 5/29/18 6:27 AM, Stefan Johansson wrote: >> Hi Coleen, >> >> On 2018-05-25 17:32, coleen.phillimore at oracle.com wrote: >>> Summary: SystemDictionary has all strong roots.? The weak oop_storage >>> is processed by the WeakProcessor so it can be scanned and cleared >>> concurrently and/or by parallel threads. >>> >>> Please see bug for explanation. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/8202813.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8202813 >>> >>> There might be more GC code that can be cleaned out since >>> G1RootClosures->weak_oops() seems unused now, but I'd like to defer >>> that to a smaller cleanup.? Also ClassLoaderData is still processed >>> inconsistently by the GCs, which would be nice to clean up. >>> >>> Tested with mach5 hs-tier1-5 with no failures.? Per's gc-test-suite, >>> dev-submit for performance testing, and runThese with all collectors, >>> with and without -XX:-ClassUnloading.? Also tested with class >>> unloading gc tests (which are also in hs-tier number tests). >> >> The changes looks good, just a couple of questions: >> * In your performance runs did you check the pause times? From what I >> can see some of the processing have moved from a parallel phase to a >> serial one, which could be problematic if the vm_weak_oop_storage is >> large. > > Thank you for looking at this Stefan.? The > SystemDictionary::vm_weak_oop_storage is not large (~500 entries avg in > my measurements with SPECjbb2015).? I don't think it benefited from > parallelism.? That said, I don't think processing the few strong roots > in SystemDictionary are worth processing in their own thread anymore > either (it used to be a lot larger when this was all set up).?? It might > make sense to move all of these strong roots together into Universe. > > The goal is to make the oop_storage processing parallel, which Kim is > working on, and not have these separate. > Sounds good. > I did generate all of the gc.log files from a performance run but > haven't dug them out yet.? Eric Caspole's been helping me graph the > pause times so can we talk offline for what to look for in these files? > Graphs are always nice :) One good thing, that works again with the latest JFR changes, is to do the aurora-perf runs with JFR enabled. That will give some basic GC events which are automatically parsed and displayed in the report. That makes it a lot easier to spot pause time regressions since they seldom show up in the benchmark result. What I would look out for is longer Young GC pause times, looking at the average over a run is usually a good first step. >> >> * When turning of ClassUnloading we treated the vm_weak_oop_storage as >> strong roots in the past, was that not needed? I don't see this being >> done after this change. > > It's not needed because the oops in the vm_weak_oop_storage relating to > ClassUnloading will be marked when we process the ClassLoaderDataGraph > so this extra walk only marked the additional ResolvedMethodTable oops. > This table has appropriate NULL checks and should probably be removed > from SystemDictionary.? It was added there for convenience. > Thanks for the explanation, with that the change looks good code wise and if you feel comfortable with the performance I don't see any problem moving forward with the change. Cheers, Stefan > Thank you! > Coleen > >> >> Thanks, >> Stefan >> >>> >>> Thanks, >>> Coleen > From coleen.phillimore at oracle.com Tue May 29 13:13:36 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 29 May 2018 09:13:36 -0400 Subject: RFR (M) 8202813: Move vm_weak processing from SystemDictionary to WeakProcessor In-Reply-To: <959bffd0-d4c0-1c32-6e85-f488a726e3e2@oracle.com> References: <4e7c8f9f-e1fa-0a29-ac22-5215b55dee69@oracle.com> <727eb265-3a7d-6e91-6677-b630de24274a@oracle.com> <959bffd0-d4c0-1c32-6e85-f488a726e3e2@oracle.com> Message-ID: On 5/29/18 8:38 AM, Stefan Johansson wrote: > > > On 2018-05-29 13:46, coleen.phillimore at oracle.com wrote: >> >> >> On 5/29/18 6:27 AM, Stefan Johansson wrote: >>> Hi Coleen, >>> >>> On 2018-05-25 17:32, coleen.phillimore at oracle.com wrote: >>>> Summary: SystemDictionary has all strong roots.? The weak >>>> oop_storage is processed by the WeakProcessor so it can be scanned >>>> and cleared concurrently and/or by parallel threads. >>>> >>>> Please see bug for explanation. >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/8202813.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8202813 >>>> >>>> There might be more GC code that can be cleaned out since >>>> G1RootClosures->weak_oops() seems unused now, but I'd like to defer >>>> that to a smaller cleanup.? Also ClassLoaderData is still processed >>>> inconsistently by the GCs, which would be nice to clean up. >>>> >>>> Tested with mach5 hs-tier1-5 with no failures.? Per's >>>> gc-test-suite, dev-submit for performance testing, and runThese >>>> with all collectors, with and without -XX:-ClassUnloading.? Also >>>> tested with class unloading gc tests (which are also in hs-tier >>>> number tests). >>> >>> The changes looks good, just a couple of questions: >>> * In your performance runs did you check the pause times? From what >>> I can see some of the processing have moved from a parallel phase to >>> a serial one, which could be problematic if the vm_weak_oop_storage >>> is large. >> >> Thank you for looking at this Stefan.? The >> SystemDictionary::vm_weak_oop_storage is not large (~500 entries avg >> in my measurements with SPECjbb2015).? I don't think it benefited >> from parallelism.? That said, I don't think processing the few strong >> roots in SystemDictionary are worth processing in their own thread >> anymore either (it used to be a lot larger when this was all set >> up).?? It might make sense to move all of these strong roots together >> into Universe. >> >> The goal is to make the oop_storage processing parallel, which Kim is >> working on, and not have these separate. >> > Sounds good. > >> I did generate all of the gc.log files from a performance run but >> haven't dug them out yet.? Eric Caspole's been helping me graph the >> pause times so can we talk offline for what to look for in these files? >> > Graphs are always nice :) One good thing, that works again with the > latest JFR changes, is to do the aurora-perf runs with JFR enabled. > That will give some basic GC events which are automatically parsed and > displayed in the report. That makes it a lot easier to spot pause time > regressions since they seldom show up in the benchmark result. > > What I would look out for is longer Young GC pause times, looking at > the average over a run is usually a good first step. > >>> >>> * When turning of ClassUnloading we treated the vm_weak_oop_storage >>> as strong roots in the past, was that not needed? I don't see this >>> being done after this change. >> >> It's not needed because the oops in the vm_weak_oop_storage relating >> to ClassUnloading will be marked when we process the >> ClassLoaderDataGraph so this extra walk only marked the additional >> ResolvedMethodTable oops. This table has appropriate NULL checks and >> should probably be removed from SystemDictionary.? It was added there >> for convenience. >> > > Thanks for the explanation, with that the change looks good code wise > and if you feel comfortable with the performance I don't see any > problem moving forward with the change. The overall performance results were neutral (critical jOPS), so I think they are fine, but I'll dig out the data and post it in the bug report for the record (and to double check nothing is amiss). Thanks! Coleen > > Cheers, > Stefan > >> Thank you! >> Coleen >> >>> >>> Thanks, >>> Stefan >>> >>>> >>>> Thanks, >>>> Coleen >> From gromero at linux.vnet.ibm.com Tue May 29 13:23:04 2018 From: gromero at linux.vnet.ibm.com (Gustavo Romero) Date: Tue, 29 May 2018 10:23:04 -0300 Subject: UseNUMA membind Issue in openJDK In-Reply-To: References: <9a0310b7-2880-db69-cfbc-7abba844ecbf@oracle.com> <187f278f-5069-3afa-c61f-f49a1fc0d790@linux.vnet.ibm.com> Message-ID: <36dc8538-f984-d2df-e401-8362a429c6f3@linux.vnet.ibm.com> Hi Swati, On 05/29/2018 06:12 AM, Swati Sharma wrote: > I have incorporated some changes suggested by you. > > The use of struct bitmask's ?maskp for checking 64 bit in single iteration > is more optimized compared to numa_bitmask_isbitset() ?as by using this we > need to check each bit for ?1024 times(SUSE case) and 64 times(Ubuntu Case). > If its fine to iterate at initialization time then I can change. Yes, I know, your version is more optimized. libnuma API should provide a ready-made solution for that... but that's another story. I'm curious to know what the time difference is on the worst case for both ways tho. Anyway, I just would like to point out that, regardless performance, it's possible to achieve the same result with current libnuma API. > For the answer to your question: > If it picks up node 16, not so bad, but what if it picks up node 0 or 1? > It can be checked based on numa_distance instead of picking up the lgrps randomly. That seems a good solution. You can do the checking very early, so lgrp_spaces()->find() does not even fail (return -1), i.e. by changing the CPU to node mapping on initialization (avoiding to change cas_allocate()). On that checking both numa distance and if the node is bound (or not) would be considered to generate the map. Best regards, Gustavo > Thanks, > Swati > > > On Fri, May 25, 2018 at 4:54 AM, Gustavo Romero > wrote: > > Hi Swati, > > > Thanks for CC:ing me. Sorry for the delay replying it, I had to reserve a few > specific machines before trying your patch :-) > > I think that UseNUMA's original task was to figure out the best binding > setup for the JVM automatically but I understand that it also has to be aware > that sometimes, for some (new) particular reasons, its binding task is > "modulated" by other external agents. Thanks for proposing a fix. > > I have just a question/concern on the proposal: how the JVM should behave if > CPUs are not bound in accordance to the bound memory nodes? For instance, what > happens if no '--cpunodebind' is passed and '--membind=0,1,16' is passed at > the same time on this numa topology: > > brianh at p215n12:~$ numactl -H > available: 4 nodes (0-1,16-17) > node 0 cpus: 0 1 2 3 8 9 10 11 16 17 18 19 24 25 26 27 32 33 34 35 > node 0 size: 65342 MB > node 0 free: 56902 MB > node 1 cpus: 40 41 42 43 48 49 50 51 56 57 58 59 64 65 66 67 72 73 74 75 > node 1 size: 65447 MB > node 1 free: 58322 MB > node 16 cpus: 80 81 82 83 88 89 90 91 96 97 98 99 104 105 106 107 112 113 114 115 > node 16 size: 65448 MB > node 16 free: 63096 MB > node 17 cpus: 120 121 122 123 128 129 130 131 136 137 138 139 144 145 146 147 152 153 154 155 > node 17 size: 65175 MB > node 17 free: 61522 MB > node distances: > node? ?0? ?1? 16? 17 > ? 0:? 10? 20? 40? 40 > ? 1:? 20? 10? 40? 40 > ?16:? 40? 40? 10? 20 > ?17:? 40? 40? 20? 10 > > > In that case JVM will spawn threads that will run on all CPUs, including those > CPUs in numa node 17. Then once in > src/hotspot/share/gc/parallel/mutableNUMASpace.cpp, in cas_allocate(): > > ?834 // This version is lock-free. > ?835 HeapWord* MutableNUMASpace::cas_allocate(size_t size) { > ?836? ?Thread* thr = Thread::current(); > ?837? ?int lgrp_id = thr->lgrp_id(); > ?838? ?if (lgrp_id == -1 || !os::numa_has_group_homing()) { > ?839? ? ?lgrp_id = os::numa_get_group_id(); > ?840? ? ?thr->set_lgrp_id(lgrp_id); > ?841? ?} > > a newly created thread will try to be mapped to a numa node given your CPU ID. > So if that CPU is in numa node 17 it will then not find it in: > > ?843? ?int i = lgrp_spaces()->find(&lgrp_id, LGRPSpace::equals); > > and will fallback to a random map, picking up a random numa node among nodes > 0, 1, and 16: > > ?846? ?if (i == -1) { > ?847? ? ?i = os::random() % lgrp_spaces()->length(); > ?848? ?} > > If it picks up node 16, not so bad, but what if it picks up node 0 or 1? > > I see that if one binds mem but leaves CPU unbound one has to know exactly what > she/he is doing, because it can be likely suboptimal. On the other hand, letting > the node being picked up randomly when there are memory nodes bound but no CPUs > seems even more suboptimal in some scenarios. Thus, should the JVM deal with it? > > @Zhengyu, do you have any opinion on that? > > Please find a few nits / comments inline. > > Note that I'm not a (R)eviewer so you still need two official reviews. > > > Best regards, > Gustavo > > On 05/21/2018 01:44 PM, Swati Sharma wrote: > > ======================PATCH============================== > diff --git a/src/hotspot/os/linux/os_linux.cpp b/src/hotspot/os/linux/os_linux.cpp > --- a/src/hotspot/os/linux/os_linux.cpp > +++ b/src/hotspot/os/linux/os_linux.cpp > @@ -2832,14 +2832,42 @@ > ? ? // Map all node ids in which is possible to allocate memory. Also nodes are > ? ? // not always consecutively available, i.e. available from 0 to the highest > ? ? // node number. > +? // If the nodes have been bound explicitly using numactl membind, then > +? // allocate memory from those nodes only. > > > I think ok to place that comment on the same existing line, like: > > -? // node number. > +? // node number. If the nodes have been bound explicitly using numactl membind, > +? // then allocate memory from these nodes only. > > > ? ? for (size_t node = 0; node <= highest_node_number; node++) { > -? ? if (Linux::isnode_in_configured_nodes(node)) { > +? ? if (Linux::isnode_in_bounded_nodes(node)) { > > ---------------------------------^ s/bounded/bound/ > > > ? ? ? ? ids[i++] = node; > ? ? ? } > ? ? } > ? ? return i; > ? } > +extern "C"? struct bitmask { > +? unsigned long size; /* number of bits in the map */ > +? unsigned long *maskp; > +}; > > > I think it's possible to move the function below to os_linux.hpp with its > friends and cope with the forward declaration of 'struct bitmask*` by using the > functions from numa API, notably numa_bitmask_nbytes() and > numa_bitmask_isbitset() only,? avoiding the member dereferecing issue and the > need to add the above struct explicitly. > > > +// Check if single memory node bound. > +// Returns true if single memory node bound. > > > I suggest a minuscule improvement, something like: > > +// Check if bound to only one numa node. > +// Returns true if bound to a single numa node, otherwise returns false. > > > +bool os::Linux::issingle_node_bound() { > > > What about s/issingle_node_bound/isbound_to_single_node/ ? > > > +? struct bitmask* bmp = _numa_get_membind != NULL ? _numa_get_membind() : NULL; > +? if(!(bmp != NULL && bmp->maskp != NULL)) return false; > > ? ? ? ? ? ? ? ? ? ? ? ? ? -----^ > Are you sure this checking is necessary? I think if numa_get_membind succeed > bmp->maskp is always != NULL. > > Indentation here is odd. No space before 'if' and return on the same line. > > I would try to avoid lines over 80 chars. > > > +? int issingle = 0; > +? // System can have more than 64 nodes so check in all the elements of > +? // unsigned long array > +? for (unsigned long i = 0; i < (bmp->size / (8 * sizeof(unsigned long))); i++) { > +? ? if (bmp->maskp[i] == 0) { > +? ? ? continue; > +? ? } else if ((bmp->maskp[i] & (bmp->maskp[i] - 1)) == 0) { > +? ? ? issingle++; > +? ? } else { > +? ? ? return false; > +? ? } > +? } > +? if (issingle == 1) > +? ? return true; > +? return false; > +} > + > > > As I mentioned, I think it could be moved to os_linux.hpp instead. Also, it > could be something like: > > +bool os::Linux::isbound_to_single_node(void) { > +? struct bitmask* bmp; > +? unsigned long mask; // a mask element in the mask array > +? unsigned long max_num_masks; > +? int single_node = 0; > + > +? if (_numa_get_membind != NULL) { > +? ? bmp = _numa_get_membind(); > +? } else { > +? ? return false; > +? } > + > +? max_num_masks = bmp->size / (8 * sizeof(unsigned long)); > + > +? for (mask = 0; mask < max_num_masks; mask++) { > +? ? if (bmp->maskp[mask] != 0) { // at least one numa node in the mask > +? ? ? if (bmp->maskp[mask] & (bmp->maskp[mask] - 1) == 0) { > +? ? ? ? single_node++; // a single numa node in the mask > +? ? ? } else { > +? ? ? ? return false; > +? ? ? } > +? ? } > +? } > + > +? if (single_node == 1) { > +? ? return true; // only a single mask with a single numa node > +? } else { > +? ? return false; > +? } > +} > > > ? bool os::get_page_info(char *start, page_info* info) { > ? ? return false; > ? } > @@ -2930,6 +2958,10 @@ > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?libnuma_dlsym(handle, "numa_bitmask_isbitset"))); > ? ? ? ? set_numa_distance(CAST_TO_FN_PTR(numa_distance_func_t, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?libnuma_dlsym(handle, "numa_distance"))); > +? ? ? set_numa_set_membind(CAST_TO_FN_PTR(numa_set_membind_func_t, > +? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? libnuma_dlsym(handle, "numa_set_membind"))); > +? ? ? set_numa_get_membind(CAST_TO_FN_PTR(numa_get_membind_func_t, > +? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? libnuma_v2_dlsym(handle, "numa_get_membind"))); > ? ? ? ? if (numa_available() != -1) { > ? ? ? ? ? set_numa_all_nodes((unsigned long*)libnuma_dlsym(handle, "numa_all_nodes")); > @@ -3054,6 +3086,8 @@ > ? os::Linux::numa_set_bind_policy_func_t os::Linux::_numa_set_bind_policy; > ? os::Linux::numa_bitmask_isbitset_func_t os::Linux::_numa_bitmask_isbitset; > ? os::Linux::numa_distance_func_t os::Linux::_numa_distance; > +os::Linux::numa_set_membind_func_t os::Linux::_numa_set_membind; > +os::Linux::numa_get_membind_func_t os::Linux::_numa_get_membind; > ? unsigned long* os::Linux::_numa_all_nodes; > ? struct bitmask* os::Linux::_numa_all_nodes_ptr; > ? struct bitmask* os::Linux::_numa_nodes_ptr; > @@ -4962,8 +4996,9 @@ > ? ? ? if (!Linux::libnuma_init()) { > ? ? ? ? UseNUMA = false; > ? ? ? } else { > -? ? ? if ((Linux::numa_max_node() < 1)) { > -? ? ? ? // There's only one node(they start from 0), disable NUMA. > +? ? ? if ((Linux::numa_max_node() < 1) || Linux::issingle_node_bound()) { > +? ? ? ? // If there's only one node(they start from 0) or if the process > +? ? ? ? // is bound explicitly to a single node using membind, disable NUMA. > ? ? ? ? ? UseNUMA = false; > ? ? ? ? } > ? ? ? } > diff --git a/src/hotspot/os/linux/os_linux.hpp b/src/hotspot/os/linux/os_linux.hpp > --- a/src/hotspot/os/linux/os_linux.hpp > +++ b/src/hotspot/os/linux/os_linux.hpp > @@ -228,6 +228,8 @@ > ? ? typedef int (*numa_tonode_memory_func_t)(void *start, size_t size, int node); > ? ? typedef void (*numa_interleave_memory_func_t)(void *start, size_t size, unsigned long *nodemask); > ? ? typedef void (*numa_interleave_memory_v2_func_t)(void *start, size_t size, struct bitmask* mask); > +? typedef void (*numa_set_membind_func_t)(struct bitmask *mask); > +? typedef struct bitmask* (*numa_get_membind_func_t)(void); > ? ? typedef void (*numa_set_bind_policy_func_t)(int policy); > ? ? typedef int (*numa_bitmask_isbitset_func_t)(struct bitmask *bmp, unsigned int n); > @@ -244,6 +246,8 @@ > ? ? static numa_set_bind_policy_func_t _numa_set_bind_policy; > ? ? static numa_bitmask_isbitset_func_t _numa_bitmask_isbitset; > ? ? static numa_distance_func_t _numa_distance; > +? static numa_set_membind_func_t _numa_set_membind; > +? static numa_get_membind_func_t _numa_get_membind; > ? ? static unsigned long* _numa_all_nodes; > ? ? static struct bitmask* _numa_all_nodes_ptr; > ? ? static struct bitmask* _numa_nodes_ptr; > @@ -259,6 +263,8 @@ > ? ? static void set_numa_set_bind_policy(numa_set_bind_policy_func_t func) { _numa_set_bind_policy = func; } > ? ? static void set_numa_bitmask_isbitset(numa_bitmask_isbitset_func_t func) { _numa_bitmask_isbitset = func; } > ? ? static void set_numa_distance(numa_distance_func_t func) { _numa_distance = func; } > +? static void set_numa_set_membind(numa_set_membind_func_t func) { _numa_set_membind = func; } > +? static void set_numa_get_membind(numa_get_membind_func_t func) { _numa_get_membind = func; } > ? ? static void set_numa_all_nodes(unsigned long* ptr) { _numa_all_nodes = ptr; } > ? ? static void set_numa_all_nodes_ptr(struct bitmask **ptr) { _numa_all_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > ? ? static void set_numa_nodes_ptr(struct bitmask **ptr) { _numa_nodes_ptr = (ptr == NULL ? NULL : *ptr); } > @@ -320,6 +326,15 @@ > ? ? ? } else > ? ? ? ? return 0; > ? ? } > +? // Check if node in bounded nodes > > > +? // Check if node is in bound node set. Maybe? > > > +? static bool isnode_in_bounded_nodes(int node) { > +? ? struct bitmask* bmp = _numa_get_membind != NULL ? _numa_get_membind() : NULL; > +? ? if (bmp != NULL && _numa_bitmask_isbitset != NULL && _numa_bitmask_isbitset(bmp, node)) { > +? ? ? return true; > +? ? } else > +? ? ? return false; > +? } > +? static bool issingle_node_bound(); > > > Looks like it can be re-written like: > > +? static bool isnode_in_bound_nodes(int node) { > +? ? if (_numa_get_membind != NULL && _numa_bitmask_isbitset != NULL) { > +? ? ? return _numa_bitmask_isbitset(_numa_get_membind(), node); > +? ? } else { > +? ? ? return false; > +? ? } > +? } > > ? > > > ? }; > ? #endif // OS_LINUX_VM_OS_LINUX_HPP > > > From stefan.karlsson at oracle.com Tue May 29 14:21:09 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 29 May 2018 16:21:09 +0200 Subject: RFR: 8203923: Add @requires feature to check flag values for the running JVM Message-ID: Hi all, Please review this patch to add @requires vm.opt.final., as a way to filter jtreg tests on the final value of flags in the test VM. http://cr.openjdk.java.net/~stefank/8203923/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8203923 This patch is needed to allow tests to be filtered out when ZGC is run. ZGC doesn't support UseCompressedOops (and currently doesn't support ClassUnloading). Instead of adding @requires vm.gc != "Z" for tests that rely on ClassUnloading, I think it's nicer to write @requires vm.opt.final.ClassUnloading. This will filter out the test if any of the following conditions are met: 1) The currently executing VM doesn't support ClassUnloading 2) -XX:-ClassUnloading is passed to the test The second case can today be handled by @requires vm.opt.ClassUnloading == NULL | vm.opt.ClassUnloading == true, or maybe simpler @requires vm.opt.ClassUnloading != false, but if you need both @requires vm.opt.final.ClassUnloading should be used. The current solution requires the queried flags to be listed in vmOptFinalFlags. Maybe there's a way to get this to cover all flags automatically? Thanks, StefanK From lois.foltan at oracle.com Tue May 29 14:57:50 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 29 May 2018 10:57:50 -0400 Subject: RFR (XS) 8189766: whitebox failure with -Xcheck:jni In-Reply-To: References: Message-ID: <51007b0e-da67-b459-6121-723cbae33376@oracle.com> Looks good. Lois On 5/29/2018 3:06 AM, David Holmes wrote: > bug: https://bugs.openjdk.java.net/browse/JDK-8189766 > webrev: http://cr.openjdk.java.net/~dholmes/8189766/webrev/ > > A WB_ENTRY already handles the pending JNI exception check by > explicitly clearing it via a helper: > > #define WB_ENTRY(result_type, header) JNI_ENTRY(result_type, header) \ > ? ClearPendingJniExcCheck _clearCheck(env); > > #define WB_END JNI_END > > But the whitebox function then does e.g: > > WB_ENTRY(jobject, WB_GetBooleanVMFlag(JNIEnv* env, jobject o, jstring > name)) > ? bool result; > ? if (GetVMFlag (thread, env, name, &result, &JVMFlag::boolAt)) { > ??? ThreadToNativeFromVM ttnfv(thread); // can't be in VM when we call > JNI > ??? return booleanBox(thread, env, result); > ? } > ? return NULL; > WB_END > > where booleanBox is defined eventually as: > > template > static jobject box(JavaThread* thread, JNIEnv* env, Symbol* name, > Symbol* sig, T value) { > ? ResourceMark rm(thread); > ? jclass clazz = env->FindClass(name->as_C_string()); > ? CHECK_JNI_EXCEPTION_(env, NULL); > ? jmethodID methodID = env->GetStaticMethodID(clazz, > ??????? vmSymbols::valueOf_name()->as_C_string(), > ??????? sig->as_C_string()); > ? CHECK_JNI_EXCEPTION_(env, NULL); > ? jobject result = env->CallStaticObjectMethod(clazz, methodID, value); > ? CHECK_JNI_EXCEPTION_(env, NULL); > ? return result; > } > > so we call JNI methods that will set the pending_jni_exception_check > flag, but CHECK_JNI_EXCEPTION is defined as: > > #define CHECK_JNI_EXCEPTION_(env, value) \ > ? do { \ > ??? JavaThread* THREAD = JavaThread::thread_from_jni_environment(env); \ > ??? if (HAS_PENDING_EXCEPTION) { \ > ????? return(value); \ > ??? } \ > ? } while (0) > > which means it is not clearing the pending_jni_exception_check flag > even though it is checking for exceptions! > > So we explicitly clear the flag. Trying to use the JNI > ExceptionOccurred function fails as not all the methods that call > CHECK_JNI_EXCEPTION are _thread_in_native. > > Testing (in progress): > ???????? hotspot tier1 and tier2 with -Xcheck:jni > ???????? regular CI testing > > Thanks, > David From boris.ulasevich at bell-sw.com Tue May 29 15:46:53 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Tue, 29 May 2018 18:46:53 +0300 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <71dbee89-451e-a151-474a-74326a2fa1c6@oracle.com> References: <53feed71-0ab9-7743-6034-3c95bb3b20e8@bell-sw.com> <848b45e0-df51-7d74-e4eb-60819d101330@oracle.com> <9a8eadf0-e745-ee38-fb37-b29d99fa96c0@bell-sw.com> <71dbee89-451e-a151-474a-74326a2fa1c6@oracle.com> Message-ID: Hi David, ARM32 part of the change looks good for me. Do you want me to apply latest (v4-incr?) patch and run tests? Boris On 24.05.2018 00:39, David Holmes wrote: > Hi Boris, > > Thanks for verifying my changes. In case you didn't see you can apply > the updated patch to my v1 changes from: > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2/ > > One follow up below. > > On 23/05/2018 11:40 PM, Boris Ulasevich wrote: >> Hi David, >> >> some minor comments below.. >> >> On 23.05.2018 09:57, David Holmes wrote: >>> Hi Boris, >>> >>> On 17/05/2018 7:23 PM, Boris Ulasevich wrote: >>>> Hi David, >>>> >>>> You are right! My bad, R2_tmp parameter conflicts with Rklass on >>>> check_klass_subtype(..) call. See correct patch in attach. Now all >>>> runtime/Nestmates tests passed! :) >>> >>> I went to aplpy your patch and found it's not a diff against my patch >>> but against the original non-nestmate code, >> >> Yes, sorry. For me it was natural to apply patch from your review to >> local repo, rework it and just send updated diff from my machine :) >> >>> so I want to be clear on the changes. AFAICS the differences are: >>> >>> -? const Register Rklass? = R3_tmp; >>> +? const Register Rklass? = R2_tmp; // Note! Same register with Rrecv >> >> Yes. >> >>> This initially concerned me as we stomp on Rrecv when we do the >>> load_klass, but then you moved? the: >>> >>> ??? __ load_klass(Rklass, Rrecv); >>> >>> to after the object case, which used Rrecv. I had assumed Rrecv was >>> somehow used when we actually do the call, but I'm assuming >>> jump_from_interpreted is done in such as way that the receiver is >>> still available on the interpreter stack and is used from there. >> >> Ok. I'm not sure I understand you well. When I moved load_klass call >> down my point was to skip unnecessary load for the notObjectMethod >> case (Rklass is not required in this case) and to make possible to >> reuse same R2 register by both Rklass and Rrecv register. > > Right. My concern was an assumption that as Rrecv was the receiver > object that we had to leave it intact ready for the actual method > invocation. But that isn't the case. The real method invocation happens > elsewhere and the receiver is obtained by other means. > > Thanks, > David > >>> -? __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >>> +? __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R3_tmp, noreg, >>> subtype); >>> >>> Okay - reworking of tmp regs. >> >> Yes, ARM32 requires one more tmp reg, and we can't spoil R0. >> >>> -? __ jump_from_interpreted(Rindex); >>> +? __ jump_from_interpreted(Rmethod); >>> >>> I'm going to trust this is okay :) It's not at all clear to me how >>> the f2 Method* gets passed through on different platforms - sometimes >>> its in the "index" and sometimes the "method" registers. >> >> I see. On ARM/AARCH64 we have preallocated register Rmethod/rmethod to >> hold current method pointer and prepare_invoke call to setup registers >> properly. >> >>> (I _really_ wish there was consistency in terminology across the >>> different platforms - this code is awful for trying to compare the >>> different platforms to figure out what to do on a new one.) >>> >>> Thanks, >>> David >>> >>>> regards, >>>> Boris >>>> >>>> On 17.05.2018 11:24, David Holmes wrote: >>>>> On 17/05/2018 6:13 PM, Boris Ulasevich wrote: >>>>>> Hi David, >>>>>> >>>>>> I see three tests failing: >>>>>> ?> NullPointerException at >>>>>> TestInterfaceMethodSelection.doInvoke(TestInterfaceMethodSelection.java:191) >>>>>> >>>>>> ?> NullPointerException at TestInvoke.access_priv(TestInvoke.java:54) >>>>>> ?> InvocationTargetException at >>>>>> TestReflection.access_priv(TestReflection.java:61) >>>>>> >>>>>> I will send you test details in a separate mail. >>>>> >>>>> Ok. This indicates a bug in the assembly code. The NPE's will >>>>> likely be SEGVs caused by a zero register. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> >>>>>> Boris >>>>>> >>>>>> On 17.05.2018 00:23, David Holmes wrote: >>>>>>> Hi Boris, >>>>>>> >>>>>>> Many thanks for looking at this and working through the ARM code. >>>>>>> >>>>>>> I will study your patch in detail but am concerned by the "passes >>>>>>> most of runtime/Nestmates tests Ok."! What tests are not passing? >>>>>>> >>>>>>> David >>>>>>> >>>>>>> On 17/05/2018 1:05 AM, Boris Ulasevich wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>> Let us look to the change in templateTable_arm.cpp. I have >>>>>>>> several notes. >>>>>>>> >>>>>>>> 1. We have compilation error because check_klass_subtype call >>>>>>>> needs one more temporary register parameter. I think correct >>>>>>>> change is: >>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >>>>>>>> -> >>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, >>>>>>>> subtype); >>>>>>>> >>>>>>>> 2. R0_tmp holds mdp used for profilation several lines below, so >>>>>>>> we can't spoil it. I think correct change is: >>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, >>>>>>>> subtype); >>>>>>>> -> >>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R2_tmp, noreg, >>>>>>>> subtype); >>>>>>>> >>>>>>>> 3. We can't jump to Rindex. I believe it was supposed to jump to >>>>>>>> Rmethod: >>>>>>>> jump_from_interpreted(Rindex); >>>>>>>> -> >>>>>>>> jump_from_interpreted(Rmethod); >>>>>>>> >>>>>>>> 4. Please note than Rklass and Rflags reuses same register. >>>>>>>> Before the change their life ranges had no intersection. I would >>>>>>>> propose to use R2 for Rklass (same with Rrecv) and move >>>>>>>> load_klass call after invokevirtual_helper call to avoid life >>>>>>>> range intersecton. >>>>>>>> >>>>>>>> See my patch against original templateTable_arm.cpp version in >>>>>>>> attach - with this change ARM build compiles and passes most of >>>>>>>> runtime/Nestmates tests Ok. >>>>>>>> >>>>>>>> regards, >>>>>>>> Boris >>>>>>>> >>>>>>>> On 15.05.2018 03:52, David Holmes wrote: >>>>>>>>> This review is being spread across four groups: langtools, >>>>>>>>> core-libs, hotspot and serviceability. This is the specific >>>>>>>>> review thread for hotspot - webrev: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>>>>>>> >>>>>>>>> >>>>>>>>> See below for full details - including annotated full webrev >>>>>>>>> guiding the review. >>>>>>>>> >>>>>>>>> The intent is to have JEP-181 targeted and integrated by the >>>>>>>>> end of this month. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> David >>>>>>>>> ----- >>>>>>>>> >>>>>>>>> The nestmates project (JEP-181) introduces new classfile >>>>>>>>> attributes to identify classes and interfaces in the same nest, >>>>>>>>> so that the VM can perform access control based on those >>>>>>>>> attributes and so allow direct private access between nestmates >>>>>>>>> without requiring javac to generate synthetic accessor methods. >>>>>>>>> These access control changes also extend to core reflection and >>>>>>>>> the MethodHandle.Lookup contexts. >>>>>>>>> >>>>>>>>> Direct private calls between nestmates requires a more general >>>>>>>>> calling context than is permitted by invokespecial, and so the >>>>>>>>> JVMS is updated to allow, and javac updated to use, >>>>>>>>> invokevirtual and invokeinterface for private class and >>>>>>>>> interface method calls respectively. These changed semantics >>>>>>>>> also extend to MethodHandle findXXX operations. >>>>>>>>> >>>>>>>>> At this time we are only concerned with static nest >>>>>>>>> definitions, which map to a top-level class/interface as the >>>>>>>>> nest-host and all its nested types as nest-members. >>>>>>>>> >>>>>>>>> Please see the JEP for further details. >>>>>>>>> >>>>>>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>>>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>>>>>>> >>>>>>>>> All of the specification changes have been previously been >>>>>>>>> worked out by the Valhalla Project Expert Group, and the >>>>>>>>> implementation reviewed by the various contributors and >>>>>>>>> discussed on the valhalla-dev mailing list. >>>>>>>>> >>>>>>>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>>>>>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, >>>>>>>>> Karen Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei >>>>>>>>> Spitsyn, Kumar Srinivasan >>>>>>>>> >>>>>>>>> Master webrev of all changes: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>>>>>>> >>>>>>>>> Annotated master webrev index: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>>>>>>> >>>>>>>>> >>>>>>>>> Performance: this is expected to be performance neutral in a >>>>>>>>> general sense. Benchmarking and performance runs are about to >>>>>>>>> start. >>>>>>>>> >>>>>>>>> Testing Discussion: >>>>>>>>> ------------------ >>>>>>>>> >>>>>>>>> The testing for nestmates can be broken into four main groups: >>>>>>>>> >>>>>>>>> -? New tests specifically related to nestmates and currently in >>>>>>>>> the runtime/Nestmates directory >>>>>>>>> >>>>>>>>> - New tests to complement existing tests by adding in testcases >>>>>>>>> not previously expressible. >>>>>>>>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java >>>>>>>>> tests use of invokespecial for private interface methods and >>>>>>>>> performing receiver typechecks, so we add >>>>>>>>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests >>>>>>>>> for invokeinterface. >>>>>>>>> >>>>>>>>> -? New JVM TI tests to verify the spec changes related to nest >>>>>>>>> attributes. >>>>>>>>> >>>>>>>>> -? Existing tests significantly affected by the nestmates >>>>>>>>> changes, primarily: >>>>>>>>> ??? -? runtime/SelectionResolution >>>>>>>>> >>>>>>>>> ??? In most cases the nestmate changes makes certain >>>>>>>>> invocations that were illegal, legal (e.g. not requiring >>>>>>>>> invokespecial to invoke private interface methods; allowing >>>>>>>>> access to private members via reflection/Methodhandles that >>>>>>>>> were previously not allowed). >>>>>>>>> >>>>>>>>> - Existing tests incidentally affected by the nestmate changes >>>>>>>>> >>>>>>>>> ?? This includes tests of things utilising class >>>>>>>>> redefinition/retransformation to alter nested types but which >>>>>>>>> unintentionally alter nest relationships (which is not permitted). >>>>>>>>> >>>>>>>>> There are still a number of tests problem-listed with issues >>>>>>>>> filed against them to have them adapted to work with nestmates. >>>>>>>>> Some of these are intended to be addressed in the short-term, >>>>>>>>> while some (such as the runtime/SelectionResolution test >>>>>>>>> changes) may not eventuate. >>>>>>>>> >>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>>>>>>> >>>>>>>>> There is also further test work still to be completed (the JNI >>>>>>>>> and JDI invocation tests): >>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>>>>>>> which will continue in parallel with the main RFR. >>>>>>>>> >>>>>>>>> Pre-integration Testing: >>>>>>>>> ??- General: >>>>>>>>> ???? - Mach5: hs/jdk tier1,2 >>>>>>>>> ???? - Mach5: hs-nightly (tiers 1 -3) >>>>>>>>> ??- Targetted >>>>>>>>> ??? - nashorn (for asm changes) >>>>>>>>> ??? - hotspot: runtime/* >>>>>>>>> ?????????????? serviceability/* >>>>>>>>> ?????????????? compiler/* >>>>>>>>> ?????????????? vmTestbase/* >>>>>>>>> ??? - jdk: java/lang/invoke/* >>>>>>>>> ?????????? java/lang/reflect/* >>>>>>>>> ?????????? java/lang/instrument/* >>>>>>>>> ?????????? java/lang/Class/* >>>>>>>>> ?????????? java/lang/management/* >>>>>>>>> ?? - langtools: tools/javac >>>>>>>>> ??????????????? tools/javap >>>>>>>>> From vladimir.kozlov at oracle.com Tue May 29 16:14:38 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 29 May 2018 09:14:38 -0700 Subject: RFR: 8203923: Add @requires feature to check flag values for the running JVM In-Reply-To: References: Message-ID: <35f1c0e4-8eac-746e-0869-4d3d88e888ee@oracle.com> Looks good. When you do tests changes you need to filter out running Graal with ZGC because it does not support it yet. Vladimir On 5/29/18 7:21 AM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to add @requires vm.opt.final., > as a way to filter jtreg tests on the final value of flags in the test VM. > > http://cr.openjdk.java.net/~stefank/8203923/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8203923 > > This patch is needed to allow tests to be filtered out when ZGC is run. > > ZGC doesn't support UseCompressedOops (and currently doesn't support > ClassUnloading). Instead of adding @requires vm.gc != "Z" for tests that > rely on ClassUnloading, I think it's nicer to write @requires > vm.opt.final.ClassUnloading. > > This will filter out the test if any of the following conditions are met: > 1) The currently executing VM doesn't support ClassUnloading > 2) -XX:-ClassUnloading is passed to the test > > The second case can today be handled by @requires vm.opt.ClassUnloading > == NULL | vm.opt.ClassUnloading == true, or maybe simpler @requires > vm.opt.ClassUnloading != false, but if you need both @requires > vm.opt.final.ClassUnloading should be used. > > The current solution requires the queried flags to be listed in > vmOptFinalFlags. Maybe there's a way to get this to cover all flags > automatically? > > Thanks, > StefanK From stefan.karlsson at oracle.com Tue May 29 17:10:18 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 29 May 2018 19:10:18 +0200 Subject: RFR: 8203923: Add @requires feature to check flag values for the running JVM In-Reply-To: <35f1c0e4-8eac-746e-0869-4d3d88e888ee@oracle.com> References: <35f1c0e4-8eac-746e-0869-4d3d88e888ee@oracle.com> Message-ID: On 2018-05-29 18:14, Vladimir Kozlov wrote: > Looks good. Thanks for reviewing. > > When you do tests changes you need to filter out running Graal with > ZGC because it does not support it yet. Yes. Thanks, StefanK > > Vladimir > > On 5/29/18 7:21 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to add @requires vm.opt.final.> name>, as a way to filter jtreg tests on the final value of flags in >> the test VM. >> >> http://cr.openjdk.java.net/~stefank/8203923/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8203923 >> >> This patch is needed to allow tests to be filtered out when ZGC is run. >> >> ZGC doesn't support UseCompressedOops (and currently doesn't support >> ClassUnloading). Instead of adding @requires vm.gc != "Z" for tests >> that rely on ClassUnloading, I think it's nicer to write @requires >> vm.opt.final.ClassUnloading. >> >> This will filter out the test if any of the following conditions are >> met: >> 1) The currently executing VM doesn't support ClassUnloading >> 2) -XX:-ClassUnloading is passed to the test >> >> The second case can today be handled by @requires >> vm.opt.ClassUnloading == NULL | vm.opt.ClassUnloading == true, or >> maybe simpler @requires vm.opt.ClassUnloading != false, but if you >> need both @requires vm.opt.final.ClassUnloading should be used. >> >> The current solution requires the queried flags to be listed in >> vmOptFinalFlags. Maybe there's a way to get this to cover all flags >> automatically? >> >> Thanks, >> StefanK From kim.barrett at oracle.com Tue May 29 17:37:22 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 29 May 2018 13:37:22 -0400 Subject: RFR: 8203923: Add @requires feature to check flag values for the running JVM In-Reply-To: References: Message-ID: <3B15B6F3-C06E-4334-AB4F-BCE31413A0FF@oracle.com> > On May 29, 2018, at 10:21 AM, Stefan Karlsson wrote: > > Hi all, > > Please review this patch to add @requires vm.opt.final., as a way to filter jtreg tests on the final value of flags in the test VM. > > http://cr.openjdk.java.net/~stefank/8203923/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8203923 > > This patch is needed to allow tests to be filtered out when ZGC is run. > > ZGC doesn't support UseCompressedOops (and currently doesn't support ClassUnloading). Instead of adding @requires vm.gc != "Z" for tests that rely on ClassUnloading, I think it's nicer to write @requires vm.opt.final.ClassUnloading. > > This will filter out the test if any of the following conditions are met: > 1) The currently executing VM doesn't support ClassUnloading > 2) -XX:-ClassUnloading is passed to the test > > The second case can today be handled by @requires vm.opt.ClassUnloading == NULL | vm.opt.ClassUnloading == true, or maybe simpler @requires vm.opt.ClassUnloading != false, but if you need both @requires vm.opt.final.ClassUnloading should be used. > > The current solution requires the queried flags to be listed in vmOptFinalFlags. Maybe there's a way to get this to cover all flags automatically? > > Thanks, > StefanK Looks good. From kim.barrett at oracle.com Tue May 29 17:41:59 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 29 May 2018 13:41:59 -0400 Subject: RFR (XS) 8189766: whitebox failure with -Xcheck:jni In-Reply-To: References: Message-ID: > On May 29, 2018, at 3:06 AM, David Holmes wrote: > > bug: https://bugs.openjdk.java.net/browse/JDK-8189766 > webrev: http://cr.openjdk.java.net/~dholmes/8189766/webrev/ > > [?] > Thanks, > David Looks good. From stefan.karlsson at oracle.com Tue May 29 17:49:20 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Tue, 29 May 2018 19:49:20 +0200 Subject: RFR: 8203923: Add @requires feature to check flag values for the running JVM In-Reply-To: <3B15B6F3-C06E-4334-AB4F-BCE31413A0FF@oracle.com> References: <3B15B6F3-C06E-4334-AB4F-BCE31413A0FF@oracle.com> Message-ID: <5d70d8b7-b87e-f280-721a-91352e4a1012@oracle.com> Thanks for the review, Kim. StefanK On 2018-05-29 19:37, Kim Barrett wrote: >> On May 29, 2018, at 10:21 AM, Stefan Karlsson wrote: >> >> Hi all, >> >> Please review this patch to add @requires vm.opt.final., as a way to filter jtreg tests on the final value of flags in the test VM. >> >> http://cr.openjdk.java.net/~stefank/8203923/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8203923 >> >> This patch is needed to allow tests to be filtered out when ZGC is run. >> >> ZGC doesn't support UseCompressedOops (and currently doesn't support ClassUnloading). Instead of adding @requires vm.gc != "Z" for tests that rely on ClassUnloading, I think it's nicer to write @requires vm.opt.final.ClassUnloading. >> >> This will filter out the test if any of the following conditions are met: >> 1) The currently executing VM doesn't support ClassUnloading >> 2) -XX:-ClassUnloading is passed to the test >> >> The second case can today be handled by @requires vm.opt.ClassUnloading == NULL | vm.opt.ClassUnloading == true, or maybe simpler @requires vm.opt.ClassUnloading != false, but if you need both @requires vm.opt.final.ClassUnloading should be used. >> >> The current solution requires the queried flags to be listed in vmOptFinalFlags. Maybe there's a way to get this to cover all flags automatically? >> >> Thanks, >> StefanK > Looks good. > From kim.barrett at oracle.com Tue May 29 18:31:09 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 29 May 2018 14:31:09 -0400 Subject: RFR(M): 8154736: enhancement of cmpxchg and copy_to_survivor for ppc64 In-Reply-To: <3e05cc86c9d4406d8a9875b705fbf1fc@sap.com> References: <5625a595-1165-8d48-afbd-8229cdc4ac07@oracle.com> <7e7fc484287e4da4926176e0b5ae1b64@sap.com> <339D50B6-09D5-4342-A687-918A9A096B39@oracle.com> <6D2268EC-C1E7-4C90-BCD3-90D02D21FA08@oracle.com> <3e05cc86c9d4406d8a9875b705fbf1fc@sap.com> Message-ID: > On May 29, 2018, at 6:30 AM, Doerr, Martin wrote: > > Hi Kim, > > I'm trying to understand how this is related to Michihiro's change. The else path of the initial test is not affected by it AFAICS. The value being tested and the forwardee obtained in the else branch are the result of some earlier CAS, e.g. some CAS (in this thread or some other) has already won the race and the result is seen here. Weakening the CAS fencing applies to that earlier CAS too, and so affects this code path. From david.holmes at oracle.com Tue May 29 20:52:56 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 30 May 2018 06:52:56 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <53feed71-0ab9-7743-6034-3c95bb3b20e8@bell-sw.com> <848b45e0-df51-7d74-e4eb-60819d101330@oracle.com> <9a8eadf0-e745-ee38-fb37-b29d99fa96c0@bell-sw.com> <71dbee89-451e-a151-474a-74326a2fa1c6@oracle.com> Message-ID: <61ded1ef-b713-d200-324c-6efe5cc981d8@oracle.com> Hi Boris, On 30/05/2018 1:46 AM, Boris Ulasevich wrote: > Hi David, > > ARM32 part of the change looks good for me. > Do you want me to apply latest (v4-incr?) patch and run tests? Please do. Thanks, David > Boris > > On 24.05.2018 00:39, David Holmes wrote: >> Hi Boris, >> >> Thanks for verifying my changes. In case you didn't see you can apply >> the updated patch to my v1 changes from: >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2/ >> >> One follow up below. >> >> On 23/05/2018 11:40 PM, Boris Ulasevich wrote: >>> Hi David, >>> >>> some minor comments below.. >>> >>> On 23.05.2018 09:57, David Holmes wrote: >>>> Hi Boris, >>>> >>>> On 17/05/2018 7:23 PM, Boris Ulasevich wrote: >>>>> Hi David, >>>>> >>>>> You are right! My bad, R2_tmp parameter conflicts with Rklass on >>>>> check_klass_subtype(..) call. See correct patch in attach. Now all >>>>> runtime/Nestmates tests passed! :) >>>> >>>> I went to aplpy your patch and found it's not a diff against my >>>> patch but against the original non-nestmate code, >>> >>> Yes, sorry. For me it was natural to apply patch from your review to >>> local repo, rework it and just send updated diff from my machine :) >>> >>>> so I want to be clear on the changes. AFAICS the differences are: >>>> >>>> -? const Register Rklass? = R3_tmp; >>>> +? const Register Rklass? = R2_tmp; // Note! Same register with Rrecv >>> >>> Yes. >>> >>>> This initially concerned me as we stomp on Rrecv when we do the >>>> load_klass, but then you moved? the: >>>> >>>> ??? __ load_klass(Rklass, Rrecv); >>>> >>>> to after the object case, which used Rrecv. I had assumed Rrecv was >>>> somehow used when we actually do the call, but I'm assuming >>>> jump_from_interpreted is done in such as way that the receiver is >>>> still available on the interpreter stack and is used from there. >>> >>> Ok. I'm not sure I understand you well. When I moved load_klass call >>> down my point was to skip unnecessary load for the notObjectMethod >>> case (Rklass is not required in this case) and to make possible to >>> reuse same R2 register by both Rklass and Rrecv register. >> >> Right. My concern was an assumption that as Rrecv was the receiver >> object that we had to leave it intact ready for the actual method >> invocation. But that isn't the case. The real method invocation >> happens elsewhere and the receiver is obtained by other means. >> >> Thanks, >> David >> >>>> -? __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >>>> +? __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R3_tmp, noreg, >>>> subtype); >>>> >>>> Okay - reworking of tmp regs. >>> >>> Yes, ARM32 requires one more tmp reg, and we can't spoil R0. >>> >>>> -? __ jump_from_interpreted(Rindex); >>>> +? __ jump_from_interpreted(Rmethod); >>>> >>>> I'm going to trust this is okay :) It's not at all clear to me how >>>> the f2 Method* gets passed through on different platforms - >>>> sometimes its in the "index" and sometimes the "method" registers. >>> >>> I see. On ARM/AARCH64 we have preallocated register Rmethod/rmethod >>> to hold current method pointer and prepare_invoke call to setup >>> registers properly. >>> >>>> (I _really_ wish there was consistency in terminology across the >>>> different platforms - this code is awful for trying to compare the >>>> different platforms to figure out what to do on a new one.) >>>> >>>> Thanks, >>>> David >>>> >>>>> regards, >>>>> Boris >>>>> >>>>> On 17.05.2018 11:24, David Holmes wrote: >>>>>> On 17/05/2018 6:13 PM, Boris Ulasevich wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> I see three tests failing: >>>>>>> ?> NullPointerException at >>>>>>> TestInterfaceMethodSelection.doInvoke(TestInterfaceMethodSelection.java:191) >>>>>>> >>>>>>> ?> NullPointerException at >>>>>>> TestInvoke.access_priv(TestInvoke.java:54) >>>>>>> ?> InvocationTargetException at >>>>>>> TestReflection.access_priv(TestReflection.java:61) >>>>>>> >>>>>>> I will send you test details in a separate mail. >>>>>> >>>>>> Ok. This indicates a bug in the assembly code. The NPE's will >>>>>> likely be SEGVs caused by a zero register. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>> >>>>>>> Boris >>>>>>> >>>>>>> On 17.05.2018 00:23, David Holmes wrote: >>>>>>>> Hi Boris, >>>>>>>> >>>>>>>> Many thanks for looking at this and working through the ARM code. >>>>>>>> >>>>>>>> I will study your patch in detail but am concerned by the >>>>>>>> "passes most of runtime/Nestmates tests Ok."! What tests are not >>>>>>>> passing? >>>>>>>> >>>>>>>> David >>>>>>>> >>>>>>>> On 17/05/2018 1:05 AM, Boris Ulasevich wrote: >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> Let us look to the change in templateTable_arm.cpp. I have >>>>>>>>> several notes. >>>>>>>>> >>>>>>>>> 1. We have compilation error because check_klass_subtype call >>>>>>>>> needs one more temporary register parameter. I think correct >>>>>>>>> change is: >>>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >>>>>>>>> -> >>>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, >>>>>>>>> subtype); >>>>>>>>> >>>>>>>>> 2. R0_tmp holds mdp used for profilation several lines below, >>>>>>>>> so we can't spoil it. I think correct change is: >>>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, >>>>>>>>> subtype); >>>>>>>>> -> >>>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R2_tmp, noreg, >>>>>>>>> subtype); >>>>>>>>> >>>>>>>>> 3. We can't jump to Rindex. I believe it was supposed to jump >>>>>>>>> to Rmethod: >>>>>>>>> jump_from_interpreted(Rindex); >>>>>>>>> -> >>>>>>>>> jump_from_interpreted(Rmethod); >>>>>>>>> >>>>>>>>> 4. Please note than Rklass and Rflags reuses same register. >>>>>>>>> Before the change their life ranges had no intersection. I >>>>>>>>> would propose to use R2 for Rklass (same with Rrecv) and move >>>>>>>>> load_klass call after invokevirtual_helper call to avoid life >>>>>>>>> range intersecton. >>>>>>>>> >>>>>>>>> See my patch against original templateTable_arm.cpp version in >>>>>>>>> attach - with this change ARM build compiles and passes most of >>>>>>>>> runtime/Nestmates tests Ok. >>>>>>>>> >>>>>>>>> regards, >>>>>>>>> Boris >>>>>>>>> >>>>>>>>> On 15.05.2018 03:52, David Holmes wrote: >>>>>>>>>> This review is being spread across four groups: langtools, >>>>>>>>>> core-libs, hotspot and serviceability. This is the specific >>>>>>>>>> review thread for hotspot - webrev: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> See below for full details - including annotated full webrev >>>>>>>>>> guiding the review. >>>>>>>>>> >>>>>>>>>> The intent is to have JEP-181 targeted and integrated by the >>>>>>>>>> end of this month. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> David >>>>>>>>>> ----- >>>>>>>>>> >>>>>>>>>> The nestmates project (JEP-181) introduces new classfile >>>>>>>>>> attributes to identify classes and interfaces in the same >>>>>>>>>> nest, so that the VM can perform access control based on those >>>>>>>>>> attributes and so allow direct private access between >>>>>>>>>> nestmates without requiring javac to generate synthetic >>>>>>>>>> accessor methods. These access control changes also extend to >>>>>>>>>> core reflection and the MethodHandle.Lookup contexts. >>>>>>>>>> >>>>>>>>>> Direct private calls between nestmates requires a more general >>>>>>>>>> calling context than is permitted by invokespecial, and so the >>>>>>>>>> JVMS is updated to allow, and javac updated to use, >>>>>>>>>> invokevirtual and invokeinterface for private class and >>>>>>>>>> interface method calls respectively. These changed semantics >>>>>>>>>> also extend to MethodHandle findXXX operations. >>>>>>>>>> >>>>>>>>>> At this time we are only concerned with static nest >>>>>>>>>> definitions, which map to a top-level class/interface as the >>>>>>>>>> nest-host and all its nested types as nest-members. >>>>>>>>>> >>>>>>>>>> Please see the JEP for further details. >>>>>>>>>> >>>>>>>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>>>>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>>>>>>>> >>>>>>>>>> All of the specification changes have been previously been >>>>>>>>>> worked out by the Valhalla Project Expert Group, and the >>>>>>>>>> implementation reviewed by the various contributors and >>>>>>>>>> discussed on the valhalla-dev mailing list. >>>>>>>>>> >>>>>>>>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>>>>>>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, >>>>>>>>>> Karen Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei >>>>>>>>>> Spitsyn, Kumar Srinivasan >>>>>>>>>> >>>>>>>>>> Master webrev of all changes: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Annotated master webrev index: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Performance: this is expected to be performance neutral in a >>>>>>>>>> general sense. Benchmarking and performance runs are about to >>>>>>>>>> start. >>>>>>>>>> >>>>>>>>>> Testing Discussion: >>>>>>>>>> ------------------ >>>>>>>>>> >>>>>>>>>> The testing for nestmates can be broken into four main groups: >>>>>>>>>> >>>>>>>>>> -? New tests specifically related to nestmates and currently >>>>>>>>>> in the runtime/Nestmates directory >>>>>>>>>> >>>>>>>>>> - New tests to complement existing tests by adding in >>>>>>>>>> testcases not previously expressible. >>>>>>>>>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java >>>>>>>>>> tests use of invokespecial for private interface methods and >>>>>>>>>> performing receiver typechecks, so we add >>>>>>>>>> java/lang/invoke/PrivateInterfaceCall.java to do similar tests >>>>>>>>>> for invokeinterface. >>>>>>>>>> >>>>>>>>>> -? New JVM TI tests to verify the spec changes related to nest >>>>>>>>>> attributes. >>>>>>>>>> >>>>>>>>>> -? Existing tests significantly affected by the nestmates >>>>>>>>>> changes, primarily: >>>>>>>>>> ??? -? runtime/SelectionResolution >>>>>>>>>> >>>>>>>>>> ??? In most cases the nestmate changes makes certain >>>>>>>>>> invocations that were illegal, legal (e.g. not requiring >>>>>>>>>> invokespecial to invoke private interface methods; allowing >>>>>>>>>> access to private members via reflection/Methodhandles that >>>>>>>>>> were previously not allowed). >>>>>>>>>> >>>>>>>>>> - Existing tests incidentally affected by the nestmate changes >>>>>>>>>> >>>>>>>>>> ?? This includes tests of things utilising class >>>>>>>>>> redefinition/retransformation to alter nested types but which >>>>>>>>>> unintentionally alter nest relationships (which is not >>>>>>>>>> permitted). >>>>>>>>>> >>>>>>>>>> There are still a number of tests problem-listed with issues >>>>>>>>>> filed against them to have them adapted to work with >>>>>>>>>> nestmates. Some of these are intended to be addressed in the >>>>>>>>>> short-term, while some (such as the >>>>>>>>>> runtime/SelectionResolution test changes) may not eventuate. >>>>>>>>>> >>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>>>>>>>> >>>>>>>>>> There is also further test work still to be completed (the JNI >>>>>>>>>> and JDI invocation tests): >>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>>>>>>>> which will continue in parallel with the main RFR. >>>>>>>>>> >>>>>>>>>> Pre-integration Testing: >>>>>>>>>> ??- General: >>>>>>>>>> ???? - Mach5: hs/jdk tier1,2 >>>>>>>>>> ???? - Mach5: hs-nightly (tiers 1 -3) >>>>>>>>>> ??- Targetted >>>>>>>>>> ??? - nashorn (for asm changes) >>>>>>>>>> ??? - hotspot: runtime/* >>>>>>>>>> ?????????????? serviceability/* >>>>>>>>>> ?????????????? compiler/* >>>>>>>>>> ?????????????? vmTestbase/* >>>>>>>>>> ??? - jdk: java/lang/invoke/* >>>>>>>>>> ?????????? java/lang/reflect/* >>>>>>>>>> ?????????? java/lang/instrument/* >>>>>>>>>> ?????????? java/lang/Class/* >>>>>>>>>> ?????????? java/lang/management/* >>>>>>>>>> ?? - langtools: tools/javac >>>>>>>>>> ??????????????? tools/javap >>>>>>>>>> From karen.kinnear at oracle.com Tue May 29 21:13:27 2018 From: karen.kinnear at oracle.com (Karen Kinnear) Date: Tue, 29 May 2018 17:13:27 -0400 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <97f8cedf-4ebc-610f-0528-e1b91f35eece@oracle.com> References: <06529fc3-2eba-101b-9aee-2757893cb8fb@oracle.com> <97f8cedf-4ebc-610f-0528-e1b91f35eece@oracle.com> Message-ID: Looks good. Thank you for the detailed JNI tests. thanks, Karen > On May 28, 2018, at 7:20 AM, David Holmes wrote: > > I've added some missing JNI tests for the basic access checks. Given JNI ignores access it should go without saying that JNI access to nestmates will work fine, but it doesn't hurt to verify that. > > http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v4-incr/ > > Thanks, > David > > On 24/05/2018 7:48 PM, David Holmes wrote: >> Here are the further updates based on review comments and rebasing to get the vmTestbase updates for which some closed test changes now have to be applied to the open versions. >> Incremental hotspot webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3-incr/ >> Full hotspot webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3/ >> Change summary: >> test/hotspot/jtreg/ProblemList.txt >> - Exclude vmTestbase/nsk/stress/except/except004.java under 8203046 >> test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/BasicTest.java >> test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/PrivateMethodsTest.java >> - updated to work with new invokeinterface rules and nestmate changes >> - misc cleanups >> src/hotspot/share/runtime/reflection.?pp >> - rename verify_field_access to verify_member_access (it's always been mis-named and I nearly forgot to do this cleanup!) and rename field_class to member_class >> - add TRAPS to verify_member_access to allow use with CHECK macros >> src/hotspot/share/ci/ciField.cpp >> src/hotspot/share/classfile/classFileParser.cpp >> src/hotspot/share/interpreter/linkResolver.cpp >> - updated to use THREAD/CHECK with verify_member_access >> - for ciField rename thread to THREAD so it can be used with HAS_PENDING_EXCEPTION >> src/hotspot/share/oops/instanceKlass.cpp >> - use CHECK_false when calling nest_host() >> - fix indent near nestmate code >> src/hotspot/share/oops/instanceKlass.hpp >> - make has_nest_member private >> Thanks, >> David >> ----- >> On 23/05/2018 4:57 PM, David Holmes wrote: >>> Here are the updates so far in response to all the review comments. >>> >>> Incremental webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2-incr/ >>> >>> Full webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2/ >>> >>> Change summary: >>> >>> test/runtime/Nestmates/reflectionAPI/* >>> - moved to java/lang/reflect/Nestmates >>> >>> src/hotspot/cpu/arm/templateTable_arm.cpp >>> - Fixed ARM invocation logic as provided by Boris. >>> >>> src/hotspot/share/interpreter/linkResolver.cpp >>> - expanded comment regarding exceptions >>> - Removed leftover debugging code >>> >>> src/hotspot/share/oops/instanceKlass.cpp >>> - Removed FIXME comments >>> - corrected incorrect comment >>> - Fixed if/else formatting >>> >>> src/hotspot/share/oops/instanceKlass.hpp >>> - removed unused debug method >>> >>> src/hotspot/share/oops/klassVtable.cpp >>> - added comment by request of Karen >>> >>> src/hotspot/share/runtime/reflection.cpp >>> - Removed FIXME comments >>> - expanded comments in places >>> - used CHECK_false >>> >>> Thanks, >>> David >>> >>> On 15/05/2018 10:52 AM, David Holmes wrote: >>>> This review is being spread across four groups: langtools, core-libs, hotspot and serviceability. This is the specific review thread for hotspot - webrev: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>> >>>> See below for full details - including annotated full webrev guiding the review. >>>> >>>> The intent is to have JEP-181 targeted and integrated by the end of this month. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> The nestmates project (JEP-181) introduces new classfile attributes to identify classes and interfaces in the same nest, so that the VM can perform access control based on those attributes and so allow direct private access between nestmates without requiring javac to generate synthetic accessor methods. These access control changes also extend to core reflection and the MethodHandle.Lookup contexts. >>>> >>>> Direct private calls between nestmates requires a more general calling context than is permitted by invokespecial, and so the JVMS is updated to allow, and javac updated to use, invokevirtual and invokeinterface for private class and interface method calls respectively. These changed semantics also extend to MethodHandle findXXX operations. >>>> >>>> At this time we are only concerned with static nest definitions, which map to a top-level class/interface as the nest-host and all its nested types as nest-members. >>>> >>>> Please see the JEP for further details. >>>> >>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>> >>>> All of the specification changes have been previously been worked out by the Valhalla Project Expert Group, and the implementation reviewed by the various contributors and discussed on the valhalla-dev mailing list. >>>> >>>> Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan >>>> >>>> Master webrev of all changes: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>> >>>> Annotated master webrev index: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>> >>>> Performance: this is expected to be performance neutral in a general sense. Benchmarking and performance runs are about to start. >>>> >>>> Testing Discussion: >>>> ------------------ >>>> >>>> The testing for nestmates can be broken into four main groups: >>>> >>>> - New tests specifically related to nestmates and currently in the runtime/Nestmates directory >>>> >>>> - New tests to complement existing tests by adding in testcases not previously expressible. >>>> - For example java/lang/invoke/SpecialInterfaceCall.java tests use of invokespecial for private interface methods and performing receiver typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do similar tests for invokeinterface. >>>> >>>> - New JVM TI tests to verify the spec changes related to nest attributes. >>>> >>>> - Existing tests significantly affected by the nestmates changes, primarily: >>>> - runtime/SelectionResolution >>>> >>>> In most cases the nestmate changes makes certain invocations that were illegal, legal (e.g. not requiring invokespecial to invoke private interface methods; allowing access to private members via reflection/Methodhandles that were previously not allowed). >>>> >>>> - Existing tests incidentally affected by the nestmate changes >>>> >>>> This includes tests of things utilising class redefinition/retransformation to alter nested types but which unintentionally alter nest relationships (which is not permitted). >>>> >>>> There are still a number of tests problem-listed with issues filed against them to have them adapted to work with nestmates. Some of these are intended to be addressed in the short-term, while some (such as the runtime/SelectionResolution test changes) may not eventuate. >>>> >>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>> >>>> There is also further test work still to be completed (the JNI and JDI invocation tests): >>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>> which will continue in parallel with the main RFR. >>>> >>>> Pre-integration Testing: >>>> - General: >>>> - Mach5: hs/jdk tier1,2 >>>> - Mach5: hs-nightly (tiers 1 -3) >>>> - Targetted >>>> - nashorn (for asm changes) >>>> - hotspot: runtime/* >>>> serviceability/* >>>> compiler/* >>>> vmTestbase/* >>>> - jdk: java/lang/invoke/* >>>> java/lang/reflect/* >>>> java/lang/instrument/* >>>> java/lang/Class/* >>>> java/lang/management/* >>>> - langtools: tools/javac >>>> tools/javap >>>> From david.holmes at oracle.com Tue May 29 21:13:27 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 30 May 2018 07:13:27 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <06529fc3-2eba-101b-9aee-2757893cb8fb@oracle.com> <97f8cedf-4ebc-610f-0528-e1b91f35eece@oracle.com> Message-ID: <7aa56a5e-0c95-f1a2-0b27-3f759ca5c702@oracle.com> Thanks Karen! David On 30/05/2018 7:13 AM, Karen Kinnear wrote: > Looks good. Thank you for the detailed JNI tests. > > thanks, > Karen > > >> On May 28, 2018, at 7:20 AM, David Holmes wrote: >> >> I've added some missing JNI tests for the basic access checks. Given JNI ignores access it should go without saying that JNI access to nestmates will work fine, but it doesn't hurt to verify that. >> >> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v4-incr/ >> >> Thanks, >> David >> >> On 24/05/2018 7:48 PM, David Holmes wrote: >>> Here are the further updates based on review comments and rebasing to get the vmTestbase updates for which some closed test changes now have to be applied to the open versions. >>> Incremental hotspot webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3-incr/ >>> Full hotspot webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v3/ >>> Change summary: >>> test/hotspot/jtreg/ProblemList.txt >>> - Exclude vmTestbase/nsk/stress/except/except004.java under 8203046 >>> test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/BasicTest.java >>> test/hotspot/jtreg/vmTestbase/vm/runtime/defmeth/PrivateMethodsTest.java >>> - updated to work with new invokeinterface rules and nestmate changes >>> - misc cleanups >>> src/hotspot/share/runtime/reflection.?pp >>> - rename verify_field_access to verify_member_access (it's always been mis-named and I nearly forgot to do this cleanup!) and rename field_class to member_class >>> - add TRAPS to verify_member_access to allow use with CHECK macros >>> src/hotspot/share/ci/ciField.cpp >>> src/hotspot/share/classfile/classFileParser.cpp >>> src/hotspot/share/interpreter/linkResolver.cpp >>> - updated to use THREAD/CHECK with verify_member_access >>> - for ciField rename thread to THREAD so it can be used with HAS_PENDING_EXCEPTION >>> src/hotspot/share/oops/instanceKlass.cpp >>> - use CHECK_false when calling nest_host() >>> - fix indent near nestmate code >>> src/hotspot/share/oops/instanceKlass.hpp >>> - make has_nest_member private >>> Thanks, >>> David >>> ----- >>> On 23/05/2018 4:57 PM, David Holmes wrote: >>>> Here are the updates so far in response to all the review comments. >>>> >>>> Incremental webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2-incr/ >>>> >>>> Full webrev: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2/ >>>> >>>> Change summary: >>>> >>>> test/runtime/Nestmates/reflectionAPI/* >>>> - moved to java/lang/reflect/Nestmates >>>> >>>> src/hotspot/cpu/arm/templateTable_arm.cpp >>>> - Fixed ARM invocation logic as provided by Boris. >>>> >>>> src/hotspot/share/interpreter/linkResolver.cpp >>>> - expanded comment regarding exceptions >>>> - Removed leftover debugging code >>>> >>>> src/hotspot/share/oops/instanceKlass.cpp >>>> - Removed FIXME comments >>>> - corrected incorrect comment >>>> - Fixed if/else formatting >>>> >>>> src/hotspot/share/oops/instanceKlass.hpp >>>> - removed unused debug method >>>> >>>> src/hotspot/share/oops/klassVtable.cpp >>>> - added comment by request of Karen >>>> >>>> src/hotspot/share/runtime/reflection.cpp >>>> - Removed FIXME comments >>>> - expanded comments in places >>>> - used CHECK_false >>>> >>>> Thanks, >>>> David >>>> >>>> On 15/05/2018 10:52 AM, David Holmes wrote: >>>>> This review is being spread across four groups: langtools, core-libs, hotspot and serviceability. This is the specific review thread for hotspot - webrev: >>>>> >>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>>> >>>>> See below for full details - including annotated full webrev guiding the review. >>>>> >>>>> The intent is to have JEP-181 targeted and integrated by the end of this month. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>> The nestmates project (JEP-181) introduces new classfile attributes to identify classes and interfaces in the same nest, so that the VM can perform access control based on those attributes and so allow direct private access between nestmates without requiring javac to generate synthetic accessor methods. These access control changes also extend to core reflection and the MethodHandle.Lookup contexts. >>>>> >>>>> Direct private calls between nestmates requires a more general calling context than is permitted by invokespecial, and so the JVMS is updated to allow, and javac updated to use, invokevirtual and invokeinterface for private class and interface method calls respectively. These changed semantics also extend to MethodHandle findXXX operations. >>>>> >>>>> At this time we are only concerned with static nest definitions, which map to a top-level class/interface as the nest-host and all its nested types as nest-members. >>>>> >>>>> Please see the JEP for further details. >>>>> >>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>>> >>>>> All of the specification changes have been previously been worked out by the Valhalla Project Expert Group, and the implementation reviewed by the various contributors and discussed on the valhalla-dev mailing list. >>>>> >>>>> Acknowledgments and contributions: Alex Buckley, Maurizio Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, Karen Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei Spitsyn, Kumar Srinivasan >>>>> >>>>> Master webrev of all changes: >>>>> >>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>>> >>>>> Annotated master webrev index: >>>>> >>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>>> >>>>> Performance: this is expected to be performance neutral in a general sense. Benchmarking and performance runs are about to start. >>>>> >>>>> Testing Discussion: >>>>> ------------------ >>>>> >>>>> The testing for nestmates can be broken into four main groups: >>>>> >>>>> - New tests specifically related to nestmates and currently in the runtime/Nestmates directory >>>>> >>>>> - New tests to complement existing tests by adding in testcases not previously expressible. >>>>> - For example java/lang/invoke/SpecialInterfaceCall.java tests use of invokespecial for private interface methods and performing receiver typechecks, so we add java/lang/invoke/PrivateInterfaceCall.java to do similar tests for invokeinterface. >>>>> >>>>> - New JVM TI tests to verify the spec changes related to nest attributes. >>>>> >>>>> - Existing tests significantly affected by the nestmates changes, primarily: >>>>> - runtime/SelectionResolution >>>>> >>>>> In most cases the nestmate changes makes certain invocations that were illegal, legal (e.g. not requiring invokespecial to invoke private interface methods; allowing access to private members via reflection/Methodhandles that were previously not allowed). >>>>> >>>>> - Existing tests incidentally affected by the nestmate changes >>>>> >>>>> This includes tests of things utilising class redefinition/retransformation to alter nested types but which unintentionally alter nest relationships (which is not permitted). >>>>> >>>>> There are still a number of tests problem-listed with issues filed against them to have them adapted to work with nestmates. Some of these are intended to be addressed in the short-term, while some (such as the runtime/SelectionResolution test changes) may not eventuate. >>>>> >>>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>>> >>>>> There is also further test work still to be completed (the JNI and JDI invocation tests): >>>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>>> which will continue in parallel with the main RFR. >>>>> >>>>> Pre-integration Testing: >>>>> - General: >>>>> - Mach5: hs/jdk tier1,2 >>>>> - Mach5: hs-nightly (tiers 1 -3) >>>>> - Targetted >>>>> - nashorn (for asm changes) >>>>> - hotspot: runtime/* >>>>> serviceability/* >>>>> compiler/* >>>>> vmTestbase/* >>>>> - jdk: java/lang/invoke/* >>>>> java/lang/reflect/* >>>>> java/lang/instrument/* >>>>> java/lang/Class/* >>>>> java/lang/management/* >>>>> - langtools: tools/javac >>>>> tools/javap >>>>> > From david.holmes at oracle.com Tue May 29 21:17:30 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 30 May 2018 07:17:30 +1000 Subject: RFR (XS) 8189766: whitebox failure with -Xcheck:jni In-Reply-To: <51007b0e-da67-b459-6121-723cbae33376@oracle.com> References: <51007b0e-da67-b459-6121-723cbae33376@oracle.com> Message-ID: <48310b51-2a6e-121d-d8ce-f032f23ffe80@oracle.com> Thanks Lois! David On 30/05/2018 12:57 AM, Lois Foltan wrote: > Looks good. > Lois > > On 5/29/2018 3:06 AM, David Holmes wrote: >> bug: https://bugs.openjdk.java.net/browse/JDK-8189766 >> webrev: http://cr.openjdk.java.net/~dholmes/8189766/webrev/ >> >> A WB_ENTRY already handles the pending JNI exception check by >> explicitly clearing it via a helper: >> >> #define WB_ENTRY(result_type, header) JNI_ENTRY(result_type, header) \ >> ? ClearPendingJniExcCheck _clearCheck(env); >> >> #define WB_END JNI_END >> >> But the whitebox function then does e.g: >> >> WB_ENTRY(jobject, WB_GetBooleanVMFlag(JNIEnv* env, jobject o, jstring >> name)) >> ? bool result; >> ? if (GetVMFlag (thread, env, name, &result, &JVMFlag::boolAt)) { >> ??? ThreadToNativeFromVM ttnfv(thread); // can't be in VM when we call >> JNI >> ??? return booleanBox(thread, env, result); >> ? } >> ? return NULL; >> WB_END >> >> where booleanBox is defined eventually as: >> >> template >> static jobject box(JavaThread* thread, JNIEnv* env, Symbol* name, >> Symbol* sig, T value) { >> ? ResourceMark rm(thread); >> ? jclass clazz = env->FindClass(name->as_C_string()); >> ? CHECK_JNI_EXCEPTION_(env, NULL); >> ? jmethodID methodID = env->GetStaticMethodID(clazz, >> ??????? vmSymbols::valueOf_name()->as_C_string(), >> ??????? sig->as_C_string()); >> ? CHECK_JNI_EXCEPTION_(env, NULL); >> ? jobject result = env->CallStaticObjectMethod(clazz, methodID, value); >> ? CHECK_JNI_EXCEPTION_(env, NULL); >> ? return result; >> } >> >> so we call JNI methods that will set the pending_jni_exception_check >> flag, but CHECK_JNI_EXCEPTION is defined as: >> >> #define CHECK_JNI_EXCEPTION_(env, value) \ >> ? do { \ >> ??? JavaThread* THREAD = JavaThread::thread_from_jni_environment(env); \ >> ??? if (HAS_PENDING_EXCEPTION) { \ >> ????? return(value); \ >> ??? } \ >> ? } while (0) >> >> which means it is not clearing the pending_jni_exception_check flag >> even though it is checking for exceptions! >> >> So we explicitly clear the flag. Trying to use the JNI >> ExceptionOccurred function fails as not all the methods that call >> CHECK_JNI_EXCEPTION are _thread_in_native. >> >> Testing (in progress): >> ???????? hotspot tier1 and tier2 with -Xcheck:jni >> ???????? regular CI testing >> >> Thanks, >> David > From david.holmes at oracle.com Tue May 29 21:17:49 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 30 May 2018 07:17:49 +1000 Subject: RFR (XS) 8189766: whitebox failure with -Xcheck:jni In-Reply-To: References: Message-ID: <2f199175-73f5-c747-6f1c-5a3792acdf2d@oracle.com> Thanks Kim! David On 30/05/2018 3:41 AM, Kim Barrett wrote: >> On May 29, 2018, at 3:06 AM, David Holmes wrote: >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8189766 >> webrev: http://cr.openjdk.java.net/~dholmes/8189766/webrev/ >> >> [?] >> Thanks, >> David > > Looks good. > From david.holmes at oracle.com Tue May 29 21:53:10 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 30 May 2018 07:53:10 +1000 Subject: RFR: 8203923: Add @requires feature to check flag values for the running JVM In-Reply-To: References: Message-ID: <8e9694e9-1148-11bb-560d-5ae70936fb7e@oracle.com> Hi Stefan, On 30/05/2018 12:21 AM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to add @requires vm.opt.final., > as a way to filter jtreg tests on the final value of flags in the test VM. > > http://cr.openjdk.java.net/~stefank/8203923/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8203923 > > This patch is needed to allow tests to be filtered out when ZGC is run. > > ZGC doesn't support UseCompressedOops (and currently doesn't support > ClassUnloading). Instead of adding @requires vm.gc != "Z" for tests that > rely on ClassUnloading, I think it's nicer to write @requires > vm.opt.final.ClassUnloading. > > This will filter out the test if any of the following conditions are met: > 1) The currently executing VM doesn't support ClassUnloading > 2) -XX:-ClassUnloading is passed to the test > > The second case can today be handled by @requires vm.opt.ClassUnloading > == NULL | vm.opt.ClassUnloading == true, or maybe simpler @requires > vm.opt.ClassUnloading != false, but if you need both @requires > vm.opt.final.ClassUnloading should be used. I'm not sure about the use of vm.opt.final as the name here. IIUC vm.opt is set by jtreg based on the vmoptions passed to jtreg for running the tests. Whereas this is the final flag value as seen in the running VM. And given we can only inspect the final value this way, perhaps just vm.flag would be better? > The current solution requires the queried flags to be listed in > vmOptFinalFlags. Maybe there's a way to get this to cover all flags > automatically? Yes it is a shame that this has to be manually configured. I think ideally this would need to be built into jtreg so that it ran a dcmd(?) to get all final flag values and then populated the properties. Though I suppose VMProps could also do that ... (The crude way of doing this is via PrintFlagsFinal and parsing the output, of course). Anyway other than my query on the naming, this seems fine for moving ahead. Thanks, David > Thanks, > StefanK From stefan.karlsson at oracle.com Tue May 29 22:07:37 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 30 May 2018 00:07:37 +0200 Subject: RFR: 8203923: Add @requires feature to check flag values for the running JVM In-Reply-To: <8e9694e9-1148-11bb-560d-5ae70936fb7e@oracle.com> References: <8e9694e9-1148-11bb-560d-5ae70936fb7e@oracle.com> Message-ID: <58b03b15-9094-dfbf-3575-0c31f9af479a@oracle.com> Hi David, On 2018-05-29 23:53, David Holmes wrote: > Hi Stefan, > > On 30/05/2018 12:21 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to add @requires vm.opt.final.> name>, as a way to filter jtreg tests on the final value of flags in >> the test VM. >> >> http://cr.openjdk.java.net/~stefank/8203923/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8203923 >> >> This patch is needed to allow tests to be filtered out when ZGC is run. >> >> ZGC doesn't support UseCompressedOops (and currently doesn't support >> ClassUnloading). Instead of adding @requires vm.gc != "Z" for tests >> that rely on ClassUnloading, I think it's nicer to write @requires >> vm.opt.final.ClassUnloading. >> >> This will filter out the test if any of the following conditions are >> met: >> 1) The currently executing VM doesn't support ClassUnloading >> 2) -XX:-ClassUnloading is passed to the test >> >> The second case can today be handled by @requires >> vm.opt.ClassUnloading == NULL | vm.opt.ClassUnloading == true, or >> maybe simpler @requires vm.opt.ClassUnloading != false, but if you >> need both @requires vm.opt.final.ClassUnloading should be used. > > I'm not sure about the use of vm.opt.final as the name here. IIUC > vm.opt is set by jtreg based on the vmoptions passed to jtreg for > running the tests. Whereas this is the final flag value as seen in the > running VM. And given we can only inspect the final value this way, > perhaps just vm.flag would be better? One reason why I chose to use the vm.opt.final prefix, instead of some other prefix (like vm.flag), is that jtreg "conveniently" accepts any property prefixed with vm.opt. If I change the prefix to vm.flag I need to add every single vm.flag to the requires.properties in all TEST.ROOT files of the test directories where these properties are used (E.g. test/hotspot/jtreg/TEST.ROOT). See isValidName in: http://hg.openjdk.java.net/code-tools/jtreg/file/7b1496d2790e/src/share/classes/com/sun/javatest/regtest/config/RegressionContext.java#l109 I can change the name, but I think that would come with a maintenance cost. What do you think? > >> The current solution requires the queried flags to be listed in >> vmOptFinalFlags. Maybe there's a way to get this to cover all flags >> automatically? > > Yes it is a shame that this has to be manually configured. I think > ideally this would need to be built into jtreg so that it ran a > dcmd(?) to get all final flag values and then populated the > properties. Though I suppose VMProps could also do that ... (The crude > way of doing this is via PrintFlagsFinal and parsing the output, of > course). > > Anyway other than my query on the naming, this seems fine for moving > ahead. Thanks, StefanK > > Thanks, > David > >> Thanks, >> StefanK From igor.ignatyev at oracle.com Tue May 29 23:22:21 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 29 May 2018 16:22:21 -0700 Subject: RFR(L) : 8199371 : [TESTBUG] Open source vm testbase JDWP tests Message-ID: <967B07D4-A563-4D86-B82F-02A8043D4CDE@oracle.com> http://cr.openjdk.java.net/~iignatyev//8199371/webrev.00/index.html > 64675 lines changed: 64675 ins; 0 del; 0 mod; Hi all, could you please review this patch which open sources JDWP tests from VM testbase? As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. JBS: https://bugs.openjdk.java.net/browse/JDK-8199371 webrev: http://cr.openjdk.java.net/~iignatyev//8199371/webrev.00/index.html testing: :vmTestbase_vm_jdwp test group Thanks, -- Igor From david.holmes at oracle.com Tue May 29 23:44:53 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 30 May 2018 09:44:53 +1000 Subject: RFR: 8203923: Add @requires feature to check flag values for the running JVM In-Reply-To: <58b03b15-9094-dfbf-3575-0c31f9af479a@oracle.com> References: <8e9694e9-1148-11bb-560d-5ae70936fb7e@oracle.com> <58b03b15-9094-dfbf-3575-0c31f9af479a@oracle.com> Message-ID: <3c96c1cf-031b-8bce-3065-da3eb05c5364@oracle.com> Hi Stefan, On 30/05/2018 8:07 AM, Stefan Karlsson wrote: > Hi David, > > On 2018-05-29 23:53, David Holmes wrote: >> Hi Stefan, >> >> On 30/05/2018 12:21 AM, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to add @requires vm.opt.final.>> name>, as a way to filter jtreg tests on the final value of flags in >>> the test VM. >>> >>> http://cr.openjdk.java.net/~stefank/8203923/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8203923 >>> >>> This patch is needed to allow tests to be filtered out when ZGC is run. >>> >>> ZGC doesn't support UseCompressedOops (and currently doesn't support >>> ClassUnloading). Instead of adding @requires vm.gc != "Z" for tests >>> that rely on ClassUnloading, I think it's nicer to write @requires >>> vm.opt.final.ClassUnloading. >>> >>> This will filter out the test if any of the following conditions are >>> met: >>> 1) The currently executing VM doesn't support ClassUnloading >>> 2) -XX:-ClassUnloading is passed to the test >>> >>> The second case can today be handled by @requires >>> vm.opt.ClassUnloading == NULL | vm.opt.ClassUnloading == true, or >>> maybe simpler @requires vm.opt.ClassUnloading != false, but if you >>> need both @requires vm.opt.final.ClassUnloading should be used. >> >> I'm not sure about the use of vm.opt.final as the name here. IIUC >> vm.opt is set by jtreg based on the vmoptions passed to jtreg for >> running the tests. Whereas this is the final flag value as seen in the >> running VM. And given we can only inspect the final value this way, >> perhaps just vm.flag would be better? > > One reason why I chose to use the vm.opt.final prefix, instead of some > other prefix (like vm.flag), is that jtreg "conveniently" accepts any > property prefixed with vm.opt. If I change the prefix to vm.flag I need > to add every single vm.flag to the requires.properties in all TEST.ROOT > files of the test directories where these properties are used (E.g. > test/hotspot/jtreg/TEST.ROOT). > > See isValidName in: > http://hg.openjdk.java.net/code-tools/jtreg/file/7b1496d2790e/src/share/classes/com/sun/javatest/regtest/config/RegressionContext.java#l109 I see. That is unfortunate. I think we need to discuss with Jon how to make this more flexible. vm.flag seems much better (and more accurate) than vm.opt... > I can change the name, but I think that would come with a maintenance > cost. What do you think? We can live with current name for now and see how to improve this. Thanks, David >> >>> The current solution requires the queried flags to be listed in >>> vmOptFinalFlags. Maybe there's a way to get this to cover all flags >>> automatically? >> >> Yes it is a shame that this has to be manually configured. I think >> ideally this would need to be built into jtreg so that it ran a >> dcmd(?) to get all final flag values and then populated the >> properties. Though I suppose VMProps could also do that ... (The crude >> way of doing this is via PrintFlagsFinal and parsing the output, of >> course). >> >> Anyway other than my query on the naming, this seems fine for moving >> ahead. > > Thanks, > StefanK > >> >> Thanks, >> David >> >>> Thanks, >>> StefanK > > From stefan.karlsson at oracle.com Wed May 30 05:36:43 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Wed, 30 May 2018 07:36:43 +0200 Subject: RFR: 8203923: Add @requires feature to check flag values for the running JVM In-Reply-To: <3c96c1cf-031b-8bce-3065-da3eb05c5364@oracle.com> References: <8e9694e9-1148-11bb-560d-5ae70936fb7e@oracle.com> <58b03b15-9094-dfbf-3575-0c31f9af479a@oracle.com> <3c96c1cf-031b-8bce-3065-da3eb05c5364@oracle.com> Message-ID: <11f21c3f-fa66-b6cf-687d-2b57b7d0a418@oracle.com> On 2018-05-30 01:44, David Holmes wrote: > Hi Stefan, > > On 30/05/2018 8:07 AM, Stefan Karlsson wrote: >> Hi David, >> >> On 2018-05-29 23:53, David Holmes wrote: >>> Hi Stefan, >>> >>> On 30/05/2018 12:21 AM, Stefan Karlsson wrote: >>>> Hi all, >>>> >>>> Please review this patch to add @requires vm.opt.final.>>> name>, as a way to filter jtreg tests on the final value of flags >>>> in the test VM. >>>> >>>> http://cr.openjdk.java.net/~stefank/8203923/webrev.01/ >>>> https://bugs.openjdk.java.net/browse/JDK-8203923 >>>> >>>> This patch is needed to allow tests to be filtered out when ZGC is >>>> run. >>>> >>>> ZGC doesn't support UseCompressedOops (and currently doesn't >>>> support ClassUnloading). Instead of adding @requires vm.gc != "Z" >>>> for tests that rely on ClassUnloading, I think it's nicer to write >>>> @requires vm.opt.final.ClassUnloading. >>>> >>>> This will filter out the test if any of the following conditions >>>> are met: >>>> 1) The currently executing VM doesn't support ClassUnloading >>>> 2) -XX:-ClassUnloading is passed to the test >>>> >>>> The second case can today be handled by @requires >>>> vm.opt.ClassUnloading == NULL | vm.opt.ClassUnloading == true, or >>>> maybe simpler @requires vm.opt.ClassUnloading != false, but if you >>>> need both @requires vm.opt.final.ClassUnloading should be used. >>> >>> I'm not sure about the use of vm.opt.final as the name here. IIUC >>> vm.opt is set by jtreg based on the vmoptions passed to jtreg for >>> running the tests. Whereas this is the final flag value as seen in >>> the running VM. And given we can only inspect the final value this >>> way, perhaps just vm.flag would be better? >> >> One reason why I chose to use the vm.opt.final prefix, instead of >> some other prefix (like vm.flag), is that jtreg "conveniently" >> accepts any property prefixed with vm.opt. If I change the prefix to >> vm.flag I need to add every single vm.flag to the requires.properties >> in all TEST.ROOT files of the test directories where these properties >> are used (E.g. test/hotspot/jtreg/TEST.ROOT). >> >> See isValidName in: >> http://hg.openjdk.java.net/code-tools/jtreg/file/7b1496d2790e/src/share/classes/com/sun/javatest/regtest/config/RegressionContext.java#l109 > > > I see. That is unfortunate. I think we need to discuss with Jon how to > make this more flexible. vm.flag seems much better (and more accurate) > than vm.opt... > >> I can change the name, but I think that would come with a maintenance >> cost. What do you think? > > We can live with current name for now and see how to improve this. Thanks, StefanK > > Thanks, > David > >>> >>>> The current solution requires the queried flags to be listed in >>>> vmOptFinalFlags. Maybe there's a way to get this to cover all flags >>>> automatically? >>> >>> Yes it is a shame that this has to be manually configured. I think >>> ideally this would need to be built into jtreg so that it ran a >>> dcmd(?) to get all final flag values and then populated the >>> properties. Though I suppose VMProps could also do that ... (The >>> crude way of doing this is via PrintFlagsFinal and parsing the >>> output, of course). >>> >>> Anyway other than my query on the naming, this seems fine for moving >>> ahead. >> >> Thanks, >> StefanK >> >>> >>> Thanks, >>> David >>> >>>> Thanks, >>>> StefanK >> >> From igor.ignatyev at oracle.com Wed May 30 06:38:21 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 29 May 2018 23:38:21 -0700 Subject: RFR(M) : 8202946 : [TESTBUG] Open source VM testbase OOM tests In-Reply-To: <23ea1dbe-f6bc-b76a-237a-ff6fc1a6dc58@oracle.com> References: <2DD8F9C6-8471-4BF6-8573-0DA3F2B6C66B@oracle.com> <23ea1dbe-f6bc-b76a-237a-ff6fc1a6dc58@oracle.com> Message-ID: <95398DD6-5014-443E-A5B5-9834DE21F551@oracle.com> Serguei, thanks for your review! GC team, can someone please review this? Thanks, -- Igor > On May 21, 2018, at 10:33 PM, serguei.spitsyn at oracle.com wrote: > > Hi Igor, > > This looks good. > This change has to be reviewed by someone from the GC team. > > Thanks, > Serguei > > > On 5/15/18 16:16, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8202946/webrev.00/index.html >>> 1619 lines changed: 1619 ins; 0 del; 0 mod; >> Hi all, >> >> could you please review this patch which open sources OOM tests from VM testbase? these tests test OutOfMemoryError throwing in different scenarios. >> >> As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8202946 >> webrev: http://cr.openjdk.java.net/~iignatyev//8202946/webrev.00/index.html >> testing: :vmTestbase_vm_oom test group >> >> Thanks, >> -- Igor > From boris.ulasevich at bell-sw.com Wed May 30 09:04:25 2018 From: boris.ulasevich at bell-sw.com (Boris Ulasevich) Date: Wed, 30 May 2018 12:04:25 +0300 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <61ded1ef-b713-d200-324c-6efe5cc981d8@oracle.com> References: <53feed71-0ab9-7743-6034-3c95bb3b20e8@bell-sw.com> <848b45e0-df51-7d74-e4eb-60819d101330@oracle.com> <9a8eadf0-e745-ee38-fb37-b29d99fa96c0@bell-sw.com> <71dbee89-451e-a151-474a-74326a2fa1c6@oracle.com> <61ded1ef-b713-d200-324c-6efe5cc981d8@oracle.com> Message-ID: Hi David, With v2,v3,v4 patches applied I have runtime/Nestmates passed on ARM32. regards, Boris On 29.05.2018 23:52, David Holmes wrote: > Hi Boris, > > On 30/05/2018 1:46 AM, Boris Ulasevich wrote: >> Hi David, >> >> ARM32 part of the change looks good for me. >> Do you want me to apply latest (v4-incr?) patch and run tests? > > Please do. > > Thanks, > David > >> Boris >> >> On 24.05.2018 00:39, David Holmes wrote: >>> Hi Boris, >>> >>> Thanks for verifying my changes. In case you didn't see you can apply >>> the updated patch to my v1 changes from: >>> >>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2/ >>> >>> One follow up below. >>> >>> On 23/05/2018 11:40 PM, Boris Ulasevich wrote: >>>> Hi David, >>>> >>>> some minor comments below.. >>>> >>>> On 23.05.2018 09:57, David Holmes wrote: >>>>> Hi Boris, >>>>> >>>>> On 17/05/2018 7:23 PM, Boris Ulasevich wrote: >>>>>> Hi David, >>>>>> >>>>>> You are right! My bad, R2_tmp parameter conflicts with Rklass on >>>>>> check_klass_subtype(..) call. See correct patch in attach. Now all >>>>>> runtime/Nestmates tests passed! :) >>>>> >>>>> I went to aplpy your patch and found it's not a diff against my >>>>> patch but against the original non-nestmate code, >>>> >>>> Yes, sorry. For me it was natural to apply patch from your review to >>>> local repo, rework it and just send updated diff from my machine :) >>>> >>>>> so I want to be clear on the changes. AFAICS the differences are: >>>>> >>>>> -? const Register Rklass? = R3_tmp; >>>>> +? const Register Rklass? = R2_tmp; // Note! Same register with Rrecv >>>> >>>> Yes. >>>> >>>>> This initially concerned me as we stomp on Rrecv when we do the >>>>> load_klass, but then you moved? the: >>>>> >>>>> ??? __ load_klass(Rklass, Rrecv); >>>>> >>>>> to after the object case, which used Rrecv. I had assumed Rrecv was >>>>> somehow used when we actually do the call, but I'm assuming >>>>> jump_from_interpreted is done in such as way that the receiver is >>>>> still available on the interpreter stack and is used from there. >>>> >>>> Ok. I'm not sure I understand you well. When I moved load_klass call >>>> down my point was to skip unnecessary load for the notObjectMethod >>>> case (Rklass is not required in this case) and to make possible to >>>> reuse same R2 register by both Rklass and Rrecv register. >>> >>> Right. My concern was an assumption that as Rrecv was the receiver >>> object that we had to leave it intact ready for the actual method >>> invocation. But that isn't the case. The real method invocation >>> happens elsewhere and the receiver is obtained by other means. >>> >>> Thanks, >>> David >>> >>>>> -? __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >>>>> +? __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R3_tmp, noreg, >>>>> subtype); >>>>> >>>>> Okay - reworking of tmp regs. >>>> >>>> Yes, ARM32 requires one more tmp reg, and we can't spoil R0. >>>> >>>>> -? __ jump_from_interpreted(Rindex); >>>>> +? __ jump_from_interpreted(Rmethod); >>>>> >>>>> I'm going to trust this is okay :) It's not at all clear to me how >>>>> the f2 Method* gets passed through on different platforms - >>>>> sometimes its in the "index" and sometimes the "method" registers. >>>> >>>> I see. On ARM/AARCH64 we have preallocated register Rmethod/rmethod >>>> to hold current method pointer and prepare_invoke call to setup >>>> registers properly. >>>> >>>>> (I _really_ wish there was consistency in terminology across the >>>>> different platforms - this code is awful for trying to compare the >>>>> different platforms to figure out what to do on a new one.) >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> regards, >>>>>> Boris >>>>>> >>>>>> On 17.05.2018 11:24, David Holmes wrote: >>>>>>> On 17/05/2018 6:13 PM, Boris Ulasevich wrote: >>>>>>>> Hi David, >>>>>>>> >>>>>>>> I see three tests failing: >>>>>>>> ?> NullPointerException at >>>>>>>> TestInterfaceMethodSelection.doInvoke(TestInterfaceMethodSelection.java:191) >>>>>>>> >>>>>>>> ?> NullPointerException at >>>>>>>> TestInvoke.access_priv(TestInvoke.java:54) >>>>>>>> ?> InvocationTargetException at >>>>>>>> TestReflection.access_priv(TestReflection.java:61) >>>>>>>> >>>>>>>> I will send you test details in a separate mail. >>>>>>> >>>>>>> Ok. This indicates a bug in the assembly code. The NPE's will >>>>>>> likely be SEGVs caused by a zero register. >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>> >>>>>>>> Boris >>>>>>>> >>>>>>>> On 17.05.2018 00:23, David Holmes wrote: >>>>>>>>> Hi Boris, >>>>>>>>> >>>>>>>>> Many thanks for looking at this and working through the ARM code. >>>>>>>>> >>>>>>>>> I will study your patch in detail but am concerned by the >>>>>>>>> "passes most of runtime/Nestmates tests Ok."! What tests are >>>>>>>>> not passing? >>>>>>>>> >>>>>>>>> David >>>>>>>>> >>>>>>>>> On 17/05/2018 1:05 AM, Boris Ulasevich wrote: >>>>>>>>>> Hi David, >>>>>>>>>> >>>>>>>>>> Let us look to the change in templateTable_arm.cpp. I have >>>>>>>>>> several notes. >>>>>>>>>> >>>>>>>>>> 1. We have compilation error because check_klass_subtype call >>>>>>>>>> needs one more temporary register parameter. I think correct >>>>>>>>>> change is: >>>>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >>>>>>>>>> -> >>>>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, >>>>>>>>>> subtype); >>>>>>>>>> >>>>>>>>>> 2. R0_tmp holds mdp used for profilation several lines below, >>>>>>>>>> so we can't spoil it. I think correct change is: >>>>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, >>>>>>>>>> subtype); >>>>>>>>>> -> >>>>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R2_tmp, noreg, >>>>>>>>>> subtype); >>>>>>>>>> >>>>>>>>>> 3. We can't jump to Rindex. I believe it was supposed to jump >>>>>>>>>> to Rmethod: >>>>>>>>>> jump_from_interpreted(Rindex); >>>>>>>>>> -> >>>>>>>>>> jump_from_interpreted(Rmethod); >>>>>>>>>> >>>>>>>>>> 4. Please note than Rklass and Rflags reuses same register. >>>>>>>>>> Before the change their life ranges had no intersection. I >>>>>>>>>> would propose to use R2 for Rklass (same with Rrecv) and move >>>>>>>>>> load_klass call after invokevirtual_helper call to avoid life >>>>>>>>>> range intersecton. >>>>>>>>>> >>>>>>>>>> See my patch against original templateTable_arm.cpp version in >>>>>>>>>> attach - with this change ARM build compiles and passes most >>>>>>>>>> of runtime/Nestmates tests Ok. >>>>>>>>>> >>>>>>>>>> regards, >>>>>>>>>> Boris >>>>>>>>>> >>>>>>>>>> On 15.05.2018 03:52, David Holmes wrote: >>>>>>>>>>> This review is being spread across four groups: langtools, >>>>>>>>>>> core-libs, hotspot and serviceability. This is the specific >>>>>>>>>>> review thread for hotspot - webrev: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> See below for full details - including annotated full webrev >>>>>>>>>>> guiding the review. >>>>>>>>>>> >>>>>>>>>>> The intent is to have JEP-181 targeted and integrated by the >>>>>>>>>>> end of this month. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> David >>>>>>>>>>> ----- >>>>>>>>>>> >>>>>>>>>>> The nestmates project (JEP-181) introduces new classfile >>>>>>>>>>> attributes to identify classes and interfaces in the same >>>>>>>>>>> nest, so that the VM can perform access control based on >>>>>>>>>>> those attributes and so allow direct private access between >>>>>>>>>>> nestmates without requiring javac to generate synthetic >>>>>>>>>>> accessor methods. These access control changes also extend to >>>>>>>>>>> core reflection and the MethodHandle.Lookup contexts. >>>>>>>>>>> >>>>>>>>>>> Direct private calls between nestmates requires a more >>>>>>>>>>> general calling context than is permitted by invokespecial, >>>>>>>>>>> and so the JVMS is updated to allow, and javac updated to >>>>>>>>>>> use, invokevirtual and invokeinterface for private class and >>>>>>>>>>> interface method calls respectively. These changed semantics >>>>>>>>>>> also extend to MethodHandle findXXX operations. >>>>>>>>>>> >>>>>>>>>>> At this time we are only concerned with static nest >>>>>>>>>>> definitions, which map to a top-level class/interface as the >>>>>>>>>>> nest-host and all its nested types as nest-members. >>>>>>>>>>> >>>>>>>>>>> Please see the JEP for further details. >>>>>>>>>>> >>>>>>>>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>>>>>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>>>>>>>>> >>>>>>>>>>> All of the specification changes have been previously been >>>>>>>>>>> worked out by the Valhalla Project Expert Group, and the >>>>>>>>>>> implementation reviewed by the various contributors and >>>>>>>>>>> discussed on the valhalla-dev mailing list. >>>>>>>>>>> >>>>>>>>>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>>>>>>>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, >>>>>>>>>>> Karen Kinnear, Vladimir Kozlov, John Rose, Dan Smith, Serguei >>>>>>>>>>> Spitsyn, Kumar Srinivasan >>>>>>>>>>> >>>>>>>>>>> Master webrev of all changes: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Annotated master webrev index: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Performance: this is expected to be performance neutral in a >>>>>>>>>>> general sense. Benchmarking and performance runs are about to >>>>>>>>>>> start. >>>>>>>>>>> >>>>>>>>>>> Testing Discussion: >>>>>>>>>>> ------------------ >>>>>>>>>>> >>>>>>>>>>> The testing for nestmates can be broken into four main groups: >>>>>>>>>>> >>>>>>>>>>> -? New tests specifically related to nestmates and currently >>>>>>>>>>> in the runtime/Nestmates directory >>>>>>>>>>> >>>>>>>>>>> - New tests to complement existing tests by adding in >>>>>>>>>>> testcases not previously expressible. >>>>>>>>>>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java >>>>>>>>>>> tests use of invokespecial for private interface methods and >>>>>>>>>>> performing receiver typechecks, so we add >>>>>>>>>>> java/lang/invoke/PrivateInterfaceCall.java to do similar >>>>>>>>>>> tests for invokeinterface. >>>>>>>>>>> >>>>>>>>>>> -? New JVM TI tests to verify the spec changes related to >>>>>>>>>>> nest attributes. >>>>>>>>>>> >>>>>>>>>>> -? Existing tests significantly affected by the nestmates >>>>>>>>>>> changes, primarily: >>>>>>>>>>> ??? -? runtime/SelectionResolution >>>>>>>>>>> >>>>>>>>>>> ??? In most cases the nestmate changes makes certain >>>>>>>>>>> invocations that were illegal, legal (e.g. not requiring >>>>>>>>>>> invokespecial to invoke private interface methods; allowing >>>>>>>>>>> access to private members via reflection/Methodhandles that >>>>>>>>>>> were previously not allowed). >>>>>>>>>>> >>>>>>>>>>> - Existing tests incidentally affected by the nestmate changes >>>>>>>>>>> >>>>>>>>>>> ?? This includes tests of things utilising class >>>>>>>>>>> redefinition/retransformation to alter nested types but which >>>>>>>>>>> unintentionally alter nest relationships (which is not >>>>>>>>>>> permitted). >>>>>>>>>>> >>>>>>>>>>> There are still a number of tests problem-listed with issues >>>>>>>>>>> filed against them to have them adapted to work with >>>>>>>>>>> nestmates. Some of these are intended to be addressed in the >>>>>>>>>>> short-term, while some (such as the >>>>>>>>>>> runtime/SelectionResolution test changes) may not eventuate. >>>>>>>>>>> >>>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>>>>>>>>> >>>>>>>>>>> There is also further test work still to be completed (the >>>>>>>>>>> JNI and JDI invocation tests): >>>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>>>>>>>>> which will continue in parallel with the main RFR. >>>>>>>>>>> >>>>>>>>>>> Pre-integration Testing: >>>>>>>>>>> ??- General: >>>>>>>>>>> ???? - Mach5: hs/jdk tier1,2 >>>>>>>>>>> ???? - Mach5: hs-nightly (tiers 1 -3) >>>>>>>>>>> ??- Targetted >>>>>>>>>>> ??? - nashorn (for asm changes) >>>>>>>>>>> ??? - hotspot: runtime/* >>>>>>>>>>> ?????????????? serviceability/* >>>>>>>>>>> ?????????????? compiler/* >>>>>>>>>>> ?????????????? vmTestbase/* >>>>>>>>>>> ??? - jdk: java/lang/invoke/* >>>>>>>>>>> ?????????? java/lang/reflect/* >>>>>>>>>>> ?????????? java/lang/instrument/* >>>>>>>>>>> ?????????? java/lang/Class/* >>>>>>>>>>> ?????????? java/lang/management/* >>>>>>>>>>> ?? - langtools: tools/javac >>>>>>>>>>> ??????????????? tools/javap >>>>>>>>>>> From david.holmes at oracle.com Wed May 30 09:42:37 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 30 May 2018 19:42:37 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: References: <53feed71-0ab9-7743-6034-3c95bb3b20e8@bell-sw.com> <848b45e0-df51-7d74-e4eb-60819d101330@oracle.com> <9a8eadf0-e745-ee38-fb37-b29d99fa96c0@bell-sw.com> <71dbee89-451e-a151-474a-74326a2fa1c6@oracle.com> <61ded1ef-b713-d200-324c-6efe5cc981d8@oracle.com> Message-ID: <76951a9d-a72a-1b76-a97c-a03179f8d8f4@oracle.com> Good to hear! Thanks Boris. David On 30/05/2018 7:04 PM, Boris Ulasevich wrote: > Hi David, > > ? With v2,v3,v4 patches applied I have runtime/Nestmates passed on ARM32. > > regards, > Boris > > On 29.05.2018 23:52, David Holmes wrote: >> Hi Boris, >> >> On 30/05/2018 1:46 AM, Boris Ulasevich wrote: >>> Hi David, >>> >>> ARM32 part of the change looks good for me. >>> Do you want me to apply latest (v4-incr?) patch and run tests? >> >> Please do. >> >> Thanks, >> David >> >>> Boris >>> >>> On 24.05.2018 00:39, David Holmes wrote: >>>> Hi Boris, >>>> >>>> Thanks for verifying my changes. In case you didn't see you can >>>> apply the updated patch to my v1 changes from: >>>> >>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v2/ >>>> >>>> One follow up below. >>>> >>>> On 23/05/2018 11:40 PM, Boris Ulasevich wrote: >>>>> Hi David, >>>>> >>>>> some minor comments below.. >>>>> >>>>> On 23.05.2018 09:57, David Holmes wrote: >>>>>> Hi Boris, >>>>>> >>>>>> On 17/05/2018 7:23 PM, Boris Ulasevich wrote: >>>>>>> Hi David, >>>>>>> >>>>>>> You are right! My bad, R2_tmp parameter conflicts with Rklass on >>>>>>> check_klass_subtype(..) call. See correct patch in attach. Now >>>>>>> all runtime/Nestmates tests passed! :) >>>>>> >>>>>> I went to aplpy your patch and found it's not a diff against my >>>>>> patch but against the original non-nestmate code, >>>>> >>>>> Yes, sorry. For me it was natural to apply patch from your review >>>>> to local repo, rework it and just send updated diff from my machine :) >>>>> >>>>>> so I want to be clear on the changes. AFAICS the differences are: >>>>>> >>>>>> -? const Register Rklass? = R3_tmp; >>>>>> +? const Register Rklass? = R2_tmp; // Note! Same register with Rrecv >>>>> >>>>> Yes. >>>>> >>>>>> This initially concerned me as we stomp on Rrecv when we do the >>>>>> load_klass, but then you moved? the: >>>>>> >>>>>> ??? __ load_klass(Rklass, Rrecv); >>>>>> >>>>>> to after the object case, which used Rrecv. I had assumed Rrecv >>>>>> was somehow used when we actually do the call, but I'm assuming >>>>>> jump_from_interpreted is done in such as way that the receiver is >>>>>> still available on the interpreter stack and is used from there. >>>>> >>>>> Ok. I'm not sure I understand you well. When I moved load_klass >>>>> call down my point was to skip unnecessary load for the >>>>> notObjectMethod case (Rklass is not required in this case) and to >>>>> make possible to reuse same R2 register by both Rklass and Rrecv >>>>> register. >>>> >>>> Right. My concern was an assumption that as Rrecv was the receiver >>>> object that we had to leave it intact ready for the actual method >>>> invocation. But that isn't the case. The real method invocation >>>> happens elsewhere and the receiver is obtained by other means. >>>> >>>> Thanks, >>>> David >>>> >>>>>> -? __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >>>>>> +? __ check_klass_subtype(Rklass, Rinterf, R1_tmp, R3_tmp, noreg, >>>>>> subtype); >>>>>> >>>>>> Okay - reworking of tmp regs. >>>>> >>>>> Yes, ARM32 requires one more tmp reg, and we can't spoil R0. >>>>> >>>>>> -? __ jump_from_interpreted(Rindex); >>>>>> +? __ jump_from_interpreted(Rmethod); >>>>>> >>>>>> I'm going to trust this is okay :) It's not at all clear to me how >>>>>> the f2 Method* gets passed through on different platforms - >>>>>> sometimes its in the "index" and sometimes the "method" registers. >>>>> >>>>> I see. On ARM/AARCH64 we have preallocated register Rmethod/rmethod >>>>> to hold current method pointer and prepare_invoke call to setup >>>>> registers properly. >>>>> >>>>>> (I _really_ wish there was consistency in terminology across the >>>>>> different platforms - this code is awful for trying to compare the >>>>>> different platforms to figure out what to do on a new one.) >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> regards, >>>>>>> Boris >>>>>>> >>>>>>> On 17.05.2018 11:24, David Holmes wrote: >>>>>>>> On 17/05/2018 6:13 PM, Boris Ulasevich wrote: >>>>>>>>> Hi David, >>>>>>>>> >>>>>>>>> I see three tests failing: >>>>>>>>> ?> NullPointerException at >>>>>>>>> TestInterfaceMethodSelection.doInvoke(TestInterfaceMethodSelection.java:191) >>>>>>>>> >>>>>>>>> ?> NullPointerException at >>>>>>>>> TestInvoke.access_priv(TestInvoke.java:54) >>>>>>>>> ?> InvocationTargetException at >>>>>>>>> TestReflection.access_priv(TestReflection.java:61) >>>>>>>>> >>>>>>>>> I will send you test details in a separate mail. >>>>>>>> >>>>>>>> Ok. This indicates a bug in the assembly code. The NPE's will >>>>>>>> likely be SEGVs caused by a zero register. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>> >>>>>>>>> Boris >>>>>>>>> >>>>>>>>> On 17.05.2018 00:23, David Holmes wrote: >>>>>>>>>> Hi Boris, >>>>>>>>>> >>>>>>>>>> Many thanks for looking at this and working through the ARM code. >>>>>>>>>> >>>>>>>>>> I will study your patch in detail but am concerned by the >>>>>>>>>> "passes most of runtime/Nestmates tests Ok."! What tests are >>>>>>>>>> not passing? >>>>>>>>>> >>>>>>>>>> David >>>>>>>>>> >>>>>>>>>> On 17/05/2018 1:05 AM, Boris Ulasevich wrote: >>>>>>>>>>> Hi David, >>>>>>>>>>> >>>>>>>>>>> Let us look to the change in templateTable_arm.cpp. I have >>>>>>>>>>> several notes. >>>>>>>>>>> >>>>>>>>>>> 1. We have compilation error because check_klass_subtype call >>>>>>>>>>> needs one more temporary register parameter. I think correct >>>>>>>>>>> change is: >>>>>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, subtype); >>>>>>>>>>> -> >>>>>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, >>>>>>>>>>> subtype); >>>>>>>>>>> >>>>>>>>>>> 2. R0_tmp holds mdp used for profilation several lines below, >>>>>>>>>>> so we can't spoil it. I think correct change is: >>>>>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R0_tmp, noreg, >>>>>>>>>>> subtype); >>>>>>>>>>> -> >>>>>>>>>>> check_klass_subtype(Rklass, Rinterf, R1_tmp, R2_tmp, noreg, >>>>>>>>>>> subtype); >>>>>>>>>>> >>>>>>>>>>> 3. We can't jump to Rindex. I believe it was supposed to jump >>>>>>>>>>> to Rmethod: >>>>>>>>>>> jump_from_interpreted(Rindex); >>>>>>>>>>> -> >>>>>>>>>>> jump_from_interpreted(Rmethod); >>>>>>>>>>> >>>>>>>>>>> 4. Please note than Rklass and Rflags reuses same register. >>>>>>>>>>> Before the change their life ranges had no intersection. I >>>>>>>>>>> would propose to use R2 for Rklass (same with Rrecv) and move >>>>>>>>>>> load_klass call after invokevirtual_helper call to avoid life >>>>>>>>>>> range intersecton. >>>>>>>>>>> >>>>>>>>>>> See my patch against original templateTable_arm.cpp version >>>>>>>>>>> in attach - with this change ARM build compiles and passes >>>>>>>>>>> most of runtime/Nestmates tests Ok. >>>>>>>>>>> >>>>>>>>>>> regards, >>>>>>>>>>> Boris >>>>>>>>>>> >>>>>>>>>>> On 15.05.2018 03:52, David Holmes wrote: >>>>>>>>>>>> This review is being spread across four groups: langtools, >>>>>>>>>>>> core-libs, hotspot and serviceability. This is the specific >>>>>>>>>>>> review thread for hotspot - webrev: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.hotspot.v1/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> See below for full details - including annotated full webrev >>>>>>>>>>>> guiding the review. >>>>>>>>>>>> >>>>>>>>>>>> The intent is to have JEP-181 targeted and integrated by the >>>>>>>>>>>> end of this month. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> David >>>>>>>>>>>> ----- >>>>>>>>>>>> >>>>>>>>>>>> The nestmates project (JEP-181) introduces new classfile >>>>>>>>>>>> attributes to identify classes and interfaces in the same >>>>>>>>>>>> nest, so that the VM can perform access control based on >>>>>>>>>>>> those attributes and so allow direct private access between >>>>>>>>>>>> nestmates without requiring javac to generate synthetic >>>>>>>>>>>> accessor methods. These access control changes also extend >>>>>>>>>>>> to core reflection and the MethodHandle.Lookup contexts. >>>>>>>>>>>> >>>>>>>>>>>> Direct private calls between nestmates requires a more >>>>>>>>>>>> general calling context than is permitted by invokespecial, >>>>>>>>>>>> and so the JVMS is updated to allow, and javac updated to >>>>>>>>>>>> use, invokevirtual and invokeinterface for private class and >>>>>>>>>>>> interface method calls respectively. These changed semantics >>>>>>>>>>>> also extend to MethodHandle findXXX operations. >>>>>>>>>>>> >>>>>>>>>>>> At this time we are only concerned with static nest >>>>>>>>>>>> definitions, which map to a top-level class/interface as the >>>>>>>>>>>> nest-host and all its nested types as nest-members. >>>>>>>>>>>> >>>>>>>>>>>> Please see the JEP for further details. >>>>>>>>>>>> >>>>>>>>>>>> JEP: https://bugs.openjdk.java.net/browse/JDK-8046171 >>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8010319 >>>>>>>>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8197445 >>>>>>>>>>>> >>>>>>>>>>>> All of the specification changes have been previously been >>>>>>>>>>>> worked out by the Valhalla Project Expert Group, and the >>>>>>>>>>>> implementation reviewed by the various contributors and >>>>>>>>>>>> discussed on the valhalla-dev mailing list. >>>>>>>>>>>> >>>>>>>>>>>> Acknowledgments and contributions: Alex Buckley, Maurizio >>>>>>>>>>>> Cimadamore, Mandy Chung, Tobias Hartmann, Vladimir Ivanov, >>>>>>>>>>>> Karen Kinnear, Vladimir Kozlov, John Rose, Dan Smith, >>>>>>>>>>>> Serguei Spitsyn, Kumar Srinivasan >>>>>>>>>>>> >>>>>>>>>>>> Master webrev of all changes: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v1/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Annotated master webrev index: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~dholmes/8010319-JEP181/jep181-webrev.html >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Performance: this is expected to be performance neutral in a >>>>>>>>>>>> general sense. Benchmarking and performance runs are about >>>>>>>>>>>> to start. >>>>>>>>>>>> >>>>>>>>>>>> Testing Discussion: >>>>>>>>>>>> ------------------ >>>>>>>>>>>> >>>>>>>>>>>> The testing for nestmates can be broken into four main groups: >>>>>>>>>>>> >>>>>>>>>>>> -? New tests specifically related to nestmates and currently >>>>>>>>>>>> in the runtime/Nestmates directory >>>>>>>>>>>> >>>>>>>>>>>> - New tests to complement existing tests by adding in >>>>>>>>>>>> testcases not previously expressible. >>>>>>>>>>>> ?? -? For example java/lang/invoke/SpecialInterfaceCall.java >>>>>>>>>>>> tests use of invokespecial for private interface methods and >>>>>>>>>>>> performing receiver typechecks, so we add >>>>>>>>>>>> java/lang/invoke/PrivateInterfaceCall.java to do similar >>>>>>>>>>>> tests for invokeinterface. >>>>>>>>>>>> >>>>>>>>>>>> -? New JVM TI tests to verify the spec changes related to >>>>>>>>>>>> nest attributes. >>>>>>>>>>>> >>>>>>>>>>>> -? Existing tests significantly affected by the nestmates >>>>>>>>>>>> changes, primarily: >>>>>>>>>>>> ??? -? runtime/SelectionResolution >>>>>>>>>>>> >>>>>>>>>>>> ??? In most cases the nestmate changes makes certain >>>>>>>>>>>> invocations that were illegal, legal (e.g. not requiring >>>>>>>>>>>> invokespecial to invoke private interface methods; allowing >>>>>>>>>>>> access to private members via reflection/Methodhandles that >>>>>>>>>>>> were previously not allowed). >>>>>>>>>>>> >>>>>>>>>>>> - Existing tests incidentally affected by the nestmate changes >>>>>>>>>>>> >>>>>>>>>>>> ?? This includes tests of things utilising class >>>>>>>>>>>> redefinition/retransformation to alter nested types but >>>>>>>>>>>> which unintentionally alter nest relationships (which is not >>>>>>>>>>>> permitted). >>>>>>>>>>>> >>>>>>>>>>>> There are still a number of tests problem-listed with issues >>>>>>>>>>>> filed against them to have them adapted to work with >>>>>>>>>>>> nestmates. Some of these are intended to be addressed in the >>>>>>>>>>>> short-term, while some (such as the >>>>>>>>>>>> runtime/SelectionResolution test changes) may not eventuate. >>>>>>>>>>>> >>>>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8203033 >>>>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8199450 >>>>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8196855 >>>>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8194857 >>>>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8187655 >>>>>>>>>>>> >>>>>>>>>>>> There is also further test work still to be completed (the >>>>>>>>>>>> JNI and JDI invocation tests): >>>>>>>>>>>> - https://bugs.openjdk.java.net/browse/JDK-8191117 >>>>>>>>>>>> which will continue in parallel with the main RFR. >>>>>>>>>>>> >>>>>>>>>>>> Pre-integration Testing: >>>>>>>>>>>> ??- General: >>>>>>>>>>>> ???? - Mach5: hs/jdk tier1,2 >>>>>>>>>>>> ???? - Mach5: hs-nightly (tiers 1 -3) >>>>>>>>>>>> ??- Targetted >>>>>>>>>>>> ??? - nashorn (for asm changes) >>>>>>>>>>>> ??? - hotspot: runtime/* >>>>>>>>>>>> ?????????????? serviceability/* >>>>>>>>>>>> ?????????????? compiler/* >>>>>>>>>>>> ?????????????? vmTestbase/* >>>>>>>>>>>> ??? - jdk: java/lang/invoke/* >>>>>>>>>>>> ?????????? java/lang/reflect/* >>>>>>>>>>>> ?????????? java/lang/instrument/* >>>>>>>>>>>> ?????????? java/lang/Class/* >>>>>>>>>>>> ?????????? java/lang/management/* >>>>>>>>>>>> ?? - langtools: tools/javac >>>>>>>>>>>> ??????????????? tools/javap >>>>>>>>>>>> From martin.doerr at sap.com Wed May 30 10:26:12 2018 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 30 May 2018 10:26:12 +0000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <7aa56a5e-0c95-f1a2-0b27-3f759ca5c702@oracle.com> References: <06529fc3-2eba-101b-9aee-2757893cb8fb@oracle.com> <97f8cedf-4ebc-610f-0528-e1b91f35eece@oracle.com> <7aa56a5e-0c95-f1a2-0b27-3f759ca5c702@oracle.com> Message-ID: <3d40c5996b2549d9ae18f9a9486b3f50@sap.com> Hi David, can you apply the following small patch to fix ppc64 and s390 builds, please? Best regards, Martin diff -r 70d41ef93cf0 src/hotspot/cpu/ppc/templateTable_ppc_64.cpp --- a/src/hotspot/cpu/ppc/templateTable_ppc_64.cpp Wed May 30 12:08:14 2018 +0200 +++ b/src/hotspot/cpu/ppc/templateTable_ppc_64.cpp Wed May 30 12:24:00 2018 +0200 @@ -3608,10 +3608,10 @@ __ testbitdi(CCR0, R0, Rflags, ConstantPoolCacheEntry::is_vfinal_shift); __ bfalse(CCR0, LnotVFinal); - __ check_klass_subtype(Rrecv_klass, Rinterface_klass, Rscratch, Rscratch1, subtype); + __ check_klass_subtype(Rrecv_klass, Rinterface_klass, Rscratch1, Rscratch2, L_subtype); // If we get here the typecheck failed __ b(L_no_such_interface); - __ bind(subtype); + __ bind(L_subtype); // do the call diff -r 70d41ef93cf0 src/hotspot/cpu/s390/templateTable_s390.cpp --- a/src/hotspot/cpu/s390/templateTable_s390.cpp Wed May 30 12:08:14 2018 +0200 +++ b/src/hotspot/cpu/s390/templateTable_s390.cpp Wed May 30 12:24:00 2018 +0200 @@ -3636,7 +3636,7 @@ NearLabel subtype, no_such_interface; - __ check_klass_subtype(klass, interface, Z_tmp_2, Z_temp_3, subtype); + __ check_klass_subtype(klass, interface, Z_tmp_2, Z_tmp_3, subtype); // If we get here the typecheck failed __ z_bru(no_such_interface); __ bind(subtype); From david.holmes at oracle.com Wed May 30 11:00:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 30 May 2018 21:00:12 +1000 Subject: [hs] RFR (L): 8010319: Implementation of JEP 181: Nest-Based Access Control In-Reply-To: <3d40c5996b2549d9ae18f9a9486b3f50@sap.com> References: <06529fc3-2eba-101b-9aee-2757893cb8fb@oracle.com> <97f8cedf-4ebc-610f-0528-e1b91f35eece@oracle.com> <7aa56a5e-0c95-f1a2-0b27-3f759ca5c702@oracle.com> <3d40c5996b2549d9ae18f9a9486b3f50@sap.com> Message-ID: <8ee54dd5-2e30-93aa-b138-f9ee78834690@oracle.com> Hi Martin, On 30/05/2018 8:26 PM, Doerr, Martin wrote: > Hi David, > > can you apply the following small patch to fix ppc64 and s390 builds, please? Absolutely! Many thanks for testing it out! I will update the v4 webrev at: http://cr.openjdk.java.net/~dholmes/8010319-JEP181/webrev.full.v4/ David > Best regards, > Martin > > > diff -r 70d41ef93cf0 src/hotspot/cpu/ppc/templateTable_ppc_64.cpp > --- a/src/hotspot/cpu/ppc/templateTable_ppc_64.cpp Wed May 30 12:08:14 2018 +0200 > +++ b/src/hotspot/cpu/ppc/templateTable_ppc_64.cpp Wed May 30 12:24:00 2018 +0200 > @@ -3608,10 +3608,10 @@ > __ testbitdi(CCR0, R0, Rflags, ConstantPoolCacheEntry::is_vfinal_shift); > __ bfalse(CCR0, LnotVFinal); > > - __ check_klass_subtype(Rrecv_klass, Rinterface_klass, Rscratch, Rscratch1, subtype); > + __ check_klass_subtype(Rrecv_klass, Rinterface_klass, Rscratch1, Rscratch2, L_subtype); > // If we get here the typecheck failed > __ b(L_no_such_interface); > - __ bind(subtype); > + __ bind(L_subtype); > > // do the call > > diff -r 70d41ef93cf0 src/hotspot/cpu/s390/templateTable_s390.cpp > --- a/src/hotspot/cpu/s390/templateTable_s390.cpp Wed May 30 12:08:14 2018 +0200 > +++ b/src/hotspot/cpu/s390/templateTable_s390.cpp Wed May 30 12:24:00 2018 +0200 > @@ -3636,7 +3636,7 @@ > > NearLabel subtype, no_such_interface; > > - __ check_klass_subtype(klass, interface, Z_tmp_2, Z_temp_3, subtype); > + __ check_klass_subtype(klass, interface, Z_tmp_2, Z_tmp_3, subtype); > // If we get here the typecheck failed > __ z_bru(no_such_interface); > __ bind(subtype); > From coleen.phillimore at oracle.com Wed May 30 12:23:51 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 30 May 2018 08:23:51 -0400 Subject: RFR (M) 8203837: Split nmethod unloading from nmethod cache cleaning Message-ID: <7847553d-0f61-f7ce-146f-1e6663cdca95@oracle.com> Summary: Refactor cleaning inline caches to after GC do_unloading. See CR for more information.? This patch refactors CompiledMethod::do_unloading() to unload nmethods in case of !is_alive oop.? If the nmethod is not unloaded, cleans the inline caches, and exception cache, for unloaded classes and unloaded nmethods.? The CodeCache walk in gc_epilogue is moved earlier to combine with cleanup for class unloading. It doesn't add CodeCache walks to any of the GCs, and keeps the G1 parallel nmethod unloading intact.? This patch also uses common code for CompiledMethod::clean_inline_caches which was duplicated by the G1 functions. The patch also fixed a case in AOT where clear_inline_caches should be called instead of clean_inline_caches.?? I think neither is necessary for the nmethods that are deoptimized because of redefinition, but clear_inline_caches clears up redefined Methods* not for unloaded nmethods.? Once the method is cleaned by the sweeper, clean_inline_caches will be called on it.? clear vs. clean ... The patch also converts TraceScavenge to -Xlog:gc+nmethod=trace.? I can revert this part and do it separately; I had just converted it while looking at the output. open webrev at http://cr.openjdk.java.net/~coleenp/8203837.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8203837 Tested with mach5 hs-tier1-5, the gc-test-suite (including specjbb2015, dacapo, gcbasher), runThese with all GCs with and without class unloading. This is an enhancement that we can use for making nmethod cleaning concurrent in ZGC. Thanks, Coleen From kim.barrett at oracle.com Wed May 30 16:28:07 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 30 May 2018 12:28:07 -0400 Subject: RFR: 8203948: Expand JVMTI callback notion of "internal threads" Message-ID: <28C16D3C-7C8F-44B3-8ED1-A24CAE4EA8AB@oracle.com> Please review this change to JVMTI to relax some restrictions on where some functions may be called. Presently they are restricted to being called from either a Java thread, the VM thread, or a thread for which is_ConcurrentGC_thread() returns true (e.g. ConcurrentGCThreads and AbstractGangWorkers (if the containing WorkGang is configured as concurrent). We now allow these functions to be called for either a Java thread or a Named thread, e.g. any of the VM thread, ConcurrentGCThreads, or WorkerThreads. This allows the restricted functions to be called from any WorkerThreads, rather than only AbstractGangWorkers in concurrent WorkGangs. The change to jvmtiEnter.xsl affects the following generated functions: jvmti_Allocate jvmti_Deallocate jvmti_CreateRawMonitor jvmti_DestroyRawMonitor jvmti_RawMonitorEnter jvmti_RawMonitorExit jvmti_RawMonitorWait jvmti_RawMonitorNotify jvmti_RawMonitorNotifyAll jvmti_GetTimerInfo jvmti_GetTime jvmti_SetEnvironmentLocalStorage jvmti_GetEnvironmentLocalStorage CR: https://bugs.openjdk.java.net/browse/JDK-8203948 Webrev: http://cr.openjdk.java.net/~kbarrett/8203948/open.00/ Testing: Mach5 tier{1,2,3} and hs-tier{4,5} There are no tests that directly exercise the relaxed restrictions yet. We'll start hitting that as part of in-progress work to parallelize WeakProcessor, which calls JvmtiExport::weak_oops_do, which generates ObjectFree events, which uses the Raw Monitor API. From coleen.phillimore at oracle.com Wed May 30 17:22:08 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 30 May 2018 13:22:08 -0400 Subject: RFR: 8203948: Expand JVMTI callback notion of "internal threads" In-Reply-To: <28C16D3C-7C8F-44B3-8ED1-A24CAE4EA8AB@oracle.com> References: <28C16D3C-7C8F-44B3-8ED1-A24CAE4EA8AB@oracle.com> Message-ID: Looks good to me. Coleen On 5/30/18 12:28 PM, Kim Barrett wrote: > Please review this change to JVMTI to relax some restrictions on where > some functions may be called. Presently they are restricted to being > called from either a Java thread, the VM thread, or a thread for which > is_ConcurrentGC_thread() returns true (e.g. ConcurrentGCThreads and > AbstractGangWorkers (if the containing WorkGang is configured as > concurrent). > > We now allow these functions to be called for either a Java thread or > a Named thread, e.g. any of the VM thread, ConcurrentGCThreads, or > WorkerThreads. This allows the restricted functions to be called from > any WorkerThreads, rather than only AbstractGangWorkers in concurrent > WorkGangs. > > The change to jvmtiEnter.xsl affects the following generated > functions: > jvmti_Allocate > jvmti_Deallocate > jvmti_CreateRawMonitor > jvmti_DestroyRawMonitor > jvmti_RawMonitorEnter > jvmti_RawMonitorExit > jvmti_RawMonitorWait > jvmti_RawMonitorNotify > jvmti_RawMonitorNotifyAll > jvmti_GetTimerInfo > jvmti_GetTime > jvmti_SetEnvironmentLocalStorage > jvmti_GetEnvironmentLocalStorage > > CR: > https://bugs.openjdk.java.net/browse/JDK-8203948 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8203948/open.00/ > > Testing: > Mach5 tier{1,2,3} and hs-tier{4,5} > > There are no tests that directly exercise the relaxed restrictions > yet. We'll start hitting that as part of in-progress work to > parallelize WeakProcessor, which calls JvmtiExport::weak_oops_do, > which generates ObjectFree events, which uses the Raw Monitor API. > From coleen.phillimore at oracle.com Wed May 30 17:25:09 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 30 May 2018 13:25:09 -0400 Subject: RFR (S) 8204087: C++ Interpreter code left over in MethodData Message-ID: <0cdbb8db-acf2-0973-b51e-8011a779ec95@oracle.com> Summary: remove unused code Tested with linux-x64 Zero build, and mach5 hs-tier1-2 on 4 Oracle platforms. open webrev at http://cr.openjdk.java.net/~coleenp/8204087.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8204087 The reason for removing this old code is for initial work to support for concurrent weak methodData cleaning, ie, I didn't want to fix this code. Thanks, Coleen From vladimir.kozlov at oracle.com Wed May 30 18:17:12 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Wed, 30 May 2018 11:17:12 -0700 (PDT) Subject: RFR (S) 8204087: C++ Interpreter code left over in MethodData In-Reply-To: <0cdbb8db-acf2-0973-b51e-8011a779ec95@oracle.com> References: <0cdbb8db-acf2-0973-b51e-8011a779ec95@oracle.com> Message-ID: <28c583a1-6ca1-9bfc-6a84-6ad83631f8e7@oracle.com> Good. Thanks, Vladimir On 5/30/18 10:25 AM, coleen.phillimore at oracle.com wrote: > Summary: remove unused code > > Tested with linux-x64 Zero build, and mach5 hs-tier1-2 on 4 Oracle > platforms. > > open webrev at http://cr.openjdk.java.net/~coleenp/8204087.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8204087 > > The reason for removing this old code is for initial work to support for > concurrent weak methodData cleaning, ie, I didn't want to fix this code. > > Thanks, > Coleen From bob.vandette at oracle.com Wed May 30 19:45:40 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Wed, 30 May 2018 15:45:40 -0400 Subject: RFR: 8203357 Container Metrics Message-ID: <510E46BF-A220-40CC-8752-687C849D9114@oracle.com> Please review the following RFE which adds an internal API, along with jtreg tests that provide access to Docker container configuration data and metrics. In addition to the API which we hope to take advantage of in the future with Java Flight Recorder and a JMX Mbean, I?ve added an additional option to -XshowSettings:system than dumps out the container or host cgroup confguration information. See the sample output below: RFE: Container Metrics https://bugs.openjdk.java.net/browse/JDK-8203357 WEBREV: http://cr.openjdk.java.net/~bobv/8203357/webrev.00 This commit will also include a fix for the following bug. BUG: [TESTBUG] Test /runtime/containers/cgroup/PlainRead.java fails https://bugs.openjdk.java.net/browse/JDK-8203691 WEBREV: http://cr.openjdk.java.net/~bobv/8203357/webrev.00/test/hotspot/jtreg/runtime/containers/cgroup/PlainRead.java.sdiff.html SAMPLE USAGE and OUTPUT: docker run ?memory=256m --cpuset-cpus 4-7 -it ubuntu bash ./java -XshowSettings:system Operating System Metrics: Provider: cgroupv1 Effective CPU Count: 4 CPU Period: 100000 CPU Quota: -1 CPU Shares: -1 List of Processors, 4 total: 4 5 6 7 List of Effective Processors, 4 total: 4 5 6 7 List of Memory Nodes, 2 total: 0 1 List of Available Memory Nodes, 2 total: 0 1 CPUSet Memory Pressure Enabled: false Memory Limit: 256.00M Memory Soft Limit: Unlimited Memory & Swap Limit: 512.00M Kernel Memory Limit: Unlimited TCP Memory Limit: Unlimited Out Of Memory Killer Enabled: true TEST RESULTS: testing runtime container APIs Directory "JTwork" not found: creating Passed: runtime/containers/cgroup/PlainRead.java Passed: runtime/containers/docker/DockerBasicTest.java Passed: runtime/containers/docker/TestCPUAwareness.java Passed: runtime/containers/docker/TestCPUSets.java Passed: runtime/containers/docker/TestMemoryAwareness.java Passed: runtime/containers/docker/TestMisc.java Test results: passed: 6 Results written to /export/users/bobv/jdk11/build/jtreg/JTwork testing jdk.internal.platform APIs Passed: jdk/internal/platform/cgroup/TestCgroupMetrics.java Passed: jdk/internal/platform/docker/TestDockerCpuMetrics.java Passed: jdk/internal/platform/docker/TestDockerMemoryMetrics.java Passed: jdk/internal/platform/docker/TestSystemMetrics.java Test results: passed: 4 Results written to /export/users/bobv/jdk11/build/jtreg/JTwork testing -XshowSettings:system launcher option Passed: tools/launcher/Settings.java Test results: passed: 1 Bob. From coleen.phillimore at oracle.com Wed May 30 19:59:45 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Wed, 30 May 2018 15:59:45 -0400 Subject: RFR (S) 8204087: C++ Interpreter code left over in MethodData In-Reply-To: <28c583a1-6ca1-9bfc-6a84-6ad83631f8e7@oracle.com> References: <0cdbb8db-acf2-0973-b51e-8011a779ec95@oracle.com> <28c583a1-6ca1-9bfc-6a84-6ad83631f8e7@oracle.com> Message-ID: <9903e91b-6c4a-39f8-af2e-a80581da37b3@oracle.com> Thank you! Coleen On 5/30/18 2:17 PM, Vladimir Kozlov wrote: > Good. > > Thanks, > Vladimir > > On 5/30/18 10:25 AM, coleen.phillimore at oracle.com wrote: >> Summary: remove unused code >> >> Tested with linux-x64 Zero build, and mach5 hs-tier1-2 on 4 Oracle >> platforms. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8204087.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8204087 >> >> The reason for removing this old code is for initial work to support >> for concurrent weak methodData cleaning, ie, I didn't want to fix >> this code. >> >> Thanks, >> Coleen From lois.foltan at oracle.com Wed May 30 20:00:00 2018 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 30 May 2018 16:00:00 -0400 Subject: RFR (S) 8204087: C++ Interpreter code left over in MethodData In-Reply-To: <0cdbb8db-acf2-0973-b51e-8011a779ec95@oracle.com> References: <0cdbb8db-acf2-0973-b51e-8011a779ec95@oracle.com> Message-ID: Looks good. Lois On 5/30/2018 1:25 PM, coleen.phillimore at oracle.com wrote: > Summary: remove unused code > > Tested with linux-x64 Zero build, and mach5 hs-tier1-2 on 4 Oracle > platforms. > > open webrev at http://cr.openjdk.java.net/~coleenp/8204087.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8204087 > > The reason for removing this old code is for initial work to support > for concurrent weak methodData cleaning, ie, I didn't want to fix this > code. > > Thanks, > Coleen From kim.barrett at oracle.com Wed May 30 20:19:16 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 30 May 2018 16:19:16 -0400 Subject: RFR: 8204097: Simplify OopStorage::AllocateList block entry access Message-ID: <2A6B793E-AD54-430F-8C68-D92F964C0A37@oracle.com> Please review this simplification of OopStorage::AllocateList, removing the no longer used support for blocks being in multiple lists simultaneously. There is now only one list of blocks, the _allocate_list. CR: https://bugs.openjdk.java.net/browse/JDK-8204097 Webrev: http://cr.openjdk.java.net/~kbarrett/8204097/open.00/ Testing: Mach5 tier{1,2,3} From kim.barrett at oracle.com Wed May 30 20:35:31 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 30 May 2018 16:35:31 -0400 Subject: RFR: 8203948: Expand JVMTI callback notion of "internal threads" In-Reply-To: References: <28C16D3C-7C8F-44B3-8ED1-A24CAE4EA8AB@oracle.com> Message-ID: <383B7E05-1555-4044-A6AE-26A82544DDD6@oracle.com> > On May 30, 2018, at 1:22 PM, coleen.phillimore at oracle.com wrote: > > > Looks good to me. > Coleen Thanks. From serguei.spitsyn at oracle.com Wed May 30 22:43:22 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 30 May 2018 15:43:22 -0700 Subject: RFR(L) : 8199371 : [TESTBUG] Open source vm testbase JDWP tests In-Reply-To: <967B07D4-A563-4D86-B82F-02A8043D4CDE@oracle.com> References: <967B07D4-A563-4D86-B82F-02A8043D4CDE@oracle.com> Message-ID: <26a89890-0680-ffe2-a66c-ab8e706cd669@oracle.com> Hi Igor, The fix looks good. At least, I do not see any issues. Thanks, Serguei On 5/29/18 16:22, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8199371/webrev.00/index.html >> 64675 lines changed: 64675 ins; 0 del; 0 mod; > Hi all, > > could you please review this patch which open sources JDWP tests from VM testbase? > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199371 > webrev: http://cr.openjdk.java.net/~iignatyev//8199371/webrev.00/index.html > testing: :vmTestbase_vm_jdwp test group > > Thanks, > -- Igor From serguei.spitsyn at oracle.com Wed May 30 22:50:59 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 30 May 2018 15:50:59 -0700 Subject: RFR(M) : 8199380 : [TESTBUG] Open source VM testbase AOD tests In-Reply-To: <5E87B35F-FC4E-4814-A4A6-CE1E24D3C479@oracle.com> References: <5E87B35F-FC4E-4814-A4A6-CE1E24D3C479@oracle.com> Message-ID: Igor, It looks good. Thank you a lot for taking care about this! Thanks, Serguei On 5/23/18 12:44, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8199380/webrev.00/ >> 2370 lines changed: 2370 ins; 0 del; 0 mod; > Hi all, > > could you please review this patch which open sources AOD tests from VM testbase? these tests test hotspot's attach-on-demand. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8199380 > webrev: http://cr.openjdk.java.net/~iignatyev//8199380/webrev.00/index.html > testing: :vmTestbase_vm_aod test group > > Thanks, > -- Igor From serguei.spitsyn at oracle.com Wed May 30 22:57:02 2018 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 30 May 2018 15:57:02 -0700 Subject: RFR: 8203031: segfaults from jvmti_AddToBootstrapClassLoaderSearch In-Reply-To: <6fafe794-a209-25b7-b65c-509e09c1b23f@oracle.com> References: <6fafe794-a209-25b7-b65c-509e09c1b23f@oracle.com> Message-ID: <12bbd950-9f14-6d20-4e47-a85f74bf75c9@oracle.com> Hi Alex, The fix looks good to me. It'd be nice if someone from the CDS/App-CDS experts looked at this. I've extended the mailing list. Thanks, Serguei On 5/30/18 11:52, Alex Menkov wrote: > Hi all, > > Please review a fix for > bug: https://bugs.openjdk.java.net/browse/JDK-8203031 > fix: http://cr.openjdk.java.net/~amenkov/addToBootstrapCLSearch/webrev/ > > The issue is a regression from JDK-8193213 (Make the UseAppCDS option > obsolete) > > FileMapInfo::current_info() is initialized by FileMapInfo ctor only > when UseSharedSpaces == true: > Metaspace::global_initialize() > ?-> MetaspaceShared::initialize_runtime_shared_and_meta_spaces > ?? -> new FileMapInfo() > > --alex From ioi.lam at oracle.com Wed May 30 23:29:03 2018 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 30 May 2018 16:29:03 -0700 Subject: RFR: 8203031: segfaults from jvmti_AddToBootstrapClassLoaderSearch In-Reply-To: <12bbd950-9f14-6d20-4e47-a85f74bf75c9@oracle.com> References: <6fafe794-a209-25b7-b65c-509e09c1b23f@oracle.com> <12bbd950-9f14-6d20-4e47-a85f74bf75c9@oracle.com> Message-ID: <465ccbf5-a1fa-53fb-daaa-400b888981e5@oracle.com> Hi Alex, The changes look good. Thank you so much for fixing it. - Ioi On 5/30/18 3:57 PM, serguei.spitsyn at oracle.com wrote: > Hi Alex, > > The fix looks good to me. > > It'd be nice if someone from the CDS/App-CDS experts looked at this. > I've extended the mailing list. > > Thanks, > Serguei > > > On 5/30/18 11:52, Alex Menkov wrote: >> Hi all, >> >> Please review a fix for >> bug: https://bugs.openjdk.java.net/browse/JDK-8203031 >> fix: http://cr.openjdk.java.net/~amenkov/addToBootstrapCLSearch/webrev/ >> >> The issue is a regression from JDK-8193213 (Make the UseAppCDS option >> obsolete) >> >> FileMapInfo::current_info() is initialized by FileMapInfo ctor only >> when UseSharedSpaces == true: >> Metaspace::global_initialize() >> ?-> MetaspaceShared::initialize_runtime_shared_and_meta_spaces >> ?? -> new FileMapInfo() >> >> --alex > From jiangli.zhou at oracle.com Thu May 31 00:36:57 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Wed, 30 May 2018 17:36:57 -0700 Subject: RFR: 8203031: segfaults from jvmti_AddToBootstrapClassLoaderSearch In-Reply-To: <465ccbf5-a1fa-53fb-daaa-400b888981e5@oracle.com> References: <6fafe794-a209-25b7-b65c-509e09c1b23f@oracle.com> <12bbd950-9f14-6d20-4e47-a85f74bf75c9@oracle.com> <465ccbf5-a1fa-53fb-daaa-400b888981e5@oracle.com> Message-ID: <239AFD8B-5A55-4DE0-86DA-CE17311DB80A@oracle.com> +1 Thanks for fixing it! Jiangli > On May 30, 2018, at 4:29 PM, Ioi Lam wrote: > > Hi Alex, > > The changes look good. Thank you so much for fixing it. > > - Ioi > > >> On 5/30/18 3:57 PM, serguei.spitsyn at oracle.com wrote: >> Hi Alex, >> >> The fix looks good to me. >> >> It'd be nice if someone from the CDS/App-CDS experts looked at this. >> I've extended the mailing list. >> >> Thanks, >> Serguei >> >> >>> On 5/30/18 11:52, Alex Menkov wrote: >>> Hi all, >>> >>> Please review a fix for >>> bug: https://bugs.openjdk.java.net/browse/JDK-8203031 >>> fix: http://cr.openjdk.java.net/~amenkov/addToBootstrapCLSearch/webrev/ >>> >>> The issue is a regression from JDK-8193213 (Make the UseAppCDS option obsolete) >>> >>> FileMapInfo::current_info() is initialized by FileMapInfo ctor only when UseSharedSpaces == true: >>> Metaspace::global_initialize() >>> -> MetaspaceShared::initialize_runtime_shared_and_meta_spaces >>> -> new FileMapInfo() >>> >>> --alex >> > From david.holmes at oracle.com Thu May 31 02:18:31 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 31 May 2018 12:18:31 +1000 Subject: RFR: 8203948: Expand JVMTI callback notion of "internal threads" In-Reply-To: <28C16D3C-7C8F-44B3-8ED1-A24CAE4EA8AB@oracle.com> References: <28C16D3C-7C8F-44B3-8ED1-A24CAE4EA8AB@oracle.com> Message-ID: <28b38b3c-c17a-b5f3-6bcc-ce872af975e7@oracle.com> Hi Kim, I guess I'd like this more if NamedThread were called something like InternalThread as that captures the actual commonality of these threads (we don't really care they they have names!). But that's not your fault. I'm still tempted to just remove the check altogether now there is only one thread we're excluding - do we really need to guard against the WatcherThread using this code? But its fine to push as-is. Thanks, David On 31/05/2018 2:28 AM, Kim Barrett wrote: > Please review this change to JVMTI to relax some restrictions on where > some functions may be called. Presently they are restricted to being > called from either a Java thread, the VM thread, or a thread for which > is_ConcurrentGC_thread() returns true (e.g. ConcurrentGCThreads and > AbstractGangWorkers (if the containing WorkGang is configured as > concurrent). > > We now allow these functions to be called for either a Java thread or > a Named thread, e.g. any of the VM thread, ConcurrentGCThreads, or > WorkerThreads. This allows the restricted functions to be called from > any WorkerThreads, rather than only AbstractGangWorkers in concurrent > WorkGangs. > > The change to jvmtiEnter.xsl affects the following generated > functions: > jvmti_Allocate > jvmti_Deallocate > jvmti_CreateRawMonitor > jvmti_DestroyRawMonitor > jvmti_RawMonitorEnter > jvmti_RawMonitorExit > jvmti_RawMonitorWait > jvmti_RawMonitorNotify > jvmti_RawMonitorNotifyAll > jvmti_GetTimerInfo > jvmti_GetTime > jvmti_SetEnvironmentLocalStorage > jvmti_GetEnvironmentLocalStorage > > CR: > https://bugs.openjdk.java.net/browse/JDK-8203948 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8203948/open.00/ > > Testing: > Mach5 tier{1,2,3} and hs-tier{4,5} > > There are no tests that directly exercise the relaxed restrictions > yet. We'll start hitting that as part of in-progress work to > parallelize WeakProcessor, which calls JvmtiExport::weak_oops_do, > which generates ObjectFree events, which uses the Raw Monitor API. > From mikhailo.seledtsov at oracle.com Thu May 31 03:41:06 2018 From: mikhailo.seledtsov at oracle.com (Mikhailo Seledtsov) Date: Wed, 30 May 2018 20:41:06 -0700 Subject: RFR(L) : 8199371 : [TESTBUG] Open source vm testbase JDWP tests In-Reply-To: <26a89890-0680-ffe2-a66c-ab8e706cd669@oracle.com> References: <967B07D4-A563-4D86-B82F-02A8043D4CDE@oracle.com> <26a89890-0680-ffe2-a66c-ab8e706cd669@oracle.com> Message-ID: <5B0F6ED2.10005@oracle.com> +1 Misha On 5/30/18, 3:43 PM, serguei.spitsyn at oracle.com wrote: > Hi Igor, > > The fix looks good. > At least, I do not see any issues. > > Thanks, > Serguei > > > On 5/29/18 16:22, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8199371/webrev.00/index.html >>> 64675 lines changed: 64675 ins; 0 del; 0 mod; >> Hi all, >> >> could you please review this patch which open sources JDWP tests from >> VM testbase? >> >> As usually w/ VM testbase code, these tests are old, they have been >> run in hotspot testing for a long period of time. Originally, these >> tests were run by a test harness different from jtreg and had >> different build and execution schemes, some parts couldn't be easily >> translated to jtreg, so tests might have actions or pieces of code >> which look weird. In a long term, we are planning to rework them. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8199371 >> webrev: >> http://cr.openjdk.java.net/~iignatyev//8199371/webrev.00/index.html >> testing: :vmTestbase_vm_jdwp test group >> >> Thanks, >> -- Igor > From rohitarulraj at gmail.com Thu May 31 04:55:51 2018 From: rohitarulraj at gmail.com (Rohit Arul Raj) Date: Thu, 31 May 2018 10:25:51 +0530 Subject: RFC: C2 Object Initialization - Using XMM/YMM registers In-Reply-To: <6ddfaac5-a5ad-dc27-8185-d2237a5b1695@oracle.com> References: <6ddfaac5-a5ad-dc27-8185-d2237a5b1695@oracle.com> Message-ID: Thanks Vladimir, I made the changes as you had suggested and it works now. Please find attached the updated patch, relevant test case as well as the micro-benchmark performance data. Sorry for the delay. **************** P A T C H ************** diff --git a/src/hotspot/cpu/x86/globals_x86.hpp b/src/hotspot/cpu/x86/globals_x86.hpp --- a/src/hotspot/cpu/x86/globals_x86.hpp +++ b/src/hotspot/cpu/x86/globals_x86.hpp @@ -150,6 +150,9 @@ product(bool, UseUnalignedLoadStores, false, \ "Use SSE2 MOVDQU instruction for Arraycopy") \ \ + product(bool, UseXMMForObjInit, false, \ + "Use XMM/YMM MOVDQU instruction for Object Initialization") \ + \ product(bool, UseFastStosb, false, \ "Use fast-string operation for zeroing: rep stosb") \ \ diff --git a/src/hotspot/cpu/x86/macroAssembler_x86.cpp b/src/hotspot/cpu/x86/macroAssembler_x86.cpp --- a/src/hotspot/cpu/x86/macroAssembler_x86.cpp +++ b/src/hotspot/cpu/x86/macroAssembler_x86.cpp @@ -6775,7 +6775,58 @@ } -void MacroAssembler::clear_mem(Register base, Register cnt, Register tmp, bool is_large) { +// clear memory of size 'cnt' qwords, starting at 'base' using XMM/YMM registers +void MacroAssembler::xmm_clear_mem(Register base, Register cnt, XMMRegister xtmp) { + // cnt - number of qwords (8-byte words). + // base - start address, qword aligned. + Label L_zero_64_bytes, L_loop, L_sloop, L_tail, L_end; + if (UseAVX >= 2) + vpxor(xtmp, xtmp, xtmp, AVX_256bit); + else + vpxor(xtmp, xtmp, xtmp, AVX_128bit); + jmp(L_zero_64_bytes); + + BIND(L_loop); + if (UseAVX >= 2) { + vmovdqu(Address(base, 0), xtmp); + vmovdqu(Address(base, 32), xtmp); + } else { + movdqu(Address(base, 0), xtmp); + movdqu(Address(base, 16), xtmp); + movdqu(Address(base, 32), xtmp); + movdqu(Address(base, 48), xtmp); + } + addptr(base, 64); + + BIND(L_zero_64_bytes); + subptr(cnt, 8); + jccb(Assembler::greaterEqual, L_loop); + addptr(cnt, 4); + jccb(Assembler::less, L_tail); + // Copy trailing 32 bytes + if (UseAVX >= 2) { + vmovdqu(Address(base, 0), xtmp); + } else { + movdqu(Address(base, 0), xtmp); + movdqu(Address(base, 16), xtmp); + } + addptr(base, 32); + subptr(cnt, 4); + + BIND(L_tail); + addptr(cnt, 4); + jccb(Assembler::lessEqual, L_end); + decrement(cnt); + + BIND(L_sloop); + movq(Address(base, 0), xtmp); + addptr(base, 8); + decrement(cnt); + jccb(Assembler::greaterEqual, L_sloop); + BIND(L_end); +} + +void MacroAssembler::clear_mem(Register base, Register cnt, Register tmp, XMMRegister xtmp, bool is_large) { // cnt - number of qwords (8-byte words). // base - start address, qword aligned. // is_large - if optimizers know cnt is larger than InitArrayShortSize @@ -6787,7 +6838,9 @@ Label DONE; - xorptr(tmp, tmp); + if (!is_large || !(UseXMMForObjInit && UseUnalignedLoadStores)) { + xorptr(tmp, tmp); + } if (!is_large) { Label LOOP, LONG; @@ -6813,6 +6866,9 @@ if (UseFastStosb) { shlptr(cnt, 3); // convert to number of bytes rep_stosb(); + } else if (UseXMMForObjInit && UseUnalignedLoadStores) { + movptr(tmp, base); + xmm_clear_mem(tmp, cnt, xtmp); } else { NOT_LP64(shlptr(cnt, 1);) // convert to number of 32-bit words for 32-bit VM rep_stos(); diff --git a/src/hotspot/cpu/x86/macroAssembler_x86.hpp b/src/hotspot/cpu/x86/macroAssembler_x86.hpp --- a/src/hotspot/cpu/x86/macroAssembler_x86.hpp +++ b/src/hotspot/cpu/x86/macroAssembler_x86.hpp @@ -1578,7 +1578,10 @@ // clear memory of size 'cnt' qwords, starting at 'base'; // if 'is_large' is set, do not try to produce short loop - void clear_mem(Register base, Register cnt, Register rtmp, bool is_large); + void clear_mem(Register base, Register cnt, Register rtmp, XMMRegister xtmp, bool is_large); + + // clear memory of size 'cnt' qwords, starting at 'base' using XMM/YMM registers + void xmm_clear_mem(Register base, Register cnt, XMMRegister xtmp); #ifdef COMPILER2 void string_indexof_char(Register str1, Register cnt1, Register ch, Register result, diff --git a/src/hotspot/cpu/x86/x86_32.ad b/src/hotspot/cpu/x86/x86_32.ad --- a/src/hotspot/cpu/x86/x86_32.ad +++ b/src/hotspot/cpu/x86/x86_32.ad @@ -11482,13 +11482,15 @@ // ======================================================================= // fast clearing of an array -instruct rep_stos(eCXRegI cnt, eDIRegP base, eAXRegI zero, Universe dummy, eFlagsReg cr) %{ +instruct rep_stos(eCXRegI cnt, eDIRegP base, regD tmp, eAXRegI zero, Universe dummy, eFlagsReg cr) %{ predicate(!((ClearArrayNode*)n)->is_large()); match(Set dummy (ClearArray cnt base)); - effect(USE_KILL cnt, USE_KILL base, KILL zero, KILL cr); + effect(USE_KILL cnt, USE_KILL base, TEMP tmp, KILL zero, KILL cr); format %{ $$template - $$emit$$"XOR EAX,EAX\t# ClearArray:\n\t" + if (!is_large || !(UseXMMForObjInit && UseUnalignedLoadStores)) { + $$emit$$"XOR EAX,EAX\t# ClearArray:\n\t" + } $$emit$$"CMP InitArrayShortSize,rcx\n\t" $$emit$$"JG LARGE\n\t" $$emit$$"SHL ECX, 1\n\t" @@ -11502,6 +11504,32 @@ if (UseFastStosb) { $$emit$$"SHL ECX,3\t# Convert doublewords to bytes\n\t" $$emit$$"REP STOSB\t# store EAX into [EDI++] while ECX--\n\t" + } else if (UseXMMForObjInit && UseUnalignedLoadStores) { + $$emit$$"MOV RDI,RAX\n\t" + $$emit$$"VPXOR YMM0,YMM0,YMM0\n\t" + $$emit$$"JMPQ L_zero_64_bytes\n\t" + $$emit$$"# L_loop:\t# 64-byte LOOP\n\t" + $$emit$$"VMOVDQU YMM0,(RAX)\n\t" + $$emit$$"VMOVDQU YMM0,0x20(RAX)\n\t" + $$emit$$"ADD 0x40,RAX\n\t" + $$emit$$"# L_zero_64_bytes:\n\t" + $$emit$$"SUB 0x8,RCX\n\t" + $$emit$$"JGE L_loop\n\t" + $$emit$$"ADD 0x4,RCX\n\t" + $$emit$$"JL L_tail\n\t" + $$emit$$"VMOVDQU YMM0,(RAX)\n\t" + $$emit$$"ADD 0x20,RAX\n\t" + $$emit$$"SUB 0x4,RCX\n\t" + $$emit$$"# L_tail:\t# Clearing tail bytes\n\t" + $$emit$$"ADD 0x4,RCX\n\t" + $$emit$$"JLE L_end\n\t" + $$emit$$"DEC RCX\n\t" + $$emit$$"# L_sloop:\t# 8-byte short loop\n\t" + $$emit$$"VMOVQ XMM0,(RAX)\n\t" + $$emit$$"ADD 0x8,RAX\n\t" + $$emit$$"DEC RCX\n\t" + $$emit$$"JGE L_sloop\n\t" + $$emit$$"# L_end:\n\t" } else { $$emit$$"SHL ECX,1\t# Convert doublewords to words\n\t" $$emit$$"REP STOS\t# store EAX into [EDI++] while ECX--\n\t" @@ -11509,20 +11537,49 @@ $$emit$$"# DONE" %} ins_encode %{ - __ clear_mem($base$$Register, $cnt$$Register, $zero$$Register, false); - %} - ins_pipe( pipe_slow ); -%} - -instruct rep_stos_large(eCXRegI cnt, eDIRegP base, eAXRegI zero, Universe dummy, eFlagsReg cr) %{ + __ clear_mem($base$$Register, $cnt$$Register, $zero$$Register, + $tmp$$XMMRegister, false); + %} + ins_pipe( pipe_slow ); +%} + +instruct rep_stos_large(eCXRegI cnt, eDIRegP base, regD tmp, eAXRegI zero, Universe dummy, eFlagsReg cr) %{ predicate(((ClearArrayNode*)n)->is_large()); match(Set dummy (ClearArray cnt base)); - effect(USE_KILL cnt, USE_KILL base, KILL zero, KILL cr); + effect(USE_KILL cnt, USE_KILL base, TEMP tmp, KILL zero, KILL cr); format %{ $$template - $$emit$$"XOR EAX,EAX\t# ClearArray:\n\t" + if (!is_large || !(UseXMMForObjInit && UseUnalignedLoadStores)) { + $$emit$$"XOR EAX,EAX\t# ClearArray:\n\t" + } if (UseFastStosb) { $$emit$$"SHL ECX,3\t# Convert doublewords to bytes\n\t" $$emit$$"REP STOSB\t# store EAX into [EDI++] while ECX--\n\t" + } else if (UseXMMForObjInit && UseUnalignedLoadStores) { + $$emit$$"MOV RDI,RAX\n\t" + $$emit$$"VPXOR YMM0,YMM0,YMM0\n\t" + $$emit$$"JMPQ L_zero_64_bytes\n\t" + $$emit$$"# L_loop:\t# 64-byte LOOP\n\t" + $$emit$$"VMOVDQU YMM0,(RAX)\n\t" + $$emit$$"VMOVDQU YMM0,0x20(RAX)\n\t" + $$emit$$"ADD 0x40,RAX\n\t" + $$emit$$"# L_zero_64_bytes:\n\t" + $$emit$$"SUB 0x8,RCX\n\t" + $$emit$$"JGE L_loop\n\t" + $$emit$$"ADD 0x4,RCX\n\t" + $$emit$$"JL L_tail\n\t" + $$emit$$"VMOVDQU YMM0,(RAX)\n\t" + $$emit$$"ADD 0x20,RAX\n\t" + $$emit$$"SUB 0x4,RCX\n\t" + $$emit$$"# L_tail:\t# Clearing tail bytes\n\t" + $$emit$$"ADD 0x4,RCX\n\t" + $$emit$$"JLE L_end\n\t" + $$emit$$"DEC RCX\n\t" + $$emit$$"# L_sloop:\t# 8-byte short loop\n\t" + $$emit$$"VMOVQ XMM0,(RAX)\n\t" + $$emit$$"ADD 0x8,RAX\n\t" + $$emit$$"DEC RCX\n\t" + $$emit$$"JGE L_sloop\n\t" + $$emit$$"# L_end:\n\t" } else { $$emit$$"SHL ECX,1\t# Convert doublewords to words\n\t" $$emit$$"REP STOS\t# store EAX into [EDI++] while ECX--\n\t" @@ -11530,7 +11587,8 @@ $$emit$$"# DONE" %} ins_encode %{ - __ clear_mem($base$$Register, $cnt$$Register, $zero$$Register, true); + __ clear_mem($base$$Register, $cnt$$Register, $zero$$Register, + $tmp$$XMMRegister, true); %} ins_pipe( pipe_slow ); %} diff --git a/src/hotspot/cpu/x86/x86_64.ad b/src/hotspot/cpu/x86/x86_64.ad --- a/src/hotspot/cpu/x86/x86_64.ad +++ b/src/hotspot/cpu/x86/x86_64.ad @@ -10625,15 +10625,17 @@ // ======================================================================= // fast clearing of an array -instruct rep_stos(rcx_RegL cnt, rdi_RegP base, rax_RegI zero, Universe dummy, - rFlagsReg cr) +instruct rep_stos(rcx_RegL cnt, rdi_RegP base, regD tmp, rax_RegI zero, + Universe dummy, rFlagsReg cr) %{ predicate(!((ClearArrayNode*)n)->is_large()); match(Set dummy (ClearArray cnt base)); - effect(USE_KILL cnt, USE_KILL base, KILL zero, KILL cr); + effect(USE_KILL cnt, USE_KILL base, TEMP tmp, KILL zero, KILL cr); format %{ $$template - $$emit$$"xorq rax, rax\t# ClearArray:\n\t" + if (!is_large || !(UseXMMForObjInit && UseUnalignedLoadStores)) { + $$emit$$"xorq rax, rax\t# ClearArray:\n\t" + } $$emit$$"cmp InitArrayShortSize,rcx\n\t" $$emit$$"jg LARGE\n\t" $$emit$$"dec rcx\n\t" @@ -10646,35 +10648,91 @@ if (UseFastStosb) { $$emit$$"shlq rcx,3\t# Convert doublewords to bytes\n\t" $$emit$$"rep stosb\t# Store rax to *rdi++ while rcx--\n\t" + } else if (UseXMMForObjInit && UseUnalignedLoadStores) { + $$emit$$"mov rdi,rax\n\t" + $$emit$$"vpxor ymm0,ymm0,ymm0\n\t" + $$emit$$"jmpq L_zero_64_bytes\n\t" + $$emit$$"# L_loop:\t# 64-byte LOOP\n\t" + $$emit$$"vmovdqu ymm0,(rax)\n\t" + $$emit$$"vmovdqu ymm0,0x20(rax)\n\t" + $$emit$$"add 0x40,rax\n\t" + $$emit$$"# L_zero_64_bytes:\n\t" + $$emit$$"sub 0x8,rcx\n\t" + $$emit$$"jge L_loop\n\t" + $$emit$$"add 0x4,rcx\n\t" + $$emit$$"jl L_tail\n\t" + $$emit$$"vmovdqu ymm0,(rax)\n\t" + $$emit$$"add 0x20,rax\n\t" + $$emit$$"sub 0x4,rcx\n\t" + $$emit$$"# L_tail:\t# Clearing tail bytes\n\t" + $$emit$$"add 0x4,rcx\n\t" + $$emit$$"jle L_end\n\t" + $$emit$$"dec rcx\n\t" + $$emit$$"# L_sloop:\t# 8-byte short loop\n\t" + $$emit$$"vmovq xmm0,(rax)\n\t" + $$emit$$"add 0x8,rax\n\t" + $$emit$$"dec rcx\n\t" + $$emit$$"jge L_sloop\n\t" + $$emit$$"# L_end:\n\t" } else { $$emit$$"rep stosq\t# Store rax to *rdi++ while rcx--\n\t" } $$emit$$"# DONE" %} ins_encode %{ - __ clear_mem($base$$Register, $cnt$$Register, $zero$$Register, false); + __ clear_mem($base$$Register, $cnt$$Register, $zero$$Register, + $tmp$$XMMRegister, false); %} ins_pipe(pipe_slow); %} -instruct rep_stos_large(rcx_RegL cnt, rdi_RegP base, rax_RegI zero, Universe dummy, - rFlagsReg cr) +instruct rep_stos_large(rcx_RegL cnt, rdi_RegP base, regD tmp, rax_RegI zero, + Universe dummy, rFlagsReg cr) %{ predicate(((ClearArrayNode*)n)->is_large()); match(Set dummy (ClearArray cnt base)); - effect(USE_KILL cnt, USE_KILL base, KILL zero, KILL cr); + effect(USE_KILL cnt, USE_KILL base, TEMP tmp, KILL zero, KILL cr); format %{ $$template - $$emit$$"xorq rax, rax\t# ClearArray:\n\t" + if (!is_large || !(UseXMMForObjInit && UseUnalignedLoadStores)) { + $$emit$$"xorq rax, rax\t# ClearArray:\n\t" + } if (UseFastStosb) { $$emit$$"shlq rcx,3\t# Convert doublewords to bytes\n\t" $$emit$$"rep stosb\t# Store rax to *rdi++ while rcx--" + } else if (UseXMMForObjInit && UseUnalignedLoadStores) { + $$emit$$"mov rdi,rax\n\t" + $$emit$$"vpxor ymm0,ymm0,ymm0\n\t" + $$emit$$"jmpq L_zero_64_bytes\n\t" + $$emit$$"# L_loop:\t# 64-byte LOOP\n\t" + $$emit$$"vmovdqu ymm0,(rax)\n\t" + $$emit$$"vmovdqu ymm0,0x20(rax)\n\t" + $$emit$$"add 0x40,rax\n\t" + $$emit$$"# L_zero_64_bytes:\n\t" + $$emit$$"sub 0x8,rcx\n\t" + $$emit$$"jge L_loop\n\t" + $$emit$$"add 0x4,rcx\n\t" + $$emit$$"jl L_tail\n\t" + $$emit$$"vmovdqu ymm0,(rax)\n\t" + $$emit$$"add 0x20,rax\n\t" + $$emit$$"sub 0x4,rcx\n\t" + $$emit$$"# L_tail:\t# Clearing tail bytes\n\t" + $$emit$$"add 0x4,rcx\n\t" + $$emit$$"jle L_end\n\t" + $$emit$$"dec rcx\n\t" + $$emit$$"# L_sloop:\t# 8-byte short loop\n\t" + $$emit$$"vmovq xmm0,(rax)\n\t" + $$emit$$"add 0x8,rax\n\t" + $$emit$$"dec rcx\n\t" + $$emit$$"jge L_sloop\n\t" + $$emit$$"# L_end:\n\t" } else { $$emit$$"rep stosq\t# Store rax to *rdi++ while rcx--" } %} ins_encode %{ - __ clear_mem($base$$Register, $cnt$$Register, $zero$$Register, true); + __ clear_mem($base$$Register, $cnt$$Register, $zero$$Register, + $tmp$$XMMRegister, true); %} ins_pipe(pipe_slow); %} *********************** END of P A T C H ******************* Generated assembly code after change: ------------------------------------------------------ 0x00002b771c0016e4: mov %rdx,%rdi 0x00002b771c0016e7: add $0x10,%rdi 0x00002b771c0016eb: mov $0x14,%ecx 0x00002b771c0016f0: mov %rdi,%rax 0x00002b771c0016f3: vpxor %ymm0,%ymm0,%ymm0 0x00002b771c0016f7: jmpq 0x00002b771c001709 0x00002b771c0016fc: vmovdqu %ymm0,(%rax) 0x00002b771c001700: vmovdqu %ymm0,0x20(%rax) 0x00002b771c001705: add $0x40,%rax 0x00002b771c001709: sub $0x8,%rcx 0x00002b771c00170d: jge 0x00002b771c0016fc 0x00002b771c00170f: add $0x4,%rcx 0x00002b771c001713: jl 0x00002b771c001721 0x00002b771c001715: vmovdqu %ymm0,(%rax) 0x00002b771c001719: add $0x20,%rax 0x00002b771c00171d: sub $0x4,%rcx 0x00002b771c001721: add $0x4,%rcx 0x00002b771c001725: jle 0x00002b771c001737 0x00002b771c001727: dec %rcx 0x00002b771c00172a: vmovq %xmm0,(%rax) 0x00002b771c00172e: add $0x8,%rax 0x00002b771c001732: dec %rcx 0x00002b771c001735: jge 0x00002b771c00172a 0x00002b771c001737: I have done regression testing (changeset: 50250:04f9bb270ab8/24May2018) on 32-bit as well as 64-bit builds and didn't find any regressions. $make run-test TEST="tier1 tier2" JTREG="JOBS=1" CONF=linux-x86_64-normal-server-release Please let me know your comments. Regards, Rohit On Tue, Apr 24, 2018 at 12:33 AM, Vladimir Kozlov wrote: > Sorry for delay. > > In general you can't use arbitrary registers without letting know JIT > compilers that you use it. It will definitely cause problems. > You need to pass it as additional XMMRegister argument and described it as > TEMP in .ad files. > > See byte_array_inflate() as example. > > > On 4/11/18 7:25 PM, Rohit Arul Raj wrote: >>>> >>>> When I use XMM0 as a temporary register, the micro-benchmark crashes. >>>> Saving and Restoring the XMM0 register before and after use works >>>> fine. >>>> >>>> Looking at the "hotspot/src/cpu/x86/vm/x86.ad" file, XMM0 as with >>>> other XMM registers has been mentioned as Save-On-Call registers and >>>> on Linux ABI, no register is preserved across function calls though >>>> XMM0-XMM7 might hold parameters. So I assumed using XMM0 without >>>> saving/restoring should be fine. >>>> >>>> Is it incorrect use XMM* registers without saving/restoring them? >>>> Using XMM10 register as temporary register works fine without having >>>> to save and restore it. >> >> >> Any comments/suggestions on the usage of XMM* registers? >> >> Thanks, >> Rohit >> >> On Thu, Apr 5, 2018 at 11:38 PM, Vladimir Kozlov >> wrote: >>> >>> Good suggestion, Rohit >>> >>> I created new RFE. Please add you suggestion and performance data there: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8201193 >>> >>> Thanks, >>> Vladimir >>> >>> >>> On 4/5/18 12:19 AM, Rohit Arul Raj wrote: >>>> >>>> >>>> Hi All, >>>> >>>> I was going through the C2 object initialization (zeroing) code based >>>> on the below bug entry: >>>> https://bugs.openjdk.java.net/browse/JDK-8146801 >>>> >>>> Right now, for longer lengths we use "rep stos" instructions on x86. I >>>> was experimenting with using XMM/YMM registers (on AMD EPYC processor) >>>> and found that they do improve performance for certain lengths: >>>> >>>> For lengths > 64 bytes - 512 bytes : improvement is in the range of 8% >>>> to >>>> 44% >>>> For lengths > 512bytes : some lengths show slight >>>> improvement in the range of 2% to 7%, others almost same as "rep stos" >>>> numbers. >>>> >>>> I have attached the complete performance data (data.txt) for reference . >>>> Can we add this as an user option similar to UseXMMForArrayCopy? >>>> >>>> I have used the same test case as in >>>> (http://cr.openjdk.java.net/~shade/8146801/benchmarks.jar) with >>>> additional sizes. >>>> >>>> Initial Patch: >>>> I haven't added the check for 32-bit mode as I need some help with the >>>> code (description given below the patch). >>>> The code is similar to the one used in array copy stubs >>>> (copy_bytes_forward). >>>> >>>> diff --git a/src/hotspot/cpu/x86/globals_x86.hpp >>>> b/src/hotspot/cpu/x86/globals_x86.hpp >>>> --- a/src/hotspot/cpu/x86/globals_x86.hpp >>>> +++ b/src/hotspot/cpu/x86/globals_x86.hpp >>>> @@ -150,6 +150,9 @@ >>>> product(bool, UseUnalignedLoadStores, false, >>>> \ >>>> "Use SSE2 MOVDQU instruction for Arraycopy") >>>> \ >>>> >>>> \ >>>> + product(bool, UseXMMForObjInit, false, >>>> \ >>>> + "Use XMM/YMM MOVDQU instruction for Object Initialization") >>>> \ >>>> + >>>> \ >>>> product(bool, UseFastStosb, false, >>>> \ >>>> "Use fast-string operation for zeroing: rep stosb") >>>> \ >>>> >>>> \ >>>> diff --git a/src/hotspot/cpu/x86/macroAssembler_x86.cpp >>>> b/src/hotspot/cpu/x86/macroAssembler_x86.cpp >>>> --- a/src/hotspot/cpu/x86/macroAssembler_x86.cpp >>>> +++ b/src/hotspot/cpu/x86/macroAssembler_x86.cpp >>>> @@ -7106,6 +7106,56 @@ >>>> if (UseFastStosb) { >>>> shlptr(cnt, 3); // convert to number of bytes >>>> rep_stosb(); >>>> + } else if (UseXMMForObjInit && UseUnalignedLoadStores) { >>>> + Label L_loop, L_sloop, L_check, L_tail, L_end; >>>> + push(base); >>>> + if (UseAVX >= 2) >>>> + vpxor(xmm10, xmm10, xmm10, AVX_256bit); >>>> + else >>>> + vpxor(xmm10, xmm10, xmm10, AVX_128bit); >>>> + >>>> + jmp(L_check); >>>> + >>>> + BIND(L_loop); >>>> + if (UseAVX >= 2) { >>>> + vmovdqu(Address(base, 0), xmm10); >>>> + vmovdqu(Address(base, 32), xmm10); >>>> + } else { >>>> + movdqu(Address(base, 0), xmm10); >>>> + movdqu(Address(base, 16), xmm10); >>>> + movdqu(Address(base, 32), xmm10); >>>> + movdqu(Address(base, 48), xmm10); >>>> + } >>>> + addptr(base, 64); >>>> + >>>> + BIND(L_check); >>>> + subptr(cnt, 8); >>>> + jccb(Assembler::greaterEqual, L_loop); >>>> + addptr(cnt, 4); >>>> + jccb(Assembler::less, L_tail); >>>> + // Copy trailing 32 bytes >>>> + if (UseAVX >= 2) { >>>> + vmovdqu(Address(base, 0), xmm10); >>>> + } else { >>>> + movdqu(Address(base, 0), xmm10); >>>> + movdqu(Address(base, 16), xmm10); >>>> + } >>>> + addptr(base, 32); >>>> + subptr(cnt, 4); >>>> + >>>> + BIND(L_tail); >>>> + addptr(cnt, 4); >>>> + jccb(Assembler::lessEqual, L_end); >>>> + decrement(cnt); >>>> + >>>> + BIND(L_sloop); >>>> + movptr(Address(base, 0), tmp); >>>> + addptr(base, 8); >>>> + decrement(cnt); >>>> + jccb(Assembler::greaterEqual, L_sloop); >>>> + >>>> + BIND(L_end); >>>> + pop(base); >>>> } else { >>>> NOT_LP64(shlptr(cnt, 1);) // convert to number of 32-bit words >>>> for 32-bit VM >>>> rep_stos(); >>>> >>>> >>>> When I use XMM0 as a temporary register, the micro-benchmark crashes. >>>> Saving and Restoring the XMM0 register before and after use works >>>> fine. >>>> >>>> Looking at the "hotspot/src/cpu/x86/vm/x86.ad" file, XMM0 as with >>>> other XMM registers has been mentioned as Save-On-Call registers and >>>> on Linux ABI, no register is preserved across function calls though >>>> XMM0-XMM7 might hold parameters. So I assumed using XMM0 without >>>> saving/restoring should be fine. >>>> >>>> Is it incorrect use XMM* registers without saving/restoring them? >>>> Using XMM10 register as temporary register works fine without having >>>> to save and restore it. >>>> >>>> Please let me know your comments. >>>> >>>> Regards, >>>> Rohit >>>> >>> > -------------- next part -------------- +------+-----------+------------------------------------------------------+--------------------------------------------------------------------------+ | | Array | Total | JDK11 trunk code ns/op | JDK11 trunk - YMM 64b loop ns/op (UseAVX=2) | | S.No | Size | Size +-----------+----------+-----------+----------+--------------------------------------------------------------------------+ | | |in bytes| Const | variance | Variable | variance | Const | variance | Variable | variance | %diff Const | %diff Var | +------+-----------+--------+-----------+----------+-----------+----------+--------------+----------+-----------+----------+-------------+-----------+ | 1 | 0 | 0 | 8.43 | 0.28 | 8.97 | 0.01 | 8.37 | 0.33 | 8.96 | 0.00 | 0.70% | 0.09% | | 2 | 1 | 8 | 8.97 | 0.00 | 9.47 | 0.01 | 8.97 | 0.00 | 9.46 | 0.01 | 0.01% | 0.10% | | 3 | 2 | 8 | 8.97 | 0.00 | 9.46 | 0.01 | 8.97 | 0.00 | 9.47 | 0.03 | 0.00% | -0.19% | | 4 | 4 | 16 | 9.36 | 0.00 | 9.96 | 0.09 | 9.36 | 0.00 | 9.92 | 0.05 | -0.02% | 0.41% | | 5 | 8 | 32 | 10.26 | 0.01 | 11.02 | 0.02 | 10.26 | 0.00 | 10.82 | 0.41 | -0.01% | 1.86% | | 6 | 16 | 64 | 12.17 | 0.01 | 13.29 | 0.18 | 12.16 | 0.01 | 13.15 | 0.08 | 0.05% | 1.02% | | 7 | 24 | 96 | 17.47 | 0.28 | 20.54 | 0.15 | 12.20 | 0.23 | 12.62 | 0.06 | 30.15% | 38.56% |<== | 8 | 32 | 128 | 19.94 | 0.28 | 24.14 | 0.11 | 15.27 | 0.05 | 15.51 | 0.06 | 23.41% | 35.75% | | 9 | 40 | 160 | 22.68 | 0.11 | 27.66 | 1.23 | 17.30 | 0.03 | 17.47 | 0.03 | 23.73% | 36.83% | | 10 | 56 | 224 | 31.28 | 0.20 | 36.04 | 0.32 | 26.35 | 0.28 | 26.82 | 0.20 | 15.75% | 25.57% | | 11 | 64 | 256 | 38.62 | 0.42 | 62.32 | 0.15 | 34.48 | 4.25 | 34.42 | 1.24 | 10.71% | 44.77% | | 12 | 96 | 384 | 70.78 | 0.16 | 70.89 | 0.29 | 57.70 | 0.06 | 59.16 | 0.06 | 18.48% | 16.55% | | 13 | 128 | 512 | 77.92 | 0.44 | 78.54 | 0.45 | 77.71 | 0.10 | 76.35 | 0.14 | 0.28% | 2.78% |<== | 14 | 136 | 544 | 80.49 | 0.14 | 82.03 | 0.17 | 76.95 | 4.72 | 79.51 | 5.65 | 4.40% | 3.07% | | 15 | 256 | 1 KB | 131.03 | 0.23 | 132.21 | 0.40 | 128.17 | 3.01 | 129.65 | 0.65 | 2.18% | 1.93% | | 16 | 512 | 2 KB | 249.43 | 1.91 | 252.00 | 2.45 | 247.95 | 1.26 | 249.26 | 3.46 | 0.60% | 1.09% | | 17 | 808 | 3 KB | 411.56 | 4.11 | 412.80 | 1.05 | 403.64 | 1.06 | 399.41 | 1.55 | 1.92% | 3.24% | | 18 | 1024 | 4 KB | 493.10 | 5.21 | 496.55 | 0.45 | 471.13 | 0.72 | 486.47 | 0.46 | 4.46% | 2.03% | | 19 | 2048 | 8 KB | 932.67 | 2.80 | 927.03 | 1.30 | 916.10 | 0.89 | 925.54 | 1.29 | 1.78% | 0.16% | | 20 | 4096 | 16 KB | 1788.73 | 7.96 | 1798.35 | 2.64 | 1785.60 | 5.71 | 1805.24 | 1.74 | 0.18% | -0.38% | | 21 | 8192 | 32 KB | 3492.33 | 3.17 | 3503.26 | 4.34 | 3500.11 | 3.48 | 3491.26 | 2.26 | -0.22% | 0.34% | | 22 | 16384 | 64 KB | 7033.47 | 13.10 | 7016.00 | 4.89 | 6997.37 | 4.47 | 6980.20 | 8.71 | 0.51% | 0.51% | | 23 | 32768 | 128KB | 14001.69 | 21.46 | 13995.99 | 23.87 | 13953.19 | 25.24 | 13985.93 | 158.40 | 0.35% | 0.07% | | 24 | 65536 | 256KB | 28077.64 | 31.21 | 27824.47 | 99.05 | 27928.072 | 17.93 | 27946.04 | 29.16 | 0.53% | -0.44% | | 25 | 131072 | 512KB | 50649.21 | 266.61 | 51081.06 | 664.10 | 50758.384 | 182.51 | 51237.59 | 359.25 | -0.22% | -0.31% | | 26 | 262144 | 1 MB | 101764.54 | 3618.86 | 103088.84 | 3324.95 | 102189.665 | 3482.84 | 101844.83 | 3659.51 | -0.42% | 1.21% | | 27 | 524288 | 2 MB | 227482.10 | 475.35 | 226808.13 | 611.68 | 227001.467 | 400.84 | 224605.92 | 492.21 | 0.21% | 0.97% | | 28 | 1048576 | 4 MB | 447029.14 | 691.55 | 448340.14 | 806.06 | 448135.881 | 615.82 | 450134.22 | 687.74 | -0.25% | -0.40% | | 29 | 2097152 | 8 MB | 883839.80 | 1305.72 | 887155.13 | 422.39 | 887501.62 | 1045.55 | 888845.74 | 1289.69 | -0.41% | -0.19% | +------+-----------+--------+-----------+----------+-----------+----------+--------------+----------+-----------+----------+-------------+-----------+ From tobias.hartmann at oracle.com Thu May 31 06:38:24 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 31 May 2018 08:38:24 +0200 Subject: JEP: https://bugs.openjdk.java.net/browse/JDK-8203832 In-Reply-To: References: Message-ID: <0ca282db-5b4f-607d-512a-a2183dbd4b73@oracle.com> Hi Yumin, This reminds me of a project we did for a student's bachelor thesis in 2015: https://github.com/mohlerm/hotspot/blob/master/report/profile_caching_mohlerm.pdf We also published a paper on that topic: https://dl.acm.org/citation.cfm?id=3132210 Thanks for submitting the JEP, very interesting! Here are the things we've learned from the "cached profiles" project, maybe you can correct this from your experience with JWarmup: - Startup: We were seeing great improvements for some benchmarks but also large regressions for others. Problems like the overhead of reading the profiles, overloading the compile queue and increased compile time due to more optimizations affect the startup time. - Peak performance: Using profile information from a previous run might cause significant performance regressions in early stages of the execution. This is because a "late" profile is usually also the one with the fewest optimistic assumptions. For example, the latest profile from a previous run might have been updated right when the application was about to shut down. If this triggered class loading or has other side effects, we might not be able to inline some methods or perform other optimistic optimizations. Using this profile right from the beginning in a subsequent run limits peak performance significantly. Here are some questions more detailed questions: - How is it implemented? Is it based on the replay compilation framework? - How do you handle dynamically generated code (for example, lambda forms)? - What information is stored/re-used and which profile is cached? - Does it work for C1 and C2? - How is the tiered compilation policy affected? - When do we compile (if method is first used or are there still thresholds)? - How do you avoid overloading the compile queue? - Is re-profiling/re-compilation supported? - What if a method is deoptimized? Is the cached profile update and re-used? Best regards, Tobias On 29.05.2018 06:09, yumin qi wrote: > Hi? Experts > > This is a newly filed JEP (JWarmup) for working on resolving java > performance issue caused by both application load peaking up and JIT > threads compiling java hot methods happen at same time. > > https://bugs.openjdk.java.net/browse/JDK-8203832 > > For a large java application, the load comes in short period of time, > like the 'Single Day' sale on Alibaba's e-commerce application, this > massive load comes in and makes many java methods ready for JIT compilation > to convert them into native methods. The compiler threads will kick in to > do the complication work and take system resource from mutator java > threads which are busy on processing requests thus lead to peak time > performance degradation. > > The JWarmup technique was proposed to avoid such issue by precompiling > the hot methods at application startup and it has been successfully applied > to Alibaba's e-commerce applications. We would like to contribute it to > OpenJDK and wish it can help java developers overcome the same issue. > > Please review and give your feedback. > > Thanks > Yumin > > (Alibaba Group Inc) > From tobias.hartmann at oracle.com Thu May 31 08:44:57 2018 From: tobias.hartmann at oracle.com (Tobias Hartmann) Date: Thu, 31 May 2018 10:44:57 +0200 Subject: RFR (S) 8204087: C++ Interpreter code left over in MethodData In-Reply-To: <0cdbb8db-acf2-0973-b51e-8011a779ec95@oracle.com> References: <0cdbb8db-acf2-0973-b51e-8011a779ec95@oracle.com> Message-ID: Hi Coleen, looks good to me. Best regards, Tobias On 30.05.2018 19:25, coleen.phillimore at oracle.com wrote: > Summary: remove unused code > > Tested with linux-x64 Zero build, and mach5 hs-tier1-2 on 4 Oracle platforms. > > open webrev at http://cr.openjdk.java.net/~coleenp/8204087.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8204087 > > The reason for removing this old code is for initial work to support for concurrent weak methodData > cleaning, ie, I didn't want to fix this code. > > Thanks, > Coleen From adam.farley at uk.ibm.com Thu May 31 11:10:32 2018 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Thu, 31 May 2018 12:10:32 +0100 Subject: RFR: JDK-8190187: Follow-up. In-Reply-To: <8c61fe81-ff96-286e-63cb-b1af24a5981c@oracle.com> References: <460d4e0b-ee30-3d32-b482-2efb571f18db@oracle.com> <8c61fe81-ff96-286e-63cb-b1af24a5981c@oracle.com> Message-ID: Hi David, Alan, Lacking community support, both in terms of effort to resolve further issues with the fix, and also the effort required to commit the fix, it would be silly for me to continue to invest time into this. I am still convinced that exit(0) is a bad idea in this scenario, but until a committer becomes convinced of this (to the point where it outweighs existing priorities), I will not be pursuing the matter further. My appreciation to the people who have invested time on this, regardless of the end result. Since I now know much more about the Hostpot and launcher code, we can call this a win from a skills-transfer perspective. :) On with the next bug! Best Regards Adam Farley David Holmes wrote on 18/05/2018 07:27:58: > From: David Holmes > To: Alan Bateman , Adam Farley8 > , "hotspot-dev at openjdk.java.net developers" > > Date: 18/05/2018 07:28 > Subject: Re: RFR: JDK-8190187: Follow-up. > > On 18/05/2018 2:27 AM, Alan Bateman wrote: > > On 17/05/2018 13:57, Adam Farley8 wrote: > >> Hi Alan, > >> > >> As requested. > >> > >> Any ideas? > > The issue here is that the patch is just one part of changing long > > standing behavior. There is still work needed to flush out other > > interactions in the JVM TI spec, e.g. the multiple agent case and > > whether the Agent_OnUnLoad should be invoked for agents loaded before > > some other agent returns JNI_SILENT_EXIT. Also the late-binding agent > > case needs examination to see if wording is needed for the case that > > Agent_OnAttach returns this status. There is also an update needed to > > the JNI spec as we discussed. As per the mails on core-libs-dev, it's > > not clear that it's worth it. In any case, I don't know if anyone cycles > > to work with you on this. I think the help you are looking for is to > > ensure that all the spec issues are covered, maybe help with the tests > > and testing, and then the process work that is the CSR and release note. > > +1 > > I'm not convinced this really needs fixing (as per previous > discussions). A proper fix is more involved than what has been proposed > and needs to be given due indepth consideration - none of which I have > time for. Even reviewing this would be a considerable time investment to > be sure that all the issues had indeed been properly examined and covered. > > David > > > > -Alan > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From stefan.karlsson at oracle.com Thu May 31 12:01:29 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 14:01:29 +0200 Subject: RFR: 8204165: Filter out tests requiring class unloading when ClassUnloading is disabled Message-ID: <6dc0988c-7324-7d87-826a-a66cdbfa2a50@oracle.com> Hi all, Please review this patch to annotate jtreg tests that require ClassUnloading with @requires vm.opt.final.ClassUnloading. http://cr.openjdk.java.net/~stefank/8204165/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8204165 This @requires tag will filter out these tests when run with -XX:-ClassUnloading or when run with GC that doesn't support class unloading. For the discussion around the introduction of the vm.opt.final tags, see: http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-May/032586.html Thanks, StefanK From coleen.phillimore at oracle.com Thu May 31 12:10:35 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 31 May 2018 08:10:35 -0400 Subject: RFR (S) 8204087: C++ Interpreter code left over in MethodData In-Reply-To: References: <0cdbb8db-acf2-0973-b51e-8011a779ec95@oracle.com> Message-ID: <87b0fad5-c976-f048-9343-c021d622314e@oracle.com> Thank you, Tobias. Coleen On 5/31/18 4:44 AM, Tobias Hartmann wrote: > Hi Coleen, > > looks good to me. > > Best regards, > Tobias > > On 30.05.2018 19:25, coleen.phillimore at oracle.com wrote: >> Summary: remove unused code >> >> Tested with linux-x64 Zero build, and mach5 hs-tier1-2 on 4 Oracle platforms. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8204087.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8204087 >> >> The reason for removing this old code is for initial work to support for concurrent weak methodData >> cleaning, ie, I didn't want to fix this code. >> >> Thanks, >> Coleen From stefan.karlsson at oracle.com Thu May 31 12:09:00 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 14:09:00 +0200 Subject: RFR: 8204167: Filter out tests requiring compressed oops when CompressedOops is disabled Message-ID: <2da42530-7549-dfe3-7950-a45eb258e522@oracle.com> Hi all, Please review this patch to add @requires vm.opt.final.UseCompressedOops to those jtreg tests that requires compressed oops. http://cr.openjdk.java.net/~stefank/8204167/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8204167 With this patch we now filter out tests that require compressed oops, when -XX:-UseCompressedOops is passed or ZGC is tested. Thanks, StefanK From coleen.phillimore at oracle.com Thu May 31 12:21:33 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 31 May 2018 08:21:33 -0400 Subject: RFR: 8203948: Expand JVMTI callback notion of "internal threads" In-Reply-To: <28b38b3c-c17a-b5f3-6bcc-ce872af975e7@oracle.com> References: <28C16D3C-7C8F-44B3-8ED1-A24CAE4EA8AB@oracle.com> <28b38b3c-c17a-b5f3-6bcc-ce872af975e7@oracle.com> Message-ID: On 5/30/18 10:18 PM, David Holmes wrote: > Hi Kim, > > I guess I'd like this more if NamedThread were called something like > InternalThread as that captures the actual commonality of these > threads (we don't really care they they have names!). But that's not > your fault. > > I'm still tempted to just remove the check altogether now there is > only one thread we're excluding - do we really need to guard against > the WatcherThread using this code? Yes our discussion in the office went something like this.?? Not sure if it's guarding against the WatcherThread or what this code is thinks it's guarding against. Coleen > > But its fine to push as-is. > > Thanks, > David > > On 31/05/2018 2:28 AM, Kim Barrett wrote: >> Please review this change to JVMTI to relax some restrictions on where >> some functions may be called.? Presently they are restricted to being >> called from either a Java thread, the VM thread, or a thread for which >> is_ConcurrentGC_thread() returns true (e.g. ConcurrentGCThreads and >> AbstractGangWorkers (if the containing WorkGang is configured as >> concurrent). >> >> We now allow these functions to be called for either a Java thread or >> a Named thread, e.g. any of the VM thread, ConcurrentGCThreads, or >> WorkerThreads.? This allows the restricted functions to be called from >> any WorkerThreads, rather than only AbstractGangWorkers in concurrent >> WorkGangs. >> >> The change to jvmtiEnter.xsl affects the following generated >> functions: >> jvmti_Allocate >> jvmti_Deallocate >> jvmti_CreateRawMonitor >> jvmti_DestroyRawMonitor >> jvmti_RawMonitorEnter >> jvmti_RawMonitorExit >> jvmti_RawMonitorWait >> jvmti_RawMonitorNotify >> jvmti_RawMonitorNotifyAll >> jvmti_GetTimerInfo >> jvmti_GetTime >> jvmti_SetEnvironmentLocalStorage >> jvmti_GetEnvironmentLocalStorage >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8203948 >> >> Webrev: >> http://cr.openjdk.java.net/~kbarrett/8203948/open.00/ >> >> Testing: >> Mach5 tier{1,2,3} and hs-tier{4,5} >> >> There are no tests that directly exercise the relaxed restrictions >> yet.? We'll start hitting that as part of in-progress work to >> parallelize WeakProcessor, which calls JvmtiExport::weak_oops_do, >> which generates ObjectFree events, which uses the Raw Monitor API. >> From stefan.karlsson at oracle.com Thu May 31 12:32:53 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 14:32:53 +0200 Subject: RFR: 8204168: Increase small heap sizes in tests to accommodate ZGC Message-ID: Hi all, Please review this patch to increase the heap size for tests that sets a small heap size. http://cr.openjdk.java.net/~stefank/8204168/webrev.01 https://bugs.openjdk.java.net/browse/JDK-8204168 This change is needed to test ZGC with these tests. ZGC doesn't use compressed oops and has an allocation memory reserve for the GC thread, and hence have a higher smallest heap size compared to other GCs in the code base. There are some alternatives to this patch that we could consider (but I prefer the suggested patch): 1) Disable these tests when running with ZGC 2) Split all these tests into two copies, one copy for ZGC and another for the other GCs. I've been running all of these changes through tier{1,2,3} and most of them has been run in tier{4,5,6}. Thanks, StefanK From coleen.phillimore at oracle.com Thu May 31 12:35:42 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 31 May 2018 08:35:42 -0400 Subject: RFR: 8204165: Filter out tests requiring class unloading when ClassUnloading is disabled In-Reply-To: <6dc0988c-7324-7d87-826a-a66cdbfa2a50@oracle.com> References: <6dc0988c-7324-7d87-826a-a66cdbfa2a50@oracle.com> Message-ID: <4a1816b7-cc65-0a37-863a-696c183df9ef@oracle.com> If? you look for ClassUnloadCommon, there are a couple others.? It would be good to have these tagged as well, even though the last one tests two things. runtime/appcds/customLoader/UnloadUnregisteredLoaderTest.java runtime/appcds/customLoader/test-classes/UnloadUnregisteredLoader.java runtime/logging/LoaderConstraintsTest.java runtime/BadObjectClass/TestUnloadClassError.java If you look for triggerUnloading there are more: e.g. vmTestbase/metaspace/stressHierarchy.? I assume they pass without class unloading though. Thanks, Coleen On 5/31/18 8:01 AM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to annotate jtreg tests that require > ClassUnloading with @requires vm.opt.final.ClassUnloading. > > http://cr.openjdk.java.net/~stefank/8204165/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8204165 > > This @requires tag will filter out these tests when run with > -XX:-ClassUnloading or when run with GC that doesn't support class > unloading. > > For the discussion around the introduction of the vm.opt.final tags, > see: > http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-May/032586.html > > Thanks, > StefanK From stefan.karlsson at oracle.com Thu May 31 12:57:34 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 14:57:34 +0200 Subject: RFR: 8204165: Filter out tests requiring class unloading when ClassUnloading is disabled In-Reply-To: <4a1816b7-cc65-0a37-863a-696c183df9ef@oracle.com> References: <6dc0988c-7324-7d87-826a-a66cdbfa2a50@oracle.com> <4a1816b7-cc65-0a37-863a-696c183df9ef@oracle.com> Message-ID: <70964597-4cab-7fba-a71b-9982a4990eb2@oracle.com> Hi Coleen, On 2018-05-31 14:35, coleen.phillimore at oracle.com wrote: > > If? you look for ClassUnloadCommon, there are a couple others.? It would > be good to have these tagged as well, even though the last one tests two > things. I've been very ZGC centric and only tagged problematic tests I've found, but of course I can tag more when it makes sense. > > runtime/appcds/customLoader/UnloadUnregisteredLoaderTest.java > runtime/appcds/customLoader/test-classes/UnloadUnregisteredLoader.java Added the tag and tested with -XX:+UseG1GC -XX:-ClassUnloading. > runtime/logging/LoaderConstraintsTest.java This tests seems to spawn a new test with a ProcessBuilder and therefore doesn't propagate the -vmoptions flags. Even if I try to turn off class unloading or run ZGC, it will end up running G1 with class unloading. I think this tests needs to be changed, before it makes sense to tag it. If you still want me to add the tag, I can do so. > runtime/BadObjectClass/TestUnloadClassError.java This test doesn't fail if I turn off class unloading. Do you still want me to add the tag? > > If you look for triggerUnloading there are more: e.g. > vmTestbase/metaspace/stressHierarchy.? I assume they pass without class > unloading though. Maybe we can handle those in a future RFE? Thanks, StefanK > > Thanks, > Coleen > > > On 5/31/18 8:01 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to annotate jtreg tests that require >> ClassUnloading with @requires vm.opt.final.ClassUnloading. >> >> http://cr.openjdk.java.net/~stefank/8204165/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8204165 >> >> This @requires tag will filter out these tests when run with >> -XX:-ClassUnloading or when run with GC that doesn't support class >> unloading. >> >> For the discussion around the introduction of the vm.opt.final tags, >> see: >> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-May/032586.html >> >> Thanks, >> StefanK > From coleen.phillimore at oracle.com Thu May 31 13:12:51 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 31 May 2018 09:12:51 -0400 Subject: RFR: 8204165: Filter out tests requiring class unloading when ClassUnloading is disabled In-Reply-To: <70964597-4cab-7fba-a71b-9982a4990eb2@oracle.com> References: <6dc0988c-7324-7d87-826a-a66cdbfa2a50@oracle.com> <4a1816b7-cc65-0a37-863a-696c183df9ef@oracle.com> <70964597-4cab-7fba-a71b-9982a4990eb2@oracle.com> Message-ID: <4f748bd8-f958-3dba-7cb0-6c7989bfe2f2@oracle.com> On 5/31/18 8:57 AM, Stefan Karlsson wrote: > Hi Coleen, > > On 2018-05-31 14:35, coleen.phillimore at oracle.com wrote: >> >> If? you look for ClassUnloadCommon, there are a couple others. It >> would be good to have these tagged as well, even though the last one >> tests two things. > > I've been very ZGC centric and only tagged problematic tests I've > found, but of course I can tag more when it makes sense. Yes, I thought that.? I'm on the fence.? I don't think it's worth tagging more.? I may make a test group at some point though. > >> >> runtime/appcds/customLoader/UnloadUnregisteredLoaderTest.java >> runtime/appcds/customLoader/test-classes/UnloadUnregisteredLoader.java > > Added the tag and tested with -XX:+UseG1GC -XX:-ClassUnloading. Great, these seemed to want it. > >> runtime/logging/LoaderConstraintsTest.java > > This tests seems to spawn a new test with a ProcessBuilder and > therefore doesn't propagate the -vmoptions flags. Even if I try to > turn off class unloading or run ZGC, it will end up running G1 with > class unloading. I think this tests needs to be changed, before it > makes sense to tag it. > > If you still want me to add the tag, I can do so. > Ugh, no, that's fine. >> runtime/BadObjectClass/TestUnloadClassError.java > > This test doesn't fail if I turn off class unloading. Do you still > want me to add the tag? > It does two things so no, that's ok, leave the tag off. >> >> If you look for triggerUnloading there are more: e.g. >> vmTestbase/metaspace/stressHierarchy.? I assume they pass without >> class unloading though. > > Maybe we can handle those in a future RFE? > Yes, or an RFE in the future to mark, in some other way, the tests that exercise class unloading. Thanks - change looks good. Coleen > Thanks, > StefanK > >> >> Thanks, >> Coleen >> >> >> On 5/31/18 8:01 AM, Stefan Karlsson wrote: >>> Hi all, >>> >>> Please review this patch to annotate jtreg tests that require >>> ClassUnloading with @requires vm.opt.final.ClassUnloading. >>> >>> http://cr.openjdk.java.net/~stefank/8204165/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8204165 >>> >>> This @requires tag will filter out these tests when run with >>> -XX:-ClassUnloading or when run with GC that doesn't support class >>> unloading. >>> >>> For the discussion around the introduction of the vm.opt.final tags, >>> see: >>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-May/032586.html >>> >>> Thanks, >>> StefanK >> From coleen.phillimore at oracle.com Thu May 31 13:19:20 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 31 May 2018 09:19:20 -0400 Subject: RFR (S) 8204087: C++ Interpreter code left over in MethodData In-Reply-To: References: <0cdbb8db-acf2-0973-b51e-8011a779ec95@oracle.com> Message-ID: <25b5dcd3-4f61-06f5-368a-189083767b32@oracle.com> Thanks Lois! Coleen On 5/30/18 4:00 PM, Lois Foltan wrote: > Looks good. > Lois > > On 5/30/2018 1:25 PM, coleen.phillimore at oracle.com wrote: >> Summary: remove unused code >> >> Tested with linux-x64 Zero build, and mach5 hs-tier1-2 on 4 Oracle >> platforms. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/8204087.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8204087 >> >> The reason for removing this old code is for initial work to support >> for concurrent weak methodData cleaning, ie, I didn't want to fix >> this code. >> >> Thanks, >> Coleen > From stefan.karlsson at oracle.com Thu May 31 13:27:55 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 15:27:55 +0200 Subject: RFR: 8204165: Filter out tests requiring class unloading when ClassUnloading is disabled In-Reply-To: <4f748bd8-f958-3dba-7cb0-6c7989bfe2f2@oracle.com> References: <6dc0988c-7324-7d87-826a-a66cdbfa2a50@oracle.com> <4a1816b7-cc65-0a37-863a-696c183df9ef@oracle.com> <70964597-4cab-7fba-a71b-9982a4990eb2@oracle.com> <4f748bd8-f958-3dba-7cb0-6c7989bfe2f2@oracle.com> Message-ID: <322cd976-04f7-eba7-e950-4b82612ed863@oracle.com> Thanks for reviewing! stefanK On 2018-05-31 15:12, coleen.phillimore at oracle.com wrote: > > > On 5/31/18 8:57 AM, Stefan Karlsson wrote: >> Hi Coleen, >> >> On 2018-05-31 14:35, coleen.phillimore at oracle.com wrote: >>> >>> If? you look for ClassUnloadCommon, there are a couple others. It >>> would be good to have these tagged as well, even though the last one >>> tests two things. >> >> I've been very ZGC centric and only tagged problematic tests I've >> found, but of course I can tag more when it makes sense. > > Yes, I thought that.? I'm on the fence.? I don't think it's worth > tagging more.? I may make a test group at some point though. >> >>> >>> runtime/appcds/customLoader/UnloadUnregisteredLoaderTest.java >>> runtime/appcds/customLoader/test-classes/UnloadUnregisteredLoader.java >> >> Added the tag and tested with -XX:+UseG1GC -XX:-ClassUnloading. > > Great, these seemed to want it. >> >>> runtime/logging/LoaderConstraintsTest.java >> >> This tests seems to spawn a new test with a ProcessBuilder and >> therefore doesn't propagate the -vmoptions flags. Even if I try to >> turn off class unloading or run ZGC, it will end up running G1 with >> class unloading. I think this tests needs to be changed, before it >> makes sense to tag it. >> >> If you still want me to add the tag, I can do so. >> > > Ugh, no, that's fine. >>> runtime/BadObjectClass/TestUnloadClassError.java >> >> This test doesn't fail if I turn off class unloading. Do you still >> want me to add the tag? >> > > It does two things so no, that's ok, leave the tag off. > >>> >>> If you look for triggerUnloading there are more: e.g. >>> vmTestbase/metaspace/stressHierarchy.? I assume they pass without >>> class unloading though. >> >> Maybe we can handle those in a future RFE? >> > > Yes, or an RFE in the future to mark, in some other way, the tests that > exercise class unloading. > > Thanks - change looks good. > Coleen > >> Thanks, >> StefanK >> >>> >>> Thanks, >>> Coleen >>> >>> >>> On 5/31/18 8:01 AM, Stefan Karlsson wrote: >>>> Hi all, >>>> >>>> Please review this patch to annotate jtreg tests that require >>>> ClassUnloading with @requires vm.opt.final.ClassUnloading. >>>> >>>> http://cr.openjdk.java.net/~stefank/8204165/webrev.01/ >>>> https://bugs.openjdk.java.net/browse/JDK-8204165 >>>> >>>> This @requires tag will filter out these tests when run with >>>> -XX:-ClassUnloading or when run with GC that doesn't support class >>>> unloading. >>>> >>>> For the discussion around the introduction of the vm.opt.final tags, >>>> see: >>>> http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-May/032586.html >>>> >>>> Thanks, >>>> StefanK >>> > From rkennke at redhat.com Thu May 31 13:40:26 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 31 May 2018 15:40:26 +0200 Subject: RFR: 8203353: Fixup inferred decorators in the interpreter In-Reply-To: <5AFD8F66.1070006@oracle.com> References: <5AFD8F66.1070006@oracle.com> Message-ID: <6db04a41-069c-0680-ff6c-02dab0aeaecf@redhat.com> > Hi, > > All subsystems that the access API touches, except the interpreter, > infers a set of decorators according to rules specified in > accessDecorators.hpp. These are typically sane defaults for things like > reference strength, memory ordering, as well as inferring that > IN_HEAP_ARRAY is IN_HEAP, and IN_CONCURRENT_ROOT is IN_ROOT. The > interpreter should do that too. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8203353 Hi Erik, this seems reasonable and patch looks good. Thanks! Roman From stefan.karlsson at oracle.com Thu May 31 13:38:48 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 15:38:48 +0200 Subject: RFR: 8204170: MXBeanException.java assumes the existence of a pool that doesn't support usage threshold Message-ID: <3c76b94a-b99e-b617-88ff-7251c1c98083@oracle.com> Hi all, Please review this patch to deal with the case when all available MemoryPoolMXBeans support usage thresholds. http://cr.openjdk.java.net/~stefank/8204170/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8204170 The tests searches for a MemoryPoolMXBean that returns false for isUsageThresholdSupported(), and then uses one of those pools to trigger UnsupportedOperationExceptions when setUsageThreshold is used. ZGC doesn't provide a pool that doesn't support usage threshold, and the test therefore throws an NPE. The suggested fix is to add: if (pool == null) { return; // All pools support usage threshold } An alternative fix would be to filter out the test if ZGC is used. Thanks, StefanK From rkennke at redhat.com Thu May 31 13:46:45 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 31 May 2018 15:46:45 +0200 Subject: RFR: 8202547: Move G1 runtime calls used by generated code to G1BarrierSetRuntime In-Reply-To: <5AFEEB58.3080404@oracle.com> References: <5AFEEB58.3080404@oracle.com> Message-ID: <2bbd8c2b-ed6e-b8e2-a3bc-e307f67ec17a@redhat.com> > Hi, > > Generated code occasionally needs to call into the runtime for G1 > barriers. Some of these slow-path runtime calls are in SharedRuntime, > some are in G1BarrierSet. It would be nice to have them collected in the > same place. This patch move these slow-path C++ functions for G1 into a > new class, g1BarrierSetRuntime. > > Webrev: > http://cr.openjdk.java.net/~eosterlund/8202547/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8202547 > This is a very nice cleanup. I was wondering the same just recently. Patch looks fine. Thanks! Roman From erik.osterlund at oracle.com Thu May 31 13:51:09 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 31 May 2018 15:51:09 +0200 Subject: RFR: 8202547: Move G1 runtime calls used by generated code to G1BarrierSetRuntime In-Reply-To: <2bbd8c2b-ed6e-b8e2-a3bc-e307f67ec17a@redhat.com> References: <5AFEEB58.3080404@oracle.com> <2bbd8c2b-ed6e-b8e2-a3bc-e307f67ec17a@redhat.com> Message-ID: <5B0FFDCD.2040007@oracle.com> Hi Roman, Thanks for the review. /Erik On 2018-05-31 15:46, Roman Kennke wrote: >> Hi, >> >> Generated code occasionally needs to call into the runtime for G1 >> barriers. Some of these slow-path runtime calls are in SharedRuntime, >> some are in G1BarrierSet. It would be nice to have them collected in the >> same place. This patch move these slow-path C++ functions for G1 into a >> new class, g1BarrierSetRuntime. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8202547/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8202547 >> > This is a very nice cleanup. I was wondering the same just recently. > Patch looks fine. Thanks! > > Roman > From stefan.karlsson at oracle.com Thu May 31 13:53:43 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 15:53:43 +0200 Subject: RFR: 8204173: Lower the minimum number of heap memory pools in MemoryTest.java Message-ID: <6280012d-35ef-5670-0429-65d6647792e4@oracle.com> Hi all, Please review this patch to lower the minimum number of heap memory pools in MemoryTest.java. http://cr.openjdk.java.net/~stefank/8204173/webrev.01/ https://bugs.openjdk.java.net/browse/JDK-8204173 Just like the comment in the test says: * NOTE: This expected result is hardcoded in this test and this test * will be affected if the heap memory layout is changed in * the future implementation. */ we need to update this test to support ZGC, which only has one heap memory pool. Thanks, StefanK From erik.osterlund at oracle.com Thu May 31 14:23:07 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 31 May 2018 16:23:07 +0200 Subject: RFR: 8203353: Fixup inferred decorators in the interpreter In-Reply-To: <6db04a41-069c-0680-ff6c-02dab0aeaecf@redhat.com> References: <5AFD8F66.1070006@oracle.com> <6db04a41-069c-0680-ff6c-02dab0aeaecf@redhat.com> Message-ID: <5B10054B.6030206@oracle.com> Hi Roman, Thanks for the review! /Erik On 2018-05-31 15:40, Roman Kennke wrote: >> Hi, >> >> All subsystems that the access API touches, except the interpreter, >> infers a set of decorators according to rules specified in >> accessDecorators.hpp. These are typically sane defaults for things like >> reference strength, memory ordering, as well as inferring that >> IN_HEAP_ARRAY is IN_HEAP, and IN_CONCURRENT_ROOT is IN_ROOT. The >> interpreter should do that too. >> >> Webrev: >> http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/ >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8203353 > > Hi Erik, this seems reasonable and patch looks good. Thanks! > Roman > From erik.osterlund at oracle.com Thu May 31 14:26:24 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 31 May 2018 16:26:24 +0200 Subject: RFR: 8204173: Lower the minimum number of heap memory pools in MemoryTest.java In-Reply-To: <6280012d-35ef-5670-0429-65d6647792e4@oracle.com> References: <6280012d-35ef-5670-0429-65d6647792e4@oracle.com> Message-ID: <5B100610.6000906@oracle.com> Hi Stefan, Looks good. Thanks, /Erik On 2018-05-31 15:53, Stefan Karlsson wrote: > Hi all, > > Please review this patch to lower the minimum number of heap memory > pools in MemoryTest.java. > > http://cr.openjdk.java.net/~stefank/8204173/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8204173 > > Just like the comment in the test says: > > * NOTE: This expected result is hardcoded in this test and this test > * will be affected if the heap memory layout is changed in > * the future implementation. > */ > > we need to update this test to support ZGC, which only has one heap > memory pool. > > Thanks, > StefanK > From erik.osterlund at oracle.com Thu May 31 14:27:58 2018 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 31 May 2018 16:27:58 +0200 Subject: RFR: 8204170: MXBeanException.java assumes the existence of a pool that doesn't support usage threshold In-Reply-To: <3c76b94a-b99e-b617-88ff-7251c1c98083@oracle.com> References: <3c76b94a-b99e-b617-88ff-7251c1c98083@oracle.com> Message-ID: <5B10066E.9030009@oracle.com> Hi Stefan, Looks good. Thanks, /Erik On 2018-05-31 15:38, Stefan Karlsson wrote: > Hi all, > > Please review this patch to deal with the case when all available > MemoryPoolMXBeans support usage thresholds. > > http://cr.openjdk.java.net/~stefank/8204170/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8204170 > > The tests searches for a MemoryPoolMXBean that returns false for > isUsageThresholdSupported(), and then uses one of those pools to > trigger UnsupportedOperationExceptions when setUsageThreshold is used. > > ZGC doesn't provide a pool that doesn't support usage threshold, and > the test therefore throws an NPE. > > The suggested fix is to add: > if (pool == null) { > return; // All pools support usage threshold > } > > An alternative fix would be to filter out the test if ZGC is used. > > Thanks, > StefanK From igor.ignatyev at oracle.com Thu May 31 15:56:21 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 31 May 2018 08:56:21 -0700 Subject: [11] RFR (M) 8202611: [GRAAL] Exclude CMS GC testing from runs with Graal In-Reply-To: References: Message-ID: <71278F80-14A3-43B0-870E-213E81643763@oracle.com> Hi Vladimir, looks good to me. -- Igor > On May 7, 2018, at 11:33 AM, Vladimir Kozlov wrote: > > http://cr.openjdk.java.net/~kvn/8202611/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8202611 > > JDK-8184349 handles Hotspot changes to add check that Graal support GC and exit on error if it does not. > Since tests changes are big I decided to do it separate and may be push first. > > These changes excluded tests which use CMS from Graal runs. > > @requires !vm.graal.enabled was added for tests which use only CMS GC explicitly. > > Undocumented jtreg functionality is use to split @run command with -XX:+UseConcMarkSweepGC into separate jtreg comment block if test has several @run commands. Jtreg treats such test as 2 tests. > > New test API sun/hotspot/code/Compiler.java (which uses WhiteBox API) is added to check if particular JIT compiler is enabled. The Graal related code was copied from VMProps.java. > This was added for cases when a test forks new process and use CMS GC. New API is used to guard such code to not run with Graal. > > Tested with Graal enabled as JIT. > > -- > Thanks, > Vladimir From stefan.karlsson at oracle.com Thu May 31 16:12:46 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 18:12:46 +0200 Subject: RFR: 8204173: Lower the minimum number of heap memory pools in MemoryTest.java In-Reply-To: <5B100610.6000906@oracle.com> References: <6280012d-35ef-5670-0429-65d6647792e4@oracle.com> <5B100610.6000906@oracle.com> Message-ID: <0c38d000-9d41-a130-7dfd-adc2422053d6@oracle.com> Thanks, Erik. StefanK On 2018-05-31 16:26, Erik ?sterlund wrote: > Hi Stefan, > > Looks good. > > Thanks, > /Erik > > On 2018-05-31 15:53, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to lower the minimum number of heap memory >> pools in MemoryTest.java. >> >> http://cr.openjdk.java.net/~stefank/8204173/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8204173 >> >> Just like the comment in the test says: >> >> ?* NOTE: This expected result is hardcoded in this test and this test >> ?* will be affected if the heap memory layout is changed in >> ?* the future implementation. >> ?*/ >> >> we need to update this test to support ZGC, which only has one heap >> memory pool. >> >> Thanks, >> StefanK >> > From stefan.karlsson at oracle.com Thu May 31 16:13:31 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 18:13:31 +0200 Subject: RFR: 8204170: MXBeanException.java assumes the existence of a pool that doesn't support usage threshold In-Reply-To: <5B10066E.9030009@oracle.com> References: <3c76b94a-b99e-b617-88ff-7251c1c98083@oracle.com> <5B10066E.9030009@oracle.com> Message-ID: Thanks, Erik. StefanK On 2018-05-31 16:27, Erik ?sterlund wrote: > Hi Stefan, > > Looks good. > > Thanks, > /Erik > > On 2018-05-31 15:38, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to deal with the case when all available >> MemoryPoolMXBeans support usage thresholds. >> >> http://cr.openjdk.java.net/~stefank/8204170/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8204170 >> >> The tests searches for a MemoryPoolMXBean that returns false for >> isUsageThresholdSupported(), and then uses one of those pools to >> trigger UnsupportedOperationExceptions when setUsageThreshold is used. >> >> ZGC doesn't provide a pool that doesn't support usage threshold, and >> the test therefore throws an NPE. >> >> The suggested fix is to add: >> ??????? if (pool == null) { >> ????????? return; // All pools support usage threshold >> ??????? } >> >> An alternative fix would be to filter out the test if ZGC is used. >> >> Thanks, >> StefanK > From mandy.chung at oracle.com Thu May 31 16:19:07 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 31 May 2018 09:19:07 -0700 Subject: RFR: 8204173: Lower the minimum number of heap memory pools in MemoryTest.java In-Reply-To: <6280012d-35ef-5670-0429-65d6647792e4@oracle.com> References: <6280012d-35ef-5670-0429-65d6647792e4@oracle.com> Message-ID: <8bfbae10-44b0-088c-7679-eacdf5aff986@oracle.com> Hi Stefan, This change looks okay. Can you add a comment to describe the expected memory pools for ZGC so that it explains why the min number of heap memory pools is 1. I think this test and possibly other memory pool tests should be re-examined and determine what proper verification should be done, for example, expected heap memory pools for a specific GC. It may be worth to file a RFE to follow up. Mandy On 5/31/18 6:53 AM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to lower the minimum number of heap memory > pools in MemoryTest.java. > > http://cr.openjdk.java.net/~stefank/8204173/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8204173 > > Just like the comment in the test says: > > ?* NOTE: This expected result is hardcoded in this test and this test > ?* will be affected if the heap memory layout is changed in > ?* the future implementation. > ?*/ > > we need to update this test to support ZGC, which only has one heap > memory pool. > > Thanks, > StefanK > From mandy.chung at oracle.com Thu May 31 16:44:37 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 31 May 2018 09:44:37 -0700 Subject: RFR: 8204170: MXBeanException.java assumes the existence of a pool that doesn't support usage threshold In-Reply-To: <3c76b94a-b99e-b617-88ff-7251c1c98083@oracle.com> References: <3c76b94a-b99e-b617-88ff-7251c1c98083@oracle.com> Message-ID: Hi Stefan, I think a better fix is to find another operation that will throw an exception. ThreadMXBean::getThreadInfo throws IAE and it can be a candidate. Mandy On 5/31/18 6:38 AM, Stefan Karlsson wrote: > Hi all, > > Please review this patch to deal with the case when all available > MemoryPoolMXBeans support usage thresholds. > > http://cr.openjdk.java.net/~stefank/8204170/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8204170 > > The tests searches for a MemoryPoolMXBean that returns false for > isUsageThresholdSupported(), and then uses one of those pools to trigger > UnsupportedOperationExceptions when setUsageThreshold is used. > > ZGC doesn't provide a pool that doesn't support usage threshold, and the > test therefore throws an NPE. > > The suggested fix is to add: > ??????? if (pool == null) { > ????????? return; // All pools support usage threshold > ??????? } > > An alternative fix would be to filter out the test if ZGC is used. > > Thanks, > StefanK From stefan.karlsson at oracle.com Thu May 31 17:01:53 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 19:01:53 +0200 Subject: RFR: 8204170: MXBeanException.java assumes the existence of a pool that doesn't support usage threshold In-Reply-To: References: <3c76b94a-b99e-b617-88ff-7251c1c98083@oracle.com> Message-ID: Hi Mandy, On 2018-05-31 18:44, mandy chung wrote: > Hi Stefan, > > I think a better fix is to find another operation that will throw an > exception.? ThreadMXBean::getThreadInfo throws IAE and it can be a > candidate. Thanks for taking a look at this patch. The test checks a number of different operations, and the suggestion would change the test significantly. I'm not sure this is the time to completely rewrite this test. If the null check is problematic, then I think it would be better to just exclude this test when running with ZGC, and get the current testing when running with the other GCs. Does this sound reasonable? Thanks, StefanK > Mandy > > On 5/31/18 6:38 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to deal with the case when all available >> MemoryPoolMXBeans support usage thresholds. >> >> http://cr.openjdk.java.net/~stefank/8204170/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8204170 >> >> The tests searches for a MemoryPoolMXBean that returns false for >> isUsageThresholdSupported(), and then uses one of those pools to >> trigger UnsupportedOperationExceptions when setUsageThreshold is used. >> >> ZGC doesn't provide a pool that doesn't support usage threshold, and >> the test therefore throws an NPE. >> >> The suggested fix is to add: >> ???????? if (pool == null) { >> ?????????? return; // All pools support usage threshold >> ???????? } >> >> An alternative fix would be to filter out the test if ZGC is used. >> >> Thanks, >> StefanK From mandy.chung at oracle.com Thu May 31 17:04:22 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 31 May 2018 10:04:22 -0700 Subject: RFR: 8204170: MXBeanException.java assumes the existence of a pool that doesn't support usage threshold In-Reply-To: References: <3c76b94a-b99e-b617-88ff-7251c1c98083@oracle.com> Message-ID: On 5/31/18 10:01 AM, Stefan Karlsson wrote: > Hi Mandy, > > On 2018-05-31 18:44, mandy chung wrote: >> Hi Stefan, >> >> I think a better fix is to find another operation that will throw an >> exception.? ThreadMXBean::getThreadInfo throws IAE and it can be a >> candidate. > > Thanks for taking a look at this patch. > > The test checks a number of different operations, and the suggestion > would change the test significantly. I'm not sure this is the time to > completely rewrite this test. If the null check is problematic, then I > think it would be better to just exclude this test when running with > ZGC, and get the current testing when running with the other GCs. > > Does this sound reasonable? Sounds good with me. Filtering out the test when running with ZGC is a good interim fix. The serviceability team can modify this test to use a different operation as a separate issue. Mandy From stefan.karlsson at oracle.com Thu May 31 17:06:42 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 19:06:42 +0200 Subject: RFR: 8204170: MXBeanException.java assumes the existence of a pool that doesn't support usage threshold In-Reply-To: References: <3c76b94a-b99e-b617-88ff-7251c1c98083@oracle.com> Message-ID: <674e786c-f13e-96ab-dc68-4fa7f009526a@oracle.com> On 2018-05-31 19:04, mandy chung wrote: > > > On 5/31/18 10:01 AM, Stefan Karlsson wrote: >> Hi Mandy, >> >> On 2018-05-31 18:44, mandy chung wrote: >>> Hi Stefan, >>> >>> I think a better fix is to find another operation that will throw an >>> exception.? ThreadMXBean::getThreadInfo throws IAE and it can be a >>> candidate. >> >> Thanks for taking a look at this patch. >> >> The test checks a number of different operations, and the suggestion >> would change the test significantly. I'm not sure this is the time to >> completely rewrite this test. If the null check is problematic, then >> I think it would be better to just exclude this test when running >> with ZGC, and get the current testing when running with the other GCs. >> >> Does this sound reasonable? > > Sounds good with me. > > Filtering out the test when running with ZGC is a good interim fix.? > The serviceability team can modify this test to use a different > operation as a separate issue. OK. Then I'm revoking this RFE and will add the ZGC filtering to the ZGC patch. Thanks, StefanK > > Mandy From stefan.karlsson at oracle.com Thu May 31 17:36:29 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 19:36:29 +0200 Subject: RFR: 8204173: Lower the minimum number of heap memory pools in MemoryTest.java In-Reply-To: <8bfbae10-44b0-088c-7679-eacdf5aff986@oracle.com> References: <6280012d-35ef-5670-0429-65d6647792e4@oracle.com> <8bfbae10-44b0-088c-7679-eacdf5aff986@oracle.com> Message-ID: <42662a7a-a3fb-7bfe-ad9b-d50e5ade4661@oracle.com> Hi Mandy, On 2018-05-31 18:19, mandy chung wrote: > Hi Stefan, > > This change looks okay.? Can you add a comment to describe the > expected memory pools for ZGC so that it explains why the min number > of heap memory pools is 1. > > I think this test and possibly other memory pool tests should be > re-examined and determine what proper verification should be done, for > example, expected heap memory pools for a specific GC.? It may be > worth to file a RFE to follow up. What if I change the code like this: http://cr.openjdk.java.net/~stefank/8204173/webrev.02/ and then later add a new MemoryTestZGC.sh test, that calls MemoryTest 1 1 and explains that ZGC has one memory manager and one heap memory pool? Thanks, StefanK > > Mandy > > On 5/31/18 6:53 AM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this patch to lower the minimum number of heap memory >> pools in MemoryTest.java. >> >> http://cr.openjdk.java.net/~stefank/8204173/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8204173 >> >> Just like the comment in the test says: >> >> ??* NOTE: This expected result is hardcoded in this test and this test >> ??* will be affected if the heap memory layout is changed in >> ??* the future implementation. >> ??*/ >> >> we need to update this test to support ZGC, which only has one heap >> memory pool. >> >> Thanks, >> StefanK >> From kim.barrett at oracle.com Thu May 31 17:59:33 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 31 May 2018 13:59:33 -0400 Subject: [11] RFR (M) 8202611: [GRAAL] Exclude CMS GC testing from runs with Graal In-Reply-To: References: Message-ID: <923C4D3F-AE62-4473-B39C-6C93B3ED8C10@oracle.com> > On May 7, 2018, at 2:33 PM, Vladimir Kozlov wrote: > > http://cr.openjdk.java.net/~kvn/8202611/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8202611 > > JDK-8184349 handles Hotspot changes to add check that Graal support GC and exit on error if it does not. > Since tests changes are big I decided to do it separate and may be push first. > > These changes excluded tests which use CMS from Graal runs. > > @requires !vm.graal.enabled was added for tests which use only CMS GC explicitly. > > Undocumented jtreg functionality is use to split @run command with -XX:+UseConcMarkSweepGC into separate jtreg comment block if test has several @run commands. Jtreg treats such test as 2 tests. > > New test API sun/hotspot/code/Compiler.java (which uses WhiteBox API) is added to check if particular JIT compiler is enabled. The Graal related code was copied from VMProps.java. > This was added for cases when a test forks new process and use CMS GC. New API is used to guard such code to not run with Graal. > > Tested with Graal enabled as JIT. > > -- > Thanks, > Vladimir Looks good. From coleen.phillimore at Oracle.com Thu May 31 17:21:16 2018 From: coleen.phillimore at Oracle.com (coleen.phillimore at Oracle.com) Date: Thu, 31 May 2018 13:21:16 -0400 Subject: RFR: 8203353: Fixup inferred decorators in the interpreter In-Reply-To: <5B10054B.6030206@oracle.com> References: <5AFD8F66.1070006@oracle.com> <6db04a41-069c-0680-ff6c-02dab0aeaecf@redhat.com> <5B10054B.6030206@oracle.com> Message-ID: http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/src/hotspot/cpu/x86/macroAssembler_x86.cpp.frames.html 6334 void MacroAssembler::access_store_at(BasicType type, DecoratorSet decorators, Address dst, Register src, 6335 Register tmp1, Register tmp2) { 6336 BarrierSetAssembler* bs = BarrierSet::barrier_set()->barrier_set_assembler(); 6337 decorators = AccessInternal::decorator_fixup(decorators); 6338 bool as_raw = (decorators & AS_RAW) != 0; 6339 if (as_raw) { 6340 bs->BarrierSetAssembler::store_at(this, decorators, type, dst, src, tmp1, tmp2); 6341 } else { 6342 bs->store_at(this, decorators, type, dst, src, tmp1, tmp2); 6343 } 6344 } This is a very strange function.? From the macroAssembler, how are we to know that the cases in this if statement are different?? It seems like this distinction should be made in BarrierSetAssembler class.?? It also seems like whatever decorator fixup should also happen inside the BarrierSetAssembler, because the placement of these calls to fixup the decorators seems unknowable. thanks, Coleen On 5/31/18 10:23 AM, Erik ?sterlund wrote: > Hi Roman, > > Thanks for the review! > > /Erik > > On 2018-05-31 15:40, Roman Kennke wrote: >>> Hi, >>> >>> All subsystems that the access API touches, except the interpreter, >>> infers a set of decorators according to rules specified in >>> accessDecorators.hpp. These are typically sane defaults for things like >>> reference strength, memory ordering, as well as inferring that >>> IN_HEAP_ARRAY is IN_HEAP, and IN_CONCURRENT_ROOT is IN_ROOT. The >>> interpreter should do that too. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/ >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8203353 >> >> Hi Erik, this seems reasonable and patch looks good. Thanks! >> Roman >> > From mandy.chung at oracle.com Thu May 31 18:09:45 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 31 May 2018 11:09:45 -0700 Subject: RFR: 8204173: Lower the minimum number of heap memory pools in MemoryTest.java In-Reply-To: <42662a7a-a3fb-7bfe-ad9b-d50e5ade4661@oracle.com> References: <6280012d-35ef-5670-0429-65d6647792e4@oracle.com> <8bfbae10-44b0-088c-7679-eacdf5aff986@oracle.com> <42662a7a-a3fb-7bfe-ad9b-d50e5ade4661@oracle.com> Message-ID: <4b5718fc-3206-9146-1722-07306bbdb74a@oracle.com> On 5/31/18 10:36 AM, Stefan Karlsson wrote: > Hi Mandy, > > On 2018-05-31 18:19, mandy chung wrote: >> Hi Stefan, >> >> This change looks okay.? Can you add a comment to describe the >> expected memory pools for ZGC so that it explains why the min number >> of heap memory pools is 1. >> >> I think this test and possibly other memory pool tests should be >> re-examined and determine what proper verification should be done, for >> example, expected heap memory pools for a specific GC.? It may be >> worth to file a RFE to follow up. > > What if I change the code like this: > http://cr.openjdk.java.net/~stefank/8204173/webrev.02/ > > and then later add a new MemoryTestZGC.sh test, that calls MemoryTest 1 > 1 and explains that ZGC has one memory manager and one heap memory pool HotSpot nightly testing used to rotate among different GC configuration. MemoryTest.java would still fail if the test is running with ZGC when the VM option is specified via jtreg -vmoption configured for automated testing. Do you need to concern that case? If so, this version won't work. It's okay with me to follow up this in a separate issue. Maybe we have to dynamically set up the expected values if there is a way to determine which collector is running. Mandy From rkennke at redhat.com Thu May 31 18:10:15 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 31 May 2018 20:10:15 +0200 Subject: RFR: 8203353: Fixup inferred decorators in the interpreter In-Reply-To: References: <5AFD8F66.1070006@oracle.com> <6db04a41-069c-0680-ff6c-02dab0aeaecf@redhat.com> <5B10054B.6030206@oracle.com> Message-ID: <26d076df-4a3a-4171-3f57-82e4fad74166@redhat.com> Am 31.05.2018 um 19:21 schrieb coleen.phillimore at Oracle.com: > > http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/src/hotspot/cpu/x86/macroAssembler_x86.cpp.frames.html > > > 6334 void MacroAssembler::access_store_at(BasicType type, DecoratorSet > decorators, Address dst, Register src, > 6335????????????????????????????????????? Register tmp1, Register tmp2) { > 6336?? BarrierSetAssembler* bs = > BarrierSet::barrier_set()->barrier_set_assembler(); > 6337 decorators = AccessInternal::decorator_fixup(decorators); > 6338?? bool as_raw = (decorators & AS_RAW) != 0; > 6339?? if (as_raw) { > 6340???? bs->BarrierSetAssembler::store_at(this, decorators, type, dst, > src, tmp1, tmp2); > 6341?? } else { > 6342???? bs->store_at(this, decorators, type, dst, src, tmp1, tmp2); > 6343?? } > 6344 } > > This is a very strange function.? From the macroAssembler, how are we to > know that the cases in this if statement are different?? It seems like > this distinction should be made in BarrierSetAssembler class.?? It also > seems like whatever decorator fixup should also happen inside the > BarrierSetAssembler, because the placement of these calls to fixup the > decorators seems unknowable. The if statement will go away with this change: https://bugs.openjdk.java.net/browse/JDK-8203232 Roman > thanks, > Coleen > > On 5/31/18 10:23 AM, Erik ?sterlund wrote: >> Hi Roman, >> >> Thanks for the review! >> >> /Erik >> >> On 2018-05-31 15:40, Roman Kennke wrote: >>>> Hi, >>>> >>>> All subsystems that the access API touches, except the interpreter, >>>> infers a set of decorators according to rules specified in >>>> accessDecorators.hpp. These are typically sane defaults for things like >>>> reference strength, memory ordering, as well as inferring that >>>> IN_HEAP_ARRAY is IN_HEAP, and IN_CONCURRENT_ROOT is IN_ROOT. The >>>> interpreter should do that too. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/ >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8203353 >>> >>> Hi Erik, this seems reasonable and patch looks good. Thanks! >>> Roman >>> >> > From coleen.phillimore at oracle.com Thu May 31 18:13:50 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 31 May 2018 14:13:50 -0400 Subject: RFR: 8203353: Fixup inferred decorators in the interpreter In-Reply-To: <26d076df-4a3a-4171-3f57-82e4fad74166@redhat.com> References: <5AFD8F66.1070006@oracle.com> <6db04a41-069c-0680-ff6c-02dab0aeaecf@redhat.com> <5B10054B.6030206@oracle.com> <26d076df-4a3a-4171-3f57-82e4fad74166@redhat.com> Message-ID: <7200563f-87e8-3e82-e2ae-243ca0c25b12@oracle.com> On 5/31/18 2:10 PM, Roman Kennke wrote: > Am 31.05.2018 um 19:21 schrieb coleen.phillimore at Oracle.com: >> http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/src/hotspot/cpu/x86/macroAssembler_x86.cpp.frames.html >> >> >> 6334 void MacroAssembler::access_store_at(BasicType type, DecoratorSet >> decorators, Address dst, Register src, >> 6335????????????????????????????????????? Register tmp1, Register tmp2) { >> 6336?? BarrierSetAssembler* bs = >> BarrierSet::barrier_set()->barrier_set_assembler(); >> 6337 decorators = AccessInternal::decorator_fixup(decorators); >> 6338?? bool as_raw = (decorators & AS_RAW) != 0; >> 6339?? if (as_raw) { >> 6340???? bs->BarrierSetAssembler::store_at(this, decorators, type, dst, >> src, tmp1, tmp2); >> 6341?? } else { >> 6342???? bs->store_at(this, decorators, type, dst, src, tmp1, tmp2); >> 6343?? } >> 6344 } >> >> This is a very strange function.? From the macroAssembler, how are we to >> know that the cases in this if statement are different?? It seems like >> this distinction should be made in BarrierSetAssembler class.?? It also >> seems like whatever decorator fixup should also happen inside the >> BarrierSetAssembler, because the placement of these calls to fixup the >> decorators seems unknowable. > The if statement will go away with this change: > https://bugs.openjdk.java.net/browse/JDK-8203232 Ok, thanks!? Should the fixup decorators adjustment be done in the callee then? thanks, Coleen > > Roman > >> thanks, >> Coleen >> >> On 5/31/18 10:23 AM, Erik ?sterlund wrote: >>> Hi Roman, >>> >>> Thanks for the review! >>> >>> /Erik >>> >>> On 2018-05-31 15:40, Roman Kennke wrote: >>>>> Hi, >>>>> >>>>> All subsystems that the access API touches, except the interpreter, >>>>> infers a set of decorators according to rules specified in >>>>> accessDecorators.hpp. These are typically sane defaults for things like >>>>> reference strength, memory ordering, as well as inferring that >>>>> IN_HEAP_ARRAY is IN_HEAP, and IN_CONCURRENT_ROOT is IN_ROOT. The >>>>> interpreter should do that too. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/ >>>>> >>>>> Bug: >>>>> https://bugs.openjdk.java.net/browse/JDK-8203353 >>>> Hi Erik, this seems reasonable and patch looks good. Thanks! >>>> Roman >>>> > From stefan.karlsson at oracle.com Thu May 31 18:14:25 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 20:14:25 +0200 Subject: RFR: 8204173: Lower the minimum number of heap memory pools in MemoryTest.java In-Reply-To: <4b5718fc-3206-9146-1722-07306bbdb74a@oracle.com> References: <6280012d-35ef-5670-0429-65d6647792e4@oracle.com> <8bfbae10-44b0-088c-7679-eacdf5aff986@oracle.com> <42662a7a-a3fb-7bfe-ad9b-d50e5ade4661@oracle.com> <4b5718fc-3206-9146-1722-07306bbdb74a@oracle.com> Message-ID: On 2018-05-31 20:09, mandy chung wrote: > > > On 5/31/18 10:36 AM, Stefan Karlsson wrote: >> Hi Mandy, >> >> On 2018-05-31 18:19, mandy chung wrote: >>> Hi Stefan, >>> >>> This change looks okay.? Can you add a comment to describe the >>> expected memory pools for ZGC so that it explains why the min number >>> of heap memory pools is 1. >>> >>> I think this test and possibly other memory pool tests should be >>> re-examined and determine what proper verification should be done, >>> for example, expected heap memory pools for a specific GC.? It may >>> be worth to file a RFE to follow up. >> >> What if I change the code like this: >> http://cr.openjdk.java.net/~stefank/8204173/webrev.02/ >> >> and then later add a new MemoryTestZGC.sh test, that calls MemoryTest >> 1 1 and explains that ZGC has one memory manager and one heap memory >> pool > HotSpot nightly testing used to rotate among different GC > configuration. MemoryTest.java would still fail if the test is running > with ZGC when the VM option is specified via jtreg -vmoption > configured for automated testing.? Do you need to concern that case?? > If so, this version won't work. Right. The ZGC patch has a @requires vm.gc != "Z" row in MemoryTest.java. MemoryTestAllGC.sh is already filtered with @requires vm.gc=="Parallel" | vm.gc=="null". > > It's okay with me to follow up this in a separate issue.? Maybe we > have to dynamically set up the expected values if there is a way to > determine which collector is running. OK. Thanks, StefanK > > > Mandy From kim.barrett at oracle.com Thu May 31 18:24:06 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 31 May 2018 14:24:06 -0400 Subject: RFR: 8204165: Filter out tests requiring class unloading when ClassUnloading is disabled In-Reply-To: <6dc0988c-7324-7d87-826a-a66cdbfa2a50@oracle.com> References: <6dc0988c-7324-7d87-826a-a66cdbfa2a50@oracle.com> Message-ID: <9C8BF495-4FF5-4497-AB74-94EC6770798C@oracle.com> > On May 31, 2018, at 8:01 AM, Stefan Karlsson wrote: > > Hi all, > > Please review this patch to annotate jtreg tests that require ClassUnloading with @requires vm.opt.final.ClassUnloading. > > http://cr.openjdk.java.net/~stefank/8204165/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8204165 > > This @requires tag will filter out these tests when run with -XX:-ClassUnloading or when run with GC that doesn't support class unloading. > > For the discussion around the introduction of the vm.opt.final tags, see: http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-May/032586.html > > Thanks, > StefanK Looks good. (Some copyrights to update?) From stefan.karlsson at oracle.com Thu May 31 18:25:07 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 20:25:07 +0200 Subject: RFR: 8204165: Filter out tests requiring class unloading when ClassUnloading is disabled In-Reply-To: <9C8BF495-4FF5-4497-AB74-94EC6770798C@oracle.com> References: <6dc0988c-7324-7d87-826a-a66cdbfa2a50@oracle.com> <9C8BF495-4FF5-4497-AB74-94EC6770798C@oracle.com> Message-ID: Thanks, Kim. StefanK On 2018-05-31 20:24, Kim Barrett wrote: >> On May 31, 2018, at 8:01 AM, Stefan Karlsson wrote: >> >> Hi all, >> >> Please review this patch to annotate jtreg tests that require ClassUnloading with @requires vm.opt.final.ClassUnloading. >> >> http://cr.openjdk.java.net/~stefank/8204165/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8204165 >> >> This @requires tag will filter out these tests when run with -XX:-ClassUnloading or when run with GC that doesn't support class unloading. >> >> For the discussion around the introduction of the vm.opt.final tags, see: http://mail.openjdk.java.net/pipermail/hotspot-dev/2018-May/032586.html >> >> Thanks, >> StefanK > Looks good. (Some copyrights to update?) > From mandy.chung at oracle.com Thu May 31 18:38:29 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 31 May 2018 11:38:29 -0700 Subject: RFR: 8204173: Lower the minimum number of heap memory pools in MemoryTest.java In-Reply-To: References: <6280012d-35ef-5670-0429-65d6647792e4@oracle.com> <8bfbae10-44b0-088c-7679-eacdf5aff986@oracle.com> <42662a7a-a3fb-7bfe-ad9b-d50e5ade4661@oracle.com> <4b5718fc-3206-9146-1722-07306bbdb74a@oracle.com> Message-ID: On 5/31/18 11:14 AM, Stefan Karlsson wrote: > Right. The ZGC patch has a @requires vm.gc != "Z" row in > MemoryTest.java. Ah. This change is fine then. Mandy From vladimir.kozlov at oracle.com Thu May 31 18:47:08 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 31 May 2018 11:47:08 -0700 Subject: [11] RFR (M) 8202611: [GRAAL] Exclude CMS GC testing from runs with Graal In-Reply-To: <923C4D3F-AE62-4473-B39C-6C93B3ED8C10@oracle.com> References: <923C4D3F-AE62-4473-B39C-6C93B3ED8C10@oracle.com> Message-ID: <2e4f42e0-72ca-0895-f1a5-32cf4d072f61@oracle.com> Thank you, Kim Vladimir On 5/31/18 10:59 AM, Kim Barrett wrote: >> On May 7, 2018, at 2:33 PM, Vladimir Kozlov wrote: >> >> http://cr.openjdk.java.net/~kvn/8202611/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8202611 >> >> JDK-8184349 handles Hotspot changes to add check that Graal support GC and exit on error if it does not. >> Since tests changes are big I decided to do it separate and may be push first. >> >> These changes excluded tests which use CMS from Graal runs. >> >> @requires !vm.graal.enabled was added for tests which use only CMS GC explicitly. >> >> Undocumented jtreg functionality is use to split @run command with -XX:+UseConcMarkSweepGC into separate jtreg comment block if test has several @run commands. Jtreg treats such test as 2 tests. >> >> New test API sun/hotspot/code/Compiler.java (which uses WhiteBox API) is added to check if particular JIT compiler is enabled. The Graal related code was copied from VMProps.java. >> This was added for cases when a test forks new process and use CMS GC. New API is used to guard such code to not run with Graal. >> >> Tested with Graal enabled as JIT. >> >> -- >> Thanks, >> Vladimir > > Looks good. > From rkennke at redhat.com Thu May 31 19:01:06 2018 From: rkennke at redhat.com (Roman Kennke) Date: Thu, 31 May 2018 21:01:06 +0200 Subject: RFR: 8203353: Fixup inferred decorators in the interpreter In-Reply-To: <7200563f-87e8-3e82-e2ae-243ca0c25b12@oracle.com> References: <5AFD8F66.1070006@oracle.com> <6db04a41-069c-0680-ff6c-02dab0aeaecf@redhat.com> <5B10054B.6030206@oracle.com> <26d076df-4a3a-4171-3f57-82e4fad74166@redhat.com> <7200563f-87e8-3e82-e2ae-243ca0c25b12@oracle.com> Message-ID: <3f1011a4-0250-75f3-4e75-44510a8d33df@redhat.com> Am 31.05.2018 um 20:13 schrieb coleen.phillimore at oracle.com: > > > On 5/31/18 2:10 PM, Roman Kennke wrote: >> Am 31.05.2018 um 19:21 schrieb coleen.phillimore at Oracle.com: >>> http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/src/hotspot/cpu/x86/macroAssembler_x86.cpp.frames.html >>> >>> >>> >>> 6334 void MacroAssembler::access_store_at(BasicType type, DecoratorSet >>> decorators, Address dst, Register src, >>> 6335????????????????????????????????????? Register tmp1, Register >>> tmp2) { >>> 6336?? BarrierSetAssembler* bs = >>> BarrierSet::barrier_set()->barrier_set_assembler(); >>> 6337 decorators = AccessInternal::decorator_fixup(decorators); >>> 6338?? bool as_raw = (decorators & AS_RAW) != 0; >>> 6339?? if (as_raw) { >>> 6340???? bs->BarrierSetAssembler::store_at(this, decorators, type, dst, >>> src, tmp1, tmp2); >>> 6341?? } else { >>> 6342???? bs->store_at(this, decorators, type, dst, src, tmp1, tmp2); >>> 6343?? } >>> 6344 } >>> >>> This is a very strange function.? From the macroAssembler, how are we to >>> know that the cases in this if statement are different?? It seems like >>> this distinction should be made in BarrierSetAssembler class.?? It also >>> seems like whatever decorator fixup should also happen inside the >>> BarrierSetAssembler, because the placement of these calls to fixup the >>> decorators seems unknowable. >> The if statement will go away with this change: >> https://bugs.openjdk.java.net/browse/JDK-8203232 > > Ok, thanks!? Should the fixup decorators adjustment be done in the > callee then? I think doing it in the access_X_at() helper functions is ok. It should be done before calling into the BarrierSetAssembler methods (unless we want to require each implementation to do the fixup, which would be ugly), but not at each callee. Roman > > thanks, > Coleen >> >> Roman >> >>> thanks, >>> Coleen >>> >>> On 5/31/18 10:23 AM, Erik ?sterlund wrote: >>>> Hi Roman, >>>> >>>> Thanks for the review! >>>> >>>> /Erik >>>> >>>> On 2018-05-31 15:40, Roman Kennke wrote: >>>>>> Hi, >>>>>> >>>>>> All subsystems that the access API touches, except the interpreter, >>>>>> infers a set of decorators according to rules specified in >>>>>> accessDecorators.hpp. These are typically sane defaults for things >>>>>> like >>>>>> reference strength, memory ordering, as well as inferring that >>>>>> IN_HEAP_ARRAY is IN_HEAP, and IN_CONCURRENT_ROOT is IN_ROOT. The >>>>>> interpreter should do that too. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/ >>>>>> >>>>>> Bug: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8203353 >>>>> Hi Erik, this seems reasonable and patch looks good. Thanks! >>>>> Roman >>>>> >> > From vladimir.kozlov at oracle.com Thu May 31 19:06:45 2018 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Thu, 31 May 2018 12:06:45 -0700 Subject: RFR(L) : 8202812 : [TESTBUG] Open source VM testbase compiler tests In-Reply-To: References: Message-ID: <6abc20ff-f125-80cf-98d6-2e69f0181673@oracle.com> Looks good to me. I don't understand how these tests trigger JIT compilation? main() does not have any loops. Where is GoldChecker class? What --java flag mean in @run command? Thanks, Vladimir On 5/18/18 10:50 AM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8202812/webrev.00/index.html >> 56733 lines changed: 56732 ins; 0 del; 1 mod; 1 > > Hi all, > > could you please review the patch which open sources compiler tests from vm testbase? these tests were developed in different time to cover different parts of JITs. > > As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8202812 > webrev: http://cr.openjdk.java.net/~iignatyev/8202812/webrev.00/index.html > testing: :vmTestbase_vm_compiler test group > > Thanks, > -- Igor > From igor.ignatyev at oracle.com Thu May 31 19:19:30 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 31 May 2018 12:19:30 -0700 Subject: RFR(L) : 8202812 : [TESTBUG] Open source VM testbase compiler tests In-Reply-To: <6abc20ff-f125-80cf-98d6-2e69f0181673@oracle.com> References: <6abc20ff-f125-80cf-98d6-2e69f0181673@oracle.com> Message-ID: > On May 31, 2018, at 12:06 PM, Vladimir Kozlov wrote: > > Looks good to me. thanks for reviewing it! > I don't understand how these tests trigger JIT compilation? main() does not have any loops. which test/tests are you asking about? I've looked at a few tests, all of them do have loops (not necessary right in main). in any case, this is how these tests were for a long time, they definitely require reevaluation and rewriting. > Where is GoldChecker class? GoldChecker is defined in test/hotspot/jtreg/vmTestbase/nsk/share/GoldChecker.java, which has been integrated by 8199643: [TESTBUG] Open source common VM testbase code. > What --java flag mean in @run command? "--java" isn't a flag of @run, it's the flag of ExecDriver (test/hotspot/jtreg/vmTestbase/ExecDriver.java) which makes it to interpret the rest of arguments as regular arguments of java. ExecDriver is needed by some tests b/c they either run package-private classes (which regular jtreg's @run can't do) or use zero-exit code to signalize passed status (jtreg treats only 95 as passed). ExecDriver was used to reduce changes in the tests during their conversion to jtreg format and eventually it should go for good. Thanks, -- Igor > > Thanks, > Vladimir > > On 5/18/18 10:50 AM, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8202812/webrev.00/index.html >>> 56733 lines changed: 56732 ins; 0 del; 1 mod; 1 >> Hi all, >> could you please review the patch which open sources compiler tests from vm testbase? these tests were developed in different time to cover different parts of JITs. >> As usually w/ VM testbase code, these tests are old, they have been run in hotspot testing for a long period of time. Originally, these tests were run by a test harness different from jtreg and had different build and execution schemes, some parts couldn't be easily translated to jtreg, so tests might have actions or pieces of code which look weird. In a long term, we are planning to rework them. >> JBS: https://bugs.openjdk.java.net/browse/JDK-8202812 >> webrev: http://cr.openjdk.java.net/~iignatyev/8202812/webrev.00/index.html >> testing: :vmTestbase_vm_compiler test group >> Thanks, >> -- Igor From kim.barrett at oracle.com Thu May 31 19:20:54 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 31 May 2018 15:20:54 -0400 Subject: RFR: 8204167: Filter out tests requiring compressed oops when CompressedOops is disabled In-Reply-To: <2da42530-7549-dfe3-7950-a45eb258e522@oracle.com> References: <2da42530-7549-dfe3-7950-a45eb258e522@oracle.com> Message-ID: <3944DDF8-5C4C-4C69-A3E0-A71CD75A5C16@oracle.com> > On May 31, 2018, at 8:09 AM, Stefan Karlsson wrote: > > Hi all, > > Please review this patch to add @requires vm.opt.final.UseCompressedOops to those jtreg tests that requires compressed oops. > > http://cr.openjdk.java.net/~stefank/8204167/webrev.01/ > https://bugs.openjdk.java.net/browse/JDK-8204167 > > With this patch we now filter out tests that require compressed oops, when -XX:-UseCompressedOops is passed or ZGC is tested. > > Thanks, > StefanK test/hotspot/jtreg/runtime/CompressedOops/CompressedClassPointers.java test/hotspot/jtreg/runtime/CompressedOops/CompressedClassSpaceSize.java test/hotspot/jtreg/runtime/Metaspace/MaxMetaspaceSizeTest.java These seem to be testing UseCompressedClassPointers rather than UseCompressedOops. Shouldn't they require UseCompressedClassPointers instead? I know UCCP requires UCO, but requiring UCO here seems odd. From coleen.phillimore at oracle.com Thu May 31 19:23:41 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 31 May 2018 15:23:41 -0400 Subject: RFR: 8203353: Fixup inferred decorators in the interpreter In-Reply-To: <3f1011a4-0250-75f3-4e75-44510a8d33df@redhat.com> References: <5AFD8F66.1070006@oracle.com> <6db04a41-069c-0680-ff6c-02dab0aeaecf@redhat.com> <5B10054B.6030206@oracle.com> <26d076df-4a3a-4171-3f57-82e4fad74166@redhat.com> <7200563f-87e8-3e82-e2ae-243ca0c25b12@oracle.com> <3f1011a4-0250-75f3-4e75-44510a8d33df@redhat.com> Message-ID: On 5/31/18 3:01 PM, Roman Kennke wrote: > Am 31.05.2018 um 20:13 schrieb coleen.phillimore at oracle.com: >> >> On 5/31/18 2:10 PM, Roman Kennke wrote: >>> Am 31.05.2018 um 19:21 schrieb coleen.phillimore at Oracle.com: >>>> http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/src/hotspot/cpu/x86/macroAssembler_x86.cpp.frames.html >>>> >>>> >>>> >>>> 6334 void MacroAssembler::access_store_at(BasicType type, DecoratorSet >>>> decorators, Address dst, Register src, >>>> 6335????????????????????????????????????? Register tmp1, Register >>>> tmp2) { >>>> 6336?? BarrierSetAssembler* bs = >>>> BarrierSet::barrier_set()->barrier_set_assembler(); >>>> 6337 decorators = AccessInternal::decorator_fixup(decorators); >>>> 6338?? bool as_raw = (decorators & AS_RAW) != 0; >>>> 6339?? if (as_raw) { >>>> 6340???? bs->BarrierSetAssembler::store_at(this, decorators, type, dst, >>>> src, tmp1, tmp2); >>>> 6341?? } else { >>>> 6342???? bs->store_at(this, decorators, type, dst, src, tmp1, tmp2); >>>> 6343?? } >>>> 6344 } >>>> >>>> This is a very strange function.? From the macroAssembler, how are we to >>>> know that the cases in this if statement are different?? It seems like >>>> this distinction should be made in BarrierSetAssembler class.?? It also >>>> seems like whatever decorator fixup should also happen inside the >>>> BarrierSetAssembler, because the placement of these calls to fixup the >>>> decorators seems unknowable. >>> The if statement will go away with this change: >>> https://bugs.openjdk.java.net/browse/JDK-8203232 >> Ok, thanks!? Should the fixup decorators adjustment be done in the >> callee then? > I think doing it in the access_X_at() helper functions is ok. It should > be done before calling into the BarrierSetAssembler methods (unless we > want to require each implementation to do the fixup, which would be > ugly), but not at each callee. Ok.? I'm fine with the change then. Thanks, Coleen > Roman > > >> thanks, >> Coleen >>> Roman >>> >>>> thanks, >>>> Coleen >>>> >>>> On 5/31/18 10:23 AM, Erik ?sterlund wrote: >>>>> Hi Roman, >>>>> >>>>> Thanks for the review! >>>>> >>>>> /Erik >>>>> >>>>> On 2018-05-31 15:40, Roman Kennke wrote: >>>>>>> Hi, >>>>>>> >>>>>>> All subsystems that the access API touches, except the interpreter, >>>>>>> infers a set of decorators according to rules specified in >>>>>>> accessDecorators.hpp. These are typically sane defaults for things >>>>>>> like >>>>>>> reference strength, memory ordering, as well as inferring that >>>>>>> IN_HEAP_ARRAY is IN_HEAP, and IN_CONCURRENT_ROOT is IN_ROOT. The >>>>>>> interpreter should do that too. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/ >>>>>>> >>>>>>> Bug: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8203353 >>>>>> Hi Erik, this seems reasonable and patch looks good. Thanks! >>>>>> Roman >>>>>> > From kim.barrett at oracle.com Thu May 31 19:24:38 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 31 May 2018 15:24:38 -0400 Subject: RFR: 8204179: [BACKOUT] OopStorage should use GlobalCounter Message-ID: <71B06950-E9F1-43F2-B972-E967BEBD202B@oracle.com> Please review this backout of JDK-8202945. See CR for details. The changeset was generated using hg backout and applies cleanly. CR: https://bugs.openjdk.java.net/browse/JDK-8204179 Webrev: http://cr.openjdk.java.net/~kbarrett/8204179/open.00/ Testing: Mach5 tier1 From per.liden at oracle.com Thu May 31 19:31:05 2018 From: per.liden at oracle.com (Per Liden) Date: Thu, 31 May 2018 21:31:05 +0200 Subject: RFR: 8204179: [BACKOUT] OopStorage should use GlobalCounter In-Reply-To: <71B06950-E9F1-43F2-B972-E967BEBD202B@oracle.com> References: <71B06950-E9F1-43F2-B972-E967BEBD202B@oracle.com> Message-ID: <0d631439-6e47-9cc6-3aa6-d14e5568eba7@oracle.com> Looks good! /Per On 05/31/2018 09:24 PM, Kim Barrett wrote: > Please review this backout of JDK-8202945. See CR for details. > The changeset was generated using hg backout and applies cleanly. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8204179 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8204179/open.00/ > > Testing: > Mach5 tier1 > From erik.osterlund at oracle.com Thu May 31 19:32:11 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Thu, 31 May 2018 21:32:11 +0200 Subject: RFR: 8204179: [BACKOUT] OopStorage should use GlobalCounter In-Reply-To: <71B06950-E9F1-43F2-B972-E967BEBD202B@oracle.com> References: <71B06950-E9F1-43F2-B972-E967BEBD202B@oracle.com> Message-ID: Hi Kim, The backout looks good. Thanks, /Erik > On 31 May 2018, at 21:24, Kim Barrett wrote: > > Please review this backout of JDK-8202945. See CR for details. > The changeset was generated using hg backout and applies cleanly. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8204179 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8204179/open.00/ > > Testing: > Mach5 tier1 > From erik.osterlund at oracle.com Thu May 31 19:33:46 2018 From: erik.osterlund at oracle.com (Erik Osterlund) Date: Thu, 31 May 2018 21:33:46 +0200 Subject: RFR: 8203353: Fixup inferred decorators in the interpreter In-Reply-To: References: <5AFD8F66.1070006@oracle.com> <6db04a41-069c-0680-ff6c-02dab0aeaecf@redhat.com> <5B10054B.6030206@oracle.com> <26d076df-4a3a-4171-3f57-82e4fad74166@redhat.com> <7200563f-87e8-3e82-e2ae-243ca0c25b12@oracle.com> <3f1011a4-0250-75f3-4e75-44510a8d33df@redhat.com> Message-ID: <183353B0-4218-4548-A3C1-6E84213D4C4C@oracle.com> Hi Coleen, Thanks for the review. /Erik > On 31 May 2018, at 21:23, coleen.phillimore at oracle.com wrote: > > > >> On 5/31/18 3:01 PM, Roman Kennke wrote: >>> Am 31.05.2018 um 20:13 schrieb coleen.phillimore at oracle.com: >>> >>>> On 5/31/18 2:10 PM, Roman Kennke wrote: >>>>> Am 31.05.2018 um 19:21 schrieb coleen.phillimore at Oracle.com: >>>>> http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/src/hotspot/cpu/x86/macroAssembler_x86.cpp.frames.html >>>>> >>>>> >>>>> >>>>> 6334 void MacroAssembler::access_store_at(BasicType type, DecoratorSet >>>>> decorators, Address dst, Register src, >>>>> 6335 Register tmp1, Register >>>>> tmp2) { >>>>> 6336 BarrierSetAssembler* bs = >>>>> BarrierSet::barrier_set()->barrier_set_assembler(); >>>>> 6337 decorators = AccessInternal::decorator_fixup(decorators); >>>>> 6338 bool as_raw = (decorators & AS_RAW) != 0; >>>>> 6339 if (as_raw) { >>>>> 6340 bs->BarrierSetAssembler::store_at(this, decorators, type, dst, >>>>> src, tmp1, tmp2); >>>>> 6341 } else { >>>>> 6342 bs->store_at(this, decorators, type, dst, src, tmp1, tmp2); >>>>> 6343 } >>>>> 6344 } >>>>> >>>>> This is a very strange function. From the macroAssembler, how are we to >>>>> know that the cases in this if statement are different? It seems like >>>>> this distinction should be made in BarrierSetAssembler class. It also >>>>> seems like whatever decorator fixup should also happen inside the >>>>> BarrierSetAssembler, because the placement of these calls to fixup the >>>>> decorators seems unknowable. >>>> The if statement will go away with this change: >>>> https://bugs.openjdk.java.net/browse/JDK-8203232 >>> Ok, thanks! Should the fixup decorators adjustment be done in the >>> callee then? >> I think doing it in the access_X_at() helper functions is ok. It should >> be done before calling into the BarrierSetAssembler methods (unless we >> want to require each implementation to do the fixup, which would be >> ugly), but not at each callee. > > Ok. I'm fine with the change then. > > Thanks, > Coleen >> Roman >> >> >>> thanks, >>> Coleen >>>> Roman >>>> >>>>> thanks, >>>>> Coleen >>>>> >>>>>> On 5/31/18 10:23 AM, Erik ?sterlund wrote: >>>>>> Hi Roman, >>>>>> >>>>>> Thanks for the review! >>>>>> >>>>>> /Erik >>>>>> >>>>>> On 2018-05-31 15:40, Roman Kennke wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> All subsystems that the access API touches, except the interpreter, >>>>>>>> infers a set of decorators according to rules specified in >>>>>>>> accessDecorators.hpp. These are typically sane defaults for things >>>>>>>> like >>>>>>>> reference strength, memory ordering, as well as inferring that >>>>>>>> IN_HEAP_ARRAY is IN_HEAP, and IN_CONCURRENT_ROOT is IN_ROOT. The >>>>>>>> interpreter should do that too. >>>>>>>> >>>>>>>> Webrev: >>>>>>>> http://cr.openjdk.java.net/~eosterlund/8203353/webrev.00/ >>>>>>>> >>>>>>>> Bug: >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8203353 >>>>>>> Hi Erik, this seems reasonable and patch looks good. Thanks! >>>>>>> Roman >>>>>>> >> > From kim.barrett at oracle.com Thu May 31 19:35:04 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 31 May 2018 15:35:04 -0400 Subject: RFR: 8204179: [BACKOUT] OopStorage should use GlobalCounter In-Reply-To: References: <71B06950-E9F1-43F2-B972-E967BEBD202B@oracle.com> Message-ID: <13D2E82B-DC35-4DB7-99A0-C39E9D330776@oracle.com> > On May 31, 2018, at 3:32 PM, Erik Osterlund wrote: > > Hi Kim, > > The backout looks good. > > Thanks, > /Erik Thanks, Erik and Per. From stefan.karlsson at oracle.com Thu May 31 19:59:19 2018 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Thu, 31 May 2018 21:59:19 +0200 Subject: RFR: 8204167: Filter out tests requiring compressed oops when CompressedOops is disabled In-Reply-To: <3944DDF8-5C4C-4C69-A3E0-A71CD75A5C16@oracle.com> References: <2da42530-7549-dfe3-7950-a45eb258e522@oracle.com> <3944DDF8-5C4C-4C69-A3E0-A71CD75A5C16@oracle.com> Message-ID: On 2018-05-31 21:20, Kim Barrett wrote: >> On May 31, 2018, at 8:09 AM, Stefan Karlsson wrote: >> >> Hi all, >> >> Please review this patch to add @requires vm.opt.final.UseCompressedOops to those jtreg tests that requires compressed oops. >> >> http://cr.openjdk.java.net/~stefank/8204167/webrev.01/ >> https://bugs.openjdk.java.net/browse/JDK-8204167 >> >> With this patch we now filter out tests that require compressed oops, when -XX:-UseCompressedOops is passed or ZGC is tested. >> >> Thanks, >> StefanK > test/hotspot/jtreg/runtime/CompressedOops/CompressedClassPointers.java > test/hotspot/jtreg/runtime/CompressedOops/CompressedClassSpaceSize.java > test/hotspot/jtreg/runtime/Metaspace/MaxMetaspaceSizeTest.java > > These seem to be testing UseCompressedClassPointers rather than > UseCompressedOops. Shouldn't they require UseCompressedClassPointers > instead? I know UCCP requires UCO, but requiring UCO here seems odd. Yes, that makes sense. New webrevs: http://cr.openjdk.java.net/~stefank/8204167/webrev.02.delta/ http://cr.openjdk.java.net/~stefank/8204167/webrev.02/ Thanks, StefanK From kim.barrett at oracle.com Thu May 31 21:05:19 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 31 May 2018 17:05:19 -0400 Subject: RFR: 8204167: Filter out tests requiring compressed oops when CompressedOops is disabled In-Reply-To: References: <2da42530-7549-dfe3-7950-a45eb258e522@oracle.com> <3944DDF8-5C4C-4C69-A3E0-A71CD75A5C16@oracle.com> Message-ID: <8171DA99-ECA6-4038-BA03-DA0B20BB1C9C@oracle.com> > On May 31, 2018, at 3:59 PM, Stefan Karlsson wrote: > > On 2018-05-31 21:20, Kim Barrett wrote: >>> On May 31, 2018, at 8:09 AM, Stefan Karlsson wrote: >>> >>> Hi all, >>> >>> Please review this patch to add @requires vm.opt.final.UseCompressedOops to those jtreg tests that requires compressed oops. >>> >>> http://cr.openjdk.java.net/~stefank/8204167/webrev.01/ >>> https://bugs.openjdk.java.net/browse/JDK-8204167 >>> >>> With this patch we now filter out tests that require compressed oops, when -XX:-UseCompressedOops is passed or ZGC is tested. >>> >>> Thanks, >>> StefanK >> test/hotspot/jtreg/runtime/CompressedOops/CompressedClassPointers.java >> test/hotspot/jtreg/runtime/CompressedOops/CompressedClassSpaceSize.java >> test/hotspot/jtreg/runtime/Metaspace/MaxMetaspaceSizeTest.java >> >> These seem to be testing UseCompressedClassPointers rather than >> UseCompressedOops. Shouldn't they require UseCompressedClassPointers >> instead? I know UCCP requires UCO, but requiring UCO here seems odd. > > Yes, that makes sense. New webrevs: > > http://cr.openjdk.java.net/~stefank/8204167/webrev.02.delta/ > http://cr.openjdk.java.net/~stefank/8204167/webrev.02/ > > Thanks, > StefanK Looks good. From kim.barrett at oracle.com Thu May 31 21:35:23 2018 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 31 May 2018 17:35:23 -0400 Subject: RFR: 8203948: Expand JVMTI callback notion of "internal threads" In-Reply-To: <28b38b3c-c17a-b5f3-6bcc-ce872af975e7@oracle.com> References: <28C16D3C-7C8F-44B3-8ED1-A24CAE4EA8AB@oracle.com> <28b38b3c-c17a-b5f3-6bcc-ce872af975e7@oracle.com> Message-ID: <873EFDBD-EDD8-4A1B-AEE4-67DC7C8B93CF@oracle.com> > On May 30, 2018, at 10:18 PM, David Holmes wrote: > > Hi Kim, > > I guess I'd like this more if NamedThread were called something like InternalThread as that captures the actual commonality of these threads (we don't really care they they have names!). But that's not your fault. Agreed. There don?t seem to be that many places where that naming is used, so a little nomenclature improvement wouldn?t be a major project. Just not something I want to take on as part of this. > I'm still tempted to just remove the check altogether now there is only one thread we're excluding - do we really need to guard against the WatcherThread using this code? I waffled on that, and ended up not. > But its fine to push as-is. Thanks. > Thanks, > David From coleen.phillimore at oracle.com Thu May 31 22:14:57 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 31 May 2018 18:14:57 -0400 Subject: RFR: 8204167: Filter out tests requiring compressed oops when CompressedOops is disabled In-Reply-To: <8171DA99-ECA6-4038-BA03-DA0B20BB1C9C@oracle.com> References: <2da42530-7549-dfe3-7950-a45eb258e522@oracle.com> <3944DDF8-5C4C-4C69-A3E0-A71CD75A5C16@oracle.com> <8171DA99-ECA6-4038-BA03-DA0B20BB1C9C@oracle.com> Message-ID: Looks good! Coleen On 5/31/18 5:05 PM, Kim Barrett wrote: >> On May 31, 2018, at 3:59 PM, Stefan Karlsson wrote: >> >> On 2018-05-31 21:20, Kim Barrett wrote: >>>> On May 31, 2018, at 8:09 AM, Stefan Karlsson wrote: >>>> >>>> Hi all, >>>> >>>> Please review this patch to add @requires vm.opt.final.UseCompressedOops to those jtreg tests that requires compressed oops. >>>> >>>> http://cr.openjdk.java.net/~stefank/8204167/webrev.01/ >>>> https://bugs.openjdk.java.net/browse/JDK-8204167 >>>> >>>> With this patch we now filter out tests that require compressed oops, when -XX:-UseCompressedOops is passed or ZGC is tested. >>>> >>>> Thanks, >>>> StefanK >>> test/hotspot/jtreg/runtime/CompressedOops/CompressedClassPointers.java >>> test/hotspot/jtreg/runtime/CompressedOops/CompressedClassSpaceSize.java >>> test/hotspot/jtreg/runtime/Metaspace/MaxMetaspaceSizeTest.java >>> >>> These seem to be testing UseCompressedClassPointers rather than >>> UseCompressedOops. Shouldn't they require UseCompressedClassPointers >>> instead? I know UCCP requires UCO, but requiring UCO here seems odd. >> Yes, that makes sense. New webrevs: >> >> http://cr.openjdk.java.net/~stefank/8204167/webrev.02.delta/ >> http://cr.openjdk.java.net/~stefank/8204167/webrev.02/ >> >> Thanks, >> StefanK > Looks good. > From coleen.phillimore at oracle.com Thu May 31 23:10:33 2018 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Thu, 31 May 2018 19:10:33 -0400 Subject: RFR (S) 8204195: Clean up macroAssembler.inline.hpp and other inline.hpp files included in .hpp files Message-ID: <335ac0b6-84f4-8ae5-ad32-0ba7d7260009@oracle.com> Summary: Moved macroAssembler.inline.hpp out of header file and distributed to .cpp files that included them: ie. c1_MacroAssembler.hpp and interp_masm.hpp. Also freeList.inline.hpp and allocation.inline.hpp. open webrev at http://cr.openjdk.java.net/~coleenp/8204195.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8204195 Tested with mach5 hs-tier1,2 on Oracle platforms: linux-x64, solaris-sparcv9, macosx-x64 and windows-x64.? Also tested zero and aarch64 fastdebug builds, and linux-x64 without precompiled headers. ? Please test other platforms, like arm32, ppc and s390!? I think these are the last platform dependent inline files that are included by .hpp files. Thanks, Coleen From jiangli.zhou at oracle.com Thu May 31 23:52:31 2018 From: jiangli.zhou at oracle.com (Jiangli Zhou) Date: Thu, 31 May 2018 16:52:31 -0700 Subject: RFR (S) 8204195: Clean up macroAssembler.inline.hpp and other inline.hpp files included in .hpp files In-Reply-To: <335ac0b6-84f4-8ae5-ad32-0ba7d7260009@oracle.com> References: <335ac0b6-84f4-8ae5-ad32-0ba7d7260009@oracle.com> Message-ID: <4A4CB4E4-7340-4798-B4DA-D3D48F9A30DA@oracle.com> The changes look good to me. Thanks, Jiangli > On May 31, 2018, at 4:10 PM, coleen.phillimore at oracle.com wrote: > > Summary: Moved macroAssembler.inline.hpp out of header file and distributed to .cpp files that included them: ie. c1_MacroAssembler.hpp and interp_masm.hpp. Also freeList.inline.hpp and allocation.inline.hpp. > > open webrev at http://cr.openjdk.java.net/~coleenp/8204195.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8204195 > > Tested with mach5 hs-tier1,2 on Oracle platforms: linux-x64, solaris-sparcv9, macosx-x64 and windows-x64. Also tested zero and aarch64 fastdebug builds, and linux-x64 without precompiled headers. Please test other platforms, like arm32, ppc and s390! I think these are the last platform dependent inline files that are included by .hpp files. > > Thanks, > Coleen