From thomas.stuefe at gmail.com Sat Aug 1 05:22:10 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 1 Aug 2020 07:22:10 +0200 Subject: RFR(xxs): 8250911: [windows] os::pd_map_memory() error detection broken Message-ID: Hi, While trying to find out why https://bugs.openjdk.java.net/browse/JDK-8249943 consistently fails at SAP nightlies, I stumbled upon a small zero-day (AFAICS) bug in the Windows version of os::pd_map_memory(). JBS: https://bugs.openjdk.java.net/browse/JDK-8250911 webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8250911---windows--os--pd_map_memory-error-detection-broken/webrev.00/webrev/ While this is annoying I do not expect fixing this bug will shake loose many follow up bugs. Currently, if the CreateFile call fails but we don't get it one of the follow up system calls will fail. All this fix provides is better error analysis. Thanks, Thomas From ioi.lam at oracle.com Sat Aug 1 05:41:48 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 31 Jul 2020 22:41:48 -0700 Subject: RFR(xxs): 8250911: [windows] os::pd_map_memory() error detection broken In-Reply-To: References: Message-ID: Hi Thomas, Looks good to me. Thanks for fixing this. Thanks - Ioi On 7/31/20 10:22 PM, Thomas St?fe wrote: > Hi, > > While trying to find out why > https://bugs.openjdk.java.net/browse/JDK-8249943 consistently fails at SAP > nightlies, I stumbled upon a small zero-day (AFAICS) bug in the Windows > version of os::pd_map_memory(). > > JBS: https://bugs.openjdk.java.net/browse/JDK-8250911 > webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8250911---windows--os--pd_map_memory-error-detection-broken/webrev.00/webrev/ > > While this is annoying I do not expect fixing this bug will shake loose > many follow up bugs. Currently, if the CreateFile call fails but we don't > get it one of the follow up system calls will fail. All this fix provides > is better error analysis. > > Thanks, Thomas From ioi.lam at oracle.com Sat Aug 1 06:36:58 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 31 Jul 2020 23:36:58 -0700 Subject: RFR(M) 8234693 Consolidate CDS static and dynamic archive dumping code Message-ID: <07b2ee18-0a5f-0839-0a6c-b54d788e4cd2@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8234693 https://cr.openjdk.java.net/~iklam/jdk16/8234693-consolidate-static-dynamic-cds-dump.v01/ https://wiki.openjdk.java.net/display/HotSpot/How+CDS+Copies+Class+Metadata+into+the+Archive Currently there are two very similar implementations for static CDS archive dumping (metaspaceShared.cpp) and dynamic CDS archive dumping (dynamicArchive.cpp). I've merged the two versions and moved to a new file, archiveBuilder.cpp. I also improved the copying algorithm to speed up dumping. Times for dumping 20000+ classes: ? ?? ????? static??????? dynamic ??? Old??? 42.655 sec??? 67.014 sec ??? New??? 37.027 sec??? 34.974 sec Normally I would not try to optimize when refactoring code. I made an earlier, naive attempt to use the old dynamic dumper for static dumping (because dynamic dumping is essentially a superset of static dumping). However, that caused significant regression in static dumping speed. Hopefully the new algorithm is both faster and easier to understand. Please see the wiki document for more information. Tested with mach5 tiers 1-4. Thanks - Ioi From thomas.stuefe at gmail.com Sat Aug 1 06:41:26 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 1 Aug 2020 08:41:26 +0200 Subject: RFR(xxs): 8250911: [windows] os::pd_map_memory() error detection broken In-Reply-To: References: Message-ID: Thank you Ioi! On Sat, Aug 1, 2020 at 7:42 AM Ioi Lam wrote: > Hi Thomas, > > Looks good to me. Thanks for fixing this. > > Thanks > - Ioi > > On 7/31/20 10:22 PM, Thomas St?fe wrote: > > Hi, > > > > While trying to find out why > > https://bugs.openjdk.java.net/browse/JDK-8249943 consistently fails at > SAP > > nightlies, I stumbled upon a small zero-day (AFAICS) bug in the Windows > > version of os::pd_map_memory(). > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8250911 > > webrev: > > > http://cr.openjdk.java.net/~stuefe/webrevs/8250911---windows--os--pd_map_memory-error-detection-broken/webrev.00/webrev/ > > > > While this is annoying I do not expect fixing this bug will shake loose > > many follow up bugs. Currently, if the CreateFile call fails but we don't > > get it one of the follow up system calls will fail. All this fix provides > > is better error analysis. > > > > Thanks, Thomas > > From kim.barrett at oracle.com Sat Aug 1 09:47:13 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 1 Aug 2020 05:47:13 -0400 Subject: RFR(S): JDK-8247938: Change various JVM enums like LinkInfo::AccessCheck and Klass::DefaultsLookupMode to enum class In-Reply-To: References: Message-ID: <3A1268FE-1231-4043-8CA8-06C9F3160421@oracle.com> > On Jul 31, 2020, at 1:55 PM, Lois Foltan wrote: > > Please review the webrev to change the following unscoped enumeration declarations to scoped enumerations. > > Klass::DefaultsLookupMode > Klass::OverpassLookupMode > Klass::StaticLookupMode > Klass::PrivateLookupMode > LinkInfo::AccessCheck > > With C++11/14 enum class provides the added type safety to prevent 0 (integral) or false (boolean) to be implicitly converted to AccessCheck::needs_access_check for example. > > open webrev at:http://cr.openjdk.java.net/~lfoltan/bug_jdk8247938.0/webrev/ > bug link: https://bugs.openjdk.java.net/browse/JDK-8247938 > > Testing: hs-tier1 & 2 complete, hs-tier3-6 in progress > > Thanks, > Lois Looks good. For what it's worth, I didn't notice any cases here that seemed like they would significantly benefit from C++20 "using enum": http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1099r5.html (Though it could have been used to signficantly reduce the size of the change.) However, it does seem like some of the enumerators might benefit from a name change, since they are now referred to by names that are qualified by the enum type name. For example, instead of this: 199 enum class DefaultsLookupMode { find_defaults, skip_defaults }; 200 enum class OverpassLookupMode { find_overpass, skip_overpass }; 201 enum class StaticLookupMode { find_static, skip_static }; 202 enum class PrivateLookupMode { find_private, skip_private }; maybe use this: 199 enum class DefaultsLookupMode { find, skip }; 200 enum class OverpassLookupMode { find, skip }; 201 enum class StaticLookupMode { find, skip }; 202 enum class PrivateLookupMode { find, skip }; since (for example) DefaultsLookupMode::find is obviously distinct from StaticLookupMode::find, with the numerator suffixes being redundant. Similarly, in class LinkInfo, perhaps: enum class AccessCheck { required, skip }; But that sort of thing can be done separately, assuming it's deemed desirable. Getting the benfit of the improved type safety is the important part of this change. From kim.barrett at oracle.com Sat Aug 1 09:56:38 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 1 Aug 2020 05:56:38 -0400 Subject: RFR(xxs): 8250911: [windows] os::pd_map_memory() error detection broken In-Reply-To: References: Message-ID: <354E432C-3C8A-47BE-A485-484FD233FB21@oracle.com> > On Aug 1, 2020, at 1:22 AM, Thomas St?fe wrote: > > Hi, > > While trying to find out why > https://bugs.openjdk.java.net/browse/JDK-8249943 consistently fails at SAP > nightlies, I stumbled upon a small zero-day (AFAICS) bug in the Windows > version of os::pd_map_memory(). > > JBS: https://bugs.openjdk.java.net/browse/JDK-8250911 > webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8250911---windows--os--pd_map_memory-error-detection-broken/webrev.00/webrev/ > > While this is annoying I do not expect fixing this bug will shake loose > many follow up bugs. Currently, if the CreateFile call fails but we don't > get it one of the follow up system calls will fail. All this fix provides > is better error analysis. > > Thanks, Thomas Looks good. From thomas.stuefe at gmail.com Sat Aug 1 10:33:43 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 1 Aug 2020 12:33:43 +0200 Subject: RFR(xxs): 8250911: [windows] os::pd_map_memory() error detection broken In-Reply-To: <354E432C-3C8A-47BE-A485-484FD233FB21@oracle.com> References: <354E432C-3C8A-47BE-A485-484FD233FB21@oracle.com> Message-ID: Thanks Kim. On Sat, Aug 1, 2020, 11:58 Kim Barrett wrote: > > On Aug 1, 2020, at 1:22 AM, Thomas St?fe > wrote: > > > > Hi, > > > > While trying to find out why > > https://bugs.openjdk.java.net/browse/JDK-8249943 consistently fails at > SAP > > nightlies, I stumbled upon a small zero-day (AFAICS) bug in the Windows > > version of os::pd_map_memory(). > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8250911 > > webrev: > > > http://cr.openjdk.java.net/~stuefe/webrevs/8250911---windows--os--pd_map_memory-error-detection-broken/webrev.00/webrev/ > > > > While this is annoying I do not expect fixing this bug will shake loose > > many follow up bugs. Currently, if the CreateFile call fails but we don't > > get it one of the follow up system calls will fail. All this fix provides > > is better error analysis. > > > > Thanks, Thomas > > Looks good. > > From shade at redhat.com Sun Aug 2 15:02:32 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Sun, 2 Aug 2020 17:02:32 +0200 Subject: RFR (S) 8250844: Make sure {type, obj}ArrayOopDesc accessors check the bounds In-Reply-To: <4db2ac51-2120-5139-28c5-bd9321c45565@oracle.com> References: <9a8f1e02-c24d-c8e1-1b2c-1b928ddbe23e@redhat.com> <935516a9-231f-ef6e-2de0-3bf3451f8322@redhat.com> <4db2ac51-2120-5139-28c5-bd9321c45565@oracle.com> Message-ID: <2e36cdaf-e1fa-f872-3fb2-eb54e972efb3@redhat.com> On 7/31/20 4:12 PM, coleen.phillimore at oracle.com wrote: > I like this change a lot! Thanks! Pushed. > I'm surprised but relieved that it didn't find any existing bugs. Higher tiers would tell ;) -- Thanks, -Aleksey From felix.yang at huawei.com Mon Aug 3 01:18:30 2020 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Mon, 3 Aug 2020 01:18:30 +0000 Subject: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic In-Reply-To: <5350e745-87af-ce34-8711-2b652a38e323@redhat.com> References: <654e8a2e-87a1-9c3d-4c58-36b4215bb0ad@redhat.com> <5350e745-87af-ce34-8711-2b652a38e323@redhat.com> Message-ID: Hi, > -----Original Message----- > From: Andrew Haley [mailto:aph at redhat.com] > Sent: Friday, July 31, 2020 11:56 PM > To: Yangfei (Felix) ; hotspot-runtime- > dev at openjdk.java.net > Cc: aarch64-port-dev at openjdk.java.net > Subject: Re: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic > > On 7/31/20 10:57 AM, Yangfei (Felix) wrote: > > Since this will not be enabled when the hardware feature is missing, do > you think we need any more changes? > > I do. I do not want the accelerator intrinsic to be auto-enabled until it has > been tested on hardware. OK. I simply comment out the following code changes in vm_version_aarch64.cpp: 393 if (UseSHA && (auxv & HWCAP_SHA512)) { 394 // Do not auto-enable UseSHA512Intrinsics until it has been fully tested on hardware 395 // if (FLAG_IS_DEFAULT(UseSHA512Intrinsics)) { 396 // FLAG_SET_DEFAULT(UseSHA512Intrinsics, true); 397 // } 398 } else if (UseSHA512Intrinsics) { Once this has been fully tested on real hardware, it will be easy to auto-enable. New webrev: http://cr.openjdk.java.net/~fyang/8165404/webrev.01/ Thanks, Felix From david.holmes at oracle.com Mon Aug 3 06:12:29 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 3 Aug 2020 16:12:29 +1000 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> Message-ID: <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> Hi Patricio, I've cc'd JFR dev folk as well. Thanks for working on this Valhalla-preview support work. I'm not quite clear if this part of the work is intended to be an isolated and independent change relative to the broader scope of work outlined in the draft JEP: JDK-8249100 https://bugs.openjdk.java.net/browse/JDK-8249100 so I'm seeking clarification on that from the Valhalla folk. Meanwhile ... On 29/07/2020 5:16 am, Patricio Chilano wrote: > Hi all, > > Please review the following change that adds diagnostic capabilities > when synchronizing on primitive wrapper classes. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8242263 > Webrev: http://cr.openjdk.java.net/~pchilanomate/8242263/v1/webrev/ > > The new flag allows to identify synchronization on these classes and to > take one of the following actions: exit the VM with fatal error, log a > warning message, or issue a JFR event. I'm not clear how conditionally enabled events should be exposed in JFR so will need some input from JFR folk on that part. > The implementation uses a simple > approach where a check is added at the beginning of the monitorenter > generated code when the flag is enabled to check whether the object is > of a primitive wrapper class. If it is, we jump to the slow path, I like this simple approach! > otherwise we just continue as always. The extra instructions will be: > load the klass of the object, load the _access_flags field for that > klass, AND with a constant, and branch based on the result. The code > will only be generated whenever the new opt-in diagnostic flag is > enabled so performance won't be affected when off. In some cases you've factored out code that was only executed with/without biased-locking so that it is always executed and so available for your new code e.g src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp - if (UseBiasedLocking) { - // load object str(obj, Address(disp_hdr, obj_offset)); - null_check_offset = biased_locking_enter(obj, hdr/*scratched*/, tmp1, false, tmp2, done, slow_case); - } but the same code on Aarch64 already does the same thing, so performance wise this shouldn't be a concern. On the other hand the change from null_check_offset = biased_locking_enter(...) to: null_check_offset = offset(); biased_locking_enter(...) does seem to penalize the biased-locking code. Can you not just do: if (DiagnoseSyncOnPrimitiveWrappers) { + null_check_offset = offset(); load_klass(hdr, obj); ldr(hdr, Address(hdr, Klass::access_flags_offset())); tst(hdr, JVM_ACC_IS_BOX_CLASS); br(Assembler::NE, slow_case); } so that only the new code is impacted? --- General comment - you use this a lot: if (DiagnoseSyncOnPrimitiveWrappers) but DiagnoseSyncOnPrimitiveWrappers is not a boolean and hotspot-style says to avoid implicit booleans, so those checks need to be expanded to: if (DiagnoseSyncOnPrimitiveWrappers != 0) --- src/hotspot/share/runtime/synchronizer.cpp To avoid the need to re-manifest the current thread you can do: void ObjectSynchronizer::handle_sync_on_primitive_wrapper(Handle obj, Thread* current) { assert(current->is_Java_thread(), "must be for java object synchronization"); JavaThread* self = (JavaThread) current; 805 assert(!do_lock || DiagnoseSyncOnPrimitiveWrappers != ObjectSynchronizer::FATAL_EXIT || !obj->klass()->is_box(), "ObjectLocker synchronizing on primitive wrapper"); I don't quite follow that assertion - why is the middle part not just "DiagnoseSyncOnPrimitiveWrappers != 0" ? Or why not just assert it is not a box so that we locate all places where we do sync on boxes? --- test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java Please add "@bug 8242263". 32 * @run main/othervm SyncOnPrimitiveWrapperTest As the test only launches other VMs this can just be: 32 * @run driver SyncOnPrimitiveWrapperTest It would be nice to factor out the common options rather than repeating so many of the string literals. I don't think you want to use ProcessTools.createTestJvm as that will apply flags from the test run that will potentially conflict with the explicit flag combinations you are testing. Just use ProcessTools.createJavaProcessBuilder. Both the FatalTest and the LogTest should be testing wrappers of each type. 99 Integer integer = new Integer(10); Use Integer.valueOf(10) (or allow implicit boxing) as direct construction of wrapper types is something that will itself be subject to warnings in the future. 101 throw new RuntimeException("synchronization on primitve wrapper did not failed"); typos: primitve -> primitive failed -> fail I don't understand why the Log test is so complex. I just expected to see a simple "synchronized(o){}" for an 'o' of each primitive type, and a check that the log contained the expected message. --- test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java The licence header should not be the version with the classpath exception. Please add "@bug 8242263". The test should test each of the wrapper types, not just Integer > In addition to the purpose described in the description of the bug, this > flag will also be useful when trying to diagnose possible > synchronization issues if these classes ever become inline types as part > of the Valhalla project. > > I added test SyncOnPrimitiveWrapperTest.java that tests for the exit and > logging cases. I added test TestSyncOnPrimitiveWrapperEvent.java to test > for the JFR event case. > I tested the patch running tiers1-6 in mach5 with the flag set to > DiagnoseSyncOnPrimitiveWrappers=2. Okay. > I checked it builds with arm32 and ppc but can't run any tests on those > platforms, so it would be good if somebody can run the new test included > in the patch. > > Let me know if you think I should run or add any more tests. The primary testing here is the new tests. Thanks, David ----- > Thanks! > > Patricio From david.holmes at oracle.com Mon Aug 3 07:53:38 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 3 Aug 2020 17:53:38 +1000 Subject: RFR: 8250636: iso8601_time returns incorrect offset part on MacOS In-Reply-To: <0e1ed1ae-2011-2e4a-4f58-d99b57ede74c@azul.com> References: <777fc30a-45ef-aa45-8f04-0bbe80c5a83d@azul.com> <0e1ed1ae-2011-2e4a-4f58-d99b57ede74c@azul.com> Message-ID: <934a8a40-dd51-512f-f010-909ed312783d@oracle.com> Hi Dmitry, On 31/07/2020 8:29 pm, Dmitry Cherepanov wrote: > Hi Gerard, > > Thanks for reviewing this, moving the logic of get_timezone to > iso8601_time looks good to me too. > Here's an updated webrev > http://cr.openjdk.java.net/~dcherepanov/8250636/webrev.v4/ This version also looks good to me. But now I see all the code clearly in one place I have to wonder why we are not using tm_gmtoff on all platforms except Windows? That would simplify the logic even further. That goes beyond fixing the current actual bug though so I'll leave that choice to you. Thanks, David > Thanks, > > Dmitry > > On 30.07.2020 19:52, gerard ziemski wrote: >> hi Dmitry, >> >> Looks good when it comes to the core approach of the fix, but may I >> suggest simplifying the code a bit? Perhaps something like this: >> >> http://cr.openjdk.java.net/~gziemski/8250636_rev1/index.html >> >> I do not like the duplication of "const time_t? constants and I don?t >> think that including more logic code in ?get_timezone()? does much for >> readability of the code here. >> >> Thank you for catching and fixing it! >> >> >> cheers >> >> >> On 7/30/20 4:32 AM, Dmitry Cherepanov wrote: >>> Hello, >>> >>> Please review a small change for fixing offset (timezone) part of the >>> string returned by os::iso8601_time on MacOS. The patch negates >>> tm_gmtoff and skips the DST adjustment when tm_gmtoff is used. The >>> behavior on other platforms should remain unchanged. More details are in >>> the bug report. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8250636 >>> Webrev: http://cr.openjdk.java.net/~dcherepanov/8250636/webrev.v3/ >>> >>> Thanks, >>> >>> Dmitry >>> > From lois.foltan at oracle.com Mon Aug 3 15:12:12 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 3 Aug 2020 11:12:12 -0400 Subject: RFR(S): JDK-8247938: Change various JVM enums like LinkInfo::AccessCheck and Klass::DefaultsLookupMode to enum class In-Reply-To: <3A1268FE-1231-4043-8CA8-06C9F3160421@oracle.com> References: <3A1268FE-1231-4043-8CA8-06C9F3160421@oracle.com> Message-ID: On 8/1/2020 5:47 AM, Kim Barrett wrote: >> On Jul 31, 2020, at 1:55 PM, Lois Foltan wrote: >> >> Please review the webrev to change the following unscoped enumeration declarations to scoped enumerations. >> >> Klass::DefaultsLookupMode >> Klass::OverpassLookupMode >> Klass::StaticLookupMode >> Klass::PrivateLookupMode >> LinkInfo::AccessCheck >> >> With C++11/14 enum class provides the added type safety to prevent 0 (integral) or false (boolean) to be implicitly converted to AccessCheck::needs_access_check for example. >> >> open webrev at:http://cr.openjdk.java.net/~lfoltan/bug_jdk8247938.0/webrev/ >> bug link: https://bugs.openjdk.java.net/browse/JDK-8247938 >> >> Testing: hs-tier1 & 2 complete, hs-tier3-6 in progress >> >> Thanks, >> Lois > Looks good. > > For what it's worth, I didn't notice any cases here that seemed like they > would significantly benefit from C++20 "using enum": > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1099r5.html > (Though it could have been used to signficantly reduce the size of the change.) > > However, it does seem like some of the enumerators might benefit from a name > change, since they are now referred to by names that are qualified by the > enum type name. For example, instead of this: > > 199 enum class DefaultsLookupMode { find_defaults, skip_defaults }; > 200 enum class OverpassLookupMode { find_overpass, skip_overpass }; > 201 enum class StaticLookupMode { find_static, skip_static }; > 202 enum class PrivateLookupMode { find_private, skip_private }; > > maybe use this: > > 199 enum class DefaultsLookupMode { find, skip }; > 200 enum class OverpassLookupMode { find, skip }; > 201 enum class StaticLookupMode { find, skip }; > 202 enum class PrivateLookupMode { find, skip }; > > since (for example) DefaultsLookupMode::find is obviously distinct from > StaticLookupMode::find, with the numerator suffixes being redundant. > > Similarly, in class LinkInfo, perhaps: > enum class AccessCheck { required, skip }; > > But that sort of thing can be done separately, assuming it's deemed > desirable. Getting the benfit of the improved type safety is the important > part of this change. > > Thank you Kim for the review!? I do like your suggestion of changing the names of the enumerators.? New webrev to include that renaming as part of this change. New webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8247938.1/webrev Retesting hs-tier1-3. Thanks, Lois From alexander.scherbatiy at bell-sw.com Mon Aug 3 15:33:24 2020 From: alexander.scherbatiy at bell-sw.com (Alexander Scherbatiy) Date: Mon, 3 Aug 2020 18:33:24 +0300 Subject: RFR 8241053: Hotspot runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java test fails on Alpine Linux with debug build Message-ID: <87a32c5b-8e96-f323-1e20-8df181ab3ec0@bell-sw.com> Hello, Could you review the fix for the issue: ? Bug: https://bugs.openjdk.java.net/browse/JDK-8241053 ? Webrev: http://cr.openjdk.java.net/~alexsch/8241053/weberv.00 The following tests fail on Linux Alpine with musl libc: ? test/hotspot/jtreg/runtime/Thread/TestThreadStackSizes.java test/hotspot/jtreg/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java This is because muls pthread_attr_setstacksize(3) function implementation [1] returns EINVAL when stack size exceeds the certain limit. According to POSIX pthread_attr_setstacksize() function description [2]: ?The pthread_attr_setstacksize() function will fail if: ??? [EINVAL]? The value of stacksize is less than PTHREAD_STACK_MIN or exceeds a system-imposed limit. The proposed fix excludes EINVAL value from assert_status() and for the EINVAL return value makes the same cleanup actions and returns false from os::create_thread() function as it is done for AIX[3]. To reproduce the issue it needs to build Portola from repository [4] using instructions from JEP 386: Alpine Linux/x64 Port [5]. After the fix both TestThreadStackSizes.java and TestOptionsWithRanges.java tests pass on Alpine Linux with musl libc and on Ubuntu 18.0.4. [1] https://git.musl-libc.org/cgit/musl/tree/src/thread/pthread_attr_setstacksize.c [2] https://pubs.opengroup.org/onlinepubs/007908799/xsh/pthread_attr_setstacksize.html [3] https://hg.openjdk.java.net/jdk/jdk/file/c8102e6fc512/src/hotspot/os/aix/os_aix.cpp#l890 [4] https://github.com/openjdk/portola [5] https://openjdk.java.net/jeps/386 Thanks, Alexander. From harold.seigel at oracle.com Mon Aug 3 15:39:34 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Mon, 3 Aug 2020 11:39:34 -0400 Subject: RFR(S): JDK-8247938: Change various JVM enums like LinkInfo::AccessCheck and Klass::DefaultsLookupMode to enum class In-Reply-To: References: <3A1268FE-1231-4043-8CA8-06C9F3160421@oracle.com> Message-ID: Hi Lois, The new changes look good! Thanks, Harold On 8/3/2020 11:12 AM, Lois Foltan wrote: > > > > > On 8/1/2020 5:47 AM, Kim Barrett wrote: >>> On Jul 31, 2020, at 1:55 PM, Lois Foltan >>> wrote: >>> >>> Please review the webrev to change the following unscoped >>> enumeration declarations to scoped enumerations. >>> >>> Klass::DefaultsLookupMode >>> Klass::OverpassLookupMode >>> Klass::StaticLookupMode >>> Klass::PrivateLookupMode >>> LinkInfo::AccessCheck >>> >>> With C++11/14 enum class provides the added type safety to prevent 0 >>> (integral) or false (boolean) to be implicitly converted to >>> AccessCheck::needs_access_check for example. >>> >>> open webrev >>> at:http://cr.openjdk.java.net/~lfoltan/bug_jdk8247938.0/webrev/ >>> >>> >>> bug link: https://bugs.openjdk.java.net/browse/JDK-8247938 >>> >>> Testing: hs-tier1 & 2 complete, hs-tier3-6 in progress >>> >>> Thanks, >>> Lois >> Looks good. >> >> For what it's worth, I didn't notice any cases here that seemed like >> they >> would significantly benefit from C++20 "using enum": >> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1099r5.html >> (Though it could have been used to signficantly reduce the size of >> the change.) >> >> However, it does seem like some of the enumerators might benefit from >> a name >> change, since they are now referred to by names that are qualified by >> the >> enum type name.? For example, instead of this: >> >> ? 199?? enum class DefaultsLookupMode { find_defaults, skip_defaults }; >> ? 200?? enum class OverpassLookupMode { find_overpass, skip_overpass }; >> ? 201?? enum class StaticLookupMode?? { find_static, skip_static }; >> ? 202?? enum class PrivateLookupMode? { find_private, skip_private }; >> >> maybe use this: >> >> ? 199?? enum class DefaultsLookupMode { find, skip }; >> ? 200?? enum class OverpassLookupMode { find, skip }; >> ? 201?? enum class StaticLookupMode?? { find, skip }; >> ? 202?? enum class PrivateLookupMode? { find, skip }; >> >> since (for example) DefaultsLookupMode::find is obviously distinct from >> StaticLookupMode::find, with the numerator suffixes being redundant. >> >> Similarly, in class LinkInfo, perhaps: >> ?? enum class AccessCheck { required, skip }; >> >> But that sort of thing can be done separately, assuming it's deemed >> desirable.? Getting the benfit of the improved type safety is the >> important >> part of this change. >> >> > > Thank you Kim for the review!? I do like your suggestion of changing > the names of the enumerators.? New webrev to include that renaming as > part of this change. > > New webrev at: > http://cr.openjdk.java.net/~lfoltan/bug_jdk8247938.1/webrev > > Retesting hs-tier1-3. > > Thanks, > Lois From lois.foltan at oracle.com Mon Aug 3 15:40:43 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 3 Aug 2020 11:40:43 -0400 Subject: RFR(S): JDK-8247938: Change various JVM enums like LinkInfo::AccessCheck and Klass::DefaultsLookupMode to enum class In-Reply-To: References: <3A1268FE-1231-4043-8CA8-06C9F3160421@oracle.com> Message-ID: <3cec28a1-a273-a47d-83c1-4c080c02bd84@oracle.com> Thanks Harold! Lois On 8/3/2020 11:39 AM, Harold Seigel wrote: > Hi Lois, > > The new changes look good! > > Thanks, Harold > > On 8/3/2020 11:12 AM, Lois Foltan wrote: >> >> >> >> >> On 8/1/2020 5:47 AM, Kim Barrett wrote: >>>> On Jul 31, 2020, at 1:55 PM, Lois Foltan >>>> wrote: >>>> >>>> Please review the webrev to change the following unscoped >>>> enumeration declarations to scoped enumerations. >>>> >>>> Klass::DefaultsLookupMode >>>> Klass::OverpassLookupMode >>>> Klass::StaticLookupMode >>>> Klass::PrivateLookupMode >>>> LinkInfo::AccessCheck >>>> >>>> With C++11/14 enum class provides the added type safety to prevent >>>> 0 (integral) or false (boolean) to be implicitly converted to >>>> AccessCheck::needs_access_check for example. >>>> >>>> open webrev >>>> at:http://cr.openjdk.java.net/~lfoltan/bug_jdk8247938.0/webrev/ >>>> >>>> >>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8247938 >>>> >>>> Testing: hs-tier1 & 2 complete, hs-tier3-6 in progress >>>> >>>> Thanks, >>>> Lois >>> Looks good. >>> >>> For what it's worth, I didn't notice any cases here that seemed like >>> they >>> would significantly benefit from C++20 "using enum": >>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1099r5.html >>> (Though it could have been used to signficantly reduce the size of >>> the change.) >>> >>> However, it does seem like some of the enumerators might benefit >>> from a name >>> change, since they are now referred to by names that are qualified >>> by the >>> enum type name.? For example, instead of this: >>> >>> ? 199?? enum class DefaultsLookupMode { find_defaults, skip_defaults }; >>> ? 200?? enum class OverpassLookupMode { find_overpass, skip_overpass }; >>> ? 201?? enum class StaticLookupMode?? { find_static, skip_static }; >>> ? 202?? enum class PrivateLookupMode? { find_private, skip_private }; >>> >>> maybe use this: >>> >>> ? 199?? enum class DefaultsLookupMode { find, skip }; >>> ? 200?? enum class OverpassLookupMode { find, skip }; >>> ? 201?? enum class StaticLookupMode?? { find, skip }; >>> ? 202?? enum class PrivateLookupMode? { find, skip }; >>> >>> since (for example) DefaultsLookupMode::find is obviously distinct from >>> StaticLookupMode::find, with the numerator suffixes being redundant. >>> >>> Similarly, in class LinkInfo, perhaps: >>> ?? enum class AccessCheck { required, skip }; >>> >>> But that sort of thing can be done separately, assuming it's deemed >>> desirable.? Getting the benfit of the improved type safety is the >>> important >>> part of this change. >>> >>> >> >> Thank you Kim for the review!? I do like your suggestion of changing >> the names of the enumerators.? New webrev to include that renaming as >> part of this change. >> >> New webrev at: >> http://cr.openjdk.java.net/~lfoltan/bug_jdk8247938.1/webrev >> >> Retesting hs-tier1-3. >> >> Thanks, >> Lois From coleen.phillimore at oracle.com Mon Aug 3 15:44:22 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Mon, 3 Aug 2020 11:44:22 -0400 Subject: RFR(S): JDK-8247938: Change various JVM enums like LinkInfo::AccessCheck and Klass::DefaultsLookupMode to enum class In-Reply-To: References: <3A1268FE-1231-4043-8CA8-06C9F3160421@oracle.com> Message-ID: +1.? It was a good suggestion. Coleen On 8/3/20 11:39 AM, Harold Seigel wrote: > Hi Lois, > > The new changes look good! > > Thanks, Harold > > On 8/3/2020 11:12 AM, Lois Foltan wrote: >> >> >> >> >> On 8/1/2020 5:47 AM, Kim Barrett wrote: >>>> On Jul 31, 2020, at 1:55 PM, Lois Foltan >>>> wrote: >>>> >>>> Please review the webrev to change the following unscoped >>>> enumeration declarations to scoped enumerations. >>>> >>>> Klass::DefaultsLookupMode >>>> Klass::OverpassLookupMode >>>> Klass::StaticLookupMode >>>> Klass::PrivateLookupMode >>>> LinkInfo::AccessCheck >>>> >>>> With C++11/14 enum class provides the added type safety to prevent >>>> 0 (integral) or false (boolean) to be implicitly converted to >>>> AccessCheck::needs_access_check for example. >>>> >>>> open webrev >>>> at:http://cr.openjdk.java.net/~lfoltan/bug_jdk8247938.0/webrev/ >>>> >>>> >>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8247938 >>>> >>>> Testing: hs-tier1 & 2 complete, hs-tier3-6 in progress >>>> >>>> Thanks, >>>> Lois >>> Looks good. >>> >>> For what it's worth, I didn't notice any cases here that seemed like >>> they >>> would significantly benefit from C++20 "using enum": >>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1099r5.html >>> (Though it could have been used to signficantly reduce the size of >>> the change.) >>> >>> However, it does seem like some of the enumerators might benefit >>> from a name >>> change, since they are now referred to by names that are qualified >>> by the >>> enum type name.? For example, instead of this: >>> >>> ? 199?? enum class DefaultsLookupMode { find_defaults, skip_defaults }; >>> ? 200?? enum class OverpassLookupMode { find_overpass, skip_overpass }; >>> ? 201?? enum class StaticLookupMode?? { find_static, skip_static }; >>> ? 202?? enum class PrivateLookupMode? { find_private, skip_private }; >>> >>> maybe use this: >>> >>> ? 199?? enum class DefaultsLookupMode { find, skip }; >>> ? 200?? enum class OverpassLookupMode { find, skip }; >>> ? 201?? enum class StaticLookupMode?? { find, skip }; >>> ? 202?? enum class PrivateLookupMode? { find, skip }; >>> >>> since (for example) DefaultsLookupMode::find is obviously distinct from >>> StaticLookupMode::find, with the numerator suffixes being redundant. >>> >>> Similarly, in class LinkInfo, perhaps: >>> ?? enum class AccessCheck { required, skip }; >>> >>> But that sort of thing can be done separately, assuming it's deemed >>> desirable.? Getting the benfit of the improved type safety is the >>> important >>> part of this change. >>> >>> >> >> Thank you Kim for the review!? I do like your suggestion of changing >> the names of the enumerators.? New webrev to include that renaming as >> part of this change. >> >> New webrev at: >> http://cr.openjdk.java.net/~lfoltan/bug_jdk8247938.1/webrev >> >> Retesting hs-tier1-3. >> >> Thanks, >> Lois From lois.foltan at oracle.com Mon Aug 3 15:45:24 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 3 Aug 2020 11:45:24 -0400 Subject: RFR(S): JDK-8247938: Change various JVM enums like LinkInfo::AccessCheck and Klass::DefaultsLookupMode to enum class In-Reply-To: References: <3A1268FE-1231-4043-8CA8-06C9F3160421@oracle.com> Message-ID: Thanks Coleen! Lois On 8/3/2020 11:44 AM, coleen.phillimore at oracle.com wrote: > +1.? It was a good suggestion. > Coleen > > On 8/3/20 11:39 AM, Harold Seigel wrote: >> Hi Lois, >> >> The new changes look good! >> >> Thanks, Harold >> >> On 8/3/2020 11:12 AM, Lois Foltan wrote: >>> >>> >>> >>> >>> On 8/1/2020 5:47 AM, Kim Barrett wrote: >>>>> On Jul 31, 2020, at 1:55 PM, Lois Foltan >>>>> wrote: >>>>> >>>>> Please review the webrev to change the following unscoped >>>>> enumeration declarations to scoped enumerations. >>>>> >>>>> Klass::DefaultsLookupMode >>>>> Klass::OverpassLookupMode >>>>> Klass::StaticLookupMode >>>>> Klass::PrivateLookupMode >>>>> LinkInfo::AccessCheck >>>>> >>>>> With C++11/14 enum class provides the added type safety to prevent >>>>> 0 (integral) or false (boolean) to be implicitly converted to >>>>> AccessCheck::needs_access_check for example. >>>>> >>>>> open webrev >>>>> at:http://cr.openjdk.java.net/~lfoltan/bug_jdk8247938.0/webrev/ >>>>> >>>>> >>>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8247938 >>>>> >>>>> Testing: hs-tier1 & 2 complete, hs-tier3-6 in progress >>>>> >>>>> Thanks, >>>>> Lois >>>> Looks good. >>>> >>>> For what it's worth, I didn't notice any cases here that seemed >>>> like they >>>> would significantly benefit from C++20 "using enum": >>>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1099r5.html >>>> (Though it could have been used to signficantly reduce the size of >>>> the change.) >>>> >>>> However, it does seem like some of the enumerators might benefit >>>> from a name >>>> change, since they are now referred to by names that are qualified >>>> by the >>>> enum type name.? For example, instead of this: >>>> >>>> ? 199?? enum class DefaultsLookupMode { find_defaults, >>>> skip_defaults }; >>>> ? 200?? enum class OverpassLookupMode { find_overpass, >>>> skip_overpass }; >>>> ? 201?? enum class StaticLookupMode?? { find_static, skip_static }; >>>> ? 202?? enum class PrivateLookupMode? { find_private, skip_private }; >>>> >>>> maybe use this: >>>> >>>> ? 199?? enum class DefaultsLookupMode { find, skip }; >>>> ? 200?? enum class OverpassLookupMode { find, skip }; >>>> ? 201?? enum class StaticLookupMode?? { find, skip }; >>>> ? 202?? enum class PrivateLookupMode? { find, skip }; >>>> >>>> since (for example) DefaultsLookupMode::find is obviously distinct >>>> from >>>> StaticLookupMode::find, with the numerator suffixes being redundant. >>>> >>>> Similarly, in class LinkInfo, perhaps: >>>> ?? enum class AccessCheck { required, skip }; >>>> >>>> But that sort of thing can be done separately, assuming it's deemed >>>> desirable.? Getting the benfit of the improved type safety is the >>>> important >>>> part of this change. >>>> >>>> >>> >>> Thank you Kim for the review!? I do like your suggestion of changing >>> the names of the enumerators.? New webrev to include that renaming >>> as part of this change. >>> >>> New webrev at: >>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8247938.1/webrev >>> >>> Retesting hs-tier1-3. >>> >>> Thanks, >>> Lois > From harold.seigel at oracle.com Mon Aug 3 18:11:32 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Mon, 3 Aug 2020 14:11:32 -0400 Subject: RFR 8139875: [TESTBUG] Improve nsk/stress/stack/* tests Message-ID: <62a030ca-99e1-1b5c-ade7-e8685aa9f8e3@oracle.com> Hi, Please review these small changes to reduce the amount of time required by some of the nsk/stress/stack tests in order to prevent them from timing out.? The changes include using -Xss to run the tests with smaller stacks, prevent running with -Xcomp, and, for test stack019, reduce the number of test iterations. Test stack019 was excluded from windows targets because, even with the reduced iterations, the test was running for > 48 minutes. Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8139875/webrev/index.html JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8139875 The tests were tested by running them in Mach5 tier5 and tier7 and looking at their execution time. Thanks, Harold From mikhailo.seledtsov at oracle.com Mon Aug 3 21:15:17 2020 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Mon, 3 Aug 2020 14:15:17 -0700 Subject: RFR(T): 8250986: Problem list docker/TestMemoryAwareness.java and docker/TestDockerMemoryMetrics.java for linux-5.4.0-1019-oracle Message-ID: <7c29a474-2bed-4eb7-720e-e7774855c82f@oracle.com> Please review this trivial fix that problem lists these 2 tests specifically for linux-5.4.0-1019-oracle, until JDK-8250984 is fixed. Testing: ? - ran on the linux-5.4.0-1019-oracle host, the problem listed tests do not get executed ? - ran on OL 7.6, the problem listed tests execute and pass ===== Change: diff --git a/test/hotspot/jtreg/ProblemList.txt b/test/hotspot/jtreg/ProblemList.txt --- a/test/hotspot/jtreg/ProblemList.txt +++ b/test/hotspot/jtreg/ProblemList.txt @@ -85,6 +85,7 @@ ?runtime/jni/terminatedThread/TestTerminatedThread.java 8219652 aix-ppc64 ?runtime/ReservedStack/ReservedStackTest.java 8231031 generic-all +containers/docker/TestMemoryAwareness.java 8250986 linux-5.4.0-1019-oracle ?############################################################################# diff --git a/test/jdk/ProblemList.txt b/test/jdk/ProblemList.txt --- a/test/jdk/ProblemList.txt +++ b/test/jdk/ProblemList.txt @@ -916,6 +916,8 @@ ?# jdk_internal +jdk/internal/platform/docker/TestDockerMemoryMetrics.java 8250986 linux-5.4.0-1019-oracle + ?############################################################################ ?# jdk_jpackage Thank you, Misha From ioi.lam at oracle.com Mon Aug 3 22:11:18 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 3 Aug 2020 15:11:18 -0700 Subject: RFR(M) 8250990 Consolidate object copying code for CDS static/dynamic archive dumping In-Reply-To: <07b2ee18-0a5f-0839-0a6c-b54d788e4cd2@oracle.com> References: <07b2ee18-0a5f-0839-0a6c-b54d788e4cd2@oracle.com> Message-ID: <39423dce-d769-bb92-4d22-9dc848d1086e@oracle.com> There turns out to be a lot of duplication in the static/dynamic dumping code. To keep the changes manageable, I've reorganized the REF into a few sub-tasks. This particular review is for: https://bugs.openjdk.java.net/browse/JDK-8250990 http://cr.openjdk.java.net/~iklam/jdk16/8250990-consolidate-static-dynamic-cds-objcopy.v01/ https://wiki.openjdk.java.net/display/HotSpot/How+CDS+Copies+Class+Metadata+into+the+Archive I've also updated the e-mail subject to use the new JBS ID (JDK-8250990). This patch is an intermediate step. Some code might look like they are in the wrong place -- for example: _estimated_metsapceobj_bytes has been moved to archiveBuilder.hpp, but _estimated_hashtable_bytes is still in dynamicArchive.cpp. The latter will eventually be moved as part of the second step (JDK-8250989 Consolidate buffer allocation code for CDS static/dynamic archive dumping). (Coleen has given me feedback off-line. I'll post a new version soon to limit the API in archiveBuilder.hpp) Thanks - Ioi On 7/31/20 11:36 PM, Ioi Lam wrote: > > Currently there are two very similar implementations for static CDS > archive > dumping (metaspaceShared.cpp) and dynamic CDS archive dumping > (dynamicArchive.cpp). > > I've merged the two versions and moved to a new file, archiveBuilder.cpp. > > I also improved the copying algorithm to speed up dumping. Times for > dumping > 20000+ classes: > > ? ?? ????? static??????? dynamic > ??? Old??? 42.655 sec??? 67.014 sec > ??? New??? 37.027 sec??? 34.974 sec > > Normally I would not try to optimize when refactoring code. I made an > earlier, naive attempt to use the old dynamic dumper for static dumping > (because dynamic dumping is essentially a superset of static dumping). > However, that caused significant regression in static dumping speed. > > Hopefully the new algorithm is both faster and easier to understand. > Please see the wiki document for more information. > > Tested with mach5 tiers 1-4. > > Thanks > - Ioi From kim.barrett at oracle.com Mon Aug 3 22:42:31 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 3 Aug 2020 18:42:31 -0400 Subject: RFR(S): JDK-8247938: Change various JVM enums like LinkInfo::AccessCheck and Klass::DefaultsLookupMode to enum class In-Reply-To: References: <3A1268FE-1231-4043-8CA8-06C9F3160421@oracle.com> Message-ID: > On Aug 3, 2020, at 11:12 AM, Lois Foltan wrote: > > Thank you Kim for the review! I do like your suggestion of changing the names of the enumerators. New webrev to include that renaming as part of this change. > > New webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8247938.1/webrev > > Retesting hs-tier1-3. > > Thanks, > Lois Looks good. BTW, for possible future work, LinkInfo might be a candidate for using delegating constructors. From david.holmes at oracle.com Mon Aug 3 22:51:11 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Aug 2020 08:51:11 +1000 Subject: RFR 8241053: Hotspot runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java test fails on Alpine Linux with debug build In-Reply-To: <87a32c5b-8e96-f323-1e20-8df181ab3ec0@bell-sw.com> References: <87a32c5b-8e96-f323-1e20-8df181ab3ec0@bell-sw.com> Message-ID: <5b530109-bda2-820e-6070-d779f08b64f6@oracle.com> Hi Alexander, Thanks for fixing this. On 4/08/2020 1:33 am, Alexander Scherbatiy wrote: > Hello, > > Could you review the fix for the issue: > ? Bug: https://bugs.openjdk.java.net/browse/JDK-8241053 > ? Webrev: http://cr.openjdk.java.net/~alexsch/8241053/weberv.00 I would simplify this a little: assert_status(status == 0 || status == EINVAL, status, "pthread_attr_setstacksize"); // pthread_attr_setstacksize() function can fail // if the stack size exceeds a system-imposed limit. if (status == EINVAL) { to if (status != 0) { // pthread_attr_setstacksize() function can fail // if the stack size exceeds a system-imposed limit. assert_status(status == EINVAL, status, "pthread_attr_setstacksize"); Thanks, David ----- > The following tests fail on Linux Alpine with musl libc: > ? test/hotspot/jtreg/runtime/Thread/TestThreadStackSizes.java > test/hotspot/jtreg/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java > > > This is because muls pthread_attr_setstacksize(3) function > implementation [1] returns EINVAL when stack size exceeds the certain > limit. > > According to POSIX pthread_attr_setstacksize() function description [2]: > ?The pthread_attr_setstacksize() function will fail if: > ??? [EINVAL]? The value of stacksize is less than PTHREAD_STACK_MIN or > exceeds a system-imposed limit. > > The proposed fix excludes EINVAL value from assert_status() and for the > EINVAL return value makes the same cleanup actions and returns false > from os::create_thread() function as it is done for AIX[3]. > > To reproduce the issue it needs to build Portola from repository [4] > using instructions from JEP 386: Alpine Linux/x64 Port [5]. > After the fix both TestThreadStackSizes.java and > TestOptionsWithRanges.java tests pass on Alpine Linux with musl libc and > on Ubuntu 18.0.4. > > [1] > https://git.musl-libc.org/cgit/musl/tree/src/thread/pthread_attr_setstacksize.c > > [2] > https://pubs.opengroup.org/onlinepubs/007908799/xsh/pthread_attr_setstacksize.html > > [3] > https://hg.openjdk.java.net/jdk/jdk/file/c8102e6fc512/src/hotspot/os/aix/os_aix.cpp#l890 > > [4] https://github.com/openjdk/portola > [5] https://openjdk.java.net/jeps/386 > > Thanks, > Alexander. > From david.holmes at oracle.com Mon Aug 3 23:04:52 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Aug 2020 09:04:52 +1000 Subject: RFR 8139875: [TESTBUG] Improve nsk/stress/stack/* tests In-Reply-To: <62a030ca-99e1-1b5c-ade7-e8685aa9f8e3@oracle.com> References: <62a030ca-99e1-1b5c-ade7-e8685aa9f8e3@oracle.com> Message-ID: <262e8bd2-5759-0b88-4977-233be3161739@oracle.com> Hi Harold, On 4/08/2020 4:11 am, Harold Seigel wrote: > Hi, > > Please review these small changes to reduce the amount of time required > by some of the nsk/stress/stack tests in order to prevent them from > timing out.? The changes include using -Xss to run the tests with > smaller stacks, prevent running with -Xcomp, and, for test stack019, > reduce the number of test iterations. Is there a reason to use a range of -Xss values in different tests instead of just, say, using -Xss200K in all of them and reducing the execution time further. Thanks, David ----- > Test stack019 was excluded from windows targets because, even with the > reduced iterations, the test was running for > 48 minutes. > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8139875/webrev/index.html > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8139875 > > The tests were tested by running them in Mach5 tier5 and tier7 and > looking at their execution time. > > Thanks, Harold > From david.holmes at oracle.com Mon Aug 3 23:33:34 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Aug 2020 09:33:34 +1000 Subject: RFR(T): 8250986: Problem list docker/TestMemoryAwareness.java and docker/TestDockerMemoryMetrics.java for linux-5.4.0-1019-oracle In-Reply-To: <7c29a474-2bed-4eb7-720e-e7774855c82f@oracle.com> References: <7c29a474-2bed-4eb7-720e-e7774855c82f@oracle.com> Message-ID: Hi Misha, On 4/08/2020 7:15 am, mikhailo.seledtsov at oracle.com wrote: > Please review this trivial fix that problem lists these 2 tests > specifically for linux-5.4.0-1019-oracle, until JDK-8250984 is fixed. In that case 8250984 is what should be in the entry not 8250986: containers/docker/TestMemoryAwareness.java 8250986 linux-5.4.0-1019-oracle I was unaware that these statements could specify a detailed OS version like that. But where does that version string come from? Thanks, David > Testing: > > ? - ran on the linux-5.4.0-1019-oracle host, the problem listed tests > do not get executed > > ? - ran on OL 7.6, the problem listed tests execute and pass > > > ===== Change: > > diff --git a/test/hotspot/jtreg/ProblemList.txt > b/test/hotspot/jtreg/ProblemList.txt > --- a/test/hotspot/jtreg/ProblemList.txt > +++ b/test/hotspot/jtreg/ProblemList.txt > @@ -85,6 +85,7 @@ > > ?runtime/jni/terminatedThread/TestTerminatedThread.java 8219652 aix-ppc64 > ?runtime/ReservedStack/ReservedStackTest.java 8231031 generic-all > +containers/docker/TestMemoryAwareness.java 8250986 linux-5.4.0-1019-oracle > > ?############################################################################# > > diff --git a/test/jdk/ProblemList.txt b/test/jdk/ProblemList.txt > --- a/test/jdk/ProblemList.txt > +++ b/test/jdk/ProblemList.txt > @@ -916,6 +916,8 @@ > > ?# jdk_internal > > +jdk/internal/platform/docker/TestDockerMemoryMetrics.java 8250986 > linux-5.4.0-1019-oracle > + > ?############################################################################ > > ?# jdk_jpackage > > > Thank you, > > Misha > From mikhailo.seledtsov at oracle.com Tue Aug 4 00:07:00 2020 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Mon, 3 Aug 2020 17:07:00 -0700 Subject: RFR(T): 8250986: Problem list docker/TestMemoryAwareness.java and docker/TestDockerMemoryMetrics.java for linux-5.4.0-1019-oracle In-Reply-To: References: <7c29a474-2bed-4eb7-720e-e7774855c82f@oracle.com> Message-ID: <324834bb-f63e-284e-7d45-97d639e25440@oracle.com> Hi David, On 8/3/20 4:33 PM, David Holmes wrote: > Hi Misha, > > On 4/08/2020 7:15 am, mikhailo.seledtsov at oracle.com wrote: >> Please review this trivial fix that problem lists these 2 tests >> specifically for linux-5.4.0-1019-oracle, until JDK-8250984 is fixed. > > In that case 8250984 is what should be in the entry not 8250986: > > containers/docker/TestMemoryAwareness.java 8250986 > linux-5.4.0-1019-oracle Thanks for the catch. I have replaced 8250986 to be 8250984. > > I was unaware that these statements could specify a detailed OS > version like that. But where does that version string come from? JTreg allows to specify a 'flavor' of OS. JTreg reports this value in the test log as "javatestOS". In this case: "javatestOS=Linux 5.4.0-1019-oracle", which seems to be a kernel version on Linux hosts. Thanks, Misha > > Thanks, > David > > >> Testing: >> >> ?? - ran on the linux-5.4.0-1019-oracle host, the problem listed >> tests do not get executed >> >> ?? - ran on OL 7.6, the problem listed tests execute and pass >> >> >> ===== Change: >> >> diff --git a/test/hotspot/jtreg/ProblemList.txt >> b/test/hotspot/jtreg/ProblemList.txt >> --- a/test/hotspot/jtreg/ProblemList.txt >> +++ b/test/hotspot/jtreg/ProblemList.txt >> @@ -85,6 +85,7 @@ >> >> ??runtime/jni/terminatedThread/TestTerminatedThread.java 8219652 >> aix-ppc64 >> ??runtime/ReservedStack/ReservedStackTest.java 8231031 generic-all >> +containers/docker/TestMemoryAwareness.java 8250986 >> linux-5.4.0-1019-oracle >> >> ??############################################################################# >> >> >> diff --git a/test/jdk/ProblemList.txt b/test/jdk/ProblemList.txt >> --- a/test/jdk/ProblemList.txt >> +++ b/test/jdk/ProblemList.txt >> @@ -916,6 +916,8 @@ >> >> ??# jdk_internal >> >> +jdk/internal/platform/docker/TestDockerMemoryMetrics.java 8250986 >> linux-5.4.0-1019-oracle >> + >> ??############################################################################ >> >> >> ??# jdk_jpackage >> >> >> Thank you, >> >> Misha >> From david.holmes at oracle.com Tue Aug 4 00:41:20 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Aug 2020 10:41:20 +1000 Subject: RFR(T): 8250986: Problem list docker/TestMemoryAwareness.java and docker/TestDockerMemoryMetrics.java for linux-5.4.0-1019-oracle In-Reply-To: <324834bb-f63e-284e-7d45-97d639e25440@oracle.com> References: <7c29a474-2bed-4eb7-720e-e7774855c82f@oracle.com> <324834bb-f63e-284e-7d45-97d639e25440@oracle.com> Message-ID: <50b62994-2764-6716-017c-2b7f952ffdea@oracle.com> On 4/08/2020 10:07 am, mikhailo.seledtsov at oracle.com wrote: > Hi David, > > On 8/3/20 4:33 PM, David Holmes wrote: >> Hi Misha, >> >> On 4/08/2020 7:15 am, mikhailo.seledtsov at oracle.com wrote: >>> Please review this trivial fix that problem lists these 2 tests >>> specifically for linux-5.4.0-1019-oracle, until JDK-8250984 is fixed. >> >> In that case 8250984 is what should be in the entry not 8250986: >> >> containers/docker/TestMemoryAwareness.java 8250986 >> linux-5.4.0-1019-oracle > Thanks for the catch. I have replaced 8250986 to be 8250984. Good to go then. :) >> >> I was unaware that these statements could specify a detailed OS >> version like that. But where does that version string come from? > JTreg allows to specify a 'flavor' of OS. JTreg reports this value in > the test log as "javatestOS". In this case: "javatestOS=Linux > 5.4.0-1019-oracle", which seems to be a kernel version on Linux hosts. That is useful to know. Thanks, David > Thanks, > Misha > >> >> Thanks, >> David >> >> >>> Testing: >>> >>> ?? - ran on the linux-5.4.0-1019-oracle host, the problem listed >>> tests do not get executed >>> >>> ?? - ran on OL 7.6, the problem listed tests execute and pass >>> >>> >>> ===== Change: >>> >>> diff --git a/test/hotspot/jtreg/ProblemList.txt >>> b/test/hotspot/jtreg/ProblemList.txt >>> --- a/test/hotspot/jtreg/ProblemList.txt >>> +++ b/test/hotspot/jtreg/ProblemList.txt >>> @@ -85,6 +85,7 @@ >>> >>> ??runtime/jni/terminatedThread/TestTerminatedThread.java 8219652 >>> aix-ppc64 >>> ??runtime/ReservedStack/ReservedStackTest.java 8231031 generic-all >>> +containers/docker/TestMemoryAwareness.java 8250986 >>> linux-5.4.0-1019-oracle >>> >>> ??############################################################################# >>> >>> >>> diff --git a/test/jdk/ProblemList.txt b/test/jdk/ProblemList.txt >>> --- a/test/jdk/ProblemList.txt >>> +++ b/test/jdk/ProblemList.txt >>> @@ -916,6 +916,8 @@ >>> >>> ??# jdk_internal >>> >>> +jdk/internal/platform/docker/TestDockerMemoryMetrics.java 8250986 >>> linux-5.4.0-1019-oracle >>> + >>> ??############################################################################ >>> >>> >>> ??# jdk_jpackage >>> >>> >>> Thank you, >>> >>> Misha >>> From david.holmes at oracle.com Tue Aug 4 01:13:52 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Aug 2020 11:13:52 +1000 Subject: RFR(S) 8249276 CDS archived objects must have "neutral" markwords In-Reply-To: <5d5b4b17-a40f-b0f4-fa50-fdd783def9d4@oracle.com> References: <5d5b4b17-a40f-b0f4-fa50-fdd783def9d4@oracle.com> Message-ID: <37245599-dfb8-0481-6c74-89cffdc5b457@oracle.com> Hi Ioi, On 27/07/2020 3:20 pm, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8249276 > http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v01/ > > > Please review this change (initial patch provided by David Holmes; I > rearranged it a little and added more comments). > > We reinitialize the markWord of all archived objects to remove any side > effect of GC/locking/etc during "java -Xshare:dump". Changes look good to me (naturally :) ). A couple of nits: int hash_archived = archived_oop->identity_hash(); assert(hash_original == hash_archived, "Different hash codes: original %x, archived %x", hash_original, hash_archived); the has_archived setting should be in ifdef DEBUG. 42 Object.class.wait(); 43 } 44 System.out.println("Huh?? notified??"); That could be a spurious wakeup. You could just use Thread.sleep with a suitably long sleep time. that said I'm not sure how the test is intended to operate. If premain returns after starting the thread there is no guarantee the thread will start executing and lock the object before the dump actually commences. Even the sighting of "Let's wait ....." is not sufficient to guarantee that unless you move it inside the sync block. > (David mentioned in the bug comments about assertions inside the > identity_hash() call, but maybe this should be fixed in a different bug?) Yes those assertions are being removed under: https://bugs.openjdk.java.net/browse/JDK-8250606 Thanks, David ----- > Tested with mach5 tiers 1-4. > > Thanks > - Ioi From mikhailo.seledtsov at oracle.com Tue Aug 4 02:00:11 2020 From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com) Date: Mon, 3 Aug 2020 19:00:11 -0700 Subject: RFR(T): 8250986: Problem list docker/TestMemoryAwareness.java and docker/TestDockerMemoryMetrics.java for linux-5.4.0-1019-oracle In-Reply-To: <50b62994-2764-6716-017c-2b7f952ffdea@oracle.com> References: <7c29a474-2bed-4eb7-720e-e7774855c82f@oracle.com> <324834bb-f63e-284e-7d45-97d639e25440@oracle.com> <50b62994-2764-6716-017c-2b7f952ffdea@oracle.com> Message-ID: <181e49c4-ba8a-58ed-3abb-61341feaeaeb@oracle.com> Thank you David. I have pushed the change. Misha On 8/3/20 5:41 PM, David Holmes wrote: > On 4/08/2020 10:07 am, mikhailo.seledtsov at oracle.com wrote: >> Hi David, >> >> On 8/3/20 4:33 PM, David Holmes wrote: >>> Hi Misha, >>> >>> On 4/08/2020 7:15 am, mikhailo.seledtsov at oracle.com wrote: >>>> Please review this trivial fix that problem lists these 2 tests >>>> specifically for linux-5.4.0-1019-oracle, until JDK-8250984 is fixed. >>> >>> In that case 8250984 is what should be in the entry not 8250986: >>> >>> containers/docker/TestMemoryAwareness.java 8250986 >>> linux-5.4.0-1019-oracle >> Thanks for the catch. I have replaced 8250986 to be 8250984. > > Good to go then. :) > >>> >>> I was unaware that these statements could specify a detailed OS >>> version like that. But where does that version string come from? >> JTreg allows to specify a 'flavor' of OS. JTreg reports this value in >> the test log as "javatestOS". In this case: "javatestOS=Linux >> 5.4.0-1019-oracle", which seems to be a kernel version on Linux hosts. > > That is useful to know. > > Thanks, > David > >> Thanks, >> Misha >> >>> >>> Thanks, >>> David >>> >>> >>>> Testing: >>>> >>>> ?? - ran on the linux-5.4.0-1019-oracle host, the problem listed >>>> tests do not get executed >>>> >>>> ?? - ran on OL 7.6, the problem listed tests execute and pass >>>> >>>> >>>> ===== Change: >>>> >>>> diff --git a/test/hotspot/jtreg/ProblemList.txt >>>> b/test/hotspot/jtreg/ProblemList.txt >>>> --- a/test/hotspot/jtreg/ProblemList.txt >>>> +++ b/test/hotspot/jtreg/ProblemList.txt >>>> @@ -85,6 +85,7 @@ >>>> >>>> ??runtime/jni/terminatedThread/TestTerminatedThread.java 8219652 >>>> aix-ppc64 >>>> ??runtime/ReservedStack/ReservedStackTest.java 8231031 generic-all >>>> +containers/docker/TestMemoryAwareness.java 8250986 >>>> linux-5.4.0-1019-oracle >>>> >>>> ??############################################################################# >>>> >>>> >>>> diff --git a/test/jdk/ProblemList.txt b/test/jdk/ProblemList.txt >>>> --- a/test/jdk/ProblemList.txt >>>> +++ b/test/jdk/ProblemList.txt >>>> @@ -916,6 +916,8 @@ >>>> >>>> ??# jdk_internal >>>> >>>> +jdk/internal/platform/docker/TestDockerMemoryMetrics.java 8250986 >>>> linux-5.4.0-1019-oracle >>>> + >>>> ??############################################################################ >>>> >>>> >>>> ??# jdk_jpackage >>>> >>>> >>>> Thank you, >>>> >>>> Misha >>>> From david.holmes at oracle.com Tue Aug 4 04:44:52 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Aug 2020 14:44:52 +1000 Subject: RFR (trivial): 8248906: runtime/Thread/ThreadObjAccessAtExit.java fails due to OutOfMemoryErrors Message-ID: Bug: https://bugs.openjdk.java.net/browse/JDK-8248906 webrev: http://cr.openjdk.java.net/~dholmes/8248906/webrev/ This rare OOME with ZGC doesn't reproduce readily and has not reappeared in the mainline CI. However it has failed twice in Panama testing for the same reasons. A personal test job also showed that ZGC plus JFR fails in a more extreme way (VM initialization failure) and that failure is completely reproducible. Bumping the minimum heap to 11M fixed that issue so that is what I will commit for this fix. Testing via tiers 1-3 plus manual on Linux. Thanks, David From patricio.chilano.mateo at oracle.com Tue Aug 4 05:11:27 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Tue, 4 Aug 2020 02:11:27 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> Message-ID: <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> Hi David, Thanks for looking at this. On 8/3/20 3:12 AM, David Holmes wrote: > Hi Patricio, > > I've cc'd JFR dev folk as well. > > Thanks for working on this Valhalla-preview support work. I'm not > quite clear if this part of the work is intended to be an isolated and > independent change relative to the broader scope of work outlined in > the draft JEP: JDK-8249100 > > https://bugs.openjdk.java.net/browse/JDK-8249100 > > so I'm seeking clarification on that from the Valhalla folk. > > Meanwhile ... > > On 29/07/2020 5:16 am, Patricio Chilano wrote: >> Hi all, >> >> Please review the following change that adds diagnostic capabilities >> when synchronizing on primitive wrapper classes. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8242263 >> Webrev: http://cr.openjdk.java.net/~pchilanomate/8242263/v1/webrev/ >> >> The new flag allows to identify synchronization on these classes and >> to take one of the following actions: exit the VM with fatal error, >> log a warning message, or issue a JFR event. > > I'm not clear how conditionally enabled events should be exposed in > JFR so will need some input from JFR folk on that part. Sounds good. >> The implementation uses a simple approach where a check is added at >> the beginning of the monitorenter generated code when the flag is >> enabled to check whether the object is of a primitive wrapper class. >> If it is, we jump to the slow path, > > I like this simple approach! > >> otherwise we just continue as always. The extra instructions will be: >> load the klass of the object, load the _access_flags field for that >> klass, AND with a constant, and branch based on the result. The code >> will only be generated whenever the new opt-in diagnostic flag is >> enabled so performance won't be affected when off. > > In some cases you've factored out code that was only executed > with/without biased-locking so that it is always executed and so > available for your new code e.g > > src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp > > -?? if (UseBiasedLocking) { > -???? // load object > ????? str(obj, Address(disp_hdr, obj_offset)); > -???? null_check_offset = biased_locking_enter(obj, hdr/*scratched*/, > tmp1, false, tmp2, done, slow_case); > -?? } > > but the same code on Aarch64 already does the same thing, so > performance wise this shouldn't be a concern. But I removed a later "str(obj, Address(disp_hdr, obj_offset));" in c1_MacroAssembler_arm.cpp, so it should be the same. I basically changed the code to match the pattern we have for other platforms where the first thing we do is store the oop in the BasicObjectLock object. In fact in arm32 now we are executing that store twice when biased locking is enabled. > On the other hand the change from > > null_check_offset = biased_locking_enter(...) > > to: > > null_check_offset = offset(); > biased_locking_enter(...) > > does seem to penalize the biased-locking code. Can you not just do: > > ?? if (DiagnoseSyncOnPrimitiveWrappers) { > ?+?? null_check_offset = offset(); > ???? load_klass(hdr, obj); > ???? ldr(hdr, Address(hdr, Klass::access_flags_offset())); > ???? tst(hdr, JVM_ACC_IS_BOX_CLASS); > ???? br(Assembler::NE, slow_case); > ?? } > > so that only the new code is impacted? The call to offset() should not generate extra code, it is only executed by the compiler thread. I think this is to know where to put the code that tests for a NULL oop, and it has to be done right before the actual dereference of the oop.? I see what you mean though that we are making a call to offset() again in biased_locking_enter() and discarding the value. Maybe I can remove those calls instead? I just don't see the point in having "null_check_offset = offset()" inside conditionals since regardless of the flags we pass we always dereference the oop right at the beginning (after storing the object in the BasicObjectLock). > --- > > General comment - you use this a lot: > > if (DiagnoseSyncOnPrimitiveWrappers) > > but DiagnoseSyncOnPrimitiveWrappers is not a boolean and hotspot-style > says to avoid implicit booleans, so those checks need to be expanded to: > > if (DiagnoseSyncOnPrimitiveWrappers != 0) Fixed. > --- > > src/hotspot/share/runtime/synchronizer.cpp > > To avoid the need to re-manifest the current thread you can do: > > void ObjectSynchronizer::handle_sync_on_primitive_wrapper(Handle obj, > Thread* current) { > ?? assert(current->is_Java_thread(), "must be for java object > synchronization"); > ?? JavaThread* self = (JavaThread) current; Done. > ?805 assert(!do_lock || DiagnoseSyncOnPrimitiveWrappers != > ObjectSynchronizer::FATAL_EXIT || !obj->klass()->is_box(), > "ObjectLocker synchronizing on primitive wrapper"); > > I don't quite follow that assertion - why is the middle part not just > "DiagnoseSyncOnPrimitiveWrappers != 0" ? Or why not just assert it is > not a box so that we locate all places where we do sync on boxes? Ah, that was a leftover from when "1" meant throw IMSE exception. It didn't made sense to me to throw an IMSE exception there since that was an internal synchronization attempt, so I was asserting instead. I removed that line. > --- > > test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java > > Please add "@bug 8242263". Done. > ?32? * @run main/othervm SyncOnPrimitiveWrapperTest > > As the test only launches other VMs this can just be: > ?32? * @run driver SyncOnPrimitiveWrapperTest Done. > It would be nice to factor out the common options rather than > repeating so many of the string literals. Added generateTests() to deal with that. > I don't think you want to use ProcessTools.createTestJvm as that will > apply flags from the test run that will potentially conflict with the > explicit flag combinations you are testing. Just use > ProcessTools.createJavaProcessBuilder. Fixed. > Both the FatalTest and the LogTest should be testing wrappers of each > type. Done. LogTest was already testing all the types so I only had to add them for FatalTest. Since testing each type for the FatalTest requires spawning a new VM each time, I implemented that in generateTests() as additional tests with one more argument (an index to the testObjects list). > ?99 Integer integer = new Integer(10); > > Use Integer.valueOf(10) (or allow implicit boxing) as direct > construction of wrapper types is something that will itself be subject > to warnings in the future. Ok, as part of the above changes I'm now testing the objects in testObjects list which use valueOf(). > 101 throw new RuntimeException("synchronization on primitve wrapper > did not failed"); > > typos: primitve -> primitive > ?????? failed -> fail Fixed. > I don't understand why the Log test is so complex. I just expected to > see a simple "synchronized(o){}" for an 'o' of each primitive type, > and a check that the log contained the expected message. I added the "for" loop to test C2 properly since I found the special flags alone are not enough. I also added additional threads so that contention paths are also exercised. With the nested synchronized statements the idea was to test more than the simple case and test recursive locking too. Maybe I can remove the nested synchronized statements? (I changed "sharedCounter += 2;" to just an increment since it otherwise might confuse people reading it that it has some special meaning. I think it did in some prior version). > --- > > ?test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java > > The licence header should not be the version with the classpath > exception. Fixed. > Please add "@bug 8242263". Done. > The test should test each of the wrapper types, not just Integer Done. I also realized I should have added -XX:StartFlightRecording to the test otherwise right now it asserts because of the check I added in arguments.cpp. In my tests it never failed because I had a small error where I had an extra closing bracket inside the #if INCLUDE_JFR macro in arguments.cpp that I fixed before sending v1. In any case, now that I look at it I think it doesn't make sense to enforce -XX:StartFlightRecording since it can work without it. So I just removed that extra check in arguments.cpp. Let's see also what the JFR folk say. >> In addition to the purpose described in the description of the bug, >> this flag will also be useful when trying to diagnose possible >> synchronization issues if these classes ever become inline types as >> part of the Valhalla project. >> >> I added test SyncOnPrimitiveWrapperTest.java that tests for the exit >> and logging cases. I added test TestSyncOnPrimitiveWrapperEvent.java >> to test for the JFR event case. >> I tested the patch running tiers1-6 in mach5 with the flag set to >> DiagnoseSyncOnPrimitiveWrappers=2. > > Okay. > >> I checked it builds with arm32 and ppc but can't run any tests on >> those platforms, so it would be good if somebody can run the new test >> included in the patch. >> >> Let me know if you think I should run or add any more tests. > > The primary testing here is the new tests. Here is v2: Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v2/webrev/ Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v2/inc/webrev/ Thanks David! Patricio > Thanks, > David > ----- > >> Thanks! >> >> Patricio From ioi.lam at oracle.com Tue Aug 4 05:59:06 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 3 Aug 2020 22:59:06 -0700 Subject: RFR(M) 8250990 Consolidate object copying code for CDS static/dynamic archive dumping In-Reply-To: <39423dce-d769-bb92-4d22-9dc848d1086e@oracle.com> References: <07b2ee18-0a5f-0839-0a6c-b54d788e4cd2@oracle.com> <39423dce-d769-bb92-4d22-9dc848d1086e@oracle.com> Message-ID: Here's an updated version after offline discussion with Coleen: http://cr.openjdk.java.net/~iklam/jdk16/8250990-consolidate-static-dynamic-cds-objcopy.v02/ http://cr.openjdk.java.net/~iklam/jdk16/8250990-consolidate-static-dynamic-cds-objcopy.v02-delta/ - Moved DumpAllocStats into its own cpp/hpp files - Removed the use of inline.hpp files from archiveBuilder.hpp - Changed comments about the old ArchiveCompactor class to ? the new AcrchiveBuilder class. - Removed CDSAddSymbol_lock which is no longer used. Thanks - Ioi On 8/3/20 3:11 PM, Ioi Lam wrote: > There turns out to be a lot of duplication in the static/dynamic > dumping code. To keep the changes manageable, I've reorganized the REF > into a few sub-tasks. This particular review is for: > > https://bugs.openjdk.java.net/browse/JDK-8250990 > http://cr.openjdk.java.net/~iklam/jdk16/8250990-consolidate-static-dynamic-cds-objcopy.v01/ > > https://wiki.openjdk.java.net/display/HotSpot/How+CDS+Copies+Class+Metadata+into+the+Archive > > > I've also updated the e-mail subject to use the new JBS ID (JDK-8250990). > > This patch is an intermediate step. Some code might look like they are > in the wrong place -- for example: _estimated_metsapceobj_bytes has > been moved to archiveBuilder.hpp, but _estimated_hashtable_bytes is > still in dynamicArchive.cpp. The latter will eventually be moved as > part of the second step (JDK-8250989 > Consolidate buffer allocation code for CDS static/dynamic archive > dumping). > > (Coleen has given me feedback off-line. I'll post a new version soon > to limit the API in archiveBuilder.hpp) > > Thanks > - Ioi > > On 7/31/20 11:36 PM, Ioi Lam wrote: >> >> Currently there are two very similar implementations for static CDS >> archive >> dumping (metaspaceShared.cpp) and dynamic CDS archive dumping >> (dynamicArchive.cpp). >> >> I've merged the two versions and moved to a new file, >> archiveBuilder.cpp. >> >> I also improved the copying algorithm to speed up dumping. Times for >> dumping >> 20000+ classes: >> >> ? ?? ????? static??????? dynamic >> ??? Old??? 42.655 sec??? 67.014 sec >> ??? New??? 37.027 sec??? 34.974 sec >> >> Normally I would not try to optimize when refactoring code. I made an >> earlier, naive attempt to use the old dynamic dumper for static dumping >> (because dynamic dumping is essentially a superset of static dumping). >> However, that caused significant regression in static dumping speed. >> >> Hopefully the new algorithm is both faster and easier to understand. >> Please see the wiki document for more information. >> >> Tested with mach5 tiers 1-4. >> >> Thanks >> - Ioi > From xxinliu at amazon.com Tue Aug 4 06:39:52 2020 From: xxinliu at amazon.com (Liu, Xin) Date: Tue, 4 Aug 2020 06:39:52 +0000 Subject: RFR(S): 8247732: validate user-input intrinsic_ids in ControlIntrinsic In-Reply-To: <916b3a4a-5617-941d-6161-840f3ea900bd@oracle.com> References: <821e3d29-c95b-aafc-8ee5-6e49a1bdde82@amazon.com> <9b324805-eb86-27e1-5dcb-96a823f8495b@amazon.com> <82cba5e4-2020-ce0a-4576-e8e0cc2e5ae5@oracle.com> <1595401959932.33284@amazon.com> <1595520162373.22868@amazon.com>, <916b3a4a-5617-941d-6161-840f3ea900bd@oracle.com> Message-ID: <1596523192072.15354@amazon.com> hi, Nils, Tobias would like to keep the parser behavior consistency. I think it means that the hotspot need to suppress the warning if the intrinsic_id doesn't exists in compiler directive. eg. -XX:CompileCommand=option,,ControlIntrinsic=-_nonexist. What do you think about it? Here is the latest webrev: http://cr.openjdk.java.net/~xliu/8247732/01/webrev/ thanks, --lx ________________________________________ From: Tobias Hartmann Sent: Friday, July 24, 2020 2:52 AM To: Liu, Xin; Nils Eliasson; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev Subject: RE: [EXTERNAL] RFR(S): 8247732: validate user-input intrinsic_ids in ControlIntrinsic CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hi Liu, On 23.07.20 18:02, Liu, Xin wrote: > That is my intention too, but CompilerOracle doesn't exit JVM when it encounters parsing errors. > It just exacts information from CompileCommand as many as possible. That makes sense because compiler "directives" are supposed to be optional for program execution. > > I do put the error message in parser's errorbuf. I set a flag "exit_on_error" to quit JVM after it dumps parser errors. yes, I treat undefined intrinsics as fatal errors. > This behavior is from Nils comment: "I want to see an error on startup if the user has specified unknown intrinsic names." It is also consistent with JVM option -XX:ControlIntrinsic=. Okay, thanks for the explanation! I would prefer consistency in error handling of compiler directives, i.e., handle all parser failures the same way. But I leave it to Nils to decide. Best regards, Tobias From martin.doerr at sap.com Tue Aug 4 07:53:23 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 4 Aug 2020 07:53:23 +0000 Subject: RFR (trivial): 8248906: runtime/Thread/ThreadObjAccessAtExit.java fails due to OutOfMemoryErrors In-Reply-To: References: Message-ID: Hi David, I'm fine with this fix. Looks good and trivial. Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev > On Behalf Of David Holmes > Sent: Dienstag, 4. August 2020 06:45 > To: hotspot-runtime-dev at openjdk.java.net > Subject: RFR (trivial): 8248906: runtime/Thread/ThreadObjAccessAtExit.java > fails due to OutOfMemoryErrors > > Bug: https://bugs.openjdk.java.net/browse/JDK-8248906 > webrev: http://cr.openjdk.java.net/~dholmes/8248906/webrev/ > > This rare OOME with ZGC doesn't reproduce readily and has not reappeared > in the mainline CI. However it has failed twice in Panama testing for > the same reasons. A personal test job also showed that ZGC plus JFR > fails in a more extreme way (VM initialization failure) and that failure > is completely reproducible. Bumping the minimum heap to 11M fixed that > issue so that is what I will commit for this fix. > > Testing via tiers 1-3 plus manual on Linux. > > Thanks, > David From jiefu at tencent.com Tue Aug 4 09:08:28 2020 From: jiefu at tencent.com (=?utf-8?B?amllZnUo5YKF5p2wKQ==?=) Date: Tue, 4 Aug 2020 09:08:28 +0000 Subject: RFR: 8251031: Some vmTestbase/nsk/monitoring/RuntimeMXBean tests fail with hostnames starting from digits Message-ID: <8A8E8751-AF91-471A-B668-5D6461B72356@tencent.com> Hi all, JBS: https://bugs.openjdk.java.net/browse/JDK-8251031 Webrev: http://cr.openjdk.java.net/~jiefu/8251031/webrev.00/ Some vmTestbase/nsk/monitoring/RuntimeMXBean tests failed in our test infrastructure. The reason is that these tests reject hostnames starting with digits. However, hostnames starting from digits are actually valid according to RFC1123 [1][2]. It would be better to fix it. Thanks a lot. Best regards, Jie [1] https://tools.ietf.org/html/rfc1123#page-13 [2] https://en.wikipedia.org/wiki/Hostname From alexander.scherbatiy at bell-sw.com Tue Aug 4 09:11:03 2020 From: alexander.scherbatiy at bell-sw.com (Alexander Scherbatiy) Date: Tue, 4 Aug 2020 12:11:03 +0300 Subject: RFR 8241053: Hotspot runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java test fails on Alpine Linux with debug build In-Reply-To: <5b530109-bda2-820e-6070-d779f08b64f6@oracle.com> References: <87a32c5b-8e96-f323-1e20-8df181ab3ec0@bell-sw.com> <5b530109-bda2-820e-6070-d779f08b64f6@oracle.com> Message-ID: <64281c1f-970d-b792-eb42-63ae1488694b@bell-sw.com> Could you review the updated fix: ? http://cr.openjdk.java.net/~alexsch/8241053/webrev.01 The assert_status() and the thread cleanup is moved under the "if (status != 0)" check. Thanks, Alexander. On 04.08.2020 01:51, David Holmes wrote: > Hi Alexander, > > Thanks for fixing this. > > On 4/08/2020 1:33 am, Alexander Scherbatiy wrote: >> Hello, >> >> Could you review the fix for the issue: >> ?? Bug: https://bugs.openjdk.java.net/browse/JDK-8241053 >> ?? Webrev: http://cr.openjdk.java.net/~alexsch/8241053/weberv.00 > > I would simplify this a little: > > ?? assert_status(status == 0 || status == EINVAL, status, > "pthread_attr_setstacksize"); > ?? // pthread_attr_setstacksize() function can fail > ?? // if the stack size exceeds a system-imposed limit. > ?? if (status == EINVAL) { > > to > ?? if (status != 0) { > ???? // pthread_attr_setstacksize() function can fail > ???? // if the stack size exceeds a system-imposed limit. > ???? assert_status(status == EINVAL, status, > "pthread_attr_setstacksize"); > > Thanks, > David > ----- > > >> The following tests fail on Linux Alpine with musl libc: >> ?? test/hotspot/jtreg/runtime/Thread/TestThreadStackSizes.java >> test/hotspot/jtreg/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >> >> >> This is because muls pthread_attr_setstacksize(3) function >> implementation [1] returns EINVAL when stack size exceeds the certain >> limit. >> >> According to POSIX pthread_attr_setstacksize() function description [2]: >> ??The pthread_attr_setstacksize() function will fail if: >> ???? [EINVAL]? The value of stacksize is less than PTHREAD_STACK_MIN >> or exceeds a system-imposed limit. >> >> The proposed fix excludes EINVAL value from assert_status() and for >> the EINVAL return value makes the same cleanup actions and returns >> false from os::create_thread() function as it is done for AIX[3]. >> >> To reproduce the issue it needs to build Portola from repository [4] >> using instructions from JEP 386: Alpine Linux/x64 Port [5]. >> After the fix both TestThreadStackSizes.java and >> TestOptionsWithRanges.java tests pass on Alpine Linux with musl libc >> and on Ubuntu 18.0.4. >> >> [1] >> https://git.musl-libc.org/cgit/musl/tree/src/thread/pthread_attr_setstacksize.c >> >> [2] >> https://pubs.opengroup.org/onlinepubs/007908799/xsh/pthread_attr_setstacksize.html >> >> [3] >> https://hg.openjdk.java.net/jdk/jdk/file/c8102e6fc512/src/hotspot/os/aix/os_aix.cpp#l890 >> >> [4] https://github.com/openjdk/portola >> [5] https://openjdk.java.net/jeps/386 >> >> Thanks, >> Alexander. >> From dcherepanov at azul.com Tue Aug 4 09:57:10 2020 From: dcherepanov at azul.com (Dmitry Cherepanov) Date: Tue, 4 Aug 2020 12:57:10 +0300 Subject: RFR: 8250636: iso8601_time returns incorrect offset part on MacOS In-Reply-To: <934a8a40-dd51-512f-f010-909ed312783d@oracle.com> References: <777fc30a-45ef-aa45-8f04-0bbe80c5a83d@azul.com> <0e1ed1ae-2011-2e4a-4f58-d99b57ede74c@azul.com> <934a8a40-dd51-512f-f010-909ed312783d@oracle.com> Message-ID: Hi David, On 03.08.2020 10:53, David Holmes wrote: > Hi Dmitry, > > On 31/07/2020 8:29 pm, Dmitry Cherepanov wrote: >> Hi Gerard, >> >> Thanks for reviewing this, moving the logic of get_timezone to >> iso8601_time looks good to me too. >> Here's an updated webrev >> http://cr.openjdk.java.net/~dcherepanov/8250636/webrev.v4/ > > This version also looks good to me. But now I see all the code clearly in one place I have to wonder why we are not using tm_gmtoff on all platforms except Windows? That would simplify the logic even further. > > That goes beyond fixing the current actual bug though so I'll leave that choice to you. Thanks for the suggestion. This change would not work on Solaris. But given the removal of Solaris port, I also see no other reasons why not to use tm_gmtoff on platforms except Windows. Updated webrev: http://cr.openjdk.java.net/~dcherepanov/8250636/webrev.v5/ Jdk-submit testing passed with this change Thanks, Dmitry From aph at redhat.com Tue Aug 4 10:26:59 2020 From: aph at redhat.com (Andrew Haley) Date: Tue, 4 Aug 2020 11:26:59 +0100 Subject: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic In-Reply-To: References: <654e8a2e-87a1-9c3d-4c58-36b4215bb0ad@redhat.com> <5350e745-87af-ce34-8711-2b652a38e323@redhat.com> Message-ID: <4d1f7179-2cad-620d-f833-949e8b39ee06@redhat.com> On 8/3/20 2:18 AM, Yangfei (Felix) wrote: > OK. I simply comment out the following code changes in vm_version_aarch64.cpp: > > 393 if (UseSHA && (auxv & HWCAP_SHA512)) { > 394 // Do not auto-enable UseSHA512Intrinsics until it has been fully tested on hardware > 395 // if (FLAG_IS_DEFAULT(UseSHA512Intrinsics)) { > 396 // FLAG_SET_DEFAULT(UseSHA512Intrinsics, true); > 397 // } > 398 } else if (UseSHA512Intrinsics) { > > Once this has been fully tested on real hardware, it will be easy to auto-enable. > > New webrev: http://cr.openjdk.java.net/~fyang/8165404/webrev.01/ Looks good, thanks. My apologies, there is another thing I should have mentioned: please add test cases for the new instructions to aarch64-asmtest.py and regenerate the code between // BEGIN Generated code -- do not edit // Generated by aarch64-asmtest.py and // END Generated code -- do not edit I'm trying to be more strict about making sure all of the instructions get tested. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From david.holmes at oracle.com Tue Aug 4 10:46:57 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Aug 2020 20:46:57 +1000 Subject: RFR (trivial): 8248906: runtime/Thread/ThreadObjAccessAtExit.java fails due to OutOfMemoryErrors In-Reply-To: References: Message-ID: <42492958-a279-b1c2-1c8c-e38505a277a9@oracle.com> Hi Martin, Thanks for the review. Cheers, David On 4/08/2020 5:53 pm, Doerr, Martin wrote: > Hi David, > > I'm fine with this fix. Looks good and trivial. > > Best regards, > Martin > > >> -----Original Message----- >> From: hotspot-runtime-dev >> On Behalf Of David Holmes >> Sent: Dienstag, 4. August 2020 06:45 >> To: hotspot-runtime-dev at openjdk.java.net >> Subject: RFR (trivial): 8248906: runtime/Thread/ThreadObjAccessAtExit.java >> fails due to OutOfMemoryErrors >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8248906 >> webrev: http://cr.openjdk.java.net/~dholmes/8248906/webrev/ >> >> This rare OOME with ZGC doesn't reproduce readily and has not reappeared >> in the mainline CI. However it has failed twice in Panama testing for >> the same reasons. A personal test job also showed that ZGC plus JFR >> fails in a more extreme way (VM initialization failure) and that failure >> is completely reproducible. Bumping the minimum heap to 11M fixed that >> issue so that is what I will commit for this fix. >> >> Testing via tiers 1-3 plus manual on Linux. >> >> Thanks, >> David From david.holmes at oracle.com Tue Aug 4 10:51:35 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Aug 2020 20:51:35 +1000 Subject: RFR: 8251031: Some vmTestbase/nsk/monitoring/RuntimeMXBean tests fail with hostnames starting from digits In-Reply-To: <8A8E8751-AF91-471A-B668-5D6461B72356@tencent.com> References: <8A8E8751-AF91-471A-B668-5D6461B72356@tencent.com> Message-ID: <5a9156f2-6d03-129c-ecaf-aacb21e4a47e@oracle.com> Hi Jie, On 4/08/2020 7:08 pm, jiefu(??) wrote: > Hi all, > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251031 > Webrev: http://cr.openjdk.java.net/~jiefu/8251031/webrev.00/ > > Some vmTestbase/nsk/monitoring/RuntimeMXBean tests failed in our test infrastructure. > The reason is that these tests reject hostnames starting with digits. > > However, hostnames starting from digits are actually valid according to RFC1123 [1][2]. > It would be better to fix it. I don't process regular expressions very well but I can see the addition of the 0-9 character class in the expression so that appears fine. :) What I don't understand is why we aren't simply splitting the string at '@' and taking the pid and hostname directly. (That's somewhat rhetorical I don't expect you to change the test.) Thanks, David ----- > Thanks a lot. > Best regards, > Jie > > [1] https://tools.ietf.org/html/rfc1123#page-13 > [2] https://en.wikipedia.org/wiki/Hostname > From david.holmes at oracle.com Tue Aug 4 10:55:46 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Aug 2020 20:55:46 +1000 Subject: RFR 8241053: Hotspot runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java test fails on Alpine Linux with debug build In-Reply-To: <64281c1f-970d-b792-eb42-63ae1488694b@bell-sw.com> References: <87a32c5b-8e96-f323-1e20-8df181ab3ec0@bell-sw.com> <5b530109-bda2-820e-6070-d779f08b64f6@oracle.com> <64281c1f-970d-b792-eb42-63ae1488694b@bell-sw.com> Message-ID: Looks good! Thanks, David ----- On 4/08/2020 7:11 pm, Alexander Scherbatiy wrote: > Could you review the updated fix: > ? http://cr.openjdk.java.net/~alexsch/8241053/webrev.01 > > The assert_status() and the thread cleanup is moved under the "if > (status != 0)" check. > > Thanks, > Alexander. > > On 04.08.2020 01:51, David Holmes wrote: >> Hi Alexander, >> >> Thanks for fixing this. >> >> On 4/08/2020 1:33 am, Alexander Scherbatiy wrote: >>> Hello, >>> >>> Could you review the fix for the issue: >>> ?? Bug: https://bugs.openjdk.java.net/browse/JDK-8241053 >>> ?? Webrev: http://cr.openjdk.java.net/~alexsch/8241053/weberv.00 >> >> I would simplify this a little: >> >> ?? assert_status(status == 0 || status == EINVAL, status, >> "pthread_attr_setstacksize"); >> ?? // pthread_attr_setstacksize() function can fail >> ?? // if the stack size exceeds a system-imposed limit. >> ?? if (status == EINVAL) { >> >> to >> ?? if (status != 0) { >> ???? // pthread_attr_setstacksize() function can fail >> ???? // if the stack size exceeds a system-imposed limit. >> ???? assert_status(status == EINVAL, status, >> "pthread_attr_setstacksize"); >> >> Thanks, >> David >> ----- >> >> >>> The following tests fail on Linux Alpine with musl libc: >>> ?? test/hotspot/jtreg/runtime/Thread/TestThreadStackSizes.java >>> test/hotspot/jtreg/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>> >>> >>> This is because muls pthread_attr_setstacksize(3) function >>> implementation [1] returns EINVAL when stack size exceeds the certain >>> limit. >>> >>> According to POSIX pthread_attr_setstacksize() function description [2]: >>> ??The pthread_attr_setstacksize() function will fail if: >>> ???? [EINVAL]? The value of stacksize is less than PTHREAD_STACK_MIN >>> or exceeds a system-imposed limit. >>> >>> The proposed fix excludes EINVAL value from assert_status() and for >>> the EINVAL return value makes the same cleanup actions and returns >>> false from os::create_thread() function as it is done for AIX[3]. >>> >>> To reproduce the issue it needs to build Portola from repository [4] >>> using instructions from JEP 386: Alpine Linux/x64 Port [5]. >>> After the fix both TestThreadStackSizes.java and >>> TestOptionsWithRanges.java tests pass on Alpine Linux with musl libc >>> and on Ubuntu 18.0.4. >>> >>> [1] >>> https://git.musl-libc.org/cgit/musl/tree/src/thread/pthread_attr_setstacksize.c >>> >>> [2] >>> https://pubs.opengroup.org/onlinepubs/007908799/xsh/pthread_attr_setstacksize.html >>> >>> [3] >>> https://hg.openjdk.java.net/jdk/jdk/file/c8102e6fc512/src/hotspot/os/aix/os_aix.cpp#l890 >>> >>> [4] https://github.com/openjdk/portola >>> [5] https://openjdk.java.net/jeps/386 >>> >>> Thanks, >>> Alexander. >>> From david.holmes at oracle.com Tue Aug 4 11:00:24 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Aug 2020 21:00:24 +1000 Subject: RFR: 8250636: iso8601_time returns incorrect offset part on MacOS In-Reply-To: References: <777fc30a-45ef-aa45-8f04-0bbe80c5a83d@azul.com> <0e1ed1ae-2011-2e4a-4f58-d99b57ede74c@azul.com> <934a8a40-dd51-512f-f010-909ed312783d@oracle.com> Message-ID: <3bd85e9c-3544-d55c-41fa-631d1db6c483@oracle.com> This looks good to me! Much simpler! Thanks, David ----- On 4/08/2020 7:57 pm, Dmitry Cherepanov wrote: > Hi David, > > On 03.08.2020 10:53, David Holmes wrote: >> Hi Dmitry, >> >> On 31/07/2020 8:29 pm, Dmitry Cherepanov wrote: >>> Hi Gerard, >>> >>> Thanks for reviewing this, moving the logic of get_timezone to >>> iso8601_time looks good to me too. >>> Here's an updated webrev >>> http://cr.openjdk.java.net/~dcherepanov/8250636/webrev.v4/ >> >> This version also looks good to me. But now I see all the code clearly in one place I have to wonder why we are not using tm_gmtoff on all platforms except Windows? That would simplify the logic even further. >> >> That goes beyond fixing the current actual bug though so I'll leave that choice to you. > > Thanks for the suggestion. This change would not work on Solaris. But given the removal of Solaris port, I also see no other reasons why not to use tm_gmtoff on platforms except Windows. > > Updated webrev: http://cr.openjdk.java.net/~dcherepanov/8250636/webrev.v5/ > Jdk-submit testing passed with this change > > Thanks, > > Dmitry > From martin.doerr at sap.com Tue Aug 4 11:19:46 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 4 Aug 2020 11:19:46 +0000 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> Message-ID: Hi Patricio, thanks for taking care of PPC64, too. Unfortunately, your version is not correct. "andi" instructions don't set CCR0. I'd implement it as follows (including s390). Not yet tested. Best regards, Martin diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 13:10:51 2020 +0200 @@ -108,7 +108,7 @@ if (DiagnoseSyncOnPrimitiveWrappers != 0) { load_klass(Rscratch, Roop); ld(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); - andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); + testbitdi(CCR0, R0, Rscratch, exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); bne(CCR0, slow_case); } diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 13:10:51 2020 +0200 @@ -913,7 +913,7 @@ if (DiagnoseSyncOnPrimitiveWrappers != 0) { load_klass(tmp, object); ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); - andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); + testbitdi(CCR0, R0, tmp, exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); bne(CCR0, slow_case); } diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 13:10:51 2020 +0200 @@ -2839,7 +2839,7 @@ if (DiagnoseSyncOnPrimitiveWrappers != 0) { load_klass(temp, oop); ld(temp, in_bytes(Klass::access_flags_offset()), temp); - andi(temp, temp, JVM_ACC_IS_BOX_CLASS); + testbitdi(CCR0, R0, temp, exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); bne(CCR0, cont); } diff -r 77852e129401 src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 13:10:51 2020 +0200 @@ -91,6 +91,13 @@ // Save object being locked into the BasicObjectLock... z_stg(obj, Address(disp_hdr, BasicObjectLock::obj_offset_in_bytes())); + if (DiagnoseSyncOnPrimitiveWrappers != 0) { + load_klass(Z_R1_scratch, obj); + testbit(Address(Z_R1_scratch, in_bytes(Klass::access_flags_offset())), + exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); + z_btrue(slow_case); + } + if (UseBiasedLocking) { biased_locking_enter(obj, hdr, Z_R1_scratch, Z_R0_scratch, done, &slow_case); } diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 13:10:51 2020 +0200 @@ -1000,6 +1000,13 @@ // Load markWord from object into displaced_header. z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), object); + if (DiagnoseSyncOnPrimitiveWrappers != 0) { + load_klass(Z_R1_scratch, object); + testbit(Address(Z_R1_scratch, in_bytes(Klass::access_flags_offset())), + exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); + z_btrue(slow_case); + } + if (UseBiasedLocking) { biased_locking_enter(object, displaced_header, Z_R1, Z_R0, done, &slow_case); } diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 13:10:51 2020 +0200 @@ -3358,6 +3358,13 @@ // Load markWord from oop into mark. z_lg(displacedHeader, 0, oop); + if (DiagnoseSyncOnPrimitiveWrappers != 0) { + load_klass(Z_R1_scratch, oop); + testbit(Address(Z_R1_scratch, in_bytes(Klass::access_flags_offset())), + exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); + z_btrue(done); + } + if (try_bias) { biased_locking_enter(oop, displacedHeader, temp, Z_R0, done); } d056149 at lu0482:/priv/d056149/jdk$ hg diff diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 13:17:06 2020 +0200 @@ -108,7 +108,7 @@ if (DiagnoseSyncOnPrimitiveWrappers != 0) { load_klass(Rscratch, Roop); ld(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); - andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); + testbitdi(CCR0, R0, Rscratch, exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); bne(CCR0, slow_case); } diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 13:17:06 2020 +0200 @@ -913,7 +913,7 @@ if (DiagnoseSyncOnPrimitiveWrappers != 0) { load_klass(tmp, object); ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); - andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); + testbitdi(CCR0, R0, tmp, exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); bne(CCR0, slow_case); } diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 13:17:06 2020 +0200 @@ -2839,7 +2839,7 @@ if (DiagnoseSyncOnPrimitiveWrappers != 0) { load_klass(temp, oop); ld(temp, in_bytes(Klass::access_flags_offset()), temp); - andi(temp, temp, JVM_ACC_IS_BOX_CLASS); + testbitdi(CCR0, R0, temp, exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); bne(CCR0, cont); } diff -r 77852e129401 src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 13:17:06 2020 +0200 @@ -91,6 +91,13 @@ // Save object being locked into the BasicObjectLock... z_stg(obj, Address(disp_hdr, BasicObjectLock::obj_offset_in_bytes())); + if (DiagnoseSyncOnPrimitiveWrappers != 0) { + load_klass(Z_R1_scratch, obj); + testbit(Address(Z_R1_scratch, in_bytes(Klass::access_flags_offset())), + exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); + z_btrue(slow_case); + } + if (UseBiasedLocking) { biased_locking_enter(obj, hdr, Z_R1_scratch, Z_R0_scratch, done, &slow_case); } diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 13:17:06 2020 +0200 @@ -1000,6 +1000,13 @@ // Load markWord from object into displaced_header. z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), object); + if (DiagnoseSyncOnPrimitiveWrappers != 0) { + load_klass(Z_R1_scratch, object); + testbit(Address(Z_R1_scratch, in_bytes(Klass::access_flags_offset())), + exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); + z_btrue(slow_case); + } + if (UseBiasedLocking) { biased_locking_enter(object, displaced_header, Z_R1, Z_R0, done, &slow_case); } diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 13:17:06 2020 +0200 @@ -3358,6 +3358,13 @@ // Load markWord from oop into mark. z_lg(displacedHeader, 0, oop); + if (DiagnoseSyncOnPrimitiveWrappers != 0) { + load_klass(Z_R1_scratch, oop); + testbit(Address(Z_R1_scratch, in_bytes(Klass::access_flags_offset())), + exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); + z_btrue(done); + } + if (try_bias) { biased_locking_enter(oop, displacedHeader, temp, Z_R0, done); } From coleen.phillimore at oracle.com Tue Aug 4 12:36:08 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 4 Aug 2020 05:36:08 -0700 (PDT) Subject: RFR(M) 8250990 Consolidate object copying code for CDS static/dynamic archive dumping Message-ID: Ioi, This version looks better. I still think there's too many internal details that are exposed in archiveBuilder.hpp, but this can be cleaned up in another pass. Can you fix the typo: _estimated_metsapceobj_bytes Otherwise, this is reviewed. Coleen ----- Original Message ----- From: ioi.lam at oracle.com To: hotspot-runtime-dev at openjdk.java.net Sent: Tuesday, August 4, 2020 1:59:27 AM GMT -05:00 US/Canada Eastern Subject: Re: RFR(M) 8250990 Consolidate object copying code for CDS static/dynamic archive dumping Here's an updated version after offline discussion with Coleen: http://cr.openjdk.java.net/~iklam/jdk16/8250990-consolidate-static-dynamic-cds-objcopy.v02/ http://cr.openjdk.java.net/~iklam/jdk16/8250990-consolidate-static-dynamic-cds-objcopy.v02-delta/ - Moved DumpAllocStats into its own cpp/hpp files - Removed the use of inline.hpp files from archiveBuilder.hpp - Changed comments about the old ArchiveCompactor class to ? the new AcrchiveBuilder class. - Removed CDSAddSymbol_lock which is no longer used. Thanks - Ioi On 8/3/20 3:11 PM, Ioi Lam wrote: > There turns out to be a lot of duplication in the static/dynamic > dumping code. To keep the changes manageable, I've reorganized the REF > into a few sub-tasks. This particular review is for: > > https://bugs.openjdk.java.net/browse/JDK-8250990 > http://cr.openjdk.java.net/~iklam/jdk16/8250990-consolidate-static-dynamic-cds-objcopy.v01/ > > https://wiki.openjdk.java.net/display/HotSpot/How+CDS+Copies+Class+Metadata+into+the+Archive > > > I've also updated the e-mail subject to use the new JBS ID (JDK-8250990). > > This patch is an intermediate step. Some code might look like they are > in the wrong place -- for example: _estimated_metsapceobj_bytes has > been moved to archiveBuilder.hpp, but _estimated_hashtable_bytes is > still in dynamicArchive.cpp. The latter will eventually be moved as > part of the second step (JDK-8250989 > Consolidate buffer allocation code for CDS static/dynamic archive > dumping). > > (Coleen has given me feedback off-line. I'll post a new version soon > to limit the API in archiveBuilder.hpp) > > Thanks > - Ioi > > On 7/31/20 11:36 PM, Ioi Lam wrote: >> >> Currently there are two very similar implementations for static CDS >> archive >> dumping (metaspaceShared.cpp) and dynamic CDS archive dumping >> (dynamicArchive.cpp). >> >> I've merged the two versions and moved to a new file, >> archiveBuilder.cpp. >> >> I also improved the copying algorithm to speed up dumping. Times for >> dumping >> 20000+ classes: >> >> ? ?? ????? static??????? dynamic >> ??? Old??? 42.655 sec??? 67.014 sec >> ??? New??? 37.027 sec??? 34.974 sec >> >> Normally I would not try to optimize when refactoring code. I made an >> earlier, naive attempt to use the old dynamic dumper for static dumping >> (because dynamic dumping is essentially a superset of static dumping). >> However, that caused significant regression in static dumping speed. >> >> Hopefully the new algorithm is both faster and easier to understand. >> Please see the wiki document for more information. >> >> Tested with mach5 tiers 1-4. >> >> Thanks >> - Ioi > From martin.doerr at sap.com Tue Aug 4 13:12:49 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 4 Aug 2020 13:12:49 +0000 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> Message-ID: Hi again, I just noticed that the flags are 32 bit: Class AccessFlags contains jint _flags. So using 64 bit loads is incorrect (movptr on x64, PPC64 and s390 versions need to get fixed, too). May work on little Endian, but should get fixed. In addition to that, x86 can combine load + and in one instruction (e.g. one of the test instructions). Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev > On Behalf Of Doerr, Martin > Sent: Dienstag, 4. August 2020 13:20 > To: Patricio Chilano ; David Holmes > ; hotspot-runtime-dev at openjdk.java.net; > hotspot-jfr-dev at openjdk.java.net > Subject: [CAUTION] RE: RFR 8242263: Diagnose synchronization on primitive > wrappers > > Hi Patricio, > > thanks for taking care of PPC64, too. > Unfortunately, your version is not correct. "andi" instructions don't set CCR0. > > I'd implement it as follows (including s390). Not yet tested. > > Best regards, > Martin > > > diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp > --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 > 10:03:57 2020 +0200 > +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 > 13:10:51 2020 +0200 > @@ -108,7 +108,7 @@ > if (DiagnoseSyncOnPrimitiveWrappers != 0) { > load_klass(Rscratch, Roop); > ld(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); > - andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); > + testbitdi(CCR0, R0, Rscratch, > exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); > bne(CCR0, slow_case); > } > > diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp > --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 10:03:57 > 2020 +0200 > +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 > 13:10:51 2020 +0200 > @@ -913,7 +913,7 @@ > if (DiagnoseSyncOnPrimitiveWrappers != 0) { > load_klass(tmp, object); > ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); > - andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); > + testbitdi(CCR0, R0, tmp, > exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); > bne(CCR0, slow_case); > } > > diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp > --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 10:03:57 > 2020 +0200 > +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 > 13:10:51 2020 +0200 > @@ -2839,7 +2839,7 @@ > if (DiagnoseSyncOnPrimitiveWrappers != 0) { > load_klass(temp, oop); > ld(temp, in_bytes(Klass::access_flags_offset()), temp); > - andi(temp, temp, JVM_ACC_IS_BOX_CLASS); > + testbitdi(CCR0, R0, temp, > exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); > bne(CCR0, cont); > } > > diff -r 77852e129401 src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp > --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 > 10:03:57 2020 +0200 > +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 > 13:10:51 2020 +0200 > @@ -91,6 +91,13 @@ > // Save object being locked into the BasicObjectLock... > z_stg(obj, Address(disp_hdr, BasicObjectLock::obj_offset_in_bytes())); > > + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > + load_klass(Z_R1_scratch, obj); > + testbit(Address(Z_R1_scratch, in_bytes(Klass::access_flags_offset())), > + exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); > + z_btrue(slow_case); > + } > + > if (UseBiasedLocking) { > biased_locking_enter(obj, hdr, Z_R1_scratch, Z_R0_scratch, done, > &slow_case); > } > diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp > --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 10:03:57 2020 > +0200 > +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 13:10:51 > 2020 +0200 > @@ -1000,6 +1000,13 @@ > // Load markWord from object into displaced_header. > z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), object); > > + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > + load_klass(Z_R1_scratch, object); > + testbit(Address(Z_R1_scratch, in_bytes(Klass::access_flags_offset())), > + exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); > + z_btrue(slow_case); > + } > + > if (UseBiasedLocking) { > biased_locking_enter(object, displaced_header, Z_R1, Z_R0, done, > &slow_case); > } > diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp > --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 > 10:03:57 2020 +0200 > +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 > 13:10:51 2020 +0200 > @@ -3358,6 +3358,13 @@ > // Load markWord from oop into mark. > z_lg(displacedHeader, 0, oop); > > + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > + load_klass(Z_R1_scratch, oop); > + testbit(Address(Z_R1_scratch, in_bytes(Klass::access_flags_offset())), > + exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); > + z_btrue(done); > + } > + > if (try_bias) { > biased_locking_enter(oop, displacedHeader, temp, Z_R0, done); > } > d056149 at lu0482:/priv/d056149/jdk$ hg diff > diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp > --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 > 10:03:57 2020 +0200 > +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 > 13:17:06 2020 +0200 > @@ -108,7 +108,7 @@ > if (DiagnoseSyncOnPrimitiveWrappers != 0) { > load_klass(Rscratch, Roop); > ld(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); > - andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); > + testbitdi(CCR0, R0, Rscratch, > exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); > bne(CCR0, slow_case); > } > > diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp > --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 10:03:57 > 2020 +0200 > +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 > 13:17:06 2020 +0200 > @@ -913,7 +913,7 @@ > if (DiagnoseSyncOnPrimitiveWrappers != 0) { > load_klass(tmp, object); > ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); > - andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); > + testbitdi(CCR0, R0, tmp, > exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); > bne(CCR0, slow_case); > } > > diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp > --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 10:03:57 > 2020 +0200 > +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 > 13:17:06 2020 +0200 > @@ -2839,7 +2839,7 @@ > if (DiagnoseSyncOnPrimitiveWrappers != 0) { > load_klass(temp, oop); > ld(temp, in_bytes(Klass::access_flags_offset()), temp); > - andi(temp, temp, JVM_ACC_IS_BOX_CLASS); > + testbitdi(CCR0, R0, temp, > exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); > bne(CCR0, cont); > } > > diff -r 77852e129401 src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp > --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 > 10:03:57 2020 +0200 > +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 > 13:17:06 2020 +0200 > @@ -91,6 +91,13 @@ > // Save object being locked into the BasicObjectLock... > z_stg(obj, Address(disp_hdr, BasicObjectLock::obj_offset_in_bytes())); > > + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > + load_klass(Z_R1_scratch, obj); > + testbit(Address(Z_R1_scratch, in_bytes(Klass::access_flags_offset())), > + exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); > + z_btrue(slow_case); > + } > + > if (UseBiasedLocking) { > biased_locking_enter(obj, hdr, Z_R1_scratch, Z_R0_scratch, done, > &slow_case); > } > diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp > --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 10:03:57 2020 > +0200 > +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 13:17:06 > 2020 +0200 > @@ -1000,6 +1000,13 @@ > // Load markWord from object into displaced_header. > z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), object); > > + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > + load_klass(Z_R1_scratch, object); > + testbit(Address(Z_R1_scratch, in_bytes(Klass::access_flags_offset())), > + exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); > + z_btrue(slow_case); > + } > + > if (UseBiasedLocking) { > biased_locking_enter(object, displaced_header, Z_R1, Z_R0, done, > &slow_case); > } > diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp > --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 > 10:03:57 2020 +0200 > +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 > 13:17:06 2020 +0200 > @@ -3358,6 +3358,13 @@ > // Load markWord from oop into mark. > z_lg(displacedHeader, 0, oop); > > + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > + load_klass(Z_R1_scratch, oop); > + testbit(Address(Z_R1_scratch, in_bytes(Klass::access_flags_offset())), > + exact_log2((intptr_t)JVM_ACC_IS_BOX_CLASS)); > + z_btrue(done); > + } > + > if (try_bias) { > biased_locking_enter(oop, displacedHeader, temp, Z_R0, done); > } From coleen.phillimore at oracle.com Tue Aug 4 14:15:56 2020 From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com) Date: Tue, 4 Aug 2020 10:15:56 -0400 Subject: RFR(S) 8249276 CDS archived objects must have "neutral" markwords In-Reply-To: <5d5b4b17-a40f-b0f4-fa50-fdd783def9d4@oracle.com> References: <5d5b4b17-a40f-b0f4-fa50-fdd783def9d4@oracle.com> Message-ID: <2bf36735-bc9e-a4f3-dad2-ca02c2634a53@oracle.com> 150 archived_oop->set_mark_raw(markWord::prototype().copy_set_hash(hash_original)); This seems very specific to the content of MarkWord, but it seems like the content and meaning of markWord is splashed all over the vm so it's fine. The change looks good. Coleen On 7/27/20 1:20 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8249276 > http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v01/ > > > Please review this change (initial patch provided by David Holmes; I > rearranged it a little and added more comments). > > We reinitialize the markWord of all archived objects to remove any > side effect of GC/locking/etc during "java -Xshare:dump". > > (David mentioned in the bug comments about assertions inside the > identity_hash() call, but maybe this should be fixed in a different bug?) > > Tested with mach5 tiers 1-4. > > Thanks > - Ioi From martin.doerr at sap.com Tue Aug 4 14:18:41 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 4 Aug 2020 14:18:41 +0000 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> Message-ID: Hi Patricio, I suggest to use movl + testl for checking the access flag as for other access flags on x64. New version for PPC64 and s390 see below. The test SyncOnPrimitiveWrapperTest produces hs_err files as expected. However, I'm getting timeout issues: timed out (timeout set to 120000ms, elapsed time including timeout handling was 122372ms) Can we provide more time? Best regards, Martin diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 16:04:31 2020 +0200 @@ -107,8 +107,8 @@ if (DiagnoseSyncOnPrimitiveWrappers != 0) { load_klass(Rscratch, Roop); - ld(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); - andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); + lwz(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); + testbitdi(CCR0, R0, Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS)); bne(CCR0, slow_case); } diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 16:04:31 2020 +0200 @@ -912,8 +912,8 @@ if (DiagnoseSyncOnPrimitiveWrappers != 0) { load_klass(tmp, object); - ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); - andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); + lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); + testbitdi(CCR0, R0, tmp, exact_log2(JVM_ACC_IS_BOX_CLASS)); bne(CCR0, slow_case); } diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 16:04:31 2020 +0200 @@ -2838,8 +2838,8 @@ if (DiagnoseSyncOnPrimitiveWrappers != 0) { load_klass(temp, oop); - ld(temp, in_bytes(Klass::access_flags_offset()), temp); - andi(temp, temp, JVM_ACC_IS_BOX_CLASS); + lwz(temp, in_bytes(Klass::access_flags_offset()), temp); + testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); bne(CCR0, cont); } diff -r 77852e129401 src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 16:04:31 2020 +0200 @@ -91,6 +91,12 @@ // Save object being locked into the BasicObjectLock... z_stg(obj, Address(disp_hdr, BasicObjectLock::obj_offset_in_bytes())); + if (DiagnoseSyncOnPrimitiveWrappers != 0) { + load_klass(Z_R1_scratch, obj); + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), exact_log2(JVM_ACC_IS_BOX_CLASS)); + z_btrue(slow_case); + } + if (UseBiasedLocking) { biased_locking_enter(obj, hdr, Z_R1_scratch, Z_R0_scratch, done, &slow_case); } diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 16:04:31 2020 +0200 @@ -1000,6 +1000,12 @@ // Load markWord from object into displaced_header. z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), object); + if (DiagnoseSyncOnPrimitiveWrappers != 0) { + load_klass(Z_R1_scratch, object); + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), exact_log2(JVM_ACC_IS_BOX_CLASS)); + z_btrue(slow_case); + } + if (UseBiasedLocking) { biased_locking_enter(object, displaced_header, Z_R1, Z_R0, done, &slow_case); } diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 10:03:57 2020 +0200 +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 16:04:31 2020 +0200 @@ -3358,6 +3358,12 @@ // Load markWord from oop into mark. z_lg(displacedHeader, 0, oop); + if (DiagnoseSyncOnPrimitiveWrappers != 0) { + load_klass(Z_R1_scratch, oop); + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), exact_log2(JVM_ACC_IS_BOX_CLASS)); + z_btrue(done); + } + if (try_bias) { biased_locking_enter(oop, displacedHeader, temp, Z_R0, done); } From coleen.phillimore at oracle.com Tue Aug 4 14:49:54 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 4 Aug 2020 10:49:54 -0400 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: References: Message-ID: This is sort of massive, so I'll only make a couple initial comments. http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html All this new code in ClassLoaderData should go somewhere else, like maybe classLoaderDataShared.hpp/cpp ?? I don't see how it needs to be owned by CLD, except maybe the serialized function. + if (loader_data) { There are many checks that should be != NULL here. http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/javaClasses.cpp.udiff.html I don't see why compute_offset/s() has been exported outside javaClasses.cpp. http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/moduleEntry.cpp.udiff.html http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/packageEntry.cpp.udiff.html These look like the same code to me.? Maybe they can be generalized and put together in a moduleEntryShared.cpp/hpp ? http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/systemDictionary.cpp.udiff.html Why do you register the loaders here? I don't know enough about the Java code to review that. I assume that this going to be rebased on other changes, so this is only an incomplete review. Coleen On 7/22/20 3:36 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8244778 > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/ > > > Please review this patch that stores the full module graph in the CDS > archive heap. This reduces the initialization time of the basic JVM by > about 22%: > > $ perf stat -r 100 bin/java -version > before: 98,219,329 instructions 0.03971 secs elapsed (+- 0.44%) > after:? 55,835,815 instructions 0.03109 secs elapsed (+- 0.65%) > > [1] Start with ModuleBootstrap.java. The current implementation is > ??? quite restrictive: the archived module graph is used only when no > ??? module options are specified. > > ??? See ModuleBootstrap.mayUseArchivedBootLayer(). > > ??? We can probably support options such as main module and module path > ??? in a future RFE. > > [2] In the current JDK implementation, there is no single object > ??? that represents "the module graph". Most of the information > ??? is stored in the archive bootLayer object, but a few additional > ??? restoration operations need to be performed: > > ??? + See ModuleBootstrap.getArchivedBootLayer() > ??? + Some static fields need to be archived/restored in > ????? Module.java, BuiltinClassLoader.java, ClassLoaders.java > ????? and BootLoader.java > > [3] I ran into a complication with two loader instances of > ??? PlatformClassLoader and AppClassLoader. They are stored in > ??? multiple tables inside the module graph (e.g., > ??? BuiltinClassLoader$LoadedModule) so I cannot easily recreate > ??? them at runtime. > > ??? However, these two loaders contain information specific to the > ??? dump time VM lifecycle (such as the classes that were loaded > ??? during CDS dumping) that need to be scrubbed. I couldn't find an > ??? elegant way of doing this, so I added a private "resetArchivedStates" > ??? method to a few classes. They are called inside > ??? HeapShared::reset_archived_object_states(). > > [4] Related native data structures (PackageEntry and ModuleEntry) > ??? are also archived. Start with classLoaderData.cpp > > Passes mach5 tiers 1-4. I will test with additional tiers. > > Thanks > - Ioi From jiefu at tencent.com Tue Aug 4 14:58:30 2020 From: jiefu at tencent.com (=?gb2312?B?amllZnUouLW93Ck=?=) Date: Tue, 4 Aug 2020 14:58:30 +0000 Subject: RFR: 8251031: Some vmTestbase/nsk/monitoring/RuntimeMXBean tests fail with hostnames starting from digits(Internet mail) In-Reply-To: <5a9156f2-6d03-129c-ecaf-aacb21e4a47e@oracle.com> References: <8A8E8751-AF91-471A-B668-5D6461B72356@tencent.com>, <5a9156f2-6d03-129c-ecaf-aacb21e4a47e@oracle.com> Message-ID: Thanks David for your review. Best regards, Jie ________________________________ From: David Holmes Sent: Tuesday, August 4, 2020 6:51 PM To: jiefu(??); hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR: 8251031: Some vmTestbase/nsk/monitoring/RuntimeMXBean tests fail with hostnames starting from digits(Internet mail) Hi Jie, On 4/08/2020 7:08 pm, jiefu(??) wrote: > Hi all, > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251031 > Webrev: http://cr.openjdk.java.net/~jiefu/8251031/webrev.00/ > > Some vmTestbase/nsk/monitoring/RuntimeMXBean tests failed in our test infrastructure. > The reason is that these tests reject hostnames starting with digits. > > However, hostnames starting from digits are actually valid according to RFC1123 [1][2]. > It would be better to fix it. I don't process regular expressions very well but I can see the addition of the 0-9 character class in the expression so that appears fine. :) What I don't understand is why we aren't simply splitting the string at '@' and taking the pid and hostname directly. (That's somewhat rhetorical I don't expect you to change the test.) Thanks, David ----- > Thanks a lot. > Best regards, > Jie > > [1] https://tools.ietf.org/html/rfc1123#page-13 > [2] https://en.wikipedia.org/wiki/Hostname > From matthias.baesken at sap.com Tue Aug 4 15:02:16 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 4 Aug 2020 15:02:16 +0000 Subject: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> Message-ID: Hi Suenaga, sorry for answering very late . I did not see any jck / jtreg errors in our internal nightly tests. I also checked the hserr on another Hyper-V guest and still get the expected " HyperV virtualization detected" (which is good ?? ). However on a host, I would still prefer to get some output (probably different than the guest output now) . Regarding the usage of #pragma comment(lib, "wbemuuid.lib") This seem to add an additional lib dependency , should some people from build/deploy area comment on this maybe ? (not saying it is a very bad thing ) Best regards, Matthias -----Original Message----- From: Yasumasa Suenaga Sent: Samstag, 1. August 2020 01:04 To: Baesken, Matthias ; hotspot-runtime-dev at openjdk.java.net Cc: David Holmes Subject: Re: RFR: 8250598: Hyper-V is detected in spite of running on host OS Hi Matthias, Have you got the result from your internal nightly build / test? Also all comments are welcome. Thanks, Yasumasa On 2020/07/27 21:21, Yasumasa Suenaga wrote: > On 2020/07/27 19:36, Baesken, Matthias wrote: >> Hi Yasumasa, I put your patch into our internal? nightly? build/test? queue . > > Thanks! > >>> ?? - Hyper-V is detected on Windows in spite of running on host OS >> >> So it is in your case a host that? runs? Hyper-V guests.? And the host? reports too? "HyperV virtualization detected" . >> Do I get you? right ? > > Yes, that's right! > >> Maybe we should be more specific in this case (however I did not see much problem so far with the? current output ). >> >>> ?? - Call CPUID with other than EAX = 40000000h (it is not described in the spec [1]) >> >> I think we had? some virtualizations that needed checking? CPUID with other than EAX = 40000000h? for detection, >> ? But might be these were old or buggy versions , cannot find much details about it atm . > > As David shows, virt-what checks CPUID with other than EAX = 40000000h, but at least, current supported hypervisor (VMware, Hyper-V, KVM, Xen) do not seem to need to do it. > >>> ?? - Does not check CPUID hypervisor present bit [1] >> >> Yes, this could/should indeed be added? ( we even did this in our internal JVM a while ago). >> >>> ?? - Does not support x86 (32bit) platform >> >> This is true , it was not added because?? I think 32bit support has not much importance any more in current jdk . >> However in case the new version does? support 32bit too, it is for sure a good thing! > > I found out "TODO support 32 bit" in current code, so I attempt to fix it :) > > > Thanks, > > Yasumasa > > >> Best regards, Matthias >> >> >> >> -----Original Message----- >> From: Yasumasa Suenaga >> Sent: Montag, 27. Juli 2020 06:25 >> To: hotspot-runtime-dev at openjdk.java.net; Baesken, Matthias >> Cc: David Holmes >> Subject: RFR: 8250598: Hyper-V is detected in spite of running on host OS >> >> Hi all, >> >> Please review this change: >> >> ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8250598 >> ??? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.00/ >> >> When I got hs_err log on Windows, I saw "HyperV virtualization detected" in it in spite of running on host OS. >> >> Hypervisor detector has been introduced in JDK-8219241, but it has some problems as below: >> >> ??? - Hyper-V is detected on Windows in spite of running on host OS >> ??? - Call CPUID with other than EAX = 40000000h (it is not described in the spec [1]) >> ??? - Does not check CPUID hypervisor present bit [1] >> ??? - Does not support x86 (32bit) platform >> >> I've tested this change on submit repo, and have checked output from VM.info jcmd on following environment: >> >> ??? - Windows x64 (host) >> ??? - Windows x64 (Hyper-V guest) >> ??? - Fedora32 x64 (Hyper-V guest) >> ??? - 32 bit JDK on Fedora32 x64 (Hyper-V guest) >> >> >> Thanks, >> >> Yasumasa >> >> >> [1] https://kb.vmware.com/s/article/1009458 >> From luhenry at microsoft.com Tue Aug 4 15:34:22 2020 From: luhenry at microsoft.com (Ludovic Henry) Date: Tue, 4 Aug 2020 15:34:22 +0000 Subject: RFR(S): Use Vectored Exception Handling on Windows In-Reply-To: References: , , Message-ID: Hi, An update on my current exploration is long overdue. # On the use of Unhandled Exception Handling. Unfortunately, this is a dead-end because UEH itself relies on Structured Exception Handling to catch any uncaught exception at the top-most frame, and then needs a properly unwindable stack. This is not the case for Java frames as Hotspot follows its own internal ABI, making it impossible for the native unwinder (which doesn't know about the Java ABI) to walk the stack frames. The easy fallback is simply to keep the existing `__try {} __except (topLevelExceptionFilter(...)) {}` and call to `RtlAddFunctionTable` to add such try/except block to the code cache. # On the use of Vectored Exception Handling Despite UEH not being usable in our case, VEH is still helpful in getting Hotspot to treat Hotspot related exceptions first and to separate the normal exception handling from the crash handling. I'll post an updated webrev with these changes later today. Thank you, Ludovic From patricio.chilano.mateo at oracle.com Tue Aug 4 15:47:10 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Tue, 4 Aug 2020 12:47:10 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> Message-ID: Hi Martin, Thanks for fixing PPC and taking care of s390! On 8/4/20 11:18 AM, Doerr, Martin wrote: > Hi Patricio, > > I suggest to use movl + testl for checking the access flag as for other access flags on x64. Ok, I'll change it to movl + testl and test it out before sending v3. > New version for PPC64 and s390 see below. > > The test SyncOnPrimitiveWrapperTest produces hs_err files as expected. However, I'm getting timeout issues: > timed out (timeout set to 120000ms, elapsed time including timeout handling was 122372ms) > Can we provide more time? I see that some tests use @run driver/timeout=xxxx. Maybe you can specify that and see if that fixes it? Let me know if that works and I can add it to the test. Thanks, Patricio > Best regards, > Martin > > > diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp > --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 10:03:57 2020 +0200 > +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 16:04:31 2020 +0200 > @@ -107,8 +107,8 @@ > > if (DiagnoseSyncOnPrimitiveWrappers != 0) { > load_klass(Rscratch, Roop); > - ld(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); > - andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); > + lwz(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); > + testbitdi(CCR0, R0, Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS)); > bne(CCR0, slow_case); > } > > diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp > --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 10:03:57 2020 +0200 > +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 16:04:31 2020 +0200 > @@ -912,8 +912,8 @@ > > if (DiagnoseSyncOnPrimitiveWrappers != 0) { > load_klass(tmp, object); > - ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); > - andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); > + lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); > + testbitdi(CCR0, R0, tmp, exact_log2(JVM_ACC_IS_BOX_CLASS)); > bne(CCR0, slow_case); > } > > diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp > --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 10:03:57 2020 +0200 > +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 16:04:31 2020 +0200 > @@ -2838,8 +2838,8 @@ > > if (DiagnoseSyncOnPrimitiveWrappers != 0) { > load_klass(temp, oop); > - ld(temp, in_bytes(Klass::access_flags_offset()), temp); > - andi(temp, temp, JVM_ACC_IS_BOX_CLASS); > + lwz(temp, in_bytes(Klass::access_flags_offset()), temp); > + testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); > bne(CCR0, cont); > } > > diff -r 77852e129401 src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp > --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 10:03:57 2020 +0200 > +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 16:04:31 2020 +0200 > @@ -91,6 +91,12 @@ > // Save object being locked into the BasicObjectLock... > z_stg(obj, Address(disp_hdr, BasicObjectLock::obj_offset_in_bytes())); > > + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > + load_klass(Z_R1_scratch, obj); > + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), exact_log2(JVM_ACC_IS_BOX_CLASS)); > + z_btrue(slow_case); > + } > + > if (UseBiasedLocking) { > biased_locking_enter(obj, hdr, Z_R1_scratch, Z_R0_scratch, done, &slow_case); > } > diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp > --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 10:03:57 2020 +0200 > +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 16:04:31 2020 +0200 > @@ -1000,6 +1000,12 @@ > // Load markWord from object into displaced_header. > z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), object); > > + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > + load_klass(Z_R1_scratch, object); > + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), exact_log2(JVM_ACC_IS_BOX_CLASS)); > + z_btrue(slow_case); > + } > + > if (UseBiasedLocking) { > biased_locking_enter(object, displaced_header, Z_R1, Z_R0, done, &slow_case); > } > diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp > --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 10:03:57 2020 +0200 > +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 16:04:31 2020 +0200 > @@ -3358,6 +3358,12 @@ > // Load markWord from oop into mark. > z_lg(displacedHeader, 0, oop); > > + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > + load_klass(Z_R1_scratch, oop); > + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), exact_log2(JVM_ACC_IS_BOX_CLASS)); > + z_btrue(done); > + } > + > if (try_bias) { > biased_locking_enter(oop, displacedHeader, temp, Z_R0, done); > } > From martin.doerr at sap.com Tue Aug 4 16:35:59 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 4 Aug 2020 16:35:59 +0000 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> Message-ID: Hi Patricio, > Ok, I'll change it to movl + testl and test it out before sending v3. Thanks. I forgot to mention arm + aarch64. Aarch64 uses ldrw + tbnz. Arm 32 bit uses ldr_u32 + tbnz. I remember having seen the same mistake ?? And nobody noticed it on little Endian platforms. > I see that some tests use @run driver/timeout=xxxx. Maybe you can > specify that and see if that fixes it? Let me know if that works and I > can add it to the test. That seems to have an effect. But now I'm not patient enough to wait for the test to finish. Maybe the problem is that I'm using slow debug builds. But is there a chance to make the test quicker without losing coverage significantly? I think the test is too slow this way to run it on a regular basis. We'd need to spend dedicated machines for it. Best regards, Martin > -----Original Message----- > From: Patricio Chilano > Sent: Dienstag, 4. August 2020 17:47 > To: Doerr, Martin ; David Holmes > ; hotspot-runtime-dev at openjdk.java.net; > hotspot-jfr-dev at openjdk.java.net > Subject: Re: RFR 8242263: Diagnose synchronization on primitive wrappers > > Hi Martin, > > Thanks for fixing PPC and taking care of s390! > > On 8/4/20 11:18 AM, Doerr, Martin wrote: > > Hi Patricio, > > > > I suggest to use movl + testl for checking the access flag as for other access > flags on x64. > Ok, I'll change it to movl + testl and test it out before sending v3. > > > New version for PPC64 and s390 see below. > > > > The test SyncOnPrimitiveWrapperTest produces hs_err files as expected. > However, I'm getting timeout issues: > > timed out (timeout set to 120000ms, elapsed time including timeout > handling was 122372ms) > > Can we provide more time? > I see that some tests use @run driver/timeout=xxxx. Maybe you can > specify that and see if that fixes it? Let me know if that works and I > can add it to the test. > > Thanks, > Patricio > > Best regards, > > Martin > > > > > > diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp > > --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 > 10:03:57 2020 +0200 > > +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 > 16:04:31 2020 +0200 > > @@ -107,8 +107,8 @@ > > > > if (DiagnoseSyncOnPrimitiveWrappers != 0) { > > load_klass(Rscratch, Roop); > > - ld(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); > > - andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); > > + lwz(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); > > + testbitdi(CCR0, R0, Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS)); > > bne(CCR0, slow_case); > > } > > > > diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp > > --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 > 10:03:57 2020 +0200 > > +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 > 16:04:31 2020 +0200 > > @@ -912,8 +912,8 @@ > > > > if (DiagnoseSyncOnPrimitiveWrappers != 0) { > > load_klass(tmp, object); > > - ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); > > - andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); > > + lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); > > + testbitdi(CCR0, R0, tmp, exact_log2(JVM_ACC_IS_BOX_CLASS)); > > bne(CCR0, slow_case); > > } > > > > diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp > > --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 > 10:03:57 2020 +0200 > > +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 > 16:04:31 2020 +0200 > > @@ -2838,8 +2838,8 @@ > > > > if (DiagnoseSyncOnPrimitiveWrappers != 0) { > > load_klass(temp, oop); > > - ld(temp, in_bytes(Klass::access_flags_offset()), temp); > > - andi(temp, temp, JVM_ACC_IS_BOX_CLASS); > > + lwz(temp, in_bytes(Klass::access_flags_offset()), temp); > > + testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); > > bne(CCR0, cont); > > } > > > > diff -r 77852e129401 src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp > > --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 > 10:03:57 2020 +0200 > > +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 > 16:04:31 2020 +0200 > > @@ -91,6 +91,12 @@ > > // Save object being locked into the BasicObjectLock... > > z_stg(obj, Address(disp_hdr, BasicObjectLock::obj_offset_in_bytes())); > > > > + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > > + load_klass(Z_R1_scratch, obj); > > + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), > exact_log2(JVM_ACC_IS_BOX_CLASS)); > > + z_btrue(slow_case); > > + } > > + > > if (UseBiasedLocking) { > > biased_locking_enter(obj, hdr, Z_R1_scratch, Z_R0_scratch, done, > &slow_case); > > } > > diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp > > --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 10:03:57 > 2020 +0200 > > +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 16:04:31 > 2020 +0200 > > @@ -1000,6 +1000,12 @@ > > // Load markWord from object into displaced_header. > > z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), object); > > > > + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > > + load_klass(Z_R1_scratch, object); > > + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), > exact_log2(JVM_ACC_IS_BOX_CLASS)); > > + z_btrue(slow_case); > > + } > > + > > if (UseBiasedLocking) { > > biased_locking_enter(object, displaced_header, Z_R1, Z_R0, done, > &slow_case); > > } > > diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp > > --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 > 10:03:57 2020 +0200 > > +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 > 16:04:31 2020 +0200 > > @@ -3358,6 +3358,12 @@ > > // Load markWord from oop into mark. > > z_lg(displacedHeader, 0, oop); > > > > + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > > + load_klass(Z_R1_scratch, oop); > > + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), > exact_log2(JVM_ACC_IS_BOX_CLASS)); > > + z_btrue(done); > > + } > > + > > if (try_bias) { > > biased_locking_enter(oop, displacedHeader, temp, Z_R0, done); > > } > > From gerard.ziemski at oracle.com Tue Aug 4 17:02:02 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Tue, 4 Aug 2020 12:02:02 -0500 Subject: RFR: 8250636: iso8601_time returns incorrect offset part on MacOS In-Reply-To: References: <777fc30a-45ef-aa45-8f04-0bbe80c5a83d@azul.com> <0e1ed1ae-2011-2e4a-4f58-d99b57ede74c@azul.com> <934a8a40-dd51-512f-f010-909ed312783d@oracle.com> Message-ID: Looks great! On 8/4/20 4:57 AM, Dmitry Cherepanov wrote: > Hi David, > > On 03.08.2020 10:53, David Holmes wrote: >> Hi Dmitry, >> >> On 31/07/2020 8:29 pm, Dmitry Cherepanov wrote: >>> Hi Gerard, >>> >>> Thanks for reviewing this, moving the logic of get_timezone to >>> iso8601_time looks good to me too. >>> Here's an updated webrev >>> http://cr.openjdk.java.net/~dcherepanov/8250636/webrev.v4/ >> This version also looks good to me. But now I see all the code clearly in one place I have to wonder why we are not using tm_gmtoff on all platforms except Windows? That would simplify the logic even further. >> >> That goes beyond fixing the current actual bug though so I'll leave that choice to you. > Thanks for the suggestion. This change would not work on Solaris. But given the removal of Solaris port, I also see no other reasons why not to use tm_gmtoff on platforms except Windows. > > Updated webrev: http://cr.openjdk.java.net/~dcherepanov/8250636/webrev.v5/ > Jdk-submit testing passed with this change > > Thanks, > > Dmitry > From gerard.ziemski at oracle.com Tue Aug 4 17:03:53 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Tue, 4 Aug 2020 12:03:53 -0500 Subject: PING Re: RFR (S) 8237591: Mac: include OS X version in hs_err_pid crash log file In-Reply-To: References: <49AB6201-C2C2-4862-A019-B60EEE44E515@me.com> <74c08f37-673c-84ae-a512-6f5afbe08050@oracle.com> <70bb6f74-e626-bd54-ddf0-568bebe933e9@oracle.com> <56bb935f-1277-e97e-aeb2-3f488bc825a7@oracle.com> Message-ID: hi all, Can I please have someone 2nd review it? cheers On 7/30/20 11:54 AM, gerard ziemski wrote: > Thank you for the review. > > May I please have a 2nd reviewer? (or would this be considered trivial?) > > > cheers > > On 7/30/20 1:10 AM, David Holmes wrote: >> On 30/07/2020 4:51 am, gerard ziemski wrote: >>> On 7/27/20 6:21 PM, David Holmes wrote: >>>> On 28/07/2020 2:12 am, gerard ziemski wrote: >>>>> Thank you David for taking a look. >>>>> >>>>> >>>>> On 7/19/20 11:37 PM, David Holmes wrote: >>>>>> Hi Gerard, >>>>>> >>>>>> On 18/07/2020 5:19 am, gerard ziemski wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> Please review this small fix that adds the OS version and the OS >>>>>>> build number to the hs_err_pidXXX.log output in the ?Summary? >>>>>>> section for Mac platform (it?s easier to use for developers than >>>>>>> the Darwin kernel version that we display right now). >>>>>>> >>>>>>> This is how things used to look: >>>>>>> >>>>>>> >>>>>>> --------------- S U M M A R Y ------------ >>>>>>> >>>>>>> Command Line: Crasher >>>>>>> >>>>>>> Host: Gerards-MBP-16, MacBookPro16,1 x86_64 2600 MHz, 12 cores, >>>>>>> 32G, Darwin 19.5.0 >>>>>>> Time: Thu Jul 16 14:01:46 2020 CDT elapsed time: 1.089465 >>>>>>> seconds (0d 0h 0m 1s) >>>>>>> >>>>>>> >>>>>>> And this is how the ?Summary? section looks like with the >>>>>>> proposed change: >>>>>>> >>>>>>> >>>>>>> --------------- S U M M A R Y ------------ >>>>>>> >>>>>>> Command Line: Crasher >>>>>>> >>>>>>> Host: Gerards-MBP-16, MacBookPro16,1 x86_64 2600 MHz, 12 cores, >>>>>>> 32G, Darwin 19.5.0, macOS 10.15.5 (19F101) >>>>>>> Time: Thu Jul 16 14:02:29 2020 CDT elapsed time: 0.360881 >>>>>>> seconds (0d 0h 0m 0s) >>>>>>> >>>>>>> >>>>>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8237591 >>>>>>> open webrev at http://cr.openjdk.java.net/~gziemski/8237591_rev1 >>>>>>> testing Mach5 hs_tier1,2,3,4,5 in progress >>>>>> >>>>>> Just to be clear, the changes prior to: >>>>>> >>>>>> 1555 #ifdef __APPLE__ >>>>>> >>>>>> are just fixing up existing indentation errors - correct? >>>>> >>>>> Yes, hope that's OK, as this was the only spot in the function >>>>> that stood out with inconsistent indentation. >>>> >>>> Yes that is fine. >>>> >>>>>> >>>>>> The actual change seems okay, just one query: >>>>>> >>>>>> 1562???? int mib_build[] = { CTL_KERN, KERN_OSVERSION }; >>>>>> >>>>>> I couldn't find KERN_OSVERSION documented for sysctl - is it a >>>>>> "recent" addition? >>>>> >>>>> Yes it is. Apple added it back in 2018 (see bug comments or this >>>>> link >>>>> https://github.com/apple/darwin-xnu/commit/5bbb823c13f3ab1ab58878f96b35433a29882676?diff=split#diff-6651b0c84a045f400bc45faa9f61c9e1 >>>>> ) >>>> >>>> That link shows the addition of sysctl_osproductversion which I >>>> assume underpins "kern.osproductversion". But my question was on >>>> KERN_OSVERSION. That definition seems to already exist prior to the >>>> change you link. My concern is whether it was also fairly recently >>>> introduced and so referring to it would require a minimum macOS >>>> version on the build machine? >>> >>> Sorry, I thought you meant "kern.osproductversion", not >>> KERN_OSVERSION, but that's a valid question. >>> >>> I found Apple using KERN_OSVERSION in its own code since macOS 10.7, >>> i.e. >>> https://opensource.apple.com/source/Libc/Libc-763.11/gen/assumes.c.auto.html >>> , though I could not find any documentation of it either. >> >> I found it in sysctl.h from 10.5 dev kit as well, so that looks fine >> to use. >> >> Thanks for checking. >> David >> ----- > From calvin.cheung at oracle.com Tue Aug 4 17:06:11 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 4 Aug 2020 10:06:11 -0700 Subject: RFR(T) 8249586: Test runtime/cds/appcds/DirClasspathTest.java will fail if run twice Message-ID: <2a1fd752-a851-359f-ad3f-471119b74fee@oracle.com> bug: https://bugs.openjdk.java.net/browse/JDK-8249586 webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8249586/webrev.00/ A trivial fix by adding the StandardCopyOption.REPLACE_EXISTING option when calling Files.copy(). Tested by running the jtreg test multiple times in the same directory. thanks, Calvin From patricio.chilano.mateo at oracle.com Tue Aug 4 18:25:34 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Tue, 4 Aug 2020 15:25:34 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> Message-ID: <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> Hi Martin, On 8/4/20 1:35 PM, Doerr, Martin wrote: > Hi Patricio, > >> Ok, I'll change it to movl + testl and test it out before sending v3. > Thanks. I forgot to mention arm + aarch64. > Aarch64 uses ldrw + tbnz. > Arm 32 bit uses ldr_u32 + tbnz. > > I remember having seen the same mistake ?? > And nobody noticed it on little Endian platforms. Ok, I can use a tbnz instead of test and then a branch on arm32 and aarch64. Shouldn't a normal ldr in arm32 work fine? Also in 64 bits (either x64 or aarch64) I don't see the issue of using a 64 bit load, besides the fact that we only care about the first 32 bits. Regardless of the endianness, aren't we masking out the upper part when we do AND/TEST or if we test a bit in the 0-31 bit range? Otherwise it seems it should have failed for one of those platforms in my tests. >> I see that some tests use @run driver/timeout=xxxx. Maybe you can >> specify that and see if that fixes it? Let me know if that works and I >> can add it to the test. > That seems to have an effect. But now I'm not patient enough to wait for the test to finish. > Maybe the problem is that I'm using slow debug builds. > But is there a chance to make the test quicker without losing coverage significantly? > I think the test is too slow this way to run it on a regular basis. We'd need to spend dedicated machines for it. We could remove the nested synchronized statements in the run() method so that we don't generate that much logging. We could also lower LOOP_COUNT. I think the issue is also because we are running LogTest with multiple flag combinations. Not sure what we should touch first. Maybe the synchronized statements, have only one or two and test that first? Thanks, Patricio > Best regards, > Martin > > >> -----Original Message----- >> From: Patricio Chilano >> Sent: Dienstag, 4. August 2020 17:47 >> To: Doerr, Martin ; David Holmes >> ; hotspot-runtime-dev at openjdk.java.net; >> hotspot-jfr-dev at openjdk.java.net >> Subject: Re: RFR 8242263: Diagnose synchronization on primitive wrappers >> >> Hi Martin, >> >> Thanks for fixing PPC and taking care of s390! >> >> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>> Hi Patricio, >>> >>> I suggest to use movl + testl for checking the access flag as for other access >> flags on x64. >> Ok, I'll change it to movl + testl and test it out before sending v3. >> >>> New version for PPC64 and s390 see below. >>> >>> The test SyncOnPrimitiveWrapperTest produces hs_err files as expected. >> However, I'm getting timeout issues: >>> timed out (timeout set to 120000ms, elapsed time including timeout >> handling was 122372ms) >>> Can we provide more time? >> I see that some tests use @run driver/timeout=xxxx. Maybe you can >> specify that and see if that fixes it? Let me know if that works and I >> can add it to the test. >> >> Thanks, >> Patricio >>> Best regards, >>> Martin >>> >>> >>> diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >> 10:03:57 2020 +0200 >>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >> 16:04:31 2020 +0200 >>> @@ -107,8 +107,8 @@ >>> >>> if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>> load_klass(Rscratch, Roop); >>> - ld(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); >>> - andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>> + lwz(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); >>> + testbitdi(CCR0, R0, Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>> bne(CCR0, slow_case); >>> } >>> >>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >> 10:03:57 2020 +0200 >>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >> 16:04:31 2020 +0200 >>> @@ -912,8 +912,8 @@ >>> >>> if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>> load_klass(tmp, object); >>> - ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>> - andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>> + lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>> + testbitdi(CCR0, R0, tmp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>> bne(CCR0, slow_case); >>> } >>> >>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >> 10:03:57 2020 +0200 >>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >> 16:04:31 2020 +0200 >>> @@ -2838,8 +2838,8 @@ >>> >>> if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>> load_klass(temp, oop); >>> - ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>> - andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>> + lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>> + testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>> bne(CCR0, cont); >>> } >>> >>> diff -r 77852e129401 src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >> 10:03:57 2020 +0200 >>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >> 16:04:31 2020 +0200 >>> @@ -91,6 +91,12 @@ >>> // Save object being locked into the BasicObjectLock... >>> z_stg(obj, Address(disp_hdr, BasicObjectLock::obj_offset_in_bytes())); >>> >>> + if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>> + load_klass(Z_R1_scratch, obj); >>> + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>> + z_btrue(slow_case); >>> + } >>> + >>> if (UseBiasedLocking) { >>> biased_locking_enter(obj, hdr, Z_R1_scratch, Z_R0_scratch, done, >> &slow_case); >>> } >>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 10:03:57 >> 2020 +0200 >>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 16:04:31 >> 2020 +0200 >>> @@ -1000,6 +1000,12 @@ >>> // Load markWord from object into displaced_header. >>> z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), object); >>> >>> + if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>> + load_klass(Z_R1_scratch, object); >>> + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>> + z_btrue(slow_case); >>> + } >>> + >>> if (UseBiasedLocking) { >>> biased_locking_enter(object, displaced_header, Z_R1, Z_R0, done, >> &slow_case); >>> } >>> diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp >>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >> 10:03:57 2020 +0200 >>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >> 16:04:31 2020 +0200 >>> @@ -3358,6 +3358,12 @@ >>> // Load markWord from oop into mark. >>> z_lg(displacedHeader, 0, oop); >>> >>> + if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>> + load_klass(Z_R1_scratch, oop); >>> + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>> + z_btrue(done); >>> + } >>> + >>> if (try_bias) { >>> biased_locking_enter(oop, displacedHeader, temp, Z_R0, done); >>> } >>> From ioi.lam at oracle.com Tue Aug 4 18:42:39 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 4 Aug 2020 11:42:39 -0700 Subject: RFR(T) 8249586: Test runtime/cds/appcds/DirClasspathTest.java will fail if run twice In-Reply-To: <2a1fd752-a851-359f-ad3f-471119b74fee@oracle.com> References: <2a1fd752-a851-359f-ad3f-471119b74fee@oracle.com> Message-ID: <4ebd0ce7-831d-33fd-b207-0425898b1849@oracle.com> Hi Calvin, Looks good, and trivial. Thanks - Ioi On 8/4/20 10:06 AM, Calvin Cheung wrote: > bug: https://bugs.openjdk.java.net/browse/JDK-8249586 > > webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8249586/webrev.00/ > > A trivial fix by adding the StandardCopyOption.REPLACE_EXISTING option > when calling Files.copy(). > > Tested by running the jtreg test multiple times in the same directory. > > thanks, > > Calvin > From harold.seigel at oracle.com Tue Aug 4 19:16:38 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 4 Aug 2020 15:16:38 -0400 Subject: RFR 8139875: [TESTBUG] Improve nsk/stress/stack/* tests In-Reply-To: <262e8bd2-5759-0b88-4977-233be3161739@oracle.com> References: <62a030ca-99e1-1b5c-ade7-e8685aa9f8e3@oracle.com> <262e8bd2-5759-0b88-4977-233be3161739@oracle.com> Message-ID: <0a1a7076-b9f8-37ca-3c53-a4afeca1f406@oracle.com> Hi David, Thanks for the suggestion.? Please review this updated webrev that uses -Xss200K for all of the tests. http://cr.openjdk.java.net/~hseigel/bug_8139875.2/webrev/index.html Thanks, Harold On 8/3/2020 7:04 PM, David Holmes wrote: > Hi Harold, > > On 4/08/2020 4:11 am, Harold Seigel wrote: >> Hi, >> >> Please review these small changes to reduce the amount of time >> required by some of the nsk/stress/stack tests in order to prevent >> them from timing out.? The changes include using -Xss to run the >> tests with smaller stacks, prevent running with -Xcomp, and, for test >> stack019, reduce the number of test iterations. > > Is there a reason to use a range of -Xss values in different tests > instead of just, say, using -Xss200K in all of them and reducing the > execution time further. > > Thanks, > David > ----- > >> Test stack019 was excluded from windows targets because, even with >> the reduced iterations, the test was running for > 48 minutes. >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8139875/webrev/index.html >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8139875 >> >> The tests were tested by running them in Mach5 tier5 and tier7 and >> looking at their execution time. >> >> Thanks, Harold >> From calvin.cheung at oracle.com Tue Aug 4 19:18:37 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Tue, 4 Aug 2020 12:18:37 -0700 Subject: RFR(T) 8249586: Test runtime/cds/appcds/DirClasspathTest.java will fail if run twice In-Reply-To: <4ebd0ce7-831d-33fd-b207-0425898b1849@oracle.com> References: <2a1fd752-a851-359f-ad3f-471119b74fee@oracle.com> <4ebd0ce7-831d-33fd-b207-0425898b1849@oracle.com> Message-ID: <6e02247d-b946-eabc-753a-4faf7350d89e@oracle.com> Thanks, Ioi. I've pushed the change. Calvin On 8/4/20 11:42 AM, Ioi Lam wrote: > Hi Calvin, > > Looks good, and trivial. > > Thanks > - Ioi > > On 8/4/20 10:06 AM, Calvin Cheung wrote: >> bug: https://bugs.openjdk.java.net/browse/JDK-8249586 >> >> webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8249586/webrev.00/ >> >> A trivial fix by adding the StandardCopyOption.REPLACE_EXISTING >> option when calling Files.copy(). >> >> Tested by running the jtreg test multiple times in the same directory. >> >> thanks, >> >> Calvin >> > From luhenry at microsoft.com Tue Aug 4 21:03:10 2020 From: luhenry at microsoft.com (Ludovic Henry) Date: Tue, 4 Aug 2020 21:03:10 +0000 Subject: RFR(S): 8247941: Use Vectored Exception Handling on Windows Message-ID: (Renaming subject to include bug id, original discussion at [1]). Webrev: http://cr.openjdk.java.net/~luhenry/8247941/webrev.00/ JBS: https://bugs.openjdk.java.net/browse/JDK-8247941 I've uploaded the webrev at the link above. [1] https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040228.html -----Original Message----- From: hotspot-runtime-dev On Behalf Of Ludovic Henry Sent: Tuesday, August 4, 2020 8:34 AM To: Thomas St?fe ; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(S): Use Vectored Exception Handling on Windows Hi, An update on my current exploration is long overdue. # On the use of Unhandled Exception Handling. Unfortunately, this is a dead-end because UEH itself relies on Structured Exception Handling to catch any uncaught exception at the top-most frame, and then needs a properly unwindable stack. This is not the case for Java frames as Hotspot follows its own internal ABI, making it impossible for the native unwinder (which doesn't know about the Java ABI) to walk the stack frames. The easy fallback is simply to keep the existing `__try {} __except (topLevelExceptionFilter(...)) {}` and call to `RtlAddFunctionTable` to add such try/except block to the code cache. # On the use of Vectored Exception Handling Despite UEH not being usable in our case, VEH is still helpful in getting Hotspot to treat Hotspot related exceptions first and to separate the normal exception handling from the crash handling. I'll post an updated webrev with these changes later today. Thank you, Ludovic From suenaga at oss.nttdata.com Wed Aug 5 00:14:29 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 5 Aug 2020 09:14:29 +0900 Subject: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> Message-ID: <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> Hi Matthias, On 2020/08/05 0:02, Baesken, Matthias wrote: > Hi Suenaga, sorry for answering very late . > I did not see any jck / jtreg errors in our internal nightly tests. Thanks! > I also checked the hserr on another Hyper-V guest and still get the expected " HyperV virtualization detected" (which is good ?? ). > > > However on a host, I would still prefer to get some output (probably different than the guest output now) . What message do you expect? How about "Hyper-V role detected" ? > Regarding the usage of > > #pragma comment(lib, "wbemuuid.lib") > > This seem to add an additional lib dependency , should some people from build/deploy area comment on this maybe ? > (not saying it is a very bad thing ) We add wbemuuid.lib to make/autoconf/libraries.m4 . I will change it in next webrev, and I will CCing to build-dev on next turn. Thanks, Yasumasa > Best regards, Matthias > > > > -----Original Message----- > From: Yasumasa Suenaga > Sent: Samstag, 1. August 2020 01:04 > To: Baesken, Matthias ; hotspot-runtime-dev at openjdk.java.net > Cc: David Holmes > Subject: Re: RFR: 8250598: Hyper-V is detected in spite of running on host OS > > Hi Matthias, > > Have you got the result from your internal nightly build / test? > Also all comments are welcome. > > > Thanks, > > Yasumasa > > > On 2020/07/27 21:21, Yasumasa Suenaga wrote: >> On 2020/07/27 19:36, Baesken, Matthias wrote: >>> Hi Yasumasa, I put your patch into our internal? nightly? build/test? queue . >> >> Thanks! >> >>>> ?? - Hyper-V is detected on Windows in spite of running on host OS >>> >>> So it is in your case a host that? runs? Hyper-V guests.? And the host? reports too? "HyperV virtualization detected" . >>> Do I get you? right ? >> >> Yes, that's right! >> >>> Maybe we should be more specific in this case (however I did not see much problem so far with the? current output ). >>> >>>> ?? - Call CPUID with other than EAX = 40000000h (it is not described in the spec [1]) >>> >>> I think we had? some virtualizations that needed checking? CPUID with other than EAX = 40000000h? for detection, >>> ? But might be these were old or buggy versions , cannot find much details about it atm . >> >> As David shows, virt-what checks CPUID with other than EAX = 40000000h, but at least, current supported hypervisor (VMware, Hyper-V, KVM, Xen) do not seem to need to do it. >> >>>> ?? - Does not check CPUID hypervisor present bit [1] >>> >>> Yes, this could/should indeed be added? ( we even did this in our internal JVM a while ago). >>> >>>> ?? - Does not support x86 (32bit) platform >>> >>> This is true , it was not added because?? I think 32bit support has not much importance any more in current jdk . >>> However in case the new version does? support 32bit too, it is for sure a good thing! >> >> I found out "TODO support 32 bit" in current code, so I attempt to fix it :) >> >> >> Thanks, >> >> Yasumasa >> >> >>> Best regards, Matthias >>> >>> >>> >>> -----Original Message----- >>> From: Yasumasa Suenaga >>> Sent: Montag, 27. Juli 2020 06:25 >>> To: hotspot-runtime-dev at openjdk.java.net; Baesken, Matthias >>> Cc: David Holmes >>> Subject: RFR: 8250598: Hyper-V is detected in spite of running on host OS >>> >>> Hi all, >>> >>> Please review this change: >>> >>> ??? JBS: https://bugs.openjdk.java.net/browse/JDK-8250598 >>> ??? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.00/ >>> >>> When I got hs_err log on Windows, I saw "HyperV virtualization detected" in it in spite of running on host OS. >>> >>> Hypervisor detector has been introduced in JDK-8219241, but it has some problems as below: >>> >>> ??? - Hyper-V is detected on Windows in spite of running on host OS >>> ??? - Call CPUID with other than EAX = 40000000h (it is not described in the spec [1]) >>> ??? - Does not check CPUID hypervisor present bit [1] >>> ??? - Does not support x86 (32bit) platform >>> >>> I've tested this change on submit repo, and have checked output from VM.info jcmd on following environment: >>> >>> ??? - Windows x64 (host) >>> ??? - Windows x64 (Hyper-V guest) >>> ??? - Fedora32 x64 (Hyper-V guest) >>> ??? - 32 bit JDK on Fedora32 x64 (Hyper-V guest) >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> [1] https://kb.vmware.com/s/article/1009458 >>> From david.holmes at oracle.com Wed Aug 5 02:12:53 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 5 Aug 2020 12:12:53 +1000 Subject: RFR 8139875: [TESTBUG] Improve nsk/stress/stack/* tests In-Reply-To: <0a1a7076-b9f8-37ca-3c53-a4afeca1f406@oracle.com> References: <62a030ca-99e1-1b5c-ade7-e8685aa9f8e3@oracle.com> <262e8bd2-5759-0b88-4977-233be3161739@oracle.com> <0a1a7076-b9f8-37ca-3c53-a4afeca1f406@oracle.com> Message-ID: <30716c9b-25c2-842f-9d2e-8c2d0fa193d8@oracle.com> Hi Harold, On 5/08/2020 5:16 am, Harold Seigel wrote: > Hi David, > > Thanks for the suggestion.? Please review this updated webrev that uses > -Xss200K for all of the tests. > > http://cr.openjdk.java.net/~hseigel/bug_8139875.2/webrev/index.html That all seems reasonable. I guess we will find out. :) Thanks, David ----- > Thanks, Harold > > On 8/3/2020 7:04 PM, David Holmes wrote: >> Hi Harold, >> >> On 4/08/2020 4:11 am, Harold Seigel wrote: >>> Hi, >>> >>> Please review these small changes to reduce the amount of time >>> required by some of the nsk/stress/stack tests in order to prevent >>> them from timing out.? The changes include using -Xss to run the >>> tests with smaller stacks, prevent running with -Xcomp, and, for test >>> stack019, reduce the number of test iterations. >> >> Is there a reason to use a range of -Xss values in different tests >> instead of just, say, using -Xss200K in all of them and reducing the >> execution time further. >> >> Thanks, >> David >> ----- >> >>> Test stack019 was excluded from windows targets because, even with >>> the reduced iterations, the test was running for > 48 minutes. >>> >>> Open Webrev: >>> http://cr.openjdk.java.net/~hseigel/bug_8139875/webrev/index.html >>> >>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8139875 >>> >>> The tests were tested by running them in Mach5 tier5 and tier7 and >>> looking at their execution time. >>> >>> Thanks, Harold >>> From david.holmes at oracle.com Wed Aug 5 02:52:42 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 5 Aug 2020 12:52:42 +1000 Subject: RFR(S): 8247941: Use Vectored Exception Handling on Windows In-Reply-To: References: Message-ID: <7ab1d205-1163-fd58-6a6f-546432f801eb@oracle.com> Hi Ludovic, I can't review the details here as it is not something I know about. The AOT folk (Bob V. cc'd) were the last to make changes in this area so it would be good if they chimed in on the review thread. :) With regard to the test a couple of points. First please fix the copyright lines so there is only the single year "2020," (the comma is still needed). Second rather than using ifdefs in the native code you can exclude it from being built other than on Windows. This should work: diff -r 3b3965c0cce5 make/test/JtregNativeHotspot.gmk --- a/make/test/JtregNativeHotspot.gmk +++ b/make/test/JtregNativeHotspot.gmk @@ -874,6 +874,7 @@ BUILD_HOTSPOT_JTREG_EXCLUDE += exesigtest.c libterminatedThread.c BUILD_HOTSPOT_JTREG_LIBRARIES_LIBS_libatExit := jvm.lib else + BUILD_HOTSPOT_JTREG_EXCLUDE += libExternalExceptions.c BUILD_HOTSPOT_JTREG_LIBRARIES_LIBS_libbootclssearch_agent += -lpthread BUILD_HOTSPOT_JTREG_LIBRARIES_LIBS_libsystemclssearch_agent += -lpthread BUILD_HOTSPOT_JTREG_LIBRARIES_LIBS_libgetsysprop001 += -lpthread Thanks, David ----- On 5/08/2020 7:03 am, Ludovic Henry wrote: > (Renaming subject to include bug id, original discussion at [1]). > > Webrev: http://cr.openjdk.java.net/~luhenry/8247941/webrev.00/ > JBS: https://bugs.openjdk.java.net/browse/JDK-8247941 > > I've uploaded the webrev at the link above. > > [1] https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040228.html > > -----Original Message----- > From: hotspot-runtime-dev On Behalf Of Ludovic Henry > Sent: Tuesday, August 4, 2020 8:34 AM > To: Thomas St?fe ; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(S): Use Vectored Exception Handling on Windows > > Hi, > > An update on my current exploration is long overdue. > > # On the use of Unhandled Exception Handling. > Unfortunately, this is a dead-end because UEH itself relies on Structured Exception Handling to catch any uncaught exception at the top-most frame, and then needs a properly unwindable stack. This is not the case for Java frames as Hotspot follows its own internal ABI, making it impossible for the native unwinder (which doesn't know about the Java ABI) to walk the stack frames. The easy fallback is simply to keep the existing `__try {} __except (topLevelExceptionFilter(...)) {}` and call to `RtlAddFunctionTable` to add such try/except block to the code cache. > > # On the use of Vectored Exception Handling > Despite UEH not being usable in our case, VEH is still helpful in getting Hotspot to treat Hotspot related exceptions first and to separate the normal exception handling from the crash handling. > > I'll post an updated webrev with these changes later today. > > Thank you, > Ludovic > From dms at samersoff.net Wed Aug 5 07:16:32 2020 From: dms at samersoff.net (Dmitry Samersoff) Date: Wed, 5 Aug 2020 10:16:32 +0300 Subject: RFR 8241053: Hotspot runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java test fails on Alpine Linux with debug build In-Reply-To: <64281c1f-970d-b792-eb42-63ae1488694b@bell-sw.com> References: <87a32c5b-8e96-f323-1e20-8df181ab3ec0@bell-sw.com> <5b530109-bda2-820e-6070-d779f08b64f6@oracle.com> <64281c1f-970d-b792-eb42-63ae1488694b@bell-sw.com> Message-ID: <19c11cfa-480f-f0ea-1cd8-8431522de8b5@samersoff.net> Hello Alexander, Sorry for being later to the party. It might be better to write it as: if (status == EINVAL) { // pthread_attr_setstacksize() function can fail // if the stack size exceeds a system-imposed limit. // ... return false; } assert_status(status == 0, status, "pthread_attr_setstacksize"); I.e. just move "assert" after "if" in your initial fix. But I will not push for it. Fix looks good to me. -Dmitry On 04.08.2020 12:11, Alexander Scherbatiy wrote: > Could you review the updated fix: > ? http://cr.openjdk.java.net/~alexsch/8241053/webrev.01 > > The assert_status() and the thread cleanup is moved under the "if > (status != 0)" check. > > Thanks, > Alexander. > > On 04.08.2020 01:51, David Holmes wrote: >> Hi Alexander, >> >> Thanks for fixing this. >> >> On 4/08/2020 1:33 am, Alexander Scherbatiy wrote: >>> Hello, >>> >>> Could you review the fix for the issue: >>> ?? Bug: https://bugs.openjdk.java.net/browse/JDK-8241053 >>> ?? Webrev: http://cr.openjdk.java.net/~alexsch/8241053/weberv.00 >> >> I would simplify this a little: >> >> ?? assert_status(status == 0 || status == EINVAL, status, >> "pthread_attr_setstacksize"); >> ?? // pthread_attr_setstacksize() function can fail >> ?? // if the stack size exceeds a system-imposed limit. >> ?? if (status == EINVAL) { >> >> to >> ?? if (status != 0) { >> ???? // pthread_attr_setstacksize() function can fail >> ???? // if the stack size exceeds a system-imposed limit. >> ???? assert_status(status == EINVAL, status, >> "pthread_attr_setstacksize"); >> >> Thanks, >> David >> ----- >> >> >>> The following tests fail on Linux Alpine with musl libc: >>> ?? test/hotspot/jtreg/runtime/Thread/TestThreadStackSizes.java >>> test/hotspot/jtreg/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>> >>> >>> This is because muls pthread_attr_setstacksize(3) function >>> implementation [1] returns EINVAL when stack size exceeds the certain >>> limit. >>> >>> According to POSIX pthread_attr_setstacksize() function description [2]: >>> ??The pthread_attr_setstacksize() function will fail if: >>> ???? [EINVAL]? The value of stacksize is less than PTHREAD_STACK_MIN >>> or exceeds a system-imposed limit. >>> >>> The proposed fix excludes EINVAL value from assert_status() and for >>> the EINVAL return value makes the same cleanup actions and returns >>> false from os::create_thread() function as it is done for AIX[3]. >>> >>> To reproduce the issue it needs to build Portola from repository [4] >>> using instructions from JEP 386: Alpine Linux/x64 Port [5]. >>> After the fix both TestThreadStackSizes.java and >>> TestOptionsWithRanges.java tests pass on Alpine Linux with musl libc >>> and on Ubuntu 18.0.4. >>> >>> [1] >>> https://git.musl-libc.org/cgit/musl/tree/src/thread/pthread_attr_setstacksize.c >>> >>> [2] >>> https://pubs.opengroup.org/onlinepubs/007908799/xsh/pthread_attr_setstacksize.html >>> >>> [3] >>> https://hg.openjdk.java.net/jdk/jdk/file/c8102e6fc512/src/hotspot/os/aix/os_aix.cpp#l890 >>> >>> [4] https://github.com/openjdk/portola >>> [5] https://openjdk.java.net/jeps/386 >>> >>> Thanks, >>> Alexander. >>> From matthias.baesken at sap.com Wed Aug 5 07:26:04 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 5 Aug 2020 07:26:04 +0000 Subject: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> Message-ID: Hello, >> I also checked the hserr on another Hyper-V guest and still get the expected " HyperV virtualization detected" (which is good ?? ). >> >> >> However on a host, I would still prefer to get some output (probably different than the guest output now) . > >What message do you expect? How about "Hyper-V role detected" ? Thats okay with me (I think some people here say "Hyper-V host" so that might be an idea as well). >> Regarding the usage of >> >> #pragma comment(lib, "wbemuuid.lib") >> >> This seem to add an additional lib dependency , should some people from build/deploy area comment on this maybe ? >> (not saying it is a very bad thing ) > >We add wbemuuid.lib to make/autoconf/libraries.m4 . >I will change it in next webrev, and I will CCing to build-dev on next turn. Let's see what they have to say. Best regards, Matthias From thomas.stuefe at gmail.com Wed Aug 5 07:33:07 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 5 Aug 2020 09:33:07 +0200 Subject: RFR 8241053: Hotspot runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java test fails on Alpine Linux with debug build In-Reply-To: <64281c1f-970d-b792-eb42-63ae1488694b@bell-sw.com> References: <87a32c5b-8e96-f323-1e20-8df181ab3ec0@bell-sw.com> <5b530109-bda2-820e-6070-d779f08b64f6@oracle.com> <64281c1f-970d-b792-eb42-63ae1488694b@bell-sw.com> Message-ID: Fix looks good to me too (though you already have your two reviewers). Since I am here I have a question about muslc, if I may: Is there a way to execute code conditionally for glibc only? I ran into this question a couple of times in the past. Related to that, would that have to be a runtime check or would a compile time check sufficient? (In other words, should a JDK built for linux x64 be runnable on distros with muslc as well as glibc?) Thank you! ..Thomas On Tue, Aug 4, 2020 at 11:13 AM Alexander Scherbatiy < alexander.scherbatiy at bell-sw.com> wrote: > Could you review the updated fix: > http://cr.openjdk.java.net/~alexsch/8241053/webrev.01 > > The assert_status() and the thread cleanup is moved under the "if > (status != 0)" check. > > Thanks, > Alexander. > > On 04.08.2020 01:51, David Holmes wrote: > > Hi Alexander, > > > > Thanks for fixing this. > > > > On 4/08/2020 1:33 am, Alexander Scherbatiy wrote: > >> Hello, > >> > >> Could you review the fix for the issue: > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8241053 > >> Webrev: http://cr.openjdk.java.net/~alexsch/8241053/weberv.00 > > > > I would simplify this a little: > > > > assert_status(status == 0 || status == EINVAL, status, > > "pthread_attr_setstacksize"); > > // pthread_attr_setstacksize() function can fail > > // if the stack size exceeds a system-imposed limit. > > if (status == EINVAL) { > > > > to > > if (status != 0) { > > // pthread_attr_setstacksize() function can fail > > // if the stack size exceeds a system-imposed limit. > > assert_status(status == EINVAL, status, > > "pthread_attr_setstacksize"); > > > > Thanks, > > David > > ----- > > > > > >> The following tests fail on Linux Alpine with musl libc: > >> test/hotspot/jtreg/runtime/Thread/TestThreadStackSizes.java > >> > test/hotspot/jtreg/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java > > >> > >> > >> This is because muls pthread_attr_setstacksize(3) function > >> implementation [1] returns EINVAL when stack size exceeds the certain > >> limit. > >> > >> According to POSIX pthread_attr_setstacksize() function description [2]: > >> The pthread_attr_setstacksize() function will fail if: > >> [EINVAL] The value of stacksize is less than PTHREAD_STACK_MIN > >> or exceeds a system-imposed limit. > >> > >> The proposed fix excludes EINVAL value from assert_status() and for > >> the EINVAL return value makes the same cleanup actions and returns > >> false from os::create_thread() function as it is done for AIX[3]. > >> > >> To reproduce the issue it needs to build Portola from repository [4] > >> using instructions from JEP 386: Alpine Linux/x64 Port [5]. > >> After the fix both TestThreadStackSizes.java and > >> TestOptionsWithRanges.java tests pass on Alpine Linux with musl libc > >> and on Ubuntu 18.0.4. > >> > >> [1] > >> > https://git.musl-libc.org/cgit/musl/tree/src/thread/pthread_attr_setstacksize.c > >> > >> [2] > >> > https://pubs.opengroup.org/onlinepubs/007908799/xsh/pthread_attr_setstacksize.html > >> > >> [3] > >> > https://hg.openjdk.java.net/jdk/jdk/file/c8102e6fc512/src/hotspot/os/aix/os_aix.cpp#l890 > >> > >> [4] https://github.com/openjdk/portola > >> [5] https://openjdk.java.net/jeps/386 > >> > >> Thanks, > >> Alexander. > >> > From martin.doerr at sap.com Wed Aug 5 07:47:44 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 5 Aug 2020 07:47:44 +0000 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> Message-ID: Hi Patricio, using 8 Byte load instructions for jint fields is a terrible coding style! Someone else may see it and use an 8 Byte store. Will result in great fun for debugging! And on Big Endian you will simply access the wrong bits. Note that Big Endian Platforms are AIX, old linux ppc, s390, SPARC. I don't think you have tested on them. > We could remove the nested synchronized statements in the run() method > so that we don't generate that much logging. We could also lower > LOOP_COUNT. I think the issue is also because we are running LogTest > with multiple flag combinations. Not sure what we should touch first. > Maybe the synchronized statements, have only one or two and test that > first? Sounds like helpful ideas. Please go ahead and strip things down. Thanks for taking care of it. Best regards, Martin > -----Original Message----- > From: Patricio Chilano > Sent: Dienstag, 4. August 2020 20:26 > To: Doerr, Martin ; David Holmes > ; hotspot-runtime-dev at openjdk.java.net; > hotspot-jfr-dev at openjdk.java.net > Subject: Re: RFR 8242263: Diagnose synchronization on primitive wrappers > > Hi Martin, > > On 8/4/20 1:35 PM, Doerr, Martin wrote: > > Hi Patricio, > > > >> Ok, I'll change it to movl + testl and test it out before sending v3. > > Thanks. I forgot to mention arm + aarch64. > > Aarch64 uses ldrw + tbnz. > > Arm 32 bit uses ldr_u32 + tbnz. > > > > I remember having seen the same mistake ?? > > And nobody noticed it on little Endian platforms. > Ok, I can use a tbnz instead of test and then a branch on arm32 and aarch64. > Shouldn't a normal ldr in arm32 work fine? > Also in 64 bits (either x64 or aarch64) I don't see the issue of using a > 64 bit load, besides the fact that we only care about the first 32 bits. > Regardless of the endianness, aren't we masking out the upper part when > we do AND/TEST or if we test a bit in the 0-31 bit range? Otherwise it > seems it should have failed for one of those platforms in my tests. > > >> I see that some tests use @run driver/timeout=xxxx. Maybe you can > >> specify that and see if that fixes it? Let me know if that works and I > >> can add it to the test. > > That seems to have an effect. But now I'm not patient enough to wait for > the test to finish. > > Maybe the problem is that I'm using slow debug builds. > > But is there a chance to make the test quicker without losing coverage > significantly? > > I think the test is too slow this way to run it on a regular basis. We'd need to > spend dedicated machines for it. > We could remove the nested synchronized statements in the run() method > so that we don't generate that much logging. We could also lower > LOOP_COUNT. I think the issue is also because we are running LogTest > with multiple flag combinations. Not sure what we should touch first. > Maybe the synchronized statements, have only one or two and test that > first? > > > Thanks, > Patricio > > Best regards, > > Martin > > > > > >> -----Original Message----- > >> From: Patricio Chilano > >> Sent: Dienstag, 4. August 2020 17:47 > >> To: Doerr, Martin ; David Holmes > >> ; hotspot-runtime-dev at openjdk.java.net; > >> hotspot-jfr-dev at openjdk.java.net > >> Subject: Re: RFR 8242263: Diagnose synchronization on primitive wrappers > >> > >> Hi Martin, > >> > >> Thanks for fixing PPC and taking care of s390! > >> > >> On 8/4/20 11:18 AM, Doerr, Martin wrote: > >>> Hi Patricio, > >>> > >>> I suggest to use movl + testl for checking the access flag as for other > access > >> flags on x64. > >> Ok, I'll change it to movl + testl and test it out before sending v3. > >> > >>> New version for PPC64 and s390 see below. > >>> > >>> The test SyncOnPrimitiveWrapperTest produces hs_err files as > expected. > >> However, I'm getting timeout issues: > >>> timed out (timeout set to 120000ms, elapsed time including timeout > >> handling was 122372ms) > >>> Can we provide more time? > >> I see that some tests use @run driver/timeout=xxxx. Maybe you can > >> specify that and see if that fixes it? Let me know if that works and I > >> can add it to the test. > >> > >> Thanks, > >> Patricio > >>> Best regards, > >>> Martin > >>> > >>> > >>> diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp > >>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 > >> 10:03:57 2020 +0200 > >>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 > >> 16:04:31 2020 +0200 > >>> @@ -107,8 +107,8 @@ > >>> > >>> if (DiagnoseSyncOnPrimitiveWrappers != 0) { > >>> load_klass(Rscratch, Roop); > >>> - ld(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); > >>> - andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); > >>> + lwz(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); > >>> + testbitdi(CCR0, R0, Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS)); > >>> bne(CCR0, slow_case); > >>> } > >>> > >>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp > >>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 > >> 10:03:57 2020 +0200 > >>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 > >> 16:04:31 2020 +0200 > >>> @@ -912,8 +912,8 @@ > >>> > >>> if (DiagnoseSyncOnPrimitiveWrappers != 0) { > >>> load_klass(tmp, object); > >>> - ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); > >>> - andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); > >>> + lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); > >>> + testbitdi(CCR0, R0, tmp, exact_log2(JVM_ACC_IS_BOX_CLASS)); > >>> bne(CCR0, slow_case); > >>> } > >>> > >>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp > >>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 > >> 10:03:57 2020 +0200 > >>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 > >> 16:04:31 2020 +0200 > >>> @@ -2838,8 +2838,8 @@ > >>> > >>> if (DiagnoseSyncOnPrimitiveWrappers != 0) { > >>> load_klass(temp, oop); > >>> - ld(temp, in_bytes(Klass::access_flags_offset()), temp); > >>> - andi(temp, temp, JVM_ACC_IS_BOX_CLASS); > >>> + lwz(temp, in_bytes(Klass::access_flags_offset()), temp); > >>> + testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); > >>> bne(CCR0, cont); > >>> } > >>> > >>> diff -r 77852e129401 > src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp > >>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 > >> 10:03:57 2020 +0200 > >>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 > >> 16:04:31 2020 +0200 > >>> @@ -91,6 +91,12 @@ > >>> // Save object being locked into the BasicObjectLock... > >>> z_stg(obj, Address(disp_hdr, > BasicObjectLock::obj_offset_in_bytes())); > >>> > >>> + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > >>> + load_klass(Z_R1_scratch, obj); > >>> + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), > >> exact_log2(JVM_ACC_IS_BOX_CLASS)); > >>> + z_btrue(slow_case); > >>> + } > >>> + > >>> if (UseBiasedLocking) { > >>> biased_locking_enter(obj, hdr, Z_R1_scratch, Z_R0_scratch, done, > >> &slow_case); > >>> } > >>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp > >>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 10:03:57 > >> 2020 +0200 > >>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 > 16:04:31 > >> 2020 +0200 > >>> @@ -1000,6 +1000,12 @@ > >>> // Load markWord from object into displaced_header. > >>> z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), object); > >>> > >>> + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > >>> + load_klass(Z_R1_scratch, object); > >>> + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), > >> exact_log2(JVM_ACC_IS_BOX_CLASS)); > >>> + z_btrue(slow_case); > >>> + } > >>> + > >>> if (UseBiasedLocking) { > >>> biased_locking_enter(object, displaced_header, Z_R1, Z_R0, done, > >> &slow_case); > >>> } > >>> diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp > >>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 > >> 10:03:57 2020 +0200 > >>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 > >> 16:04:31 2020 +0200 > >>> @@ -3358,6 +3358,12 @@ > >>> // Load markWord from oop into mark. > >>> z_lg(displacedHeader, 0, oop); > >>> > >>> + if (DiagnoseSyncOnPrimitiveWrappers != 0) { > >>> + load_klass(Z_R1_scratch, oop); > >>> + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), > >> exact_log2(JVM_ACC_IS_BOX_CLASS)); > >>> + z_btrue(done); > >>> + } > >>> + > >>> if (try_bias) { > >>> biased_locking_enter(oop, displacedHeader, temp, Z_R0, done); > >>> } > >>> From david.holmes at oracle.com Wed Aug 5 07:58:31 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 5 Aug 2020 17:58:31 +1000 Subject: RFR (s): 8250606: Remove unnecessary assertions in ObjectSynchronizer FastHashcode and inflate Message-ID: <3e4c931a-593b-0336-97b9-3adc4de96062@oracle.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8250606 webrev: http://cr.openjdk.java.net/~dholmes/8250606/webrev/ tl;dr I removed some assertions that were deemed unnecessary after interfering with other work. :) ObjectSynchronizer::inflate had the following assertion added back in JDK 6 under JDK-5030359: ObjectMonitor * ObjectSynchronizer::inflate (oop object) { // Inflate mutates the heap ... assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; That same change added FastHashcode which had the following assertions: intptr_t ObjectSynchronizer::FastHashCode (Thread * Self, oop obj) { assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; assert (Self->is_Java_thread(), "invariant") ; assert (((JavaThread *)Self)->thread_state() != _thread_blocked, "invariant") ; These methods had assertions that they were not executed at a safepoint in case the modification of the markWord could cause interference with GC. JDK-6320749 then relaxed the assertions to allow for GC verification - which does happen at a safepoint and so obviously not in a JavaThread. The FastHashCode assertions were further relaxed in JDK-8015086 to allow the CDS dump VM safepoint operation to call it. But FastHashCode can also call inflate so that was a mismatch in assertions just waiting to happen - which is how I hit this in a VM that always forces inflation :) I initially intended to extend the FastHashCode assertion to match inflate, but then a question arose about the safety of the CDS dumping code using FastHashcode which showed that the code is in fact safe. Notwithstanding what the state of the world may have been in the JDK 6 timeframe, there is now no potential interference possible between execution of these methods at an arbitrary safepoint and any concurrent GC use of the markWord. So the original !is_at_safepoint assertion can just be dropped from both methods. That in turn means we either drop the JavaThread assertion or else expand it to be "JavaThread or VMThread" - but that itself seems unnecessarily restrictive: why can't a GC thread use this code for example? So I chose to drop it. That leaves the "if I'm a JavaThread I'm not _thread_blocked" assertion. I don't see that this really guards against anything useful (a JavaThread should be _thread_in_vm when executing this code ... though there may be leaf calls where the thread is _thread_in_java). So again I just dropped the assertion. So a long winded justification for dropping a few assertions. :) Thanks, David From alexander.scherbatiy at bell-sw.com Wed Aug 5 08:41:18 2020 From: alexander.scherbatiy at bell-sw.com (Alexander Scherbatiy) Date: Wed, 5 Aug 2020 11:41:18 +0300 Subject: RFR 8241053: Hotspot runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java test fails on Alpine Linux with debug build In-Reply-To: <19c11cfa-480f-f0ea-1cd8-8431522de8b5@samersoff.net> References: <87a32c5b-8e96-f323-1e20-8df181ab3ec0@bell-sw.com> <5b530109-bda2-820e-6070-d779f08b64f6@oracle.com> <64281c1f-970d-b792-eb42-63ae1488694b@bell-sw.com> <19c11cfa-480f-f0ea-1cd8-8431522de8b5@samersoff.net> Message-ID: <5267bc12-ae35-4efa-b06a-bc98f69863a6@bell-sw.com> On 05.08.2020 10:16, Dmitry Samersoff wrote: > Hello Alexander, > > Sorry for being later to the party. It might be better to write it as: > > if (status == EINVAL) { > ????? // pthread_attr_setstacksize() function can fail > ????? // if the stack size exceeds a system-imposed limit. > ????? // > ... > > ??? return false; > } > > assert_status(status == 0, status, "pthread_attr_setstacksize"); > > I.e. just move "assert" after "if" in your initial fix. ? It looks like the behavior would be different when pthread_attr_setstacksize() returns error other than EINVAL on non-debug build. ? In the former case there will be the thread cleanup and return from the os::create_thread(). ? In the later case the execution will be continued. ? I am not sure which one is better. ? Thanks, ? Alexander. > > But I will not push for it. Fix looks good to me. > > -Dmitry > > On 04.08.2020 12:11, Alexander Scherbatiy wrote: >> Could you review the updated fix: >> ?? http://cr.openjdk.java.net/~alexsch/8241053/webrev.01 >> >> The assert_status() and the thread cleanup is moved under the "if >> (status != 0)" check. >> >> Thanks, >> Alexander. >> >> On 04.08.2020 01:51, David Holmes wrote: >>> Hi Alexander, >>> >>> Thanks for fixing this. >>> >>> On 4/08/2020 1:33 am, Alexander Scherbatiy wrote: >>>> Hello, >>>> >>>> Could you review the fix for the issue: >>>> ?? Bug: https://bugs.openjdk.java.net/browse/JDK-8241053 >>>> ?? Webrev: http://cr.openjdk.java.net/~alexsch/8241053/weberv.00 >>> >>> I would simplify this a little: >>> >>> ?? assert_status(status == 0 || status == EINVAL, status, >>> "pthread_attr_setstacksize"); >>> ?? // pthread_attr_setstacksize() function can fail >>> ?? // if the stack size exceeds a system-imposed limit. >>> ?? if (status == EINVAL) { >>> >>> to >>> ?? if (status != 0) { >>> ???? // pthread_attr_setstacksize() function can fail >>> ???? // if the stack size exceeds a system-imposed limit. >>> ???? assert_status(status == EINVAL, status, >>> "pthread_attr_setstacksize"); >>> >>> Thanks, >>> David >>> ----- >>> >>> >>>> The following tests fail on Linux Alpine with musl libc: >>>> test/hotspot/jtreg/runtime/Thread/TestThreadStackSizes.java >>>> test/hotspot/jtreg/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>>> >>>> >>>> This is because muls pthread_attr_setstacksize(3) function >>>> implementation [1] returns EINVAL when stack size exceeds the >>>> certain limit. >>>> >>>> According to POSIX pthread_attr_setstacksize() function description >>>> [2]: >>>> ??The pthread_attr_setstacksize() function will fail if: >>>> ???? [EINVAL]? The value of stacksize is less than >>>> PTHREAD_STACK_MIN or exceeds a system-imposed limit. >>>> >>>> The proposed fix excludes EINVAL value from assert_status() and for >>>> the EINVAL return value makes the same cleanup actions and returns >>>> false from os::create_thread() function as it is done for AIX[3]. >>>> >>>> To reproduce the issue it needs to build Portola from repository >>>> [4] using instructions from JEP 386: Alpine Linux/x64 Port [5]. >>>> After the fix both TestThreadStackSizes.java and >>>> TestOptionsWithRanges.java tests pass on Alpine Linux with musl >>>> libc and on Ubuntu 18.0.4. >>>> >>>> [1] >>>> https://git.musl-libc.org/cgit/musl/tree/src/thread/pthread_attr_setstacksize.c >>>> >>>> [2] >>>> https://pubs.opengroup.org/onlinepubs/007908799/xsh/pthread_attr_setstacksize.html >>>> >>>> [3] >>>> https://hg.openjdk.java.net/jdk/jdk/file/c8102e6fc512/src/hotspot/os/aix/os_aix.cpp#l890 >>>> >>>> [4] https://github.com/openjdk/portola >>>> [5] https://openjdk.java.net/jeps/386 >>>> >>>> Thanks, >>>> Alexander. >>>> > From suenaga at oss.nttdata.com Wed Aug 5 09:30:31 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 5 Aug 2020 18:30:31 +0900 Subject: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> Message-ID: Hi Matthias, (CC'ed build-dev due to makefile change) I uploaded new webrev. Could you review it? http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/ On 2020/08/05 16:26, Baesken, Matthias wrote: > Hello, > >>> I also checked the hserr on another Hyper-V guest and still get the expected " HyperV virtualization detected" (which is good ?? ). >>> >>> >>> However on a host, I would still prefer to get some output (probably different than the guest output now) . >> >> What message do you expect? How about "Hyper-V role detected" ? > > Thats okay with me (I think some people here say "Hyper-V host" so that might be an idea as well). I said "Hyper-V host" in previous mails, but it is not strictly correct. In Hyper-V, it should be "root partision". However it's a little difficult to understand, so I choose "role" in this place. >>> Regarding the usage of >>> >>> #pragma comment(lib, "wbemuuid.lib") >>> >>> This seem to add an additional lib dependency , should some people from build/deploy area comment on this maybe ? >>> (not saying it is a very bad thing ) I modified libraries.m4. wbemuuid.lib is needed for WMI call. Thanks, Yasumasa >> We add wbemuuid.lib to make/autoconf/libraries.m4 . >> I will change it in next webrev, and I will CCing to build-dev on next turn. > > > Let's see what they have to say. > > Best regards, Matthias > > From suenaga at oss.nttdata.com Wed Aug 5 09:36:43 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 5 Aug 2020 18:36:43 +0900 Subject: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> Message-ID: CC'ing build-dev On 2020/08/05 18:30, Yasumasa Suenaga wrote: > Hi Matthias, > (CC'ed build-dev due to makefile change) > > I uploaded new webrev. Could you review it? > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/ > > On 2020/08/05 16:26, Baesken, Matthias wrote: >> Hello, >> >>>> I also checked the? hserr on another Hyper-V guest and still get the expected " HyperV virtualization detected"?? (which is good ?? ). >>>> >>>> >>>> However on a host, I would still prefer to get some output (probably different than the guest output now) . >>> >>> What message do you expect? How about "Hyper-V role detected" ? >> >> Thats okay with me? (I think some people here say "Hyper-V host" so that might be an idea as well). > > I said "Hyper-V host" in previous mails, but it is not strictly correct. In Hyper-V, it should be "root partision". However it's a little difficult to understand, so I choose "role" in this place. > > >>>> Regarding the usage of >>>> >>>> #pragma comment(lib, "wbemuuid.lib") >>>> >>>> This seem to add an additional lib dependency ,? should some people from build/deploy? area comment on this maybe ? >>>> (not saying it is a very bad thing ) > > I modified libraries.m4. > wbemuuid.lib is needed for WMI call. > > > Thanks, > > Yasumasa > > >>> We add wbemuuid.lib to make/autoconf/libraries.m4 . >>> I will change it in next webrev, and I will CCing to build-dev on next turn. >> >> >> Let's see what they have to say. >> >> Best regards, Matthias >> >> From harold.seigel at oracle.com Wed Aug 5 12:28:53 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Wed, 5 Aug 2020 08:28:53 -0400 Subject: RFR 8139875: [TESTBUG] Improve nsk/stress/stack/* tests In-Reply-To: <30716c9b-25c2-842f-9d2e-8c2d0fa193d8@oracle.com> References: <62a030ca-99e1-1b5c-ade7-e8685aa9f8e3@oracle.com> <262e8bd2-5759-0b88-4977-233be3161739@oracle.com> <0a1a7076-b9f8-37ca-3c53-a4afeca1f406@oracle.com> <30716c9b-25c2-842f-9d2e-8c2d0fa193d8@oracle.com> Message-ID: <4c874694-2e78-d87e-8b38-f44fd906dcaf@oracle.com> Thanks David! Harold On 8/4/2020 10:12 PM, David Holmes wrote: > Hi Harold, > > On 5/08/2020 5:16 am, Harold Seigel wrote: >> Hi David, >> >> Thanks for the suggestion.? Please review this updated webrev that >> uses -Xss200K for all of the tests. >> >> http://cr.openjdk.java.net/~hseigel/bug_8139875.2/webrev/index.html > > That all seems reasonable. I guess we will find out. :) > > Thanks, > David > ----- > >> Thanks, Harold >> >> On 8/3/2020 7:04 PM, David Holmes wrote: >>> Hi Harold, >>> >>> On 4/08/2020 4:11 am, Harold Seigel wrote: >>>> Hi, >>>> >>>> Please review these small changes to reduce the amount of time >>>> required by some of the nsk/stress/stack tests in order to prevent >>>> them from timing out.? The changes include using -Xss to run the >>>> tests with smaller stacks, prevent running with -Xcomp, and, for >>>> test stack019, reduce the number of test iterations. >>> >>> Is there a reason to use a range of -Xss values in different tests >>> instead of just, say, using -Xss200K in all of them and reducing the >>> execution time further. >>> >>> Thanks, >>> David >>> ----- >>> >>>> Test stack019 was excluded from windows targets because, even with >>>> the reduced iterations, the test was running for > 48 minutes. >>>> >>>> Open Webrev: >>>> http://cr.openjdk.java.net/~hseigel/bug_8139875/webrev/index.html >>>> >>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8139875 >>>> >>>> The tests were tested by running them in Mach5 tier5 and tier7 and >>>> looking at their execution time. >>>> >>>> Thanks, Harold >>>> From erik.joelsson at oracle.com Wed Aug 5 12:30:35 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 5 Aug 2020 05:30:35 -0700 Subject: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> Message-ID: <58a6ddb7-2aa0-595b-e999-2e4277f9e7c5@oracle.com> Build change is ok. /Erik On 2020-08-05 02:36, Yasumasa Suenaga wrote: > CC'ing build-dev > > On 2020/08/05 18:30, Yasumasa Suenaga wrote: >> Hi Matthias, >> (CC'ed build-dev due to makefile change) >> >> I uploaded new webrev. Could you review it? >> >> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/ >> >> On 2020/08/05 16:26, Baesken, Matthias wrote: >>> Hello, >>> >>>>> I also checked the? hserr on another Hyper-V guest and still get >>>>> the expected " HyperV virtualization detected"?? (which is good ?? ). >>>>> >>>>> >>>>> However on a host, I would still prefer to get some output >>>>> (probably different than the guest output now) . >>>> >>>> What message do you expect? How about "Hyper-V role detected" ? >>> >>> Thats okay with me? (I think some people here say "Hyper-V host" so >>> that might be an idea as well). >> >> I said "Hyper-V host" in previous mails, but it is not strictly >> correct. In Hyper-V, it should be "root partision". However it's a >> little difficult to understand, so I choose "role" in this place. >> >> >>>>> Regarding the usage of >>>>> >>>>> #pragma comment(lib, "wbemuuid.lib") >>>>> >>>>> This seem to add an additional lib dependency ,? should some >>>>> people from build/deploy? area comment on this maybe ? >>>>> (not saying it is a very bad thing ) >> >> I modified libraries.m4. >> wbemuuid.lib is needed for WMI call. >> >> >> Thanks, >> >> Yasumasa >> >> >>>> We add wbemuuid.lib to make/autoconf/libraries.m4 . >>>> I will change it in next webrev, and I will CCing to build-dev on >>>> next turn. >>> >>> >>> Let's see what they have to say. >>> >>> Best regards, Matthias >>> >>> From patricio.chilano.mateo at oracle.com Wed Aug 5 13:01:34 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Wed, 5 Aug 2020 10:01:34 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> Message-ID: <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> Hi Martin, On 8/5/20 4:47 AM, Doerr, Martin wrote: > Hi Patricio, > > using 8 Byte load instructions for jint fields is a terrible coding style! > Someone else may see it and use an 8 Byte store. Will result in great fun for debugging! > > And on Big Endian you will simply access the wrong bits. Ah, of course! Those 32 bits will be in the wrong place when doing the test. > Note that Big Endian Platforms are AIX, old linux ppc, s390, SPARC. I don't think you have tested on them. > >> We could remove the nested synchronized statements in the run() method >> so that we don't generate that much logging. We could also lower >> LOOP_COUNT. I think the issue is also because we are running LogTest >> with multiple flag combinations. Not sure what we should touch first. >> Maybe the synchronized statements, have only one or two and test that >> first? > Sounds like helpful ideas. Please go ahead and strip things down. Great, I will send v3 later with those changes. Thanks Martin! Patricio > Thanks for taking care of it. > > Best regards, > Martin > > >> -----Original Message----- >> From: Patricio Chilano >> Sent: Dienstag, 4. August 2020 20:26 >> To: Doerr, Martin ; David Holmes >> ; hotspot-runtime-dev at openjdk.java.net; >> hotspot-jfr-dev at openjdk.java.net >> Subject: Re: RFR 8242263: Diagnose synchronization on primitive wrappers >> >> Hi Martin, >> >> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>> Hi Patricio, >>> >>>> Ok, I'll change it to movl + testl and test it out before sending v3. >>> Thanks. I forgot to mention arm + aarch64. >>> Aarch64 uses ldrw + tbnz. >>> Arm 32 bit uses ldr_u32 + tbnz. >>> >>> I remember having seen the same mistake ?? >>> And nobody noticed it on little Endian platforms. >> Ok, I can use a tbnz instead of test and then a branch on arm32 and aarch64. >> Shouldn't a normal ldr in arm32 work fine? >> Also in 64 bits (either x64 or aarch64) I don't see the issue of using a >> 64 bit load, besides the fact that we only care about the first 32 bits. >> Regardless of the endianness, aren't we masking out the upper part when >> we do AND/TEST or if we test a bit in the 0-31 bit range? Otherwise it >> seems it should have failed for one of those platforms in my tests. >> >>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>> specify that and see if that fixes it? Let me know if that works and I >>>> can add it to the test. >>> That seems to have an effect. But now I'm not patient enough to wait for >> the test to finish. >>> Maybe the problem is that I'm using slow debug builds. >>> But is there a chance to make the test quicker without losing coverage >> significantly? >>> I think the test is too slow this way to run it on a regular basis. We'd need to >> spend dedicated machines for it. >> We could remove the nested synchronized statements in the run() method >> so that we don't generate that much logging. We could also lower >> LOOP_COUNT. I think the issue is also because we are running LogTest >> with multiple flag combinations. Not sure what we should touch first. >> Maybe the synchronized statements, have only one or two and test that >> first? >> >> >> Thanks, >> Patricio >>> Best regards, >>> Martin >>> >>> >>>> -----Original Message----- >>>> From: Patricio Chilano >>>> Sent: Dienstag, 4. August 2020 17:47 >>>> To: Doerr, Martin ; David Holmes >>>> ; hotspot-runtime-dev at openjdk.java.net; >>>> hotspot-jfr-dev at openjdk.java.net >>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive wrappers >>>> >>>> Hi Martin, >>>> >>>> Thanks for fixing PPC and taking care of s390! >>>> >>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>> Hi Patricio, >>>>> >>>>> I suggest to use movl + testl for checking the access flag as for other >> access >>>> flags on x64. >>>> Ok, I'll change it to movl + testl and test it out before sending v3. >>>> >>>>> New version for PPC64 and s390 see below. >>>>> >>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >> expected. >>>> However, I'm getting timeout issues: >>>>> timed out (timeout set to 120000ms, elapsed time including timeout >>>> handling was 122372ms) >>>>> Can we provide more time? >>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>> specify that and see if that fixes it? Let me know if that works and I >>>> can add it to the test. >>>> >>>> Thanks, >>>> Patricio >>>>> Best regards, >>>>> Martin >>>>> >>>>> >>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>> 10:03:57 2020 +0200 >>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>> 16:04:31 2020 +0200 >>>>> @@ -107,8 +107,8 @@ >>>>> >>>>> if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>> load_klass(Rscratch, Roop); >>>>> - ld(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); >>>>> - andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>> + lwz(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); >>>>> + testbitdi(CCR0, R0, Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>> bne(CCR0, slow_case); >>>>> } >>>>> >>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>> 10:03:57 2020 +0200 >>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>> 16:04:31 2020 +0200 >>>>> @@ -912,8 +912,8 @@ >>>>> >>>>> if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>> load_klass(tmp, object); >>>>> - ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>> - andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>> + lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>> + testbitdi(CCR0, R0, tmp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>> bne(CCR0, slow_case); >>>>> } >>>>> >>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>> 10:03:57 2020 +0200 >>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>> 16:04:31 2020 +0200 >>>>> @@ -2838,8 +2838,8 @@ >>>>> >>>>> if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>> load_klass(temp, oop); >>>>> - ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>> - andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>> + lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>> + testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>> bne(CCR0, cont); >>>>> } >>>>> >>>>> diff -r 77852e129401 >> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>> 10:03:57 2020 +0200 >>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>> 16:04:31 2020 +0200 >>>>> @@ -91,6 +91,12 @@ >>>>> // Save object being locked into the BasicObjectLock... >>>>> z_stg(obj, Address(disp_hdr, >> BasicObjectLock::obj_offset_in_bytes())); >>>>> + if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>> + load_klass(Z_R1_scratch, obj); >>>>> + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>> + z_btrue(slow_case); >>>>> + } >>>>> + >>>>> if (UseBiasedLocking) { >>>>> biased_locking_enter(obj, hdr, Z_R1_scratch, Z_R0_scratch, done, >>>> &slow_case); >>>>> } >>>>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 10:03:57 >>>> 2020 +0200 >>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >> 16:04:31 >>>> 2020 +0200 >>>>> @@ -1000,6 +1000,12 @@ >>>>> // Load markWord from object into displaced_header. >>>>> z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), object); >>>>> >>>>> + if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>> + load_klass(Z_R1_scratch, object); >>>>> + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>> + z_btrue(slow_case); >>>>> + } >>>>> + >>>>> if (UseBiasedLocking) { >>>>> biased_locking_enter(object, displaced_header, Z_R1, Z_R0, done, >>>> &slow_case); >>>>> } >>>>> diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>> 10:03:57 2020 +0200 >>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>> 16:04:31 2020 +0200 >>>>> @@ -3358,6 +3358,12 @@ >>>>> // Load markWord from oop into mark. >>>>> z_lg(displacedHeader, 0, oop); >>>>> >>>>> + if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>> + load_klass(Z_R1_scratch, oop); >>>>> + testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>> + z_btrue(done); >>>>> + } >>>>> + >>>>> if (try_bias) { >>>>> biased_locking_enter(oop, displacedHeader, temp, Z_R0, done); >>>>> } >>>>> From lois.foltan at oracle.com Wed Aug 5 13:13:43 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 5 Aug 2020 09:13:43 -0400 Subject: RFR 8139875: [TESTBUG] Improve nsk/stress/stack/* tests In-Reply-To: <62a030ca-99e1-1b5c-ade7-e8685aa9f8e3@oracle.com> References: <62a030ca-99e1-1b5c-ade7-e8685aa9f8e3@oracle.com> Message-ID: <8c4a2eac-371c-ccb8-62af-5595ce0d460c@oracle.com> Looks good. Lois On 8/3/2020 2:11 PM, Harold Seigel wrote: > Hi, > > Please review these small changes to reduce the amount of time > required by some of the nsk/stress/stack tests in order to prevent > them from timing out.? The changes include using -Xss to run the tests > with smaller stacks, prevent running with -Xcomp, and, for test > stack019, reduce the number of test iterations. > > Test stack019 was excluded from windows targets because, even with the > reduced iterations, the test was running for > 48 minutes. > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8139875/webrev/index.html > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8139875 > > The tests were tested by running them in Mach5 tier5 and tier7 and > looking at their execution time. > > Thanks, Harold > From harold.seigel at oracle.com Wed Aug 5 13:14:08 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Wed, 5 Aug 2020 09:14:08 -0400 Subject: RFR 8139875: [TESTBUG] Improve nsk/stress/stack/* tests In-Reply-To: <8c4a2eac-371c-ccb8-62af-5595ce0d460c@oracle.com> References: <62a030ca-99e1-1b5c-ade7-e8685aa9f8e3@oracle.com> <8c4a2eac-371c-ccb8-62af-5595ce0d460c@oracle.com> Message-ID: Thanks Lois! Harold On 8/5/2020 9:13 AM, Lois Foltan wrote: > Looks good. > Lois > > On 8/3/2020 2:11 PM, Harold Seigel wrote: >> Hi, >> >> Please review these small changes to reduce the amount of time >> required by some of the nsk/stress/stack tests in order to prevent >> them from timing out.? The changes include using -Xss to run the >> tests with smaller stacks, prevent running with -Xcomp, and, for test >> stack019, reduce the number of test iterations. >> >> Test stack019 was excluded from windows targets because, even with >> the reduced iterations, the test was running for > 48 minutes. >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8139875/webrev/index.html >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8139875 >> >> The tests were tested by running them in Mach5 tier5 and tier7 and >> looking at their execution time. >> >> Thanks, Harold >> > From lois.foltan at oracle.com Wed Aug 5 13:20:40 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 5 Aug 2020 09:20:40 -0400 Subject: RFR(S): JDK-8247938: Change various JVM enums like LinkInfo::AccessCheck and Klass::DefaultsLookupMode to enum class In-Reply-To: References: <3A1268FE-1231-4043-8CA8-06C9F3160421@oracle.com> Message-ID: On 8/3/2020 6:42 PM, Kim Barrett wrote: >> On Aug 3, 2020, at 11:12 AM, Lois Foltan wrote: >> >> Thank you Kim for the review! I do like your suggestion of changing the names of the enumerators. New webrev to include that renaming as part of this change. >> >> New webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8247938.1/webrev >> >> Retesting hs-tier1-3. >> >> Thanks, >> Lois > Looks good. > > BTW, for possible future work, LinkInfo might be a candidate for using > delegating constructors. > Thanks Kim for the review.? I will look into delegating constructors for LinkInfo and file a RFE if needed. Lois From gerard.ziemski at oracle.com Wed Aug 5 14:49:28 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Wed, 5 Aug 2020 09:49:28 -0500 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> Message-ID: <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> hi all, Please review this new test that replaces a closed (and flawed) one, which was causing time out issues on Mac OS X. The new test takes the main characteristic of the old test, which is creating a large number of threads, but in a robust and deterministic way - we create as many threads as the process allows and expect the VM to throw OOM exception to let us know when we reach the limit - then we perform a simple work unit for each thread (setting its name and verifying it back), then simply exit. To make it more robust we use the Thread.wait(TIMEOUT) instead of the Thread.wait() as in the original. To keep things nice and simple we use CountDownLatch objects for synchronization between the main and worker threads (thank you David). bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev1 testing: passes Mach5 hs-tier1,2,3,4,5 cheers From daniel.daugherty at oracle.com Wed Aug 5 18:08:48 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 5 Aug 2020 14:08:48 -0400 Subject: RFR (s): 8250606: Remove unnecessary assertions in ObjectSynchronizer FastHashcode and inflate In-Reply-To: <3e4c931a-593b-0336-97b9-3adc4de96062@oracle.com> References: <3e4c931a-593b-0336-97b9-3adc4de96062@oracle.com> Message-ID: <17eef440-3575-73f5-1413-0a25da6a13df@oracle.com> On 8/5/20 3:58 AM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8250606 > webrev: http://cr.openjdk.java.net/~dholmes/8250606/webrev/ src/hotspot/share/runtime/synchronizer.cpp ??? L1010: ????? // Relaxing assertion for bug 6320749. ??? L1011: ????? assert(Universe::verify_in_progress() || ??? L1012: ???????????? !SafepointSynchronize::is_at_safepoint(), ??? L1013: ???????????? "biases should not be seen by VM thread here"); ??????? This assertion is similar and tagged with the same bug ID (6320749) ??????? as the ones that you removed. Granted that the Biased Locking code ??????? is on the way out, but... ??????? I'm fine if you plan to leave this one for Biased Locking removal. Thumbs up.? Thanks for digging up the history behind these very old asserts. Dan > > tl;dr I removed some assertions that were deemed unnecessary after > interfering with other work. :) > > ObjectSynchronizer::inflate had the following assertion added back in > JDK 6 under JDK-5030359: > > ObjectMonitor * ObjectSynchronizer::inflate (oop object) { > ? // Inflate mutates the heap ... > ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; > > That same change added FastHashcode which had the following assertions: > > intptr_t ObjectSynchronizer::FastHashCode (Thread * Self, oop obj) { > ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; > ? assert (Self->is_Java_thread(), "invariant") ; > ? assert (((JavaThread *)Self)->thread_state() != _thread_blocked, > "invariant") ; > > These methods had assertions that they were not executed at a > safepoint in case the modification of the markWord could cause > interference with GC. > > JDK-6320749 then relaxed the assertions to allow for GC verification - > which does happen at a safepoint and so obviously not in a JavaThread. > > The FastHashCode assertions were further relaxed in JDK-8015086 to > allow the CDS dump VM safepoint operation to call it. But FastHashCode > can also call inflate so that was a mismatch in assertions just > waiting to happen - which is how I hit this in a VM that always forces > inflation :) > > I initially intended to extend the FastHashCode assertion to match > inflate, but then a question arose about the safety of the CDS dumping > code using FastHashcode which showed that the code is in fact safe. > Notwithstanding what the state of the world may have been in the JDK 6 > timeframe, there is now no potential interference possible between > execution of these methods at an arbitrary safepoint and any > concurrent GC use of the markWord. So the original !is_at_safepoint > assertion can just be dropped from both methods. That in turn means we > either drop the JavaThread assertion or else expand it to be > "JavaThread or VMThread" - but that itself seems unnecessarily > restrictive: why can't a GC thread use this code for example? So I > chose to drop it. > > That leaves the "if I'm a JavaThread I'm not _thread_blocked" > assertion. I don't see that this really guards against anything useful > (a JavaThread should be _thread_in_vm when executing this code ... > though there may be leaf calls where the thread is _thread_in_java). > So again I just dropped the assertion. > > So a long winded justification for dropping a few assertions. :) > > Thanks, > David > > From daniel.daugherty at oracle.com Wed Aug 5 21:47:35 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 5 Aug 2020 17:47:35 -0400 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> Message-ID: I'm peeking ahead to the next webrev... :-) > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ General comments: ? - check files for copyright year updates. src/hotspot/share/runtime/synchronizer.hpp ??? No comments. src/hotspot/share/runtime/synchronizer.cpp ??? L507: ? const markWord mark = obj->mark(); ??? L508: ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && obj->klass()->is_box()) { ??? L510: ??? return false; ??? L511: ? } ??????? The new bailout on L509-511 can move above L507. ??? L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass %s", p2i(obj()), obj->klass()->external_name()); ??????? This external_name() call does not have a ResourceMark. src/hotspot/share/logging/logTag.hpp ??? No comments. src/hotspot/share/oops/klass.hpp ??? No comments. src/hotspot/share/utilities/accessFlags.hpp ??? No comments. src/hotspot/share/runtime/globals.hpp ??? L814: ???????????? "0: off " ??????? Missing a ';' after "off". ??? L816: ???????????? "2: log message to stdout. ??????? Perhaps add "(by default)" after "stdout" or ??????? don't say where log output is at all. src/hotspot/share/runtime/arguments.cpp ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, true, LOG_TAGS(primitivewrappers)); ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp ??????? since it looks like logging output is configured to 'stdout'. ??????? I'm assuming that other log options to put output elsewhere ??????? are overridden by this code? src/hotspot/share/classfile/systemDictionary.cpp ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { ??????? nit - s/T_LONG+1/T_LONG + 1/ ??? L2191: _box_klasses[i]->set_prototype_header(markWord::prototype()); ??????? I assume we're keeping the prototype_header field when Biased Locking ??????? goes away? The reason I ask: ? ? ? ? static markWord prototype() { ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); ? ? ? ? } ??????? is because without Biased Locking, do we really need the prototype ??????? anymore? The initial markWord won't need possible variants... src/hotspot/share/jfr/metadata/metadata.xml ??? L69: ? ??? L3578: ??? __ bind(cont); ??? L3579: ??? // flag == EQ indicates success ??? L3580: ??? // flag == NE indicates failure ??????? If tbnz() branches to "cont" when we have a box class, what's ??????? the flag value set to (EQ or NE)? And what set that flag value? ??????? The reason I ask is I don't think tbnz() sets condition codes... src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp ??? No comments. src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp ??? No comments. src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp ??? int MacroAssembler::biased_locking_enter(Register lock_reg, ??????????? ? ?? ??????????????????????????? Register obj_reg, ?????????????????????????????? ? ?? ???????? Register swap_reg, ?????????????????????????????????? ? ?? ???? Register tmp_reg, ?????????????????????????????????? ? ?? ???? bool swap_reg_contains_mark, ?????????????????????????????????? ? ?? ???? Label& done, ??????????????????????????????????? ? ?? ??? Label* slow_case, BiasedLockingCounters* counters) { ??????? I think you've changed the only callers of biased_locking_enter() ??????? that cared about the return value with this changeset so it can ??????? be changed to a void function. src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp ??? No comments. src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), done); ??? L131: ? bind(done); ??? L132: ??? L133: ? // At this point flags are set as follows: ??? L134: ? //? EQ -> Success ??? L135: ? //? NE -> Failure, branch to slow path ??????? If tbnz() branches to "done" when we have a box class, what's ??????? the flag value set to (EQ or NE)? And what set that flag value? ??????? The reason I ask is I don't think tbnz() sets condition codes... src/hotspot/cpu/arm/interp_masm_arm.cpp ??? No comments. src/hotspot/cpu/arm/macroAssembler_arm.cpp ??? int MacroAssembler::biased_locking_enter(Register obj_reg, Register swap_reg, Register tmp_reg, ????????????????????? ? ?? ????????????????? bool swap_reg_contains_mark, ???????????????????????? ? ?? ?????????????? Register tmp2, ????????????????????????? ? ?? ????????????? Label& done, Label& slow_case, BiasedLockingCounters* counters) { ??????? I think you've changed the only callers of biased_locking_enter() ??????? that cared about the return value with this changeset so it can ??????? be changed to a void function. src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp src/hotspot/cpu/ppc/macroAssembler_ppc.cpp ??? No comments on the PPC code. src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp src/hotspot/cpu/s390/interp_masm_s390.cpp src/hotspot/cpu/s390/macroAssembler_s390.cpp ??? No comments on the S390 code. src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); ??????? What will this do with a 'noreg' value? (I need a refresher.) src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp ??? No comments. src/hotspot/cpu/x86/interp_masm_x86.cpp ??? No comments. src/hotspot/cpu/x86/macroAssembler_x86.cpp ??? int MacroAssembler::biased_locking_enter(Register lock_reg, ????????????????????????????????? ? ?? ????? Register obj_reg, ????????????????????????????????? ? ?? ????? Register swap_reg, ????????????????????????????????? ? ?? ????? Register tmp_reg, ?????????????????????????????????? ? ?? ???? Register tmp_reg2, ?????????????????????????????????????? ? ? ? bool swap_reg_contains_mark, ?????????????????????????????????? ? ?? ???? Label& done, ?????????????????????????????????? ? ?? ???? Label* slow_case, BiasedLockingCounters* counters) { ??????? I think you've changed the only caller of biased_locking_enter() ??????? that cared about the return value with this changeset so it can ??????? be changed to a void function. test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java ??? L30:? * @test SyncOnPrimitiveWrapperTest ??????? No parameter to @test directive. ??? L136: ??????? private static long sharedCounter = 0L; ??????? Since you don't do anything with sharedCounter other than increment it, ??????? can the compilers optimize it away? If it can be optimized away, does ??????? that mean that: ??????????? L142: ??????????????? synchronized (obj) { ??????? can also be optimized away? ??????? I don't think that: ??????????? L161: ??????????????? synchronized (sharedLock1) { ??????? can be optimized away because it is shared by multiple threads. test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java ??? Similar questions about 'counter' being optimized away. ??? Similar question about "synchronized (obj) {" being optimized away. Dan On 8/5/20 9:01 AM, Patricio Chilano wrote: > Hi Martin, > > On 8/5/20 4:47 AM, Doerr, Martin wrote: >> Hi Patricio, >> >> using 8 Byte load instructions for jint fields is a terrible coding >> style! >> Someone else may see it and use an 8 Byte store. Will result in great >> fun for debugging! >> >> And on Big Endian you will simply access the wrong bits. > Ah, of course! Those 32 bits will be in the wrong place when doing the > test. > >> Note that Big Endian Platforms are AIX, old linux ppc, s390, SPARC. I >> don't think you have tested on them. >> >>> We could remove the nested synchronized statements in the run() method >>> so that we don't generate that much logging. We could also lower >>> LOOP_COUNT. I think the issue is also because we are running LogTest >>> with multiple flag combinations. Not sure what we should touch first. >>> Maybe the synchronized statements, have only one or two and test that >>> first? >> Sounds like helpful ideas. Please go ahead and strip things down. > Great, I will send v3 later with those changes. > > Thanks Martin! > > Patricio >> Thanks for taking care of it. >> >> Best regards, >> Martin >> >> >>> -----Original Message----- >>> From: Patricio Chilano >>> Sent: Dienstag, 4. August 2020 20:26 >>> To: Doerr, Martin ; David Holmes >>> ; hotspot-runtime-dev at openjdk.java.net; >>> hotspot-jfr-dev at openjdk.java.net >>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>> wrappers >>> >>> Hi Martin, >>> >>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>> Hi Patricio, >>>> >>>>> Ok, I'll change it to movl + testl and test it out before sending v3. >>>> Thanks. I forgot to mention arm + aarch64. >>>> Aarch64 uses ldrw + tbnz. >>>> Arm 32 bit uses ldr_u32 + tbnz. >>>> >>>> I remember having seen the same mistake ?? >>>> And nobody noticed it on little Endian platforms. >>> Ok, I can use a tbnz instead of test and then a branch on arm32 and >>> aarch64. >>> Shouldn't a normal ldr in arm32 work fine? >>> Also in 64 bits (either x64 or aarch64) I don't see the issue of >>> using a >>> 64 bit load, besides the fact that we only care about the first 32 >>> bits. >>> Regardless of the endianness, aren't we masking out the upper part when >>> we do AND/TEST or if we test a bit in the 0-31 bit range? Otherwise it >>> seems it should have failed for one of those platforms in my tests. >>> >>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>> specify that and see if that fixes it? Let me know if that works >>>>> and I >>>>> can add it to the test. >>>> That seems to have an effect. But now I'm not patient enough to >>>> wait for >>> the test to finish. >>>> Maybe the problem is that I'm using slow debug builds. >>>> But is there a chance to make the test quicker without losing coverage >>> significantly? >>>> I think the test is too slow this way to run it on a regular basis. >>>> We'd need to >>> spend dedicated machines for it. >>> We could remove the nested synchronized statements in the run() method >>> so that we don't generate that much logging. We could also lower >>> LOOP_COUNT. I think the issue is also because we are running LogTest >>> with multiple flag combinations. Not sure what we should touch first. >>> Maybe the synchronized statements, have only one or two and test that >>> first? >>> >>> >>> Thanks, >>> Patricio >>>> Best regards, >>>> Martin >>>> >>>> >>>>> -----Original Message----- >>>>> From: Patricio Chilano >>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>> To: Doerr, Martin ; David Holmes >>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>> hotspot-jfr-dev at openjdk.java.net >>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>> wrappers >>>>> >>>>> Hi Martin, >>>>> >>>>> Thanks for fixing PPC and taking care of s390! >>>>> >>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>> Hi Patricio, >>>>>> >>>>>> I suggest to use movl + testl for checking the access flag as for >>>>>> other >>> access >>>>> flags on x64. >>>>> Ok, I'll change it to movl + testl and test it out before sending v3. >>>>> >>>>>> New version for PPC64 and s390 see below. >>>>>> >>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>> expected. >>>>> However, I'm getting timeout issues: >>>>>> timed out (timeout set to 120000ms, elapsed time including timeout >>>>> handling was 122372ms) >>>>>> Can we provide more time? >>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>> specify that and see if that fixes it? Let me know if that works >>>>> and I >>>>> can add it to the test. >>>>> >>>>> Thanks, >>>>> Patricio >>>>>> Best regards, >>>>>> Martin >>>>>> >>>>>> >>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>> 10:03:57 2020 +0200 >>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp???? Tue Aug 04 >>>>> 16:04:31 2020 +0200 >>>>>> @@ -107,8 +107,8 @@ >>>>>> >>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>> ??????? load_klass(Rscratch, Roop); >>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); >>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>> +??? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>> Rscratch); >>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>> ??????? bne(CCR0, slow_case); >>>>>> ????? } >>>>>> >>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>> 10:03:57 2020 +0200 >>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp??????? Tue Aug 04 >>>>> 16:04:31 2020 +0200 >>>>>> @@ -912,8 +912,8 @@ >>>>>> >>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>> ????????? load_klass(tmp, object); >>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>> +????? testbitdi(CCR0, R0, tmp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>> ????????? bne(CCR0, slow_case); >>>>>> ??????? } >>>>>> >>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>> 10:03:57 2020 +0200 >>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp??????? Tue Aug 04 >>>>> 16:04:31 2020 +0200 >>>>>> @@ -2838,8 +2838,8 @@ >>>>>> >>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>> ??????? load_klass(temp, oop); >>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>> +??? testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>> ??????? bne(CCR0, cont); >>>>>> ????? } >>>>>> >>>>>> diff -r 77852e129401 >>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp?? Tue Aug 04 >>>>> 10:03:57 2020 +0200 >>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp?? Tue Aug 04 >>>>> 16:04:31 2020 +0200 >>>>>> @@ -91,6 +91,12 @@ >>>>>> ????? // Save object being locked into the BasicObjectLock... >>>>>> ????? z_stg(obj, Address(disp_hdr, >>> BasicObjectLock::obj_offset_in_bytes())); >>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>> +??? z_btrue(slow_case); >>>>>> +? } >>>>>> + >>>>>> ????? if (UseBiasedLocking) { >>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>> Z_R0_scratch, done, >>>>> &slow_case); >>>>>> ????? } >>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 10:03:57 >>>>> 2020 +0200 >>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>> 16:04:31 >>>>> 2020 +0200 >>>>>> @@ -1000,6 +1000,12 @@ >>>>>> ????? // Load markWord from object into displaced_header. >>>>>> ????? z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), >>>>>> object); >>>>>> >>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>> +??? z_btrue(slow_case); >>>>>> +? } >>>>>> + >>>>>> ????? if (UseBiasedLocking) { >>>>>> ??????? biased_locking_enter(object, displaced_header, Z_R1, >>>>>> Z_R0, done, >>>>> &slow_case); >>>>>> ????? } >>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>> 10:03:57 2020 +0200 >>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp????? Tue Aug 04 >>>>> 16:04:31 2020 +0200 >>>>>> @@ -3358,6 +3358,12 @@ >>>>>> ????? // Load markWord from oop into mark. >>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>> >>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>> +??? z_btrue(done); >>>>>> +? } >>>>>> + >>>>>> ????? if (try_bias) { >>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, Z_R0, >>>>>> done); >>>>>> ????? } >>>>>> > From ioi.lam at oracle.com Wed Aug 5 22:26:24 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 5 Aug 2020 15:26:24 -0700 Subject: RFR(S) 8249276 CDS archived objects must have "neutral" markwords In-Reply-To: <37245599-dfb8-0481-6c74-89cffdc5b457@oracle.com> References: <5d5b4b17-a40f-b0f4-fa50-fdd783def9d4@oracle.com> <37245599-dfb8-0481-6c74-89cffdc5b457@oracle.com> Message-ID: <7a81547c-8221-da80-2d1a-edb42f4078fc@oracle.com> Hi David & Coleen, Thanks for the review. I have created a new version: http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ I created a new test case (LockDuringDump), and incorporated David's fix for JDK-8250606 [1] during my testing: + Without my patch, LockDuringDumpApp would crash at runtime, because ? we have an invalid markword for the Helper.class object. + Without the JDK-8250606 patch, we would assert during CDS dump time. I also removed the assert in HeapShared::oop_hash() since it's irrelevant after JDK-8250606. I intend to push my patch after JDK-8250606 is integrated. ------- More comments below On 8/3/20 6:13 PM, David Holmes wrote: > Hi Ioi, > > On 27/07/2020 3:20 pm, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8249276 >> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v01/ >> >> >> Please review this change (initial patch provided by David Holmes; I >> rearranged it a little and added more comments). >> >> We reinitialize the markWord of all archived objects to remove any >> side effect of GC/locking/etc during "java -Xshare:dump". > > Changes look good to me (naturally :) ). A couple of nits: > > ?? int hash_archived = archived_oop->identity_hash(); > ?? assert(hash_original == hash_archived, "Different hash codes: > original %x, archived %x", hash_original, hash_archived); > > the has_archived setting should be in ifdef DEBUG. > Fixed > 42???????????????????? Object.class.wait(); > ? 43???????????????? } > ? 44???????????????? System.out.println("Huh?? notified??"); > > That could be a spurious wakeup. You could just use Thread.sleep with > a suitably long sleep time. that said I'm not sure how the test is > intended to operate. If premain returns after starting the thread > there is no guarantee the thread will start executing and lock the > object before the dump actually commences. Even the sighting of "Let's > wait ....." is not sufficient to guarantee that unless you move it > inside the sync block. > I have fixed the test case so that the "Let's hold the lock" message is printed after the lock is held, and the lock is never released afterwards. I also added synchronization to make sure the premain() method doesn't return until the child thread has held the lock. The log file shows that theses messages are printed before CDS tries to load classes from the classlist: [0.005s][info][cds] Allocated shared space: 3221225472 bytes at 0x0000000800000000 inside LockDuringDumpAgent: jdk.internal.loader.ClassLoaders$AppClassLoader at 707f7052 Helper class is initialized Class = class LockDuringDumpApp$Helper Let's hold the lock on Helper.class forever ..... Thread has started [0.145s][info][cds] Loading classes to share ... >> (David mentioned in the bug comments about assertions inside the >> identity_hash() call, but maybe this should be fixed in a different >> bug?) > > Yes those assertions are being removed under: > > https://bugs.openjdk.java.net/browse/JDK-8250606 > Thanks - Ioi ------- [1] https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/040985.html From david.holmes at oracle.com Wed Aug 5 22:31:56 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 6 Aug 2020 08:31:56 +1000 Subject: RFR (s): 8250606: Remove unnecessary assertions in ObjectSynchronizer FastHashcode and inflate In-Reply-To: <17eef440-3575-73f5-1413-0a25da6a13df@oracle.com> References: <3e4c931a-593b-0336-97b9-3adc4de96062@oracle.com> <17eef440-3575-73f5-1413-0a25da6a13df@oracle.com> Message-ID: Hi Dan, Thanks for taking a look at this. On 6/08/2020 4:08 am, Daniel D. Daugherty wrote: > On 8/5/20 3:58 AM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8250606 >> webrev: http://cr.openjdk.java.net/~dholmes/8250606/webrev/ > > src/hotspot/share/runtime/synchronizer.cpp > ??? L1010: ????? // Relaxing assertion for bug 6320749. > ??? L1011: ????? assert(Universe::verify_in_progress() || > ??? L1012: ???????????? !SafepointSynchronize::is_at_safepoint(), > ??? L1013: ???????????? "biases should not be seen by VM thread here"); > ??????? This assertion is similar and tagged with the same bug ID > (6320749) > ??????? as the ones that you removed. Granted that the Biased Locking code > ??????? is on the way out, but... > > ??????? I'm fine if you plan to leave this one for Biased Locking removal. Yeah I confess I had my biased-locking filter on when looking at this part of the code. You've actually exposed a bit of a mess. :) It is unclear why the same !at_safepoint assertion was initially added to the biased-locking revocation chunk of code given the same assertion would then be executed after it. But I think it is more to do with the line following the assertion (original code): assert(!SafepointSynchronize::is_at_safepoint(), "biases should not be seen by VM thread here"); BiasedLocking::revoke_and_rebias(hobj, false, JavaThread::current()); If we are at a safepoint in the VMThread then JavaThread::current() will assert! So even after the assertion was apparently relaxed with the verify_in_progress() check, we could never have hit this code as the JavaThread::current() would still assert. So I can (and should) remove the assertion here as well (CDS dump could reach this in theory), but I also need to correct this current code: - BiasedLocking::revoke(hobj, JavaThread::current()); + BiasedLocking::revoke(hobj, self); as re-manifesting the current thread is pointless. That said the above code (nor the original) also can't execute at a safepoint: void BiasedLocking::revoke(Handle obj, TRAPS) { assert(!SafepointSynchronize::is_at_safepoint(), "must not be called while at safepoint"); so even larger changes will be needed to make this code work at a safepoint, and that is going well beyond the intended scope of this particular bug. :( I'll need to get back to you. I also need to check our GC verify tests to see how we manage to apparently never encounter a biased-locked object, otherwise we would have seen this inconsistency in the code. Thanks, David ----- > Thumbs up.? Thanks for digging up the history behind these very old > asserts. Thanks, David > Dan > > >> >> tl;dr I removed some assertions that were deemed unnecessary after >> interfering with other work. :) >> >> ObjectSynchronizer::inflate had the following assertion added back in >> JDK 6 under JDK-5030359: >> >> ObjectMonitor * ObjectSynchronizer::inflate (oop object) { >> ? // Inflate mutates the heap ... >> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >> >> That same change added FastHashcode which had the following assertions: >> >> intptr_t ObjectSynchronizer::FastHashCode (Thread * Self, oop obj) { >> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >> ? assert (Self->is_Java_thread(), "invariant") ; >> ? assert (((JavaThread *)Self)->thread_state() != _thread_blocked, >> "invariant") ; >> >> These methods had assertions that they were not executed at a >> safepoint in case the modification of the markWord could cause >> interference with GC. >> >> JDK-6320749 then relaxed the assertions to allow for GC verification - >> which does happen at a safepoint and so obviously not in a JavaThread. >> >> The FastHashCode assertions were further relaxed in JDK-8015086 to >> allow the CDS dump VM safepoint operation to call it. But FastHashCode >> can also call inflate so that was a mismatch in assertions just >> waiting to happen - which is how I hit this in a VM that always forces >> inflation :) >> >> I initially intended to extend the FastHashCode assertion to match >> inflate, but then a question arose about the safety of the CDS dumping >> code using FastHashcode which showed that the code is in fact safe. >> Notwithstanding what the state of the world may have been in the JDK 6 >> timeframe, there is now no potential interference possible between >> execution of these methods at an arbitrary safepoint and any >> concurrent GC use of the markWord. So the original !is_at_safepoint >> assertion can just be dropped from both methods. That in turn means we >> either drop the JavaThread assertion or else expand it to be >> "JavaThread or VMThread" - but that itself seems unnecessarily >> restrictive: why can't a GC thread use this code for example? So I >> chose to drop it. >> >> That leaves the "if I'm a JavaThread I'm not _thread_blocked" >> assertion. I don't see that this really guards against anything useful >> (a JavaThread should be _thread_in_vm when executing this code ... >> though there may be leaf calls where the thread is _thread_in_java). >> So again I just dropped the assertion. >> >> So a long winded justification for dropping a few assertions. :) >> >> Thanks, >> David >> >> > From suenaga at oss.nttdata.com Thu Aug 6 00:09:35 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 6 Aug 2020 09:09:35 +0900 Subject: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: <58a6ddb7-2aa0-595b-e999-2e4277f9e7c5@oracle.com> References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <58a6ddb7-2aa0-595b-e999-2e4277f9e7c5@oracle.com> Message-ID: <488d9fdc-60cd-732b-b5fc-185cb0846868@oss.nttdata.com> Thanks Erik! Yasumasa On 2020/08/05 21:30, Erik Joelsson wrote: > Build change is ok. > > /Erik > > On 2020-08-05 02:36, Yasumasa Suenaga wrote: >> CC'ing build-dev >> >> On 2020/08/05 18:30, Yasumasa Suenaga wrote: >>> Hi Matthias, >>> (CC'ed build-dev due to makefile change) >>> >>> I uploaded new webrev. Could you review it? >>> >>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/ >>> >>> On 2020/08/05 16:26, Baesken, Matthias wrote: >>>> Hello, >>>> >>>>>> I also checked the? hserr on another Hyper-V guest and still get the expected " HyperV virtualization detected"?? (which is good ?? ). >>>>>> >>>>>> >>>>>> However on a host, I would still prefer to get some output (probably different than the guest output now) . >>>>> >>>>> What message do you expect? How about "Hyper-V role detected" ? >>>> >>>> Thats okay with me? (I think some people here say "Hyper-V host" so that might be an idea as well). >>> >>> I said "Hyper-V host" in previous mails, but it is not strictly correct. In Hyper-V, it should be "root partision". However it's a little difficult to understand, so I choose "role" in this place. >>> >>> >>>>>> Regarding the usage of >>>>>> >>>>>> #pragma comment(lib, "wbemuuid.lib") >>>>>> >>>>>> This seem to add an additional lib dependency ,? should some people from build/deploy? area comment on this maybe ? >>>>>> (not saying it is a very bad thing ) >>> >>> I modified libraries.m4. >>> wbemuuid.lib is needed for WMI call. >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>>>> We add wbemuuid.lib to make/autoconf/libraries.m4 . >>>>> I will change it in next webrev, and I will CCing to build-dev on next turn. >>>> >>>> >>>> Let's see what they have to say. >>>> >>>> Best regards, Matthias >>>> >>>> From felix.yang at huawei.com Thu Aug 6 01:13:44 2020 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Thu, 6 Aug 2020 01:13:44 +0000 Subject: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic In-Reply-To: <4d1f7179-2cad-620d-f833-949e8b39ee06@redhat.com> References: <654e8a2e-87a1-9c3d-4c58-36b4215bb0ad@redhat.com> <5350e745-87af-ce34-8711-2b652a38e323@redhat.com> <4d1f7179-2cad-620d-f833-949e8b39ee06@redhat.com> Message-ID: Hi, > -----Original Message----- > From: Andrew Haley [mailto:aph at redhat.com] > Sent: Tuesday, August 4, 2020 6:27 PM > To: Yangfei (Felix) ; hotspot-runtime- > dev at openjdk.java.net > Cc: aarch64-port-dev at openjdk.java.net > Subject: Re: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic > > On 8/3/20 2:18 AM, Yangfei (Felix) wrote: > > OK. I simply comment out the following code changes in > vm_version_aarch64.cpp: > > > > 393 if (UseSHA && (auxv & HWCAP_SHA512)) { > > 394 // Do not auto-enable UseSHA512Intrinsics until it has been fully > tested on hardware > > 395 // if (FLAG_IS_DEFAULT(UseSHA512Intrinsics)) { > > 396 // FLAG_SET_DEFAULT(UseSHA512Intrinsics, true); > > 397 // } > > 398 } else if (UseSHA512Intrinsics) { > > > > Once this has been fully tested on real hardware, it will be easy to auto- > enable. > > > > New webrev: http://cr.openjdk.java.net/~fyang/8165404/webrev.01/ > > Looks good, thanks. Thanks for reviewing this. > My apologies, there is another thing I should have mentioned: please add > test cases for the new instructions to aarch64-asmtest.py and regenerate the > code between > > // BEGIN Generated code -- do not edit > // Generated by aarch64-asmtest.py > > and > > // END Generated code -- do not edit > > I'm trying to be more strict about making sure all of the instructions get > tested. Sure. I have added tests for sha512 crypto instructions in this script and regenerated the code. Tier1-3 tested with an aarch64 fastdebug build on my aarch64 linux host to cover this part. Updated webrev: http://cr.openjdk.java.net/~fyang/8165404/webrev.02/ Is it OK? Felix From ioi.lam at oracle.com Thu Aug 6 06:21:21 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 5 Aug 2020 23:21:21 -0700 Subject: RFR JDK-8251213 [TESTBUG] CDS tests shouldn't write output files into test.classes directory Message-ID: https://bugs.openjdk.java.net/browse/JDK-8251213 http://cr.openjdk.java.net/~iklam/jdk16/8251213-cds-shouldnt-write-test-classes-dir.v01/ Many of the CDS tests would write output files to the directory given by System.getProperty("test.classes"). I changed them to write to the scratch directory, i.e., System.getProperty("user.dir", "."). This way, it's easier to examine the output of the tests. Please start the review with CDSTestUtils.java. I validated the change by going to the jtreg work directory and: ??? $ find $(find . -name classes) -type f | grep -v '[.]class$' | wc -l ??? 0 (There used to be a lot of .jsa, .classlist, .stdout, etc). Executed all the tests locally on Linux. Running mach5 tiers1-4. Thanks - Ioi From david.holmes at oracle.com Thu Aug 6 07:33:17 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 6 Aug 2020 17:33:17 +1000 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> Message-ID: Hi Gerard, On 6/08/2020 12:49 am, gerard ziemski wrote: > hi all, > > Please review this new test that replaces a closed (and flawed) one, > which was causing time out issues on Mac OS X. > > The new test takes the main characteristic of the old test, which is > creating a large number of threads, but in a robust and deterministic > way - we create as many threads as the process allows and expect the VM > to throw OOM exception to let us know when we reach the limit - then we > perform a simple work unit for each thread (setting its name and > verifying it back), then simply exit. Basing the new test around the existing thread stress tests was not unreasonable but I don't think we're actively adding to those tests. I would check with Igor whether it is better to just add this under runtime/thread rather than vmTestBase. > To make it more robust we use the Thread.wait(TIMEOUT) instead of the > Thread.wait() as in the original. It doesn't actually make it more robust as we need to ensure the TIMEOUT is compatible with the jtreg timeout that is applied, which is itself subject to a timeout adjustment factor to allow for slower machines. If you just use await() you don't need to care about timeouts or time value parsing etc and it simplifies the test. Timeouts are best handled at the jtreg level. Overall copying the test from the existing nsk stress tests makes it more complicated than necessary and also perpetuates some poor design choices made in those tests (or at least choices that may have made sense for a single known testing environment but which do not generalise). In particular the goal of this test should be to check that hitting the OS limit does not cause a crash or other strange behaviour; it isn't about ensuring a minimum number of threads can be created by the JVM. By setting an artificial minimum you just set the test up to fail in a new test environment (such as a for a small, memory constrained device). I've said before that calling yield() is unnecessary and may not even do anything useful, but if you insist of using it it is a static method so should be invoked as Thread.yield() (all static Thread methods apply to the current thread). > To keep things nice and simple we use CountDownLatch objects for > synchronization between the main and worker threads (thank you David). 173 for (int i = limit; i < NUMBER_OF_THREADS; i++) { 174 doneSignal.countDown(); 175 } The main thread shouldn't call countDown on the doneSignal. It could also simply join() them rather than using the doneSignal. Thanks, David ----- > bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 > open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev1 > testing: passes Mach5 hs-tier1,2,3,4,5 > > > cheers From aph at redhat.com Thu Aug 6 11:41:39 2020 From: aph at redhat.com (Andrew Haley) Date: Thu, 6 Aug 2020 12:41:39 +0100 Subject: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic In-Reply-To: References: <654e8a2e-87a1-9c3d-4c58-36b4215bb0ad@redhat.com> <5350e745-87af-ce34-8711-2b652a38e323@redhat.com> <4d1f7179-2cad-620d-f833-949e8b39ee06@redhat.com> Message-ID: <7cedaced-9628-32b3-6d01-beae760fc964@redhat.com> On 8/6/20 2:13 AM, Yangfei (Felix) wrote: > I have added tests for sha512 crypto instructions in this script and regenerated the code. > Tier1-3 tested with an aarch64 fastdebug build on my aarch64 linux host to cover this part. > Updated webrev: http://cr.openjdk.java.net/~fyang/8165404/webrev.02/ > Is it OK? Yes, thanks. It's a bit annoying that the indentation seems to change so frequently. I'm wondering if perhaps some people have been running this on Windows, which has different tab sizes, but that seems very unlikely, given that this is a Linux port. But never mind, that's not your problem. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From jamsheed.c.m at oracle.com Thu Aug 6 12:07:40 2020 From: jamsheed.c.m at oracle.com (Jamsheed C M) Date: Thu, 6 Aug 2020 17:37:40 +0530 Subject: RFR: 8249451: Unconditional exceptions clearing logic in compiler code should honor Async Exceptions Message-ID: Hi all, JBS: https://bugs.openjdk.java.net/browse/JDK-8249451 webrev: http://cr.openjdk.java.net/~jcm/8249451/webrev.00/ testing : mach1-5(links in jbs) While working on JDK-8246381 it was noticed that compilation request path clears all exceptions(including async) and doesn't propagate[1]. Fix: patch restores the propagation behavior for the probable async exceptions. Compilation request path propagate exception as in [2]. MDO and MethodCounter doesn't expect any exception other than metaspace OOM(added comments). Deoptimization path doesn't clear probable async exceptions and take unpack_exception path for non uncommontraps. Added java_lang_InternalError to well known classes. Request for review. Best Regards, Jamsheed [1] w.r.t changes done for JDK-7131259 [2] ??? (a) ??? -----> c1_Runtime1.cpp/interpreterRuntime.cpp/compilerRuntime.cpp ????? | ?????? ----- compilationPolicy.cpp/tieredThresholdPolicy.cpp ???????? | ????????? ------ compileBroker.cpp ??? (b) ??? Xcomp versions ??? ------> compilationPolicy.cpp ?????? | ??????? ------> compileBroker.cpp ??? (c) ??? Direct call to? compile_method in compileBroker.cpp ??? JVMCI bootstrap, whitebox, replayCompile. From christoph.dreis at freenet.de Thu Aug 6 13:56:22 2020 From: christoph.dreis at freenet.de (Christoph Dreis) Date: Thu, 06 Aug 2020 15:56:22 +0200 Subject: Performance of instanceof with interfaces is multiple times slower than with classes In-Reply-To: References: <6A5EB60C-88C2-4160-9C55-A82B8FA95ECF@oracle.com> Message-ID: Hi John, I haven't heard back from you after my last mail. Do you have any further thoughts on the topic? I'd appreciate any input. Cheers, Christoph ==== ORIGINAL MESSAGE ==== ?Hi John, thanks for your answer. I think I need to elaborate a bit more on the background of this question. I was looking at Set.copyOf() and noticed that it copies the collection into a HashSet if it isn?t an immutable collection. Yet, this doesn?t seem to be necessary in case the passed collection is already a Set, which I think is not that uncommon. So I changed it slightly to the following: static Set copyOf(Collection coll) { if (coll instanceof ImmutableCollections.AbstractImmutableSet) { return (Set)coll; } else if (coll instanceof Set) { // that is the new bit return (Set)Set.of(coll.toArray()); } else { return (Set)Set.of(new HashSet<>(coll).toArray()); } } Doing that showed the wanted performance boost for Sets. When passed a Collections.emptySet() this seems to make it allocation free even. But for other inputs, like a List, it degraded drastically. I tried to benchmark this change with the following benchmark: @BenchmarkMode(Mode.AverageTime) @OutputTimeUnit(TimeUnit.NANOSECONDS) @State(Scope.Thread) public class MyBenchmark { @State(Scope.Thread) public static class BenchmarkState { private Collection emptySet = Collections.emptySet(); private Collection emptyList = Collections.emptyList(); } @Benchmark public Set testCopyEmptySet(BenchmarkState state) { return Set.copyOf(state.emptySet); } @Benchmark public Set testCopyEmptyList(BenchmarkState state) { return Set.copyOf(state.emptyList); } } Which showed the following results Before Benchmark Mode Cnt Score Error Units MyBenchmark.testCopyEmptyList avgt 10 5,436 ? 0,286 ns/op MyBenchmark.testCopyEmptyList:?gc.alloc.rate.norm avgt 10 48,002 ? 0,001 B/op MyBenchmark.testCopyEmptySet avgt 10 5,319 ? 0,472 ns/op MyBenchmark.testCopyEmptySet:?gc.alloc.rate.norm avgt 10 48,002 ? 0,001 B/op After Benchmark Mode Cnt Score Error Units MyBenchmark.testCopyEmptyList avgt 10 24,835 ? 0,621 ns/op MyBenchmark.testCopyEmptyList:?gc.alloc.rate.norm avgt 10 48,004 ? 0,001 B/op MyBenchmark.testCopyEmptySet avgt 10 2,494 ? 0,179 ns/op MyBenchmark.testCopyEmptySet:?gc.alloc.rate.norm avgt 10 ? 10?? B/op I was surprised to see such a big difference, so I played around with it. My first suspect was inlining problems with the new solution, but that didn't turn out to be true. I ended up pinning it down to the instanceof check, because when I changed the additional instanceof check to AbstractSet for a test I saw no regression for the list case and still saw my improvement for the empty set. All I was trying to do with the provided benchmark in my first mail was to isolate the "problem" and to showcase that this seems to be independent from hierarchy, which exists in the collection API test above. In retrospect, this didn't help here, which I'm sorry about. > You ask ?point me to the code? which suggests you haven?t looked at the code yet. When you do (and I hope you do!) you will find that there are many, many algorithms and decisions in the JVM that affect the treatment of runtime > type tests. You could start by grepping around for ?subtype? (case insensitive) in the *.[ch]pp and *.ad files under src/hotspot. I have in fact looked at the code already before the first mail and found emit_typecheck_helper in src/hotspot/cpu/x86/c1_LIRAssembler_x86.cpp, which I think is involved - please correct me if I'm wrong. And MacroAssembler::check_klass_subtype_fast_path in src/hotspot/cpu/x86/macroAssembler_x86.cpp etc. Depending on the architecture (x86, aarch etc.) of course. Like you said, there are many things involved and I was hoping for a good starting point really from an experienced developer or a rough explanation what might be involved in the example. I understand now that there doesn't seem to be a "simple" explanation. Unfortunately, the C++ side of things is not as good documented as the Java side of things and that makes it relatively complicated for people like me who aren't familiar with the code to follow the flows sometimes. I can assure you: I do want to look at code, hence the - admittedly clumsy - question. I didn't do a great job of explaining that I looked upfront before asking, so that's on me. Nonetheless, I hope this sheds some light on the background of this and I hope the collection case earlier justifies as a more real-life example. Cheers, Christoph Von: John Rose Datum: Samstag, 18. Juli 2020 um 01:13 An: Christoph Dreis Cc: hotspot-runtime-dev Betreff: Re: Performance of instanceof with interfaces is multiple times slower than with classes On Jul 15, 2020, at 2:08 AM, Christoph Dreis wrote: Could you enlighten me what the cause for this is and maybe point me to the code where this is done? A microbenchmark like that doesn?t demonstrate much. The JVM optimizations which are designed for real applications may or may not apply to this code. They will certainly apply differently in real-life application execution. In real life, class hierarchies are more complex, and so the JVM expects to do more work telling types apart; at that point differences between single inheritance and multiple inheritance requires differing algorithms with different complexities and costs. Also, in real life, inputs to such hot paths are not just all one type (as in your micro), which is another reason you get into a different set of costs and complexities. If this were related to a real-world application, or a realistic benchmark, I might get assembly code for the hot paths in the two methods, and see what the CPU is doing that makes a difference in speed. Then I might meditate on what decisions the JVM made to choose that code, and see if there is an improvement possible. You ask ?point me to the code? which suggests you haven?t looked at the code yet. When you do (and I hope you do!) you will find that there are many, many algorithms and decisions in the JVM that affect the treatment of runtime type tests. You could start by grepping around for ?subtype? (case insensitive) in the *.[ch]pp and *.ad files under src/hotspot. Is this maybe even a bug/regression? Can we maybe do something to improve the interface case? I will stick my neck out to say that microbenchmarks alone by definition never demonstrate performance regressions or bugs. Yes, we can always do something to improve it. Improving this micro could involve attempting to treat trivial interface hierarchies like trivial class hierarchies: But who would this benefit? Our investment of effort is surely driven by a hope to spend limited effort to get the most benefit on real applications. ? John From thomas.stuefe at gmail.com Thu Aug 6 14:01:44 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 6 Aug 2020 16:01:44 +0200 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info Message-ID: Hi, may I please have reviews for this addition: JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 Webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs-err/webrev.00/webrev/ When analysing a customer memory leak recently I found for all the information we have in the error files we miss some really fundamental statistics like process virtual size and rss. Either that or I am blind :) In case I am not blind this patch adds those statistics. I decided against dumping proc files wholesale to the hs-err file, I just print out what I really find useful. Note that among other things I print out the total size of outstanding heap allocations. Note that still the best way to do this is via mallinfo, and that means the returned value is int and may wrap, see coding and comment. Even with this caveat though this info is still very useful. I did a number of manual tests, the numbers seem legit. Looks like this: Process Memory: Virtual Size: 7494372K (peak: 7559908K) Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: 44804K, shmem: 0K) Swapped out: 0K C-Heap outstanding allocations: 37845K Here I simulate a C-Heap memory leak: Process Memory: Virtual Size: 9062260K (peak: 9118772K) Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, file: 44904K, shmem: 0K) Swapped out: 0K C-Heap outstanding allocations: 1291984K Same leak, with the mallinfo value wrapped around: Process Memory: Virtual Size: 12343352K (peak: 12445556K) Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, file: 45032K, shmem: 0K) Swapped out: 0K C-Heap outstanding allocations: 15454K (may have wrapped) Thanks, Thomas From thomas.stuefe at gmail.com Thu Aug 6 15:07:32 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 6 Aug 2020 17:07:32 +0200 Subject: RFR(xxs): 8251257: NMT: jcmd VM.native_memory scale=1 crashes target VM Message-ID: Hi, very simple small fix. jcmd VM.native_memory allows for scale=1 to print exact byte values, but crashes when printing. JBS: https://bugs.openjdk.java.net/browse/JDK-8251257 webrev: http://cr.openjdk.java.net/~stuefe/webrevs/8251257-nmt-scale-1-crash/webrev.00/webrev/ Thanks, Thomas From zgu at redhat.com Thu Aug 6 15:26:32 2020 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 6 Aug 2020 11:26:32 -0400 Subject: RFR(xxs): 8251257: NMT: jcmd VM.native_memory scale=1 crashes target VM In-Reply-To: References: Message-ID: Other than updating copyright year, looks good to me. -Zhengyu On 8/6/20 11:07 AM, Thomas St?fe wrote: > Hi, > > very simple small fix. > > jcmd VM.native_memory allows for scale=1 to print exact byte values, but > crashes when printing. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251257 > webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8251257-nmt-scale-1-crash/webrev.00/webrev/ > > Thanks, Thomas > From thomas.stuefe at gmail.com Thu Aug 6 15:28:50 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 6 Aug 2020 17:28:50 +0200 Subject: NMT does not seem to count malloc correctly Message-ID: Hi, maybe I am missing something fundamental, or there is something wrong with NMT. As an experiment for an unrelated issue, I created a C-Heap leak by simply commenting out the ::free() call inside os::free(): --- a/src/hotspot/share/runtime/os.cpp Thu Aug 06 17:08:02 2020 +0200 +++ b/src/hotspot/share/runtime/os.cpp Thu Aug 06 17:08:08 2020 +0200 @@ -804,7 +804,7 @@ size_t size = guarded.get_user_size(); inc_stat_counter(&free_bytes, size); membase = guarded.release_for_freeing(); - ::free(membase); +// ::free(membase); #else void* membase = MemTracker::record_free(memblock, MemTracker::tracking_level()); ::free(membase); As expected, this leads quickly to a very high memory usage. In my tiny example (which really does not matter) I quickly end up with ~4G RSS, and the glibc reports about 3G outstanding allocations: Process Memory: Virtual Size: 11136852K (peak: 11136852K) Resident Set Size: 4057860K (peak: 4057860K) (anon: 4013076K, file: 44784K, shmem: 0K) Swapped out: 0K C-Heap outstanding allocations: 3070399K However, NMT reports just about 600M committed memory (into this number should fall all outstanding allocations done with os::malloc() AFAICS): thomas at mainframe:~$ jjjcmd Interl VM.native_memory scale=1 19441: Native Memory Tracking: Total: reserved=6227876720, committed=626285424 .... And the malloc information adds up to about only 68M: thomas at mainframe:~$ jjjcmd Interl VM.native_memory scale=1 | grep malloc (malloc=4044913 #55107) (malloc=38688 #146) (malloc=10812522 #258557) (malloc=18157899 #3629) (malloc=42496 #311) (malloc=216 #7) (malloc=594920 #1788) (malloc=27304 #2) (malloc=2781872 #60716) (malloc=543008 #5144) (malloc=200416) (malloc=152 #7) (malloc=6788 #214) (malloc=18977 #485) (malloc=994280 #2797) (malloc=153248 #744) (malloc=2152 #12) I must be missing something obvious here. What is it? Thank you, Thomas From thomas.stuefe at gmail.com Thu Aug 6 15:29:12 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 6 Aug 2020 17:29:12 +0200 Subject: RFR(xxs): 8251257: NMT: jcmd VM.native_memory scale=1 crashes target VM In-Reply-To: References: Message-ID: Thanks Zhengyu. On Thu, Aug 6, 2020 at 5:26 PM Zhengyu Gu wrote: > Other than updating copyright year, looks good to me. > > -Zhengyu > > On 8/6/20 11:07 AM, Thomas St?fe wrote: > > Hi, > > > > very simple small fix. > > > > jcmd VM.native_memory allows for scale=1 to print exact byte values, but > > crashes when printing. > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251257 > > webrev: > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251257-nmt-scale-1-crash/webrev.00/webrev/ > > > > Thanks, Thomas > > > > From zgu at redhat.com Thu Aug 6 15:38:31 2020 From: zgu at redhat.com (Zhengyu Gu) Date: Thu, 6 Aug 2020 11:38:31 -0400 Subject: NMT does not seem to count malloc correctly In-Reply-To: References: Message-ID: By commenting out ::free(), you created real memory leak, but not from NMT perspective, because it already adjusted internal counting to reflect os:free(). -Zhengyu On 8/6/20 11:28 AM, Thomas St?fe wrote: > Hi, > > maybe I am missing something fundamental, or there is something wrong with > NMT. > > As an experiment for an unrelated issue, I created a C-Heap leak by simply > commenting out the ::free() call inside os::free(): > > --- a/src/hotspot/share/runtime/os.cpp Thu Aug 06 17:08:02 2020 +0200 > +++ b/src/hotspot/share/runtime/os.cpp Thu Aug 06 17:08:08 2020 +0200 > @@ -804,7 +804,7 @@ > size_t size = guarded.get_user_size(); > inc_stat_counter(&free_bytes, size); > membase = guarded.release_for_freeing(); > - ::free(membase); > +// ::free(membase); > #else > void* membase = MemTracker::record_free(memblock, > MemTracker::tracking_level()); > ::free(membase); > > As expected, this leads quickly to a very high memory usage. In my tiny > example (which really does not matter) I quickly end up with ~4G RSS, and > the glibc reports about 3G outstanding allocations: > > Process Memory: > Virtual Size: 11136852K (peak: 11136852K) > Resident Set Size: 4057860K (peak: 4057860K) (anon: 4013076K, file: 44784K, > shmem: 0K) > Swapped out: 0K > C-Heap outstanding allocations: 3070399K > > > However, NMT reports just about 600M committed memory (into this number > should fall all outstanding allocations done with os::malloc() AFAICS): > > thomas at mainframe:~$ jjjcmd Interl VM.native_memory scale=1 > > 19441: > > Native Memory Tracking: > > Total: reserved=6227876720, committed=626285424 > > .... > > And the malloc information adds up to about only 68M: > > thomas at mainframe:~$ jjjcmd Interl VM.native_memory scale=1 | grep malloc > (malloc=4044913 #55107) > (malloc=38688 #146) > (malloc=10812522 #258557) > (malloc=18157899 #3629) > (malloc=42496 #311) > (malloc=216 #7) > (malloc=594920 #1788) > (malloc=27304 #2) > (malloc=2781872 #60716) > (malloc=543008 #5144) > (malloc=200416) > (malloc=152 #7) > (malloc=6788 #214) > (malloc=18977 #485) > (malloc=994280 #2797) > (malloc=153248 #744) > (malloc=2152 #12) > > I must be missing something obvious here. What is it? > > Thank you, > > Thomas > From thomas.stuefe at gmail.com Thu Aug 6 15:45:59 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 6 Aug 2020 17:45:59 +0200 Subject: NMT does not seem to count malloc correctly In-Reply-To: References: Message-ID: Oh of course. How silly of me. Thanks Zhengyu. On Thu, Aug 6, 2020, 17:38 Zhengyu Gu wrote: > By commenting out ::free(), you created real memory leak, but not from > NMT perspective, because it already adjusted internal counting to > reflect os:free(). > > -Zhengyu > > > > On 8/6/20 11:28 AM, Thomas St?fe wrote: > > Hi, > > > > maybe I am missing something fundamental, or there is something wrong > with > > NMT. > > > > As an experiment for an unrelated issue, I created a C-Heap leak by > simply > > commenting out the ::free() call inside os::free(): > > > > --- a/src/hotspot/share/runtime/os.cpp Thu Aug 06 17:08:02 2020 +0200 > > +++ b/src/hotspot/share/runtime/os.cpp Thu Aug 06 17:08:08 2020 +0200 > > @@ -804,7 +804,7 @@ > > size_t size = guarded.get_user_size(); > > inc_stat_counter(&free_bytes, size); > > membase = guarded.release_for_freeing(); > > - ::free(membase); > > +// ::free(membase); > > #else > > void* membase = MemTracker::record_free(memblock, > > MemTracker::tracking_level()); > > ::free(membase); > > > > As expected, this leads quickly to a very high memory usage. In my tiny > > example (which really does not matter) I quickly end up with ~4G RSS, and > > the glibc reports about 3G outstanding allocations: > > > > Process Memory: > > Virtual Size: 11136852K (peak: 11136852K) > > Resident Set Size: 4057860K (peak: 4057860K) (anon: 4013076K, file: > 44784K, > > shmem: 0K) > > Swapped out: 0K > > C-Heap outstanding allocations: 3070399K > > > > > > However, NMT reports just about 600M committed memory (into this number > > should fall all outstanding allocations done with os::malloc() AFAICS): > > > > thomas at mainframe:~$ jjjcmd Interl VM.native_memory scale=1 > > > > 19441: > > > > Native Memory Tracking: > > > > Total: reserved=6227876720, committed=626285424 > > > > .... > > > > And the malloc information adds up to about only 68M: > > > > thomas at mainframe:~$ jjjcmd Interl VM.native_memory scale=1 | grep > malloc > > (malloc=4044913 #55107) > > (malloc=38688 #146) > > (malloc=10812522 #258557) > > (malloc=18157899 #3629) > > (malloc=42496 #311) > > (malloc=216 #7) > > (malloc=594920 #1788) > > (malloc=27304 #2) > > (malloc=2781872 #60716) > > (malloc=543008 #5144) > > (malloc=200416) > > (malloc=152 #7) > > (malloc=6788 #214) > > (malloc=18977 #485) > > (malloc=994280 #2797) > > (malloc=153248 #744) > > (malloc=2152 #12) > > > > I must be missing something obvious here. What is it? > > > > Thank you, > > > > Thomas > > > > From yumin.qi at oracle.com Thu Aug 6 16:54:32 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Thu, 6 Aug 2020 09:54:32 -0700 Subject: RFR JDK-8251213 [TESTBUG] CDS tests shouldn't write output files into test.classes directory In-Reply-To: References: Message-ID: <02603b92-df8c-9537-1dee-402aca23d62b@oracle.com> Hi, Ioi ? The change looks good to me. ? The output file name repeats test name in it. This is not related to your change, but think if you can make change to it: runtime.cds.appcds.cacheObject.OpenArchiveRegion.java-runtime.cds.appcds.cacheObject.OpenArchiveRegion.java-0000-dump.stderr runtime.cds.appcds.cacheObject.OpenArchiveRegion.java-runtime.cds.appcds.cacheObject.OpenArchiveRegion.java-0000-dump.stdout Thanks Yumin On 8/5/20 11:21 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8251213 > http://cr.openjdk.java.net/~iklam/jdk16/8251213-cds-shouldnt-write-test-classes-dir.v01/ > > > Many of the CDS tests would write output files to the directory > given by System.getProperty("test.classes"). > > I changed them to write to the scratch directory, i.e., > System.getProperty("user.dir", "."). This way, it's easier to examine the > output of the tests. > > Please start the review with CDSTestUtils.java. > > I validated the change by going to the jtreg work directory and: > > ??? $ find $(find . -name classes) -type f | grep -v '[.]class$' | wc -l > ??? 0 > > (There used to be a lot of .jsa, .classlist, .stdout, etc). > > Executed all the tests locally on Linux. Running mach5 tiers1-4. > > Thanks > - Ioi From ioi.lam at oracle.com Thu Aug 6 17:18:09 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 6 Aug 2020 10:18:09 -0700 Subject: RFR JDK-8251213 [TESTBUG] CDS tests shouldn't write output files into test.classes directory In-Reply-To: <02603b92-df8c-9537-1dee-402aca23d62b@oracle.com> References: <02603b92-df8c-9537-1dee-402aca23d62b@oracle.com> Message-ID: <4c3f2359-d3b3-c075-5d5c-8074cbda015f@oracle.com> Hi Yumin, Thanks for the review. I actually fixed the repeated name in the test stdout/stderr files, but forgot to mention it in the e-mail. The files look like this now: work/runtime/cds/appcds/HelloTest/runtime.cds.appcds.HelloTest.java-0000-dump.stderr work/runtime/cds/appcds/HelloTest/runtime.cds.appcds.HelloTest.java-0000-dump.stdout work/runtime/cds/appcds/HelloTest/runtime.cds.appcds.HelloTest.java-0001-exec.stderr work/runtime/cds/appcds/HelloTest/runtime.cds.appcds.HelloTest.java-0001-exec.stdout Thanks - Ioi On 8/6/20 9:54 AM, Yumin Qi wrote: > Hi, Ioi > > ? The change looks good to me. > > ? The output file name repeats test name in it. This is not related to > your change, but think if you can make change to it: > > runtime.cds.appcds.cacheObject.OpenArchiveRegion.java-runtime.cds.appcds.cacheObject.OpenArchiveRegion.java-0000-dump.stderr > > runtime.cds.appcds.cacheObject.OpenArchiveRegion.java-runtime.cds.appcds.cacheObject.OpenArchiveRegion.java-0000-dump.stdout > > > > Thanks > > Yumin > > > On 8/5/20 11:21 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8251213 >> http://cr.openjdk.java.net/~iklam/jdk16/8251213-cds-shouldnt-write-test-classes-dir.v01/ >> >> >> Many of the CDS tests would write output files to the directory >> given by System.getProperty("test.classes"). >> >> I changed them to write to the scratch directory, i.e., >> System.getProperty("user.dir", "."). This way, it's easier to examine >> the >> output of the tests. >> >> Please start the review with CDSTestUtils.java. >> >> I validated the change by going to the jtreg work directory and: >> >> ??? $ find $(find . -name classes) -type f | grep -v '[.]class$' | wc -l >> ??? 0 >> >> (There used to be a lot of .jsa, .classlist, .stdout, etc). >> >> Executed all the tests locally on Linux. Running mach5 tiers1-4. >> >> Thanks >> - Ioi From ioi.lam at oracle.com Thu Aug 6 18:25:42 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 6 Aug 2020 11:25:42 -0700 Subject: RFR (xs) 8251209 [TESTBUG] CDS jvmti tests should use "@modules java.instrument" Message-ID: https://bugs.openjdk.java.net/browse/JDK-8251209 http://cr.openjdk.java.net/~iklam/jdk16/8251209-cds-jvmti-tests-modules-tag.v01/ Summary -- changed the tests from (mis)using ?* @requires vm.flavor != "minimal" to ?* @modules java.instrument ... to be consistent with other jvmti tests. Thanks - Ioi From daniel.daugherty at oracle.com Thu Aug 6 18:29:19 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 6 Aug 2020 14:29:19 -0400 Subject: RFR (s): 8250606: Remove unnecessary assertions in ObjectSynchronizer FastHashcode and inflate In-Reply-To: References: <3e4c931a-593b-0336-97b9-3adc4de96062@oracle.com> <17eef440-3575-73f5-1413-0a25da6a13df@oracle.com> Message-ID: <11f913cc-8f88-0740-28d0-a7db7897d6a8@oracle.com> On 8/5/20 6:31 PM, David Holmes wrote: > Hi Dan, > > Thanks for taking a look at this. > > On 6/08/2020 4:08 am, Daniel D. Daugherty wrote: >> On 8/5/20 3:58 AM, David Holmes wrote: >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8250606 >>> webrev: http://cr.openjdk.java.net/~dholmes/8250606/webrev/ >> >> src/hotspot/share/runtime/synchronizer.cpp >> ???? L1010: ????? // Relaxing assertion for bug 6320749. >> ???? L1011: ????? assert(Universe::verify_in_progress() || >> ???? L1012: !SafepointSynchronize::is_at_safepoint(), >> ???? L1013: ???????????? "biases should not be seen by VM thread here"); >> ???????? This assertion is similar and tagged with the same bug ID >> (6320749) >> ???????? as the ones that you removed. Granted that the Biased >> Locking code >> ???????? is on the way out, but... >> >> ???????? I'm fine if you plan to leave this one for Biased Locking >> removal. > > Yeah I confess I had my biased-locking filter on when looking at this > part of the code. You've actually exposed a bit of a mess. :) > > It is unclear why the same !at_safepoint assertion was initially added > to the biased-locking revocation chunk of code given the same > assertion would then be executed after it. But I think it is more to > do with the line following the assertion (original code): > > assert(!SafepointSynchronize::is_at_safepoint(), "biases should not be > seen by VM thread here"); > BiasedLocking::revoke_and_rebias(hobj, false, JavaThread::current()); > > If we are at a safepoint in the VMThread then JavaThread::current() > will assert! So even after the assertion was apparently relaxed with > the verify_in_progress() check, we could never have hit this code as > the JavaThread::current() would still assert. So I can (and should) > remove the assertion here as well (CDS dump could reach this in > theory), but I also need to correct this current code: > > -????? BiasedLocking::revoke(hobj, JavaThread::current()); > +????? BiasedLocking::revoke(hobj, self); > > as re-manifesting the current thread is pointless. That said the above > code (nor the original) also can't execute at a safepoint: > > void BiasedLocking::revoke(Handle obj, TRAPS) { > ? assert(!SafepointSynchronize::is_at_safepoint(), "must not be called > while at safepoint"); With the assert in JavaThread::current() and with the assert in BiasedLocking::revoke(), I don't see how we could ever get away with calling FastHashCode() on a biased object at a safepoint. It simply should have blown up (with non-release bits). I'm not yet sure what keeps us from seeing a biased object at a safepoint, but there has to be something there. We aren't that lucky to have simply missed this... Dan > > so even larger changes will be needed to make this code work at a > safepoint, and that is going well beyond the intended scope of this > particular bug. :( I'll need to get back to you. > > I also need to check our GC verify tests to see how we manage to > apparently never encounter a biased-locked object, otherwise we would > have seen this inconsistency in the code. > > Thanks, > David > ----- > > >> Thumbs up.? Thanks for digging up the history behind these very old >> asserts. > > Thanks, > David > >> Dan >> >> >>> >>> tl;dr I removed some assertions that were deemed unnecessary after >>> interfering with other work. :) >>> >>> ObjectSynchronizer::inflate had the following assertion added back >>> in JDK 6 under JDK-5030359: >>> >>> ObjectMonitor * ObjectSynchronizer::inflate (oop object) { >>> ? // Inflate mutates the heap ... >>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>> >>> That same change added FastHashcode which had the following assertions: >>> >>> intptr_t ObjectSynchronizer::FastHashCode (Thread * Self, oop obj) { >>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>> ? assert (Self->is_Java_thread(), "invariant") ; >>> ? assert (((JavaThread *)Self)->thread_state() != _thread_blocked, >>> "invariant") ; >>> >>> These methods had assertions that they were not executed at a >>> safepoint in case the modification of the markWord could cause >>> interference with GC. >>> >>> JDK-6320749 then relaxed the assertions to allow for GC verification >>> - which does happen at a safepoint and so obviously not in a >>> JavaThread. >>> >>> The FastHashCode assertions were further relaxed in JDK-8015086 to >>> allow the CDS dump VM safepoint operation to call it. But >>> FastHashCode can also call inflate so that was a mismatch in >>> assertions just waiting to happen - which is how I hit this in a VM >>> that always forces inflation :) >>> >>> I initially intended to extend the FastHashCode assertion to match >>> inflate, but then a question arose about the safety of the CDS >>> dumping code using FastHashcode which showed that the code is in >>> fact safe. Notwithstanding what the state of the world may have been >>> in the JDK 6 timeframe, there is now no potential interference >>> possible between execution of these methods at an arbitrary >>> safepoint and any concurrent GC use of the markWord. So the original >>> !is_at_safepoint assertion can just be dropped from both methods. >>> That in turn means we either drop the JavaThread assertion or else >>> expand it to be "JavaThread or VMThread" - but that itself seems >>> unnecessarily restrictive: why can't a GC thread use this code for >>> example? So I chose to drop it. >>> >>> That leaves the "if I'm a JavaThread I'm not _thread_blocked" >>> assertion. I don't see that this really guards against anything >>> useful (a JavaThread should be _thread_in_vm when executing this >>> code ... though there may be leaf calls where the thread is >>> _thread_in_java). So again I just dropped the assertion. >>> >>> So a long winded justification for dropping a few assertions. :) >>> >>> Thanks, >>> David >>> >>> >> From calvin.cheung at oracle.com Thu Aug 6 18:38:09 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Thu, 6 Aug 2020 11:38:09 -0700 Subject: RFR JDK-8251213 [TESTBUG] CDS tests shouldn't write output files into test.classes directory In-Reply-To: References: Message-ID: <97dcfab3-1efc-067a-8800-8e375652af16@oracle.com> Hi Ioi, Thanks for fixing this. Couple of minor comments: 1) MultiReleaseJars.java ? 37 import jdk.test.lib.process.OutputAnalyzer; ? 38 import jdk.test.lib.cds.CDSTestUtils; Please swap the order of the above. 2) You've made changes to the tests under dynamicArchive/methodHandles. Can you apply the same change to the CDSMHTest_generate.sh since those tests are being generated by the script? No need for another webrev for the above changes. One suggestion (I think it should be done in another RFE): Several tests are doing the following: ??? File dir = new File(System.getProperty("user.dir")); I'm thinking add another method in CDSTestUtils to return the outputDirAsFile so that the above could be: ??? File dir = CDSTestUtils.getOutputDirAsFile(); thanks, Calvin On 8/5/20 11:21 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8251213 > http://cr.openjdk.java.net/~iklam/jdk16/8251213-cds-shouldnt-write-test-classes-dir.v01/ > > > Many of the CDS tests would write output files to the directory > given by System.getProperty("test.classes"). > > I changed them to write to the scratch directory, i.e., > System.getProperty("user.dir", "."). This way, it's easier to examine the > output of the tests. > > Please start the review with CDSTestUtils.java. > > I validated the change by going to the jtreg work directory and: > > ??? $ find $(find . -name classes) -type f | grep -v '[.]class$' | wc -l > ??? 0 > > (There used to be a lot of .jsa, .classlist, .stdout, etc). > > Executed all the tests locally on Linux. Running mach5 tiers1-4. > > Thanks > - Ioi From patricio.chilano.mateo at oracle.com Thu Aug 6 18:48:36 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Thu, 6 Aug 2020 15:48:36 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> Message-ID: <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> Hi Dan, On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: > I'm peeking ahead to the next webrev... :-) > > > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ : ) > General comments: > ? - check files for copyright year updates. Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. > src/hotspot/share/runtime/synchronizer.hpp > ??? No comments. > > src/hotspot/share/runtime/synchronizer.cpp > ??? L507: ? const markWord mark = obj->mark(); > ??? L508: > ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && > obj->klass()->is_box()) { > ??? L510: ??? return false; > ??? L511: ? } > ??????? The new bailout on L509-511 can move above L507. Moved. > L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass > %s", p2i(obj()), obj->klass()->external_name()); > ??????? This external_name() call does not have a ResourceMark. Good catch! I had one in a previous version but then I changed the conditionals and lost it for the fatal error case. The test worked okay because for the main JavaThread there is a ResourceMark in jni_invoke_static() (jni.cpp). > src/hotspot/share/logging/logTag.hpp > ??? No comments. > > src/hotspot/share/oops/klass.hpp > ??? No comments. > > src/hotspot/share/utilities/accessFlags.hpp > ??? No comments. > > src/hotspot/share/runtime/globals.hpp > ??? L814: ???????????? "0: off " > ??????? Missing a ';' after "off". Fixed. > L816: ???????????? "2: log message to stdout. > ??????? Perhaps add "(by default)" after "stdout" or > ??????? don't say where log output is at all. > > src/hotspot/share/runtime/arguments.cpp > ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, true, > LOG_TAGS(primitivewrappers)); > ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp > ??????? since it looks like logging output is configured to 'stdout'. > ??????? I'm assuming that other log options to put output elsewhere > ??????? are overridden by this code? Right. So the logging is done under UL with the tag primitivewrappers. If that tag is specified in Xlog then this conditional will be skipped because !log_is_enabled(Info, primitivewrappers) will be false. > src/hotspot/share/classfile/systemDictionary.cpp > ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { > ??????? nit - s/T_LONG+1/T_LONG + 1/ Fixed. > L2191: _box_klasses[i]->set_prototype_header(markWord::prototype()); > ??????? I assume we're keeping the prototype_header field when Biased > Locking > ??????? goes away? The reason I ask: > > ? ? ? ? static markWord prototype() { > ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); > ? ? ? ? } > > ??????? is because without Biased Locking, do we really need the > prototype > ??????? anymore? The initial markWord won't need possible variants... Yes, I think it can go away unless somebody finds another use for it. Maybe Valhalla. > src/hotspot/share/jfr/metadata/metadata.xml > ??? L69: ? ??????? Is the category "Java Application" because it's the application > ??????? code that did something "wrong" here? Where "application" is > loosely > ??????? defined to include the possibility of auto generated code, > library > ??????? code and the like? Or perhaps "application" because something > "above" > ??????? the "Java Virtual Machine, Runtime" did the "wrong" thing here? I don't know what the right category should be really. I saw the events JavaMonitorEnter, JavaMonitorWait and JavaMonitorInflate and thought this event should fall in the same category they do. > src/jdk.jfr/share/conf/jfr/default.jfc > ??? No comments. > > src/jdk.jfr/share/conf/jfr/profile.jfc > ??? No comments. > > test/lib/jdk/test/lib/jfr/EventNames.java > ??? No comments. > > src/hotspot/cpu/aarch64/aarch64.ad > ??? L3517: ????? __ tbnz(tmp, exact_log2(JVM_ACC_IS_BOX_CLASS), cont); > > ??? L3578: ??? __ bind(cont); > ??? L3579: ??? // flag == EQ indicates success > ??? L3580: ??? // flag == NE indicates failure > ??????? If tbnz() branches to "cont" when we have a box class, what's > ??????? the flag value set to (EQ or NE)? And what set that flag value? > ??????? The reason I ask is I don't think tbnz() sets condition codes... Right, it doesn't set condition codes, so I kept the version I had (more on that below). > src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp > ??? No comments. > > src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp > ??? No comments. > > src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp > ??? int MacroAssembler::biased_locking_enter(Register lock_reg, > ??????????? ? ?? ??????????????????????????? Register obj_reg, > ?????????????????????????????? ? ?? ???????? Register swap_reg, > ?????????????????????????????????? ? ?? ???? Register tmp_reg, > ?????????????????????????????????? ? ?? ???? bool swap_reg_contains_mark, > ?????????????????????????????????? ? ?? ???? Label& done, > ??????????????????????????????????? ? ?? ??? Label* slow_case, > BiasedLockingCounters* counters) { > ??????? I think you've changed the only callers of biased_locking_enter() > ??????? that cared about the return value with this changeset so it can > ??????? be changed to a void function. Ok, this is what I mentioned to David in a previous email. Done. > src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp > ??? No comments. > > src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp > ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), done); > > ??? L131: ? bind(done); > ??? L132: > ??? L133: ? // At this point flags are set as follows: > ??? L134: ? //? EQ -> Success > ??? L135: ? //? NE -> Failure, branch to slow path > ??????? If tbnz() branches to "done" when we have a box class, what's > ??????? the flag value set to (EQ or NE)? And what set that flag value? > ??????? The reason I ask is I don't think tbnz() sets condition codes... Right. Same as above, I kept the version I had. > src/hotspot/cpu/arm/interp_masm_arm.cpp > ??? No comments. > > src/hotspot/cpu/arm/macroAssembler_arm.cpp > ??? int MacroAssembler::biased_locking_enter(Register obj_reg, > Register swap_reg, Register tmp_reg, > ????????????????????? ? ?? ????????????????? bool swap_reg_contains_mark, > ???????????????????????? ? ?? ?????????????? Register tmp2, > ????????????????????????? ? ?? ????????????? Label& done, Label& > slow_case, > BiasedLockingCounters* counters) { > ??????? I think you've changed the only callers of biased_locking_enter() > ??????? that cared about the return value with this changeset so it can > ??????? be changed to a void function. Done. > src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp > src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp > src/hotspot/cpu/ppc/macroAssembler_ppc.cpp > ??? No comments on the PPC code. > > src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp > src/hotspot/cpu/s390/interp_masm_s390.cpp > src/hotspot/cpu/s390/macroAssembler_s390.cpp > ??? No comments on the S390 code. > > src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp > ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); > ??????? What will this do with a 'noreg' value? (I need a refresher.) When not in 64 bit mode that register just won't be used. > src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp > ??? No comments. > > src/hotspot/cpu/x86/interp_masm_x86.cpp > ??? No comments. > > src/hotspot/cpu/x86/macroAssembler_x86.cpp > ??? int MacroAssembler::biased_locking_enter(Register lock_reg, > ????????????????????????????????? ? ?? ????? Register obj_reg, > ????????????????????????????????? ? ?? ????? Register swap_reg, > ????????????????????????????????? ? ?? ????? Register tmp_reg, > ?????????????????????????????????? ? ?? ???? Register tmp_reg2, > ?????????????????????????????????????? ? ? ? bool swap_reg_contains_mark, > ?????????????????????????????????? ? ?? ???? Label& done, > ?????????????????????????????????? ? ?? ???? Label* slow_case, > BiasedLockingCounters* counters) { > ??????? I think you've changed the only caller of biased_locking_enter() > ??????? that cared about the return value with this changeset so it can > ??????? be changed to a void function. Done. > test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java > ??? L30:? * @test SyncOnPrimitiveWrapperTest > ??????? No parameter to @test directive. Removed. > L136: ??????? private static long sharedCounter = 0L; > ??????? Since you don't do anything with sharedCounter other than > increment it, > ??????? can the compilers optimize it away? If it can be optimized > away, does > ??????? that mean that: > > ??????????? L142: ??????????????? synchronized (obj) { > > ??????? can also be optimized away? > > ??????? I don't think that: > > ??????????? L161: ??????????????? synchronized (sharedLock1) { > > ??????? can be optimized away because it is shared by multiple threads. > > test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java > ??? Similar questions about 'counter' being optimized away. > ??? Similar question about "synchronized (obj) {" being optimized away. I'm not sure if the compiler will optimize it away. Seems unlikely given the counter we are incrementing is not just local to some thread. Ok, below is v3 which has the following changes: - Use a 32 bit load for the _access_flags field, instead of 64 - Martin's implementation for s390 and fix for PPC - Faster LogTest version - Above changes based on Dan review I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 only) again with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I checked it builds in arm32, ppc and s390. Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev @Martin: Please check if the test doesn't timeout for you now with the changes I made to LogTest. Also, I tried to use tbnz in aarch64 and arm32 instead of tst + br (except for c2 since we actually need to set the condition flags), but for c1 I was getting an assertion in the compiler thread: guarantee(chk == -1 || chk == 0) failed: Field too big for insn This is the stack when that happens: MacroAssembler::pd_patch_instruction_size(unsigned char*, unsigned char*)+0x398 AbstractAssembler::bind(Label&)+0x118 MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 LIR_Assembler::emit_slow_case_stubs()+0x54 Compilation::emit_code_body()+0x17c The change was simply: diff --git a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp @@ -78,7 +78,6 @@ ?? if (DiagnoseSyncOnPrimitiveWrappers != 0) { ???? load_klass(hdr, obj); -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); -??? br(Assembler::NE, slow_case); +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); ?? } So the issue must be related to where we want to jump. Thanks, Patricio > Dan > > > On 8/5/20 9:01 AM, Patricio Chilano wrote: >> Hi Martin, >> >> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>> Hi Patricio, >>> >>> using 8 Byte load instructions for jint fields is a terrible coding >>> style! >>> Someone else may see it and use an 8 Byte store. Will result in >>> great fun for debugging! >>> >>> And on Big Endian you will simply access the wrong bits. >> Ah, of course! Those 32 bits will be in the wrong place when doing >> the test. >> >>> Note that Big Endian Platforms are AIX, old linux ppc, s390, SPARC. >>> I don't think you have tested on them. >>> >>>> We could remove the nested synchronized statements in the run() method >>>> so that we don't generate that much logging. We could also lower >>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>> with multiple flag combinations. Not sure what we should touch first. >>>> Maybe the synchronized statements, have only one or two and test that >>>> first? >>> Sounds like helpful ideas. Please go ahead and strip things down. >> Great, I will send v3 later with those changes. >> >> Thanks Martin! >> >> Patricio >>> Thanks for taking care of it. >>> >>> Best regards, >>> Martin >>> >>> >>>> -----Original Message----- >>>> From: Patricio Chilano >>>> Sent: Dienstag, 4. August 2020 20:26 >>>> To: Doerr, Martin ; David Holmes >>>> ; hotspot-runtime-dev at openjdk.java.net; >>>> hotspot-jfr-dev at openjdk.java.net >>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>> wrappers >>>> >>>> Hi Martin, >>>> >>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>> Hi Patricio, >>>>> >>>>>> Ok, I'll change it to movl + testl and test it out before sending >>>>>> v3. >>>>> Thanks. I forgot to mention arm + aarch64. >>>>> Aarch64 uses ldrw + tbnz. >>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>> >>>>> I remember having seen the same mistake ?? >>>>> And nobody noticed it on little Endian platforms. >>>> Ok, I can use a tbnz instead of test and then a branch on arm32 and >>>> aarch64. >>>> Shouldn't a normal ldr in arm32 work fine? >>>> Also in 64 bits (either x64 or aarch64) I don't see the issue of >>>> using a >>>> 64 bit load, besides the fact that we only care about the first 32 >>>> bits. >>>> Regardless of the endianness, aren't we masking out the upper part >>>> when >>>> we do AND/TEST or if we test a bit in the 0-31 bit range? Otherwise it >>>> seems it should have failed for one of those platforms in my tests. >>>> >>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>> specify that and see if that fixes it? Let me know if that works >>>>>> and I >>>>>> can add it to the test. >>>>> That seems to have an effect. But now I'm not patient enough to >>>>> wait for >>>> the test to finish. >>>>> Maybe the problem is that I'm using slow debug builds. >>>>> But is there a chance to make the test quicker without losing >>>>> coverage >>>> significantly? >>>>> I think the test is too slow this way to run it on a regular >>>>> basis. We'd need to >>>> spend dedicated machines for it. >>>> We could remove the nested synchronized statements in the run() method >>>> so that we don't generate that much logging. We could also lower >>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>> with multiple flag combinations. Not sure what we should touch first. >>>> Maybe the synchronized statements, have only one or two and test that >>>> first? >>>> >>>> >>>> Thanks, >>>> Patricio >>>>> Best regards, >>>>> Martin >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: Patricio Chilano >>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>> To: Doerr, Martin ; David Holmes >>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>> wrappers >>>>>> >>>>>> Hi Martin, >>>>>> >>>>>> Thanks for fixing PPC and taking care of s390! >>>>>> >>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>> Hi Patricio, >>>>>>> >>>>>>> I suggest to use movl + testl for checking the access flag as >>>>>>> for other >>>> access >>>>>> flags on x64. >>>>>> Ok, I'll change it to movl + testl and test it out before sending >>>>>> v3. >>>>>> >>>>>>> New version for PPC64 and s390 see below. >>>>>>> >>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>>> expected. >>>>>> However, I'm getting timeout issues: >>>>>>> timed out (timeout set to 120000ms, elapsed time including timeout >>>>>> handling was 122372ms) >>>>>>> Can we provide more time? >>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>> specify that and see if that fixes it? Let me know if that works >>>>>> and I >>>>>> can add it to the test. >>>>>> >>>>>> Thanks, >>>>>> Patricio >>>>>>> Best regards, >>>>>>> Martin >>>>>>> >>>>>>> >>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>> 10:03:57 2020 +0200 >>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>> 16:04:31 2020 +0200 >>>>>>> @@ -107,8 +107,8 @@ >>>>>>> >>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>> Rscratch); >>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>> +??? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>> Rscratch); >>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>> ??????? bne(CCR0, slow_case); >>>>>>> ????? } >>>>>>> >>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>> 10:03:57 2020 +0200 >>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>> 16:04:31 2020 +0200 >>>>>>> @@ -912,8 +912,8 @@ >>>>>>> >>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>> ????????? load_klass(tmp, object); >>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>> +????? testbitdi(CCR0, R0, tmp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>> ????????? bne(CCR0, slow_case); >>>>>>> ??????? } >>>>>>> >>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>> 10:03:57 2020 +0200 >>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>> 16:04:31 2020 +0200 >>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>> >>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>> ??????? load_klass(temp, oop); >>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>> +??? testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>> ??????? bne(CCR0, cont); >>>>>>> ????? } >>>>>>> >>>>>>> diff -r 77852e129401 >>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>> 10:03:57 2020 +0200 >>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>> 16:04:31 2020 +0200 >>>>>>> @@ -91,6 +91,12 @@ >>>>>>> ????? // Save object being locked into the BasicObjectLock... >>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>> +??? z_btrue(slow_case); >>>>>>> +? } >>>>>>> + >>>>>>> ????? if (UseBiasedLocking) { >>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>> Z_R0_scratch, done, >>>>>> &slow_case); >>>>>>> ????? } >>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 10:03:57 >>>>>> 2020 +0200 >>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>> 16:04:31 >>>>>> 2020 +0200 >>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>> ????? z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), >>>>>>> object); >>>>>>> >>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>> +??? z_btrue(slow_case); >>>>>>> +? } >>>>>>> + >>>>>>> ????? if (UseBiasedLocking) { >>>>>>> ??????? biased_locking_enter(object, displaced_header, Z_R1, >>>>>>> Z_R0, done, >>>>>> &slow_case); >>>>>>> ????? } >>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>> 10:03:57 2020 +0200 >>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>> 16:04:31 2020 +0200 >>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>> ????? // Load markWord from oop into mark. >>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>> >>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>> +??? z_btrue(done); >>>>>>> +? } >>>>>>> + >>>>>>> ????? if (try_bias) { >>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, Z_R0, >>>>>>> done); >>>>>>> ????? } >>>>>>> >> > From ioi.lam at oracle.com Thu Aug 6 20:39:06 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 6 Aug 2020 13:39:06 -0700 Subject: RFR JDK-8251213 [TESTBUG] CDS tests shouldn't write output files into test.classes directory In-Reply-To: <97dcfab3-1efc-067a-8800-8e375652af16@oracle.com> References: <97dcfab3-1efc-067a-8800-8e375652af16@oracle.com> Message-ID: <410704ee-fc53-ae8f-c23d-33b29f9a7da3@oracle.com> On 8/6/20 11:38 AM, calvin.cheung at oracle.com wrote: > Hi Ioi, > > Thanks for fixing this. > > Couple of minor comments: > > 1) MultiReleaseJars.java > > ? 37 import jdk.test.lib.process.OutputAnalyzer; > ? 38 import jdk.test.lib.cds.CDSTestUtils; > > Please swap the order of the above. > Fixed > 2) You've made changes to the tests under > dynamicArchive/methodHandles. Can you apply the same change to the > CDSMHTest_generate.sh since those tests are being generated by the > script? > Fixed. I re-ran the script and it generated the exact same contents as my manually edited version. > No need for another webrev for the above changes. > > One suggestion (I think it should be done in another RFE): > > Several tests are doing the following: > > ??? File dir = new File(System.getProperty("user.dir")); > > I'm thinking add another method in CDSTestUtils to return the > outputDirAsFile so that the above could be: > > ??? File dir = CDSTestUtils.getOutputDirAsFile(); > I filed https://bugs.openjdk.java.net/browse/JDK-8251267 -- CDS tests should use CDSTestUtils.getOutputDir instead of System.getProperty("user.dir") Thanks - Ioi > thanks, > > Calvin > > On 8/5/20 11:21 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8251213 >> http://cr.openjdk.java.net/~iklam/jdk16/8251213-cds-shouldnt-write-test-classes-dir.v01/ >> >> >> Many of the CDS tests would write output files to the directory >> given by System.getProperty("test.classes"). >> >> I changed them to write to the scratch directory, i.e., >> System.getProperty("user.dir", "."). This way, it's easier to examine >> the >> output of the tests. >> >> Please start the review with CDSTestUtils.java. >> >> I validated the change by going to the jtreg work directory and: >> >> ??? $ find $(find . -name classes) -type f | grep -v '[.]class$' | wc -l >> ??? 0 >> >> (There used to be a lot of .jsa, .classlist, .stdout, etc). >> >> Executed all the tests locally on Linux. Running mach5 tiers1-4. >> >> Thanks >> - Ioi From david.holmes at oracle.com Thu Aug 6 21:46:17 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Aug 2020 07:46:17 +1000 Subject: RFR (s): 8250606: Remove unnecessary assertions in ObjectSynchronizer FastHashcode and inflate In-Reply-To: <11f913cc-8f88-0740-28d0-a7db7897d6a8@oracle.com> References: <3e4c931a-593b-0336-97b9-3adc4de96062@oracle.com> <17eef440-3575-73f5-1413-0a25da6a13df@oracle.com> <11f913cc-8f88-0740-28d0-a7db7897d6a8@oracle.com> Message-ID: <63ee179e-7133-3edb-c952-1b5fd4f334bd@oracle.com> Hi Dan, On 7/08/2020 4:29 am, Daniel D. Daugherty wrote: > On 8/5/20 6:31 PM, David Holmes wrote: >> Hi Dan, >> >> Thanks for taking a look at this. >> >> On 6/08/2020 4:08 am, Daniel D. Daugherty wrote: >>> On 8/5/20 3:58 AM, David Holmes wrote: >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8250606 >>>> webrev: http://cr.openjdk.java.net/~dholmes/8250606/webrev/ >>> >>> src/hotspot/share/runtime/synchronizer.cpp >>> ???? L1010: ????? // Relaxing assertion for bug 6320749. >>> ???? L1011: ????? assert(Universe::verify_in_progress() || >>> ???? L1012: !SafepointSynchronize::is_at_safepoint(), >>> ???? L1013: ???????????? "biases should not be seen by VM thread here"); >>> ???????? This assertion is similar and tagged with the same bug ID >>> (6320749) >>> ???????? as the ones that you removed. Granted that the Biased >>> Locking code >>> ???????? is on the way out, but... >>> >>> ???????? I'm fine if you plan to leave this one for Biased Locking >>> removal. >> >> Yeah I confess I had my biased-locking filter on when looking at this >> part of the code. You've actually exposed a bit of a mess. :) >> >> It is unclear why the same !at_safepoint assertion was initially added >> to the biased-locking revocation chunk of code given the same >> assertion would then be executed after it. But I think it is more to >> do with the line following the assertion (original code): >> >> assert(!SafepointSynchronize::is_at_safepoint(), "biases should not be >> seen by VM thread here"); >> BiasedLocking::revoke_and_rebias(hobj, false, JavaThread::current()); >> >> If we are at a safepoint in the VMThread then JavaThread::current() >> will assert! So even after the assertion was apparently relaxed with >> the verify_in_progress() check, we could never have hit this code as >> the JavaThread::current() would still assert. So I can (and should) >> remove the assertion here as well (CDS dump could reach this in >> theory), but I also need to correct this current code: >> >> -????? BiasedLocking::revoke(hobj, JavaThread::current()); >> +????? BiasedLocking::revoke(hobj, self); >> >> as re-manifesting the current thread is pointless. That said the above >> code (nor the original) also can't execute at a safepoint: >> >> void BiasedLocking::revoke(Handle obj, TRAPS) { >> ? assert(!SafepointSynchronize::is_at_safepoint(), "must not be called >> while at safepoint"); > > > With the assert in JavaThread::current() and with the assert in > BiasedLocking::revoke(), I don't see how we could ever get away > with calling FastHashCode() on a biased object at a safepoint. > It simply should have blown up (with non-release bits). > > I'm not yet sure what keeps us from seeing a biased object at > a safepoint, but there has to be something there. We aren't that > lucky to have simply missed this... Seems we are that lucky in a sense. It seems that the verification process that we allow for is not the same as it once was and so we don't hit the code that needs the FastHashCode. The only other safepoint op that gets involved is CDS dumping and there we also get lucky in that none of the objects that get archived are bias-locked. I've been using Ioi's test from http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ to force the object to be biased but I still can't hit that code fragment so something else is still in play. David ----- > Dan > >> >> so even larger changes will be needed to make this code work at a >> safepoint, and that is going well beyond the intended scope of this >> particular bug. :( I'll need to get back to you. >> >> I also need to check our GC verify tests to see how we manage to >> apparently never encounter a biased-locked object, otherwise we would >> have seen this inconsistency in the code. >> >> Thanks, >> David >> ----- >> >> >>> Thumbs up.? Thanks for digging up the history behind these very old >>> asserts. >> >> Thanks, >> David >> >>> Dan >>> >>> >>>> >>>> tl;dr I removed some assertions that were deemed unnecessary after >>>> interfering with other work. :) >>>> >>>> ObjectSynchronizer::inflate had the following assertion added back >>>> in JDK 6 under JDK-5030359: >>>> >>>> ObjectMonitor * ObjectSynchronizer::inflate (oop object) { >>>> ? // Inflate mutates the heap ... >>>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>>> >>>> That same change added FastHashcode which had the following assertions: >>>> >>>> intptr_t ObjectSynchronizer::FastHashCode (Thread * Self, oop obj) { >>>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>>> ? assert (Self->is_Java_thread(), "invariant") ; >>>> ? assert (((JavaThread *)Self)->thread_state() != _thread_blocked, >>>> "invariant") ; >>>> >>>> These methods had assertions that they were not executed at a >>>> safepoint in case the modification of the markWord could cause >>>> interference with GC. >>>> >>>> JDK-6320749 then relaxed the assertions to allow for GC verification >>>> - which does happen at a safepoint and so obviously not in a >>>> JavaThread. >>>> >>>> The FastHashCode assertions were further relaxed in JDK-8015086 to >>>> allow the CDS dump VM safepoint operation to call it. But >>>> FastHashCode can also call inflate so that was a mismatch in >>>> assertions just waiting to happen - which is how I hit this in a VM >>>> that always forces inflation :) >>>> >>>> I initially intended to extend the FastHashCode assertion to match >>>> inflate, but then a question arose about the safety of the CDS >>>> dumping code using FastHashcode which showed that the code is in >>>> fact safe. Notwithstanding what the state of the world may have been >>>> in the JDK 6 timeframe, there is now no potential interference >>>> possible between execution of these methods at an arbitrary >>>> safepoint and any concurrent GC use of the markWord. So the original >>>> !is_at_safepoint assertion can just be dropped from both methods. >>>> That in turn means we either drop the JavaThread assertion or else >>>> expand it to be "JavaThread or VMThread" - but that itself seems >>>> unnecessarily restrictive: why can't a GC thread use this code for >>>> example? So I chose to drop it. >>>> >>>> That leaves the "if I'm a JavaThread I'm not _thread_blocked" >>>> assertion. I don't see that this really guards against anything >>>> useful (a JavaThread should be _thread_in_vm when executing this >>>> code ... though there may be leaf calls where the thread is >>>> _thread_in_java). So again I just dropped the assertion. >>>> >>>> So a long winded justification for dropping a few assertions. :) >>>> >>>> Thanks, >>>> David >>>> >>>> >>> > From david.holmes at oracle.com Thu Aug 6 21:52:59 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Aug 2020 07:52:59 +1000 Subject: RFR (xs) 8251209 [TESTBUG] CDS jvmti tests should use "@modules java.instrument" In-Reply-To: References: Message-ID: <2b9000ef-9a09-caf4-8610-bb354afb722b@oracle.com> Hi Ioi, On 7/08/2020 4:25 am, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8251209 > http://cr.openjdk.java.net/~iklam/jdk16/8251209-cds-jvmti-tests-modules-tag.v01/ > > > Summary -- changed the tests from (mis)using > > ?* @requires vm.flavor != "minimal" > > to > > ?* @modules java.instrument > > ... to be consistent with other jvmti tests. That seems like an invalid precondition to me. It would have been somewhat valid in the Compact Profiles world when we did not provide "java.instrument" in the profiles which supported MinimalVM, but you can define a minimal VM in a build that still has all modules available. I don't think building the minimal VM makes any changes to the supported modules. Also AIUI the @modules statement simply adds the necessary command-line args to use the java.instrument module (if present), it doesn't ensure that the listed module has to be present. David > Thanks > - Ioi From coleen.phillimore at oracle.com Thu Aug 6 21:56:26 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 6 Aug 2020 17:56:26 -0400 Subject: RFR (s): 8250606: Remove unnecessary assertions in ObjectSynchronizer FastHashcode and inflate In-Reply-To: <11f913cc-8f88-0740-28d0-a7db7897d6a8@oracle.com> References: <3e4c931a-593b-0336-97b9-3adc4de96062@oracle.com> <17eef440-3575-73f5-1413-0a25da6a13df@oracle.com> <11f913cc-8f88-0740-28d0-a7db7897d6a8@oracle.com> Message-ID: Deleting these asserts looks good to me.? More below. On 8/6/20 2:29 PM, Daniel D. Daugherty wrote: > On 8/5/20 6:31 PM, David Holmes wrote: >> Hi Dan, >> >> Thanks for taking a look at this. >> >> On 6/08/2020 4:08 am, Daniel D. Daugherty wrote: >>> On 8/5/20 3:58 AM, David Holmes wrote: >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8250606 >>>> webrev: http://cr.openjdk.java.net/~dholmes/8250606/webrev/ >>> >>> src/hotspot/share/runtime/synchronizer.cpp >>> ???? L1010: ????? // Relaxing assertion for bug 6320749. >>> ???? L1011: ????? assert(Universe::verify_in_progress() || >>> ???? L1012: !SafepointSynchronize::is_at_safepoint(), >>> ???? L1013: ???????????? "biases should not be seen by VM thread >>> here"); >>> ???????? This assertion is similar and tagged with the same bug ID >>> (6320749) >>> ???????? as the ones that you removed. Granted that the Biased >>> Locking code >>> ???????? is on the way out, but... >>> >>> ???????? I'm fine if you plan to leave this one for Biased Locking >>> removal. >> >> Yeah I confess I had my biased-locking filter on when looking at this >> part of the code. You've actually exposed a bit of a mess. :) >> >> It is unclear why the same !at_safepoint assertion was initially >> added to the biased-locking revocation chunk of code given the same >> assertion would then be executed after it. But I think it is more to >> do with the line following the assertion (original code): >> >> assert(!SafepointSynchronize::is_at_safepoint(), "biases should not >> be seen by VM thread here"); >> BiasedLocking::revoke_and_rebias(hobj, false, JavaThread::current()); >> >> If we are at a safepoint in the VMThread then JavaThread::current() >> will assert! So even after the assertion was apparently relaxed with >> the verify_in_progress() check, we could never have hit this code as >> the JavaThread::current() would still assert. So I can (and should) >> remove the assertion here as well (CDS dump could reach this in >> theory), but I also need to correct this current code: >> >> -????? BiasedLocking::revoke(hobj, JavaThread::current()); >> +????? BiasedLocking::revoke(hobj, self); >> >> as re-manifesting the current thread is pointless. That said the >> above code (nor the original) also can't execute at a safepoint: >> >> void BiasedLocking::revoke(Handle obj, TRAPS) { >> ? assert(!SafepointSynchronize::is_at_safepoint(), "must not be >> called while at safepoint"); > > > With the assert in JavaThread::current() and with the assert in > BiasedLocking::revoke(), I don't see how we could ever get away > with calling FastHashCode() on a biased object at a safepoint. > It simply should have blown up (with non-release bits). > > I'm not yet sure what keeps us from seeing a biased object at > a safepoint, but there has to be something there. We aren't that > lucky to have simply missed this... The original reason for relaxing the asserts for verification was that we were verifying the SystemDictionary during GC, which used Object.hashCode() for Klasses because they were in PermGen.? The Klass's prototype_header was set to never used biased locking. Currently, the only use for calling FastHashCode() in a safepoint is because of dumping objects for CDS, and in this case UseBiasedLocking is switched off when DumpSharedSpaces. So neither path would go through the first 'if' statement in the FastHashCode code. I think you should remove the assert here.? It's obsolete and one less mystery to solve. 1010 // Relaxing assertion for bug 6320749. 1011 assert(Universe::verify_in_progress() || 1012 !SafepointSynchronize::is_at_safepoint(), 1013 "biases should not be seen by VM thread here"); since revoke() asserts !at_safepoint() and change JavaThread::current() to "self". Thanks, Coleen > > Dan > >> >> so even larger changes will be needed to make this code work at a >> safepoint, and that is going well beyond the intended scope of this >> particular bug. :( I'll need to get back to you. >> >> I also need to check our GC verify tests to see how we manage to >> apparently never encounter a biased-locked object, otherwise we would >> have seen this inconsistency in the code. >> >> Thanks, >> David >> ----- >> >> >>> Thumbs up.? Thanks for digging up the history behind these very old >>> asserts. >> >> Thanks, >> David >> >>> Dan >>> >>> >>>> >>>> tl;dr I removed some assertions that were deemed unnecessary after >>>> interfering with other work. :) >>>> >>>> ObjectSynchronizer::inflate had the following assertion added back >>>> in JDK 6 under JDK-5030359: >>>> >>>> ObjectMonitor * ObjectSynchronizer::inflate (oop object) { >>>> ? // Inflate mutates the heap ... >>>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>>> >>>> That same change added FastHashcode which had the following >>>> assertions: >>>> >>>> intptr_t ObjectSynchronizer::FastHashCode (Thread * Self, oop obj) { >>>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>>> ? assert (Self->is_Java_thread(), "invariant") ; >>>> ? assert (((JavaThread *)Self)->thread_state() != _thread_blocked, >>>> "invariant") ; >>>> >>>> These methods had assertions that they were not executed at a >>>> safepoint in case the modification of the markWord could cause >>>> interference with GC. >>>> >>>> JDK-6320749 then relaxed the assertions to allow for GC >>>> verification - which does happen at a safepoint and so obviously >>>> not in a JavaThread. >>>> >>>> The FastHashCode assertions were further relaxed in JDK-8015086 to >>>> allow the CDS dump VM safepoint operation to call it. But >>>> FastHashCode can also call inflate so that was a mismatch in >>>> assertions just waiting to happen - which is how I hit this in a VM >>>> that always forces inflation :) >>>> >>>> I initially intended to extend the FastHashCode assertion to match >>>> inflate, but then a question arose about the safety of the CDS >>>> dumping code using FastHashcode which showed that the code is in >>>> fact safe. Notwithstanding what the state of the world may have >>>> been in the JDK 6 timeframe, there is now no potential interference >>>> possible between execution of these methods at an arbitrary >>>> safepoint and any concurrent GC use of the markWord. So the >>>> original !is_at_safepoint assertion can just be dropped from both >>>> methods. That in turn means we either drop the JavaThread assertion >>>> or else expand it to be "JavaThread or VMThread" - but that itself >>>> seems unnecessarily restrictive: why can't a GC thread use this >>>> code for example? So I chose to drop it. >>>> >>>> That leaves the "if I'm a JavaThread I'm not _thread_blocked" >>>> assertion. I don't see that this really guards against anything >>>> useful (a JavaThread should be _thread_in_vm when executing this >>>> code ... though there may be leaf calls where the thread is >>>> _thread_in_java). So again I just dropped the assertion. >>>> >>>> So a long winded justification for dropping a few assertions. :) >>>> >>>> Thanks, >>>> David >>>> >>>> >>> > From daniel.daugherty at oracle.com Thu Aug 6 22:06:16 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 6 Aug 2020 18:06:16 -0400 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> Message-ID: <0d89c58a-0b70-4458-be91-e50f7a81b37f@oracle.com> > Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev This time on the official 'v3' rather than the 'prelim-v3'. src/hotspot/cpu/aarch64/aarch64.ad ??? No comments. src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp ??? No comments. src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp ??? No comments. src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp ??? L447: void MacroAssembler::biased_locking_enter(Register lock_reg, ??? L448: ???????????????????????????????????????? Register obj_reg, ??? : ??? L454: BiasedLockingCounters* counters) { ??????? nit - The change from 'int' to 'void' changes the indent so ?????? ? ? ?? L448-454 need one more space. src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp ??? L114: ? void biased_locking_enter(Register lock_reg, Register obj_reg, ??? L115: ?????????????????????????? Register swap_reg, Register tmp_reg, ??? : ??? L118: ?????????????????????????? BiasedLockingCounters* counters = NULL); ??????? nit - The change from 'int' to 'void' changes the indent so ????????????? L115-118 need one more space. src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp ??? No comments. src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp ??? No comments. src/hotspot/cpu/arm/interp_masm_arm.cpp ??? No comments. src/hotspot/cpu/arm/macroAssembler_arm.cpp ??? L1325: void MacroAssembler::biased_locking_enter(Register obj_reg, Register swap_reg, Register tmp_reg, ??? L1326: ???????????????????????????????????????? bool swap_reg_contains_mark, ??? : ??? L1329: BiasedLockingCounters* counters) { ??????? nit - The change from 'int' to 'void' changes the indent so ????????????? L1326-1329 need one more space. src/hotspot/cpu/arm/macroAssembler_arm.hpp ??? L381: ? void biased_locking_enter(Register obj_reg, Register swap_reg, Register tmp_reg, ??? L382: ?????????????????????????? bool swap_reg_contains_mark, ??? : ??? L385: ?????????????????????????? BiasedLockingCounters* counters = NULL); ??????? nit - The change from 'int' to 'void' changes the indent so ????????????? L382-385 need one more space. src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp src/hotspot/cpu/ppc/macroAssembler_ppc.cpp ??? No comments on the PPC files. src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp src/hotspot/cpu/s390/interp_masm_s390.cpp src/hotspot/cpu/s390/macroAssembler_s390.cpp ??? No comments on the S390 files. src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp ??? No comments. src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp ??? No comments. src/hotspot/cpu/x86/interp_masm_x86.cpp ??? No comments. src/hotspot/cpu/x86/macroAssembler_x86.cpp ??? L1084: void MacroAssembler::biased_locking_enter(Register lock_reg, ??? L1085: ???????????????????????????????????????? Register obj_reg, ??? : ??? L1092: BiasedLockingCounters* counters) { ??????? nit - The change from 'int' to 'void' changes the indent so ????????????? L1085-1092 need one more space. src/hotspot/cpu/x86/macroAssembler_x86.hpp ??? L664: ? void biased_locking_enter(Register lock_reg, Register obj_reg, ??? L665: ?????????????????????????? Register swap_reg, Register tmp_reg, ??? : ??? L668: ?????????????????????????? BiasedLockingCounters* counters = NULL); ??????? nit - The change from 'int' to 'void' changes the indent so ????????????? L665-668 need one more space. src/hotspot/share/classfile/systemDictionary.cpp ??? No comments. src/hotspot/share/jfr/metadata/metadata.xml ??? No comments. src/hotspot/share/logging/logTag.hpp ??? No comments. src/hotspot/share/oops/klass.hpp ??? No comments. src/hotspot/share/runtime/arguments.cpp ??? No comments. src/hotspot/share/runtime/globals.hpp ??? No comments. src/hotspot/share/runtime/synchronizer.cpp ??? No comments. src/hotspot/share/runtime/synchronizer.hpp ??? No comments. src/hotspot/share/utilities/accessFlags.hpp ??? No comments. src/jdk.jfr/share/conf/jfr/default.jfc ??? No comments. src/jdk.jfr/share/conf/jfr/profile.jfc ??? No comments. test/lib/jdk/test/lib/jfr/EventNames.java ??? No comments. test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java ??? No comments. test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java ??? No comments. Thumbs up. I only have nits above so no need for another webrev if you choose to fix them. Dan On 8/6/20 2:48 PM, Patricio Chilano wrote: > Hi Dan, > > On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: >> I'm peeking ahead to the next webrev... :-) >> >> > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ > : ) > >> General comments: >> ? - check files for copyright year updates. > Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, > c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. > >> src/hotspot/share/runtime/synchronizer.hpp >> ??? No comments. >> >> src/hotspot/share/runtime/synchronizer.cpp >> ??? L507: ? const markWord mark = obj->mark(); >> ??? L508: >> ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && >> obj->klass()->is_box()) { >> ??? L510: ??? return false; >> ??? L511: ? } >> ??????? The new bailout on L509-511 can move above L507. > Moved. > >> L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass >> %s", p2i(obj()), obj->klass()->external_name()); >> ??????? This external_name() call does not have a ResourceMark. > Good catch! I had one in a previous version but then I changed the > conditionals and lost it for the fatal error case. The test worked > okay because for the main JavaThread there is a ResourceMark in > jni_invoke_static() (jni.cpp). > >> src/hotspot/share/logging/logTag.hpp >> ??? No comments. >> >> src/hotspot/share/oops/klass.hpp >> ??? No comments. >> >> src/hotspot/share/utilities/accessFlags.hpp >> ??? No comments. >> >> src/hotspot/share/runtime/globals.hpp >> ??? L814: ???????????? "0: off " >> ??????? Missing a ';' after "off". > Fixed. > >> L816: ???????????? "2: log message to stdout. >> ??????? Perhaps add "(by default)" after "stdout" or >> ??????? don't say where log output is at all. >> >> src/hotspot/share/runtime/arguments.cpp >> ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, true, >> LOG_TAGS(primitivewrappers)); >> ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp >> ??????? since it looks like logging output is configured to 'stdout'. >> ??????? I'm assuming that other log options to put output elsewhere >> ??????? are overridden by this code? > Right. So the logging is done under UL with the tag primitivewrappers. > If that tag is specified in Xlog then this conditional will be skipped > because !log_is_enabled(Info, primitivewrappers) will be false. > >> src/hotspot/share/classfile/systemDictionary.cpp >> ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { >> ??????? nit - s/T_LONG+1/T_LONG + 1/ > Fixed. > >> L2191: _box_klasses[i]->set_prototype_header(markWord::prototype()); >> ??????? I assume we're keeping the prototype_header field when Biased >> Locking >> ??????? goes away? The reason I ask: >> >> ? ? ? ? static markWord prototype() { >> ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); >> ? ? ? ? } >> >> ??????? is because without Biased Locking, do we really need the >> prototype >> ??????? anymore? The initial markWord won't need possible variants... > Yes, I think it can go away unless somebody finds another use for it. > Maybe Valhalla. > >> src/hotspot/share/jfr/metadata/metadata.xml >> ??? L69: ? > ??????? Is the category "Java Application" because it's the application >> ??????? code that did something "wrong" here? Where "application" is >> loosely >> ??????? defined to include the possibility of auto generated code, >> library >> ??????? code and the like? Or perhaps "application" because something >> "above" >> ??????? the "Java Virtual Machine, Runtime" did the "wrong" thing here? > I don't know what the right category should be really. I saw the > events JavaMonitorEnter, JavaMonitorWait and JavaMonitorInflate and > thought this event should fall in the same category they do. > >> src/jdk.jfr/share/conf/jfr/default.jfc >> ??? No comments. >> >> src/jdk.jfr/share/conf/jfr/profile.jfc >> ??? No comments. >> >> test/lib/jdk/test/lib/jfr/EventNames.java >> ??? No comments. >> >> src/hotspot/cpu/aarch64/aarch64.ad >> ??? L3517: ????? __ tbnz(tmp, exact_log2(JVM_ACC_IS_BOX_CLASS), cont); >> >> ??? L3578: ??? __ bind(cont); >> ??? L3579: ??? // flag == EQ indicates success >> ??? L3580: ??? // flag == NE indicates failure >> ??????? If tbnz() branches to "cont" when we have a box class, what's >> ??????? the flag value set to (EQ or NE)? And what set that flag value? >> ??????? The reason I ask is I don't think tbnz() sets condition codes... > Right, it doesn't set condition codes, so I kept the version I had > (more on that below). > >> src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> ??? No comments. >> >> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp >> ??? No comments. >> >> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp >> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >> ??????????? ? ?? ??????????????????????????? Register obj_reg, >> ?????????????????????????????? ? ?? ???????? Register swap_reg, >> ?????????????????????????????????? ? ?? ???? Register tmp_reg, >> ?????????????????????????????????? ? ?? ???? bool >> swap_reg_contains_mark, >> ?????????????????????????????????? ? ?? ???? Label& done, >> ??????????????????????????????????? ? ?? ??? Label* slow_case, >> BiasedLockingCounters* counters) { >> ??????? I think you've changed the only callers of >> biased_locking_enter() >> ??????? that cared about the return value with this changeset so it can >> ??????? be changed to a void function. > Ok, this is what I mentioned to David in a previous email. Done. > >> src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp >> ??? No comments. >> >> src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp >> ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), done); >> >> ??? L131: ? bind(done); >> ??? L132: >> ??? L133: ? // At this point flags are set as follows: >> ??? L134: ? //? EQ -> Success >> ??? L135: ? //? NE -> Failure, branch to slow path >> ??????? If tbnz() branches to "done" when we have a box class, what's >> ??????? the flag value set to (EQ or NE)? And what set that flag value? >> ??????? The reason I ask is I don't think tbnz() sets condition codes... > Right. Same as above, I kept the version I had. > >> src/hotspot/cpu/arm/interp_masm_arm.cpp >> ??? No comments. >> >> src/hotspot/cpu/arm/macroAssembler_arm.cpp >> ??? int MacroAssembler::biased_locking_enter(Register obj_reg, >> Register swap_reg, Register tmp_reg, >> ????????????????????? ? ?? ????????????????? bool >> swap_reg_contains_mark, >> ???????????????????????? ? ?? ?????????????? Register tmp2, >> ????????????????????????? ? ?? ????????????? Label& done, Label& >> slow_case, >> BiasedLockingCounters* counters) { >> ??????? I think you've changed the only callers of >> biased_locking_enter() >> ??????? that cared about the return value with this changeset so it can >> ??????? be changed to a void function. > Done. > >> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >> ??? No comments on the PPC code. >> >> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >> src/hotspot/cpu/s390/interp_masm_s390.cpp >> src/hotspot/cpu/s390/macroAssembler_s390.cpp >> ??? No comments on the S390 code. >> >> src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp >> ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); >> ??????? What will this do with a 'noreg' value? (I need a refresher.) > When not in 64 bit mode that register just won't be used. > >> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp >> ??? No comments. >> >> src/hotspot/cpu/x86/interp_masm_x86.cpp >> ??? No comments. >> >> src/hotspot/cpu/x86/macroAssembler_x86.cpp >> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >> ????????????????????????????????? ? ?? ????? Register obj_reg, >> ????????????????????????????????? ? ?? ????? Register swap_reg, >> ????????????????????????????????? ? ?? ????? Register tmp_reg, >> ?????????????????????????????????? ? ?? ???? Register tmp_reg2, >> ?????????????????????????????????????? ? ? ? bool >> swap_reg_contains_mark, >> ?????????????????????????????????? ? ?? ???? Label& done, >> ?????????????????????????????????? ? ?? ???? Label* slow_case, >> BiasedLockingCounters* counters) { >> ??????? I think you've changed the only caller of biased_locking_enter() >> ??????? that cared about the return value with this changeset so it can >> ??????? be changed to a void function. > Done. > >> test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >> ??? L30:? * @test SyncOnPrimitiveWrapperTest >> ??????? No parameter to @test directive. > Removed. > >> L136: ??????? private static long sharedCounter = 0L; >> ??????? Since you don't do anything with sharedCounter other than >> increment it, >> ??????? can the compilers optimize it away? If it can be optimized >> away, does >> ??????? that mean that: >> >> ??????????? L142: ??????????????? synchronized (obj) { >> >> ??????? can also be optimized away? >> >> ??????? I don't think that: >> >> ??????????? L161: ??????????????? synchronized (sharedLock1) { >> >> ??????? can be optimized away because it is shared by multiple threads. >> >> test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java >> ??? Similar questions about 'counter' being optimized away. >> ??? Similar question about "synchronized (obj) {" being optimized away. > I'm not sure if the compiler will optimize it away. Seems unlikely > given the counter we are incrementing is not just local to some thread. > > > Ok, below is v3 which has the following changes: > - Use a 32 bit load for the _access_flags field, instead of 64 > - Martin's implementation for s390 and fix for PPC > - Faster LogTest version > - Above changes based on Dan review > > I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 only) > again with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I checked it builds > in arm32, ppc and s390. > > Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev > Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev > > > @Martin: > Please check if the test doesn't timeout for you now with the changes > I made to LogTest. > Also, I tried to use tbnz in aarch64 and arm32 instead of tst + br > (except for c2 since we actually need to set the condition flags), but > for c1 I was getting an assertion in the compiler thread: > > guarantee(chk == -1 || chk == 0) failed: Field too big for insn > > This is the stack when that happens: > > MacroAssembler::pd_patch_instruction_size(unsigned char*, unsigned > char*)+0x398 > AbstractAssembler::bind(Label&)+0x118 > MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 > LIR_Assembler::emit_slow_case_stubs()+0x54 > Compilation::emit_code_body()+0x17c > > The change was simply: > > diff --git a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp > b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp > --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp > +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp > @@ -78,7 +78,6 @@ > ?? if (DiagnoseSyncOnPrimitiveWrappers != 0) { > ???? load_klass(hdr, obj); > -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); > -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); > -??? br(Assembler::NE, slow_case); > +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); > +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); > ?? } > > So the issue must be related to where we want to jump. > > Thanks, > Patricio >> Dan >> >> >> On 8/5/20 9:01 AM, Patricio Chilano wrote: >>> Hi Martin, >>> >>> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>>> Hi Patricio, >>>> >>>> using 8 Byte load instructions for jint fields is a terrible coding >>>> style! >>>> Someone else may see it and use an 8 Byte store. Will result in >>>> great fun for debugging! >>>> >>>> And on Big Endian you will simply access the wrong bits. >>> Ah, of course! Those 32 bits will be in the wrong place when doing >>> the test. >>> >>>> Note that Big Endian Platforms are AIX, old linux ppc, s390, SPARC. >>>> I don't think you have tested on them. >>>> >>>>> We could remove the nested synchronized statements in the run() >>>>> method >>>>> so that we don't generate that much logging. We could also lower >>>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>>> with multiple flag combinations. Not sure what we should touch first. >>>>> Maybe the synchronized statements, have only one or two and test that >>>>> first? >>>> Sounds like helpful ideas. Please go ahead and strip things down. >>> Great, I will send v3 later with those changes. >>> >>> Thanks Martin! >>> >>> Patricio >>>> Thanks for taking care of it. >>>> >>>> Best regards, >>>> Martin >>>> >>>> >>>>> -----Original Message----- >>>>> From: Patricio Chilano >>>>> Sent: Dienstag, 4. August 2020 20:26 >>>>> To: Doerr, Martin ; David Holmes >>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>> hotspot-jfr-dev at openjdk.java.net >>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>> wrappers >>>>> >>>>> Hi Martin, >>>>> >>>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>>> Hi Patricio, >>>>>> >>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>> sending v3. >>>>>> Thanks. I forgot to mention arm + aarch64. >>>>>> Aarch64 uses ldrw + tbnz. >>>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>>> >>>>>> I remember having seen the same mistake ?? >>>>>> And nobody noticed it on little Endian platforms. >>>>> Ok, I can use a tbnz instead of test and then a branch on arm32 >>>>> and aarch64. >>>>> Shouldn't a normal ldr in arm32 work fine? >>>>> Also in 64 bits (either x64 or aarch64) I don't see the issue of >>>>> using a >>>>> 64 bit load, besides the fact that we only care about the first 32 >>>>> bits. >>>>> Regardless of the endianness, aren't we masking out the upper part >>>>> when >>>>> we do AND/TEST or if we test a bit in the 0-31 bit range? >>>>> Otherwise it >>>>> seems it should have failed for one of those platforms in my tests. >>>>> >>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>> specify that and see if that fixes it? Let me know if that works >>>>>>> and I >>>>>>> can add it to the test. >>>>>> That seems to have an effect. But now I'm not patient enough to >>>>>> wait for >>>>> the test to finish. >>>>>> Maybe the problem is that I'm using slow debug builds. >>>>>> But is there a chance to make the test quicker without losing >>>>>> coverage >>>>> significantly? >>>>>> I think the test is too slow this way to run it on a regular >>>>>> basis. We'd need to >>>>> spend dedicated machines for it. >>>>> We could remove the nested synchronized statements in the run() >>>>> method >>>>> so that we don't generate that much logging. We could also lower >>>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>>> with multiple flag combinations. Not sure what we should touch first. >>>>> Maybe the synchronized statements, have only one or two and test that >>>>> first? >>>>> >>>>> >>>>> Thanks, >>>>> Patricio >>>>>> Best regards, >>>>>> Martin >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Patricio Chilano >>>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>>> To: Doerr, Martin ; David Holmes >>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>>> wrappers >>>>>>> >>>>>>> Hi Martin, >>>>>>> >>>>>>> Thanks for fixing PPC and taking care of s390! >>>>>>> >>>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>>> Hi Patricio, >>>>>>>> >>>>>>>> I suggest to use movl + testl for checking the access flag as >>>>>>>> for other >>>>> access >>>>>>> flags on x64. >>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>> sending v3. >>>>>>> >>>>>>>> New version for PPC64 and s390 see below. >>>>>>>> >>>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>>>> expected. >>>>>>> However, I'm getting timeout issues: >>>>>>>> timed out (timeout set to 120000ms, elapsed time including timeout >>>>>>> handling was 122372ms) >>>>>>>> Can we provide more time? >>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>> specify that and see if that fixes it? Let me know if that works >>>>>>> and I >>>>>>> can add it to the test. >>>>>>> >>>>>>> Thanks, >>>>>>> Patricio >>>>>>>> Best regards, >>>>>>>> Martin >>>>>>>> >>>>>>>> >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -107,8 +107,8 @@ >>>>>>>> >>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>> Rscratch); >>>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>>> +??? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>> Rscratch); >>>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> ??????? bne(CCR0, slow_case); >>>>>>>> ????? } >>>>>>>> >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -912,8 +912,8 @@ >>>>>>>> >>>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> ????????? load_klass(tmp, object); >>>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>> +????? testbitdi(CCR0, R0, tmp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> ????????? bne(CCR0, slow_case); >>>>>>>> ??????? } >>>>>>>> >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>>> >>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> ??????? load_klass(temp, oop); >>>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>> +??? testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> ??????? bne(CCR0, cont); >>>>>>>> ????? } >>>>>>>> >>>>>>>> diff -r 77852e129401 >>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -91,6 +91,12 @@ >>>>>>>> ????? // Save object being locked into the BasicObjectLock... >>>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> +??? z_btrue(slow_case); >>>>>>>> +? } >>>>>>>> + >>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>>> Z_R0_scratch, done, >>>>>>> &slow_case); >>>>>>>> ????? } >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>> 10:03:57 >>>>>>> 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>> 16:04:31 >>>>>>> 2020 +0200 >>>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>>> ????? z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), >>>>>>>> object); >>>>>>>> >>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> +??? z_btrue(slow_case); >>>>>>>> +? } >>>>>>>> + >>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>> ??????? biased_locking_enter(object, displaced_header, Z_R1, >>>>>>>> Z_R0, done, >>>>>>> &slow_case); >>>>>>>> ????? } >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>>> ????? // Load markWord from oop into mark. >>>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>>> >>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> +??? z_btrue(done); >>>>>>>> +? } >>>>>>>> + >>>>>>>> ????? if (try_bias) { >>>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, Z_R0, >>>>>>>> done); >>>>>>>> ????? } >>>>>>>> >>> >> > From coleen.phillimore at oracle.com Thu Aug 6 22:17:25 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 6 Aug 2020 18:17:25 -0400 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> Message-ID: Patricio,? One question: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/src/hotspot/share/runtime/synchronizer.cpp.udiff.html + // adjust bcp to point back to monitorenter so that we print the correct line numbers How do the fatal and event print the correct line numbers here?? I see the logging gets it from the stack trace.? Should the abort message have more information in it?? You can get the source and line number information in the same way that print_stack_on() gets it. Otherwise, this looks good to me. Coleen On 8/6/20 2:48 PM, Patricio Chilano wrote: > Hi Dan, > > On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: >> I'm peeking ahead to the next webrev... :-) >> >> > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ > : ) > >> General comments: >> ? - check files for copyright year updates. > Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, > c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. > >> src/hotspot/share/runtime/synchronizer.hpp >> ??? No comments. >> >> src/hotspot/share/runtime/synchronizer.cpp >> ??? L507: ? const markWord mark = obj->mark(); >> ??? L508: >> ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && >> obj->klass()->is_box()) { >> ??? L510: ??? return false; >> ??? L511: ? } >> ??????? The new bailout on L509-511 can move above L507. > Moved. > >> L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass >> %s", p2i(obj()), obj->klass()->external_name()); >> ??????? This external_name() call does not have a ResourceMark. > Good catch! I had one in a previous version but then I changed the > conditionals and lost it for the fatal error case. The test worked > okay because for the main JavaThread there is a ResourceMark in > jni_invoke_static() (jni.cpp). > >> src/hotspot/share/logging/logTag.hpp >> ??? No comments. >> >> src/hotspot/share/oops/klass.hpp >> ??? No comments. >> >> src/hotspot/share/utilities/accessFlags.hpp >> ??? No comments. >> >> src/hotspot/share/runtime/globals.hpp >> ??? L814: ???????????? "0: off " >> ??????? Missing a ';' after "off". > Fixed. > >> L816: ???????????? "2: log message to stdout. >> ??????? Perhaps add "(by default)" after "stdout" or >> ??????? don't say where log output is at all. >> >> src/hotspot/share/runtime/arguments.cpp >> ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, true, >> LOG_TAGS(primitivewrappers)); >> ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp >> ??????? since it looks like logging output is configured to 'stdout'. >> ??????? I'm assuming that other log options to put output elsewhere >> ??????? are overridden by this code? > Right. So the logging is done under UL with the tag primitivewrappers. > If that tag is specified in Xlog then this conditional will be skipped > because !log_is_enabled(Info, primitivewrappers) will be false. > >> src/hotspot/share/classfile/systemDictionary.cpp >> ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { >> ??????? nit - s/T_LONG+1/T_LONG + 1/ > Fixed. > >> L2191: _box_klasses[i]->set_prototype_header(markWord::prototype()); >> ??????? I assume we're keeping the prototype_header field when Biased >> Locking >> ??????? goes away? The reason I ask: >> >> ? ? ? ? static markWord prototype() { >> ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); >> ? ? ? ? } >> >> ??????? is because without Biased Locking, do we really need the >> prototype >> ??????? anymore? The initial markWord won't need possible variants... > Yes, I think it can go away unless somebody finds another use for it. > Maybe Valhalla. > >> src/hotspot/share/jfr/metadata/metadata.xml >> ??? L69: ? > ??????? Is the category "Java Application" because it's the application >> ??????? code that did something "wrong" here? Where "application" is >> loosely >> ??????? defined to include the possibility of auto generated code, >> library >> ??????? code and the like? Or perhaps "application" because something >> "above" >> ??????? the "Java Virtual Machine, Runtime" did the "wrong" thing here? > I don't know what the right category should be really. I saw the > events JavaMonitorEnter, JavaMonitorWait and JavaMonitorInflate and > thought this event should fall in the same category they do. > >> src/jdk.jfr/share/conf/jfr/default.jfc >> ??? No comments. >> >> src/jdk.jfr/share/conf/jfr/profile.jfc >> ??? No comments. >> >> test/lib/jdk/test/lib/jfr/EventNames.java >> ??? No comments. >> >> src/hotspot/cpu/aarch64/aarch64.ad >> ??? L3517: ????? __ tbnz(tmp, exact_log2(JVM_ACC_IS_BOX_CLASS), cont); >> >> ??? L3578: ??? __ bind(cont); >> ??? L3579: ??? // flag == EQ indicates success >> ??? L3580: ??? // flag == NE indicates failure >> ??????? If tbnz() branches to "cont" when we have a box class, what's >> ??????? the flag value set to (EQ or NE)? And what set that flag value? >> ??????? The reason I ask is I don't think tbnz() sets condition codes... > Right, it doesn't set condition codes, so I kept the version I had > (more on that below). > >> src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> ??? No comments. >> >> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp >> ??? No comments. >> >> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp >> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >> ??????????? ? ?? ??????????????????????????? Register obj_reg, >> ?????????????????????????????? ? ?? ???????? Register swap_reg, >> ?????????????????????????????????? ? ?? ???? Register tmp_reg, >> ?????????????????????????????????? ? ?? ???? bool >> swap_reg_contains_mark, >> ?????????????????????????????????? ? ?? ???? Label& done, >> ??????????????????????????????????? ? ?? ??? Label* slow_case, >> BiasedLockingCounters* counters) { >> ??????? I think you've changed the only callers of >> biased_locking_enter() >> ??????? that cared about the return value with this changeset so it can >> ??????? be changed to a void function. > Ok, this is what I mentioned to David in a previous email. Done. > >> src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp >> ??? No comments. >> >> src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp >> ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), done); >> >> ??? L131: ? bind(done); >> ??? L132: >> ??? L133: ? // At this point flags are set as follows: >> ??? L134: ? //? EQ -> Success >> ??? L135: ? //? NE -> Failure, branch to slow path >> ??????? If tbnz() branches to "done" when we have a box class, what's >> ??????? the flag value set to (EQ or NE)? And what set that flag value? >> ??????? The reason I ask is I don't think tbnz() sets condition codes... > Right. Same as above, I kept the version I had. > >> src/hotspot/cpu/arm/interp_masm_arm.cpp >> ??? No comments. >> >> src/hotspot/cpu/arm/macroAssembler_arm.cpp >> ??? int MacroAssembler::biased_locking_enter(Register obj_reg, >> Register swap_reg, Register tmp_reg, >> ????????????????????? ? ?? ????????????????? bool >> swap_reg_contains_mark, >> ???????????????????????? ? ?? ?????????????? Register tmp2, >> ????????????????????????? ? ?? ????????????? Label& done, Label& >> slow_case, >> BiasedLockingCounters* counters) { >> ??????? I think you've changed the only callers of >> biased_locking_enter() >> ??????? that cared about the return value with this changeset so it can >> ??????? be changed to a void function. > Done. > >> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >> ??? No comments on the PPC code. >> >> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >> src/hotspot/cpu/s390/interp_masm_s390.cpp >> src/hotspot/cpu/s390/macroAssembler_s390.cpp >> ??? No comments on the S390 code. >> >> src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp >> ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); >> ??????? What will this do with a 'noreg' value? (I need a refresher.) > When not in 64 bit mode that register just won't be used. > >> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp >> ??? No comments. >> >> src/hotspot/cpu/x86/interp_masm_x86.cpp >> ??? No comments. >> >> src/hotspot/cpu/x86/macroAssembler_x86.cpp >> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >> ????????????????????????????????? ? ?? ????? Register obj_reg, >> ????????????????????????????????? ? ?? ????? Register swap_reg, >> ????????????????????????????????? ? ?? ????? Register tmp_reg, >> ?????????????????????????????????? ? ?? ???? Register tmp_reg2, >> ?????????????????????????????????????? ? ? ? bool >> swap_reg_contains_mark, >> ?????????????????????????????????? ? ?? ???? Label& done, >> ?????????????????????????????????? ? ?? ???? Label* slow_case, >> BiasedLockingCounters* counters) { >> ??????? I think you've changed the only caller of biased_locking_enter() >> ??????? that cared about the return value with this changeset so it can >> ??????? be changed to a void function. > Done. > >> test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >> ??? L30:? * @test SyncOnPrimitiveWrapperTest >> ??????? No parameter to @test directive. > Removed. > >> L136: ??????? private static long sharedCounter = 0L; >> ??????? Since you don't do anything with sharedCounter other than >> increment it, >> ??????? can the compilers optimize it away? If it can be optimized >> away, does >> ??????? that mean that: >> >> ??????????? L142: ??????????????? synchronized (obj) { >> >> ??????? can also be optimized away? >> >> ??????? I don't think that: >> >> ??????????? L161: ??????????????? synchronized (sharedLock1) { >> >> ??????? can be optimized away because it is shared by multiple threads. >> >> test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java >> ??? Similar questions about 'counter' being optimized away. >> ??? Similar question about "synchronized (obj) {" being optimized away. > I'm not sure if the compiler will optimize it away. Seems unlikely > given the counter we are incrementing is not just local to some thread. > > > Ok, below is v3 which has the following changes: > - Use a 32 bit load for the _access_flags field, instead of 64 > - Martin's implementation for s390 and fix for PPC > - Faster LogTest version > - Above changes based on Dan review > > I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 only) > again with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I checked it builds > in arm32, ppc and s390. > > Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev > Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev > > > @Martin: > Please check if the test doesn't timeout for you now with the changes > I made to LogTest. > Also, I tried to use tbnz in aarch64 and arm32 instead of tst + br > (except for c2 since we actually need to set the condition flags), but > for c1 I was getting an assertion in the compiler thread: > > guarantee(chk == -1 || chk == 0) failed: Field too big for insn > > This is the stack when that happens: > > MacroAssembler::pd_patch_instruction_size(unsigned char*, unsigned > char*)+0x398 > AbstractAssembler::bind(Label&)+0x118 > MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 > LIR_Assembler::emit_slow_case_stubs()+0x54 > Compilation::emit_code_body()+0x17c > > The change was simply: > > diff --git a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp > b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp > --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp > +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp > @@ -78,7 +78,6 @@ > ?? if (DiagnoseSyncOnPrimitiveWrappers != 0) { > ???? load_klass(hdr, obj); > -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); > -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); > -??? br(Assembler::NE, slow_case); > +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); > +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); > ?? } > > So the issue must be related to where we want to jump. > > Thanks, > Patricio >> Dan >> >> >> On 8/5/20 9:01 AM, Patricio Chilano wrote: >>> Hi Martin, >>> >>> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>>> Hi Patricio, >>>> >>>> using 8 Byte load instructions for jint fields is a terrible coding >>>> style! >>>> Someone else may see it and use an 8 Byte store. Will result in >>>> great fun for debugging! >>>> >>>> And on Big Endian you will simply access the wrong bits. >>> Ah, of course! Those 32 bits will be in the wrong place when doing >>> the test. >>> >>>> Note that Big Endian Platforms are AIX, old linux ppc, s390, SPARC. >>>> I don't think you have tested on them. >>>> >>>>> We could remove the nested synchronized statements in the run() >>>>> method >>>>> so that we don't generate that much logging. We could also lower >>>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>>> with multiple flag combinations. Not sure what we should touch first. >>>>> Maybe the synchronized statements, have only one or two and test that >>>>> first? >>>> Sounds like helpful ideas. Please go ahead and strip things down. >>> Great, I will send v3 later with those changes. >>> >>> Thanks Martin! >>> >>> Patricio >>>> Thanks for taking care of it. >>>> >>>> Best regards, >>>> Martin >>>> >>>> >>>>> -----Original Message----- >>>>> From: Patricio Chilano >>>>> Sent: Dienstag, 4. August 2020 20:26 >>>>> To: Doerr, Martin ; David Holmes >>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>> hotspot-jfr-dev at openjdk.java.net >>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>> wrappers >>>>> >>>>> Hi Martin, >>>>> >>>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>>> Hi Patricio, >>>>>> >>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>> sending v3. >>>>>> Thanks. I forgot to mention arm + aarch64. >>>>>> Aarch64 uses ldrw + tbnz. >>>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>>> >>>>>> I remember having seen the same mistake ?? >>>>>> And nobody noticed it on little Endian platforms. >>>>> Ok, I can use a tbnz instead of test and then a branch on arm32 >>>>> and aarch64. >>>>> Shouldn't a normal ldr in arm32 work fine? >>>>> Also in 64 bits (either x64 or aarch64) I don't see the issue of >>>>> using a >>>>> 64 bit load, besides the fact that we only care about the first 32 >>>>> bits. >>>>> Regardless of the endianness, aren't we masking out the upper part >>>>> when >>>>> we do AND/TEST or if we test a bit in the 0-31 bit range? >>>>> Otherwise it >>>>> seems it should have failed for one of those platforms in my tests. >>>>> >>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>> specify that and see if that fixes it? Let me know if that works >>>>>>> and I >>>>>>> can add it to the test. >>>>>> That seems to have an effect. But now I'm not patient enough to >>>>>> wait for >>>>> the test to finish. >>>>>> Maybe the problem is that I'm using slow debug builds. >>>>>> But is there a chance to make the test quicker without losing >>>>>> coverage >>>>> significantly? >>>>>> I think the test is too slow this way to run it on a regular >>>>>> basis. We'd need to >>>>> spend dedicated machines for it. >>>>> We could remove the nested synchronized statements in the run() >>>>> method >>>>> so that we don't generate that much logging. We could also lower >>>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>>> with multiple flag combinations. Not sure what we should touch first. >>>>> Maybe the synchronized statements, have only one or two and test that >>>>> first? >>>>> >>>>> >>>>> Thanks, >>>>> Patricio >>>>>> Best regards, >>>>>> Martin >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Patricio Chilano >>>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>>> To: Doerr, Martin ; David Holmes >>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>>> wrappers >>>>>>> >>>>>>> Hi Martin, >>>>>>> >>>>>>> Thanks for fixing PPC and taking care of s390! >>>>>>> >>>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>>> Hi Patricio, >>>>>>>> >>>>>>>> I suggest to use movl + testl for checking the access flag as >>>>>>>> for other >>>>> access >>>>>>> flags on x64. >>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>> sending v3. >>>>>>> >>>>>>>> New version for PPC64 and s390 see below. >>>>>>>> >>>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>>>> expected. >>>>>>> However, I'm getting timeout issues: >>>>>>>> timed out (timeout set to 120000ms, elapsed time including timeout >>>>>>> handling was 122372ms) >>>>>>>> Can we provide more time? >>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>> specify that and see if that fixes it? Let me know if that works >>>>>>> and I >>>>>>> can add it to the test. >>>>>>> >>>>>>> Thanks, >>>>>>> Patricio >>>>>>>> Best regards, >>>>>>>> Martin >>>>>>>> >>>>>>>> >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -107,8 +107,8 @@ >>>>>>>> >>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>> Rscratch); >>>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>>> +??? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>> Rscratch); >>>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> ??????? bne(CCR0, slow_case); >>>>>>>> ????? } >>>>>>>> >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -912,8 +912,8 @@ >>>>>>>> >>>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> ????????? load_klass(tmp, object); >>>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>> +????? testbitdi(CCR0, R0, tmp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> ????????? bne(CCR0, slow_case); >>>>>>>> ??????? } >>>>>>>> >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>>> >>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> ??????? load_klass(temp, oop); >>>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>> +??? testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> ??????? bne(CCR0, cont); >>>>>>>> ????? } >>>>>>>> >>>>>>>> diff -r 77852e129401 >>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -91,6 +91,12 @@ >>>>>>>> ????? // Save object being locked into the BasicObjectLock... >>>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> +??? z_btrue(slow_case); >>>>>>>> +? } >>>>>>>> + >>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>>> Z_R0_scratch, done, >>>>>>> &slow_case); >>>>>>>> ????? } >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>> 10:03:57 >>>>>>> 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>> 16:04:31 >>>>>>> 2020 +0200 >>>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>>> ????? z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), >>>>>>>> object); >>>>>>>> >>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> +??? z_btrue(slow_case); >>>>>>>> +? } >>>>>>>> + >>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>> ??????? biased_locking_enter(object, displaced_header, Z_R1, >>>>>>>> Z_R0, done, >>>>>>> &slow_case); >>>>>>>> ????? } >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>>> ????? // Load markWord from oop into mark. >>>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>>> >>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> +??? z_btrue(done); >>>>>>>> +? } >>>>>>>> + >>>>>>>> ????? if (try_bias) { >>>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, Z_R0, >>>>>>>> done); >>>>>>>> ????? } >>>>>>>> >>> >> > From ioi.lam at oracle.com Thu Aug 6 23:37:26 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 6 Aug 2020 16:37:26 -0700 Subject: RFR (s): 8250606: Remove unnecessary assertions in ObjectSynchronizer FastHashcode and inflate In-Reply-To: <63ee179e-7133-3edb-c952-1b5fd4f334bd@oracle.com> References: <3e4c931a-593b-0336-97b9-3adc4de96062@oracle.com> <17eef440-3575-73f5-1413-0a25da6a13df@oracle.com> <11f913cc-8f88-0740-28d0-a7db7897d6a8@oracle.com> <63ee179e-7133-3edb-c952-1b5fd4f334bd@oracle.com> Message-ID: <7370d58f-1f9c-ccfd-9056-71d0035d15c5@oracle.com> On 8/6/20 2:46 PM, David Holmes wrote: > Hi Dan, > > On 7/08/2020 4:29 am, Daniel D. Daugherty wrote: >> On 8/5/20 6:31 PM, David Holmes wrote: >>> Hi Dan, >>> >>> Thanks for taking a look at this. >>> >>> On 6/08/2020 4:08 am, Daniel D. Daugherty wrote: >>>> On 8/5/20 3:58 AM, David Holmes wrote: >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8250606 >>>>> webrev: http://cr.openjdk.java.net/~dholmes/8250606/webrev/ >>>> >>>> src/hotspot/share/runtime/synchronizer.cpp >>>> ???? L1010: ????? // Relaxing assertion for bug 6320749. >>>> ???? L1011: ????? assert(Universe::verify_in_progress() || >>>> ???? L1012: !SafepointSynchronize::is_at_safepoint(), >>>> ???? L1013: ???????????? "biases should not be seen by VM thread >>>> here"); >>>> ???????? This assertion is similar and tagged with the same bug ID >>>> (6320749) >>>> ???????? as the ones that you removed. Granted that the Biased >>>> Locking code >>>> ???????? is on the way out, but... >>>> >>>> ???????? I'm fine if you plan to leave this one for Biased Locking >>>> removal. >>> >>> Yeah I confess I had my biased-locking filter on when looking at >>> this part of the code. You've actually exposed a bit of a mess. :) >>> >>> It is unclear why the same !at_safepoint assertion was initially >>> added to the biased-locking revocation chunk of code given the same >>> assertion would then be executed after it. But I think it is more to >>> do with the line following the assertion (original code): >>> >>> assert(!SafepointSynchronize::is_at_safepoint(), "biases should not >>> be seen by VM thread here"); >>> BiasedLocking::revoke_and_rebias(hobj, false, JavaThread::current()); >>> >>> If we are at a safepoint in the VMThread then JavaThread::current() >>> will assert! So even after the assertion was apparently relaxed with >>> the verify_in_progress() check, we could never have hit this code as >>> the JavaThread::current() would still assert. So I can (and should) >>> remove the assertion here as well (CDS dump could reach this in >>> theory), but I also need to correct this current code: >>> >>> -????? BiasedLocking::revoke(hobj, JavaThread::current()); >>> +????? BiasedLocking::revoke(hobj, self); >>> >>> as re-manifesting the current thread is pointless. That said the >>> above code (nor the original) also can't execute at a safepoint: >>> >>> void BiasedLocking::revoke(Handle obj, TRAPS) { >>> ? assert(!SafepointSynchronize::is_at_safepoint(), "must not be >>> called while at safepoint"); >> >> >> With the assert in JavaThread::current() and with the assert in >> BiasedLocking::revoke(), I don't see how we could ever get away >> with calling FastHashCode() on a biased object at a safepoint. >> It simply should have blown up (with non-release bits). >> >> I'm not yet sure what keeps us from seeing a biased object at >> a safepoint, but there has to be something there. We aren't that >> lucky to have simply missed this... > > Seems we are that lucky in a sense. It seems that the verification > process that we allow for is not the same as it once was and so we > don't hit the code that needs the FastHashCode. The only other > safepoint op that gets involved is CDS dumping and there we also get > lucky in that none of the objects that get archived are bias-locked. > I've been using Ioi's test from > > http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ > > > to force the object to be biased but I still can't hit that code > fragment so something else is still in play. > > David > ----- > Oh I feel so dumb. I forgot that we force UseBiasedLocking to false when CDS is enabled (arguments.cpp)! ? if (DumpSharedSpaces) { ??? // Disable biased locking now as it interferes with the clean up of ??? // the archived Klasses and Java string objects (at dump time only). ??? UseBiasedLocking = false; Anyway, it seems like UseBiasedLocking is not compatible with CDS in many ways. I have updated the patch to disable the above line. I also use a more reliably way get a biased-locked object. The test locks on an interned string, which we know for sure no one else would lock, or compute identity_hash of it. http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03-with-debugging/ and when dumping with -XX:+UseBiasedLocking, we crash while doing GC inside a safepoint: #19 0x00007ffff5bb1a80 in oopDesc::set_mark (this=0xbcbcbcbcbcbcbcbc, m=...) #20 0x00007ffff5bb0d0b in BiasedLocking::restore_marks () #21 0x00007ffff5fa3515 in G1FullCollector::complete_collection (...) #22 0x00007ffff5f68b57 in G1CollectedHeap::do_full_collection (...) #23 0x00007ffff5f68bac in G1CollectedHeap::do_full_collection (...) #24 0x00007ffff5dc7aad in CollectedHeap::collect_as_vm_thread (...) #25 0x00007ffff60885f4 in HeapShared::run_full_gc_in_vm_thread () So it looks like the G1GC doesn't support collection inside the VM thread with biased-locked objects. Is biased locking going to be removed soon? I think we probably shouldn't spend too much time on it. Maybe instead of removing the asserts, just enclose them inside "if UseBiasedLocking"? Thanks - Ioi >> Dan >> >>> >>> so even larger changes will be needed to make this code work at a >>> safepoint, and that is going well beyond the intended scope of this >>> particular bug. :( I'll need to get back to you. >>> >>> I also need to check our GC verify tests to see how we manage to >>> apparently never encounter a biased-locked object, otherwise we >>> would have seen this inconsistency in the code. >>> >>> Thanks, >>> David >>> ----- >>> >>> >>>> Thumbs up.? Thanks for digging up the history behind these very old >>>> asserts. >>> >>> Thanks, >>> David >>> >>>> Dan >>>> >>>> >>>>> >>>>> tl;dr I removed some assertions that were deemed unnecessary after >>>>> interfering with other work. :) >>>>> >>>>> ObjectSynchronizer::inflate had the following assertion added back >>>>> in JDK 6 under JDK-5030359: >>>>> >>>>> ObjectMonitor * ObjectSynchronizer::inflate (oop object) { >>>>> ? // Inflate mutates the heap ... >>>>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>>>> >>>>> That same change added FastHashcode which had the following >>>>> assertions: >>>>> >>>>> intptr_t ObjectSynchronizer::FastHashCode (Thread * Self, oop obj) { >>>>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>>>> ? assert (Self->is_Java_thread(), "invariant") ; >>>>> ? assert (((JavaThread *)Self)->thread_state() != _thread_blocked, >>>>> "invariant") ; >>>>> >>>>> These methods had assertions that they were not executed at a >>>>> safepoint in case the modification of the markWord could cause >>>>> interference with GC. >>>>> >>>>> JDK-6320749 then relaxed the assertions to allow for GC >>>>> verification - which does happen at a safepoint and so obviously >>>>> not in a JavaThread. >>>>> >>>>> The FastHashCode assertions were further relaxed in JDK-8015086 to >>>>> allow the CDS dump VM safepoint operation to call it. But >>>>> FastHashCode can also call inflate so that was a mismatch in >>>>> assertions just waiting to happen - which is how I hit this in a >>>>> VM that always forces inflation :) >>>>> >>>>> I initially intended to extend the FastHashCode assertion to match >>>>> inflate, but then a question arose about the safety of the CDS >>>>> dumping code using FastHashcode which showed that the code is in >>>>> fact safe. Notwithstanding what the state of the world may have >>>>> been in the JDK 6 timeframe, there is now no potential >>>>> interference possible between execution of these methods at an >>>>> arbitrary safepoint and any concurrent GC use of the markWord. So >>>>> the original !is_at_safepoint assertion can just be dropped from >>>>> both methods. That in turn means we either drop the JavaThread >>>>> assertion or else expand it to be "JavaThread or VMThread" - but >>>>> that itself seems unnecessarily restrictive: why can't a GC thread >>>>> use this code for example? So I chose to drop it. >>>>> >>>>> That leaves the "if I'm a JavaThread I'm not _thread_blocked" >>>>> assertion. I don't see that this really guards against anything >>>>> useful (a JavaThread should be _thread_in_vm when executing this >>>>> code ... though there may be leaf calls where the thread is >>>>> _thread_in_java). So again I just dropped the assertion. >>>>> >>>>> So a long winded justification for dropping a few assertions. :) >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> >>>> >> From david.holmes at oracle.com Thu Aug 6 23:49:57 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Aug 2020 09:49:57 +1000 Subject: RFR (s): 8250606: Remove unnecessary assertions in ObjectSynchronizer FastHashcode and inflate In-Reply-To: <7370d58f-1f9c-ccfd-9056-71d0035d15c5@oracle.com> References: <3e4c931a-593b-0336-97b9-3adc4de96062@oracle.com> <17eef440-3575-73f5-1413-0a25da6a13df@oracle.com> <11f913cc-8f88-0740-28d0-a7db7897d6a8@oracle.com> <63ee179e-7133-3edb-c952-1b5fd4f334bd@oracle.com> <7370d58f-1f9c-ccfd-9056-71d0035d15c5@oracle.com> Message-ID: <1dfdfaeb-f0b2-f67b-2837-1660bc1ff733@oracle.com> On 7/08/2020 9:37 am, Ioi Lam wrote: > > > On 8/6/20 2:46 PM, David Holmes wrote: >> Hi Dan, >> >> On 7/08/2020 4:29 am, Daniel D. Daugherty wrote: >>> On 8/5/20 6:31 PM, David Holmes wrote: >>>> Hi Dan, >>>> >>>> Thanks for taking a look at this. >>>> >>>> On 6/08/2020 4:08 am, Daniel D. Daugherty wrote: >>>>> On 8/5/20 3:58 AM, David Holmes wrote: >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8250606 >>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8250606/webrev/ >>>>> >>>>> src/hotspot/share/runtime/synchronizer.cpp >>>>> ???? L1010: ????? // Relaxing assertion for bug 6320749. >>>>> ???? L1011: ????? assert(Universe::verify_in_progress() || >>>>> ???? L1012: !SafepointSynchronize::is_at_safepoint(), >>>>> ???? L1013: ???????????? "biases should not be seen by VM thread >>>>> here"); >>>>> ???????? This assertion is similar and tagged with the same bug ID >>>>> (6320749) >>>>> ???????? as the ones that you removed. Granted that the Biased >>>>> Locking code >>>>> ???????? is on the way out, but... >>>>> >>>>> ???????? I'm fine if you plan to leave this one for Biased Locking >>>>> removal. >>>> >>>> Yeah I confess I had my biased-locking filter on when looking at >>>> this part of the code. You've actually exposed a bit of a mess. :) >>>> >>>> It is unclear why the same !at_safepoint assertion was initially >>>> added to the biased-locking revocation chunk of code given the same >>>> assertion would then be executed after it. But I think it is more to >>>> do with the line following the assertion (original code): >>>> >>>> assert(!SafepointSynchronize::is_at_safepoint(), "biases should not >>>> be seen by VM thread here"); >>>> BiasedLocking::revoke_and_rebias(hobj, false, JavaThread::current()); >>>> >>>> If we are at a safepoint in the VMThread then JavaThread::current() >>>> will assert! So even after the assertion was apparently relaxed with >>>> the verify_in_progress() check, we could never have hit this code as >>>> the JavaThread::current() would still assert. So I can (and should) >>>> remove the assertion here as well (CDS dump could reach this in >>>> theory), but I also need to correct this current code: >>>> >>>> -????? BiasedLocking::revoke(hobj, JavaThread::current()); >>>> +????? BiasedLocking::revoke(hobj, self); >>>> >>>> as re-manifesting the current thread is pointless. That said the >>>> above code (nor the original) also can't execute at a safepoint: >>>> >>>> void BiasedLocking::revoke(Handle obj, TRAPS) { >>>> ? assert(!SafepointSynchronize::is_at_safepoint(), "must not be >>>> called while at safepoint"); >>> >>> >>> With the assert in JavaThread::current() and with the assert in >>> BiasedLocking::revoke(), I don't see how we could ever get away >>> with calling FastHashCode() on a biased object at a safepoint. >>> It simply should have blown up (with non-release bits). >>> >>> I'm not yet sure what keeps us from seeing a biased object at >>> a safepoint, but there has to be something there. We aren't that >>> lucky to have simply missed this... >> >> Seems we are that lucky in a sense. It seems that the verification >> process that we allow for is not the same as it once was and so we >> don't hit the code that needs the FastHashCode. The only other >> safepoint op that gets involved is CDS dumping and there we also get >> lucky in that none of the objects that get archived are bias-locked. >> I've been using Ioi's test from >> >> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ >> >> >> to force the object to be biased but I still can't hit that code >> fragment so something else is still in play. >> >> David >> ----- >> > > Oh I feel so dumb. I forgot that we force UseBiasedLocking to false when > CDS is enabled (arguments.cpp)! > > ? if (DumpSharedSpaces) { > ??? // Disable biased locking now as it interferes with the clean up of > ??? // the archived Klasses and Java string objects (at dump time only). > ??? UseBiasedLocking = false; Ah! :( > Anyway, it seems like UseBiasedLocking is not compatible with CDS in > many ways. I have updated the patch to disable the above line. I also > use a more reliably way get a biased-locked object. The test locks on an > interned string, which we know for sure no one else would lock, or > compute identity_hash of it. > > http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03-with-debugging/ > > > and when dumping with -XX:+UseBiasedLocking, we crash while doing GC > inside a safepoint: > > #19 0x00007ffff5bb1a80 in oopDesc::set_mark (this=0xbcbcbcbcbcbcbcbc, > m=...) > #20 0x00007ffff5bb0d0b in BiasedLocking::restore_marks () Know bug: JDK-8251118 > #21 0x00007ffff5fa3515 in G1FullCollector::complete_collection (...) > #22 0x00007ffff5f68b57 in G1CollectedHeap::do_full_collection (...) > #23 0x00007ffff5f68bac in G1CollectedHeap::do_full_collection (...) > #24 0x00007ffff5dc7aad in CollectedHeap::collect_as_vm_thread (...) > #25 0x00007ffff60885f4 in HeapShared::run_full_gc_in_vm_thread () > > So it looks like the G1GC doesn't support collection inside the VM > thread with biased-locked objects. > > Is biased locking going to be removed soon? I think we probably > shouldn't spend too much time on it. There was some feedback on the disabling that indicated it may still be useful so that decision to proceed with the removal has yet to occur. > Maybe instead of removing the asserts, just enclose them inside "if > UseBiasedLocking"? I think I'm going to fix the BL block of code separately. There's no reason I can see why a biased-locked object should be a concern in FastHashCode, we just need to change the revocation code to allow for the safepoint (which is exactly what we do elsewhere). Thanks, David > Thanks > - Ioi > > >>> Dan >>> >>>> >>>> so even larger changes will be needed to make this code work at a >>>> safepoint, and that is going well beyond the intended scope of this >>>> particular bug. :( I'll need to get back to you. >>>> >>>> I also need to check our GC verify tests to see how we manage to >>>> apparently never encounter a biased-locked object, otherwise we >>>> would have seen this inconsistency in the code. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> >>>>> Thumbs up.? Thanks for digging up the history behind these very old >>>>> asserts. >>>> >>>> Thanks, >>>> David >>>> >>>>> Dan >>>>> >>>>> >>>>>> >>>>>> tl;dr I removed some assertions that were deemed unnecessary after >>>>>> interfering with other work. :) >>>>>> >>>>>> ObjectSynchronizer::inflate had the following assertion added back >>>>>> in JDK 6 under JDK-5030359: >>>>>> >>>>>> ObjectMonitor * ObjectSynchronizer::inflate (oop object) { >>>>>> ? // Inflate mutates the heap ... >>>>>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>>>>> >>>>>> That same change added FastHashcode which had the following >>>>>> assertions: >>>>>> >>>>>> intptr_t ObjectSynchronizer::FastHashCode (Thread * Self, oop obj) { >>>>>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>>>>> ? assert (Self->is_Java_thread(), "invariant") ; >>>>>> ? assert (((JavaThread *)Self)->thread_state() != _thread_blocked, >>>>>> "invariant") ; >>>>>> >>>>>> These methods had assertions that they were not executed at a >>>>>> safepoint in case the modification of the markWord could cause >>>>>> interference with GC. >>>>>> >>>>>> JDK-6320749 then relaxed the assertions to allow for GC >>>>>> verification - which does happen at a safepoint and so obviously >>>>>> not in a JavaThread. >>>>>> >>>>>> The FastHashCode assertions were further relaxed in JDK-8015086 to >>>>>> allow the CDS dump VM safepoint operation to call it. But >>>>>> FastHashCode can also call inflate so that was a mismatch in >>>>>> assertions just waiting to happen - which is how I hit this in a >>>>>> VM that always forces inflation :) >>>>>> >>>>>> I initially intended to extend the FastHashCode assertion to match >>>>>> inflate, but then a question arose about the safety of the CDS >>>>>> dumping code using FastHashcode which showed that the code is in >>>>>> fact safe. Notwithstanding what the state of the world may have >>>>>> been in the JDK 6 timeframe, there is now no potential >>>>>> interference possible between execution of these methods at an >>>>>> arbitrary safepoint and any concurrent GC use of the markWord. So >>>>>> the original !is_at_safepoint assertion can just be dropped from >>>>>> both methods. That in turn means we either drop the JavaThread >>>>>> assertion or else expand it to be "JavaThread or VMThread" - but >>>>>> that itself seems unnecessarily restrictive: why can't a GC thread >>>>>> use this code for example? So I chose to drop it. >>>>>> >>>>>> That leaves the "if I'm a JavaThread I'm not _thread_blocked" >>>>>> assertion. I don't see that this really guards against anything >>>>>> useful (a JavaThread should be _thread_in_vm when executing this >>>>>> code ... though there may be leaf calls where the thread is >>>>>> _thread_in_java). So again I just dropped the assertion. >>>>>> >>>>>> So a long winded justification for dropping a few assertions. :) >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>> >>>>> >>> > From coleen.phillimore at oracle.com Fri Aug 7 00:25:32 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 6 Aug 2020 20:25:32 -0400 Subject: RFR (s): 8250606: Remove unnecessary assertions in ObjectSynchronizer FastHashcode and inflate In-Reply-To: <1dfdfaeb-f0b2-f67b-2837-1660bc1ff733@oracle.com> References: <3e4c931a-593b-0336-97b9-3adc4de96062@oracle.com> <17eef440-3575-73f5-1413-0a25da6a13df@oracle.com> <11f913cc-8f88-0740-28d0-a7db7897d6a8@oracle.com> <63ee179e-7133-3edb-c952-1b5fd4f334bd@oracle.com> <7370d58f-1f9c-ccfd-9056-71d0035d15c5@oracle.com> <1dfdfaeb-f0b2-f67b-2837-1660bc1ff733@oracle.com> Message-ID: <2c71ef9c-0d10-8a3e-b0af-86418392dbc1@oracle.com> On 8/6/20 7:49 PM, David Holmes wrote: > On 7/08/2020 9:37 am, Ioi Lam wrote: >> >> >> On 8/6/20 2:46 PM, David Holmes wrote: >>> Hi Dan, >>> >>> On 7/08/2020 4:29 am, Daniel D. Daugherty wrote: >>>> On 8/5/20 6:31 PM, David Holmes wrote: >>>>> Hi Dan, >>>>> >>>>> Thanks for taking a look at this. >>>>> >>>>> On 6/08/2020 4:08 am, Daniel D. Daugherty wrote: >>>>>> On 8/5/20 3:58 AM, David Holmes wrote: >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8250606 >>>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8250606/webrev/ >>>>>> >>>>>> src/hotspot/share/runtime/synchronizer.cpp >>>>>> ???? L1010: ????? // Relaxing assertion for bug 6320749. >>>>>> ???? L1011: ????? assert(Universe::verify_in_progress() || >>>>>> ???? L1012: !SafepointSynchronize::is_at_safepoint(), >>>>>> ???? L1013: ???????????? "biases should not be seen by VM thread >>>>>> here"); >>>>>> ???????? This assertion is similar and tagged with the same bug >>>>>> ID (6320749) >>>>>> ???????? as the ones that you removed. Granted that the Biased >>>>>> Locking code >>>>>> ???????? is on the way out, but... >>>>>> >>>>>> ???????? I'm fine if you plan to leave this one for Biased >>>>>> Locking removal. >>>>> >>>>> Yeah I confess I had my biased-locking filter on when looking at >>>>> this part of the code. You've actually exposed a bit of a mess. :) >>>>> >>>>> It is unclear why the same !at_safepoint assertion was initially >>>>> added to the biased-locking revocation chunk of code given the >>>>> same assertion would then be executed after it. But I think it is >>>>> more to do with the line following the assertion (original code): >>>>> >>>>> assert(!SafepointSynchronize::is_at_safepoint(), "biases should >>>>> not be seen by VM thread here"); >>>>> BiasedLocking::revoke_and_rebias(hobj, false, JavaThread::current()); >>>>> >>>>> If we are at a safepoint in the VMThread then >>>>> JavaThread::current() will assert! So even after the assertion was >>>>> apparently relaxed with the verify_in_progress() check, we could >>>>> never have hit this code as the JavaThread::current() would still >>>>> assert. So I can (and should) remove the assertion here as well >>>>> (CDS dump could reach this in theory), but I also need to correct >>>>> this current code: >>>>> >>>>> -????? BiasedLocking::revoke(hobj, JavaThread::current()); >>>>> +????? BiasedLocking::revoke(hobj, self); >>>>> >>>>> as re-manifesting the current thread is pointless. That said the >>>>> above code (nor the original) also can't execute at a safepoint: >>>>> >>>>> void BiasedLocking::revoke(Handle obj, TRAPS) { >>>>> ? assert(!SafepointSynchronize::is_at_safepoint(), "must not be >>>>> called while at safepoint"); >>>> >>>> >>>> With the assert in JavaThread::current() and with the assert in >>>> BiasedLocking::revoke(), I don't see how we could ever get away >>>> with calling FastHashCode() on a biased object at a safepoint. >>>> It simply should have blown up (with non-release bits). >>>> >>>> I'm not yet sure what keeps us from seeing a biased object at >>>> a safepoint, but there has to be something there. We aren't that >>>> lucky to have simply missed this... >>> >>> Seems we are that lucky in a sense. It seems that the verification >>> process that we allow for is not the same as it once was and so we >>> don't hit the code that needs the FastHashCode. The only other >>> safepoint op that gets involved is CDS dumping and there we also get >>> lucky in that none of the objects that get archived are bias-locked. >>> I've been using Ioi's test from >>> >>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ >>> >>> >>> to force the object to be biased but I still can't hit that code >>> fragment so something else is still in play. >>> >>> David >>> ----- >>> >> >> Oh I feel so dumb. I forgot that we force UseBiasedLocking to false >> when CDS is enabled (arguments.cpp)! >> >> ?? if (DumpSharedSpaces) { >> ???? // Disable biased locking now as it interferes with the clean up of >> ???? // the archived Klasses and Java string objects (at dump time >> only). >> ???? UseBiasedLocking = false; > > Ah! :( > >> Anyway, it seems like UseBiasedLocking is not compatible with CDS in >> many ways. I have updated the patch to disable the above line. I also >> use a more reliably way get a biased-locked object. The test locks on >> an interned string, which we know for sure no one else would lock, or >> compute identity_hash of it. >> >> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03-with-debugging/ >> >> >> and when dumping with -XX:+UseBiasedLocking, we crash while doing GC >> inside a safepoint: >> >> #19 0x00007ffff5bb1a80 in oopDesc::set_mark (this=0xbcbcbcbcbcbcbcbc, >> m=...) >> #20 0x00007ffff5bb0d0b in BiasedLocking::restore_marks () > > Know bug: JDK-8251118 > >> #21 0x00007ffff5fa3515 in G1FullCollector::complete_collection (...) >> #22 0x00007ffff5f68b57 in G1CollectedHeap::do_full_collection (...) >> #23 0x00007ffff5f68bac in G1CollectedHeap::do_full_collection (...) >> #24 0x00007ffff5dc7aad in CollectedHeap::collect_as_vm_thread (...) >> #25 0x00007ffff60885f4 in HeapShared::run_full_gc_in_vm_thread () >> >> So it looks like the G1GC doesn't support collection inside the VM >> thread with biased-locked objects. >> >> Is biased locking going to be removed soon? I think we probably >> shouldn't spend too much time on it. > > There was some feedback on the disabling that indicated it may still > be useful so that decision to proceed with the removal has yet to occur. > >> Maybe instead of removing the asserts, just enclose them inside "if >> UseBiasedLocking"? > > I think I'm going to fix the BL block of code separately. There's no > reason I can see why a biased-locked object should be a concern in > FastHashCode, we just need to change the revocation code to allow for > the safepoint (which is exactly what we do elsewhere). I'm not sure why you want to do this, since I can't see why anything would call this with BiasedLocking on.? But for what it's worth, I had a prototype of something else where I changed that code to be: ? if (SafepointSynchronize::is_at_safepoint()) { ??? BiasedLocking::revoke_at_safepoint(); ? } else { ??? BiasedLocking::revoke(); ? } and it worked fine. Coleen > > Thanks, > David > >> Thanks >> - Ioi >> >> >>>> Dan >>>> >>>>> >>>>> so even larger changes will be needed to make this code work at a >>>>> safepoint, and that is going well beyond the intended scope of >>>>> this particular bug. :( I'll need to get back to you. >>>>> >>>>> I also need to check our GC verify tests to see how we manage to >>>>> apparently never encounter a biased-locked object, otherwise we >>>>> would have seen this inconsistency in the code. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>> >>>>>> Thumbs up.? Thanks for digging up the history behind these very >>>>>> old asserts. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Dan >>>>>> >>>>>> >>>>>>> >>>>>>> tl;dr I removed some assertions that were deemed unnecessary >>>>>>> after interfering with other work. :) >>>>>>> >>>>>>> ObjectSynchronizer::inflate had the following assertion added >>>>>>> back in JDK 6 under JDK-5030359: >>>>>>> >>>>>>> ObjectMonitor * ObjectSynchronizer::inflate (oop object) { >>>>>>> ? // Inflate mutates the heap ... >>>>>>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>>>>>> >>>>>>> That same change added FastHashcode which had the following >>>>>>> assertions: >>>>>>> >>>>>>> intptr_t ObjectSynchronizer::FastHashCode (Thread * Self, oop >>>>>>> obj) { >>>>>>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>>>>>> ? assert (Self->is_Java_thread(), "invariant") ; >>>>>>> ? assert (((JavaThread *)Self)->thread_state() != >>>>>>> _thread_blocked, "invariant") ; >>>>>>> >>>>>>> These methods had assertions that they were not executed at a >>>>>>> safepoint in case the modification of the markWord could cause >>>>>>> interference with GC. >>>>>>> >>>>>>> JDK-6320749 then relaxed the assertions to allow for GC >>>>>>> verification - which does happen at a safepoint and so obviously >>>>>>> not in a JavaThread. >>>>>>> >>>>>>> The FastHashCode assertions were further relaxed in JDK-8015086 >>>>>>> to allow the CDS dump VM safepoint operation to call it. But >>>>>>> FastHashCode can also call inflate so that was a mismatch in >>>>>>> assertions just waiting to happen - which is how I hit this in a >>>>>>> VM that always forces inflation :) >>>>>>> >>>>>>> I initially intended to extend the FastHashCode assertion to >>>>>>> match inflate, but then a question arose about the safety of the >>>>>>> CDS dumping code using FastHashcode which showed that the code >>>>>>> is in fact safe. Notwithstanding what the state of the world may >>>>>>> have been in the JDK 6 timeframe, there is now no potential >>>>>>> interference possible between execution of these methods at an >>>>>>> arbitrary safepoint and any concurrent GC use of the markWord. >>>>>>> So the original !is_at_safepoint assertion can just be dropped >>>>>>> from both methods. That in turn means we either drop the >>>>>>> JavaThread assertion or else expand it to be "JavaThread or >>>>>>> VMThread" - but that itself seems unnecessarily restrictive: why >>>>>>> can't a GC thread use this code for example? So I chose to drop it. >>>>>>> >>>>>>> That leaves the "if I'm a JavaThread I'm not _thread_blocked" >>>>>>> assertion. I don't see that this really guards against anything >>>>>>> useful (a JavaThread should be _thread_in_vm when executing this >>>>>>> code ... though there may be leaf calls where the thread is >>>>>>> _thread_in_java). So again I just dropped the assertion. >>>>>>> >>>>>>> So a long winded justification for dropping a few assertions. :) >>>>>>> >>>>>>> Thanks, >>>>>>> David >>>>>>> >>>>>>> >>>>>> >>>> >> From david.holmes at oracle.com Fri Aug 7 00:49:57 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Aug 2020 10:49:57 +1000 Subject: RFR (s): 8250606: Remove unnecessary assertions in ObjectSynchronizer FastHashcode and inflate In-Reply-To: <2c71ef9c-0d10-8a3e-b0af-86418392dbc1@oracle.com> References: <3e4c931a-593b-0336-97b9-3adc4de96062@oracle.com> <17eef440-3575-73f5-1413-0a25da6a13df@oracle.com> <11f913cc-8f88-0740-28d0-a7db7897d6a8@oracle.com> <63ee179e-7133-3edb-c952-1b5fd4f334bd@oracle.com> <7370d58f-1f9c-ccfd-9056-71d0035d15c5@oracle.com> <1dfdfaeb-f0b2-f67b-2837-1660bc1ff733@oracle.com> <2c71ef9c-0d10-8a3e-b0af-86418392dbc1@oracle.com> Message-ID: <1bafeeec-314f-9948-ee7d-1fcb6c20efc0@oracle.com> On 7/08/2020 10:25 am, Coleen Phillimore wrote: > On 8/6/20 7:49 PM, David Holmes wrote: >> On 7/08/2020 9:37 am, Ioi Lam wrote: >>> >>> >>> On 8/6/20 2:46 PM, David Holmes wrote: >>>> Hi Dan, >>>> >>>> On 7/08/2020 4:29 am, Daniel D. Daugherty wrote: >>>>> On 8/5/20 6:31 PM, David Holmes wrote: >>>>>> Hi Dan, >>>>>> >>>>>> Thanks for taking a look at this. >>>>>> >>>>>> On 6/08/2020 4:08 am, Daniel D. Daugherty wrote: >>>>>>> On 8/5/20 3:58 AM, David Holmes wrote: >>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8250606 >>>>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8250606/webrev/ >>>>>>> >>>>>>> src/hotspot/share/runtime/synchronizer.cpp >>>>>>> ???? L1010: ????? // Relaxing assertion for bug 6320749. >>>>>>> ???? L1011: ????? assert(Universe::verify_in_progress() || >>>>>>> ???? L1012: !SafepointSynchronize::is_at_safepoint(), >>>>>>> ???? L1013: ???????????? "biases should not be seen by VM thread >>>>>>> here"); >>>>>>> ???????? This assertion is similar and tagged with the same bug >>>>>>> ID (6320749) >>>>>>> ???????? as the ones that you removed. Granted that the Biased >>>>>>> Locking code >>>>>>> ???????? is on the way out, but... >>>>>>> >>>>>>> ???????? I'm fine if you plan to leave this one for Biased >>>>>>> Locking removal. >>>>>> >>>>>> Yeah I confess I had my biased-locking filter on when looking at >>>>>> this part of the code. You've actually exposed a bit of a mess. :) >>>>>> >>>>>> It is unclear why the same !at_safepoint assertion was initially >>>>>> added to the biased-locking revocation chunk of code given the >>>>>> same assertion would then be executed after it. But I think it is >>>>>> more to do with the line following the assertion (original code): >>>>>> >>>>>> assert(!SafepointSynchronize::is_at_safepoint(), "biases should >>>>>> not be seen by VM thread here"); >>>>>> BiasedLocking::revoke_and_rebias(hobj, false, JavaThread::current()); >>>>>> >>>>>> If we are at a safepoint in the VMThread then >>>>>> JavaThread::current() will assert! So even after the assertion was >>>>>> apparently relaxed with the verify_in_progress() check, we could >>>>>> never have hit this code as the JavaThread::current() would still >>>>>> assert. So I can (and should) remove the assertion here as well >>>>>> (CDS dump could reach this in theory), but I also need to correct >>>>>> this current code: >>>>>> >>>>>> -????? BiasedLocking::revoke(hobj, JavaThread::current()); >>>>>> +????? BiasedLocking::revoke(hobj, self); >>>>>> >>>>>> as re-manifesting the current thread is pointless. That said the >>>>>> above code (nor the original) also can't execute at a safepoint: >>>>>> >>>>>> void BiasedLocking::revoke(Handle obj, TRAPS) { >>>>>> ? assert(!SafepointSynchronize::is_at_safepoint(), "must not be >>>>>> called while at safepoint"); >>>>> >>>>> >>>>> With the assert in JavaThread::current() and with the assert in >>>>> BiasedLocking::revoke(), I don't see how we could ever get away >>>>> with calling FastHashCode() on a biased object at a safepoint. >>>>> It simply should have blown up (with non-release bits). >>>>> >>>>> I'm not yet sure what keeps us from seeing a biased object at >>>>> a safepoint, but there has to be something there. We aren't that >>>>> lucky to have simply missed this... >>>> >>>> Seems we are that lucky in a sense. It seems that the verification >>>> process that we allow for is not the same as it once was and so we >>>> don't hit the code that needs the FastHashCode. The only other >>>> safepoint op that gets involved is CDS dumping and there we also get >>>> lucky in that none of the objects that get archived are bias-locked. >>>> I've been using Ioi's test from >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ >>>> >>>> >>>> to force the object to be biased but I still can't hit that code >>>> fragment so something else is still in play. >>>> >>>> David >>>> ----- >>>> >>> >>> Oh I feel so dumb. I forgot that we force UseBiasedLocking to false >>> when CDS is enabled (arguments.cpp)! >>> >>> ?? if (DumpSharedSpaces) { >>> ???? // Disable biased locking now as it interferes with the clean up of >>> ???? // the archived Klasses and Java string objects (at dump time >>> only). >>> ???? UseBiasedLocking = false; >> >> Ah! :( >> >>> Anyway, it seems like UseBiasedLocking is not compatible with CDS in >>> many ways. I have updated the patch to disable the above line. I also >>> use a more reliably way get a biased-locked object. The test locks on >>> an interned string, which we know for sure no one else would lock, or >>> compute identity_hash of it. >>> >>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03-with-debugging/ >>> >>> >>> and when dumping with -XX:+UseBiasedLocking, we crash while doing GC >>> inside a safepoint: >>> >>> #19 0x00007ffff5bb1a80 in oopDesc::set_mark (this=0xbcbcbcbcbcbcbcbc, >>> m=...) >>> #20 0x00007ffff5bb0d0b in BiasedLocking::restore_marks () >> >> Know bug: JDK-8251118 >> >>> #21 0x00007ffff5fa3515 in G1FullCollector::complete_collection (...) >>> #22 0x00007ffff5f68b57 in G1CollectedHeap::do_full_collection (...) >>> #23 0x00007ffff5f68bac in G1CollectedHeap::do_full_collection (...) >>> #24 0x00007ffff5dc7aad in CollectedHeap::collect_as_vm_thread (...) >>> #25 0x00007ffff60885f4 in HeapShared::run_full_gc_in_vm_thread () >>> >>> So it looks like the G1GC doesn't support collection inside the VM >>> thread with biased-locked objects. >>> >>> Is biased locking going to be removed soon? I think we probably >>> shouldn't spend too much time on it. >> >> There was some feedback on the disabling that indicated it may still >> be useful so that decision to proceed with the removal has yet to occur. >> >>> Maybe instead of removing the asserts, just enclose them inside "if >>> UseBiasedLocking"? >> >> I think I'm going to fix the BL block of code separately. There's no >> reason I can see why a biased-locked object should be a concern in >> FastHashCode, we just need to change the revocation code to allow for >> the safepoint (which is exactly what we do elsewhere). > > I'm not sure why you want to do this, since I can't see why anything > would call this with BiasedLocking on.? But for what it's worth, I had a > prototype of something else where I changed that code to be: > > ? if (SafepointSynchronize::is_at_safepoint()) { > ??? BiasedLocking::revoke_at_safepoint(); > ? } else { > ??? BiasedLocking::revoke(); > ? } > > and it worked fine. Yes that is exactly what I had in mind. If nothing else it avoids having to think about why the BL path can't happen in a safepoint. Also it may be that once this is fixed the CDS code may not have to disable BL. Thanks, David > Coleen > >> >> Thanks, >> David >> >>> Thanks >>> - Ioi >>> >>> >>>>> Dan >>>>> >>>>>> >>>>>> so even larger changes will be needed to make this code work at a >>>>>> safepoint, and that is going well beyond the intended scope of >>>>>> this particular bug. :( I'll need to get back to you. >>>>>> >>>>>> I also need to check our GC verify tests to see how we manage to >>>>>> apparently never encounter a biased-locked object, otherwise we >>>>>> would have seen this inconsistency in the code. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>> >>>>>>> Thumbs up.? Thanks for digging up the history behind these very >>>>>>> old asserts. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> >>>>>>> Dan >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> tl;dr I removed some assertions that were deemed unnecessary >>>>>>>> after interfering with other work. :) >>>>>>>> >>>>>>>> ObjectSynchronizer::inflate had the following assertion added >>>>>>>> back in JDK 6 under JDK-5030359: >>>>>>>> >>>>>>>> ObjectMonitor * ObjectSynchronizer::inflate (oop object) { >>>>>>>> ? // Inflate mutates the heap ... >>>>>>>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>>>>>>> >>>>>>>> That same change added FastHashcode which had the following >>>>>>>> assertions: >>>>>>>> >>>>>>>> intptr_t ObjectSynchronizer::FastHashCode (Thread * Self, oop >>>>>>>> obj) { >>>>>>>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>>>>>>> ? assert (Self->is_Java_thread(), "invariant") ; >>>>>>>> ? assert (((JavaThread *)Self)->thread_state() != >>>>>>>> _thread_blocked, "invariant") ; >>>>>>>> >>>>>>>> These methods had assertions that they were not executed at a >>>>>>>> safepoint in case the modification of the markWord could cause >>>>>>>> interference with GC. >>>>>>>> >>>>>>>> JDK-6320749 then relaxed the assertions to allow for GC >>>>>>>> verification - which does happen at a safepoint and so obviously >>>>>>>> not in a JavaThread. >>>>>>>> >>>>>>>> The FastHashCode assertions were further relaxed in JDK-8015086 >>>>>>>> to allow the CDS dump VM safepoint operation to call it. But >>>>>>>> FastHashCode can also call inflate so that was a mismatch in >>>>>>>> assertions just waiting to happen - which is how I hit this in a >>>>>>>> VM that always forces inflation :) >>>>>>>> >>>>>>>> I initially intended to extend the FastHashCode assertion to >>>>>>>> match inflate, but then a question arose about the safety of the >>>>>>>> CDS dumping code using FastHashcode which showed that the code >>>>>>>> is in fact safe. Notwithstanding what the state of the world may >>>>>>>> have been in the JDK 6 timeframe, there is now no potential >>>>>>>> interference possible between execution of these methods at an >>>>>>>> arbitrary safepoint and any concurrent GC use of the markWord. >>>>>>>> So the original !is_at_safepoint assertion can just be dropped >>>>>>>> from both methods. That in turn means we either drop the >>>>>>>> JavaThread assertion or else expand it to be "JavaThread or >>>>>>>> VMThread" - but that itself seems unnecessarily restrictive: why >>>>>>>> can't a GC thread use this code for example? So I chose to drop it. >>>>>>>> >>>>>>>> That leaves the "if I'm a JavaThread I'm not _thread_blocked" >>>>>>>> assertion. I don't see that this really guards against anything >>>>>>>> useful (a JavaThread should be _thread_in_vm when executing this >>>>>>>> code ... though there may be leaf calls where the thread is >>>>>>>> _thread_in_java). So again I just dropped the assertion. >>>>>>>> >>>>>>>> So a long winded justification for dropping a few assertions. :) >>>>>>>> >>>>>>>> Thanks, >>>>>>>> David >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> > From coleen.phillimore at oracle.com Fri Aug 7 00:50:42 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 6 Aug 2020 20:50:42 -0400 Subject: RFR(S) 8249276 CDS archived objects must have "neutral" markwords In-Reply-To: <7a81547c-8221-da80-2d1a-edb42f4078fc@oracle.com> References: <5d5b4b17-a40f-b0f4-fa50-fdd783def9d4@oracle.com> <37245599-dfb8-0481-6c74-89cffdc5b457@oracle.com> <7a81547c-8221-da80-2d1a-edb42f4078fc@oracle.com> Message-ID: <53850d65-5b9b-9077-b8ad-91d840b66805@oracle.com> This looks good. Coleen On 8/5/20 6:26 PM, Ioi Lam wrote: > Hi David & Coleen, > > Thanks for the review. I have created a new version: > > http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ > > > > I created a new test case (LockDuringDump), and incorporated David's > fix for JDK-8250606 [1] during my testing: > > + Without my patch, LockDuringDumpApp would crash at runtime, because > ? we have an invalid markword for the Helper.class object. > > + Without the JDK-8250606 patch, we would assert during CDS dump time. > > I also removed the assert in HeapShared::oop_hash() since it's > irrelevant after JDK-8250606. > > I intend to push my patch after JDK-8250606 is integrated. > > > ------- > > More comments below > > On 8/3/20 6:13 PM, David Holmes wrote: >> Hi Ioi, >> >> On 27/07/2020 3:20 pm, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8249276 >>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v01/ >>> >>> >>> Please review this change (initial patch provided by David Holmes; I >>> rearranged it a little and added more comments). >>> >>> We reinitialize the markWord of all archived objects to remove any >>> side effect of GC/locking/etc during "java -Xshare:dump". >> >> Changes look good to me (naturally :) ). A couple of nits: >> >> ?? int hash_archived = archived_oop->identity_hash(); >> ?? assert(hash_original == hash_archived, "Different hash codes: >> original %x, archived %x", hash_original, hash_archived); >> >> the has_archived setting should be in ifdef DEBUG. >> > > Fixed > > >> 42 Object.class.wait(); >> ? 43???????????????? } >> ? 44???????????????? System.out.println("Huh?? notified??"); >> >> That could be a spurious wakeup. You could just use Thread.sleep with >> a suitably long sleep time. that said I'm not sure how the test is >> intended to operate. If premain returns after starting the thread >> there is no guarantee the thread will start executing and lock the >> object before the dump actually commences. Even the sighting of >> "Let's wait ....." is not sufficient to guarantee that unless you >> move it inside the sync block. >> > > I have fixed the test case so that the "Let's hold the lock" message > is printed after the lock is held, and the lock is never released > afterwards. > > I also added synchronization to make sure the premain() method doesn't > return until the child thread has held the lock. The log file shows > that theses messages are printed before CDS tries to load classes from > the classlist: > > [0.005s][info][cds] Allocated shared space: 3221225472 bytes at > 0x0000000800000000 > inside LockDuringDumpAgent: > jdk.internal.loader.ClassLoaders$AppClassLoader at 707f7052 > Helper class is initialized > Class = class LockDuringDumpApp$Helper > Let's hold the lock on Helper.class forever ..... > Thread has started > [0.145s][info][cds] Loading classes to share ... > >>> (David mentioned in the bug comments about assertions inside the >>> identity_hash() call, but maybe this should be fixed in a different >>> bug?) >> >> Yes those assertions are being removed under: >> >> https://bugs.openjdk.java.net/browse/JDK-8250606 >> > > Thanks > - Ioi > > ------- > > [1] > https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/040985.html > > From david.holmes at oracle.com Fri Aug 7 00:51:23 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Aug 2020 10:51:23 +1000 Subject: RFR (s): 8250606: Remove unnecessary assertions in ObjectSynchronizer FastHashcode and inflate In-Reply-To: References: <3e4c931a-593b-0336-97b9-3adc4de96062@oracle.com> <17eef440-3575-73f5-1413-0a25da6a13df@oracle.com> <11f913cc-8f88-0740-28d0-a7db7897d6a8@oracle.com> Message-ID: On 7/08/2020 7:56 am, Coleen Phillimore wrote: > > Deleting these asserts looks good to me.? More below. Thanks for looking at this Coleen, and the info about the verification changes. I will push what is in the webrev and file a follow up bug for the BL issue in FastHashCode. Thanks, David ----- > On 8/6/20 2:29 PM, Daniel D. Daugherty wrote: >> On 8/5/20 6:31 PM, David Holmes wrote: >>> Hi Dan, >>> >>> Thanks for taking a look at this. >>> >>> On 6/08/2020 4:08 am, Daniel D. Daugherty wrote: >>>> On 8/5/20 3:58 AM, David Holmes wrote: >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8250606 >>>>> webrev: http://cr.openjdk.java.net/~dholmes/8250606/webrev/ >>>> >>>> src/hotspot/share/runtime/synchronizer.cpp >>>> ???? L1010: ????? // Relaxing assertion for bug 6320749. >>>> ???? L1011: ????? assert(Universe::verify_in_progress() || >>>> ???? L1012: !SafepointSynchronize::is_at_safepoint(), >>>> ???? L1013: ???????????? "biases should not be seen by VM thread >>>> here"); >>>> ???????? This assertion is similar and tagged with the same bug ID >>>> (6320749) >>>> ???????? as the ones that you removed. Granted that the Biased >>>> Locking code >>>> ???????? is on the way out, but... >>>> >>>> ???????? I'm fine if you plan to leave this one for Biased Locking >>>> removal. >>> >>> Yeah I confess I had my biased-locking filter on when looking at this >>> part of the code. You've actually exposed a bit of a mess. :) >>> >>> It is unclear why the same !at_safepoint assertion was initially >>> added to the biased-locking revocation chunk of code given the same >>> assertion would then be executed after it. But I think it is more to >>> do with the line following the assertion (original code): >>> >>> assert(!SafepointSynchronize::is_at_safepoint(), "biases should not >>> be seen by VM thread here"); >>> BiasedLocking::revoke_and_rebias(hobj, false, JavaThread::current()); >>> >>> If we are at a safepoint in the VMThread then JavaThread::current() >>> will assert! So even after the assertion was apparently relaxed with >>> the verify_in_progress() check, we could never have hit this code as >>> the JavaThread::current() would still assert. So I can (and should) >>> remove the assertion here as well (CDS dump could reach this in >>> theory), but I also need to correct this current code: >>> >>> -????? BiasedLocking::revoke(hobj, JavaThread::current()); >>> +????? BiasedLocking::revoke(hobj, self); >>> >>> as re-manifesting the current thread is pointless. That said the >>> above code (nor the original) also can't execute at a safepoint: >>> >>> void BiasedLocking::revoke(Handle obj, TRAPS) { >>> ? assert(!SafepointSynchronize::is_at_safepoint(), "must not be >>> called while at safepoint"); >> >> >> With the assert in JavaThread::current() and with the assert in >> BiasedLocking::revoke(), I don't see how we could ever get away >> with calling FastHashCode() on a biased object at a safepoint. >> It simply should have blown up (with non-release bits). >> >> I'm not yet sure what keeps us from seeing a biased object at >> a safepoint, but there has to be something there. We aren't that >> lucky to have simply missed this... > > The original reason for relaxing the asserts for verification was that > we were verifying the SystemDictionary during GC, which used > Object.hashCode() for Klasses because they were in PermGen.? The Klass's > prototype_header was set to never used biased locking. > > Currently, the only use for calling FastHashCode() in a safepoint is > because of dumping objects for CDS, and in this case UseBiasedLocking is > switched off when DumpSharedSpaces. > > So neither path would go through the first 'if' statement in the > FastHashCode code. > > I think you should remove the assert here.? It's obsolete and one less > mystery to solve. > > 1010?????? // Relaxing assertion for bug 6320749. > 1011?????? assert(Universe::verify_in_progress() || > 1012????????????? !SafepointSynchronize::is_at_safepoint(), > 1013????????????? "biases should not be seen by VM thread here"); > > since revoke() asserts !at_safepoint() and change JavaThread::current() > to "self". > > Thanks, > Coleen > >> >> Dan >> >>> >>> so even larger changes will be needed to make this code work at a >>> safepoint, and that is going well beyond the intended scope of this >>> particular bug. :( I'll need to get back to you. >>> >>> I also need to check our GC verify tests to see how we manage to >>> apparently never encounter a biased-locked object, otherwise we would >>> have seen this inconsistency in the code. >>> >>> Thanks, >>> David >>> ----- >>> >>> >>>> Thumbs up.? Thanks for digging up the history behind these very old >>>> asserts. >>> >>> Thanks, >>> David >>> >>>> Dan >>>> >>>> >>>>> >>>>> tl;dr I removed some assertions that were deemed unnecessary after >>>>> interfering with other work. :) >>>>> >>>>> ObjectSynchronizer::inflate had the following assertion added back >>>>> in JDK 6 under JDK-5030359: >>>>> >>>>> ObjectMonitor * ObjectSynchronizer::inflate (oop object) { >>>>> ? // Inflate mutates the heap ... >>>>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>>>> >>>>> That same change added FastHashcode which had the following >>>>> assertions: >>>>> >>>>> intptr_t ObjectSynchronizer::FastHashCode (Thread * Self, oop obj) { >>>>> ? assert (!SafepointSynchronize::is_at_safepoint(), "invariant") ; >>>>> ? assert (Self->is_Java_thread(), "invariant") ; >>>>> ? assert (((JavaThread *)Self)->thread_state() != _thread_blocked, >>>>> "invariant") ; >>>>> >>>>> These methods had assertions that they were not executed at a >>>>> safepoint in case the modification of the markWord could cause >>>>> interference with GC. >>>>> >>>>> JDK-6320749 then relaxed the assertions to allow for GC >>>>> verification - which does happen at a safepoint and so obviously >>>>> not in a JavaThread. >>>>> >>>>> The FastHashCode assertions were further relaxed in JDK-8015086 to >>>>> allow the CDS dump VM safepoint operation to call it. But >>>>> FastHashCode can also call inflate so that was a mismatch in >>>>> assertions just waiting to happen - which is how I hit this in a VM >>>>> that always forces inflation :) >>>>> >>>>> I initially intended to extend the FastHashCode assertion to match >>>>> inflate, but then a question arose about the safety of the CDS >>>>> dumping code using FastHashcode which showed that the code is in >>>>> fact safe. Notwithstanding what the state of the world may have >>>>> been in the JDK 6 timeframe, there is now no potential interference >>>>> possible between execution of these methods at an arbitrary >>>>> safepoint and any concurrent GC use of the markWord. So the >>>>> original !is_at_safepoint assertion can just be dropped from both >>>>> methods. That in turn means we either drop the JavaThread assertion >>>>> or else expand it to be "JavaThread or VMThread" - but that itself >>>>> seems unnecessarily restrictive: why can't a GC thread use this >>>>> code for example? So I chose to drop it. >>>>> >>>>> That leaves the "if I'm a JavaThread I'm not _thread_blocked" >>>>> assertion. I don't see that this really guards against anything >>>>> useful (a JavaThread should be _thread_in_vm when executing this >>>>> code ... though there may be leaf calls where the thread is >>>>> _thread_in_java). So again I just dropped the assertion. >>>>> >>>>> So a long winded justification for dropping a few assertions. :) >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>> >>>> >> > From david.holmes at oracle.com Fri Aug 7 00:55:40 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Aug 2020 10:55:40 +1000 Subject: RFR(S) 8249276 CDS archived objects must have "neutral" markwords In-Reply-To: <7a81547c-8221-da80-2d1a-edb42f4078fc@oracle.com> References: <5d5b4b17-a40f-b0f4-fa50-fdd783def9d4@oracle.com> <37245599-dfb8-0481-6c74-89cffdc5b457@oracle.com> <7a81547c-8221-da80-2d1a-edb42f4078fc@oracle.com> Message-ID: <11f16292-909d-6d61-d353-cfa9d7054506@oracle.com> Hi Ioi, On 6/08/2020 8:26 am, Ioi Lam wrote: > Hi David & Coleen, > > Thanks for the review. I have created a new version: > > http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ > Functional changes are good. For the test, as per our offline chat, the Helper class is not being archived. This line: 66 TestCommon.testDump(appJar, TestCommon.list(LockDuringDumpApp.class.getName()), should be 66 TestCommon.testDump(appJar, TestCommon.list(appClasses), Thanks, David > > I created a new test case (LockDuringDump), and incorporated David's fix > for JDK-8250606 [1] during my testing: > > + Without my patch, LockDuringDumpApp would crash at runtime, because > ? we have an invalid markword for the Helper.class object. > > + Without the JDK-8250606 patch, we would assert during CDS dump time. > > I also removed the assert in HeapShared::oop_hash() since it's > irrelevant after JDK-8250606. > > I intend to push my patch after JDK-8250606 is integrated. > > > ------- > > More comments below > > On 8/3/20 6:13 PM, David Holmes wrote: >> Hi Ioi, >> >> On 27/07/2020 3:20 pm, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8249276 >>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v01/ >>> >>> >>> Please review this change (initial patch provided by David Holmes; I >>> rearranged it a little and added more comments). >>> >>> We reinitialize the markWord of all archived objects to remove any >>> side effect of GC/locking/etc during "java -Xshare:dump". >> >> Changes look good to me (naturally :) ). A couple of nits: >> >> ?? int hash_archived = archived_oop->identity_hash(); >> ?? assert(hash_original == hash_archived, "Different hash codes: >> original %x, archived %x", hash_original, hash_archived); >> >> the has_archived setting should be in ifdef DEBUG. >> > > Fixed > > >> 42???????????????????? Object.class.wait(); >> ? 43???????????????? } >> ? 44???????????????? System.out.println("Huh?? notified??"); >> >> That could be a spurious wakeup. You could just use Thread.sleep with >> a suitably long sleep time. that said I'm not sure how the test is >> intended to operate. If premain returns after starting the thread >> there is no guarantee the thread will start executing and lock the >> object before the dump actually commences. Even the sighting of "Let's >> wait ....." is not sufficient to guarantee that unless you move it >> inside the sync block. >> > > I have fixed the test case so that the "Let's hold the lock" message is > printed after the lock is held, and the lock is never released afterwards. > > I also added synchronization to make sure the premain() method doesn't > return until the child thread has held the lock. The log file shows that > theses messages are printed before CDS tries to load classes from the > classlist: > > [0.005s][info][cds] Allocated shared space: 3221225472 bytes at > 0x0000000800000000 > inside LockDuringDumpAgent: > jdk.internal.loader.ClassLoaders$AppClassLoader at 707f7052 > Helper class is initialized > Class = class LockDuringDumpApp$Helper > Let's hold the lock on Helper.class forever ..... > Thread has started > [0.145s][info][cds] Loading classes to share ... > >>> (David mentioned in the bug comments about assertions inside the >>> identity_hash() call, but maybe this should be fixed in a different >>> bug?) >> >> Yes those assertions are being removed under: >> >> https://bugs.openjdk.java.net/browse/JDK-8250606 >> > > Thanks > - Ioi > > ------- > > [1] > https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/040985.html > > > From david.holmes at oracle.com Fri Aug 7 00:58:41 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Aug 2020 10:58:41 +1000 Subject: RFR (xs) 8251209 [TESTBUG] CDS jvmti tests should use "@modules java.instrument" In-Reply-To: <2b9000ef-9a09-caf4-8610-bb354afb722b@oracle.com> References: <2b9000ef-9a09-caf4-8610-bb354afb722b@oracle.com> Message-ID: <138c579d-2ab9-5e35-9150-dfdd74e899eb@oracle.com> Correction ... On 7/08/2020 7:52 am, David Holmes wrote: > Hi Ioi, > > On 7/08/2020 4:25 am, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8251209 >> http://cr.openjdk.java.net/~iklam/jdk16/8251209-cds-jvmti-tests-modules-tag.v01/ >> >> >> Summary -- changed the tests from (mis)using >> >> ??* @requires vm.flavor != "minimal" >> >> to >> >> ??* @modules java.instrument >> >> ... to be consistent with other jvmti tests. > > That seems like an invalid precondition to me. It would have been > somewhat valid in the Compact Profiles world when we did not provide > "java.instrument" in the profiles which supported MinimalVM, but you can > define a minimal VM in a build that still has all modules available. I > don't think building the minimal VM makes any changes to the supported > modules. > > Also AIUI the @modules statement simply adds the necessary command-line > args to use the java.instrument module (if present), it doesn't ensure > that the listed module has to be present. It does in fact ensure that: "Otherwise, a test will not be run if the system being tested does not contain all of the specified modules." http://openjdk.java.net/jtreg/tag-spec.html But as I said the module could be present in a JRE but you are still using the MinimalVM. David ----- > David > >> Thanks >> - Ioi From patricio.chilano.mateo at oracle.com Fri Aug 7 01:07:46 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Thu, 6 Aug 2020 22:07:46 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <0d89c58a-0b70-4458-be91-e50f7a81b37f@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> <0d89c58a-0b70-4458-be91-e50f7a81b37f@oracle.com> Message-ID: Hi Dan, On 8/6/20 7:06 PM, Daniel D. Daugherty wrote: >> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev > > This time on the official 'v3' rather than the 'prelim-v3'. > > src/hotspot/cpu/aarch64/aarch64.ad > ??? No comments. > > src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp > ??? No comments. > > src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp > ??? No comments. > > src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp > ??? L447: void MacroAssembler::biased_locking_enter(Register lock_reg, > ??? L448: ???????????????????????????????????????? Register obj_reg, > ??? : > ??? L454: BiasedLockingCounters* counters) { > ??????? nit - The change from 'int' to 'void' changes the indent so > ?????? ? ? ?? L448-454 need one more space. > > src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp > ??? L114: ? void biased_locking_enter(Register lock_reg, Register > obj_reg, > ??? L115: ?????????????????????????? Register swap_reg, Register tmp_reg, > ??? : > ??? L118: ?????????????????????????? BiasedLockingCounters* counters = > NULL); > ??????? nit - The change from 'int' to 'void' changes the indent so > ????????????? L115-118 need one more space. > > src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp > ??? No comments. > > src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp > ??? No comments. > > src/hotspot/cpu/arm/interp_masm_arm.cpp > ??? No comments. > > src/hotspot/cpu/arm/macroAssembler_arm.cpp > ??? L1325: void MacroAssembler::biased_locking_enter(Register obj_reg, > Register swap_reg, Register tmp_reg, > ??? L1326: ???????????????????????????????????????? bool > swap_reg_contains_mark, > ??? : > ??? L1329: BiasedLockingCounters* counters) { > ??????? nit - The change from 'int' to 'void' changes the indent so > ????????????? L1326-1329 need one more space. > > src/hotspot/cpu/arm/macroAssembler_arm.hpp > ??? L381: ? void biased_locking_enter(Register obj_reg, Register > swap_reg, Register tmp_reg, > ??? L382: ?????????????????????????? bool swap_reg_contains_mark, > ??? : > ??? L385: ?????????????????????????? BiasedLockingCounters* counters = > NULL); > ??????? nit - The change from 'int' to 'void' changes the indent so > ????????????? L382-385 need one more space. > > src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp > src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp > src/hotspot/cpu/ppc/macroAssembler_ppc.cpp > ??? No comments on the PPC files. > > src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp > src/hotspot/cpu/s390/interp_masm_s390.cpp > src/hotspot/cpu/s390/macroAssembler_s390.cpp > ??? No comments on the S390 files. > > src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp > ??? No comments. > > src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp > ??? No comments. > > src/hotspot/cpu/x86/interp_masm_x86.cpp > ??? No comments. > > src/hotspot/cpu/x86/macroAssembler_x86.cpp > ??? L1084: void MacroAssembler::biased_locking_enter(Register lock_reg, > ??? L1085: ???????????????????????????????????????? Register obj_reg, > ??? : > ??? L1092: BiasedLockingCounters* counters) { > ??????? nit - The change from 'int' to 'void' changes the indent so > ????????????? L1085-1092 need one more space. > > src/hotspot/cpu/x86/macroAssembler_x86.hpp > ??? L664: ? void biased_locking_enter(Register lock_reg, Register > obj_reg, > ??? L665: ?????????????????????????? Register swap_reg, Register tmp_reg, > ??? : > ??? L668: ?????????????????????????? BiasedLockingCounters* counters = > NULL); > ??????? nit - The change from 'int' to 'void' changes the indent so > ????????????? L665-668 need one more space. > > src/hotspot/share/classfile/systemDictionary.cpp > ??? No comments. > > src/hotspot/share/jfr/metadata/metadata.xml > ??? No comments. > > src/hotspot/share/logging/logTag.hpp > ??? No comments. > > src/hotspot/share/oops/klass.hpp > ??? No comments. > > src/hotspot/share/runtime/arguments.cpp > ??? No comments. > > src/hotspot/share/runtime/globals.hpp > ??? No comments. > > src/hotspot/share/runtime/synchronizer.cpp > ??? No comments. > > src/hotspot/share/runtime/synchronizer.hpp > ??? No comments. > > src/hotspot/share/utilities/accessFlags.hpp > ??? No comments. > > src/jdk.jfr/share/conf/jfr/default.jfc > ??? No comments. > > src/jdk.jfr/share/conf/jfr/profile.jfc > ??? No comments. > > test/lib/jdk/test/lib/jfr/EventNames.java > ??? No comments. > > test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java > ??? No comments. > > test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java > ??? No comments. > > Thumbs up. I only have nits above so no need for another webrev if > you choose to fix them. Great, thanks for reviewing this! Patricio > Dan > > > > On 8/6/20 2:48 PM, Patricio Chilano wrote: >> Hi Dan, >> >> On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: >>> I'm peeking ahead to the next webrev... :-) >>> >>> > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ >> : ) >> >>> General comments: >>> ? - check files for copyright year updates. >> Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, >> c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. >> >>> src/hotspot/share/runtime/synchronizer.hpp >>> ??? No comments. >>> >>> src/hotspot/share/runtime/synchronizer.cpp >>> ??? L507: ? const markWord mark = obj->mark(); >>> ??? L508: >>> ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && >>> obj->klass()->is_box()) { >>> ??? L510: ??? return false; >>> ??? L511: ? } >>> ??????? The new bailout on L509-511 can move above L507. >> Moved. >> >>> L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass >>> %s", p2i(obj()), obj->klass()->external_name()); >>> ??????? This external_name() call does not have a ResourceMark. >> Good catch! I had one in a previous version but then I changed the >> conditionals and lost it for the fatal error case. The test worked >> okay because for the main JavaThread there is a ResourceMark in >> jni_invoke_static() (jni.cpp). >> >>> src/hotspot/share/logging/logTag.hpp >>> ??? No comments. >>> >>> src/hotspot/share/oops/klass.hpp >>> ??? No comments. >>> >>> src/hotspot/share/utilities/accessFlags.hpp >>> ??? No comments. >>> >>> src/hotspot/share/runtime/globals.hpp >>> ??? L814: ???????????? "0: off " >>> ??????? Missing a ';' after "off". >> Fixed. >> >>> L816: ???????????? "2: log message to stdout. >>> ??????? Perhaps add "(by default)" after "stdout" or >>> ??????? don't say where log output is at all. >>> >>> src/hotspot/share/runtime/arguments.cpp >>> ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, true, >>> LOG_TAGS(primitivewrappers)); >>> ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp >>> ??????? since it looks like logging output is configured to 'stdout'. >>> ??????? I'm assuming that other log options to put output elsewhere >>> ??????? are overridden by this code? >> Right. So the logging is done under UL with the tag >> primitivewrappers. If that tag is specified in Xlog then this >> conditional will be skipped because !log_is_enabled(Info, >> primitivewrappers) will be false. >> >>> src/hotspot/share/classfile/systemDictionary.cpp >>> ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { >>> ??????? nit - s/T_LONG+1/T_LONG + 1/ >> Fixed. >> >>> L2191: _box_klasses[i]->set_prototype_header(markWord::prototype()); >>> ??????? I assume we're keeping the prototype_header field when >>> Biased Locking >>> ??????? goes away? The reason I ask: >>> >>> ? ? ? ? static markWord prototype() { >>> ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); >>> ? ? ? ? } >>> >>> ??????? is because without Biased Locking, do we really need the >>> prototype >>> ??????? anymore? The initial markWord won't need possible variants... >> Yes, I think it can go away unless somebody finds another use for it. >> Maybe Valhalla. >> >>> src/hotspot/share/jfr/metadata/metadata.xml >>> ??? L69: ? >> ??????? Is the category "Java Application" because it's the application >>> ??????? code that did something "wrong" here? Where "application" is >>> loosely >>> ??????? defined to include the possibility of auto generated code, >>> library >>> ??????? code and the like? Or perhaps "application" because >>> something "above" >>> ??????? the "Java Virtual Machine, Runtime" did the "wrong" thing here? >> I don't know what the right category should be really. I saw the >> events JavaMonitorEnter, JavaMonitorWait and JavaMonitorInflate and >> thought this event should fall in the same category they do. >> >>> src/jdk.jfr/share/conf/jfr/default.jfc >>> ??? No comments. >>> >>> src/jdk.jfr/share/conf/jfr/profile.jfc >>> ??? No comments. >>> >>> test/lib/jdk/test/lib/jfr/EventNames.java >>> ??? No comments. >>> >>> src/hotspot/cpu/aarch64/aarch64.ad >>> ??? L3517: ????? __ tbnz(tmp, exact_log2(JVM_ACC_IS_BOX_CLASS), cont); >>> >>> ??? L3578: ??? __ bind(cont); >>> ??? L3579: ??? // flag == EQ indicates success >>> ??? L3580: ??? // flag == NE indicates failure >>> ??????? If tbnz() branches to "cont" when we have a box class, what's >>> ??????? the flag value set to (EQ or NE)? And what set that flag value? >>> ??????? The reason I ask is I don't think tbnz() sets condition >>> codes... >> Right, it doesn't set condition codes, so I kept the version I had >> (more on that below). >> >>> src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp >>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>> ??????????? ? ?? ??????????????????????????? Register obj_reg, >>> ?????????????????????????????? ? ?? ???????? Register swap_reg, >>> ?????????????????????????????????? ? ?? ???? Register tmp_reg, >>> ?????????????????????????????????? ? ?? ???? bool >>> swap_reg_contains_mark, >>> ?????????????????????????????????? ? ?? ???? Label& done, >>> ??????????????????????????????????? ? ?? ??? Label* slow_case, >>> BiasedLockingCounters* counters) { >>> ??????? I think you've changed the only callers of >>> biased_locking_enter() >>> ??????? that cared about the return value with this changeset so it can >>> ??????? be changed to a void function. >> Ok, this is what I mentioned to David in a previous email. Done. >> >>> src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp >>> ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), done); >>> >>> ??? L131: ? bind(done); >>> ??? L132: >>> ??? L133: ? // At this point flags are set as follows: >>> ??? L134: ? //? EQ -> Success >>> ??? L135: ? //? NE -> Failure, branch to slow path >>> ??????? If tbnz() branches to "done" when we have a box class, what's >>> ??????? the flag value set to (EQ or NE)? And what set that flag value? >>> ??????? The reason I ask is I don't think tbnz() sets condition >>> codes... >> Right. Same as above, I kept the version I had. >> >>> src/hotspot/cpu/arm/interp_masm_arm.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/arm/macroAssembler_arm.cpp >>> ??? int MacroAssembler::biased_locking_enter(Register obj_reg, >>> Register swap_reg, Register tmp_reg, >>> ????????????????????? ? ?? ????????????????? bool >>> swap_reg_contains_mark, >>> ???????????????????????? ? ?? ?????????????? Register tmp2, >>> ????????????????????????? ? ?? ????????????? Label& done, Label& >>> slow_case, >>> BiasedLockingCounters* counters) { >>> ??????? I think you've changed the only callers of >>> biased_locking_enter() >>> ??????? that cared about the return value with this changeset so it can >>> ??????? be changed to a void function. >> Done. >> >>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>> ??? No comments on the PPC code. >>> >>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>> src/hotspot/cpu/s390/interp_masm_s390.cpp >>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>> ??? No comments on the S390 code. >>> >>> src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp >>> ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); >>> ??????? What will this do with a 'noreg' value? (I need a refresher.) >> When not in 64 bit mode that register just won't be used. >> >>> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/x86/interp_masm_x86.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/x86/macroAssembler_x86.cpp >>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>> ????????????????????????????????? ? ?? ????? Register obj_reg, >>> ????????????????????????????????? ? ?? ????? Register swap_reg, >>> ????????????????????????????????? ? ?? ????? Register tmp_reg, >>> ?????????????????????????????????? ? ?? ???? Register tmp_reg2, >>> ?????????????????????????????????????? ? ? ? bool >>> swap_reg_contains_mark, >>> ?????????????????????????????????? ? ?? ???? Label& done, >>> ?????????????????????????????????? ? ?? ???? Label* slow_case, >>> BiasedLockingCounters* counters) { >>> ??????? I think you've changed the only caller of >>> biased_locking_enter() >>> ??????? that cared about the return value with this changeset so it can >>> ??????? be changed to a void function. >> Done. >> >>> test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >>> ??? L30:? * @test SyncOnPrimitiveWrapperTest >>> ??????? No parameter to @test directive. >> Removed. >> >>> L136: ??????? private static long sharedCounter = 0L; >>> ??????? Since you don't do anything with sharedCounter other than >>> increment it, >>> ??????? can the compilers optimize it away? If it can be optimized >>> away, does >>> ??????? that mean that: >>> >>> ??????????? L142: ??????????????? synchronized (obj) { >>> >>> ??????? can also be optimized away? >>> >>> ??????? I don't think that: >>> >>> ??????????? L161: ??????????????? synchronized (sharedLock1) { >>> >>> ??????? can be optimized away because it is shared by multiple threads. >>> >>> test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java >>> ??? Similar questions about 'counter' being optimized away. >>> ??? Similar question about "synchronized (obj) {" being optimized away. >> I'm not sure if the compiler will optimize it away. Seems unlikely >> given the counter we are incrementing is not just local to some thread. >> >> >> Ok, below is v3 which has the following changes: >> - Use a 32 bit load for the _access_flags field, instead of 64 >> - Martin's implementation for s390 and fix for PPC >> - Faster LogTest version >> - Above changes based on Dan review >> >> I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 only) >> again with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I checked it builds >> in arm32, ppc and s390. >> >> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev >> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev >> >> >> @Martin: >> Please check if the test doesn't timeout for you now with the changes >> I made to LogTest. >> Also, I tried to use tbnz in aarch64 and arm32 instead of tst + br >> (except for c2 since we actually need to set the condition flags), >> but for c1 I was getting an assertion in the compiler thread: >> >> guarantee(chk == -1 || chk == 0) failed: Field too big for insn >> >> This is the stack when that happens: >> >> MacroAssembler::pd_patch_instruction_size(unsigned char*, unsigned >> char*)+0x398 >> AbstractAssembler::bind(Label&)+0x118 >> MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 >> LIR_Assembler::emit_slow_case_stubs()+0x54 >> Compilation::emit_code_body()+0x17c >> >> The change was simply: >> >> diff --git a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> @@ -78,7 +78,6 @@ >> ?? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >> ???? load_klass(hdr, obj); >> -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); >> -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); >> -??? br(Assembler::NE, slow_case); >> +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); >> +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); >> ?? } >> >> So the issue must be related to where we want to jump. >> >> Thanks, >> Patricio >>> Dan >>> >>> >>> On 8/5/20 9:01 AM, Patricio Chilano wrote: >>>> Hi Martin, >>>> >>>> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>>>> Hi Patricio, >>>>> >>>>> using 8 Byte load instructions for jint fields is a terrible >>>>> coding style! >>>>> Someone else may see it and use an 8 Byte store. Will result in >>>>> great fun for debugging! >>>>> >>>>> And on Big Endian you will simply access the wrong bits. >>>> Ah, of course! Those 32 bits will be in the wrong place when doing >>>> the test. >>>> >>>>> Note that Big Endian Platforms are AIX, old linux ppc, s390, >>>>> SPARC. I don't think you have tested on them. >>>>> >>>>>> We could remove the nested synchronized statements in the run() >>>>>> method >>>>>> so that we don't generate that much logging. We could also lower >>>>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>>>> with multiple flag combinations. Not sure what we should touch >>>>>> first. >>>>>> Maybe the synchronized statements, have only one or two and test >>>>>> that >>>>>> first? >>>>> Sounds like helpful ideas. Please go ahead and strip things down. >>>> Great, I will send v3 later with those changes. >>>> >>>> Thanks Martin! >>>> >>>> Patricio >>>>> Thanks for taking care of it. >>>>> >>>>> Best regards, >>>>> Martin >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: Patricio Chilano >>>>>> Sent: Dienstag, 4. August 2020 20:26 >>>>>> To: Doerr, Martin ; David Holmes >>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>> wrappers >>>>>> >>>>>> Hi Martin, >>>>>> >>>>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>>>> Hi Patricio, >>>>>>> >>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>> sending v3. >>>>>>> Thanks. I forgot to mention arm + aarch64. >>>>>>> Aarch64 uses ldrw + tbnz. >>>>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>>>> >>>>>>> I remember having seen the same mistake ?? >>>>>>> And nobody noticed it on little Endian platforms. >>>>>> Ok, I can use a tbnz instead of test and then a branch on arm32 >>>>>> and aarch64. >>>>>> Shouldn't a normal ldr in arm32 work fine? >>>>>> Also in 64 bits (either x64 or aarch64) I don't see the issue of >>>>>> using a >>>>>> 64 bit load, besides the fact that we only care about the first >>>>>> 32 bits. >>>>>> Regardless of the endianness, aren't we masking out the upper >>>>>> part when >>>>>> we do AND/TEST or if we test a bit in the 0-31 bit range? >>>>>> Otherwise it >>>>>> seems it should have failed for one of those platforms in my tests. >>>>>> >>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>> works and I >>>>>>>> can add it to the test. >>>>>>> That seems to have an effect. But now I'm not patient enough to >>>>>>> wait for >>>>>> the test to finish. >>>>>>> Maybe the problem is that I'm using slow debug builds. >>>>>>> But is there a chance to make the test quicker without losing >>>>>>> coverage >>>>>> significantly? >>>>>>> I think the test is too slow this way to run it on a regular >>>>>>> basis. We'd need to >>>>>> spend dedicated machines for it. >>>>>> We could remove the nested synchronized statements in the run() >>>>>> method >>>>>> so that we don't generate that much logging. We could also lower >>>>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>>>> with multiple flag combinations. Not sure what we should touch >>>>>> first. >>>>>> Maybe the synchronized statements, have only one or two and test >>>>>> that >>>>>> first? >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Patricio >>>>>>> Best regards, >>>>>>> Martin >>>>>>> >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Patricio Chilano >>>>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>>>> wrappers >>>>>>>> >>>>>>>> Hi Martin, >>>>>>>> >>>>>>>> Thanks for fixing PPC and taking care of s390! >>>>>>>> >>>>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>>>> Hi Patricio, >>>>>>>>> >>>>>>>>> I suggest to use movl + testl for checking the access flag as >>>>>>>>> for other >>>>>> access >>>>>>>> flags on x64. >>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>> sending v3. >>>>>>>> >>>>>>>>> New version for PPC64 and s390 see below. >>>>>>>>> >>>>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>>>>> expected. >>>>>>>> However, I'm getting timeout issues: >>>>>>>>> timed out (timeout set to 120000ms, elapsed time including >>>>>>>>> timeout >>>>>>>> handling was 122372ms) >>>>>>>>> Can we provide more time? >>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>> works and I >>>>>>>> can add it to the test. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Patricio >>>>>>>>> Best regards, >>>>>>>>> Martin >>>>>>>>> >>>>>>>>> >>>>>>>>> diff -r 77852e129401 >>>>>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -107,8 +107,8 @@ >>>>>>>>> >>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>> Rscratch); >>>>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>>>> +??? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>> Rscratch); >>>>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> ??????? bne(CCR0, slow_case); >>>>>>>>> ????? } >>>>>>>>> >>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -912,8 +912,8 @@ >>>>>>>>> >>>>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> ????????? load_klass(tmp, object); >>>>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>> +????? testbitdi(CCR0, R0, tmp, >>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> ????????? bne(CCR0, slow_case); >>>>>>>>> ??????? } >>>>>>>>> >>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>>>> >>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> ??????? load_klass(temp, oop); >>>>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>> +??? testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> ??????? bne(CCR0, cont); >>>>>>>>> ????? } >>>>>>>>> >>>>>>>>> diff -r 77852e129401 >>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -91,6 +91,12 @@ >>>>>>>>> ????? // Save object being locked into the BasicObjectLock... >>>>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> +??? z_btrue(slow_case); >>>>>>>>> +? } >>>>>>>>> + >>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>>>> Z_R0_scratch, done, >>>>>>>> &slow_case); >>>>>>>>> ????? } >>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>>> 10:03:57 >>>>>>>> 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>> 16:04:31 >>>>>>>> 2020 +0200 >>>>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>>>> ????? z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), >>>>>>>>> object); >>>>>>>>> >>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> +??? z_btrue(slow_case); >>>>>>>>> +? } >>>>>>>>> + >>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>> ??????? biased_locking_enter(object, displaced_header, Z_R1, >>>>>>>>> Z_R0, done, >>>>>>>> &slow_case); >>>>>>>>> ????? } >>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>>>> ????? // Load markWord from oop into mark. >>>>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>>>> >>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> +??? z_btrue(done); >>>>>>>>> +? } >>>>>>>>> + >>>>>>>>> ????? if (try_bias) { >>>>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, Z_R0, >>>>>>>>> done); >>>>>>>>> ????? } >>>>>>>>> >>>> >>> >> > From patricio.chilano.mateo at oracle.com Fri Aug 7 01:45:09 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Thu, 6 Aug 2020 22:45:09 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> Message-ID: Hi Coleen, Thanks for looking at this. On 8/6/20 7:17 PM, Coleen Phillimore wrote: > > Patricio,? One question: > > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/src/hotspot/share/runtime/synchronizer.cpp.udiff.html > > > + // adjust bcp to point back to monitorenter so that we print the > correct line numbers > > > How do the fatal and event print the correct line numbers here?? I see > the logging gets it from the stack trace.? Should the abort message > have more information in it?? You can get the source and line number > information in the same way that print_stack_on() gets it. For the fatal error case I'm not printing line numbers as I do with the warning case or as you get with JFR events. But maybe I should print the stack too and then exit the VM. You can get the stack info and the line number of the monitorenter bytecode that caused the crash from the generated hs_err file though. I don't know which exact technique JFR uses to print line numbers but it has to include reading the current bcp. Since when coming from the interpreter for monitorenter the bcp is always already pointing to the next instruction we need to decrement it to print the correct line numbers. I tested it and if I don't fix the bcp, as expected JFR too will print the next line number for the top frame. Patricio > Otherwise, this looks good to me. > Coleen > > > On 8/6/20 2:48 PM, Patricio Chilano wrote: >> Hi Dan, >> >> On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: >>> I'm peeking ahead to the next webrev... :-) >>> >>> > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ >> : ) >> >>> General comments: >>> ? - check files for copyright year updates. >> Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, >> c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. >> >>> src/hotspot/share/runtime/synchronizer.hpp >>> ??? No comments. >>> >>> src/hotspot/share/runtime/synchronizer.cpp >>> ??? L507: ? const markWord mark = obj->mark(); >>> ??? L508: >>> ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && >>> obj->klass()->is_box()) { >>> ??? L510: ??? return false; >>> ??? L511: ? } >>> ??????? The new bailout on L509-511 can move above L507. >> Moved. >> >>> L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass >>> %s", p2i(obj()), obj->klass()->external_name()); >>> ??????? This external_name() call does not have a ResourceMark. >> Good catch! I had one in a previous version but then I changed the >> conditionals and lost it for the fatal error case. The test worked >> okay because for the main JavaThread there is a ResourceMark in >> jni_invoke_static() (jni.cpp). >> >>> src/hotspot/share/logging/logTag.hpp >>> ??? No comments. >>> >>> src/hotspot/share/oops/klass.hpp >>> ??? No comments. >>> >>> src/hotspot/share/utilities/accessFlags.hpp >>> ??? No comments. >>> >>> src/hotspot/share/runtime/globals.hpp >>> ??? L814: ???????????? "0: off " >>> ??????? Missing a ';' after "off". >> Fixed. >> >>> L816: ???????????? "2: log message to stdout. >>> ??????? Perhaps add "(by default)" after "stdout" or >>> ??????? don't say where log output is at all. >>> >>> src/hotspot/share/runtime/arguments.cpp >>> ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, true, >>> LOG_TAGS(primitivewrappers)); >>> ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp >>> ??????? since it looks like logging output is configured to 'stdout'. >>> ??????? I'm assuming that other log options to put output elsewhere >>> ??????? are overridden by this code? >> Right. So the logging is done under UL with the tag >> primitivewrappers. If that tag is specified in Xlog then this >> conditional will be skipped because !log_is_enabled(Info, >> primitivewrappers) will be false. >> >>> src/hotspot/share/classfile/systemDictionary.cpp >>> ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { >>> ??????? nit - s/T_LONG+1/T_LONG + 1/ >> Fixed. >> >>> L2191: _box_klasses[i]->set_prototype_header(markWord::prototype()); >>> ??????? I assume we're keeping the prototype_header field when >>> Biased Locking >>> ??????? goes away? The reason I ask: >>> >>> ? ? ? ? static markWord prototype() { >>> ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); >>> ? ? ? ? } >>> >>> ??????? is because without Biased Locking, do we really need the >>> prototype >>> ??????? anymore? The initial markWord won't need possible variants... >> Yes, I think it can go away unless somebody finds another use for it. >> Maybe Valhalla. >> >>> src/hotspot/share/jfr/metadata/metadata.xml >>> ??? L69: ? >> ??????? Is the category "Java Application" because it's the application >>> ??????? code that did something "wrong" here? Where "application" is >>> loosely >>> ??????? defined to include the possibility of auto generated code, >>> library >>> ??????? code and the like? Or perhaps "application" because >>> something "above" >>> ??????? the "Java Virtual Machine, Runtime" did the "wrong" thing here? >> I don't know what the right category should be really. I saw the >> events JavaMonitorEnter, JavaMonitorWait and JavaMonitorInflate and >> thought this event should fall in the same category they do. >> >>> src/jdk.jfr/share/conf/jfr/default.jfc >>> ??? No comments. >>> >>> src/jdk.jfr/share/conf/jfr/profile.jfc >>> ??? No comments. >>> >>> test/lib/jdk/test/lib/jfr/EventNames.java >>> ??? No comments. >>> >>> src/hotspot/cpu/aarch64/aarch64.ad >>> ??? L3517: ????? __ tbnz(tmp, exact_log2(JVM_ACC_IS_BOX_CLASS), cont); >>> >>> ??? L3578: ??? __ bind(cont); >>> ??? L3579: ??? // flag == EQ indicates success >>> ??? L3580: ??? // flag == NE indicates failure >>> ??????? If tbnz() branches to "cont" when we have a box class, what's >>> ??????? the flag value set to (EQ or NE)? And what set that flag value? >>> ??????? The reason I ask is I don't think tbnz() sets condition >>> codes... >> Right, it doesn't set condition codes, so I kept the version I had >> (more on that below). >> >>> src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp >>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>> ??????????? ? ?? ??????????????????????????? Register obj_reg, >>> ?????????????????????????????? ? ?? ???????? Register swap_reg, >>> ?????????????????????????????????? ? ?? ???? Register tmp_reg, >>> ?????????????????????????????????? ? ?? ???? bool >>> swap_reg_contains_mark, >>> ?????????????????????????????????? ? ?? ???? Label& done, >>> ??????????????????????????????????? ? ?? ??? Label* slow_case, >>> BiasedLockingCounters* counters) { >>> ??????? I think you've changed the only callers of >>> biased_locking_enter() >>> ??????? that cared about the return value with this changeset so it can >>> ??????? be changed to a void function. >> Ok, this is what I mentioned to David in a previous email. Done. >> >>> src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp >>> ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), done); >>> >>> ??? L131: ? bind(done); >>> ??? L132: >>> ??? L133: ? // At this point flags are set as follows: >>> ??? L134: ? //? EQ -> Success >>> ??? L135: ? //? NE -> Failure, branch to slow path >>> ??????? If tbnz() branches to "done" when we have a box class, what's >>> ??????? the flag value set to (EQ or NE)? And what set that flag value? >>> ??????? The reason I ask is I don't think tbnz() sets condition >>> codes... >> Right. Same as above, I kept the version I had. >> >>> src/hotspot/cpu/arm/interp_masm_arm.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/arm/macroAssembler_arm.cpp >>> ??? int MacroAssembler::biased_locking_enter(Register obj_reg, >>> Register swap_reg, Register tmp_reg, >>> ????????????????????? ? ?? ????????????????? bool >>> swap_reg_contains_mark, >>> ???????????????????????? ? ?? ?????????????? Register tmp2, >>> ????????????????????????? ? ?? ????????????? Label& done, Label& >>> slow_case, >>> BiasedLockingCounters* counters) { >>> ??????? I think you've changed the only callers of >>> biased_locking_enter() >>> ??????? that cared about the return value with this changeset so it can >>> ??????? be changed to a void function. >> Done. >> >>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>> ??? No comments on the PPC code. >>> >>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>> src/hotspot/cpu/s390/interp_masm_s390.cpp >>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>> ??? No comments on the S390 code. >>> >>> src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp >>> ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); >>> ??????? What will this do with a 'noreg' value? (I need a refresher.) >> When not in 64 bit mode that register just won't be used. >> >>> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/x86/interp_masm_x86.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/x86/macroAssembler_x86.cpp >>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>> ????????????????????????????????? ? ?? ????? Register obj_reg, >>> ????????????????????????????????? ? ?? ????? Register swap_reg, >>> ????????????????????????????????? ? ?? ????? Register tmp_reg, >>> ?????????????????????????????????? ? ?? ???? Register tmp_reg2, >>> ?????????????????????????????????????? ? ? ? bool >>> swap_reg_contains_mark, >>> ?????????????????????????????????? ? ?? ???? Label& done, >>> ?????????????????????????????????? ? ?? ???? Label* slow_case, >>> BiasedLockingCounters* counters) { >>> ??????? I think you've changed the only caller of >>> biased_locking_enter() >>> ??????? that cared about the return value with this changeset so it can >>> ??????? be changed to a void function. >> Done. >> >>> test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >>> ??? L30:? * @test SyncOnPrimitiveWrapperTest >>> ??????? No parameter to @test directive. >> Removed. >> >>> L136: ??????? private static long sharedCounter = 0L; >>> ??????? Since you don't do anything with sharedCounter other than >>> increment it, >>> ??????? can the compilers optimize it away? If it can be optimized >>> away, does >>> ??????? that mean that: >>> >>> ??????????? L142: ??????????????? synchronized (obj) { >>> >>> ??????? can also be optimized away? >>> >>> ??????? I don't think that: >>> >>> ??????????? L161: ??????????????? synchronized (sharedLock1) { >>> >>> ??????? can be optimized away because it is shared by multiple threads. >>> >>> test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java >>> ??? Similar questions about 'counter' being optimized away. >>> ??? Similar question about "synchronized (obj) {" being optimized away. >> I'm not sure if the compiler will optimize it away. Seems unlikely >> given the counter we are incrementing is not just local to some thread. >> >> >> Ok, below is v3 which has the following changes: >> - Use a 32 bit load for the _access_flags field, instead of 64 >> - Martin's implementation for s390 and fix for PPC >> - Faster LogTest version >> - Above changes based on Dan review >> >> I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 only) >> again with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I checked it builds >> in arm32, ppc and s390. >> >> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev >> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev >> >> >> @Martin: >> Please check if the test doesn't timeout for you now with the changes >> I made to LogTest. >> Also, I tried to use tbnz in aarch64 and arm32 instead of tst + br >> (except for c2 since we actually need to set the condition flags), >> but for c1 I was getting an assertion in the compiler thread: >> >> guarantee(chk == -1 || chk == 0) failed: Field too big for insn >> >> This is the stack when that happens: >> >> MacroAssembler::pd_patch_instruction_size(unsigned char*, unsigned >> char*)+0x398 >> AbstractAssembler::bind(Label&)+0x118 >> MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 >> LIR_Assembler::emit_slow_case_stubs()+0x54 >> Compilation::emit_code_body()+0x17c >> >> The change was simply: >> >> diff --git a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> @@ -78,7 +78,6 @@ >> ?? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >> ???? load_klass(hdr, obj); >> -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); >> -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); >> -??? br(Assembler::NE, slow_case); >> +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); >> +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); >> ?? } >> >> So the issue must be related to where we want to jump. >> >> Thanks, >> Patricio >>> Dan >>> >>> >>> On 8/5/20 9:01 AM, Patricio Chilano wrote: >>>> Hi Martin, >>>> >>>> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>>>> Hi Patricio, >>>>> >>>>> using 8 Byte load instructions for jint fields is a terrible >>>>> coding style! >>>>> Someone else may see it and use an 8 Byte store. Will result in >>>>> great fun for debugging! >>>>> >>>>> And on Big Endian you will simply access the wrong bits. >>>> Ah, of course! Those 32 bits will be in the wrong place when doing >>>> the test. >>>> >>>>> Note that Big Endian Platforms are AIX, old linux ppc, s390, >>>>> SPARC. I don't think you have tested on them. >>>>> >>>>>> We could remove the nested synchronized statements in the run() >>>>>> method >>>>>> so that we don't generate that much logging. We could also lower >>>>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>>>> with multiple flag combinations. Not sure what we should touch >>>>>> first. >>>>>> Maybe the synchronized statements, have only one or two and test >>>>>> that >>>>>> first? >>>>> Sounds like helpful ideas. Please go ahead and strip things down. >>>> Great, I will send v3 later with those changes. >>>> >>>> Thanks Martin! >>>> >>>> Patricio >>>>> Thanks for taking care of it. >>>>> >>>>> Best regards, >>>>> Martin >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: Patricio Chilano >>>>>> Sent: Dienstag, 4. August 2020 20:26 >>>>>> To: Doerr, Martin ; David Holmes >>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>> wrappers >>>>>> >>>>>> Hi Martin, >>>>>> >>>>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>>>> Hi Patricio, >>>>>>> >>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>> sending v3. >>>>>>> Thanks. I forgot to mention arm + aarch64. >>>>>>> Aarch64 uses ldrw + tbnz. >>>>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>>>> >>>>>>> I remember having seen the same mistake ?? >>>>>>> And nobody noticed it on little Endian platforms. >>>>>> Ok, I can use a tbnz instead of test and then a branch on arm32 >>>>>> and aarch64. >>>>>> Shouldn't a normal ldr in arm32 work fine? >>>>>> Also in 64 bits (either x64 or aarch64) I don't see the issue of >>>>>> using a >>>>>> 64 bit load, besides the fact that we only care about the first >>>>>> 32 bits. >>>>>> Regardless of the endianness, aren't we masking out the upper >>>>>> part when >>>>>> we do AND/TEST or if we test a bit in the 0-31 bit range? >>>>>> Otherwise it >>>>>> seems it should have failed for one of those platforms in my tests. >>>>>> >>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>> works and I >>>>>>>> can add it to the test. >>>>>>> That seems to have an effect. But now I'm not patient enough to >>>>>>> wait for >>>>>> the test to finish. >>>>>>> Maybe the problem is that I'm using slow debug builds. >>>>>>> But is there a chance to make the test quicker without losing >>>>>>> coverage >>>>>> significantly? >>>>>>> I think the test is too slow this way to run it on a regular >>>>>>> basis. We'd need to >>>>>> spend dedicated machines for it. >>>>>> We could remove the nested synchronized statements in the run() >>>>>> method >>>>>> so that we don't generate that much logging. We could also lower >>>>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>>>> with multiple flag combinations. Not sure what we should touch >>>>>> first. >>>>>> Maybe the synchronized statements, have only one or two and test >>>>>> that >>>>>> first? >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Patricio >>>>>>> Best regards, >>>>>>> Martin >>>>>>> >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Patricio Chilano >>>>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>>>> wrappers >>>>>>>> >>>>>>>> Hi Martin, >>>>>>>> >>>>>>>> Thanks for fixing PPC and taking care of s390! >>>>>>>> >>>>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>>>> Hi Patricio, >>>>>>>>> >>>>>>>>> I suggest to use movl + testl for checking the access flag as >>>>>>>>> for other >>>>>> access >>>>>>>> flags on x64. >>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>> sending v3. >>>>>>>> >>>>>>>>> New version for PPC64 and s390 see below. >>>>>>>>> >>>>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>>>>> expected. >>>>>>>> However, I'm getting timeout issues: >>>>>>>>> timed out (timeout set to 120000ms, elapsed time including >>>>>>>>> timeout >>>>>>>> handling was 122372ms) >>>>>>>>> Can we provide more time? >>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>> works and I >>>>>>>> can add it to the test. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Patricio >>>>>>>>> Best regards, >>>>>>>>> Martin >>>>>>>>> >>>>>>>>> >>>>>>>>> diff -r 77852e129401 >>>>>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -107,8 +107,8 @@ >>>>>>>>> >>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>> Rscratch); >>>>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>>>> +??? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>> Rscratch); >>>>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> ??????? bne(CCR0, slow_case); >>>>>>>>> ????? } >>>>>>>>> >>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -912,8 +912,8 @@ >>>>>>>>> >>>>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> ????????? load_klass(tmp, object); >>>>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>> +????? testbitdi(CCR0, R0, tmp, >>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> ????????? bne(CCR0, slow_case); >>>>>>>>> ??????? } >>>>>>>>> >>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>>>> >>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> ??????? load_klass(temp, oop); >>>>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>> +??? testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> ??????? bne(CCR0, cont); >>>>>>>>> ????? } >>>>>>>>> >>>>>>>>> diff -r 77852e129401 >>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -91,6 +91,12 @@ >>>>>>>>> ????? // Save object being locked into the BasicObjectLock... >>>>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> +??? z_btrue(slow_case); >>>>>>>>> +? } >>>>>>>>> + >>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>>>> Z_R0_scratch, done, >>>>>>>> &slow_case); >>>>>>>>> ????? } >>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>>> 10:03:57 >>>>>>>> 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>> 16:04:31 >>>>>>>> 2020 +0200 >>>>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>>>> ????? z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), >>>>>>>>> object); >>>>>>>>> >>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> +??? z_btrue(slow_case); >>>>>>>>> +? } >>>>>>>>> + >>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>> ??????? biased_locking_enter(object, displaced_header, Z_R1, >>>>>>>>> Z_R0, done, >>>>>>>> &slow_case); >>>>>>>>> ????? } >>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>>>> ????? // Load markWord from oop into mark. >>>>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>>>> >>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> +??? z_btrue(done); >>>>>>>>> +? } >>>>>>>>> + >>>>>>>>> ????? if (try_bias) { >>>>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, Z_R0, >>>>>>>>> done); >>>>>>>>> ????? } >>>>>>>>> >>>> >>> >> > From martin.doerr at sap.com Fri Aug 7 03:58:50 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 7 Aug 2020 03:58:50 +0000 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> Message-ID: Hi Patricio, thanks for the update. Sorry for causing trouble related to the tbnz instructions on aarch64. My concern was basically to use the correct number of Bytes for the load instructions. You have fixed that in webrev v3. Thanks. Unfortunately, I did the same mistake regarding the flag requirements on PPC64 and s390. And we have a wrong label on PPC64 (otherwise branch destination can be too far). Test duration is still terrible with the slow debug build. I still got timeouts. (Maybe I should better use fast debug builds.) Fixes see below. Sometimes simple changes can require quite some time ... Best regards, Martin diff -r f629202b3862 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Thu Aug 06 22:03:39 2020 +0200 +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Thu Aug 06 23:26:02 2020 +0200 @@ -109,7 +109,7 @@ load_klass(Rscratch, Roop); lwz(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); testbitdi(CCR0, R0, Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS)); - bne(CCR0, slow_case); + bne(CCR0, slow_int); } if (UseBiasedLocking) { diff -r f629202b3862 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Thu Aug 06 22:03:39 2020 +0200 +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Thu Aug 06 23:26:02 2020 +0200 @@ -2839,8 +2839,8 @@ if (DiagnoseSyncOnPrimitiveWrappers != 0) { load_klass(temp, oop); lwz(temp, in_bytes(Klass::access_flags_offset()), temp); - testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); - bne(CCR0, cont); + testbitdi(flag, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); + bne(flag, cont); } if (try_bias) { diff -r f629202b3862 src/hotspot/cpu/s390/macroAssembler_s390.cpp --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Thu Aug 06 22:03:39 2020 +0200 +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Thu Aug 06 23:26:02 2020 +0200 @@ -3360,8 +3360,10 @@ if (DiagnoseSyncOnPrimitiveWrappers != 0) { load_klass(Z_R1_scratch, oop); - testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), exact_log2(JVM_ACC_IS_BOX_CLASS)); - z_btrue(done); + z_l(Z_R1_scratch, Address(Z_R1_scratch, Klass::access_flags_offset())); + assert((JVM_ACC_IS_BOX_CLASS & 0xFFFF) == 0, "or change following instruction"); + z_nilh(Z_R1_scratch, JVM_ACC_IS_BOX_CLASS >> 16); + z_brne(done); } if (try_bias) { diff -r f629202b3862 test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java --- a/test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java Thu Aug 06 22:03:39 2020 +0200 +++ b/test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java Thu Aug 06 23:26:02 2020 +0200 @@ -31,7 +31,7 @@ * @bug 8242263 * @summary Exercise DiagnoseSyncOnPrimitiveWrappers diagnostic flag * @library /test/lib - * @run driver SyncOnPrimitiveWrapperTest + * @run driver/timeout=180000 SyncOnPrimitiveWrapperTest */ public class SyncOnPrimitiveWrapperTest { From ioi.lam at oracle.com Fri Aug 7 04:41:55 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 6 Aug 2020 21:41:55 -0700 Subject: RFR (xs) 8251209 [TESTBUG] CDS jvmti tests should use "@modules java.instrument" In-Reply-To: <138c579d-2ab9-5e35-9150-dfdd74e899eb@oracle.com> References: <2b9000ef-9a09-caf4-8610-bb354afb722b@oracle.com> <138c579d-2ab9-5e35-9150-dfdd74e899eb@oracle.com> Message-ID: On 8/6/20 5:58 PM, David Holmes wrote: > Correction ... > > On 7/08/2020 7:52 am, David Holmes wrote: >> Hi Ioi, >> >> On 7/08/2020 4:25 am, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8251209 >>> http://cr.openjdk.java.net/~iklam/jdk16/8251209-cds-jvmti-tests-modules-tag.v01/ >>> >>> >>> Summary -- changed the tests from (mis)using >>> >>> ??* @requires vm.flavor != "minimal" >>> >>> to >>> >>> ??* @modules java.instrument >>> >>> ... to be consistent with other jvmti tests. >> >> That seems like an invalid precondition to me. It would have been >> somewhat valid in the Compact Profiles world when we did not provide >> "java.instrument" in the profiles which supported MinimalVM, but you >> can define a minimal VM in a build that still has all modules >> available. I don't think building the minimal VM makes any changes to >> the supported modules. >> >> Also AIUI the @modules statement simply adds the necessary >> command-line args to use the java.instrument module (if present), it >> doesn't ensure that the listed module has to be present. > > It does in fact ensure that: > > "Otherwise, a test will not be run if the system being tested does not > contain all of the specified modules." > > http://openjdk.java.net/jtreg/tag-spec.html > > But as I said the module could be present in a JRE but you are still > using the MinimalVM. > Hi David, As I mentioned above, I am following the same rule as other jvmti tests, which only use "@modules java.instrument" and do not check whether the VM is minimal. E.g., http://hg.openjdk.java.net/jdk/jdk/file/4d36e29a5410/test/hotspot/jtreg/serviceability/jvmti/GetObjectSizeClass.java ------- If I understand correctly, you're saying someone can build a minimal JDK (configure --with-jvm-variants=minimal), and then try to add the java.instrument module to it. I.e., adding the following module to your JDK (with jlink, or by hand). $ unzip -l ./jmods/java.instrument.jmod ? Length????? Date??? Time??? Name ---------? ---------- -----?? ---- ????? 294? 2020-08-04 17:03?? classes/module-info.class ???? 1102? 2020-08-04 17:03 classes/sun/instrument/TransformerManager$TransformerInfo.class ???? 4294? 2020-08-04 17:03 classes/sun/instrument/TransformerManager.class ????? 911? 2020-08-04 17:03 classes/sun/instrument/InstrumentationImpl$1.class ??? 16663? 2020-08-04 17:03 classes/sun/instrument/InstrumentationImpl.class ???? 1356? 2020-08-04 17:03 classes/java/lang/instrument/ClassFileTransformer.class ????? 554? 2020-08-04 17:03 classes/java/lang/instrument/IllegalClassFormatException.class ???? 1734? 2020-08-04 17:03 classes/java/lang/instrument/Instrumentation.class ????? 563? 2020-08-04 17:03 classes/java/lang/instrument/UnmodifiableModuleException.class ????? 970? 2020-08-04 17:03 classes/java/lang/instrument/ClassDefinition.class ????? 551? 2020-08-04 17:03 classes/java/lang/instrument/UnmodifiableClassException.class ???? 3244? 2020-08-04 17:03?? legal/COPYRIGHT ?????? 44? 2020-08-04 17:03?? legal/LICENSE ??? 50920? 2020-08-04 17:03?? lib/libinstrument.so<<<<<<<<< But this module has a native library, libinstrument.so, which requires JVMTI to be present in libjvm.so. E.g.: ??? jvmtiEnv * ??? retransformableEnvironment(JPLISAgent * agent) { ??? .... ??????? jnierror = (*agent->mJVM)->GetEnv(? agent->mJVM, ??????? ? ?? ????????????????????? (void **) &retransformerEnv, ??????????? ? ?? ????????????????? JVMTI_VERSION_1_1); So if you try to run the CDS JVMTI test cases, it will be executed (because your JDK says "I have java.instrument") and the test finds out that your JDK's java.instrument module isn't working properly. So the test is doing exactly what it's supposed to do. I would argue that this is better than before (which would exclude the test when the libjvm.so is a minimal build, and would will not detect such a mis-configured java.instrument module.) Thanks - Ioi From Alan.Bateman at oracle.com Fri Aug 7 05:49:40 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 7 Aug 2020 06:49:40 +0100 Subject: RFR (xs) 8251209 [TESTBUG] CDS jvmti tests should use "@modules java.instrument" In-Reply-To: <138c579d-2ab9-5e35-9150-dfdd74e899eb@oracle.com> References: <2b9000ef-9a09-caf4-8610-bb354afb722b@oracle.com> <138c579d-2ab9-5e35-9150-dfdd74e899eb@oracle.com> Message-ID: On 07/08/2020 01:58, David Holmes wrote: > > It does in fact ensure that: > > "Otherwise, a test will not be run if the system being tested does not > contain all of the specified modules." > > http://openjdk.java.net/jtreg/tag-spec.html > > But as I said the module could be present in a JRE but you are still > using the MinimalVM. Right, tests with `@modules java.instrument` will not be selected if the run-time under test doesn't contain this module. It would be a bit strange to create a run-time image with a minimal VM build that doesn't have JVM TI but include java.instrument. All usages of -javaagent would be fatal because the JPLIS agent uses JVM TI. A long time ago there were calls for a way java.management and java.instrument to express that they required specific VM features but I don't think it came to anything. So nothing in jlink to catch this at link-time, at least for the case that there is only one libjvm in the generated run-time image. So technically I think the tests would need both @requires and @modules if someone really wanted to be able to run all tests with the minimal VM and expect jtreg to not select these tests. -Alan. From david.holmes at oracle.com Fri Aug 7 06:01:50 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Aug 2020 16:01:50 +1000 Subject: RFR (xs) 8251209 [TESTBUG] CDS jvmti tests should use "@modules java.instrument" In-Reply-To: References: <2b9000ef-9a09-caf4-8610-bb354afb722b@oracle.com> <138c579d-2ab9-5e35-9150-dfdd74e899eb@oracle.com> Message-ID: On 7/08/2020 2:41 pm, Ioi Lam wrote: > On 8/6/20 5:58 PM, David Holmes wrote: >> Correction ... >> >> On 7/08/2020 7:52 am, David Holmes wrote: >>> Hi Ioi, >>> >>> On 7/08/2020 4:25 am, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8251209 >>>> http://cr.openjdk.java.net/~iklam/jdk16/8251209-cds-jvmti-tests-modules-tag.v01/ >>>> >>>> >>>> Summary -- changed the tests from (mis)using >>>> >>>> ??* @requires vm.flavor != "minimal" >>>> >>>> to >>>> >>>> ??* @modules java.instrument >>>> >>>> ... to be consistent with other jvmti tests. >>> >>> That seems like an invalid precondition to me. It would have been >>> somewhat valid in the Compact Profiles world when we did not provide >>> "java.instrument" in the profiles which supported MinimalVM, but you >>> can define a minimal VM in a build that still has all modules >>> available. I don't think building the minimal VM makes any changes to >>> the supported modules. >>> >>> Also AIUI the @modules statement simply adds the necessary >>> command-line args to use the java.instrument module (if present), it >>> doesn't ensure that the listed module has to be present. >> >> It does in fact ensure that: >> >> "Otherwise, a test will not be run if the system being tested does not >> contain all of the specified modules." >> >> http://openjdk.java.net/jtreg/tag-spec.html >> >> But as I said the module could be present in a JRE but you are still >> using the MinimalVM. >> > > Hi David, > > As I mentioned above, I am following the same rule as other jvmti tests, > which only use "@modules java.instrument" and do not check whether the > VM is minimal. E.g., > > http://hg.openjdk.java.net/jdk/jdk/file/4d36e29a5410/test/hotspot/jtreg/serviceability/jvmti/GetObjectSizeClass.java Sure but I contend those tests are wrong and the tests you are changing are right (or more right given common test configurations). > > ------- > > If I understand correctly, you're saying someone can build a minimal JDK > (configure --with-jvm-variants=minimal), and then try to add the > java.instrument module to it. I.e., adding the following module to your > JDK (with jlink, or by hand). Just build a JDK with multiple VMs present. > > $ unzip -l ./jmods/java.instrument.jmod > ? Length????? Date??? Time??? Name > ---------? ---------- -----?? ---- > ????? 294? 2020-08-04 17:03?? classes/module-info.class > ???? 1102? 2020-08-04 17:03 > classes/sun/instrument/TransformerManager$TransformerInfo.class > ???? 4294? 2020-08-04 17:03 > classes/sun/instrument/TransformerManager.class > ????? 911? 2020-08-04 17:03 > classes/sun/instrument/InstrumentationImpl$1.class > ??? 16663? 2020-08-04 17:03 > classes/sun/instrument/InstrumentationImpl.class > ???? 1356? 2020-08-04 17:03 > classes/java/lang/instrument/ClassFileTransformer.class > ????? 554? 2020-08-04 17:03 > classes/java/lang/instrument/IllegalClassFormatException.class > ???? 1734? 2020-08-04 17:03 > classes/java/lang/instrument/Instrumentation.class > ????? 563? 2020-08-04 17:03 > classes/java/lang/instrument/UnmodifiableModuleException.class > ????? 970? 2020-08-04 17:03 > classes/java/lang/instrument/ClassDefinition.class > ????? 551? 2020-08-04 17:03 > classes/java/lang/instrument/UnmodifiableClassException.class > ???? 3244? 2020-08-04 17:03?? legal/COPYRIGHT > ?????? 44? 2020-08-04 17:03?? legal/LICENSE > ??? 50920? 2020-08-04 17:03?? lib/libinstrument.so<<<<<<<<< > > But this module has a native library, libinstrument.so, which requires > JVMTI to be present in libjvm.so. E.g.: > > ??? jvmtiEnv * > ??? retransformableEnvironment(JPLISAgent * agent) { > ??? .... > ??????? jnierror = (*agent->mJVM)->GetEnv(? agent->mJVM, > ??????? ? ?? ????????????????????? (void **) &retransformerEnv, > ??????????? ? ?? ????????????????? JVMTI_VERSION_1_1); > > So if you try to run the CDS JVMTI test cases, it will be executed > (because your JDK says "I have java.instrument") and the test finds out > that your JDK's java.instrument module isn't working properly. So the > test is doing exactly what it's supposed to do. The whole point of the @requires is to not waste time and resources running a test on a platform that cannot run the test successfully. So the fully correct solution could be to have both settings: @requires vm.flavor != "minimal" @modules java.instrument if you require both a VM that supports JVM TI and you need a JRE that includes the java.instrument module. But that assumes your test does need java.instrument. Not all JVM TI tests need java.instrument, but all instrumentation tests depend on JVM TI. Just looking at the first three of tests in your webrev I don't see any dependency on java.instrument - they are CDS only tests as far as I can see and so require a VM with CDS which means not a Minimal VM - though perhaps it is sufficient to have the @requires vm.cds in those cases? For the other JVM TI related tests using -javaagent they probably need both @requires and @module. David ----- > I would argue that this is better than before (which would exclude the > test when the libjvm.so is a minimal build, and would will not detect > such a mis-configured java.instrument module.) > > > Thanks > - Ioi > > From ioi.lam at oracle.com Fri Aug 7 06:13:59 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 6 Aug 2020 23:13:59 -0700 Subject: RFR (xs) 8251209 [TESTBUG] CDS jvmti tests should use "@modules java.instrument" In-Reply-To: References: <2b9000ef-9a09-caf4-8610-bb354afb722b@oracle.com> <138c579d-2ab9-5e35-9150-dfdd74e899eb@oracle.com> Message-ID: <044d8bc3-d359-6ca6-2e38-91a158ec1203@oracle.com> On 8/6/20 11:01 PM, David Holmes wrote: > On 7/08/2020 2:41 pm, Ioi Lam wrote: >> On 8/6/20 5:58 PM, David Holmes wrote: >>> Correction ... >>> >>> On 7/08/2020 7:52 am, David Holmes wrote: >>>> Hi Ioi, >>>> >>>> On 7/08/2020 4:25 am, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8251209 >>>>> http://cr.openjdk.java.net/~iklam/jdk16/8251209-cds-jvmti-tests-modules-tag.v01/ >>>>> >>>>> >>>>> Summary -- changed the tests from (mis)using >>>>> >>>>> ??* @requires vm.flavor != "minimal" >>>>> >>>>> to >>>>> >>>>> ??* @modules java.instrument >>>>> >>>>> ... to be consistent with other jvmti tests. >>>> >>>> That seems like an invalid precondition to me. It would have been >>>> somewhat valid in the Compact Profiles world when we did not >>>> provide "java.instrument" in the profiles which supported >>>> MinimalVM, but you can define a minimal VM in a build that still >>>> has all modules available. I don't think building the minimal VM >>>> makes any changes to the supported modules. >>>> >>>> Also AIUI the @modules statement simply adds the necessary >>>> command-line args to use the java.instrument module (if present), >>>> it doesn't ensure that the listed module has to be present. >>> >>> It does in fact ensure that: >>> >>> "Otherwise, a test will not be run if the system being tested does >>> not contain all of the specified modules." >>> >>> http://openjdk.java.net/jtreg/tag-spec.html >>> >>> But as I said the module could be present in a JRE but you are still >>> using the MinimalVM. >>> >> >> Hi David, >> >> As I mentioned above, I am following the same rule as other jvmti >> tests, which only use "@modules java.instrument" and do not check >> whether the VM is minimal. E.g., >> >> http://hg.openjdk.java.net/jdk/jdk/file/4d36e29a5410/test/hotspot/jtreg/serviceability/jvmti/GetObjectSizeClass.java > > > Sure but I contend those tests are wrong and the tests you are > changing are right (or more right given common test configurations). > >> >> ------- >> >> If I understand correctly, you're saying someone can build a minimal >> JDK (configure --with-jvm-variants=minimal), and then try to add the >> java.instrument module to it. I.e., adding the following module to >> your JDK (with jlink, or by hand). > > Just build a JDK with multiple VMs present. > >> >> $ unzip -l ./jmods/java.instrument.jmod >> ?? Length????? Date??? Time??? Name >> ---------? ---------- -----?? ---- >> ?????? 294? 2020-08-04 17:03?? classes/module-info.class >> ????? 1102? 2020-08-04 17:03 >> classes/sun/instrument/TransformerManager$TransformerInfo.class >> ????? 4294? 2020-08-04 17:03 >> classes/sun/instrument/TransformerManager.class >> ?????? 911? 2020-08-04 17:03 >> classes/sun/instrument/InstrumentationImpl$1.class >> ???? 16663? 2020-08-04 17:03 >> classes/sun/instrument/InstrumentationImpl.class >> ????? 1356? 2020-08-04 17:03 >> classes/java/lang/instrument/ClassFileTransformer.class >> ?????? 554? 2020-08-04 17:03 >> classes/java/lang/instrument/IllegalClassFormatException.class >> ????? 1734? 2020-08-04 17:03 >> classes/java/lang/instrument/Instrumentation.class >> ?????? 563? 2020-08-04 17:03 >> classes/java/lang/instrument/UnmodifiableModuleException.class >> ?????? 970? 2020-08-04 17:03 >> classes/java/lang/instrument/ClassDefinition.class >> ?????? 551? 2020-08-04 17:03 >> classes/java/lang/instrument/UnmodifiableClassException.class >> ????? 3244? 2020-08-04 17:03?? legal/COPYRIGHT >> ??????? 44? 2020-08-04 17:03?? legal/LICENSE >> ???? 50920? 2020-08-04 17:03 lib/libinstrument.so<<<<<<<<< >> >> But this module has a native library, libinstrument.so, which >> requires JVMTI to be present in libjvm.so. E.g.: >> >> ???? jvmtiEnv * >> ???? retransformableEnvironment(JPLISAgent * agent) { >> ???? .... >> ???????? jnierror = (*agent->mJVM)->GetEnv( agent->mJVM, >> ???????? ? ?? ????????????????????? (void **) &retransformerEnv, >> ???????????? ? ?? ????????????????? JVMTI_VERSION_1_1); >> >> So if you try to run the CDS JVMTI test cases, it will be executed >> (because your JDK says "I have java.instrument") and the test finds >> out that your JDK's java.instrument module isn't working properly. So >> the test is doing exactly what it's supposed to do. > > The whole point of the @requires is to not waste time and resources > running a test on a platform that cannot run the test successfully. > > So the fully correct solution could be to have both settings: > > @requires vm.flavor != "minimal" > @modules java.instrument > > if you require both a VM that supports JVM TI and you need a JRE that > includes the java.instrument module. But that assumes your test does > need java.instrument. Not all JVM TI tests need java.instrument, but > all instrumentation tests depend on JVM TI. Just looking at the first > three of tests in your webrev I don't see any dependency on > java.instrument - they are CDS only tests as far as I can see and so > require a VM with CDS which means not a Minimal VM - though perhaps it > is sufficient to have the > > @requires vm.cds > > in those cases? > > For the other JVM TI related tests using -javaagent they probably need > both @requires and @module. You can disable JVMTI with "configure --disable-jvm-feature-jvmti". So checking for vm.flavor != "minimal" is not sufficient. Similarly, "@requires vm.cds" doesn't guarantee that JVMTI is supported. Maybe we should have a new VM prop so we can do @requires vm.jvmti @modules java.instrument Thanks - Ioi > > David > ----- > >> I would argue that this is better than before (which would exclude >> the test when the libjvm.so is a minimal build, and would will not >> detect such a mis-configured java.instrument module.) >> >> >> Thanks >> - Ioi >> >> From felix.yang at huawei.com Fri Aug 7 06:20:25 2020 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Fri, 7 Aug 2020 06:20:25 +0000 Subject: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic In-Reply-To: <7cedaced-9628-32b3-6d01-beae760fc964@redhat.com> References: <654e8a2e-87a1-9c3d-4c58-36b4215bb0ad@redhat.com> <5350e745-87af-ce34-8711-2b652a38e323@redhat.com> <4d1f7179-2cad-620d-f833-949e8b39ee06@redhat.com> <7cedaced-9628-32b3-6d01-beae760fc964@redhat.com> Message-ID: Hi, > -----Original Message----- > From: Andrew Haley [mailto:aph at redhat.com] > Sent: Thursday, August 6, 2020 7:42 PM > To: Yangfei (Felix) ; hotspot-runtime- > dev at openjdk.java.net > Cc: aarch64-port-dev at openjdk.java.net > Subject: Re: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic > > On 8/6/20 2:13 AM, Yangfei (Felix) wrote: > > I have added tests for sha512 crypto instructions in this script and > regenerated the code. > > Tier1-3 tested with an aarch64 fastdebug build on my aarch64 linux host to > cover this part. > > Updated webrev: http://cr.openjdk.java.net/~fyang/8165404/webrev.02/ > > Is it OK? > > Yes, thanks. > > It's a bit annoying that the indentation seems to change so frequently. I'm > wondering if perhaps some people have been running this on Windows, > which has different tab sizes, but that seems very unlikely, given that this is a > Linux port. But never mind, that's not your problem. I see the output of the python script contains tabs which is not allowed by jcheck. So I guess it may depends on the way in which people substitute these tabs. I used vim editor to do the replacement, which is simple. Open the file which contains the script output and do: :set ts=8 :set expandtab :%retab! Looks like the indentation is more consistent this way. Hope that helps. Pushed as: https://hg.openjdk.java.net/jdk/jdk/rev/09ad5b67a099 Thanks, Felix From david.holmes at oracle.com Fri Aug 7 06:41:58 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 7 Aug 2020 16:41:58 +1000 Subject: RFR (xs) 8251209 [TESTBUG] CDS jvmti tests should use "@modules java.instrument" In-Reply-To: <044d8bc3-d359-6ca6-2e38-91a158ec1203@oracle.com> References: <2b9000ef-9a09-caf4-8610-bb354afb722b@oracle.com> <138c579d-2ab9-5e35-9150-dfdd74e899eb@oracle.com> <044d8bc3-d359-6ca6-2e38-91a158ec1203@oracle.com> Message-ID: <76bb64fd-bb2d-40ed-7624-b5fe8adce3c6@oracle.com> On 7/08/2020 4:13 pm, Ioi Lam wrote: > On 8/6/20 11:01 PM, David Holmes wrote: >> On 7/08/2020 2:41 pm, Ioi Lam wrote: >>> On 8/6/20 5:58 PM, David Holmes wrote: >>>> Correction ... >>>> >>>> On 7/08/2020 7:52 am, David Holmes wrote: >>>>> Hi Ioi, >>>>> >>>>> On 7/08/2020 4:25 am, Ioi Lam wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8251209 >>>>>> http://cr.openjdk.java.net/~iklam/jdk16/8251209-cds-jvmti-tests-modules-tag.v01/ >>>>>> >>>>>> >>>>>> Summary -- changed the tests from (mis)using >>>>>> >>>>>> ??* @requires vm.flavor != "minimal" >>>>>> >>>>>> to >>>>>> >>>>>> ??* @modules java.instrument >>>>>> >>>>>> ... to be consistent with other jvmti tests. >>>>> >>>>> That seems like an invalid precondition to me. It would have been >>>>> somewhat valid in the Compact Profiles world when we did not >>>>> provide "java.instrument" in the profiles which supported >>>>> MinimalVM, but you can define a minimal VM in a build that still >>>>> has all modules available. I don't think building the minimal VM >>>>> makes any changes to the supported modules. >>>>> >>>>> Also AIUI the @modules statement simply adds the necessary >>>>> command-line args to use the java.instrument module (if present), >>>>> it doesn't ensure that the listed module has to be present. >>>> >>>> It does in fact ensure that: >>>> >>>> "Otherwise, a test will not be run if the system being tested does >>>> not contain all of the specified modules." >>>> >>>> http://openjdk.java.net/jtreg/tag-spec.html >>>> >>>> But as I said the module could be present in a JRE but you are still >>>> using the MinimalVM. >>>> >>> >>> Hi David, >>> >>> As I mentioned above, I am following the same rule as other jvmti >>> tests, which only use "@modules java.instrument" and do not check >>> whether the VM is minimal. E.g., >>> >>> http://hg.openjdk.java.net/jdk/jdk/file/4d36e29a5410/test/hotspot/jtreg/serviceability/jvmti/GetObjectSizeClass.java >> >> >> >> Sure but I contend those tests are wrong and the tests you are >> changing are right (or more right given common test configurations). >> >>> >>> ------- >>> >>> If I understand correctly, you're saying someone can build a minimal >>> JDK (configure --with-jvm-variants=minimal), and then try to add the >>> java.instrument module to it. I.e., adding the following module to >>> your JDK (with jlink, or by hand). >> >> Just build a JDK with multiple VMs present. >> >>> >>> $ unzip -l ./jmods/java.instrument.jmod >>> ?? Length????? Date??? Time??? Name >>> ---------? ---------- -----?? ---- >>> ?????? 294? 2020-08-04 17:03?? classes/module-info.class >>> ????? 1102? 2020-08-04 17:03 >>> classes/sun/instrument/TransformerManager$TransformerInfo.class >>> ????? 4294? 2020-08-04 17:03 >>> classes/sun/instrument/TransformerManager.class >>> ?????? 911? 2020-08-04 17:03 >>> classes/sun/instrument/InstrumentationImpl$1.class >>> ???? 16663? 2020-08-04 17:03 >>> classes/sun/instrument/InstrumentationImpl.class >>> ????? 1356? 2020-08-04 17:03 >>> classes/java/lang/instrument/ClassFileTransformer.class >>> ?????? 554? 2020-08-04 17:03 >>> classes/java/lang/instrument/IllegalClassFormatException.class >>> ????? 1734? 2020-08-04 17:03 >>> classes/java/lang/instrument/Instrumentation.class >>> ?????? 563? 2020-08-04 17:03 >>> classes/java/lang/instrument/UnmodifiableModuleException.class >>> ?????? 970? 2020-08-04 17:03 >>> classes/java/lang/instrument/ClassDefinition.class >>> ?????? 551? 2020-08-04 17:03 >>> classes/java/lang/instrument/UnmodifiableClassException.class >>> ????? 3244? 2020-08-04 17:03?? legal/COPYRIGHT >>> ??????? 44? 2020-08-04 17:03?? legal/LICENSE >>> ???? 50920? 2020-08-04 17:03 lib/libinstrument.so<<<<<<<<< >>> >>> But this module has a native library, libinstrument.so, which >>> requires JVMTI to be present in libjvm.so. E.g.: >>> >>> ???? jvmtiEnv * >>> ???? retransformableEnvironment(JPLISAgent * agent) { >>> ???? .... >>> ???????? jnierror = (*agent->mJVM)->GetEnv( agent->mJVM, >>> ???????? ? ?? ????????????????????? (void **) &retransformerEnv, >>> ???????????? ? ?? ????????????????? JVMTI_VERSION_1_1); >>> >>> So if you try to run the CDS JVMTI test cases, it will be executed >>> (because your JDK says "I have java.instrument") and the test finds >>> out that your JDK's java.instrument module isn't working properly. So >>> the test is doing exactly what it's supposed to do. >> >> The whole point of the @requires is to not waste time and resources >> running a test on a platform that cannot run the test successfully. >> >> So the fully correct solution could be to have both settings: >> >> @requires vm.flavor != "minimal" >> @modules java.instrument >> >> if you require both a VM that supports JVM TI and you need a JRE that >> includes the java.instrument module. But that assumes your test does >> need java.instrument. Not all JVM TI tests need java.instrument, but >> all instrumentation tests depend on JVM TI. Just looking at the first >> three of tests in your webrev I don't see any dependency on >> java.instrument - they are CDS only tests as far as I can see and so >> require a VM with CDS which means not a Minimal VM - though perhaps it >> is sufficient to have the >> >> @requires vm.cds >> >> in those cases? >> >> For the other JVM TI related tests using -javaagent they probably need >> both @requires and @module. > > You can disable JVMTI with "configure --disable-jvm-feature-jvmti". So > checking for vm.flavor != "minimal" is not sufficient. Not it isn't but we don't try to accommodate every possible VM feature and module that someone could create a JRE with. Otherwise we will need an @require capability for every selectable feature and we would need to update every test to explicitly list every dependency. But we do build a minimal VM and people can test the minimal VM, and if you build the minimal VM as part a multi-VM JRE then you do have java.instrument present. So only testing for the module, instead of testing for a non-minimal VM is not sufficient and changing tests to do that is a step in the wrong direction IMO. Cheers, David > Similarly, "@requires vm.cds" doesn't guarantee that JVMTI is supported. > > Maybe we should have a new VM prop so we can do > > @requires vm.jvmti > @modules java.instrument > > Thanks > - Ioi > > > >> >> David >> ----- >> >>> I would argue that this is better than before (which would exclude >>> the test when the libjvm.so is a minimal build, and would will not >>> detect such a mis-configured java.instrument module.) >>> >>> >>> Thanks >>> - Ioi >>> >>> > From coleen.phillimore at oracle.com Fri Aug 7 13:53:51 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 7 Aug 2020 09:53:51 -0400 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> Message-ID: <2b030b14-0dc2-2de6-f01b-cc4f9ae80149@oracle.com> On 8/6/20 9:45 PM, Patricio Chilano wrote: > Hi Coleen, > > Thanks for looking at this. > > On 8/6/20 7:17 PM, Coleen Phillimore wrote: >> >> Patricio,? One question: >> >> http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/src/hotspot/share/runtime/synchronizer.cpp.udiff.html >> >> >> + // adjust bcp to point back to monitorenter so that we print the >> correct line numbers >> >> >> How do the fatal and event print the correct line numbers here? I see >> the logging gets it from the stack trace.? Should the abort message >> have more information in it?? You can get the source and line number >> information in the same way that print_stack_on() gets it. > For the fatal error case I'm not printing line numbers as I do with > the warning case or as you get with JFR events. But maybe I should > print the stack too and then exit the VM. You can get the stack info > and the line number of the monitorenter bytecode that caused the crash > from the generated hs_err file though. Can you send a sample, please?? I guess there isn't a handy function to show the source file name and line number in the fatal error message. Thanks, Coleen > > I don't know which exact technique JFR uses to print line numbers but > it has to include reading the current bcp. Since when coming from the > interpreter for monitorenter the bcp is always already pointing to the > next instruction we need to decrement it to print the correct line > numbers. I tested it and if I don't fix the bcp, as expected JFR too > will print the next line number for the top frame. > > > Patricio >> Otherwise, this looks good to me. >> Coleen >> >> >> On 8/6/20 2:48 PM, Patricio Chilano wrote: >>> Hi Dan, >>> >>> On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: >>>> I'm peeking ahead to the next webrev... :-) >>>> >>>> > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ >>> : ) >>> >>>> General comments: >>>> ? - check files for copyright year updates. >>> Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, >>> c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. >>> >>>> src/hotspot/share/runtime/synchronizer.hpp >>>> ??? No comments. >>>> >>>> src/hotspot/share/runtime/synchronizer.cpp >>>> ??? L507: ? const markWord mark = obj->mark(); >>>> ??? L508: >>>> ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && >>>> obj->klass()->is_box()) { >>>> ??? L510: ??? return false; >>>> ??? L511: ? } >>>> ??????? The new bailout on L509-511 can move above L507. >>> Moved. >>> >>>> L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass >>>> %s", p2i(obj()), obj->klass()->external_name()); >>>> ??????? This external_name() call does not have a ResourceMark. >>> Good catch! I had one in a previous version but then I changed the >>> conditionals and lost it for the fatal error case. The test worked >>> okay because for the main JavaThread there is a ResourceMark in >>> jni_invoke_static() (jni.cpp). >>> >>>> src/hotspot/share/logging/logTag.hpp >>>> ??? No comments. >>>> >>>> src/hotspot/share/oops/klass.hpp >>>> ??? No comments. >>>> >>>> src/hotspot/share/utilities/accessFlags.hpp >>>> ??? No comments. >>>> >>>> src/hotspot/share/runtime/globals.hpp >>>> ??? L814: ???????????? "0: off " >>>> ??????? Missing a ';' after "off". >>> Fixed. >>> >>>> L816: ???????????? "2: log message to stdout. >>>> ??????? Perhaps add "(by default)" after "stdout" or >>>> ??????? don't say where log output is at all. >>>> >>>> src/hotspot/share/runtime/arguments.cpp >>>> ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, true, >>>> LOG_TAGS(primitivewrappers)); >>>> ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp >>>> ??????? since it looks like logging output is configured to 'stdout'. >>>> ??????? I'm assuming that other log options to put output elsewhere >>>> ??????? are overridden by this code? >>> Right. So the logging is done under UL with the tag >>> primitivewrappers. If that tag is specified in Xlog then this >>> conditional will be skipped because !log_is_enabled(Info, >>> primitivewrappers) will be false. >>> >>>> src/hotspot/share/classfile/systemDictionary.cpp >>>> ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { >>>> ??????? nit - s/T_LONG+1/T_LONG + 1/ >>> Fixed. >>> >>>> L2191: _box_klasses[i]->set_prototype_header(markWord::prototype()); >>>> ??????? I assume we're keeping the prototype_header field when >>>> Biased Locking >>>> ??????? goes away? The reason I ask: >>>> >>>> ? ? ? ? static markWord prototype() { >>>> ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); >>>> ? ? ? ? } >>>> >>>> ??????? is because without Biased Locking, do we really need the >>>> prototype >>>> ??????? anymore? The initial markWord won't need possible variants... >>> Yes, I think it can go away unless somebody finds another use for >>> it. Maybe Valhalla. >>> >>>> src/hotspot/share/jfr/metadata/metadata.xml >>>> ??? L69: ? >>> ??????? Is the category "Java Application" because it's the >>>> application >>>> ??????? code that did something "wrong" here? Where "application" >>>> is loosely >>>> ??????? defined to include the possibility of auto generated code, >>>> library >>>> ??????? code and the like? Or perhaps "application" because >>>> something "above" >>>> ??????? the "Java Virtual Machine, Runtime" did the "wrong" thing >>>> here? >>> I don't know what the right category should be really. I saw the >>> events JavaMonitorEnter, JavaMonitorWait and JavaMonitorInflate and >>> thought this event should fall in the same category they do. >>> >>>> src/jdk.jfr/share/conf/jfr/default.jfc >>>> ??? No comments. >>>> >>>> src/jdk.jfr/share/conf/jfr/profile.jfc >>>> ??? No comments. >>>> >>>> test/lib/jdk/test/lib/jfr/EventNames.java >>>> ??? No comments. >>>> >>>> src/hotspot/cpu/aarch64/aarch64.ad >>>> ??? L3517: ????? __ tbnz(tmp, exact_log2(JVM_ACC_IS_BOX_CLASS), cont); >>>> >>>> ??? L3578: ??? __ bind(cont); >>>> ??? L3579: ??? // flag == EQ indicates success >>>> ??? L3580: ??? // flag == NE indicates failure >>>> ??????? If tbnz() branches to "cont" when we have a box class, what's >>>> ??????? the flag value set to (EQ or NE)? And what set that flag >>>> value? >>>> ??????? The reason I ask is I don't think tbnz() sets condition >>>> codes... >>> Right, it doesn't set condition codes, so I kept the version I had >>> (more on that below). >>> >>>> src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>> ??? No comments. >>>> >>>> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp >>>> ??? No comments. >>>> >>>> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp >>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>> ??????????? ? ?? ??????????????????????????? Register obj_reg, >>>> ?????????????????????????????? ? ?? ???????? Register swap_reg, >>>> ?????????????????????????????????? ? ?? ???? Register tmp_reg, >>>> ?????????????????????????????????? ? ?? ???? bool >>>> swap_reg_contains_mark, >>>> ?????????????????????????????????? ? ?? ???? Label& done, >>>> ??????????????????????????????????? ? ?? ??? Label* slow_case, >>>> BiasedLockingCounters* counters) { >>>> ??????? I think you've changed the only callers of >>>> biased_locking_enter() >>>> ??????? that cared about the return value with this changeset so it >>>> can >>>> ??????? be changed to a void function. >>> Ok, this is what I mentioned to David in a previous email. Done. >>> >>>> src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp >>>> ??? No comments. >>>> >>>> src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp >>>> ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), done); >>>> >>>> ??? L131: ? bind(done); >>>> ??? L132: >>>> ??? L133: ? // At this point flags are set as follows: >>>> ??? L134: ? //? EQ -> Success >>>> ??? L135: ? //? NE -> Failure, branch to slow path >>>> ??????? If tbnz() branches to "done" when we have a box class, what's >>>> ??????? the flag value set to (EQ or NE)? And what set that flag >>>> value? >>>> ??????? The reason I ask is I don't think tbnz() sets condition >>>> codes... >>> Right. Same as above, I kept the version I had. >>> >>>> src/hotspot/cpu/arm/interp_masm_arm.cpp >>>> ??? No comments. >>>> >>>> src/hotspot/cpu/arm/macroAssembler_arm.cpp >>>> ??? int MacroAssembler::biased_locking_enter(Register obj_reg, >>>> Register swap_reg, Register tmp_reg, >>>> ????????????????????? ? ?? ????????????????? bool >>>> swap_reg_contains_mark, >>>> ???????????????????????? ? ?? ?????????????? Register tmp2, >>>> ????????????????????????? ? ?? ????????????? Label& done, Label& >>>> slow_case, >>>> BiasedLockingCounters* counters) { >>>> ??????? I think you've changed the only callers of >>>> biased_locking_enter() >>>> ??????? that cared about the return value with this changeset so it >>>> can >>>> ??????? be changed to a void function. >>> Done. >>> >>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>> ??? No comments on the PPC code. >>>> >>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>> src/hotspot/cpu/s390/interp_masm_s390.cpp >>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>> ??? No comments on the S390 code. >>>> >>>> src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp >>>> ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); >>>> ??????? What will this do with a 'noreg' value? (I need a refresher.) >>> When not in 64 bit mode that register just won't be used. >>> >>>> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp >>>> ??? No comments. >>>> >>>> src/hotspot/cpu/x86/interp_masm_x86.cpp >>>> ??? No comments. >>>> >>>> src/hotspot/cpu/x86/macroAssembler_x86.cpp >>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>> ????????????????????????????????? ? ?? ????? Register obj_reg, >>>> ????????????????????????????????? ? ?? ????? Register swap_reg, >>>> ????????????????????????????????? ? ?? ????? Register tmp_reg, >>>> ?????????????????????????????????? ? ?? ???? Register tmp_reg2, >>>> ?????????????????????????????????????? ? ? ? bool >>>> swap_reg_contains_mark, >>>> ?????????????????????????????????? ? ?? ???? Label& done, >>>> ?????????????????????????????????? ? ?? ???? Label* slow_case, >>>> BiasedLockingCounters* counters) { >>>> ??????? I think you've changed the only caller of >>>> biased_locking_enter() >>>> ??????? that cared about the return value with this changeset so it >>>> can >>>> ??????? be changed to a void function. >>> Done. >>> >>>> test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >>>> ??? L30:? * @test SyncOnPrimitiveWrapperTest >>>> ??????? No parameter to @test directive. >>> Removed. >>> >>>> L136: ??????? private static long sharedCounter = 0L; >>>> ??????? Since you don't do anything with sharedCounter other than >>>> increment it, >>>> ??????? can the compilers optimize it away? If it can be optimized >>>> away, does >>>> ??????? that mean that: >>>> >>>> ??????????? L142: ??????????????? synchronized (obj) { >>>> >>>> ??????? can also be optimized away? >>>> >>>> ??????? I don't think that: >>>> >>>> ??????????? L161: ??????????????? synchronized (sharedLock1) { >>>> >>>> ??????? can be optimized away because it is shared by multiple >>>> threads. >>>> >>>> test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java >>>> ??? Similar questions about 'counter' being optimized away. >>>> ??? Similar question about "synchronized (obj) {" being optimized >>>> away. >>> I'm not sure if the compiler will optimize it away. Seems unlikely >>> given the counter we are incrementing is not just local to some thread. >>> >>> >>> Ok, below is v3 which has the following changes: >>> - Use a 32 bit load for the _access_flags field, instead of 64 >>> - Martin's implementation for s390 and fix for PPC >>> - Faster LogTest version >>> - Above changes based on Dan review >>> >>> I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 only) >>> again with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I checked it >>> builds in arm32, ppc and s390. >>> >>> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev >>> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev >>> >>> >>> @Martin: >>> Please check if the test doesn't timeout for you now with the >>> changes I made to LogTest. >>> Also, I tried to use tbnz in aarch64 and arm32 instead of tst + br >>> (except for c2 since we actually need to set the condition flags), >>> but for c1 I was getting an assertion in the compiler thread: >>> >>> guarantee(chk == -1 || chk == 0) failed: Field too big for insn >>> >>> This is the stack when that happens: >>> >>> MacroAssembler::pd_patch_instruction_size(unsigned char*, unsigned >>> char*)+0x398 >>> AbstractAssembler::bind(Label&)+0x118 >>> MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 >>> LIR_Assembler::emit_slow_case_stubs()+0x54 >>> Compilation::emit_code_body()+0x17c >>> >>> The change was simply: >>> >>> diff --git a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>> b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>> --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>> +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>> @@ -78,7 +78,6 @@ >>> ?? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>> ???? load_klass(hdr, obj); >>> -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); >>> -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); >>> -??? br(Assembler::NE, slow_case); >>> +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); >>> +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); >>> ?? } >>> >>> So the issue must be related to where we want to jump. >>> >>> Thanks, >>> Patricio >>>> Dan >>>> >>>> >>>> On 8/5/20 9:01 AM, Patricio Chilano wrote: >>>>> Hi Martin, >>>>> >>>>> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>>>>> Hi Patricio, >>>>>> >>>>>> using 8 Byte load instructions for jint fields is a terrible >>>>>> coding style! >>>>>> Someone else may see it and use an 8 Byte store. Will result in >>>>>> great fun for debugging! >>>>>> >>>>>> And on Big Endian you will simply access the wrong bits. >>>>> Ah, of course! Those 32 bits will be in the wrong place when doing >>>>> the test. >>>>> >>>>>> Note that Big Endian Platforms are AIX, old linux ppc, s390, >>>>>> SPARC. I don't think you have tested on them. >>>>>> >>>>>>> We could remove the nested synchronized statements in the run() >>>>>>> method >>>>>>> so that we don't generate that much logging. We could also lower >>>>>>> LOOP_COUNT. I think the issue is also because we are running >>>>>>> LogTest >>>>>>> with multiple flag combinations. Not sure what we should touch >>>>>>> first. >>>>>>> Maybe the synchronized statements, have only one or two and test >>>>>>> that >>>>>>> first? >>>>>> Sounds like helpful ideas. Please go ahead and strip things down. >>>>> Great, I will send v3 later with those changes. >>>>> >>>>> Thanks Martin! >>>>> >>>>> Patricio >>>>>> Thanks for taking care of it. >>>>>> >>>>>> Best regards, >>>>>> Martin >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Patricio Chilano >>>>>>> Sent: Dienstag, 4. August 2020 20:26 >>>>>>> To: Doerr, Martin ; David Holmes >>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>>> wrappers >>>>>>> >>>>>>> Hi Martin, >>>>>>> >>>>>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>>>>> Hi Patricio, >>>>>>>> >>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>> sending v3. >>>>>>>> Thanks. I forgot to mention arm + aarch64. >>>>>>>> Aarch64 uses ldrw + tbnz. >>>>>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>>>>> >>>>>>>> I remember having seen the same mistake ?? >>>>>>>> And nobody noticed it on little Endian platforms. >>>>>>> Ok, I can use a tbnz instead of test and then a branch on arm32 >>>>>>> and aarch64. >>>>>>> Shouldn't a normal ldr in arm32 work fine? >>>>>>> Also in 64 bits (either x64 or aarch64) I don't see the issue of >>>>>>> using a >>>>>>> 64 bit load, besides the fact that we only care about the first >>>>>>> 32 bits. >>>>>>> Regardless of the endianness, aren't we masking out the upper >>>>>>> part when >>>>>>> we do AND/TEST or if we test a bit in the 0-31 bit range? >>>>>>> Otherwise it >>>>>>> seems it should have failed for one of those platforms in my tests. >>>>>>> >>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>>> works and I >>>>>>>>> can add it to the test. >>>>>>>> That seems to have an effect. But now I'm not patient enough to >>>>>>>> wait for >>>>>>> the test to finish. >>>>>>>> Maybe the problem is that I'm using slow debug builds. >>>>>>>> But is there a chance to make the test quicker without losing >>>>>>>> coverage >>>>>>> significantly? >>>>>>>> I think the test is too slow this way to run it on a regular >>>>>>>> basis. We'd need to >>>>>>> spend dedicated machines for it. >>>>>>> We could remove the nested synchronized statements in the run() >>>>>>> method >>>>>>> so that we don't generate that much logging. We could also lower >>>>>>> LOOP_COUNT. I think the issue is also because we are running >>>>>>> LogTest >>>>>>> with multiple flag combinations. Not sure what we should touch >>>>>>> first. >>>>>>> Maybe the synchronized statements, have only one or two and test >>>>>>> that >>>>>>> first? >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Patricio >>>>>>>> Best regards, >>>>>>>> Martin >>>>>>>> >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Patricio Chilano >>>>>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on >>>>>>>>> primitive wrappers >>>>>>>>> >>>>>>>>> Hi Martin, >>>>>>>>> >>>>>>>>> Thanks for fixing PPC and taking care of s390! >>>>>>>>> >>>>>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>>>>> Hi Patricio, >>>>>>>>>> >>>>>>>>>> I suggest to use movl + testl for checking the access flag as >>>>>>>>>> for other >>>>>>> access >>>>>>>>> flags on x64. >>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>> sending v3. >>>>>>>>> >>>>>>>>>> New version for PPC64 and s390 see below. >>>>>>>>>> >>>>>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>>>>>> expected. >>>>>>>>> However, I'm getting timeout issues: >>>>>>>>>> timed out (timeout set to 120000ms, elapsed time including >>>>>>>>>> timeout >>>>>>>>> handling was 122372ms) >>>>>>>>>> Can we provide more time? >>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>>> works and I >>>>>>>>> can add it to the test. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Patricio >>>>>>>>>> Best regards, >>>>>>>>>> Martin >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> diff -r 77852e129401 >>>>>>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>> @@ -107,8 +107,8 @@ >>>>>>>>>> >>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>>> Rscratch); >>>>>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>> +??? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>>> Rscratch); >>>>>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>> ??????? bne(CCR0, slow_case); >>>>>>>>>> ????? } >>>>>>>>>> >>>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>> @@ -912,8 +912,8 @@ >>>>>>>>>> >>>>>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>> ????????? load_klass(tmp, object); >>>>>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>>> +????? testbitdi(CCR0, R0, tmp, >>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>> ????????? bne(CCR0, slow_case); >>>>>>>>>> ??????? } >>>>>>>>>> >>>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>>>>> >>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>> ??????? load_klass(temp, oop); >>>>>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>>> +??? testbitdi(CCR0, R0, temp, >>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>> ??????? bne(CCR0, cont); >>>>>>>>>> ????? } >>>>>>>>>> >>>>>>>>>> diff -r 77852e129401 >>>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>> @@ -91,6 +91,12 @@ >>>>>>>>>> ????? // Save object being locked into the BasicObjectLock... >>>>>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>>>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>> +??? z_btrue(slow_case); >>>>>>>>>> +? } >>>>>>>>>> + >>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>>>>> Z_R0_scratch, done, >>>>>>>>> &slow_case); >>>>>>>>>> ????? } >>>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>>>> 10:03:57 >>>>>>>>> 2020 +0200 >>>>>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>> 16:04:31 >>>>>>>>> 2020 +0200 >>>>>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>>>>> ????? z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), >>>>>>>>>> object); >>>>>>>>>> >>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>> +??? z_btrue(slow_case); >>>>>>>>>> +? } >>>>>>>>>> + >>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>> ??????? biased_locking_enter(object, displaced_header, Z_R1, >>>>>>>>>> Z_R0, done, >>>>>>>>> &slow_case); >>>>>>>>>> ????? } >>>>>>>>>> diff -r 77852e129401 >>>>>>>>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>>>>> ????? // Load markWord from oop into mark. >>>>>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>>>>> >>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>> +??? z_btrue(done); >>>>>>>>>> +? } >>>>>>>>>> + >>>>>>>>>> ????? if (try_bias) { >>>>>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, >>>>>>>>>> Z_R0, done); >>>>>>>>>> ????? } >>>>>>>>>> >>>>> >>>> >>> >> > From patricio.chilano.mateo at oracle.com Fri Aug 7 15:29:14 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Fri, 7 Aug 2020 12:29:14 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> Message-ID: Hi Martin, On 8/7/20 12:58 AM, Doerr, Martin wrote: > Hi Patricio, > > thanks for the update. > > Sorry for causing trouble related to the tbnz instructions on aarch64. > My concern was basically to use the correct number of Bytes for the load instructions. > You have fixed that in webrev v3. Thanks. > > Unfortunately, I did the same mistake regarding the flag requirements on PPC64 and s390. And we have a wrong label on PPC64 (otherwise branch destination can be too far). > Test duration is still terrible with the slow debug build. I still got timeouts. (Maybe I should better use fast debug builds.) I measured the time in my Mac (fastdebug build). The FatalTest takes ~ less than a second to run, and we have 7*8=56 of those. The LogTest takes between 1-4 seconds depending on the flag, and we have 7 of those. Overall it takes 2-3 minutes to run. Maybe you can measure those times too to see where its taking so long? > Fixes see below. Great, added the new fixes. Here is v4: Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v4/webrev/ Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v4/inc/webrev/ > Sometimes simple changes can require quite some time ... Yep, no problem? : ) Thanks! Patricio > Best regards, > Martin > > > diff -r f629202b3862 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp > --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Thu Aug 06 22:03:39 2020 +0200 > +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Thu Aug 06 23:26:02 2020 +0200 > @@ -109,7 +109,7 @@ > load_klass(Rscratch, Roop); > lwz(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); > testbitdi(CCR0, R0, Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS)); > - bne(CCR0, slow_case); > + bne(CCR0, slow_int); > } > > if (UseBiasedLocking) { > diff -r f629202b3862 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp > --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Thu Aug 06 22:03:39 2020 +0200 > +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Thu Aug 06 23:26:02 2020 +0200 > @@ -2839,8 +2839,8 @@ > if (DiagnoseSyncOnPrimitiveWrappers != 0) { > load_klass(temp, oop); > lwz(temp, in_bytes(Klass::access_flags_offset()), temp); > - testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); > - bne(CCR0, cont); > + testbitdi(flag, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); > + bne(flag, cont); > } > > if (try_bias) { > diff -r f629202b3862 src/hotspot/cpu/s390/macroAssembler_s390.cpp > --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Thu Aug 06 22:03:39 2020 +0200 > +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Thu Aug 06 23:26:02 2020 +0200 > @@ -3360,8 +3360,10 @@ > > if (DiagnoseSyncOnPrimitiveWrappers != 0) { > load_klass(Z_R1_scratch, oop); > - testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), exact_log2(JVM_ACC_IS_BOX_CLASS)); > - z_btrue(done); > + z_l(Z_R1_scratch, Address(Z_R1_scratch, Klass::access_flags_offset())); > + assert((JVM_ACC_IS_BOX_CLASS & 0xFFFF) == 0, "or change following instruction"); > + z_nilh(Z_R1_scratch, JVM_ACC_IS_BOX_CLASS >> 16); > + z_brne(done); > } > > if (try_bias) { > diff -r f629202b3862 test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java > --- a/test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java Thu Aug 06 22:03:39 2020 +0200 > +++ b/test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java Thu Aug 06 23:26:02 2020 +0200 > @@ -31,7 +31,7 @@ > * @bug 8242263 > * @summary Exercise DiagnoseSyncOnPrimitiveWrappers diagnostic flag > * @library /test/lib > - * @run driver SyncOnPrimitiveWrapperTest > + * @run driver/timeout=180000 SyncOnPrimitiveWrapperTest > */ > > public class SyncOnPrimitiveWrapperTest { > From patricio.chilano.mateo at oracle.com Fri Aug 7 15:49:30 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Fri, 7 Aug 2020 12:49:30 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <2b030b14-0dc2-2de6-f01b-cc4f9ae80149@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> <2b030b14-0dc2-2de6-f01b-cc4f9ae80149@oracle.com> Message-ID: <41fd3f22-4266-de46-8d32-47ba6295218b@oracle.com> Hi Coleen, On 8/7/20 10:53 AM, Coleen Phillimore wrote: > > > On 8/6/20 9:45 PM, Patricio Chilano wrote: >> Hi Coleen, >> >> Thanks for looking at this. >> >> On 8/6/20 7:17 PM, Coleen Phillimore wrote: >>> >>> Patricio,? One question: >>> >>> http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/src/hotspot/share/runtime/synchronizer.cpp.udiff.html >>> >>> >>> + // adjust bcp to point back to monitorenter so that we print the >>> correct line numbers >>> >>> >>> How do the fatal and event print the correct line numbers here? I >>> see the logging gets it from the stack trace.? Should the abort >>> message have more information in it?? You can get the source and >>> line number information in the same way that print_stack_on() gets it. >> For the fatal error case I'm not printing line numbers as I do with >> the warning case or as you get with JFR events. But maybe I should >> print the stack too and then exit the VM. You can get the stack info >> and the line number of the monitorenter bytecode that caused the >> crash from the generated hs_err file though. > > Can you send a sample, please? At the top of the hs_err you will get: ?# ?# A fatal error has been detected by the Java Runtime Environment: ?# ?#? Internal Error (/xx/xxx/xxxx/open/src/hotspot/share/runtime/synchronizer.cpp:574), pid=48810, tid=48811 ?#? fatal error: Synchronizing on object 0x00000007ff0758a8 of klass java.lang.Integer ?# And then on the stacktrace you can see: Stack: [0x00007f982b6ce000,0x00007f982b7cf000], sp=0x00007f982b7cd5c0,? free space=1021k Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) V? [libjvm.so+0x1662385] ObjectSynchronizer::handle_sync_on_primitive_wrapper(Handle, Thread*)+0x185 V? [libjvm.so+0x16683ac]? ObjectSynchronizer::enter(Handle, BasicLock*, Thread*)+0x21c V? [libjvm.so+0xce88fa] InterpreterRuntime::monitorenter(JavaThread*, BasicObjectLock*)+0x13a j? SimpleTest.main([Ljava/lang/String;)V+24 v? ~StubRoutines::call_stub V? [libjvm.so+0xd0263a]? JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x62a V? [libjvm.so+0xe22c8e]? jni_invoke_static(JNIEnv_*, JavaValue*, _jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, Thr ?.constprop.1]+0x35e V? [libjvm.so+0xe28e6b]? jni_CallStaticVoidMethod+0x21b C? [libjli.so+0x496e]? JavaMain+0xc1e C? [libjli.so+0x7759]? ThreadJavaMain+0x9 Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) j? SimpleTest.main([Ljava/lang/String;)V+24 v? ~StubRoutines::call_stub Where +24 is the bci. But that would be with the interpreter, with c1 that line will show as: J 576 c1 SimpleTest.main([Ljava/lang/String;)V (205 bytes) @ 0x00007f97b5ae5dc7 You have the method where it failed but not the exact bci. So maybe I should print the stack and then exit the VM. > I guess there isn't a handy function to show the source file name and > line number in the fatal error message. I'll see if there is an easy way to have it in the fatal error message otherwise I can just print the stack as with the warning case and exit. Thanks, Patricio > Thanks, > Coleen >> >> I don't know which exact technique JFR uses to print line numbers but >> it has to include reading the current bcp. Since when coming from the >> interpreter for monitorenter the bcp is always already pointing to >> the next instruction we need to decrement it to print the correct >> line numbers. I tested it and if I don't fix the bcp, as expected JFR >> too will print the next line number for the top frame. >> >> >> Patricio >>> Otherwise, this looks good to me. >>> Coleen >>> >>> >>> On 8/6/20 2:48 PM, Patricio Chilano wrote: >>>> Hi Dan, >>>> >>>> On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: >>>>> I'm peeking ahead to the next webrev... :-) >>>>> >>>>> > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ >>>> : ) >>>> >>>>> General comments: >>>>> ? - check files for copyright year updates. >>>> Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, >>>> c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. >>>> >>>>> src/hotspot/share/runtime/synchronizer.hpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/share/runtime/synchronizer.cpp >>>>> ??? L507: ? const markWord mark = obj->mark(); >>>>> ??? L508: >>>>> ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && >>>>> obj->klass()->is_box()) { >>>>> ??? L510: ??? return false; >>>>> ??? L511: ? } >>>>> ??????? The new bailout on L509-511 can move above L507. >>>> Moved. >>>> >>>>> L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of >>>>> klass %s", p2i(obj()), obj->klass()->external_name()); >>>>> ??????? This external_name() call does not have a ResourceMark. >>>> Good catch! I had one in a previous version but then I changed the >>>> conditionals and lost it for the fatal error case. The test worked >>>> okay because for the main JavaThread there is a ResourceMark in >>>> jni_invoke_static() (jni.cpp). >>>> >>>>> src/hotspot/share/logging/logTag.hpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/share/oops/klass.hpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/share/utilities/accessFlags.hpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/share/runtime/globals.hpp >>>>> ??? L814: ???????????? "0: off " >>>>> ??????? Missing a ';' after "off". >>>> Fixed. >>>> >>>>> L816: ???????????? "2: log message to stdout. >>>>> ??????? Perhaps add "(by default)" after "stdout" or >>>>> ??????? don't say where log output is at all. >>>>> >>>>> src/hotspot/share/runtime/arguments.cpp >>>>> ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, >>>>> true, LOG_TAGS(primitivewrappers)); >>>>> ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp >>>>> ??????? since it looks like logging output is configured to 'stdout'. >>>>> ??????? I'm assuming that other log options to put output elsewhere >>>>> ??????? are overridden by this code? >>>> Right. So the logging is done under UL with the tag >>>> primitivewrappers. If that tag is specified in Xlog then this >>>> conditional will be skipped because !log_is_enabled(Info, >>>> primitivewrappers) will be false. >>>> >>>>> src/hotspot/share/classfile/systemDictionary.cpp >>>>> ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { >>>>> ??????? nit - s/T_LONG+1/T_LONG + 1/ >>>> Fixed. >>>> >>>>> L2191: _box_klasses[i]->set_prototype_header(markWord::prototype()); >>>>> ??????? I assume we're keeping the prototype_header field when >>>>> Biased Locking >>>>> ??????? goes away? The reason I ask: >>>>> >>>>> ? ? ? ? static markWord prototype() { >>>>> ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); >>>>> ? ? ? ? } >>>>> >>>>> ??????? is because without Biased Locking, do we really need the >>>>> prototype >>>>> ??????? anymore? The initial markWord won't need possible variants... >>>> Yes, I think it can go away unless somebody finds another use for >>>> it. Maybe Valhalla. >>>> >>>>> src/hotspot/share/jfr/metadata/metadata.xml >>>>> ??? L69: ? >>>> ??????? Is the category "Java Application" because it's the >>>>> application >>>>> ??????? code that did something "wrong" here? Where "application" >>>>> is loosely >>>>> ??????? defined to include the possibility of auto generated code, >>>>> library >>>>> ??????? code and the like? Or perhaps "application" because >>>>> something "above" >>>>> ??????? the "Java Virtual Machine, Runtime" did the "wrong" thing >>>>> here? >>>> I don't know what the right category should be really. I saw the >>>> events JavaMonitorEnter, JavaMonitorWait and JavaMonitorInflate and >>>> thought this event should fall in the same category they do. >>>> >>>>> src/jdk.jfr/share/conf/jfr/default.jfc >>>>> ??? No comments. >>>>> >>>>> src/jdk.jfr/share/conf/jfr/profile.jfc >>>>> ??? No comments. >>>>> >>>>> test/lib/jdk/test/lib/jfr/EventNames.java >>>>> ??? No comments. >>>>> >>>>> src/hotspot/cpu/aarch64/aarch64.ad >>>>> ??? L3517: ????? __ tbnz(tmp, exact_log2(JVM_ACC_IS_BOX_CLASS), >>>>> cont); >>>>> >>>>> ??? L3578: ??? __ bind(cont); >>>>> ??? L3579: ??? // flag == EQ indicates success >>>>> ??? L3580: ??? // flag == NE indicates failure >>>>> ??????? If tbnz() branches to "cont" when we have a box class, what's >>>>> ??????? the flag value set to (EQ or NE)? And what set that flag >>>>> value? >>>>> ??????? The reason I ask is I don't think tbnz() sets condition >>>>> codes... >>>> Right, it doesn't set condition codes, so I kept the version I had >>>> (more on that below). >>>> >>>>> src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp >>>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>>> ??????????? ? ?? ??????????????????????????? Register obj_reg, >>>>> ?????????????????????????????? ? ?? ???????? Register swap_reg, >>>>> ?????????????????????????????????? ? ?? ???? Register tmp_reg, >>>>> ?????????????????????????????????? ? ?? ???? bool >>>>> swap_reg_contains_mark, >>>>> ?????????????????????????????????? ? ?? ???? Label& done, >>>>> ??????????????????????????????????? ? ?? ??? Label* slow_case, >>>>> BiasedLockingCounters* counters) { >>>>> ??????? I think you've changed the only callers of >>>>> biased_locking_enter() >>>>> ??????? that cared about the return value with this changeset so >>>>> it can >>>>> ??????? be changed to a void function. >>>> Ok, this is what I mentioned to David in a previous email. Done. >>>> >>>>> src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp >>>>> ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), done); >>>>> >>>>> ??? L131: ? bind(done); >>>>> ??? L132: >>>>> ??? L133: ? // At this point flags are set as follows: >>>>> ??? L134: ? //? EQ -> Success >>>>> ??? L135: ? //? NE -> Failure, branch to slow path >>>>> ??????? If tbnz() branches to "done" when we have a box class, what's >>>>> ??????? the flag value set to (EQ or NE)? And what set that flag >>>>> value? >>>>> ??????? The reason I ask is I don't think tbnz() sets condition >>>>> codes... >>>> Right. Same as above, I kept the version I had. >>>> >>>>> src/hotspot/cpu/arm/interp_masm_arm.cpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/cpu/arm/macroAssembler_arm.cpp >>>>> ??? int MacroAssembler::biased_locking_enter(Register obj_reg, >>>>> Register swap_reg, Register tmp_reg, >>>>> ????????????????????? ? ?? ????????????????? bool >>>>> swap_reg_contains_mark, >>>>> ???????????????????????? ? ?? ?????????????? Register tmp2, >>>>> ????????????????????????? ? ?? ????????????? Label& done, Label& >>>>> slow_case, >>>>> BiasedLockingCounters* counters) { >>>>> ??????? I think you've changed the only callers of >>>>> biased_locking_enter() >>>>> ??????? that cared about the return value with this changeset so >>>>> it can >>>>> ??????? be changed to a void function. >>>> Done. >>>> >>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>> ??? No comments on the PPC code. >>>>> >>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>> src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>> ??? No comments on the S390 code. >>>>> >>>>> src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp >>>>> ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); >>>>> ??????? What will this do with a 'noreg' value? (I need a refresher.) >>>> When not in 64 bit mode that register just won't be used. >>>> >>>>> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/cpu/x86/interp_masm_x86.cpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/cpu/x86/macroAssembler_x86.cpp >>>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>>> ????????????????????????????????? ? ?? ????? Register obj_reg, >>>>> ????????????????????????????????? ? ?? ????? Register swap_reg, >>>>> ????????????????????????????????? ? ?? ????? Register tmp_reg, >>>>> ?????????????????????????????????? ? ?? ???? Register tmp_reg2, >>>>> ?????????????????????????????????????? ? ? ? bool >>>>> swap_reg_contains_mark, >>>>> ?????????????????????????????????? ? ?? ???? Label& done, >>>>> ?????????????????????????????????? ? ?? ???? Label* slow_case, >>>>> BiasedLockingCounters* counters) { >>>>> ??????? I think you've changed the only caller of >>>>> biased_locking_enter() >>>>> ??????? that cared about the return value with this changeset so >>>>> it can >>>>> ??????? be changed to a void function. >>>> Done. >>>> >>>>> test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >>>>> ??? L30:? * @test SyncOnPrimitiveWrapperTest >>>>> ??????? No parameter to @test directive. >>>> Removed. >>>> >>>>> L136: ??????? private static long sharedCounter = 0L; >>>>> ??????? Since you don't do anything with sharedCounter other than >>>>> increment it, >>>>> ??????? can the compilers optimize it away? If it can be optimized >>>>> away, does >>>>> ??????? that mean that: >>>>> >>>>> ??????????? L142: ??????????????? synchronized (obj) { >>>>> >>>>> ??????? can also be optimized away? >>>>> >>>>> ??????? I don't think that: >>>>> >>>>> ??????????? L161: ??????????????? synchronized (sharedLock1) { >>>>> >>>>> ??????? can be optimized away because it is shared by multiple >>>>> threads. >>>>> >>>>> test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java >>>>> ??? Similar questions about 'counter' being optimized away. >>>>> ??? Similar question about "synchronized (obj) {" being optimized >>>>> away. >>>> I'm not sure if the compiler will optimize it away. Seems unlikely >>>> given the counter we are incrementing is not just local to some >>>> thread. >>>> >>>> >>>> Ok, below is v3 which has the following changes: >>>> - Use a 32 bit load for the _access_flags field, instead of 64 >>>> - Martin's implementation for s390 and fix for PPC >>>> - Faster LogTest version >>>> - Above changes based on Dan review >>>> >>>> I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 only) >>>> again with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I checked it >>>> builds in arm32, ppc and s390. >>>> >>>> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev >>>> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev >>>> >>>> >>>> @Martin: >>>> Please check if the test doesn't timeout for you now with the >>>> changes I made to LogTest. >>>> Also, I tried to use tbnz in aarch64 and arm32 instead of tst + br >>>> (except for c2 since we actually need to set the condition flags), >>>> but for c1 I was getting an assertion in the compiler thread: >>>> >>>> guarantee(chk == -1 || chk == 0) failed: Field too big for insn >>>> >>>> This is the stack when that happens: >>>> >>>> MacroAssembler::pd_patch_instruction_size(unsigned char*, unsigned >>>> char*)+0x398 >>>> AbstractAssembler::bind(Label&)+0x118 >>>> MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 >>>> LIR_Assembler::emit_slow_case_stubs()+0x54 >>>> Compilation::emit_code_body()+0x17c >>>> >>>> The change was simply: >>>> >>>> diff --git a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>> b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>> --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>> +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>> @@ -78,7 +78,6 @@ >>>> ?? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>> ???? load_klass(hdr, obj); >>>> -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); >>>> -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); >>>> -??? br(Assembler::NE, slow_case); >>>> +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); >>>> +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); >>>> ?? } >>>> >>>> So the issue must be related to where we want to jump. >>>> >>>> Thanks, >>>> Patricio >>>>> Dan >>>>> >>>>> >>>>> On 8/5/20 9:01 AM, Patricio Chilano wrote: >>>>>> Hi Martin, >>>>>> >>>>>> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>>>>>> Hi Patricio, >>>>>>> >>>>>>> using 8 Byte load instructions for jint fields is a terrible >>>>>>> coding style! >>>>>>> Someone else may see it and use an 8 Byte store. Will result in >>>>>>> great fun for debugging! >>>>>>> >>>>>>> And on Big Endian you will simply access the wrong bits. >>>>>> Ah, of course! Those 32 bits will be in the wrong place when >>>>>> doing the test. >>>>>> >>>>>>> Note that Big Endian Platforms are AIX, old linux ppc, s390, >>>>>>> SPARC. I don't think you have tested on them. >>>>>>> >>>>>>>> We could remove the nested synchronized statements in the run() >>>>>>>> method >>>>>>>> so that we don't generate that much logging. We could also lower >>>>>>>> LOOP_COUNT. I think the issue is also because we are running >>>>>>>> LogTest >>>>>>>> with multiple flag combinations. Not sure what we should touch >>>>>>>> first. >>>>>>>> Maybe the synchronized statements, have only one or two and >>>>>>>> test that >>>>>>>> first? >>>>>>> Sounds like helpful ideas. Please go ahead and strip things down. >>>>>> Great, I will send v3 later with those changes. >>>>>> >>>>>> Thanks Martin! >>>>>> >>>>>> Patricio >>>>>>> Thanks for taking care of it. >>>>>>> >>>>>>> Best regards, >>>>>>> Martin >>>>>>> >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Patricio Chilano >>>>>>>> Sent: Dienstag, 4. August 2020 20:26 >>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>>>> wrappers >>>>>>>> >>>>>>>> Hi Martin, >>>>>>>> >>>>>>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>>>>>> Hi Patricio, >>>>>>>>> >>>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>>> sending v3. >>>>>>>>> Thanks. I forgot to mention arm + aarch64. >>>>>>>>> Aarch64 uses ldrw + tbnz. >>>>>>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>>>>>> >>>>>>>>> I remember having seen the same mistake ?? >>>>>>>>> And nobody noticed it on little Endian platforms. >>>>>>>> Ok, I can use a tbnz instead of test and then a branch on arm32 >>>>>>>> and aarch64. >>>>>>>> Shouldn't a normal ldr in arm32 work fine? >>>>>>>> Also in 64 bits (either x64 or aarch64) I don't see the issue >>>>>>>> of using a >>>>>>>> 64 bit load, besides the fact that we only care about the first >>>>>>>> 32 bits. >>>>>>>> Regardless of the endianness, aren't we masking out the upper >>>>>>>> part when >>>>>>>> we do AND/TEST or if we test a bit in the 0-31 bit range? >>>>>>>> Otherwise it >>>>>>>> seems it should have failed for one of those platforms in my >>>>>>>> tests. >>>>>>>> >>>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you >>>>>>>>>> can >>>>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>>>> works and I >>>>>>>>>> can add it to the test. >>>>>>>>> That seems to have an effect. But now I'm not patient enough >>>>>>>>> to wait for >>>>>>>> the test to finish. >>>>>>>>> Maybe the problem is that I'm using slow debug builds. >>>>>>>>> But is there a chance to make the test quicker without losing >>>>>>>>> coverage >>>>>>>> significantly? >>>>>>>>> I think the test is too slow this way to run it on a regular >>>>>>>>> basis. We'd need to >>>>>>>> spend dedicated machines for it. >>>>>>>> We could remove the nested synchronized statements in the run() >>>>>>>> method >>>>>>>> so that we don't generate that much logging. We could also lower >>>>>>>> LOOP_COUNT. I think the issue is also because we are running >>>>>>>> LogTest >>>>>>>> with multiple flag combinations. Not sure what we should touch >>>>>>>> first. >>>>>>>> Maybe the synchronized statements, have only one or two and >>>>>>>> test that >>>>>>>> first? >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Patricio >>>>>>>>> Best regards, >>>>>>>>> Martin >>>>>>>>> >>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Patricio Chilano >>>>>>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on >>>>>>>>>> primitive wrappers >>>>>>>>>> >>>>>>>>>> Hi Martin, >>>>>>>>>> >>>>>>>>>> Thanks for fixing PPC and taking care of s390! >>>>>>>>>> >>>>>>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>>>>>> Hi Patricio, >>>>>>>>>>> >>>>>>>>>>> I suggest to use movl + testl for checking the access flag >>>>>>>>>>> as for other >>>>>>>> access >>>>>>>>>> flags on x64. >>>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>>> sending v3. >>>>>>>>>> >>>>>>>>>>> New version for PPC64 and s390 see below. >>>>>>>>>>> >>>>>>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>>>>>>> expected. >>>>>>>>>> However, I'm getting timeout issues: >>>>>>>>>>> timed out (timeout set to 120000ms, elapsed time including >>>>>>>>>>> timeout >>>>>>>>>> handling was 122372ms) >>>>>>>>>>> Can we provide more time? >>>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you >>>>>>>>>> can >>>>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>>>> works and I >>>>>>>>>> can add it to the test. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Patricio >>>>>>>>>>> Best regards, >>>>>>>>>>> Martin >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>> @@ -107,8 +107,8 @@ >>>>>>>>>>> >>>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>> Rscratch); >>>>>>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>> +??? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>> Rscratch); >>>>>>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>> ??????? bne(CCR0, slow_case); >>>>>>>>>>> ????? } >>>>>>>>>>> >>>>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>> @@ -912,8 +912,8 @@ >>>>>>>>>>> >>>>>>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>> ????????? load_klass(tmp, object); >>>>>>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>>>> +????? testbitdi(CCR0, R0, tmp, >>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>> ????????? bne(CCR0, slow_case); >>>>>>>>>>> ??????? } >>>>>>>>>>> >>>>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>>>>>> >>>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>> ??????? load_klass(temp, oop); >>>>>>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>>>> +??? testbitdi(CCR0, R0, temp, >>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>> ??????? bne(CCR0, cont); >>>>>>>>>>> ????? } >>>>>>>>>>> >>>>>>>>>>> diff -r 77852e129401 >>>>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue >>>>>>>>>>> Aug 04 >>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue >>>>>>>>>>> Aug 04 >>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>> @@ -91,6 +91,12 @@ >>>>>>>>>>> ????? // Save object being locked into the BasicObjectLock... >>>>>>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>>>>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>> +??? z_btrue(slow_case); >>>>>>>>>>> +? } >>>>>>>>>>> + >>>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>>>>>> Z_R0_scratch, done, >>>>>>>>>> &slow_case); >>>>>>>>>>> ????? } >>>>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>>>>> 10:03:57 >>>>>>>>>> 2020 +0200 >>>>>>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>> 16:04:31 >>>>>>>>>> 2020 +0200 >>>>>>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>>>>>> ????? z_lg(displaced_header, >>>>>>>>>>> oopDesc::mark_offset_in_bytes(), object); >>>>>>>>>>> >>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>> +??? z_btrue(slow_case); >>>>>>>>>>> +? } >>>>>>>>>>> + >>>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>>> ??????? biased_locking_enter(object, displaced_header, Z_R1, >>>>>>>>>>> Z_R0, done, >>>>>>>>>> &slow_case); >>>>>>>>>>> ????? } >>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>>>>>> ????? // Load markWord from oop into mark. >>>>>>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>>>>>> >>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>> +??? z_btrue(done); >>>>>>>>>>> +? } >>>>>>>>>>> + >>>>>>>>>>> ????? if (try_bias) { >>>>>>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, >>>>>>>>>>> Z_R0, done); >>>>>>>>>>> ????? } >>>>>>>>>>> >>>>>> >>>>> >>>> >>> >> > From coleen.phillimore at oracle.com Fri Aug 7 15:58:28 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 7 Aug 2020 11:58:28 -0400 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <41fd3f22-4266-de46-8d32-47ba6295218b@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> <2b030b14-0dc2-2de6-f01b-cc4f9ae80149@oracle.com> <41fd3f22-4266-de46-8d32-47ba6295218b@oracle.com> Message-ID: On 8/7/20 11:49 AM, Patricio Chilano wrote: > Hi Coleen, > > On 8/7/20 10:53 AM, Coleen Phillimore wrote: >> >> >> On 8/6/20 9:45 PM, Patricio Chilano wrote: >>> Hi Coleen, >>> >>> Thanks for looking at this. >>> >>> On 8/6/20 7:17 PM, Coleen Phillimore wrote: >>>> >>>> Patricio,? One question: >>>> >>>> http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/src/hotspot/share/runtime/synchronizer.cpp.udiff.html >>>> >>>> >>>> + // adjust bcp to point back to monitorenter so that we print the >>>> correct line numbers >>>> >>>> >>>> How do the fatal and event print the correct line numbers here? I >>>> see the logging gets it from the stack trace. Should the abort >>>> message have more information in it?? You can get the source and >>>> line number information in the same way that print_stack_on() gets it. >>> For the fatal error case I'm not printing line numbers as I do with >>> the warning case or as you get with JFR events. But maybe I should >>> print the stack too and then exit the VM. You can get the stack info >>> and the line number of the monitorenter bytecode that caused the >>> crash from the generated hs_err file though. >> >> Can you send a sample, please? > At the top of the hs_err you will get: > > ?# > ?# A fatal error has been detected by the Java Runtime Environment: > ?# > ?#? Internal Error > (/xx/xxx/xxxx/open/src/hotspot/share/runtime/synchronizer.cpp:574), > pid=48810, tid=48811 > ?#? fatal error: Synchronizing on object 0x00000007ff0758a8 of klass > java.lang.Integer > ?# > > And then on the stacktrace you can see: > > Stack: [0x00007f982b6ce000,0x00007f982b7cf000], > sp=0x00007f982b7cd5c0,? free space=1021k > Native frames: (J=compiled Java code, A=aot compiled Java code, > j=interpreted, Vv=VM code, C=native code) > V? [libjvm.so+0x1662385] > ObjectSynchronizer::handle_sync_on_primitive_wrapper(Handle, > Thread*)+0x185 > V? [libjvm.so+0x16683ac]? ObjectSynchronizer::enter(Handle, > BasicLock*, Thread*)+0x21c > V? [libjvm.so+0xce88fa] InterpreterRuntime::monitorenter(JavaThread*, > BasicObjectLock*)+0x13a > j? SimpleTest.main([Ljava/lang/String;)V+24 > v? ~StubRoutines::call_stub > V? [libjvm.so+0xd0263a]? JavaCalls::call_helper(JavaValue*, > methodHandle const&, JavaCallArguments*, Thread*)+0x62a > V? [libjvm.so+0xe22c8e]? jni_invoke_static(JNIEnv_*, JavaValue*, > _jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, Thr > ?.constprop.1]+0x35e > V? [libjvm.so+0xe28e6b]? jni_CallStaticVoidMethod+0x21b > C? [libjli.so+0x496e]? JavaMain+0xc1e > C? [libjli.so+0x7759]? ThreadJavaMain+0x9 > > Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) > j? SimpleTest.main([Ljava/lang/String;)V+24 > v? ~StubRoutines::call_stub > > Where +24 is the bci. But that would be with the interpreter, with c1 > that line will show as: > > J 576 c1 SimpleTest.main([Ljava/lang/String;)V (205 bytes) @ > 0x00007f97b5ae5dc7 > > You have the method where it failed but not the exact bci. So maybe I > should print the stack and then exit the VM. > >> I guess there isn't a handy function to show the source file name and >> line number in the fatal error message. > I'll see if there is an easy way to have it in the fatal error message > otherwise I can just print the stack as with the warning case and exit. I don't know if we want the fatal message to print the whole stack. But an improvement would be: ?#? fatal error: Synchronizing on object 0x00000007ff0758a8 of klass java.lang.Integer in method->external_name()+bci which you have in the function and then people won't have to go to the hs_err file. Thanks, Coleen > > > Thanks, > Patricio >> Thanks, >> Coleen >>> >>> I don't know which exact technique JFR uses to print line numbers >>> but it has to include reading the current bcp. Since when coming >>> from the interpreter for monitorenter the bcp is always already >>> pointing to the next instruction we need to decrement it to print >>> the correct line numbers. I tested it and if I don't fix the bcp, as >>> expected JFR too will print the next line number for the top frame. >>> >>> >>> Patricio >>>> Otherwise, this looks good to me. >>>> Coleen >>>> >>>> >>>> On 8/6/20 2:48 PM, Patricio Chilano wrote: >>>>> Hi Dan, >>>>> >>>>> On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: >>>>>> I'm peeking ahead to the next webrev... :-) >>>>>> >>>>>> > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ >>>>> : ) >>>>> >>>>>> General comments: >>>>>> ? - check files for copyright year updates. >>>>> Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, >>>>> c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. >>>>> >>>>>> src/hotspot/share/runtime/synchronizer.hpp >>>>>> ??? No comments. >>>>>> >>>>>> src/hotspot/share/runtime/synchronizer.cpp >>>>>> ??? L507: ? const markWord mark = obj->mark(); >>>>>> ??? L508: >>>>>> ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && >>>>>> obj->klass()->is_box()) { >>>>>> ??? L510: ??? return false; >>>>>> ??? L511: ? } >>>>>> ??????? The new bailout on L509-511 can move above L507. >>>>> Moved. >>>>> >>>>>> L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of >>>>>> klass %s", p2i(obj()), obj->klass()->external_name()); >>>>>> ??????? This external_name() call does not have a ResourceMark. >>>>> Good catch! I had one in a previous version but then I changed the >>>>> conditionals and lost it for the fatal error case. The test worked >>>>> okay because for the main JavaThread there is a ResourceMark in >>>>> jni_invoke_static() (jni.cpp). >>>>> >>>>>> src/hotspot/share/logging/logTag.hpp >>>>>> ??? No comments. >>>>>> >>>>>> src/hotspot/share/oops/klass.hpp >>>>>> ??? No comments. >>>>>> >>>>>> src/hotspot/share/utilities/accessFlags.hpp >>>>>> ??? No comments. >>>>>> >>>>>> src/hotspot/share/runtime/globals.hpp >>>>>> ??? L814: ???????????? "0: off " >>>>>> ??????? Missing a ';' after "off". >>>>> Fixed. >>>>> >>>>>> L816: ???????????? "2: log message to stdout. >>>>>> ??????? Perhaps add "(by default)" after "stdout" or >>>>>> ??????? don't say where log output is at all. >>>>>> >>>>>> src/hotspot/share/runtime/arguments.cpp >>>>>> ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, >>>>>> true, LOG_TAGS(primitivewrappers)); >>>>>> ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp >>>>>> ??????? since it looks like logging output is configured to >>>>>> 'stdout'. >>>>>> ??????? I'm assuming that other log options to put output elsewhere >>>>>> ??????? are overridden by this code? >>>>> Right. So the logging is done under UL with the tag >>>>> primitivewrappers. If that tag is specified in Xlog then this >>>>> conditional will be skipped because !log_is_enabled(Info, >>>>> primitivewrappers) will be false. >>>>> >>>>>> src/hotspot/share/classfile/systemDictionary.cpp >>>>>> ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { >>>>>> ??????? nit - s/T_LONG+1/T_LONG + 1/ >>>>> Fixed. >>>>> >>>>>> L2191: _box_klasses[i]->set_prototype_header(markWord::prototype()); >>>>>> ??????? I assume we're keeping the prototype_header field when >>>>>> Biased Locking >>>>>> ??????? goes away? The reason I ask: >>>>>> >>>>>> ? ? ? ? static markWord prototype() { >>>>>> ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); >>>>>> ? ? ? ? } >>>>>> >>>>>> ??????? is because without Biased Locking, do we really need the >>>>>> prototype >>>>>> ??????? anymore? The initial markWord won't need possible >>>>>> variants... >>>>> Yes, I think it can go away unless somebody finds another use for >>>>> it. Maybe Valhalla. >>>>> >>>>>> src/hotspot/share/jfr/metadata/metadata.xml >>>>>> ??? L69: ? >>>>> ??????? Is the category "Java Application" because it's the >>>>>> application >>>>>> ??????? code that did something "wrong" here? Where "application" >>>>>> is loosely >>>>>> ??????? defined to include the possibility of auto generated >>>>>> code, library >>>>>> ??????? code and the like? Or perhaps "application" because >>>>>> something "above" >>>>>> ??????? the "Java Virtual Machine, Runtime" did the "wrong" thing >>>>>> here? >>>>> I don't know what the right category should be really. I saw the >>>>> events JavaMonitorEnter, JavaMonitorWait and JavaMonitorInflate >>>>> and thought this event should fall in the same category they do. >>>>> >>>>>> src/jdk.jfr/share/conf/jfr/default.jfc >>>>>> ??? No comments. >>>>>> >>>>>> src/jdk.jfr/share/conf/jfr/profile.jfc >>>>>> ??? No comments. >>>>>> >>>>>> test/lib/jdk/test/lib/jfr/EventNames.java >>>>>> ??? No comments. >>>>>> >>>>>> src/hotspot/cpu/aarch64/aarch64.ad >>>>>> ??? L3517: ????? __ tbnz(tmp, exact_log2(JVM_ACC_IS_BOX_CLASS), >>>>>> cont); >>>>>> >>>>>> ??? L3578: ??? __ bind(cont); >>>>>> ??? L3579: ??? // flag == EQ indicates success >>>>>> ??? L3580: ??? // flag == NE indicates failure >>>>>> ??????? If tbnz() branches to "cont" when we have a box class, >>>>>> what's >>>>>> ??????? the flag value set to (EQ or NE)? And what set that flag >>>>>> value? >>>>>> ??????? The reason I ask is I don't think tbnz() sets condition >>>>>> codes... >>>>> Right, it doesn't set condition codes, so I kept the version I had >>>>> (more on that below). >>>>> >>>>>> src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>> ??? No comments. >>>>>> >>>>>> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp >>>>>> ??? No comments. >>>>>> >>>>>> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp >>>>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>>>> ??????????? ? ?? ??????????????????????????? Register obj_reg, >>>>>> ?????????????????????????????? ? ?? ???????? Register swap_reg, >>>>>> ?????????????????????????????????? ? ?? ???? Register tmp_reg, >>>>>> ?????????????????????????????????? ? ?? ???? bool >>>>>> swap_reg_contains_mark, >>>>>> ?????????????????????????????????? ? ?? ???? Label& done, >>>>>> ??????????????????????????????????? ? ?? ??? Label* slow_case, >>>>>> BiasedLockingCounters* counters) { >>>>>> ??????? I think you've changed the only callers of >>>>>> biased_locking_enter() >>>>>> ??????? that cared about the return value with this changeset so >>>>>> it can >>>>>> ??????? be changed to a void function. >>>>> Ok, this is what I mentioned to David in a previous email. Done. >>>>> >>>>>> src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp >>>>>> ??? No comments. >>>>>> >>>>>> src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp >>>>>> ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), >>>>>> done); >>>>>> >>>>>> ??? L131: ? bind(done); >>>>>> ??? L132: >>>>>> ??? L133: ? // At this point flags are set as follows: >>>>>> ??? L134: ? //? EQ -> Success >>>>>> ??? L135: ? //? NE -> Failure, branch to slow path >>>>>> ??????? If tbnz() branches to "done" when we have a box class, >>>>>> what's >>>>>> ??????? the flag value set to (EQ or NE)? And what set that flag >>>>>> value? >>>>>> ??????? The reason I ask is I don't think tbnz() sets condition >>>>>> codes... >>>>> Right. Same as above, I kept the version I had. >>>>> >>>>>> src/hotspot/cpu/arm/interp_masm_arm.cpp >>>>>> ??? No comments. >>>>>> >>>>>> src/hotspot/cpu/arm/macroAssembler_arm.cpp >>>>>> ??? int MacroAssembler::biased_locking_enter(Register obj_reg, >>>>>> Register swap_reg, Register tmp_reg, >>>>>> ????????????????????? ? ?? ????????????????? bool >>>>>> swap_reg_contains_mark, >>>>>> ???????????????????????? ? ?? ?????????????? Register tmp2, >>>>>> ????????????????????????? ? ?? ????????????? Label& done, Label& >>>>>> slow_case, >>>>>> BiasedLockingCounters* counters) { >>>>>> ??????? I think you've changed the only callers of >>>>>> biased_locking_enter() >>>>>> ??????? that cared about the return value with this changeset so >>>>>> it can >>>>>> ??????? be changed to a void function. >>>>> Done. >>>>> >>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>> ??? No comments on the PPC code. >>>>>> >>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>> src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>> ??? No comments on the S390 code. >>>>>> >>>>>> src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp >>>>>> ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); >>>>>> ??????? What will this do with a 'noreg' value? (I need a >>>>>> refresher.) >>>>> When not in 64 bit mode that register just won't be used. >>>>> >>>>>> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp >>>>>> ??? No comments. >>>>>> >>>>>> src/hotspot/cpu/x86/interp_masm_x86.cpp >>>>>> ??? No comments. >>>>>> >>>>>> src/hotspot/cpu/x86/macroAssembler_x86.cpp >>>>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>>>> ????????????????????????????????? ? ?? ????? Register obj_reg, >>>>>> ????????????????????????????????? ? ?? ????? Register swap_reg, >>>>>> ????????????????????????????????? ? ?? ????? Register tmp_reg, >>>>>> ?????????????????????????????????? ? ?? ???? Register tmp_reg2, >>>>>> ?????????????????????????????????????? ? ? ? bool >>>>>> swap_reg_contains_mark, >>>>>> ?????????????????????????????????? ? ?? ???? Label& done, >>>>>> ?????????????????????????????????? ? ?? ???? Label* slow_case, >>>>>> BiasedLockingCounters* counters) { >>>>>> ??????? I think you've changed the only caller of >>>>>> biased_locking_enter() >>>>>> ??????? that cared about the return value with this changeset so >>>>>> it can >>>>>> ??????? be changed to a void function. >>>>> Done. >>>>> >>>>>> test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >>>>>> ??? L30:? * @test SyncOnPrimitiveWrapperTest >>>>>> ??????? No parameter to @test directive. >>>>> Removed. >>>>> >>>>>> L136: ??????? private static long sharedCounter = 0L; >>>>>> ??????? Since you don't do anything with sharedCounter other than >>>>>> increment it, >>>>>> ??????? can the compilers optimize it away? If it can be >>>>>> optimized away, does >>>>>> ??????? that mean that: >>>>>> >>>>>> ??????????? L142: ??????????????? synchronized (obj) { >>>>>> >>>>>> ??????? can also be optimized away? >>>>>> >>>>>> ??????? I don't think that: >>>>>> >>>>>> ??????????? L161: ??????????????? synchronized (sharedLock1) { >>>>>> >>>>>> ??????? can be optimized away because it is shared by multiple >>>>>> threads. >>>>>> >>>>>> test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java >>>>>> ??? Similar questions about 'counter' being optimized away. >>>>>> ??? Similar question about "synchronized (obj) {" being optimized >>>>>> away. >>>>> I'm not sure if the compiler will optimize it away. Seems unlikely >>>>> given the counter we are incrementing is not just local to some >>>>> thread. >>>>> >>>>> >>>>> Ok, below is v3 which has the following changes: >>>>> - Use a 32 bit load for the _access_flags field, instead of 64 >>>>> - Martin's implementation for s390 and fix for PPC >>>>> - Faster LogTest version >>>>> - Above changes based on Dan review >>>>> >>>>> I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 only) >>>>> again with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I checked it >>>>> builds in arm32, ppc and s390. >>>>> >>>>> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev >>>>> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev >>>>> >>>>> >>>>> @Martin: >>>>> Please check if the test doesn't timeout for you now with the >>>>> changes I made to LogTest. >>>>> Also, I tried to use tbnz in aarch64 and arm32 instead of tst + br >>>>> (except for c2 since we actually need to set the condition flags), >>>>> but for c1 I was getting an assertion in the compiler thread: >>>>> >>>>> guarantee(chk == -1 || chk == 0) failed: Field too big for insn >>>>> >>>>> This is the stack when that happens: >>>>> >>>>> MacroAssembler::pd_patch_instruction_size(unsigned char*, unsigned >>>>> char*)+0x398 >>>>> AbstractAssembler::bind(Label&)+0x118 >>>>> MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 >>>>> LIR_Assembler::emit_slow_case_stubs()+0x54 >>>>> Compilation::emit_code_body()+0x17c >>>>> >>>>> The change was simply: >>>>> >>>>> diff --git a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>> b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>> --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>> +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>> @@ -78,7 +78,6 @@ >>>>> ?? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>> ???? load_klass(hdr, obj); >>>>> -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); >>>>> -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); >>>>> -??? br(Assembler::NE, slow_case); >>>>> +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); >>>>> +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); >>>>> ?? } >>>>> >>>>> So the issue must be related to where we want to jump. >>>>> >>>>> Thanks, >>>>> Patricio >>>>>> Dan >>>>>> >>>>>> >>>>>> On 8/5/20 9:01 AM, Patricio Chilano wrote: >>>>>>> Hi Martin, >>>>>>> >>>>>>> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>>>>>>> Hi Patricio, >>>>>>>> >>>>>>>> using 8 Byte load instructions for jint fields is a terrible >>>>>>>> coding style! >>>>>>>> Someone else may see it and use an 8 Byte store. Will result in >>>>>>>> great fun for debugging! >>>>>>>> >>>>>>>> And on Big Endian you will simply access the wrong bits. >>>>>>> Ah, of course! Those 32 bits will be in the wrong place when >>>>>>> doing the test. >>>>>>> >>>>>>>> Note that Big Endian Platforms are AIX, old linux ppc, s390, >>>>>>>> SPARC. I don't think you have tested on them. >>>>>>>> >>>>>>>>> We could remove the nested synchronized statements in the >>>>>>>>> run() method >>>>>>>>> so that we don't generate that much logging. We could also lower >>>>>>>>> LOOP_COUNT. I think the issue is also because we are running >>>>>>>>> LogTest >>>>>>>>> with multiple flag combinations. Not sure what we should touch >>>>>>>>> first. >>>>>>>>> Maybe the synchronized statements, have only one or two and >>>>>>>>> test that >>>>>>>>> first? >>>>>>>> Sounds like helpful ideas. Please go ahead and strip things down. >>>>>>> Great, I will send v3 later with those changes. >>>>>>> >>>>>>> Thanks Martin! >>>>>>> >>>>>>> Patricio >>>>>>>> Thanks for taking care of it. >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Martin >>>>>>>> >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Patricio Chilano >>>>>>>>> Sent: Dienstag, 4. August 2020 20:26 >>>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on >>>>>>>>> primitive wrappers >>>>>>>>> >>>>>>>>> Hi Martin, >>>>>>>>> >>>>>>>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>>>>>>> Hi Patricio, >>>>>>>>>> >>>>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>>>> sending v3. >>>>>>>>>> Thanks. I forgot to mention arm + aarch64. >>>>>>>>>> Aarch64 uses ldrw + tbnz. >>>>>>>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>>>>>>> >>>>>>>>>> I remember having seen the same mistake ?? >>>>>>>>>> And nobody noticed it on little Endian platforms. >>>>>>>>> Ok, I can use a tbnz instead of test and then a branch on >>>>>>>>> arm32 and aarch64. >>>>>>>>> Shouldn't a normal ldr in arm32 work fine? >>>>>>>>> Also in 64 bits (either x64 or aarch64) I don't see the issue >>>>>>>>> of using a >>>>>>>>> 64 bit load, besides the fact that we only care about the >>>>>>>>> first 32 bits. >>>>>>>>> Regardless of the endianness, aren't we masking out the upper >>>>>>>>> part when >>>>>>>>> we do AND/TEST or if we test a bit in the 0-31 bit range? >>>>>>>>> Otherwise it >>>>>>>>> seems it should have failed for one of those platforms in my >>>>>>>>> tests. >>>>>>>>> >>>>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe >>>>>>>>>>> you can >>>>>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>>>>> works and I >>>>>>>>>>> can add it to the test. >>>>>>>>>> That seems to have an effect. But now I'm not patient enough >>>>>>>>>> to wait for >>>>>>>>> the test to finish. >>>>>>>>>> Maybe the problem is that I'm using slow debug builds. >>>>>>>>>> But is there a chance to make the test quicker without losing >>>>>>>>>> coverage >>>>>>>>> significantly? >>>>>>>>>> I think the test is too slow this way to run it on a regular >>>>>>>>>> basis. We'd need to >>>>>>>>> spend dedicated machines for it. >>>>>>>>> We could remove the nested synchronized statements in the >>>>>>>>> run() method >>>>>>>>> so that we don't generate that much logging. We could also lower >>>>>>>>> LOOP_COUNT. I think the issue is also because we are running >>>>>>>>> LogTest >>>>>>>>> with multiple flag combinations. Not sure what we should touch >>>>>>>>> first. >>>>>>>>> Maybe the synchronized statements, have only one or two and >>>>>>>>> test that >>>>>>>>> first? >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Patricio >>>>>>>>>> Best regards, >>>>>>>>>> Martin >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: Patricio Chilano >>>>>>>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>>>>> ; >>>>>>>>>>> hotspot-runtime-dev at openjdk.java.net; >>>>>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on >>>>>>>>>>> primitive wrappers >>>>>>>>>>> >>>>>>>>>>> Hi Martin, >>>>>>>>>>> >>>>>>>>>>> Thanks for fixing PPC and taking care of s390! >>>>>>>>>>> >>>>>>>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>>>>>>> Hi Patricio, >>>>>>>>>>>> >>>>>>>>>>>> I suggest to use movl + testl for checking the access flag >>>>>>>>>>>> as for other >>>>>>>>> access >>>>>>>>>>> flags on x64. >>>>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>>>> sending v3. >>>>>>>>>>> >>>>>>>>>>>> New version for PPC64 and s390 see below. >>>>>>>>>>>> >>>>>>>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>>>>>>>> expected. >>>>>>>>>>> However, I'm getting timeout issues: >>>>>>>>>>>> timed out (timeout set to 120000ms, elapsed time including >>>>>>>>>>>> timeout >>>>>>>>>>> handling was 122372ms) >>>>>>>>>>>> Can we provide more time? >>>>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe >>>>>>>>>>> you can >>>>>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>>>>> works and I >>>>>>>>>>> can add it to the test. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Patricio >>>>>>>>>>>> Best regards, >>>>>>>>>>>> Martin >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>> @@ -107,8 +107,8 @@ >>>>>>>>>>>> >>>>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>>>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>>> Rscratch); >>>>>>>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>>> +??? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>>> Rscratch); >>>>>>>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>> ??????? bne(CCR0, slow_case); >>>>>>>>>>>> ????? } >>>>>>>>>>>> >>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>> @@ -912,8 +912,8 @@ >>>>>>>>>>>> >>>>>>>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>> ????????? load_klass(tmp, object); >>>>>>>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>>>>> +????? testbitdi(CCR0, R0, tmp, >>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>> ????????? bne(CCR0, slow_case); >>>>>>>>>>>> ??????? } >>>>>>>>>>>> >>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>>>>>>> >>>>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>> ??????? load_klass(temp, oop); >>>>>>>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>>>>> +??? testbitdi(CCR0, R0, temp, >>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>> ??????? bne(CCR0, cont); >>>>>>>>>>>> ????? } >>>>>>>>>>>> >>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue >>>>>>>>>>>> Aug 04 >>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue >>>>>>>>>>>> Aug 04 >>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>> @@ -91,6 +91,12 @@ >>>>>>>>>>>> ????? // Save object being locked into the BasicObjectLock... >>>>>>>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>>>>>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>> + z_btrue(slow_case); >>>>>>>>>>>> +? } >>>>>>>>>>>> + >>>>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>>>>>>> Z_R0_scratch, done, >>>>>>>>>>> &slow_case); >>>>>>>>>>>> ????? } >>>>>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>>>>>> 10:03:57 >>>>>>>>>>> 2020 +0200 >>>>>>>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>>> 16:04:31 >>>>>>>>>>> 2020 +0200 >>>>>>>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>>>>>>> ????? z_lg(displaced_header, >>>>>>>>>>>> oopDesc::mark_offset_in_bytes(), object); >>>>>>>>>>>> >>>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>> + z_btrue(slow_case); >>>>>>>>>>>> +? } >>>>>>>>>>>> + >>>>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>>>> ??????? biased_locking_enter(object, displaced_header, >>>>>>>>>>>> Z_R1, Z_R0, done, >>>>>>>>>>> &slow_case); >>>>>>>>>>>> ????? } >>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>>>>>>> ????? // Load markWord from oop into mark. >>>>>>>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>>>>>>> >>>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>> +??? z_btrue(done); >>>>>>>>>>>> +? } >>>>>>>>>>>> + >>>>>>>>>>>> ????? if (try_bias) { >>>>>>>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, >>>>>>>>>>>> Z_R0, done); >>>>>>>>>>>> ????? } >>>>>>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From dms at samersoff.net Fri Aug 7 16:18:09 2020 From: dms at samersoff.net (Dmitry Samersoff) Date: Fri, 7 Aug 2020 19:18:09 +0300 Subject: RFR 8241053: Hotspot runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java test fails on Alpine Linux with debug build In-Reply-To: References: <87a32c5b-8e96-f323-1e20-8df181ab3ec0@bell-sw.com> <5b530109-bda2-820e-6070-d779f08b64f6@oracle.com> <64281c1f-970d-b792-eb42-63ae1488694b@bell-sw.com> Message-ID: <1f6193b8-5a7b-2403-38c6-9285bb9e4c9e@samersoff.net> Hello Thomas, > Since I am here I have a question about muslc, if I may: musl libc doesn't have some features that glibc has (e.g. dlvsym), so, from code perspective, you can build musl version on glibc linux but not vice versa. From runtime perspective, you can't run binaries compiled for one platform on the other one because it requires different runtime linkers for load (/lib/ld-musl-x86_64.so.1 vs /lib64/ld-linux-x86-64.so.2) So if you have strict requirement to run the same binaries on both platforms you need to bundle correct libc with JDK. -Dmitry On 05.08.2020 10:33, Thomas St?fe wrote: > Fix looks good to me too (though you already have your two reviewers). > > Since I am here I have a question about muslc, if I may: > > Is there a way to execute code conditionally for glibc only? I ran into > this question a couple of times in the past. Related to that, would that > have to be a runtime check or would a compile time check sufficient? (In > other words, should a JDK built for linux x64 be runnable on distros with > muslc as well as glibc?) > > Thank you! > > ..Thomas > > On Tue, Aug 4, 2020 at 11:13 AM Alexander Scherbatiy < > alexander.scherbatiy at bell-sw.com> wrote: > >> Could you review the updated fix: >> http://cr.openjdk.java.net/~alexsch/8241053/webrev.01 >> >> The assert_status() and the thread cleanup is moved under the "if >> (status != 0)" check. >> >> Thanks, >> Alexander. >> >> On 04.08.2020 01:51, David Holmes wrote: >>> Hi Alexander, >>> >>> Thanks for fixing this. >>> >>> On 4/08/2020 1:33 am, Alexander Scherbatiy wrote: >>>> Hello, >>>> >>>> Could you review the fix for the issue: >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8241053 >>>> Webrev: http://cr.openjdk.java.net/~alexsch/8241053/weberv.00 >>> >>> I would simplify this a little: >>> >>> assert_status(status == 0 || status == EINVAL, status, >>> "pthread_attr_setstacksize"); >>> // pthread_attr_setstacksize() function can fail >>> // if the stack size exceeds a system-imposed limit. >>> if (status == EINVAL) { >>> >>> to >>> if (status != 0) { >>> // pthread_attr_setstacksize() function can fail >>> // if the stack size exceeds a system-imposed limit. >>> assert_status(status == EINVAL, status, >>> "pthread_attr_setstacksize"); >>> >>> Thanks, >>> David >>> ----- >>> >>> >>>> The following tests fail on Linux Alpine with musl libc: >>>> test/hotspot/jtreg/runtime/Thread/TestThreadStackSizes.java >>>> >> test/hotspot/jtreg/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >> >>>> >>>> >>>> This is because muls pthread_attr_setstacksize(3) function >>>> implementation [1] returns EINVAL when stack size exceeds the certain >>>> limit. >>>> >>>> According to POSIX pthread_attr_setstacksize() function description [2]: >>>> The pthread_attr_setstacksize() function will fail if: >>>> [EINVAL] The value of stacksize is less than PTHREAD_STACK_MIN >>>> or exceeds a system-imposed limit. >>>> >>>> The proposed fix excludes EINVAL value from assert_status() and for >>>> the EINVAL return value makes the same cleanup actions and returns >>>> false from os::create_thread() function as it is done for AIX[3]. >>>> >>>> To reproduce the issue it needs to build Portola from repository [4] >>>> using instructions from JEP 386: Alpine Linux/x64 Port [5]. >>>> After the fix both TestThreadStackSizes.java and >>>> TestOptionsWithRanges.java tests pass on Alpine Linux with musl libc >>>> and on Ubuntu 18.0.4. >>>> >>>> [1] >>>> >> https://git.musl-libc.org/cgit/musl/tree/src/thread/pthread_attr_setstacksize.c >>>> >>>> [2] >>>> >> https://pubs.opengroup.org/onlinepubs/007908799/xsh/pthread_attr_setstacksize.html >>>> >>>> [3] >>>> >> https://hg.openjdk.java.net/jdk/jdk/file/c8102e6fc512/src/hotspot/os/aix/os_aix.cpp#l890 >>>> >>>> [4] https://github.com/openjdk/portola >>>> [5] https://openjdk.java.net/jeps/386 >>>> >>>> Thanks, >>>> Alexander. >>>> >> From sergey.kuksenko at oracle.com Fri Aug 7 17:34:37 2020 From: sergey.kuksenko at oracle.com (Sergey Kuksenko) Date: Fri, 7 Aug 2020 10:34:37 -0700 Subject: Performance of instanceof with interfaces is multiple times slower than with classes In-Reply-To: References: Message-ID: <40424eba-abc8-90d0-dbb2-c0857e7bd7b0@oracle.com> By the way. Hotspot has CHA (class hierarchy analysis) which works for classes, but didn't work for interfaces. Particularly for your benchmark: ?"testInstanceOfClass" should be generated into the simple "false" and "testInstanceOfInterface" should be generated into real runtime check. Did you checked generated assembly? On 7/15/20 2:08 AM, Christoph Dreis wrote: > Hi, > > please forgive me if this is a stupid question or a known problem. > > I was working on something that involved an instanceof check and needed to change it slightly. > I was surprised to see that the performance difference between an interface and a class is quite big. > > E.g. consider the following benchmark: > > @BenchmarkMode(Mode.AverageTime) > @OutputTimeUnit(TimeUnit.NANOSECONDS) > @State(Scope.Thread) > public class MyBenchmark { > > interface TestInterface {} > private static class TestClass {} > private static class AnotherClass {} > > @State(Scope.Thread) > public static class BenchmarkState { > private Object clazz = new MyBenchmark.TestClass(); > } > > @Benchmark > public boolean testInstanceOfInterface(BenchmarkState state) { > return state.clazz instanceof TestInterface; > } > > @Benchmark > public boolean testInstanceOfClass(BenchmarkState state) { > return state.clazz instanceof AnotherClass; > } > } > > Benchmark Mode Cnt Score Error Units > MyBenchmark.testInstanceOfClass avgt 10 2,085 ? 0,179 ns/op > MyBenchmark.testInstanceOfInterface avgt 10 18,783 ? 0,595 ns/op > > I was surprised to see that the interface variant is so much slower. > Both checks should return false and there is no big hierarchy that needs to be walked up/down. > > Could you enlighten me what the cause for this is and maybe point me to the code where this is done? > Is this maybe even a bug/regression? Can we maybe do something to improve the interface case? > > Thanks in advance, > Cheers, > Christoph > > From thomas.stuefe at gmail.com Fri Aug 7 17:50:12 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 7 Aug 2020 19:50:12 +0200 Subject: RFR 8241053: Hotspot runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java test fails on Alpine Linux with debug build In-Reply-To: <1f6193b8-5a7b-2403-38c6-9285bb9e4c9e@samersoff.net> References: <87a32c5b-8e96-f323-1e20-8df181ab3ec0@bell-sw.com> <5b530109-bda2-820e-6070-d779f08b64f6@oracle.com> <64281c1f-970d-b792-eb42-63ae1488694b@bell-sw.com> <1f6193b8-5a7b-2403-38c6-9285bb9e4c9e@samersoff.net> Message-ID: Hi Dmitry, On Fri, Aug 7, 2020 at 6:18 PM Dmitry Samersoff wrote: > Hello Thomas, > > > Since I am here I have a question about muslc, if I may: > > musl libc doesn't have some features that glibc has (e.g. dlvsym), so, > from code perspective, you can build musl version on glibc linux but not > vice versa. > Without having a distro to test the build - what macro would I use to guard glibc-specific code? If I know or suspect some coding is glibc specific and may not work or build on muslc? > From runtime perspective, you can't run binaries compiled for one > platform on the other one because it requires different runtime linkers > for load (/lib/ld-musl-x86_64.so.1 vs /lib64/ld-linux-x86-64.so.2) > > So if you have strict requirement to run the same binaries on both > platforms you need to bundle correct libc with JDK. > > Thank you for the information, Thomas > -Dmitry > > > On 05.08.2020 10:33, Thomas St?fe wrote: > > Fix looks good to me too (though you already have your two reviewers). > > > > Since I am here I have a question about muslc, if I may: > > > > Is there a way to execute code conditionally for glibc only? I ran into > > this question a couple of times in the past. Related to that, would that > > have to be a runtime check or would a compile time check sufficient? (In > > other words, should a JDK built for linux x64 be runnable on distros with > > muslc as well as glibc?) > > > > Thank you! > > > > ..Thomas > > > > On Tue, Aug 4, 2020 at 11:13 AM Alexander Scherbatiy < > > alexander.scherbatiy at bell-sw.com> wrote: > > > >> Could you review the updated fix: > >> http://cr.openjdk.java.net/~alexsch/8241053/webrev.01 > >> > >> The assert_status() and the thread cleanup is moved under the "if > >> (status != 0)" check. > >> > >> Thanks, > >> Alexander. > >> > >> On 04.08.2020 01:51, David Holmes wrote: > >>> Hi Alexander, > >>> > >>> Thanks for fixing this. > >>> > >>> On 4/08/2020 1:33 am, Alexander Scherbatiy wrote: > >>>> Hello, > >>>> > >>>> Could you review the fix for the issue: > >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8241053 > >>>> Webrev: http://cr.openjdk.java.net/~alexsch/8241053/weberv.00 > >>> > >>> I would simplify this a little: > >>> > >>> assert_status(status == 0 || status == EINVAL, status, > >>> "pthread_attr_setstacksize"); > >>> // pthread_attr_setstacksize() function can fail > >>> // if the stack size exceeds a system-imposed limit. > >>> if (status == EINVAL) { > >>> > >>> to > >>> if (status != 0) { > >>> // pthread_attr_setstacksize() function can fail > >>> // if the stack size exceeds a system-imposed limit. > >>> assert_status(status == EINVAL, status, > >>> "pthread_attr_setstacksize"); > >>> > >>> Thanks, > >>> David > >>> ----- > >>> > >>> > >>>> The following tests fail on Linux Alpine with musl libc: > >>>> test/hotspot/jtreg/runtime/Thread/TestThreadStackSizes.java > >>>> > >> > test/hotspot/jtreg/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java > >> > >>>> > >>>> > >>>> This is because muls pthread_attr_setstacksize(3) function > >>>> implementation [1] returns EINVAL when stack size exceeds the certain > >>>> limit. > >>>> > >>>> According to POSIX pthread_attr_setstacksize() function description > [2]: > >>>> The pthread_attr_setstacksize() function will fail if: > >>>> [EINVAL] The value of stacksize is less than PTHREAD_STACK_MIN > >>>> or exceeds a system-imposed limit. > >>>> > >>>> The proposed fix excludes EINVAL value from assert_status() and for > >>>> the EINVAL return value makes the same cleanup actions and returns > >>>> false from os::create_thread() function as it is done for AIX[3]. > >>>> > >>>> To reproduce the issue it needs to build Portola from repository [4] > >>>> using instructions from JEP 386: Alpine Linux/x64 Port [5]. > >>>> After the fix both TestThreadStackSizes.java and > >>>> TestOptionsWithRanges.java tests pass on Alpine Linux with musl libc > >>>> and on Ubuntu 18.0.4. > >>>> > >>>> [1] > >>>> > >> > https://git.musl-libc.org/cgit/musl/tree/src/thread/pthread_attr_setstacksize.c > >>>> > >>>> [2] > >>>> > >> > https://pubs.opengroup.org/onlinepubs/007908799/xsh/pthread_attr_setstacksize.html > >>>> > >>>> [3] > >>>> > >> > https://hg.openjdk.java.net/jdk/jdk/file/c8102e6fc512/src/hotspot/os/aix/os_aix.cpp#l890 > >>>> > >>>> [4] https://github.com/openjdk/portola > >>>> [5] https://openjdk.java.net/jeps/386 > >>>> > >>>> Thanks, > >>>> Alexander. > >>>> > >> > > From lois.foltan at oracle.com Fri Aug 7 19:06:24 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Fri, 7 Aug 2020 15:06:24 -0400 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: References: Message-ID: <9e6b0043-65a1-dd97-a3d2-33679c8048d4@oracle.com> Hi Ioi, Overall looks promising.? I have some review comments below, but not a complete review.? I concentrated on the JVM side primarily how the archived module graph is read in, how the ModuleEntry and PackageEntry tables are created from the archive for the 3 builtin loaders and potential timing issues during start up. On 7/22/2020 3:36 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8244778 > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/ > > > Please review this patch that stores the full module graph in the CDS > archive heap. This reduces the initialization time of the basic JVM by > about 22%: > > $ perf stat -r 100 bin/java -version > before: 98,219,329 instructions 0.03971 secs elapsed (+- 0.44%) > after:? 55,835,815 instructions 0.03109 secs elapsed (+- 0.65%) > > [1] Start with ModuleBootstrap.java. The current implementation is > ??? quite restrictive: the archived module graph is used only when no > ??? module options are specified. > > ??? See ModuleBootstrap.mayUseArchivedBootLayer(). > > ??? We can probably support options such as main module and module path > ??? in a future RFE. Yes, I noticed the restrictions.? The JBS issue discusses the possibility of using the dumped module graph in the event that the value of the options are the same.? I think for this implementation to be viable and have a positive impact you should consider comparing at least the options --add-modules, --add-exports and --add-reads. > > [2] In the current JDK implementation, there is no single object > ??? that represents "the module graph". Most of the information > ??? is stored in the archive bootLayer object, but a few additional > ??? restoration operations need to be performed: > > ??? + See ModuleBootstrap.getArchivedBootLayer() > ??? + Some static fields need to be archived/restored in > ????? Module.java, BuiltinClassLoader.java, ClassLoaders.java > ????? and BootLoader.java > > [3] I ran into a complication with two loader instances of > ??? PlatformClassLoader and AppClassLoader. They are stored in > ??? multiple tables inside the module graph (e.g., > ??? BuiltinClassLoader$LoadedModule) so I cannot easily recreate > ??? them at runtime. > > ??? However, these two loaders contain information specific to the > ??? dump time VM lifecycle (such as the classes that were loaded > ??? during CDS dumping) that need to be scrubbed. I couldn't find an > ??? elegant way of doing this, so I added a private "resetArchivedStates" > ??? method to a few classes. They are called inside > ??? HeapShared::reset_archived_object_states(). > > [4] Related native data structures (PackageEntry and ModuleEntry) > ??? are also archived. Start with classLoaderData.cpp > > Passes mach5 tiers 1-4. I will test with additional tiers. > > Thanks > - Ioi classfile/classLoader.cpp - line #1644 pulling the javabase_moduleEntry() check outside of the Module_lock maybe problematic if after you check it another ? thread initializes in the time between the check and the obtaining of the Module_lock on line #1645. classfile/classLoader.hpp - somewhere in ArchivedClassLoaderData there should be an assert to make sure that the CLD, whose package entries and module entries are being archived is not a "_has_class_mirror_holder" CLD for a weakly defined hidden class.? Those dedicated CLDs should never have package entries or module entries. classfile/moduleEntry.cpp - line #400, typo "conver" --> "convert" - line #500, maybe sort if n is greater than 1 instead of 0 (same comment for packageEntry.cpp line #270) classfile/systemDictionary.cpp - It looks like the PackageEntry and ModuleEntry tables for the system and platform class loaders are? added within SystemDictionary::compute_java_loaders() which is called from Thread::create_vm() but after the call in Thread::create_vm() to call_initPhase2 which is when Universe::_module_initialized is set to true.? Did I read this correctly?? If yes, then I think you have to be sure that Universe::_module_initialized is not set until the module graph for the 3 builtin loaders is loaded from the archive. memory/filemap.cpp - line #237 typo "modue" --> "module" oops/instanceKlass.cpp - line #2545, comment "TO DO? -- point it to the archived PackageEntry (JDK-8249262)" are you thinking that since the module graph is read in ahead of time that it can be set when an InstanceKlass is created?? There is a point before java.base is defined that InstanceKlass::set_package takes into account that could be a timing issue. Clarification/timing questions: - Since the CDS support for capturing the module graph does not archive unnamed modules, I assume Modules::set_bootloader_unnamed_module() is still called.? Is this true or does the unnamed module for the boot loader get archived? - There are some checks in modules.cpp that are valuable to have during bootstrapping.? For example, packages that are encountered during bootstrapping before java.base is defined are put into the java.base module but then verified after java.base is defined via verify_javabase_packages.? So at what point in the bootstrapping process does the boot loader's archived module's become known? Could classes have been defined to the VM before this point?? Then their packages will have to be verified to be sure that they are indeed packages within java.base.? Consider looking at other checks that occur within classfile/modules.cpp as well. I may have more review comments as I continue to look at this! Thanks, Lois From john.r.rose at oracle.com Fri Aug 7 19:27:41 2020 From: john.r.rose at oracle.com (John Rose) Date: Fri, 7 Aug 2020 12:27:41 -0700 Subject: Performance of instanceof with interfaces is multiple times slower than with classes In-Reply-To: References: <6A5EB60C-88C2-4160-9C55-A82B8FA95ECF@oracle.com> Message-ID: On Aug 6, 2020, at 6:56 AM, Christoph Dreis wrote: > > I have in fact looked at the code already before the first mail and found emit_typecheck_helper in src/hotspot/cpu/x86/c1_LIRAssembler_x86.cpp, which I think is involved - please correct me if I'm wrong. > And MacroAssembler::check_klass_subtype_fast_path in src/hotspot/cpu/x86/macroAssembler_x86.cpp etc. Depending on the architecture (x86, aarch etc.) of course. > > Like you said, there are many things involved and I was hoping for a good starting point really from an experienced developer or a rough explanation what might be involved in the example. I understand now that there doesn't seem to be a "simple" explanation. Unfortunately, the C++ side of things is not as good documented as the Java side of things and that makes it relatively complicated for people like me who aren't familiar with the code to follow the flows sometimes. I can assure you: I do want to look at code, hence the - admittedly clumsy - question. I?ll explain how I would handle it, assuming I didn?t know the code in question already. (Counterfactual, since I wrote a bunch of it. But I want to explain how to do the exploration, which would be a better use of both our time, than for me to run a class over email on HotSpot type checking.) When exploring HotSpot code, you often have to follow dependencies many layers deep. You started in a good place at emit_typecheck_helper, perhaps after tracing callees from where c1 processes Bytecodes::_checkcast or some similar bytecode. You then observed that there is a call to MacroAssembler::check_klass_subtype_fast_path, and hopefully also that there?s a ?slow path? mentioned in emit_typecheck_helper. In MacroAssembler, if you nosed around a little, you might have noticed that the function following check_klass_subtype_fast_path is called check_klass_subtype_slow_path, or perhaps you find the latter function by tracing through callees. You could then verify your understanding about the roles of these two functions by grepping for them in the whole source base, and note that they are used in a few crucial places. $ grep -nH -e check_klass_subtype_[slowfast]*_path $(find src/hotspot/{cpu/x86,share} -type f) The next step is to ask, ?who does the macro assembler functions call?? You might think that?s a dead end, but in fact those functions make use of some symbolic offsets. This is something HotSpot engineers learn to look for and exploit. See these two lines: int sc_offset = in_bytes(Klass::secondary_super_cache_offset()); int sco_offset = in_bytes(Klass::super_check_offset_offset()); And later on in the slow path: int ss_offset = in_bytes(Klass::secondary_supers_offset()); int sc_offset = in_bytes(Klass::secondary_super_cache_offset()); They are static calls into klass.cpp which produce offsets in the Klass metadata structure. That is the next link to explore in the callee chain. Next stop, in klass.hpp there is a very brief comment explaining the data structure, plus this little gem: // ? See big discussion // in doc/server_compiler/checktype.txt There are two ways forward from this point. First, look for the places where the relevant fields of Klass are initialized, and you will see algorithms for filling them with the correct data. Second, find out where that checktype.txt went. Here?s how I did it; your mileage may vary. First ?git ls-files '**/*checktype*'? comes up empty. So then google ?hotspot checktype.txt? and see a bunch of WIFI mismatches. The winning google query is ?java hotspot checktype.txt?. First hit points to some public discourse a couple years ago from people asking questions similar to the one you are asking. The necessary clues really are in the code base, usually. It seems they were in this case. As has been discussed at other times (probably on this alias), the algorithm we use is good enough for many but not all use cases, and has known weaknesses. One or two complicated things could be done to patch the weaknesses, involving a better look aside cache (less thread contention and/or more cache slots) in front of the O(N) slow path, or a less than O(N) slow path, involving complicated tables. There?s literature out there (which I don?t have handy; it?s old stuff) about perfect hashing and other coloring schemes that reach a slow and complex but honorable O(1) speed. I will guess that a log N binary search through the supers array, maybe using metadata address order, would be simpler and just as good, in practice. It?s a matter for further research. One key constraint that some people don?t see at first: The resulting algorithm has to be maintainable over the long haul. It can?t complicate unrelated paths in the JVM, so it can?t be overly ?heroic? in order to maintain some complicated perfect coloring scheme. It has to be self-contained, and easy to work with in isolation. I hope this helps. ? John From martin.doerr at sap.com Fri Aug 7 19:33:50 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 7 Aug 2020 19:33:50 +0000 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> Message-ID: Hi Patricio, thanks for incorporating my fixes. I don?t know if I can find time for measurements next week. I have a lot of other things to do. Feel free to push it after you got enough reviews. Best regards, Martin From: Patricio Chilano Sent: Freitag, 7. August 2020 17:29 To: Doerr, Martin ; daniel.daugherty at oracle.com; David Holmes ; hotspot-runtime-dev at openjdk.java.net; hotspot-jfr-dev at openjdk.java.net Subject: Re: RFR 8242263: Diagnose synchronization on primitive wrappers Hi Martin, On 8/7/20 12:58 AM, Doerr, Martin wrote: Hi Patricio, thanks for the update. Sorry for causing trouble related to the tbnz instructions on aarch64. My concern was basically to use the correct number of Bytes for the load instructions. You have fixed that in webrev v3. Thanks. Unfortunately, I did the same mistake regarding the flag requirements on PPC64 and s390. And we have a wrong label on PPC64 (otherwise branch destination can be too far). Test duration is still terrible with the slow debug build. I still got timeouts. (Maybe I should better use fast debug builds.) I measured the time in my Mac (fastdebug build). The FatalTest takes ~ less than a second to run, and we have 7*8=56 of those. The LogTest takes between 1-4 seconds depending on the flag, and we have 7 of those. Overall it takes 2-3 minutes to run. Maybe you can measure those times too to see where its taking so long? Fixes see below. Great, added the new fixes. Here is v4: Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v4/webrev/ Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v4/inc/webrev/ Sometimes simple changes can require quite some time ... Yep, no problem : ) Thanks! Patricio Best regards, Martin diff -r f629202b3862 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Thu Aug 06 22:03:39 2020 +0200 +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Thu Aug 06 23:26:02 2020 +0200 @@ -109,7 +109,7 @@ load_klass(Rscratch, Roop); lwz(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); testbitdi(CCR0, R0, Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS)); - bne(CCR0, slow_case); + bne(CCR0, slow_int); } if (UseBiasedLocking) { diff -r f629202b3862 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Thu Aug 06 22:03:39 2020 +0200 +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Thu Aug 06 23:26:02 2020 +0200 @@ -2839,8 +2839,8 @@ if (DiagnoseSyncOnPrimitiveWrappers != 0) { load_klass(temp, oop); lwz(temp, in_bytes(Klass::access_flags_offset()), temp); - testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); - bne(CCR0, cont); + testbitdi(flag, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); + bne(flag, cont); } if (try_bias) { diff -r f629202b3862 src/hotspot/cpu/s390/macroAssembler_s390.cpp --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Thu Aug 06 22:03:39 2020 +0200 +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Thu Aug 06 23:26:02 2020 +0200 @@ -3360,8 +3360,10 @@ if (DiagnoseSyncOnPrimitiveWrappers != 0) { load_klass(Z_R1_scratch, oop); - testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), exact_log2(JVM_ACC_IS_BOX_CLASS)); - z_btrue(done); + z_l(Z_R1_scratch, Address(Z_R1_scratch, Klass::access_flags_offset())); + assert((JVM_ACC_IS_BOX_CLASS & 0xFFFF) == 0, "or change following instruction"); + z_nilh(Z_R1_scratch, JVM_ACC_IS_BOX_CLASS >> 16); + z_brne(done); } if (try_bias) { diff -r f629202b3862 test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java --- a/test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java Thu Aug 06 22:03:39 2020 +0200 +++ b/test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java Thu Aug 06 23:26:02 2020 +0200 @@ -31,7 +31,7 @@ * @bug 8242263 * @summary Exercise DiagnoseSyncOnPrimitiveWrappers diagnostic flag * @library /test/lib - * @run driver SyncOnPrimitiveWrapperTest + * @run driver/timeout=180000 SyncOnPrimitiveWrapperTest */ public class SyncOnPrimitiveWrapperTest { From igor.ignatyev at oracle.com Fri Aug 7 19:41:28 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 7 Aug 2020 12:41:28 -0700 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> Message-ID: <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> > On Aug 6, 2020, at 12:33 AM, David Holmes wrote: > > Basing the new test around the existing thread stress tests was not unreasonable but I don't think we're actively adding to those tests. I would check with Igor whether it is better to just add this under runtime/thread rather than vmTestBase. Hi Gerard, David is right, we are trying to get rid of vmTestbase directory and move all tests into corresponding component directories cleaning them up at the same time. so runtime/thread directory sound like a much better location for a new test. I don't think you need to cary all this baggage of 'VM testbase keywords', 'converted from ...' and the rest, I'd suggest you just use the text from 'DESCRIPTION' as @summary. I'd also remove System.exit call and would just make thread009::run to return bool. it's also unclear to me if someone is ever going to change THREADS_EXPECTED , TIMEOUT, and DEBUG_MODE values, I'd just hardcode them in the test and get rid of parsing. it seems that debug output won't clutter output, so you might want to always print them and remove DEBUG_MODE field. using `assert` in test code isn't a good idea as it will work only if someone runs test w/ `-ea` which not all people do. it's better to replace w/ `if (p) throw new Error(...)` or use jdk.test.lib.Asserts class. Cheers, -- Igor From christoph.dreis at freenet.de Fri Aug 7 19:41:27 2020 From: christoph.dreis at freenet.de (Christoph Dreis) Date: Fri, 07 Aug 2020 21:41:27 +0200 Subject: Performance of instanceof with interfaces is multiple times slower than with classes In-Reply-To: References: <6A5EB60C-88C2-4160-9C55-A82B8FA95ECF@oracle.com> Message-ID: <30825BC9-48CE-4AB4-8111-2B1F9A1C7C4C@freenet.de> Hi John, thanks for your elaborate answer. That helps a lot indeed and explains some dead ends I - as a non Hotspot engineer - could have not solved without some hints. I will dig a bit deeper into this now and also follow up on the "big discussion" in the past. Thank you for taking the time - I really appreciate it. Cheers, Christoph Von: John Rose Datum: Freitag, 7. August 2020 um 21:27 An: Christoph Dreis Cc: hotspot-runtime-dev Betreff: Re: Performance of instanceof with interfaces is multiple times slower than with classes On Aug 6, 2020, at 6:56 AM, Christoph Dreis wrote: I have in fact looked at the code already before the first mail and found emit_typecheck_helper in src/hotspot/cpu/x86/c1_LIRAssembler_x86.cpp, which I think is involved - please correct me if I'm wrong. And MacroAssembler::check_klass_subtype_fast_path in src/hotspot/cpu/x86/macroAssembler_x86.cpp etc. Depending on the architecture (x86, aarch etc.) of course. Like you said, there are many things involved and I was hoping for a good starting point really from an experienced developer or a rough explanation what might be involved in the example. I understand now that there doesn't seem to be a "simple" explanation. Unfortunately, the C++ side of things is not as good documented as the Java side of things and that makes it relatively complicated for people like me who aren't familiar with the code to follow the flows sometimes. I can assure you: I do want to look at code, hence the - admittedly clumsy - question. I?ll explain how I would handle it, assuming I didn?t know the code in question already. ?(Counterfactual, since I wrote a bunch of it. ?But I want to explain how to do the exploration, which would be a better use of both our time, than for me to run a class over email on HotSpot type checking.) When exploring HotSpot code, you often have to follow dependencies many layers deep. ?You started in a good place at emit_typecheck_helper, perhaps after tracing callees from where c1 processes Bytecodes::_checkcast or some similar bytecode. ?You then observed that there is a call to MacroAssembler::check_klass_subtype_fast_path, and hopefully also that there?s a ?slow path? mentioned in emit_typecheck_helper. In MacroAssembler, if you nosed around a little, you might have noticed that the function following ?check_klass_subtype_fast_path is called check_klass_subtype_slow_path, or perhaps you find the latter function by tracing through callees. You could then verify your understanding about the roles of these two functions by grepping for them in the whole source base, and note that they are used in a few crucial places. $ grep ?-nH -e ?check_klass_subtype_[slowfast]*_path $(find src/hotspot/{cpu/x86,share} -type f) The next step is to ask, ?who does the macro assembler functions call?? You might think that?s a dead end, but in fact those functions make use of some symbolic offsets. ?This is something HotSpot engineers learn to look for and exploit. ?See these two lines: ? int sc_offset = in_bytes(Klass::secondary_super_cache_offset()); ? int sco_offset = in_bytes(Klass::super_check_offset_offset()); And later on in the slow path: ? int ss_offset = in_bytes(Klass::secondary_supers_offset()); ? int sc_offset = in_bytes(Klass::secondary_super_cache_offset()); They are static calls into klass.cpp which produce offsets in the Klass metadata structure. ?That is the next link to explore in the callee chain. Next stop, in klass.hpp there is a very brief comment explaining the data structure, plus this little gem: ? // ? See big discussion ? // in doc/server_compiler/checktype.txt There are two ways forward from this point. ?First, look for the places where the relevant fields of Klass are initialized, and you will see algorithms for filling them with the correct data. Second, find out where that checktype.txt went. ?Here?s how I did it; your mileage may vary. ?First ?git ls-files '**/*checktype*'? comes up empty. ?So then google ?hotspot checktype.txt? and see a bunch of WIFI mismatches. ?The winning google query is ?java hotspot checktype.txt?. ?First hit points to some public discourse a couple years ago from people asking questions similar to the one you are asking. The necessary clues really are in the code base, usually. It seems they were in this case. As has been discussed at other times (probably on this alias), the algorithm we use is good enough for many but not all use cases, and has known weaknesses. ?One or two complicated things could be done to patch the weaknesses, involving a better look aside cache (less thread contention and/or more cache slots) in front of the O(N) slow path, or a less than O(N) slow path, involving complicated tables. ?There?s literature out there (which I don?t have handy; it?s old stuff) about perfect hashing and other coloring schemes that reach a slow and complex but honorable O(1) speed. ?I will guess that a log N binary search through the supers array, maybe using metadata address order, would be simpler and just as good, in practice. It?s a matter for further research. ?One key constraint that some people don?t see at first: ?The resulting algorithm has to be maintainable over the long haul. ?It can?t complicate unrelated paths in the JVM, so it can?t be overly ?heroic? in order to maintain some complicated perfect coloring scheme. ?It has to be self-contained, and easy to work with in isolation. I hope this helps. ? John From ioi.lam at oracle.com Fri Aug 7 21:13:48 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 7 Aug 2020 14:13:48 -0700 Subject: Performance of instanceof with interfaces is multiple times slower than with classes In-Reply-To: <30825BC9-48CE-4AB4-8111-2B1F9A1C7C4C@freenet.de> References: <6A5EB60C-88C2-4160-9C55-A82B8FA95ECF@oracle.com> <30825BC9-48CE-4AB4-8111-2B1F9A1C7C4C@freenet.de> Message-ID: <4e783b3a-68a3-9a03-3ee5-9f9ba26b78b0@oracle.com> One thing that I found very useful in learning the C1 compiler is something like this: public class InterfaceTest { ? public static void main(String args[]) { ??? test(new Runnable() { public void run() {} }); ? } ? static void test (Runnable r) { ??? r.run(); ? } } Build a slow-debug version of the JVM, and then: $ gdb --args myjdk/bin/java -cp ~/tmp -Xcomp -XX:TieredStopAtLevel=1 \ ??? -XX:CompileCommand=print,InterfaceTest::test \ ??? -XX:CompileCommand=compileonly,InterfaceTest::test InterfaceTest (gdb) b nmethod::print_nmethod (gdb) r Thread 13 "C1 CompilerThre" hit Breakpoint 4, nmethod::print_nmethod (....) (gdb) finish This will print out the content of the compiled method, like: [Verified Entry Point] ? # {method} {0x00007fffa17fc3a0} 'test' '(Ljava/lang/Runnable;)V' in 'InterfaceTest' ? # parm0:??? rsi:rsi?? = 'java/lang/Runnable' ? #?????????? [sp+0x40]? (sp of caller) ?;;? block B1 [0, 0] ? 0x00007fffd8cf61c0:?? mov??? %eax,-0x16000(%rsp) ? 0x00007fffd8cf61c7:?? push?? %rbp ? 0x00007fffd8cf61c8:?? sub??? $0x30,%rsp ;*aload_0 {reexecute=0 rethrow=0 return_oop=0} Now you can set a break point there and single step the instructions to see what's happening. (gdb) b *0x00007fffd8cf61c0 Breakpoint 5 at 0x7fffd8cf61c0 (gdb) c Continuing. [Switching to Thread 0x7ffff7fb6700 (LWP 24183)] Thread 2 "java" hit Breakpoint 5, 0x00007fffd8cf61c0 in ?? () (gdb) display/i $pc 1: x/i $pc => 0x7fffd8cf61c0:??? mov??? %eax,-0x16000(%rsp) (gdb) si 0x00007fffd8cf61c7 in ?? () 1: x/i $pc => 0x7fffd8cf61c7:??? push?? %rbp (gdb) si 0x00007fffd8cf61c8 in ?? () 1: x/i $pc => 0x7fffd8cf61c8:??? sub??? $0x30,%rsp (gdb) si 0x00007fffd8cf61cc in ?? () 1: x/i $pc => 0x7fffd8cf61cc:??? nop You can also set breakpoints at places like emit_typecheck_helper. Because now only a very limited number of bytecodes are compiled, you will have less distraction and can focus on what happens with your test case. It won't be easy but will definitely be fun! (And the C2 compiler is a completely different beast .....) - Ioi On 8/7/20 12:41 PM, Christoph Dreis wrote: > Hi John, > > thanks for your elaborate answer. That helps a lot indeed and explains some dead ends I - as a non Hotspot engineer - could have not solved without some hints. > > I will dig a bit deeper into this now and also follow up on the "big discussion" in the past. > > Thank you for taking the time - I really appreciate it. > > Cheers, > Christoph > > Von: John Rose > Datum: Freitag, 7. August 2020 um 21:27 > An: Christoph Dreis > Cc: hotspot-runtime-dev > Betreff: Re: Performance of instanceof with interfaces is multiple times slower than with classes > > On Aug 6, 2020, at 6:56 AM, Christoph Dreis wrote: > > I have in fact looked at the code already before the first mail and found emit_typecheck_helper in src/hotspot/cpu/x86/c1_LIRAssembler_x86.cpp, which I think is involved - please correct me if I'm wrong. > And MacroAssembler::check_klass_subtype_fast_path in src/hotspot/cpu/x86/macroAssembler_x86.cpp etc. Depending on the architecture (x86, aarch etc.) of course. > > Like you said, there are many things involved and I was hoping for a good starting point really from an experienced developer or a rough explanation what might be involved in the example. I understand now that there doesn't seem to be a "simple" explanation. Unfortunately, the C++ side of things is not as good documented as the Java side of things and that makes it relatively complicated for people like me who aren't familiar with the code to follow the flows sometimes. I can assure you: I do want to look at code, hence the - admittedly clumsy - question. > > I?ll explain how I would handle it, assuming I didn?t know the > code in question already. ?(Counterfactual, since I wrote a bunch > of it. ?But I want to explain how to do the exploration, which would > be a better use of both our time, than for me to run a class over email > on HotSpot type checking.) > > When exploring HotSpot code, you often have to follow dependencies > many layers deep. ?You started in a good place at emit_typecheck_helper, > perhaps after tracing callees from where c1 processes Bytecodes::_checkcast > or some similar bytecode. ?You then observed that there is a call to > MacroAssembler::check_klass_subtype_fast_path, and hopefully also > that there?s a ?slow path? mentioned in emit_typecheck_helper. > > In MacroAssembler, if you nosed around a little, you might have noticed > that the function following ?check_klass_subtype_fast_path is called > check_klass_subtype_slow_path, or perhaps you find the latter function > by tracing through callees. > > You could then verify your understanding about the roles of these two > functions by grepping for them in the whole source base, and note that > they are used in a few crucial places. > > $ grep ?-nH -e ?check_klass_subtype_[slowfast]*_path $(find src/hotspot/{cpu/x86,share} -type f) > > The next step is to ask, ?who does the macro assembler functions call?? > You might think that?s a dead end, but in fact those functions make > use of some symbolic offsets. ?This is something HotSpot engineers > learn to look for and exploit. ?See these two lines: > > ? int sc_offset = in_bytes(Klass::secondary_super_cache_offset()); > ? int sco_offset = in_bytes(Klass::super_check_offset_offset()); > > And later on in the slow path: > > ? int ss_offset = in_bytes(Klass::secondary_supers_offset()); > ? int sc_offset = in_bytes(Klass::secondary_super_cache_offset()); > > They are static calls into klass.cpp which produce offsets in > the Klass metadata structure. ?That is the next link to explore > in the callee chain. > > Next stop, in klass.hpp there is a very brief comment explaining > the data structure, plus this little gem: > > ? // ? See big discussion > ? // in doc/server_compiler/checktype.txt > > There are two ways forward from this point. ?First, look for > the places where the relevant fields of Klass are initialized, > and you will see algorithms for filling them with the correct > data. > > Second, find out where that checktype.txt went. ?Here?s how > I did it; your mileage may vary. ?First ?git ls-files '**/*checktype*'? > comes up empty. ?So then google ?hotspot checktype.txt? > and see a bunch of WIFI mismatches. ?The winning google > query is ?java hotspot checktype.txt?. ?First hit points to > some public discourse a couple years ago from people asking > questions similar to the one you are asking. > > The necessary clues really are in the code base, usually. > It seems they were in this case. > > As has been discussed at other times (probably on this alias), > the algorithm we use is good enough for many but not all > use cases, and has known weaknesses. ?One or two complicated > things could be done to patch the weaknesses, involving > a better look aside cache (less thread contention and/or > more cache slots) in front of the O(N) slow path, or a less > than O(N) slow path, involving complicated tables. ?There?s > literature out there (which I don?t have handy; it?s old stuff) > about perfect hashing and other coloring schemes that > reach a slow and complex but honorable O(1) speed. ?I > will guess that a log N binary search through the supers > array, maybe using metadata address order, would be > simpler and just as good, in practice. > > It?s a matter for further research. ?One key constraint that > some people don?t see at first: ?The resulting algorithm has > to be maintainable over the long haul. ?It can?t complicate > unrelated paths in the JVM, so it can?t be overly ?heroic? > in order to maintain some complicated perfect coloring > scheme. ?It has to be self-contained, and easy to work with > in isolation. > > I hope this helps. > > ? John > > From john.r.rose at oracle.com Fri Aug 7 22:45:38 2020 From: john.r.rose at oracle.com (John Rose) Date: Fri, 7 Aug 2020 15:45:38 -0700 Subject: Performance of instanceof with interfaces is multiple times slower than with classes In-Reply-To: <30825BC9-48CE-4AB4-8111-2B1F9A1C7C4C@freenet.de> References: <6A5EB60C-88C2-4160-9C55-A82B8FA95ECF@oracle.com> <30825BC9-48CE-4AB4-8111-2B1F9A1C7C4C@freenet.de> Message-ID: <8CFC53B4-E532-4718-BAEA-5A9A22992521@oracle.com> On Aug 7, 2020, at 12:41 PM, Christoph Dreis wrote: > > thanks for your elaborate answer. That helps a lot indeed and explains some dead ends I - as a non Hotspot engineer - could have not solved without some hints. > > I will dig a bit deeper into this now and also follow up on the "big discussion" in the past. > > Thank you for taking the time - I really appreciate it. You are welcome! One thing mentioned in checktype.txt is a lookaside negative cache, of one element. That might make your micro go faster (maybe; it?s already plenty fast given the short SS array), but wouldn?t scale so well to Scala?s use cases. I suspect there?s useful future work to be done. But, to emphasize: The really interesting improvements, IMO, involve both negative and positive tests on *long* secondary super arrays, and *less frequent updates* of cross-thread state. Your benchmark doesn?t touch on either of those questions. One thing it *might* touch on is a relative slowness for traversing a zero-element SS array. I think the repne_scan idiom (scasq) might be past its sell-by date; is may no longer be faster than just a regular tight loop of load/cmp/jcc. So, a quick experiment you might wish to try is to patch out the use of scasq with a bit of replacement code and see if it helps. Further down that rabbit hole would be an attempt to recognize longer SS arrays and call an out-of-line routine, which then could be upgraded to binary search. ? John From john.r.rose at oracle.com Fri Aug 7 23:50:05 2020 From: john.r.rose at oracle.com (John Rose) Date: Fri, 7 Aug 2020 16:50:05 -0700 Subject: Performance of instanceof with interfaces is multiple times slower than with classes In-Reply-To: <8CFC53B4-E532-4718-BAEA-5A9A22992521@oracle.com> References: <6A5EB60C-88C2-4160-9C55-A82B8FA95ECF@oracle.com> <30825BC9-48CE-4AB4-8111-2B1F9A1C7C4C@freenet.de> <8CFC53B4-E532-4718-BAEA-5A9A22992521@oracle.com> Message-ID: <4AD9481C-1B70-4CC1-BFF8-4D70F42840BC@oracle.com> P.S. I filed https://bugs.openjdk.java.net/browse/JDK-8251318 to record the suggested fixes in JBS. > On Aug 7, 2020, at 3:45 PM, John Rose wrote: > > On Aug 7, 2020, at 12:41 PM, Christoph Dreis wrote: >> >> thanks for your elaborate answer. That helps a lot indeed and explains some dead ends I - as a non Hotspot engineer - could have not solved without some hints. >> >> I will dig a bit deeper into this now and also follow up on the "big discussion" in the past. >> >> Thank you for taking the time - I really appreciate it. > > You are welcome! > > One thing mentioned in checktype.txt is a lookaside negative cache, > of one element. That might make your micro go faster (maybe; it?s > already plenty fast given the short SS array), but wouldn?t scale so well > to Scala?s use cases. I suspect there?s useful future work to be done. > > But, to emphasize: The really interesting improvements, IMO, involve > both negative and positive tests on *long* secondary super arrays, > and *less frequent updates* of cross-thread state. > > Your benchmark doesn?t touch on either of those questions. > > One thing it *might* touch on is a relative slowness for traversing > a zero-element SS array. I think the repne_scan idiom (scasq) might > be past its sell-by date; is may no longer be faster than just a regular > tight loop of load/cmp/jcc. So, a quick experiment you might wish > to try is to patch out the use of scasq with a bit of replacement code > and see if it helps. Further down that rabbit hole would be an attempt > to recognize longer SS arrays and call an out-of-line routine, which > then could be upgraded to binary search. > > ? John From thomas.stuefe at gmail.com Sat Aug 8 06:14:15 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 8 Aug 2020 08:14:15 +0200 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info In-Reply-To: References: Message-ID: Hi, all tests at SAP went through successfully, as did the jdk-submit tests. ..Thomas On Thu, Aug 6, 2020 at 4:01 PM Thomas St?fe wrote: > Hi, > > may I please have reviews for this addition: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs-err/webrev.00/webrev/ > > When analysing a customer memory leak recently I found for all > the information we have in the error files we miss some really fundamental > statistics like process virtual size and rss. Either that or I am blind :) > > In case I am not blind this patch adds those statistics. I decided against > dumping proc files wholesale to the hs-err file, I just print out what I > really find useful. > > Note that among other things I print out the total size of outstanding > heap allocations. Note that still the best way to do this is via mallinfo, > and that means the returned value is int and may wrap, see coding and > comment. Even with this caveat though this info is still very useful. > > I did a number of manual tests, the numbers seem legit. > > Looks like this: > > Process Memory: > Virtual Size: 7494372K (peak: 7559908K) > Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: 44804K, > shmem: 0K) > Swapped out: 0K > C-Heap outstanding allocations: 37845K > > > Here I simulate a C-Heap memory leak: > > Process Memory: > Virtual Size: 9062260K (peak: 9118772K) > Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, file: > 44904K, shmem: 0K) > Swapped out: 0K > C-Heap outstanding allocations: 1291984K > > > Same leak, with the mallinfo value wrapped around: > > Process Memory: > Virtual Size: 12343352K (peak: 12445556K) > Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, file: > 45032K, shmem: 0K) > Swapped out: 0K > C-Heap outstanding allocations: 15454K (may have wrapped) > > Thanks, Thomas > > From thomas.stuefe at gmail.com Sat Aug 8 06:15:05 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 8 Aug 2020 08:15:05 +0200 Subject: RFR(xxs): 8251257: NMT: jcmd VM.native_memory scale=1 crashes target VM In-Reply-To: References: Message-ID: Hi, may I have a second Review please? All tests at SAP went through successfully, as did the jdk-submit tests at Oracle. Thanks! ..Thomas On Thu, Aug 6, 2020 at 5:07 PM Thomas St?fe wrote: > Hi, > > very simple small fix. > > jcmd VM.native_memory allows for scale=1 to print exact byte values, but > crashes when printing. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251257 > webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8251257-nmt-scale-1-crash/webrev.00/webrev/ > > Thanks, Thomas > From david.holmes at oracle.com Sat Aug 8 12:32:28 2020 From: david.holmes at oracle.com (David Holmes) Date: Sat, 8 Aug 2020 22:32:28 +1000 Subject: RFR(xxs): 8251257: NMT: jcmd VM.native_memory scale=1 crashes target VM In-Reply-To: References: Message-ID: <42f4f7db-6580-f1a8-78da-a928f1c19a7b@oracle.com> Hi Thomas, Wouldn't a scale name of "b" be more appropriate given the acceptance of "scale=b"? It seems weird to me that scale=1 is even accepted in the first place. And can you please add some information to the bug report. Thanks, David On 8/08/2020 4:15 pm, Thomas St?fe wrote: > Hi, > > may I have a second Review please? > > All tests at SAP went through successfully, as did the jdk-submit tests at > Oracle. > > Thanks! > > ..Thomas > > On Thu, Aug 6, 2020 at 5:07 PM Thomas St?fe wrote: > >> Hi, >> >> very simple small fix. >> >> jcmd VM.native_memory allows for scale=1 to print exact byte values, but >> crashes when printing. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8251257 >> webrev: >> http://cr.openjdk.java.net/~stuefe/webrevs/8251257-nmt-scale-1-crash/webrev.00/webrev/ >> >> Thanks, Thomas >> From thomas.stuefe at gmail.com Sat Aug 8 15:01:11 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 8 Aug 2020 17:01:11 +0200 Subject: RFR(xxs): 8251257: NMT: jcmd VM.native_memory scale=1 crashes target VM In-Reply-To: <42f4f7db-6580-f1a8-78da-a928f1c19a7b@oracle.com> References: <42f4f7db-6580-f1a8-78da-a928f1c19a7b@oracle.com> Message-ID: Hi David, On Sat, Aug 8, 2020 at 2:32 PM David Holmes wrote: > Hi Thomas, > > Wouldn't a scale name of "b" be more appropriate given the acceptance of > "scale=b"? It seems weird to me that scale=1 is even accepted in the > first place. > I am not sure what you refer to, the added case in the switch construct? If yes, this is the string of the unit NMT prints its numbers with. For byte sizes, we normally omit the unit, so I do it here too. If you object against the name of the scale as used in the scale argument, that has been there before and changing it is not subject of the patch. I just don't want the target VM to crash. > And can you please add some information to the bug report. > > Done. I thought subject + assert printout would be sufficient, sorry for the brevity. > Thanks, > David > Thank you, Thomas > > On 8/08/2020 4:15 pm, Thomas St?fe wrote: > > Hi, > > > > may I have a second Review please? > > > > All tests at SAP went through successfully, as did the jdk-submit tests > at > > Oracle. > > > > Thanks! > > > > ..Thomas > > > > On Thu, Aug 6, 2020 at 5:07 PM Thomas St?fe > wrote: > > > >> Hi, > >> > >> very simple small fix. > >> > >> jcmd VM.native_memory allows for scale=1 to print exact byte values, but > >> crashes when printing. > >> > >> JBS: https://bugs.openjdk.java.net/browse/JDK-8251257 > >> webrev: > >> > http://cr.openjdk.java.net/~stuefe/webrevs/8251257-nmt-scale-1-crash/webrev.00/webrev/ > >> > >> Thanks, Thomas > >> > From david.holmes at oracle.com Sun Aug 9 06:28:59 2020 From: david.holmes at oracle.com (David Holmes) Date: Sun, 9 Aug 2020 16:28:59 +1000 Subject: RFR(xxs): 8251257: NMT: jcmd VM.native_memory scale=1 crashes target VM In-Reply-To: References: <42f4f7db-6580-f1a8-78da-a928f1c19a7b@oracle.com> Message-ID: On 9/08/2020 1:01 am, Thomas St?fe wrote: > Hi David, > > On Sat, Aug 8, 2020 at 2:32 PM David Holmes > wrote: > > Hi Thomas, > > Wouldn't a scale name of "b" be more appropriate given the > acceptance of > "scale=b"? It seems weird to me that scale=1 is even accepted in the > first place. > > > I am not sure what you refer to, the added case in the switch construct? Yes. I expected to see 37 switch(scale) { 38 case B: return "B"; 39 case K: return "KB"; 40 case M: return "MB"; 41 case G: return "GB"; 42 } ... > If yes, this is the string of the unit NMT prints its numbers with. For > byte sizes, we normally omit the unit, so I do it here too. ... but if the convention is to omit the unit for bytes then that is fine. The use of "1" in the switch rather than B made this look even odder to me. And I recognise that for some reason the code uses the alias "1" instead of "B" as the primary way to refer to the "byte" scale. > If you object against the name of the scale as used in the scale > argument, that has been there before and changing it is not subject of > the patch. I just don't want the target VM to crash. Sorry I didn't intend to suggest that you change it, just commenting that it seems quite odd to me and inconsistent with the other scale values. > > And can you please add some information to the bug report. > > > Done. I thought subject + assert printout would be sufficient, sorry for > the brevity. Thanks. David > Thanks, > David > > > Thank you, Thomas > > > On 8/08/2020 4:15 pm, Thomas St?fe wrote: > > Hi, > > > > may I have a second Review please? > > > > All tests at SAP went through successfully, as did the jdk-submit > tests at > > Oracle. > > > > Thanks! > > > > ..Thomas > > > > On Thu, Aug 6, 2020 at 5:07 PM Thomas St?fe > > wrote: > > > >> Hi, > >> > >> very simple small fix. > >> > >> jcmd VM.native_memory allows for scale=1 to print exact byte > values, but > >> crashes when printing. > >> > >> JBS: https://bugs.openjdk.java.net/browse/JDK-8251257 > >> webrev: > >> > http://cr.openjdk.java.net/~stuefe/webrevs/8251257-nmt-scale-1-crash/webrev.00/webrev/ > >> > >> Thanks, Thomas > >> > From thomas.stuefe at gmail.com Sun Aug 9 06:36:28 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sun, 9 Aug 2020 08:36:28 +0200 Subject: RFR(xxs): 8251257: NMT: jcmd VM.native_memory scale=1 crashes target VM In-Reply-To: References: <42f4f7db-6580-f1a8-78da-a928f1c19a7b@oracle.com> Message-ID: On Sun, Aug 9, 2020 at 8:29 AM David Holmes wrote: > On 9/08/2020 1:01 am, Thomas St?fe wrote: > > Hi David, > > > > On Sat, Aug 8, 2020 at 2:32 PM David Holmes > > wrote: > > > > Hi Thomas, > > > > Wouldn't a scale name of "b" be more appropriate given the > > acceptance of > > "scale=b"? It seems weird to me that scale=1 is even accepted in the > > first place. > > > > > > I am not sure what you refer to, the added case in the switch construct? > > Yes. I expected to see > > 37 switch(scale) { > 38 case B: return "B"; > 39 case K: return "KB"; > 40 case M: return "MB"; > 41 case G: return "GB"; > 42 } > > ... > > > If yes, this is the string of the unit NMT prints its numbers with. For > > byte sizes, we normally omit the unit, so I do it here too. > > ... but if the convention is to omit the unit for bytes then that is > fine. The use of "1" in the switch rather than B made this look even > odder to me. And I recognise that for some reason the code uses the > alias "1" instead of "B" as the primary way to refer to the "byte" scale. > > Just looked at the history and I added "1" and "b" myself as values in 2018 when I unified scale handling between VM.metaspace and VM.native_memory (JDK-8201572). If it bothers you we can change this but I do not see any pressing need; for me "1" works, I think it makes sense. > > If you object against the name of the scale as used in the scale > > argument, that has been there before and changing it is not subject of > > the patch. I just don't want the target VM to crash. > > Sorry I didn't intend to suggest that you change it, just commenting > that it seems quite odd to me and inconsistent with the other scale values. > > > > > And can you please add some information to the bug report. > > > > > > Done. I thought subject + assert printout would be sufficient, sorry for > > the brevity. > > Thanks. > > David > > So, reviewed from your side? ..Thomas > > Thanks, > > David > > > > > > Thank you, Thomas > > > > > > On 8/08/2020 4:15 pm, Thomas St?fe wrote: > > > Hi, > > > > > > may I have a second Review please? > > > > > > All tests at SAP went through successfully, as did the jdk-submit > > tests at > > > Oracle. > > > > > > Thanks! > > > > > > ..Thomas > > > > > > On Thu, Aug 6, 2020 at 5:07 PM Thomas St?fe > > > wrote: > > > > > >> Hi, > > >> > > >> very simple small fix. > > >> > > >> jcmd VM.native_memory allows for scale=1 to print exact byte > > values, but > > >> crashes when printing. > > >> > > >> JBS: https://bugs.openjdk.java.net/browse/JDK-8251257 > > >> webrev: > > >> > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251257-nmt-scale-1-crash/webrev.00/webrev/ > > >> > > >> Thanks, Thomas > > >> > > > From david.holmes at oracle.com Sun Aug 9 06:45:42 2020 From: david.holmes at oracle.com (David Holmes) Date: Sun, 9 Aug 2020 16:45:42 +1000 Subject: RFR(xxs): 8251257: NMT: jcmd VM.native_memory scale=1 crashes target VM In-Reply-To: References: <42f4f7db-6580-f1a8-78da-a928f1c19a7b@oracle.com> Message-ID: Hi Thomas, Yes all reviewed. No need for any follow ups. Thanks, David On 9/08/2020 4:36 pm, Thomas St?fe wrote: > > > On Sun, Aug 9, 2020 at 8:29 AM David Holmes > wrote: > > On 9/08/2020 1:01 am, Thomas St?fe wrote: > > Hi David, > > > > On Sat, Aug 8, 2020 at 2:32 PM David Holmes > > > >> wrote: > > > >? ? ?Hi Thomas, > > > >? ? ?Wouldn't a scale name of "b" be more appropriate given the > >? ? ?acceptance of > >? ? ?"scale=b"? It seems weird to me that scale=1 is even accepted > in the > >? ? ?first place. > > > > > > I am not sure what you refer to, the added case in the switch > construct? > > Yes. I expected to see > > ? ?37? ?switch(scale) { > ? ?38? ? ?case B: return "B"; > ? ?39? ? ?case K: return "KB"; > ? ?40? ? ?case M: return "MB"; > ? ?41? ? ?case G: return "GB"; > ? ?42? ?} > > ... > > > If yes, this is the string of the unit NMT prints its numbers > with. For > > byte sizes, we normally omit the unit, so I do it here too. > > ... but if the convention is to omit the unit for bytes then that is > fine. The use of "1" in the switch rather than B made this look even > odder to me. And I recognise that for some reason the code uses the > alias "1" instead of "B" as the primary way to refer to the "byte" > scale. > > > Just looked at the history and I added "1" and "b" myself as values in > 2018 when I unified scale handling between VM.metaspace and > VM.native_memory (JDK-8201572). > > If it bothers you we can change this but I do not see any pressing need; > for me "1" works, I think it makes sense. > > > If you object against the name of the scale as used in the scale > > argument, that has been there before and changing it is not > subject of > > the patch. I just don't want the target VM to crash. > > Sorry I didn't intend to suggest that you change it, just commenting > that it seems quite odd to me and inconsistent with the other scale > values. > > > > >? ? ?And can you please add some information to the bug report. > > > > > > Done. I thought subject + assert printout would be sufficient, > sorry for > > the brevity. > > Thanks. > > David > > > So, reviewed from your side? > > ..Thomas > > >? ? ?Thanks, > >? ? ?David > > > > > > Thank you, Thomas > > > > > >? ? ?On 8/08/2020 4:15 pm, Thomas St?fe wrote: > >? ? ? > Hi, > >? ? ? > > >? ? ? > may I have a second Review please? > >? ? ? > > >? ? ? > All tests at SAP went through successfully, as did the > jdk-submit > >? ? ?tests at > >? ? ? > Oracle. > >? ? ? > > >? ? ? > Thanks! > >? ? ? > > >? ? ? > ..Thomas > >? ? ? > > >? ? ? > On Thu, Aug 6, 2020 at 5:07 PM Thomas St?fe > >? ? ? > >> > wrote: > >? ? ? > > >? ? ? >> Hi, > >? ? ? >> > >? ? ? >> very simple small fix. > >? ? ? >> > >? ? ? >> jcmd VM.native_memory allows for scale=1 to print exact byte > >? ? ?values, but > >? ? ? >> crashes when printing. > >? ? ? >> > >? ? ? >> JBS: https://bugs.openjdk.java.net/browse/JDK-8251257 > >? ? ? >> webrev: > >? ? ? >> > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251257-nmt-scale-1-crash/webrev.00/webrev/ > >? ? ? >> > >? ? ? >> Thanks, Thomas > >? ? ? >> > > > From thomas.stuefe at gmail.com Sun Aug 9 07:04:08 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sun, 9 Aug 2020 09:04:08 +0200 Subject: RFR(xxs): 8251257: NMT: jcmd VM.native_memory scale=1 crashes target VM In-Reply-To: References: <42f4f7db-6580-f1a8-78da-a928f1c19a7b@oracle.com> Message-ID: Thank you David. On Sun, Aug 9, 2020 at 8:45 AM David Holmes wrote: > Hi Thomas, > > Yes all reviewed. > > No need for any follow ups. > > Thanks, > David > > On 9/08/2020 4:36 pm, Thomas St?fe wrote: > > > > > > On Sun, Aug 9, 2020 at 8:29 AM David Holmes > > wrote: > > > > On 9/08/2020 1:01 am, Thomas St?fe wrote: > > > Hi David, > > > > > > On Sat, Aug 8, 2020 at 2:32 PM David Holmes > > > > > > >> wrote: > > > > > > Hi Thomas, > > > > > > Wouldn't a scale name of "b" be more appropriate given the > > > acceptance of > > > "scale=b"? It seems weird to me that scale=1 is even accepted > > in the > > > first place. > > > > > > > > > I am not sure what you refer to, the added case in the switch > > construct? > > > > Yes. I expected to see > > > > 37 switch(scale) { > > 38 case B: return "B"; > > 39 case K: return "KB"; > > 40 case M: return "MB"; > > 41 case G: return "GB"; > > 42 } > > > > ... > > > > > If yes, this is the string of the unit NMT prints its numbers > > with. For > > > byte sizes, we normally omit the unit, so I do it here too. > > > > ... but if the convention is to omit the unit for bytes then that is > > fine. The use of "1" in the switch rather than B made this look even > > odder to me. And I recognise that for some reason the code uses the > > alias "1" instead of "B" as the primary way to refer to the "byte" > > scale. > > > > > > Just looked at the history and I added "1" and "b" myself as values in > > 2018 when I unified scale handling between VM.metaspace and > > VM.native_memory (JDK-8201572). > > > > If it bothers you we can change this but I do not see any pressing need; > > for me "1" works, I think it makes sense. > > > > > If you object against the name of the scale as used in the scale > > > argument, that has been there before and changing it is not > > subject of > > > the patch. I just don't want the target VM to crash. > > > > Sorry I didn't intend to suggest that you change it, just commenting > > that it seems quite odd to me and inconsistent with the other scale > > values. > > > > > > > > And can you please add some information to the bug report. > > > > > > > > > Done. I thought subject + assert printout would be sufficient, > > sorry for > > > the brevity. > > > > Thanks. > > > > David > > > > > > So, reviewed from your side? > > > > ..Thomas > > > > > Thanks, > > > David > > > > > > > > > Thank you, Thomas > > > > > > > > > On 8/08/2020 4:15 pm, Thomas St?fe wrote: > > > > Hi, > > > > > > > > may I have a second Review please? > > > > > > > > All tests at SAP went through successfully, as did the > > jdk-submit > > > tests at > > > > Oracle. > > > > > > > > Thanks! > > > > > > > > ..Thomas > > > > > > > > On Thu, Aug 6, 2020 at 5:07 PM Thomas St?fe > > > > > >> > > wrote: > > > > > > > >> Hi, > > > >> > > > >> very simple small fix. > > > >> > > > >> jcmd VM.native_memory allows for scale=1 to print exact > byte > > > values, but > > > >> crashes when printing. > > > >> > > > >> JBS: https://bugs.openjdk.java.net/browse/JDK-8251257 > > > >> webrev: > > > >> > > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251257-nmt-scale-1-crash/webrev.00/webrev/ > > > >> > > > >> Thanks, Thomas > > > >> > > > > > > From david.holmes at oracle.com Mon Aug 10 02:03:35 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Aug 2020 12:03:35 +1000 Subject: RFR: 8249451: Unconditional exceptions clearing logic in compiler code should honor Async Exceptions In-Reply-To: References: Message-ID: <442caa21-ca0a-f6eb-60a5-1e74bf994894@oracle.com> Hi Jamsheed, On 6/08/2020 10:07 pm, Jamsheed C M wrote: > Hi all, > > JBS: https://bugs.openjdk.java.net/browse/JDK-8249451 > > webrev: http://cr.openjdk.java.net/~jcm/8249451/webrev.00/ Thanks for tackling this messy issue. Overall I like the use of TRAPS to more clearly document which methods can return with an exception pending. I think there are some problems with the proposed changes. I'll start with those comments and then move on to more general comments. src/hotspot/share/utilities/exceptions.cpp src/hotspot/share/utilities/exceptions.hpp I don't think the changes here are correct or safe in general. First, adding the new macro and function to only clear non-async exceptions is fine itself. But naming wise the fact only non-async exceptions are cleared should be evident, and there is no "check" involved (in the sense of the existing CHECK_ macros) so I suggest: s/CHECK_CLEAR_PENDING_EXCEPTION/CLEAR_PENDING_NONASYNC_EXCEPTIONS/ s/check_clear_pending_exception/clear_pending_nonasync_exceptions/ But changing the existing CHECK_AND_CLEAR macros to now leave async exceptions pending seems potentially dangerous as calling code may not be prepared for there to now be a pending exception. For example the use in thread.cpp: JDK_Version::set_runtime_name(get_java_runtime_name(THREAD)); JDK_Version::set_runtime_version(get_java_runtime_version(THREAD)); get_java_runtime_name() is currently guaranteed to clear all exceptions, so all the other code is known to be safe to call. But that would no longer be true. That said, this is VM initialization code and an async exception is impossible at this stage. I think I would rather see CHECK_AND_CLEAR left as-is, and an actual CHECK_AND_CLEAR_NONASYNC introduced for those users of CHECK_AND_CLEAR that can encounter async exceptions and which should not clear them. + if (!_pending_exception->is_a(SystemDictionary::ThreadDeath_klass()) && + _pending_exception->klass() != SystemDictionary::InternalError_klass()) { Flagging all InternalErrors as async exceptions is probably also not correct. I don't see a good solution to this at the moment. I think we would need to introduce a new subclass of InternalError for the unsafe access error case**. Now it may be that all the other InternalError usages are "impossible" in the context of where the new macros are to be used, but that is very difficult to establish or assert. ** Or perhaps we could inject a field that allows the VM to identify instances related to unsafe access errors ... Ideally of course these unsafe access errors would be distinct from the async exception mechanism - something I would still like to pursue. --- General comments ... There is a general change from "JavaThread* thread" to "Thread* THREAD" (or TRAPS) to allow the use of the CHECK macros. This is unfortunate because the fact the thread is restricted to being a JavaThread is no longer evident in the method signatures. That is a flaw with the TRAPS/CHECK mechanism unfortunately :( . But as the methods no longer take a JavaThread* arg, they should assert that THREAD->is_Java_thread(). I will also look at an RFE to have as_JavaThread() to avoid the need for separate assertion checks before casting from "Thread*" to "JavaThread*". Note there's no need to use CHECK when the enclosing method is going to return immediately after the call that contains the CHECK. It just adds unnecessary checking of the exception state. The use of TRAPS shows that the methods may return with an exception pending. I've flagged all such occurrences I spotted below. --- + // Only metaspace OOM is expected. no Java code executed. Nit: s/no/No src/hotspot/share/compiler/compilationPolicy.cpp 410 method_invocation_event(method, CHECK_NULL); 489 CompileBroker::compile_method(m, InvocationEntryBci, comp_level, m, hot_count, CompileTask::Reason_InvocationCount, CHECK); Nit: there's no need to use CHECK here. --- src/hotspot/share/compiler/tieredThresholdPolicy.cpp 504 method_invocation_event(method, inlinee, comp_level, nm, CHECK_NULL); 570 compile(mh, bci, CompLevel_simple, CHECK); 581 compile(mh, bci, CompLevel_simple, CHECK); 595 CompileBroker::compile_method(mh, bci, level, mh, hot_count, CompileTask::Reason_Tiered, CHECK); 1062 compile(mh, InvocationEntryBci, next_level, CHECK); Nit: there's no need to use CHECK here. 814 void TieredThresholdPolicy::create_mdo(const methodHandle& mh, Thread* THREAD) { Thank you for correcting this misuse of the THREAD name on a JavaThread* type. --- src/hotspot/share/interpreter/linkResolver.cpp 128 CompilationPolicy::compile_if_required(selected_method, CHECK); Nit: there's no need to use CHECK here. --- src/hotspot/share/jvmci/compilerRuntime.cpp 260 CompilationPolicy::policy()->event(emh, mh, InvocationEntryBci, InvocationEntryBci, CompLevel_aot, cm, CHECK); 280 nmethod* osr_nm = CompilationPolicy::policy()->event(emh, mh, branch_bci, target_bci, CompLevel_aot, cm, CHECK); Nit: there's no need to use CHECK here. --- src/hotspot/share/jvmci/jvmciRuntime.cpp 102 // Donot clear probable async exceptions. typo: s/Donot/Do not/ --- src/hotspot/share/runtime/deoptimization.cpp 1686 void Deoptimization::load_class_by_index(const constantPoolHandle& constant_pool, int index) { This method should be declared with TRAPS now. 1693 // Donot clear probable Async Exceptions. typo: s/Donot/Do not/ > testing : mach1-5(links in jbs) There is very little existing testing that will actually test the key changes you have made here. You will need to do direct fault-injection testing anywhere you now allow async exceptions to remain, to see if the calling code can tolerate that. It will be difficult to test thoroughly. Thanks again for tackling this difficult problem! David ----- > > While working on JDK-8246381 it was noticed that compilation request > path clears all exceptions(including async) and doesn't propagate[1]. > > Fix: patch restores the propagation behavior for the probable async > exceptions. > > Compilation request path propagate exception as in [2]. MDO and > MethodCounter doesn't expect any exception other than metaspace > OOM(added comments). > > Deoptimization path doesn't clear probable async exceptions and take > unpack_exception path for non uncommontraps. > > Added java_lang_InternalError to well known classes. > > Request for review. > > Best Regards, > > Jamsheed > > [1] w.r.t changes done for JDK-7131259 > > [2] > > ??? (a) > ??? -----> c1_Runtime1.cpp/interpreterRuntime.cpp/compilerRuntime.cpp > ????? | > ?????? ----- compilationPolicy.cpp/tieredThresholdPolicy.cpp > ???????? | > ????????? ------ compileBroker.cpp > > ??? (b) > ??? Xcomp versions > ??? ------> compilationPolicy.cpp > ?????? | > ??????? ------> compileBroker.cpp > > ??? (c) > > ??? Direct call to? compile_method in compileBroker.cpp > > ??? JVMCI bootstrap, whitebox, replayCompile. > > From david.holmes at oracle.com Mon Aug 10 04:12:15 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Aug 2020 14:12:15 +1000 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> Message-ID: <6960c0f3-8cbe-fb80-3529-a1c7fb794f0d@oracle.com> Hi Patricio, First, I confirmed with Valhalla folk that this change is fine to go ahead as a stand-alone change independent of the proposed JEP in this area. I've checked the incremental changes for v2, v3 and v4 and everything seems good to me. I only have a couple of nits with the tests: 54 private static void initTestObjects() { 55 testObjects.add(Character.valueOf('H')); 56 testObjects.add(Boolean.valueOf(true)); 57 Byte byteT = 0x40; 58 testObjects.add(byteT); 59 Short shortT = 0x4000; 60 testObjects.add(shortT); 61 testObjects.add(Integer.valueOf(0x40000000)); 62 testObjects.add(Long.valueOf(0x4000000000000000L)); 63 testObjects.add(Float.valueOf(1.20f)); 64 testObjects.add(Double.valueOf(1.2345)); 65 } Why are Byte and Short treated differently? 74 fatalTests[index] = Stream.of(commonFatalTestsFlags, specificFlags[i], new String[] {"SyncOnPrimitiveWrapperTest$FatalTest", Integer.toString(j)}) 75 .flatMap(Stream::of).toArray(String[]::new); The preferred Java style [1] for stream operations is: fatalTests[index] = Stream.of(commonFatalTestsFlags, specificFlags[i], new String[] {"SyncOnPrimitiveWrapperTest$FatalTest", Integer.toString(j)}) .flatMap(Stream::of) .toArray(String[]::new); Thanks, David ----- [1] As demonstrated here (but widely used in JDK code): http://cr.openjdk.java.net/~alundblad/styleguide/index-v6.html#toc-wrapping-lines On 7/08/2020 4:48 am, Patricio Chilano wrote: > Hi Dan, > > On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: >> I'm peeking ahead to the next webrev... :-) >> >> > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ > : ) > >> General comments: >> ? - check files for copyright year updates. > Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, > c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. > >> src/hotspot/share/runtime/synchronizer.hpp >> ??? No comments. >> >> src/hotspot/share/runtime/synchronizer.cpp >> ??? L507: ? const markWord mark = obj->mark(); >> ??? L508: >> ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && >> obj->klass()->is_box()) { >> ??? L510: ??? return false; >> ??? L511: ? } >> ??????? The new bailout on L509-511 can move above L507. > Moved. > >> L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass >> %s", p2i(obj()), obj->klass()->external_name()); >> ??????? This external_name() call does not have a ResourceMark. > Good catch! I had one in a previous version but then I changed the > conditionals and lost it for the fatal error case. The test worked okay > because for the main JavaThread there is a ResourceMark in > jni_invoke_static() (jni.cpp). > >> src/hotspot/share/logging/logTag.hpp >> ??? No comments. >> >> src/hotspot/share/oops/klass.hpp >> ??? No comments. >> >> src/hotspot/share/utilities/accessFlags.hpp >> ??? No comments. >> >> src/hotspot/share/runtime/globals.hpp >> ??? L814: ???????????? "0: off " >> ??????? Missing a ';' after "off". > Fixed. > >> L816: ???????????? "2: log message to stdout. >> ??????? Perhaps add "(by default)" after "stdout" or >> ??????? don't say where log output is at all. >> >> src/hotspot/share/runtime/arguments.cpp >> ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, true, >> LOG_TAGS(primitivewrappers)); >> ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp >> ??????? since it looks like logging output is configured to 'stdout'. >> ??????? I'm assuming that other log options to put output elsewhere >> ??????? are overridden by this code? > Right. So the logging is done under UL with the tag primitivewrappers. > If that tag is specified in Xlog then this conditional will be skipped > because !log_is_enabled(Info, primitivewrappers) will be false. > >> src/hotspot/share/classfile/systemDictionary.cpp >> ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { >> ??????? nit - s/T_LONG+1/T_LONG + 1/ > Fixed. > >> L2191: _box_klasses[i]->set_prototype_header(markWord::prototype()); >> ??????? I assume we're keeping the prototype_header field when Biased >> Locking >> ??????? goes away? The reason I ask: >> >> ? ? ? ? static markWord prototype() { >> ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); >> ? ? ? ? } >> >> ??????? is because without Biased Locking, do we really need the >> prototype >> ??????? anymore? The initial markWord won't need possible variants... > Yes, I think it can go away unless somebody finds another use for it. > Maybe Valhalla. > >> src/hotspot/share/jfr/metadata/metadata.xml >> ??? L69: ? > ??????? Is the category "Java Application" because it's the application >> ??????? code that did something "wrong" here? Where "application" is >> loosely >> ??????? defined to include the possibility of auto generated code, >> library >> ??????? code and the like? Or perhaps "application" because something >> "above" >> ??????? the "Java Virtual Machine, Runtime" did the "wrong" thing here? > I don't know what the right category should be really. I saw the events > JavaMonitorEnter, JavaMonitorWait and JavaMonitorInflate and thought > this event should fall in the same category they do. > >> src/jdk.jfr/share/conf/jfr/default.jfc >> ??? No comments. >> >> src/jdk.jfr/share/conf/jfr/profile.jfc >> ??? No comments. >> >> test/lib/jdk/test/lib/jfr/EventNames.java >> ??? No comments. >> >> src/hotspot/cpu/aarch64/aarch64.ad >> ??? L3517: ????? __ tbnz(tmp, exact_log2(JVM_ACC_IS_BOX_CLASS), cont); >> >> ??? L3578: ??? __ bind(cont); >> ??? L3579: ??? // flag == EQ indicates success >> ??? L3580: ??? // flag == NE indicates failure >> ??????? If tbnz() branches to "cont" when we have a box class, what's >> ??????? the flag value set to (EQ or NE)? And what set that flag value? >> ??????? The reason I ask is I don't think tbnz() sets condition codes... > Right, it doesn't set condition codes, so I kept the version I had (more > on that below). > >> src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> ??? No comments. >> >> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp >> ??? No comments. >> >> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp >> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >> ??????????? ? ?? ??????????????????????????? Register obj_reg, >> ?????????????????????????????? ? ?? ???????? Register swap_reg, >> ?????????????????????????????????? ? ?? ???? Register tmp_reg, >> ?????????????????????????????????? ? ?? ???? bool swap_reg_contains_mark, >> ?????????????????????????????????? ? ?? ???? Label& done, >> ??????????????????????????????????? ? ?? ??? Label* slow_case, >> BiasedLockingCounters* counters) { >> ??????? I think you've changed the only callers of biased_locking_enter() >> ??????? that cared about the return value with this changeset so it can >> ??????? be changed to a void function. > Ok, this is what I mentioned to David in a previous email. Done. > >> src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp >> ??? No comments. >> >> src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp >> ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), done); >> >> ??? L131: ? bind(done); >> ??? L132: >> ??? L133: ? // At this point flags are set as follows: >> ??? L134: ? //? EQ -> Success >> ??? L135: ? //? NE -> Failure, branch to slow path >> ??????? If tbnz() branches to "done" when we have a box class, what's >> ??????? the flag value set to (EQ or NE)? And what set that flag value? >> ??????? The reason I ask is I don't think tbnz() sets condition codes... > Right. Same as above, I kept the version I had. > >> src/hotspot/cpu/arm/interp_masm_arm.cpp >> ??? No comments. >> >> src/hotspot/cpu/arm/macroAssembler_arm.cpp >> ??? int MacroAssembler::biased_locking_enter(Register obj_reg, >> Register swap_reg, Register tmp_reg, >> ????????????????????? ? ?? ????????????????? bool swap_reg_contains_mark, >> ???????????????????????? ? ?? ?????????????? Register tmp2, >> ????????????????????????? ? ?? ????????????? Label& done, Label& >> slow_case, >> BiasedLockingCounters* counters) { >> ??????? I think you've changed the only callers of biased_locking_enter() >> ??????? that cared about the return value with this changeset so it can >> ??????? be changed to a void function. > Done. > >> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >> ??? No comments on the PPC code. >> >> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >> src/hotspot/cpu/s390/interp_masm_s390.cpp >> src/hotspot/cpu/s390/macroAssembler_s390.cpp >> ??? No comments on the S390 code. >> >> src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp >> ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); >> ??????? What will this do with a 'noreg' value? (I need a refresher.) > When not in 64 bit mode that register just won't be used. > >> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp >> ??? No comments. >> >> src/hotspot/cpu/x86/interp_masm_x86.cpp >> ??? No comments. >> >> src/hotspot/cpu/x86/macroAssembler_x86.cpp >> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >> ????????????????????????????????? ? ?? ????? Register obj_reg, >> ????????????????????????????????? ? ?? ????? Register swap_reg, >> ????????????????????????????????? ? ?? ????? Register tmp_reg, >> ?????????????????????????????????? ? ?? ???? Register tmp_reg2, >> ?????????????????????????????????????? ? ? ? bool swap_reg_contains_mark, >> ?????????????????????????????????? ? ?? ???? Label& done, >> ?????????????????????????????????? ? ?? ???? Label* slow_case, >> BiasedLockingCounters* counters) { >> ??????? I think you've changed the only caller of biased_locking_enter() >> ??????? that cared about the return value with this changeset so it can >> ??????? be changed to a void function. > Done. > >> test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >> ??? L30:? * @test SyncOnPrimitiveWrapperTest >> ??????? No parameter to @test directive. > Removed. > >> L136: ??????? private static long sharedCounter = 0L; >> ??????? Since you don't do anything with sharedCounter other than >> increment it, >> ??????? can the compilers optimize it away? If it can be optimized >> away, does >> ??????? that mean that: >> >> ??????????? L142: ??????????????? synchronized (obj) { >> >> ??????? can also be optimized away? >> >> ??????? I don't think that: >> >> ??????????? L161: ??????????????? synchronized (sharedLock1) { >> >> ??????? can be optimized away because it is shared by multiple threads. >> >> test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java >> ??? Similar questions about 'counter' being optimized away. >> ??? Similar question about "synchronized (obj) {" being optimized away. > I'm not sure if the compiler will optimize it away. Seems unlikely given > the counter we are incrementing is not just local to some thread. > > > Ok, below is v3 which has the following changes: > - Use a 32 bit load for the _access_flags field, instead of 64 > - Martin's implementation for s390 and fix for PPC > - Faster LogTest version > - Above changes based on Dan review > > I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 only) again > with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I checked it builds in > arm32, ppc and s390. > > Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev > Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev > > > @Martin: > Please check if the test doesn't timeout for you now with the changes I > made to LogTest. > Also, I tried to use tbnz in aarch64 and arm32 instead of tst + br > (except for c2 since we actually need to set the condition flags), but > for c1 I was getting an assertion in the compiler thread: > > guarantee(chk == -1 || chk == 0) failed: Field too big for insn > > This is the stack when that happens: > > MacroAssembler::pd_patch_instruction_size(unsigned char*, unsigned > char*)+0x398 > AbstractAssembler::bind(Label&)+0x118 > MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 > LIR_Assembler::emit_slow_case_stubs()+0x54 > Compilation::emit_code_body()+0x17c > > The change was simply: > > diff --git a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp > b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp > --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp > +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp > @@ -78,7 +78,6 @@ > ?? if (DiagnoseSyncOnPrimitiveWrappers != 0) { > ???? load_klass(hdr, obj); > -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); > -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); > -??? br(Assembler::NE, slow_case); > +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); > +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); > ?? } > > So the issue must be related to where we want to jump. > > Thanks, > Patricio >> Dan >> >> >> On 8/5/20 9:01 AM, Patricio Chilano wrote: >>> Hi Martin, >>> >>> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>>> Hi Patricio, >>>> >>>> using 8 Byte load instructions for jint fields is a terrible coding >>>> style! >>>> Someone else may see it and use an 8 Byte store. Will result in >>>> great fun for debugging! >>>> >>>> And on Big Endian you will simply access the wrong bits. >>> Ah, of course! Those 32 bits will be in the wrong place when doing >>> the test. >>> >>>> Note that Big Endian Platforms are AIX, old linux ppc, s390, SPARC. >>>> I don't think you have tested on them. >>>> >>>>> We could remove the nested synchronized statements in the run() method >>>>> so that we don't generate that much logging. We could also lower >>>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>>> with multiple flag combinations. Not sure what we should touch first. >>>>> Maybe the synchronized statements, have only one or two and test that >>>>> first? >>>> Sounds like helpful ideas. Please go ahead and strip things down. >>> Great, I will send v3 later with those changes. >>> >>> Thanks Martin! >>> >>> Patricio >>>> Thanks for taking care of it. >>>> >>>> Best regards, >>>> Martin >>>> >>>> >>>>> -----Original Message----- >>>>> From: Patricio Chilano >>>>> Sent: Dienstag, 4. August 2020 20:26 >>>>> To: Doerr, Martin ; David Holmes >>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>> hotspot-jfr-dev at openjdk.java.net >>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>> wrappers >>>>> >>>>> Hi Martin, >>>>> >>>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>>> Hi Patricio, >>>>>> >>>>>>> Ok, I'll change it to movl + testl and test it out before sending >>>>>>> v3. >>>>>> Thanks. I forgot to mention arm + aarch64. >>>>>> Aarch64 uses ldrw + tbnz. >>>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>>> >>>>>> I remember having seen the same mistake ?? >>>>>> And nobody noticed it on little Endian platforms. >>>>> Ok, I can use a tbnz instead of test and then a branch on arm32 and >>>>> aarch64. >>>>> Shouldn't a normal ldr in arm32 work fine? >>>>> Also in 64 bits (either x64 or aarch64) I don't see the issue of >>>>> using a >>>>> 64 bit load, besides the fact that we only care about the first 32 >>>>> bits. >>>>> Regardless of the endianness, aren't we masking out the upper part >>>>> when >>>>> we do AND/TEST or if we test a bit in the 0-31 bit range? Otherwise it >>>>> seems it should have failed for one of those platforms in my tests. >>>>> >>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>> specify that and see if that fixes it? Let me know if that works >>>>>>> and I >>>>>>> can add it to the test. >>>>>> That seems to have an effect. But now I'm not patient enough to >>>>>> wait for >>>>> the test to finish. >>>>>> Maybe the problem is that I'm using slow debug builds. >>>>>> But is there a chance to make the test quicker without losing >>>>>> coverage >>>>> significantly? >>>>>> I think the test is too slow this way to run it on a regular >>>>>> basis. We'd need to >>>>> spend dedicated machines for it. >>>>> We could remove the nested synchronized statements in the run() method >>>>> so that we don't generate that much logging. We could also lower >>>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>>> with multiple flag combinations. Not sure what we should touch first. >>>>> Maybe the synchronized statements, have only one or two and test that >>>>> first? >>>>> >>>>> >>>>> Thanks, >>>>> Patricio >>>>>> Best regards, >>>>>> Martin >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Patricio Chilano >>>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>>> To: Doerr, Martin ; David Holmes >>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>>> wrappers >>>>>>> >>>>>>> Hi Martin, >>>>>>> >>>>>>> Thanks for fixing PPC and taking care of s390! >>>>>>> >>>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>>> Hi Patricio, >>>>>>>> >>>>>>>> I suggest to use movl + testl for checking the access flag as >>>>>>>> for other >>>>> access >>>>>>> flags on x64. >>>>>>> Ok, I'll change it to movl + testl and test it out before sending >>>>>>> v3. >>>>>>> >>>>>>>> New version for PPC64 and s390 see below. >>>>>>>> >>>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>>>> expected. >>>>>>> However, I'm getting timeout issues: >>>>>>>> timed out (timeout set to 120000ms, elapsed time including timeout >>>>>>> handling was 122372ms) >>>>>>>> Can we provide more time? >>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>> specify that and see if that fixes it? Let me know if that works >>>>>>> and I >>>>>>> can add it to the test. >>>>>>> >>>>>>> Thanks, >>>>>>> Patricio >>>>>>>> Best regards, >>>>>>>> Martin >>>>>>>> >>>>>>>> >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -107,8 +107,8 @@ >>>>>>>> >>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>> Rscratch); >>>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>>> +??? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>> Rscratch); >>>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> ??????? bne(CCR0, slow_case); >>>>>>>> ????? } >>>>>>>> >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -912,8 +912,8 @@ >>>>>>>> >>>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> ????????? load_klass(tmp, object); >>>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>> +????? testbitdi(CCR0, R0, tmp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> ????????? bne(CCR0, slow_case); >>>>>>>> ??????? } >>>>>>>> >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>>> >>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> ??????? load_klass(temp, oop); >>>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>> +??? testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> ??????? bne(CCR0, cont); >>>>>>>> ????? } >>>>>>>> >>>>>>>> diff -r 77852e129401 >>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -91,6 +91,12 @@ >>>>>>>> ????? // Save object being locked into the BasicObjectLock... >>>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> +??? z_btrue(slow_case); >>>>>>>> +? } >>>>>>>> + >>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>>> Z_R0_scratch, done, >>>>>>> &slow_case); >>>>>>>> ????? } >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 10:03:57 >>>>>>> 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>> 16:04:31 >>>>>>> 2020 +0200 >>>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>>> ????? z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), >>>>>>>> object); >>>>>>>> >>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> +??? z_btrue(slow_case); >>>>>>>> +? } >>>>>>>> + >>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>> ??????? biased_locking_enter(object, displaced_header, Z_R1, >>>>>>>> Z_R0, done, >>>>>>> &slow_case); >>>>>>>> ????? } >>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>> 10:03:57 2020 +0200 >>>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>> 16:04:31 2020 +0200 >>>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>>> ????? // Load markWord from oop into mark. >>>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>>> >>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>> +??? z_btrue(done); >>>>>>>> +? } >>>>>>>> + >>>>>>>> ????? if (try_bias) { >>>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, Z_R0, >>>>>>>> done); >>>>>>>> ????? } >>>>>>>> >>> >> > From david.holmes at oracle.com Mon Aug 10 04:49:47 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Aug 2020 14:49:47 +1000 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info In-Reply-To: References: Message-ID: <6a8d3ac8-15b4-6f2e-6eae-4d3e3efa2b91@oracle.com> Hi Thomas, On 7/08/2020 12:01 am, Thomas St?fe wrote: > Hi, > > may I please have reviews for this addition: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs-err/webrev.00/webrev/ > > When analysing a customer memory leak recently I found for all > the information we have in the error files we miss some really fundamental > statistics like process virtual size and rss. Either that or I am blind :) This doesn't seem unreasonable, though I don't like to see continued divergence between what gets reported on Linux versus other platforms. Also is this something that is reasonable to always do or should it be added to the set of things under ExtensiveErrorReports? > In case I am not blind this patch adds those statistics. I decided against > dumping proc files wholesale to the hs-err file, I just print out what I > really find useful. + FILE* f = ::fopen("/proc/self/status", "r"); + char buf[256]; + while (::fgets(buf, sizeof(buf), f) != NULL && num_found < num_values) { But without counting the characters (and knowing what is fixed-width and what is variable) doesn't this risk getting a partial entry at the end of the buffer such that you won't see it when trying to parse? > Note that among other things I print out the total size of outstanding heap > allocations. Note that still the best way to do this is via mallinfo, and > that means the returned value is int and may wrap, see coding and comment. > Even with this caveat though this info is still very useful. I am less keen on this bit. Now I understand your query in another RFR about how to flag such glibc'isms so they don't impact building against different libs like musl. :) Is mallinfo safe to call from a signal handling context? I mean that in a practical sense (ie it doesn't try to acquire internal malloc locks) rather than being designated as async-signal-safe. Thanks, David ----- > I did a number of manual tests, the numbers seem legit. > > Looks like this: > > Process Memory: > Virtual Size: 7494372K (peak: 7559908K) > Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: 44804K, > shmem: 0K) > Swapped out: 0K > C-Heap outstanding allocations: 37845K > > > Here I simulate a C-Heap memory leak: > > Process Memory: > Virtual Size: 9062260K (peak: 9118772K) > Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, file: 44904K, > shmem: 0K) > Swapped out: 0K > C-Heap outstanding allocations: 1291984K > > > Same leak, with the mallinfo value wrapped around: > > Process Memory: > Virtual Size: 12343352K (peak: 12445556K) > Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, file: 45032K, > shmem: 0K) > Swapped out: 0K > C-Heap outstanding allocations: 15454K (may have wrapped) > > Thanks, Thomas > From thomas.stuefe at gmail.com Mon Aug 10 06:13:20 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 10 Aug 2020 08:13:20 +0200 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info In-Reply-To: <6a8d3ac8-15b4-6f2e-6eae-4d3e3efa2b91@oracle.com> References: <6a8d3ac8-15b4-6f2e-6eae-4d3e3efa2b91@oracle.com> Message-ID: On Mon, Aug 10, 2020 at 6:50 AM David Holmes wrote: > Hi Thomas, > > On 7/08/2020 12:01 am, Thomas St?fe wrote: > > Hi, > > > > may I please have reviews for this addition: > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 > > Webrev: > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs-err/webrev.00/webrev/ > > > > When analysing a customer memory leak recently I found for all > > the information we have in the error files we miss some really > fundamental > > statistics like process virtual size and rss. Either that or I am blind > :) > > This doesn't seem unreasonable, though I don't like to see continued > divergence between what gets reported on Linux versus other platforms. > > Neither do I but Linux is kind of the most important one so I do not feel bad for enhancing only this; and people interested in other platforms can always step in. My problem here is while adding similar printouts for other platforms is trivial, I do not like printing out stuff I do not have investigated and fully understood / tested before. Which would take more time than I have right now to do correctly for all platforms. See also below, my thoughts about mallinfo. > Also is this something that is reasonable to always do or should it be > added to the set of things under ExtensiveErrorReports? > Always, IMHO. There is a lot of other stuff less important which we may shove into ExtensiveErrorReports, but this is very basic. > > In case I am not blind this patch adds those statistics. I decided > against > > dumping proc files wholesale to the hs-err file, I just print out what I > > really find useful. > > + FILE* f = ::fopen("/proc/self/status", "r"); > + char buf[256]; > + while (::fgets(buf, sizeof(buf), f) != NULL && num_found < num_values) > { > > But without counting the characters (and knowing what is fixed-width and > what is variable) doesn't this risk getting a partial entry at the end > of the buffer such that you won't see it when trying to parse? > Not sure if I understand your question. If you mean what happens should the proc file contain a line longer than 256, that is not a problem: fgets stops either on newline or when the buffer is full. We would read the first 256 characters, scan, then the next characters (needlessly), scan (needlessly, not finding anything) and so forth, until we hit a newline. The subsequent call to fgets will start at the next line. Also, the proc file content is no mystery: it is clearly documented in the kernel man pages (see section about /proc/[pid]/status). So I do not have to assume an arbitrary input. > > > Note that among other things I print out the total size of outstanding > heap > > allocations. Note that still the best way to do this is via mallinfo, and > > that means the returned value is int and may wrap, see coding and > comment. > > Even with this caveat though this info is still very useful. > > I am less keen on this bit. Now I understand your query in another RFR > about how to flag such glibc'isms so they don't impact building against > different libs like musl. :) > > Funny enough, I read somewhere that mallinfo is a Posix function; I see it mentioned in very old documents from the 80s. But OpenGroup has no mention of it. About muslc, it is highly probable that it would implement mallinfo(), but the meaning of the content may be different, so the printout should be adjusted for muslc. Same goes probably for all other *nix platforms - we could print out all members, but one should investigate beforehand if their content makes sense and what it means. > Is mallinfo safe to call from a signal handling context? I mean that in > a practical sense (ie it doesn't try to acquire internal malloc locks) > rather than being designated as async-signal-safe. > completely harmless, see int_malloc() in glibc malloc/malloc.c: https://sources.debian.org/src/glibc/2.28-10/malloc/malloc.c/ Does no locks, no calldowns to other functions (check_malloc_state is empty in glibc release builds); Also similar coding has been active in our proprietary VM since about five years so I think this is safe. -- One other question just occurred to me, I wonder whether long is 64bit on all our 64bit linux builds. Or whether I should use an explicit 64bit type. Thanks, Thomas > Thanks, > David > ----- > > > I did a number of manual tests, the numbers seem legit. > > > > Looks like this: > > > > Process Memory: > > Virtual Size: 7494372K (peak: 7559908K) > > Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: 44804K, > > shmem: 0K) > > Swapped out: 0K > > C-Heap outstanding allocations: 37845K > > > > > > Here I simulate a C-Heap memory leak: > > > > Process Memory: > > Virtual Size: 9062260K (peak: 9118772K) > > Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, file: > 44904K, > > shmem: 0K) > > Swapped out: 0K > > C-Heap outstanding allocations: 1291984K > > > > > > Same leak, with the mallinfo value wrapped around: > > > > Process Memory: > > Virtual Size: 12343352K (peak: 12445556K) > > Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, file: > 45032K, > > shmem: 0K) > > Swapped out: 0K > > C-Heap outstanding allocations: 15454K (may have wrapped) > > > > Thanks, Thomas > > > From thomas.stuefe at gmail.com Mon Aug 10 07:32:02 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 10 Aug 2020 09:32:02 +0200 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info In-Reply-To: References: <6a8d3ac8-15b4-6f2e-6eae-4d3e3efa2b91@oracle.com> Message-ID: On Mon, Aug 10, 2020 at 8:13 AM Thomas St?fe wrote: > > > On Mon, Aug 10, 2020 at 6:50 AM David Holmes > wrote: > >> Hi Thomas, >> >> On 7/08/2020 12:01 am, Thomas St?fe wrote: >> > Hi, >> > >> > may I please have reviews for this addition: >> > >> > JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 >> > Webrev: >> > >> http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs-err/webrev.00/webrev/ >> > >> > When analysing a customer memory leak recently I found for all >> > the information we have in the error files we miss some really >> fundamental >> > statistics like process virtual size and rss. Either that or I am blind >> :) >> >> This doesn't seem unreasonable, though I don't like to see continued >> divergence between what gets reported on Linux versus other platforms. >> >> > Neither do I but Linux is kind of the most important one so I do not feel > bad for enhancing only this; and people interested in other platforms can > always step in. > > My problem here is while adding similar printouts for other platforms is > trivial, I do not like printing out stuff I do not have investigated and > fully understood / tested before. Which would take more time than I have > right now to do correctly for all platforms. See also below, my thoughts > about mallinfo. > > >> Also is this something that is reasonable to always do or should it be >> added to the set of things under ExtensiveErrorReports? >> > > Always, IMHO. There is a lot of other stuff less important which we may > shove into ExtensiveErrorReports, but this is very basic. > > >> > In case I am not blind this patch adds those statistics. I decided >> against >> > dumping proc files wholesale to the hs-err file, I just print out what I >> > really find useful. >> >> + FILE* f = ::fopen("/proc/self/status", "r"); >> + char buf[256]; >> + while (::fgets(buf, sizeof(buf), f) != NULL && num_found < >> num_values) { >> >> But without counting the characters (and knowing what is fixed-width and >> what is variable) doesn't this risk getting a partial entry at the end >> of the buffer such that you won't see it when trying to parse? >> > > Not sure if I understand your question. If you mean what happens should > the proc file contain a line longer than 256, that is not a problem: fgets > stops either on newline or when the buffer is full. We would read the first > 256 characters, scan, then the next characters (needlessly), scan > (needlessly, not finding anything) and so forth, until we hit a newline. > The subsequent call to fgets will start at the next line. > > Also, the proc file content is no mystery: it is clearly documented in the > kernel man pages (see section about /proc/[pid]/status). So I do not have > to assume an arbitrary input. > > >> >> > Note that among other things I print out the total size of outstanding >> heap >> > allocations. Note that still the best way to do this is via mallinfo, >> and >> > that means the returned value is int and may wrap, see coding and >> comment. >> > Even with this caveat though this info is still very useful. >> >> I am less keen on this bit. Now I understand your query in another RFR >> about how to flag such glibc'isms so they don't impact building against >> different libs like musl. :) >> >> > Funny enough, I read somewhere that mallinfo is a Posix function; I see it > mentioned in very old documents from the 80s. But OpenGroup has no mention > of it. > > About muslc, it is highly probable that it would implement mallinfo(), but > the meaning of the content may be different, so the printout should be > adjusted for muslc. Same goes probably for all other *nix platforms - we > could print out all members, but one should investigate beforehand if their > content makes sense and what it means. > > >> Is mallinfo safe to call from a signal handling context? I mean that in >> a practical sense (ie it doesn't try to acquire internal malloc locks) >> rather than being designated as async-signal-safe. >> > > completely harmless, see int_malloc() in glibc malloc/malloc.c: > https://sources.debian.org/src/glibc/2.28-10/malloc/malloc.c/ > > s/int_malloc/int_mallinfo https://sources.debian.org/src/glibc/2.28-10/malloc/malloc.c/#L4878 > Does no locks, no calldowns to other functions (check_malloc_state is > empty in glibc release builds); > > Also similar coding has been active in our proprietary VM since about five > years so I think this is safe. > > -- > > One other question just occurred to me, I wonder whether long is 64bit on > all our 64bit linux builds. Or whether I should use an explicit 64bit type. > > Thanks, Thomas > > >> Thanks, >> David >> ----- >> >> > I did a number of manual tests, the numbers seem legit. >> > >> > Looks like this: >> > >> > Process Memory: >> > Virtual Size: 7494372K (peak: 7559908K) >> > Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: 44804K, >> > shmem: 0K) >> > Swapped out: 0K >> > C-Heap outstanding allocations: 37845K >> > >> > >> > Here I simulate a C-Heap memory leak: >> > >> > Process Memory: >> > Virtual Size: 9062260K (peak: 9118772K) >> > Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, file: >> 44904K, >> > shmem: 0K) >> > Swapped out: 0K >> > C-Heap outstanding allocations: 1291984K >> > >> > >> > Same leak, with the mallinfo value wrapped around: >> > >> > Process Memory: >> > Virtual Size: 12343352K (peak: 12445556K) >> > Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, file: >> 45032K, >> > shmem: 0K) >> > Swapped out: 0K >> > C-Heap outstanding allocations: 15454K (may have wrapped) >> > >> > Thanks, Thomas >> > >> > From david.holmes at oracle.com Mon Aug 10 08:11:20 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Aug 2020 18:11:20 +1000 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info In-Reply-To: References: <6a8d3ac8-15b4-6f2e-6eae-4d3e3efa2b91@oracle.com> Message-ID: Hi Thomas, On 10/08/2020 4:13 pm, Thomas St?fe wrote: > On Mon, Aug 10, 2020 at 6:50 AM David Holmes > wrote: > > Hi Thomas, > > On 7/08/2020 12:01 am, Thomas St?fe wrote: > > Hi, > > > > may I please have reviews for this addition: > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 > > Webrev: > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs-err/webrev.00/webrev/ > > > > When analysing a customer memory leak recently I found for all > > the information we have in the error files we miss some really > fundamental > > statistics like process virtual size and rss. Either that or I am > blind :) > > This doesn't seem unreasonable, though I don't like to see continued > divergence between what gets reported on Linux versus other platforms. > > > Neither do I but Linux is kind of the most important one so I do not > feel bad for enhancing only this; and people interested in other > platforms can always step in. Ok. > My problem here is while adding similar printouts for other platforms is > trivial, I do not like printing out stuff I do not have investigated and > fully understood / tested before. Which would take more time than I have > right now to do correctly for all platforms. See also below, my thoughts > about mallinfo. Sure I understand. > Also is this something that is reasonable to always do or should it be > added to the set of things under ExtensiveErrorReports? > > > Always, IMHO. There is a lot of other stuff less important which we may > shove into ExtensiveErrorReports, but this is very basic. Ok. > > > In case I am not blind this patch adds those statistics. I > decided against > > dumping proc files wholesale to the hs-err file, I just print out > what I > > really find useful. > > +? ?FILE* f = ::fopen("/proc/self/status", "r"); > +? ?char buf[256]; > +? ?while (::fgets(buf, sizeof(buf), f) != NULL && num_found < > num_values) { > > But without counting the characters (and knowing what is fixed-width > and > what is variable) doesn't this risk getting a partial entry at the end > of the buffer such that you won't see it when trying to parse? > > > Not sure if I understand your question. If you mean what happens should > the proc file contain a line longer than 256, that is not a problem: > fgets stops either on newline or when the buffer is full. We would read > the first 256 characters, scan, then the next characters (needlessly), > scan (needlessly, not finding anything) and so forth, until we hit a > newline. The subsequent call to fgets will start at the next line. Sorry I was thinking fgets would read 256 chars at a time, not a line of up to 256 chars. > Also, the proc file content is no mystery: it is clearly documented in > the kernel man pages (see section about /proc/[pid]/status). So I do not > have to assume an arbitrary input. > > > > Note that among other things I print out the total size of > outstanding heap > > allocations. Note that still the best way to do this is via > mallinfo, and > > that means the returned value is int and may wrap, see coding and > comment. > > Even with this caveat though this info is still very useful. > > I am less keen on this bit. Now I understand your query in another RFR > about how to flag such glibc'isms so they don't impact building against > different libs like musl. :) > > > Funny enough, I read somewhere that mallinfo is a Posix function; I see > it mentioned in very old documents from the 80s. But OpenGroup has no > mention of it. I read (Linux manpage) it is from System V Interface Definition but I don't see it in the 4th edition PDF. > About muslc, it is highly probable that it would implement mallinfo(), No it doesn't and won't as it is considered fundamentally broken due to the int issue. https://www.openwall.com/lists/musl/2018/01/17/2 > but the meaning of the content may be different, so the printout should > be adjusted for muslc. Same goes probably for all other *nix platforms - > we could print out all members, but one should investigate beforehand if > their content makes sense and what it means. > > Is mallinfo safe to call from a signal handling context? I mean that in > a practical sense (ie it doesn't try to acquire internal malloc locks) > rather than being designated as async-signal-safe. > > > completely harmless, see int_malloc() in glibc malloc/malloc.c: > https://sources.debian.org/src/glibc/2.28-10/malloc/malloc.c/ > > Does no locks, no calldowns to other functions (check_malloc_state is > empty in glibc release builds); Saw your update - ok. > Also similar coding has been active in our proprietary?VM since about > five years so I think this is safe. > > -- > > One other question just occurred to me, I wonder whether long is 64bit > on all our 64bit linux builds. Or whether I should use an explicit 64bit > type. "long" will be 32-bit on 32-bit Linux (ILP32) and 64-bit on 64-bit Linux (LP64). Does /proc/self/status specify what ranges the values may have? Perhaps size_t would be better (even though same size as long) ? Thanks, David ----- > > Thanks, Thomas > > > Thanks, > David > ----- > > > I did a number of manual tests, the numbers seem legit. > > > > Looks like this: > > > > Process Memory: > > Virtual Size: 7494372K (peak: 7559908K) > > Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: > 44804K, > > shmem: 0K) > > Swapped out: 0K > > C-Heap outstanding allocations: 37845K > > > > > > Here I simulate a C-Heap memory leak: > > > > Process Memory: > > Virtual Size: 9062260K (peak: 9118772K) > > Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, > file: 44904K, > > shmem: 0K) > > Swapped out: 0K > > C-Heap outstanding allocations: 1291984K > > > > > > Same leak, with the mallinfo value wrapped around: > > > > Process Memory: > > Virtual Size: 12343352K (peak: 12445556K) > > Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, > file: 45032K, > > shmem: 0K) > > Swapped out: 0K > > C-Heap outstanding allocations: 15454K (may have wrapped) > > > > Thanks, Thomas > > > From thomas.stuefe at gmail.com Mon Aug 10 09:04:47 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 10 Aug 2020 11:04:47 +0200 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info In-Reply-To: References: <6a8d3ac8-15b4-6f2e-6eae-4d3e3efa2b91@oracle.com> Message-ID: Hi David, On Mon, Aug 10, 2020 at 10:11 AM David Holmes wrote: > Hi Thomas, > > On 10/08/2020 4:13 pm, Thomas St?fe wrote: > > On Mon, Aug 10, 2020 at 6:50 AM David Holmes > > wrote: > > > > Hi Thomas, > > > > On 7/08/2020 12:01 am, Thomas St?fe wrote: > > > Hi, > > > > > > may I please have reviews for this addition: > > > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 > > > Webrev: > > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs-err/webrev.00/webrev/ > > > > > > When analysing a customer memory leak recently I found for all > > > the information we have in the error files we miss some really > > fundamental > > > statistics like process virtual size and rss. Either that or I am > > blind :) > > > > This doesn't seem unreasonable, though I don't like to see continued > > divergence between what gets reported on Linux versus other > platforms. > > > > > > Neither do I but Linux is kind of the most important one so I do not > > feel bad for enhancing only this; and people interested in other > > platforms can always step in. > > Ok. > > > My problem here is while adding similar printouts for other platforms is > > trivial, I do not like printing out stuff I do not have investigated and > > fully understood / tested before. Which would take more time than I have > > right now to do correctly for all platforms. See also below, my thoughts > > about mallinfo. > > Sure I understand. > > > Also is this something that is reasonable to always do or should it > be > > added to the set of things under ExtensiveErrorReports? > > > > > > Always, IMHO. There is a lot of other stuff less important which we may > > shove into ExtensiveErrorReports, but this is very basic. > > Ok. > > > > > > In case I am not blind this patch adds those statistics. I > > decided against > > > dumping proc files wholesale to the hs-err file, I just print out > > what I > > > really find useful. > > > > + FILE* f = ::fopen("/proc/self/status", "r"); > > + char buf[256]; > > + while (::fgets(buf, sizeof(buf), f) != NULL && num_found < > > num_values) { > > > > But without counting the characters (and knowing what is fixed-width > > and > > what is variable) doesn't this risk getting a partial entry at the > end > > of the buffer such that you won't see it when trying to parse? > > > > > > Not sure if I understand your question. If you mean what happens should > > the proc file contain a line longer than 256, that is not a problem: > > fgets stops either on newline or when the buffer is full. We would read > > the first 256 characters, scan, then the next characters (needlessly), > > scan (needlessly, not finding anything) and so forth, until we hit a > > newline. The subsequent call to fgets will start at the next line. > > Sorry I was thinking fgets would read 256 chars at a time, not a line of > up to 256 chars. > > > Also, the proc file content is no mystery: it is clearly documented in > > the kernel man pages (see section about /proc/[pid]/status). So I do not > > have to assume an arbitrary input. > > > > > > > Note that among other things I print out the total size of > > outstanding heap > > > allocations. Note that still the best way to do this is via > > mallinfo, and > > > that means the returned value is int and may wrap, see coding and > > comment. > > > Even with this caveat though this info is still very useful. > > > > I am less keen on this bit. Now I understand your query in another > RFR > > about how to flag such glibc'isms so they don't impact building > against > > different libs like musl. :) > > > > > > Funny enough, I read somewhere that mallinfo is a Posix function; I see > > it mentioned in very old documents from the 80s. But OpenGroup has no > > mention of it. > > I read (Linux manpage) it is from System V Interface Definition but I > don't see it in the 4th edition PDF. > > > About muslc, it is highly probable that it would implement mallinfo(), > > No it doesn't and won't as it is considered fundamentally broken due to > the int issue. > > https://www.openwall.com/lists/musl/2018/01/17/2 Good catch, I'll try to work around that. > > > > but the meaning of the content may be different, so the printout should > > be adjusted for muslc. Same goes probably for all other *nix platforms - > > we could print out all members, but one should investigate beforehand if > > their content makes sense and what it means. > > > > Is mallinfo safe to call from a signal handling context? I mean that > in > > a practical sense (ie it doesn't try to acquire internal malloc > locks) > > rather than being designated as async-signal-safe. > > > > > > completely harmless, see int_malloc() in glibc malloc/malloc.c: > > https://sources.debian.org/src/glibc/2.28-10/malloc/malloc.c/ > > > > Does no locks, no calldowns to other functions (check_malloc_state is > > empty in glibc release builds); > > Saw your update - ok. > > > Also similar coding has been active in our proprietary VM since about > > five years so I think this is safe. > > > > -- > > > > One other question just occurred to me, I wonder whether long is 64bit > > on all our 64bit linux builds. Or whether I should use an explicit 64bit > > type. > > "long" will be 32-bit on 32-bit Linux (ILP32) and 64-bit on 64-bit Linux > (LP64). Does /proc/self/status specify what ranges the values may have? > Perhaps size_t would be better (even though same size as long) ? > > I will change the variables to size_t (even though the code will not as compact as it is now). > Thanks, > David > ----- > > I will post a new webrev when this is done and tested. ..Thomas > > > > Thanks, Thomas > > > > > > Thanks, > > David > > ----- > > > > > I did a number of manual tests, the numbers seem legit. > > > > > > Looks like this: > > > > > > Process Memory: > > > Virtual Size: 7494372K (peak: 7559908K) > > > Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: > > 44804K, > > > shmem: 0K) > > > Swapped out: 0K > > > C-Heap outstanding allocations: 37845K > > > > > > > > > Here I simulate a C-Heap memory leak: > > > > > > Process Memory: > > > Virtual Size: 9062260K (peak: 9118772K) > > > Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, > > file: 44904K, > > > shmem: 0K) > > > Swapped out: 0K > > > C-Heap outstanding allocations: 1291984K > > > > > > > > > Same leak, with the mallinfo value wrapped around: > > > > > > Process Memory: > > > Virtual Size: 12343352K (peak: 12445556K) > > > Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, > > file: 45032K, > > > shmem: 0K) > > > Swapped out: 0K > > > C-Heap outstanding allocations: 15454K (may have wrapped) > > > > > > Thanks, Thomas > > > > > > From thomas.stuefe at gmail.com Mon Aug 10 11:32:51 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 10 Aug 2020 13:32:51 +0200 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info In-Reply-To: References: Message-ID: Hi, new version: http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs-err/webrev.01/webrev/ - changed the type of the receiver variables to ssize_t - enclosed mallinfo call with __GLIBC__ Thanks, Thomas On Thu, Aug 6, 2020 at 4:01 PM Thomas St?fe wrote: > Hi, > > may I please have reviews for this addition: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs-err/webrev.00/webrev/ > > When analysing a customer memory leak recently I found for all > the information we have in the error files we miss some really fundamental > statistics like process virtual size and rss. Either that or I am blind :) > > In case I am not blind this patch adds those statistics. I decided against > dumping proc files wholesale to the hs-err file, I just print out what I > really find useful. > > Note that among other things I print out the total size of outstanding > heap allocations. Note that still the best way to do this is via mallinfo, > and that means the returned value is int and may wrap, see coding and > comment. Even with this caveat though this info is still very useful. > > I did a number of manual tests, the numbers seem legit. > > Looks like this: > > Process Memory: > Virtual Size: 7494372K (peak: 7559908K) > Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: 44804K, > shmem: 0K) > Swapped out: 0K > C-Heap outstanding allocations: 37845K > > > Here I simulate a C-Heap memory leak: > > Process Memory: > Virtual Size: 9062260K (peak: 9118772K) > Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, file: > 44904K, shmem: 0K) > Swapped out: 0K > C-Heap outstanding allocations: 1291984K > > > Same leak, with the mallinfo value wrapped around: > > Process Memory: > Virtual Size: 12343352K (peak: 12445556K) > Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, file: > 45032K, shmem: 0K) > Swapped out: 0K > C-Heap outstanding allocations: 15454K (may have wrapped) > > Thanks, Thomas > > From david.holmes at oracle.com Mon Aug 10 11:39:53 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 10 Aug 2020 21:39:53 +1000 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info In-Reply-To: References: Message-ID: Hi Thomas, This seems fine to me. Thanks, David On 10/08/2020 9:32 pm, Thomas St?fe wrote: > Hi, > > new version: > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs-err/webrev.01/webrev/ > > - changed the type of the receiver variables to ssize_t > - enclosed mallinfo call with __GLIBC__ > > Thanks, Thomas > > On Thu, Aug 6, 2020 at 4:01 PM Thomas St?fe > wrote: > > Hi, > > may I please have reviews for this addition: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 > Webrev: > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs-err/webrev.00/webrev/ > > When analysing a customer memory leak recently I found for?all > the?information we have in the error files we miss some really > fundamental statistics like process virtual size and rss. Either > that or I am blind :) > > In case I am not blind this patch?adds those statistics. I decided > against dumping proc files wholesale to the hs-err file, I > just?print out what I really find useful. > > Note that among other?things I print out the total size of > outstanding heap allocations. Note that still the best way to do > this is via mallinfo, and that means the returned value is int and > may wrap, see coding and comment. Even with this caveat though this > info is still very useful. > > I did a number of manual tests, the numbers seem legit. > > Looks like this: > > Process Memory: > Virtual Size: 7494372K (peak: 7559908K) > Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: > 44804K, shmem: 0K) > Swapped out: 0K > C-Heap outstanding allocations: 37845K > > > Here I simulate a C-Heap memory leak: > > Process Memory: > Virtual Size: 9062260K (peak: 9118772K) > Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, file: > 44904K, shmem: 0K) > Swapped out: 0K > C-Heap outstanding allocations: 1291984K > > > Same leak, with the mallinfo value wrapped around: > > Process Memory: > Virtual Size: 12343352K (peak: 12445556K) > Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, file: > 45032K, shmem: 0K) > Swapped out: 0K > C-Heap outstanding allocations: 15454K (may have wrapped) > > Thanks, Thomas > From thomas.stuefe at gmail.com Mon Aug 10 11:51:57 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 10 Aug 2020 13:51:57 +0200 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info In-Reply-To: References: Message-ID: Thanks for the review, David! On Mon, Aug 10, 2020 at 1:42 PM David Holmes wrote: > Hi Thomas, > > This seems fine to me. > > Thanks, > David > > On 10/08/2020 9:32 pm, Thomas St?fe wrote: > > Hi, > > > > new version: > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs-err/webrev.01/webrev/ > > > > - changed the type of the receiver variables to ssize_t > > - enclosed mallinfo call with __GLIBC__ > > > > Thanks, Thomas > > > > On Thu, Aug 6, 2020 at 4:01 PM Thomas St?fe > > wrote: > > > > Hi, > > > > may I please have reviews for this addition: > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 > > Webrev: > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs-err/webrev.00/webrev/ > > > > When analysing a customer memory leak recently I found for all > > the information we have in the error files we miss some really > > fundamental statistics like process virtual size and rss. Either > > that or I am blind :) > > > > In case I am not blind this patch adds those statistics. I decided > > against dumping proc files wholesale to the hs-err file, I > > just print out what I really find useful. > > > > Note that among other things I print out the total size of > > outstanding heap allocations. Note that still the best way to do > > this is via mallinfo, and that means the returned value is int and > > may wrap, see coding and comment. Even with this caveat though this > > info is still very useful. > > > > I did a number of manual tests, the numbers seem legit. > > > > Looks like this: > > > > Process Memory: > > Virtual Size: 7494372K (peak: 7559908K) > > Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: > > 44804K, shmem: 0K) > > Swapped out: 0K > > C-Heap outstanding allocations: 37845K > > > > > > Here I simulate a C-Heap memory leak: > > > > Process Memory: > > Virtual Size: 9062260K (peak: 9118772K) > > Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, file: > > 44904K, shmem: 0K) > > Swapped out: 0K > > C-Heap outstanding allocations: 1291984K > > > > > > Same leak, with the mallinfo value wrapped around: > > > > Process Memory: > > Virtual Size: 12343352K (peak: 12445556K) > > Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, file: > > 45032K, shmem: 0K) > > Swapped out: 0K > > C-Heap outstanding allocations: 15454K (may have wrapped) > > > > Thanks, Thomas > > > From martin.doerr at sap.com Mon Aug 10 12:24:43 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 10 Aug 2020 12:24:43 +0000 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info In-Reply-To: References: Message-ID: Hi Thomas, looks like a really helpful contribution. Minor nit: You print most values in "kB", but some other ones in "K". I think "kB" should be used consistently. I don't think kernel < 2.6.34 is still supported. I don't care if you remove the check or not. Otherwise thumbs up from my side. I don't need to see another webrev. Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev > On Behalf Of Thomas St?fe > Sent: Montag, 10. August 2020 13:52 > To: David Holmes > Cc: Hotspot dev runtime > Subject: Re: RFR(xs): 8251255: [linux] Add process-memory information to > hs-err and VM.info > > Thanks for the review, David! > > On Mon, Aug 10, 2020 at 1:42 PM David Holmes > wrote: > > > Hi Thomas, > > > > This seems fine to me. > > > > Thanks, > > David > > > > On 10/08/2020 9:32 pm, Thomas St?fe wrote: > > > Hi, > > > > > > new version: > > > > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs- > err/webrev.01/webrev/ > > > > > > - changed the type of the receiver variables to ssize_t > > > - enclosed mallinfo call with __GLIBC__ > > > > > > Thanks, Thomas > > > > > > On Thu, Aug 6, 2020 at 4:01 PM Thomas St?fe > > > > wrote: > > > > > > Hi, > > > > > > may I please have reviews for this addition: > > > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 > > > Webrev: > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs- > err/webrev.00/webrev/ > > > > > > When analysing a customer memory leak recently I found for all > > > the information we have in the error files we miss some really > > > fundamental statistics like process virtual size and rss. Either > > > that or I am blind :) > > > > > > In case I am not blind this patch adds those statistics. I decided > > > against dumping proc files wholesale to the hs-err file, I > > > just print out what I really find useful. > > > > > > Note that among other things I print out the total size of > > > outstanding heap allocations. Note that still the best way to do > > > this is via mallinfo, and that means the returned value is int and > > > may wrap, see coding and comment. Even with this caveat though this > > > info is still very useful. > > > > > > I did a number of manual tests, the numbers seem legit. > > > > > > Looks like this: > > > > > > Process Memory: > > > Virtual Size: 7494372K (peak: 7559908K) > > > Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: > > > 44804K, shmem: 0K) > > > Swapped out: 0K > > > C-Heap outstanding allocations: 37845K > > > > > > > > > Here I simulate a C-Heap memory leak: > > > > > > Process Memory: > > > Virtual Size: 9062260K (peak: 9118772K) > > > Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, file: > > > 44904K, shmem: 0K) > > > Swapped out: 0K > > > C-Heap outstanding allocations: 1291984K > > > > > > > > > Same leak, with the mallinfo value wrapped around: > > > > > > Process Memory: > > > Virtual Size: 12343352K (peak: 12445556K) > > > Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, file: > > > 45032K, shmem: 0K) > > > Swapped out: 0K > > > C-Heap outstanding allocations: 15454K (may have wrapped) > > > > > > Thanks, Thomas > > > > > From thomas.stuefe at gmail.com Mon Aug 10 12:29:19 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 10 Aug 2020 14:29:19 +0200 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info In-Reply-To: References: Message-ID: Great, thanks Martin! On Mon, Aug 10, 2020 at 2:24 PM Doerr, Martin wrote: > Hi Thomas, > > looks like a really helpful contribution. > > Minor nit: You print most values in "kB", but some other ones in "K". I > think "kB" should be used consistently. > > I don't think kernel < 2.6.34 is still supported. I don't care if you > remove the check or not. > > Otherwise thumbs up from my side. I don't need to see another webrev. > > Best regards, > Martin > > > > -----Original Message----- > > From: hotspot-runtime-dev > > On Behalf Of Thomas St?fe > > Sent: Montag, 10. August 2020 13:52 > > To: David Holmes > > Cc: Hotspot dev runtime > > Subject: Re: RFR(xs): 8251255: [linux] Add process-memory information to > > hs-err and VM.info > > > > Thanks for the review, David! > > > > On Mon, Aug 10, 2020 at 1:42 PM David Holmes > > wrote: > > > > > Hi Thomas, > > > > > > This seems fine to me. > > > > > > Thanks, > > > David > > > > > > On 10/08/2020 9:32 pm, Thomas St?fe wrote: > > > > Hi, > > > > > > > > new version: > > > > > > > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs- > > err/webrev.01/webrev/ > > > > > > > > - changed the type of the receiver variables to ssize_t > > > > - enclosed mallinfo call with __GLIBC__ > > > > > > > > Thanks, Thomas > > > > > > > > On Thu, Aug 6, 2020 at 4:01 PM Thomas St?fe > > > > > > wrote: > > > > > > > > Hi, > > > > > > > > may I please have reviews for this addition: > > > > > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 > > > > Webrev: > > > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs- > > err/webrev.00/webrev/ > > > > > > > > When analysing a customer memory leak recently I found for all > > > > the information we have in the error files we miss some really > > > > fundamental statistics like process virtual size and rss. Either > > > > that or I am blind :) > > > > > > > > In case I am not blind this patch adds those statistics. I > decided > > > > against dumping proc files wholesale to the hs-err file, I > > > > just print out what I really find useful. > > > > > > > > Note that among other things I print out the total size of > > > > outstanding heap allocations. Note that still the best way to do > > > > this is via mallinfo, and that means the returned value is int > and > > > > may wrap, see coding and comment. Even with this caveat though > this > > > > info is still very useful. > > > > > > > > I did a number of manual tests, the numbers seem legit. > > > > > > > > Looks like this: > > > > > > > > Process Memory: > > > > Virtual Size: 7494372K (peak: 7559908K) > > > > Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: > > > > 44804K, shmem: 0K) > > > > Swapped out: 0K > > > > C-Heap outstanding allocations: 37845K > > > > > > > > > > > > Here I simulate a C-Heap memory leak: > > > > > > > > Process Memory: > > > > Virtual Size: 9062260K (peak: 9118772K) > > > > Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, > file: > > > > 44904K, shmem: 0K) > > > > Swapped out: 0K > > > > C-Heap outstanding allocations: 1291984K > > > > > > > > > > > > Same leak, with the mallinfo value wrapped around: > > > > > > > > Process Memory: > > > > Virtual Size: 12343352K (peak: 12445556K) > > > > Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, > file: > > > > 45032K, shmem: 0K) > > > > Swapped out: 0K > > > > C-Heap outstanding allocations: 15454K (may have wrapped) > > > > > > > > Thanks, Thomas > > > > > > > > From thomas.stuefe at gmail.com Mon Aug 10 12:33:24 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 10 Aug 2020 14:33:24 +0200 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info In-Reply-To: References: Message-ID: Hi Martin, re-read your mail: On Mon, Aug 10, 2020 at 2:24 PM Doerr, Martin wrote: > Hi Thomas, > > looks like a really helpful contribution. > > Minor nit: You print most values in "kB", but some other ones in "K". I > think "kB" should be used consistently. > > Actually I don't, "kB" is used for sscanf()'ing the proc file. All printouts are in "K". kbyte are not consistently used - we have "k", "K", "KB". I opted for "K" since NMT and the heap summary use that. So I think if its okay with you I leave that part as it is. I don't think kernel < 2.6.34 is still supported. I don't care if you > remove the check or not. > > Otherwise thumbs up from my side. I don't need to see another webrev. > > Best regards, > Martin > Thanks, Thomas > > > > -----Original Message----- > > From: hotspot-runtime-dev > > On Behalf Of Thomas St?fe > > Sent: Montag, 10. August 2020 13:52 > > To: David Holmes > > Cc: Hotspot dev runtime > > Subject: Re: RFR(xs): 8251255: [linux] Add process-memory information to > > hs-err and VM.info > > > > Thanks for the review, David! > > > > On Mon, Aug 10, 2020 at 1:42 PM David Holmes > > wrote: > > > > > Hi Thomas, > > > > > > This seems fine to me. > > > > > > Thanks, > > > David > > > > > > On 10/08/2020 9:32 pm, Thomas St?fe wrote: > > > > Hi, > > > > > > > > new version: > > > > > > > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs- > > err/webrev.01/webrev/ > > > > > > > > - changed the type of the receiver variables to ssize_t > > > > - enclosed mallinfo call with __GLIBC__ > > > > > > > > Thanks, Thomas > > > > > > > > On Thu, Aug 6, 2020 at 4:01 PM Thomas St?fe > > > > > > wrote: > > > > > > > > Hi, > > > > > > > > may I please have reviews for this addition: > > > > > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 > > > > Webrev: > > > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs- > > err/webrev.00/webrev/ > > > > > > > > When analysing a customer memory leak recently I found for all > > > > the information we have in the error files we miss some really > > > > fundamental statistics like process virtual size and rss. Either > > > > that or I am blind :) > > > > > > > > In case I am not blind this patch adds those statistics. I > decided > > > > against dumping proc files wholesale to the hs-err file, I > > > > just print out what I really find useful. > > > > > > > > Note that among other things I print out the total size of > > > > outstanding heap allocations. Note that still the best way to do > > > > this is via mallinfo, and that means the returned value is int > and > > > > may wrap, see coding and comment. Even with this caveat though > this > > > > info is still very useful. > > > > > > > > I did a number of manual tests, the numbers seem legit. > > > > > > > > Looks like this: > > > > > > > > Process Memory: > > > > Virtual Size: 7494372K (peak: 7559908K) > > > > Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: > > > > 44804K, shmem: 0K) > > > > Swapped out: 0K > > > > C-Heap outstanding allocations: 37845K > > > > > > > > > > > > Here I simulate a C-Heap memory leak: > > > > > > > > Process Memory: > > > > Virtual Size: 9062260K (peak: 9118772K) > > > > Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, > file: > > > > 44904K, shmem: 0K) > > > > Swapped out: 0K > > > > C-Heap outstanding allocations: 1291984K > > > > > > > > > > > > Same leak, with the mallinfo value wrapped around: > > > > > > > > Process Memory: > > > > Virtual Size: 12343352K (peak: 12445556K) > > > > Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, > file: > > > > 45032K, shmem: 0K) > > > > Swapped out: 0K > > > > C-Heap outstanding allocations: 15454K (may have wrapped) > > > > > > > > Thanks, Thomas > > > > > > > > From erik.gahlin at oracle.com Mon Aug 10 13:19:06 2020 From: erik.gahlin at oracle.com (Erik Gahlin) Date: Mon, 10 Aug 2020 15:19:06 +0200 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> Message-ID: Hi Patricio, I have tried to review the JFR changes, but I need more information on how the feature is going to be used. Is this something temporary that will be used gain insight for the Valhalla project, or do you think this will be needed longer term? If it is longer term, the bar is higher on what can be accepted. If this is a temporary feature, we could mark the event as experimental in metadata.xml, similar to what we did with GC events during the development of ZGC. Events that are experimental do not show up by default in visualization tools such as JMC and can be removed when they are not needed, or we have a better solution. For events to be enabled in default.jfc, they should not cause more than 1% overhead, not even in pathological cases. To me, it seems this could happen if you make a loop where it is triggered all the time. For events in profile.jfc, the overhead should still be low (1-2%), but the target is the typical application. For example, allocation profiling is only enabled in profile.jfc. Some application that are allocation intensive could cause a higher overhead than 1%, but that is OK because that configuration is only to be used for a short period of time. That said, there is still a budget on how much space in a recording an event can take up, so it also depends on how important the information is for the user. To me, it seems this event will not be that important for the average user, which makes me think it should be disabled by default. To complicate things, I noticed that aa command line flag is also used to enable the event. This is something we have worked hard to avoid when it comes to JFR events. All configuration should happen using configuration files. We had problem with this in the past where users gets confused why their events are not enabled. When we ported JFR to Hotspot, we got rid of those of those flags/events and it is a much better situation. Again, if this a temporary event to discover usage pattern for a release or two, it might be OK, but if we believe this event to stick around, I think we should look into alternatives, or not use JFR at all for this. Thanks Erik > On 28 Jul 2020, at 21:16, Patricio Chilano wrote: > > Hi all, > > Please review the following change that adds diagnostic capabilities when synchronizing on primitive wrapper classes. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8242263 > Webrev: http://cr.openjdk.java.net/~pchilanomate/8242263/v1/webrev/ > > The new flag allows to identify synchronization on these classes and to take one of the following actions: exit the VM with fatal error, log a warning message, or issue a JFR event. The implementation uses a simple approach where a check is added at the beginning of the monitorenter generated code when the flag is enabled to check whether the object is of a primitive wrapper class. If it is, we jump to the slow path, otherwise we just continue as always. The extra instructions will be: load the klass of the object, load the _access_flags field for that klass, AND with a constant, and branch based on the result. The code will only be generated whenever the new opt-in diagnostic flag is enabled so performance won't be affected when off. > > In addition to the purpose described in the description of the bug, this flag will also be useful when trying to diagnose possible synchronization issues if these classes ever become inline types as part of the Valhalla project. > > I added test SyncOnPrimitiveWrapperTest.java that tests for the exit and logging cases. I added test TestSyncOnPrimitiveWrapperEvent.java to test for the JFR event case. > I tested the patch running tiers1-6 in mach5 with the flag set to DiagnoseSyncOnPrimitiveWrappers=2. > I checked it builds with arm32 and ppc but can't run any tests on those platforms, so it would be good if somebody can run the new test included in the patch. > > Let me know if you think I should run or add any more tests. > > Thanks! > > Patricio From martin.doerr at sap.com Mon Aug 10 13:19:28 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 10 Aug 2020 13:19:28 +0000 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info In-Reply-To: References: Message-ID: Hi Thomas, > Actually I don't, "kB" is used for sscanf()'ing the proc file. All printouts are in "K". Sorry, you?re right. > kbyte are not consistently used - we have "k", "K", "KB". What a mess! > I opted for "K" since NMT and the heap summary use that. So we report sizes in Kelvin ?? I don?t like "K", but if it?s already used, you can just ignore my complaint. Best regards, Martin From: Thomas St?fe Sent: Montag, 10. August 2020 14:33 To: Doerr, Martin Cc: David Holmes ; Hotspot dev runtime Subject: Re: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info Hi Martin, re-read your mail: On Mon, Aug 10, 2020 at 2:24 PM Doerr, Martin > wrote: Hi Thomas, looks like a really helpful contribution. Minor nit: You print most values in "kB", but some other ones in "K". I think "kB" should be used consistently. Actually I don't, "kB" is used for sscanf()'ing the proc file. All printouts are in "K". kbyte are not consistently used - we have "k", "K", "KB". I opted for "K" since NMT and the heap summary use that. So I think if its okay with you I leave that part as it is. I don't think kernel < 2.6.34 is still supported. I don't care if you remove the check or not. Otherwise thumbs up from my side. I don't need to see another webrev. Best regards, Martin Thanks, Thomas > -----Original Message----- > From: hotspot-runtime-dev > > On Behalf Of Thomas St?fe > Sent: Montag, 10. August 2020 13:52 > To: David Holmes > > Cc: Hotspot dev runtime > > Subject: Re: RFR(xs): 8251255: [linux] Add process-memory information to > hs-err and VM.info > > Thanks for the review, David! > > On Mon, Aug 10, 2020 at 1:42 PM David Holmes > > wrote: > > > Hi Thomas, > > > > This seems fine to me. > > > > Thanks, > > David > > > > On 10/08/2020 9:32 pm, Thomas St?fe wrote: > > > Hi, > > > > > > new version: > > > > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs- > err/webrev.01/webrev/ > > > > > > - changed the type of the receiver variables to ssize_t > > > - enclosed mallinfo call with __GLIBC__ > > > > > > Thanks, Thomas > > > > > > On Thu, Aug 6, 2020 at 4:01 PM Thomas St?fe > > > > >> wrote: > > > > > > Hi, > > > > > > may I please have reviews for this addition: > > > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 > > > Webrev: > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs- > err/webrev.00/webrev/ > > > > > > When analysing a customer memory leak recently I found for all > > > the information we have in the error files we miss some really > > > fundamental statistics like process virtual size and rss. Either > > > that or I am blind :) > > > > > > In case I am not blind this patch adds those statistics. I decided > > > against dumping proc files wholesale to the hs-err file, I > > > just print out what I really find useful. > > > > > > Note that among other things I print out the total size of > > > outstanding heap allocations. Note that still the best way to do > > > this is via mallinfo, and that means the returned value is int and > > > may wrap, see coding and comment. Even with this caveat though this > > > info is still very useful. > > > > > > I did a number of manual tests, the numbers seem legit. > > > > > > Looks like this: > > > > > > Process Memory: > > > Virtual Size: 7494372K (peak: 7559908K) > > > Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: > > > 44804K, shmem: 0K) > > > Swapped out: 0K > > > C-Heap outstanding allocations: 37845K > > > > > > > > > Here I simulate a C-Heap memory leak: > > > > > > Process Memory: > > > Virtual Size: 9062260K (peak: 9118772K) > > > Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, file: > > > 44904K, shmem: 0K) > > > Swapped out: 0K > > > C-Heap outstanding allocations: 1291984K > > > > > > > > > Same leak, with the mallinfo value wrapped around: > > > > > > Process Memory: > > > Virtual Size: 12343352K (peak: 12445556K) > > > Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, file: > > > 45032K, shmem: 0K) > > > Swapped out: 0K > > > C-Heap outstanding allocations: 15454K (may have wrapped) > > > > > > Thanks, Thomas > > > > > From thomas.stuefe at gmail.com Mon Aug 10 13:41:17 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 10 Aug 2020 15:41:17 +0200 Subject: RFR(xs): 8251255: [linux] Add process-memory information to hs-err and VM.info In-Reply-To: References: Message-ID: On Mon, Aug 10, 2020 at 3:19 PM Doerr, Martin wrote: > Hi Thomas, > > > > > Actually I don't, "kB" is used for sscanf()'ing the proc file. All > printouts are in "K". > > Sorry, you?re right. > > > > > kbyte are not consistently used - we have "k", "K", "KB". > > What a mess! > > > > > I opted for "K" since NMT and the heap summary use that. > > So we report sizes in Kelvin ?? > XD you are right. Well we can fix this in a follow up later. > I don?t like "K", but if it?s already used, you can just ignore my > complaint. > > > > Best regards, > > Martin > > > Thanks, Thomas > > > *From:* Thomas St?fe > *Sent:* Montag, 10. August 2020 14:33 > *To:* Doerr, Martin > *Cc:* David Holmes ; Hotspot dev runtime < > hotspot-runtime-dev at openjdk.java.net> > *Subject:* Re: RFR(xs): 8251255: [linux] Add process-memory information > to hs-err and VM.info > > > > Hi Martin, > > > > re-read your mail: > > > > On Mon, Aug 10, 2020 at 2:24 PM Doerr, Martin > wrote: > > Hi Thomas, > > looks like a really helpful contribution. > > Minor nit: You print most values in "kB", but some other ones in "K". I > think "kB" should be used consistently. > > > > Actually I don't, "kB" is used for sscanf()'ing the proc file. All > printouts are in "K". kbyte are not consistently used - we have "k", "K", > "KB". I opted for "K" since NMT and the heap summary use that. > > > > So I think if its okay with you I leave that part as it is. > > > > I don't think kernel < 2.6.34 is still supported. I don't care if you > remove the check or not. > > Otherwise thumbs up from my side. I don't need to see another webrev. > > Best regards, > Martin > > > > Thanks, Thomas > > > > > > > -----Original Message----- > > From: hotspot-runtime-dev > > On Behalf Of Thomas St?fe > > Sent: Montag, 10. August 2020 13:52 > > To: David Holmes > > Cc: Hotspot dev runtime > > Subject: Re: RFR(xs): 8251255: [linux] Add process-memory information to > > hs-err and VM.info > > > > Thanks for the review, David! > > > > On Mon, Aug 10, 2020 at 1:42 PM David Holmes > > wrote: > > > > > Hi Thomas, > > > > > > This seems fine to me. > > > > > > Thanks, > > > David > > > > > > On 10/08/2020 9:32 pm, Thomas St?fe wrote: > > > > Hi, > > > > > > > > new version: > > > > > > > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs- > > err/webrev.01/webrev/ > > > > > > > > - changed the type of the receiver variables to ssize_t > > > > - enclosed mallinfo call with __GLIBC__ > > > > > > > > Thanks, Thomas > > > > > > > > On Thu, Aug 6, 2020 at 4:01 PM Thomas St?fe > > > > > > wrote: > > > > > > > > Hi, > > > > > > > > may I please have reviews for this addition: > > > > > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251255 > > > > Webrev: > > > > > > > http://cr.openjdk.java.net/~stuefe/webrevs/8251255-add-rss-to-hs- > > err/webrev.00/webrev/ > > > > > > > > When analysing a customer memory leak recently I found for all > > > > the information we have in the error files we miss some really > > > > fundamental statistics like process virtual size and rss. Either > > > > that or I am blind :) > > > > > > > > In case I am not blind this patch adds those statistics. I > decided > > > > against dumping proc files wholesale to the hs-err file, I > > > > just print out what I really find useful. > > > > > > > > Note that among other things I print out the total size of > > > > outstanding heap allocations. Note that still the best way to do > > > > this is via mallinfo, and that means the returned value is int > and > > > > may wrap, see coding and comment. Even with this caveat though > this > > > > info is still very useful. > > > > > > > > I did a number of manual tests, the numbers seem legit. > > > > > > > > Looks like this: > > > > > > > > Process Memory: > > > > Virtual Size: 7494372K (peak: 7559908K) > > > > Resident Set Size: 271264K (peak: 571348K) (anon: 226460K, file: > > > > 44804K, shmem: 0K) > > > > Swapped out: 0K > > > > C-Heap outstanding allocations: 37845K > > > > > > > > > > > > Here I simulate a C-Heap memory leak: > > > > > > > > Process Memory: > > > > Virtual Size: 9062260K (peak: 9118772K) > > > > Resident Set Size: 1941776K (peak: 1941776K) (anon: 1896872K, > file: > > > > 44904K, shmem: 0K) > > > > Swapped out: 0K > > > > C-Heap outstanding allocations: 1291984K > > > > > > > > > > > > Same leak, with the mallinfo value wrapped around: > > > > > > > > Process Memory: > > > > Virtual Size: 12343352K (peak: 12445556K) > > > > Resident Set Size: 5232084K (peak: 5281512K) (anon: 5187052K, > file: > > > > 45032K, shmem: 0K) > > > > Swapped out: 0K > > > > C-Heap outstanding allocations: 15454K (may have wrapped) > > > > > > > > Thanks, Thomas > > > > > > > > > From patricio.chilano.mateo at oracle.com Mon Aug 10 13:44:53 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Mon, 10 Aug 2020 10:44:53 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> <2b030b14-0dc2-2de6-f01b-cc4f9ae80149@oracle.com> <41fd3f22-4266-de46-8d32-47ba6295218b@oracle.com> Message-ID: <39f9b363-8ce1-563c-94b8-8525ff03ccc7@oracle.com> Hi Coleen, On 8/7/20 12:58 PM, Coleen Phillimore wrote: > > > On 8/7/20 11:49 AM, Patricio Chilano wrote: >> Hi Coleen, >> >> On 8/7/20 10:53 AM, Coleen Phillimore wrote: >>> >>> >>> On 8/6/20 9:45 PM, Patricio Chilano wrote: >>>> Hi Coleen, >>>> >>>> Thanks for looking at this. >>>> >>>> On 8/6/20 7:17 PM, Coleen Phillimore wrote: >>>>> >>>>> Patricio,? One question: >>>>> >>>>> http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/src/hotspot/share/runtime/synchronizer.cpp.udiff.html >>>>> >>>>> >>>>> + // adjust bcp to point back to monitorenter so that we print the >>>>> correct line numbers >>>>> >>>>> >>>>> How do the fatal and event print the correct line numbers here? I >>>>> see the logging gets it from the stack trace. Should the abort >>>>> message have more information in it?? You can get the source and >>>>> line number information in the same way that print_stack_on() gets >>>>> it. >>>> For the fatal error case I'm not printing line numbers as I do with >>>> the warning case or as you get with JFR events. But maybe I should >>>> print the stack too and then exit the VM. You can get the stack >>>> info and the line number of the monitorenter bytecode that caused >>>> the crash from the generated hs_err file though. >>> >>> Can you send a sample, please? >> At the top of the hs_err you will get: >> >> ?# >> ?# A fatal error has been detected by the Java Runtime Environment: >> ?# >> ?#? Internal Error >> (/xx/xxx/xxxx/open/src/hotspot/share/runtime/synchronizer.cpp:574), >> pid=48810, tid=48811 >> ?#? fatal error: Synchronizing on object 0x00000007ff0758a8 of klass >> java.lang.Integer >> ?# >> >> And then on the stacktrace you can see: >> >> Stack: [0x00007f982b6ce000,0x00007f982b7cf000], >> sp=0x00007f982b7cd5c0,? free space=1021k >> Native frames: (J=compiled Java code, A=aot compiled Java code, >> j=interpreted, Vv=VM code, C=native code) >> V? [libjvm.so+0x1662385] >> ObjectSynchronizer::handle_sync_on_primitive_wrapper(Handle, >> Thread*)+0x185 >> V? [libjvm.so+0x16683ac]? ObjectSynchronizer::enter(Handle, >> BasicLock*, Thread*)+0x21c >> V? [libjvm.so+0xce88fa] InterpreterRuntime::monitorenter(JavaThread*, >> BasicObjectLock*)+0x13a >> j? SimpleTest.main([Ljava/lang/String;)V+24 >> v? ~StubRoutines::call_stub >> V? [libjvm.so+0xd0263a]? JavaCalls::call_helper(JavaValue*, >> methodHandle const&, JavaCallArguments*, Thread*)+0x62a >> V? [libjvm.so+0xe22c8e]? jni_invoke_static(JNIEnv_*, JavaValue*, >> _jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, Thr >> ?.constprop.1]+0x35e >> V? [libjvm.so+0xe28e6b]? jni_CallStaticVoidMethod+0x21b >> C? [libjli.so+0x496e]? JavaMain+0xc1e >> C? [libjli.so+0x7759]? ThreadJavaMain+0x9 >> >> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) >> j? SimpleTest.main([Ljava/lang/String;)V+24 >> v? ~StubRoutines::call_stub >> >> Where +24 is the bci. But that would be with the interpreter, with c1 >> that line will show as: >> >> J 576 c1 SimpleTest.main([Ljava/lang/String;)V (205 bytes) @ >> 0x00007f97b5ae5dc7 >> >> You have the method where it failed but not the exact bci. So maybe I >> should print the stack and then exit the VM. >> >>> I guess there isn't a handy function to show the source file name >>> and line number in the fatal error message. >> I'll see if there is an easy way to have it in the fatal error >> message otherwise I can just print the stack as with the warning case >> and exit. > > I don't know if we want the fatal message to print the whole stack. > But an improvement would be: > > ?#? fatal error: Synchronizing on object 0x00000007ff0758a8 of klass > java.lang.Integer in method->external_name()+bci > > which you have in the function and then people won't have to go to the > hs_err file. I tested two different changes. The first one gets the method and bci and adds it to the fatal error message as you said: diff --git a/src/hotspot/share/runtime/synchronizer.cpp b/src/hotspot/share/runtime/synchronizer.cpp --- a/src/hotspot/share/runtime/synchronizer.cpp +++ b/src/hotspot/share/runtime/synchronizer.cpp @@ -572,5 +572,9 @@ ?? if (DiagnoseSyncOnPrimitiveWrappers == FATAL_EXIT) { ???? ResourceMark rm(self); -??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass %s", p2i(obj()), obj->klass()->external_name()); +??? vframeStream vfst(self); +??? Method* method = vfst.method(); +??? int bci = vfst.bci(); +??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass %s in \"%s\" at bci:%d", +????????? p2i(obj()), obj->klass()->external_name(), method ? method->external_name() : "", bci); ?? } Which produces the following output: #? Internal Error (/scratch/pchilano/random7/open/src/hotspot/share/runtime/synchronizer.cpp:577), pid=13759, tid=13785 #? fatal error: Synchronizing on object 0x00000001013d72d8 of klass java.lang.Float in "void SimpleTest.run()" at bci:24 # In the second one I use the same print_stack_on() method I use for the warning case but print on a stringStream object instead, and then I print the first line of the buffer only. diff --git a/src/hotspot/share/runtime/synchronizer.cpp b/src/hotspot/share/runtime/synchronizer.cpp --- a/src/hotspot/share/runtime/synchronizer.cpp +++ b/src/hotspot/share/runtime/synchronizer.cpp @@ -572,5 +572,12 @@ ?? if (DiagnoseSyncOnPrimitiveWrappers == FATAL_EXIT) { ???? ResourceMark rm(self); -??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass %s", p2i(obj()), obj->klass()->external_name()); +??? stringStream ss; +??? self->print_stack_on(&ss); +??? char* base = (char*)strstr(ss.base(), "at"); +??? char* newline = (char*)strchr(ss.base(), '\n'); +??? if (newline != NULL) { +????? *newline = '\0'; +??? } +??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass %s %s", p2i(obj()), obj->klass()->external_name(), base); ?? } Which produces the following output: #? Internal Error (/scratch/pchilano/random7/open/src/hotspot/share/runtime/synchronizer.cpp:581), pid=9288, tid=9306 #? fatal error: Synchronizing on object 0x00000001013d72d8 of klass java.lang.Float at SimpleTest.run(SimpleTest.java:70) # I think the second method is better since it provides the actual file and line number, what do you think? Thanks, Patricio > Thanks, > Coleen > > >> >> >> Thanks, >> Patricio >>> Thanks, >>> Coleen >>>> >>>> I don't know which exact technique JFR uses to print line numbers >>>> but it has to include reading the current bcp. Since when coming >>>> from the interpreter for monitorenter the bcp is always already >>>> pointing to the next instruction we need to decrement it to print >>>> the correct line numbers. I tested it and if I don't fix the bcp, >>>> as expected JFR too will print the next line number for the top frame. >>>> >>>> >>>> Patricio >>>>> Otherwise, this looks good to me. >>>>> Coleen >>>>> >>>>> >>>>> On 8/6/20 2:48 PM, Patricio Chilano wrote: >>>>>> Hi Dan, >>>>>> >>>>>> On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: >>>>>>> I'm peeking ahead to the next webrev... :-) >>>>>>> >>>>>>> > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ >>>>>> : ) >>>>>> >>>>>>> General comments: >>>>>>> ? - check files for copyright year updates. >>>>>> Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, >>>>>> c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. >>>>>> >>>>>>> src/hotspot/share/runtime/synchronizer.hpp >>>>>>> ??? No comments. >>>>>>> >>>>>>> src/hotspot/share/runtime/synchronizer.cpp >>>>>>> ??? L507: ? const markWord mark = obj->mark(); >>>>>>> ??? L508: >>>>>>> ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && >>>>>>> obj->klass()->is_box()) { >>>>>>> ??? L510: ??? return false; >>>>>>> ??? L511: ? } >>>>>>> ??????? The new bailout on L509-511 can move above L507. >>>>>> Moved. >>>>>> >>>>>>> L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of >>>>>>> klass %s", p2i(obj()), obj->klass()->external_name()); >>>>>>> ??????? This external_name() call does not have a ResourceMark. >>>>>> Good catch! I had one in a previous version but then I changed >>>>>> the conditionals and lost it for the fatal error case. The test >>>>>> worked okay because for the main JavaThread there is a >>>>>> ResourceMark in jni_invoke_static() (jni.cpp). >>>>>> >>>>>>> src/hotspot/share/logging/logTag.hpp >>>>>>> ??? No comments. >>>>>>> >>>>>>> src/hotspot/share/oops/klass.hpp >>>>>>> ??? No comments. >>>>>>> >>>>>>> src/hotspot/share/utilities/accessFlags.hpp >>>>>>> ??? No comments. >>>>>>> >>>>>>> src/hotspot/share/runtime/globals.hpp >>>>>>> ??? L814: ???????????? "0: off " >>>>>>> ??????? Missing a ';' after "off". >>>>>> Fixed. >>>>>> >>>>>>> L816: ???????????? "2: log message to stdout. >>>>>>> ??????? Perhaps add "(by default)" after "stdout" or >>>>>>> ??????? don't say where log output is at all. >>>>>>> >>>>>>> src/hotspot/share/runtime/arguments.cpp >>>>>>> ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, >>>>>>> true, LOG_TAGS(primitivewrappers)); >>>>>>> ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp >>>>>>> ??????? since it looks like logging output is configured to >>>>>>> 'stdout'. >>>>>>> ??????? I'm assuming that other log options to put output elsewhere >>>>>>> ??????? are overridden by this code? >>>>>> Right. So the logging is done under UL with the tag >>>>>> primitivewrappers. If that tag is specified in Xlog then this >>>>>> conditional will be skipped because !log_is_enabled(Info, >>>>>> primitivewrappers) will be false. >>>>>> >>>>>>> src/hotspot/share/classfile/systemDictionary.cpp >>>>>>> ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { >>>>>>> ??????? nit - s/T_LONG+1/T_LONG + 1/ >>>>>> Fixed. >>>>>> >>>>>>> L2191: >>>>>>> _box_klasses[i]->set_prototype_header(markWord::prototype()); >>>>>>> ??????? I assume we're keeping the prototype_header field when >>>>>>> Biased Locking >>>>>>> ??????? goes away? The reason I ask: >>>>>>> >>>>>>> ? ? ? ? static markWord prototype() { >>>>>>> ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); >>>>>>> ? ? ? ? } >>>>>>> >>>>>>> ??????? is because without Biased Locking, do we really need the >>>>>>> prototype >>>>>>> ??????? anymore? The initial markWord won't need possible >>>>>>> variants... >>>>>> Yes, I think it can go away unless somebody finds another use for >>>>>> it. Maybe Valhalla. >>>>>> >>>>>>> src/hotspot/share/jfr/metadata/metadata.xml >>>>>>> ??? L69: ? >>>>>> ??????? Is the category "Java Application" because it's the >>>>>>> application >>>>>>> ??????? code that did something "wrong" here? Where >>>>>>> "application" is loosely >>>>>>> ??????? defined to include the possibility of auto generated >>>>>>> code, library >>>>>>> ??????? code and the like? Or perhaps "application" because >>>>>>> something "above" >>>>>>> ??????? the "Java Virtual Machine, Runtime" did the "wrong" >>>>>>> thing here? >>>>>> I don't know what the right category should be really. I saw the >>>>>> events JavaMonitorEnter, JavaMonitorWait and JavaMonitorInflate >>>>>> and thought this event should fall in the same category they do. >>>>>> >>>>>>> src/jdk.jfr/share/conf/jfr/default.jfc >>>>>>> ??? No comments. >>>>>>> >>>>>>> src/jdk.jfr/share/conf/jfr/profile.jfc >>>>>>> ??? No comments. >>>>>>> >>>>>>> test/lib/jdk/test/lib/jfr/EventNames.java >>>>>>> ??? No comments. >>>>>>> >>>>>>> src/hotspot/cpu/aarch64/aarch64.ad >>>>>>> ??? L3517: ????? __ tbnz(tmp, exact_log2(JVM_ACC_IS_BOX_CLASS), >>>>>>> cont); >>>>>>> >>>>>>> ??? L3578: ??? __ bind(cont); >>>>>>> ??? L3579: ??? // flag == EQ indicates success >>>>>>> ??? L3580: ??? // flag == NE indicates failure >>>>>>> ??????? If tbnz() branches to "cont" when we have a box class, >>>>>>> what's >>>>>>> ??????? the flag value set to (EQ or NE)? And what set that flag >>>>>>> value? >>>>>>> ??????? The reason I ask is I don't think tbnz() sets condition >>>>>>> codes... >>>>>> Right, it doesn't set condition codes, so I kept the version I >>>>>> had (more on that below). >>>>>> >>>>>>> src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>>> ??? No comments. >>>>>>> >>>>>>> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp >>>>>>> ??? No comments. >>>>>>> >>>>>>> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp >>>>>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>>>>> ??????????? ? ?? ??????????????????????????? Register obj_reg, >>>>>>> ?????????????????????????????? ? ?? ???????? Register swap_reg, >>>>>>> ?????????????????????????????????? ? ?? ???? Register tmp_reg, >>>>>>> ?????????????????????????????????? ? ?? ???? bool >>>>>>> swap_reg_contains_mark, >>>>>>> Label& done, >>>>>>> ??????????????????????????????????? ? ?? ??? Label* slow_case, >>>>>>> BiasedLockingCounters* counters) { >>>>>>> ??????? I think you've changed the only callers of >>>>>>> biased_locking_enter() >>>>>>> ??????? that cared about the return value with this changeset so >>>>>>> it can >>>>>>> ??????? be changed to a void function. >>>>>> Ok, this is what I mentioned to David in a previous email. Done. >>>>>> >>>>>>> src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp >>>>>>> ??? No comments. >>>>>>> >>>>>>> src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp >>>>>>> ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), >>>>>>> done); >>>>>>> >>>>>>> ??? L131: ? bind(done); >>>>>>> ??? L132: >>>>>>> ??? L133: ? // At this point flags are set as follows: >>>>>>> ??? L134: ? //? EQ -> Success >>>>>>> ??? L135: ? //? NE -> Failure, branch to slow path >>>>>>> ??????? If tbnz() branches to "done" when we have a box class, >>>>>>> what's >>>>>>> ??????? the flag value set to (EQ or NE)? And what set that flag >>>>>>> value? >>>>>>> ??????? The reason I ask is I don't think tbnz() sets condition >>>>>>> codes... >>>>>> Right. Same as above, I kept the version I had. >>>>>> >>>>>>> src/hotspot/cpu/arm/interp_masm_arm.cpp >>>>>>> ??? No comments. >>>>>>> >>>>>>> src/hotspot/cpu/arm/macroAssembler_arm.cpp >>>>>>> ??? int MacroAssembler::biased_locking_enter(Register obj_reg, >>>>>>> Register swap_reg, Register tmp_reg, >>>>>>> ????????????????????? ? ?? ????????????????? bool >>>>>>> swap_reg_contains_mark, >>>>>>> ???????????????????????? ? ?? ?????????????? Register tmp2, >>>>>>> Label& done, Label& slow_case, >>>>>>> BiasedLockingCounters* counters) { >>>>>>> ??????? I think you've changed the only callers of >>>>>>> biased_locking_enter() >>>>>>> ??????? that cared about the return value with this changeset so >>>>>>> it can >>>>>>> ??????? be changed to a void function. >>>>>> Done. >>>>>> >>>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>> ??? No comments on the PPC code. >>>>>>> >>>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>> src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>> ??? No comments on the S390 code. >>>>>>> >>>>>>> src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp >>>>>>> ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); >>>>>>> ??????? What will this do with a 'noreg' value? (I need a >>>>>>> refresher.) >>>>>> When not in 64 bit mode that register just won't be used. >>>>>> >>>>>>> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp >>>>>>> ??? No comments. >>>>>>> >>>>>>> src/hotspot/cpu/x86/interp_masm_x86.cpp >>>>>>> ??? No comments. >>>>>>> >>>>>>> src/hotspot/cpu/x86/macroAssembler_x86.cpp >>>>>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>>>>> ????????????????????????????????? ? ?? ????? Register obj_reg, >>>>>>> ????????????????????????????????? ? ?? ????? Register swap_reg, >>>>>>> ????????????????????????????????? ? ?? ????? Register tmp_reg, >>>>>>> ?????????????????????????????????? ? ?? ???? Register tmp_reg2, >>>>>>> ?????????????????????????????????????? ? ? ? bool >>>>>>> swap_reg_contains_mark, >>>>>>> Label& done, >>>>>>> ?????????????????????????????????? ? ?? ???? Label* slow_case, >>>>>>> BiasedLockingCounters* counters) { >>>>>>> ??????? I think you've changed the only caller of >>>>>>> biased_locking_enter() >>>>>>> ??????? that cared about the return value with this changeset so >>>>>>> it can >>>>>>> ??????? be changed to a void function. >>>>>> Done. >>>>>> >>>>>>> test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >>>>>>> ??? L30:? * @test SyncOnPrimitiveWrapperTest >>>>>>> ??????? No parameter to @test directive. >>>>>> Removed. >>>>>> >>>>>>> L136: ??????? private static long sharedCounter = 0L; >>>>>>> ??????? Since you don't do anything with sharedCounter other >>>>>>> than increment it, >>>>>>> ??????? can the compilers optimize it away? If it can be >>>>>>> optimized away, does >>>>>>> ??????? that mean that: >>>>>>> >>>>>>> ??????????? L142: ??????????????? synchronized (obj) { >>>>>>> >>>>>>> ??????? can also be optimized away? >>>>>>> >>>>>>> ??????? I don't think that: >>>>>>> >>>>>>> ??????????? L161: ??????????????? synchronized (sharedLock1) { >>>>>>> >>>>>>> ??????? can be optimized away because it is shared by multiple >>>>>>> threads. >>>>>>> >>>>>>> test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java >>>>>>> ??? Similar questions about 'counter' being optimized away. >>>>>>> ??? Similar question about "synchronized (obj) {" being >>>>>>> optimized away. >>>>>> I'm not sure if the compiler will optimize it away. Seems >>>>>> unlikely given the counter we are incrementing is not just local >>>>>> to some thread. >>>>>> >>>>>> >>>>>> Ok, below is v3 which has the following changes: >>>>>> - Use a 32 bit load for the _access_flags field, instead of 64 >>>>>> - Martin's implementation for s390 and fix for PPC >>>>>> - Faster LogTest version >>>>>> - Above changes based on Dan review >>>>>> >>>>>> I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 >>>>>> only) again with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I checked >>>>>> it builds in arm32, ppc and s390. >>>>>> >>>>>> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev >>>>>> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev >>>>>> >>>>>> >>>>>> @Martin: >>>>>> Please check if the test doesn't timeout for you now with the >>>>>> changes I made to LogTest. >>>>>> Also, I tried to use tbnz in aarch64 and arm32 instead of tst + >>>>>> br (except for c2 since we actually need to set the condition >>>>>> flags), but for c1 I was getting an assertion in the compiler >>>>>> thread: >>>>>> >>>>>> guarantee(chk == -1 || chk == 0) failed: Field too big for insn >>>>>> >>>>>> This is the stack when that happens: >>>>>> >>>>>> MacroAssembler::pd_patch_instruction_size(unsigned char*, >>>>>> unsigned char*)+0x398 >>>>>> AbstractAssembler::bind(Label&)+0x118 >>>>>> MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 >>>>>> LIR_Assembler::emit_slow_case_stubs()+0x54 >>>>>> Compilation::emit_code_body()+0x17c >>>>>> >>>>>> The change was simply: >>>>>> >>>>>> diff --git >>>>>> a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>> b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>> --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>> +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>> @@ -78,7 +78,6 @@ >>>>>> ?? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>> ???? load_klass(hdr, obj); >>>>>> -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); >>>>>> -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); >>>>>> -??? br(Assembler::NE, slow_case); >>>>>> +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); >>>>>> +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); >>>>>> ?? } >>>>>> >>>>>> So the issue must be related to where we want to jump. >>>>>> >>>>>> Thanks, >>>>>> Patricio >>>>>>> Dan >>>>>>> >>>>>>> >>>>>>> On 8/5/20 9:01 AM, Patricio Chilano wrote: >>>>>>>> Hi Martin, >>>>>>>> >>>>>>>> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>>>>>>>> Hi Patricio, >>>>>>>>> >>>>>>>>> using 8 Byte load instructions for jint fields is a terrible >>>>>>>>> coding style! >>>>>>>>> Someone else may see it and use an 8 Byte store. Will result >>>>>>>>> in great fun for debugging! >>>>>>>>> >>>>>>>>> And on Big Endian you will simply access the wrong bits. >>>>>>>> Ah, of course! Those 32 bits will be in the wrong place when >>>>>>>> doing the test. >>>>>>>> >>>>>>>>> Note that Big Endian Platforms are AIX, old linux ppc, s390, >>>>>>>>> SPARC. I don't think you have tested on them. >>>>>>>>> >>>>>>>>>> We could remove the nested synchronized statements in the >>>>>>>>>> run() method >>>>>>>>>> so that we don't generate that much logging. We could also lower >>>>>>>>>> LOOP_COUNT. I think the issue is also because we are running >>>>>>>>>> LogTest >>>>>>>>>> with multiple flag combinations. Not sure what we should >>>>>>>>>> touch first. >>>>>>>>>> Maybe the synchronized statements, have only one or two and >>>>>>>>>> test that >>>>>>>>>> first? >>>>>>>>> Sounds like helpful ideas. Please go ahead and strip things down. >>>>>>>> Great, I will send v3 later with those changes. >>>>>>>> >>>>>>>> Thanks Martin! >>>>>>>> >>>>>>>> Patricio >>>>>>>>> Thanks for taking care of it. >>>>>>>>> >>>>>>>>> Best regards, >>>>>>>>> Martin >>>>>>>>> >>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Patricio Chilano >>>>>>>>>> Sent: Dienstag, 4. August 2020 20:26 >>>>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on >>>>>>>>>> primitive wrappers >>>>>>>>>> >>>>>>>>>> Hi Martin, >>>>>>>>>> >>>>>>>>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>>>>>>>> Hi Patricio, >>>>>>>>>>> >>>>>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>>>>> sending v3. >>>>>>>>>>> Thanks. I forgot to mention arm + aarch64. >>>>>>>>>>> Aarch64 uses ldrw + tbnz. >>>>>>>>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>>>>>>>> >>>>>>>>>>> I remember having seen the same mistake ?? >>>>>>>>>>> And nobody noticed it on little Endian platforms. >>>>>>>>>> Ok, I can use a tbnz instead of test and then a branch on >>>>>>>>>> arm32 and aarch64. >>>>>>>>>> Shouldn't a normal ldr in arm32 work fine? >>>>>>>>>> Also in 64 bits (either x64 or aarch64) I don't see the issue >>>>>>>>>> of using a >>>>>>>>>> 64 bit load, besides the fact that we only care about the >>>>>>>>>> first 32 bits. >>>>>>>>>> Regardless of the endianness, aren't we masking out the upper >>>>>>>>>> part when >>>>>>>>>> we do AND/TEST or if we test a bit in the 0-31 bit range? >>>>>>>>>> Otherwise it >>>>>>>>>> seems it should have failed for one of those platforms in my >>>>>>>>>> tests. >>>>>>>>>> >>>>>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe >>>>>>>>>>>> you can >>>>>>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>>>>>> works and I >>>>>>>>>>>> can add it to the test. >>>>>>>>>>> That seems to have an effect. But now I'm not patient enough >>>>>>>>>>> to wait for >>>>>>>>>> the test to finish. >>>>>>>>>>> Maybe the problem is that I'm using slow debug builds. >>>>>>>>>>> But is there a chance to make the test quicker without >>>>>>>>>>> losing coverage >>>>>>>>>> significantly? >>>>>>>>>>> I think the test is too slow this way to run it on a regular >>>>>>>>>>> basis. We'd need to >>>>>>>>>> spend dedicated machines for it. >>>>>>>>>> We could remove the nested synchronized statements in the >>>>>>>>>> run() method >>>>>>>>>> so that we don't generate that much logging. We could also lower >>>>>>>>>> LOOP_COUNT. I think the issue is also because we are running >>>>>>>>>> LogTest >>>>>>>>>> with multiple flag combinations. Not sure what we should >>>>>>>>>> touch first. >>>>>>>>>> Maybe the synchronized statements, have only one or two and >>>>>>>>>> test that >>>>>>>>>> first? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Patricio >>>>>>>>>>> Best regards, >>>>>>>>>>> Martin >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>> From: Patricio Chilano >>>>>>>>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>>>>>> ; >>>>>>>>>>>> hotspot-runtime-dev at openjdk.java.net; >>>>>>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on >>>>>>>>>>>> primitive wrappers >>>>>>>>>>>> >>>>>>>>>>>> Hi Martin, >>>>>>>>>>>> >>>>>>>>>>>> Thanks for fixing PPC and taking care of s390! >>>>>>>>>>>> >>>>>>>>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>>>>>>>> Hi Patricio, >>>>>>>>>>>>> >>>>>>>>>>>>> I suggest to use movl + testl for checking the access flag >>>>>>>>>>>>> as for other >>>>>>>>>> access >>>>>>>>>>>> flags on x64. >>>>>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>>>>> sending v3. >>>>>>>>>>>> >>>>>>>>>>>>> New version for PPC64 and s390 see below. >>>>>>>>>>>>> >>>>>>>>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>>>>>>>>> expected. >>>>>>>>>>>> However, I'm getting timeout issues: >>>>>>>>>>>>> timed out (timeout set to 120000ms, elapsed time including >>>>>>>>>>>>> timeout >>>>>>>>>>>> handling was 122372ms) >>>>>>>>>>>>> Can we provide more time? >>>>>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe >>>>>>>>>>>> you can >>>>>>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>>>>>> works and I >>>>>>>>>>>> can add it to the test. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Patricio >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> Martin >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue >>>>>>>>>>>>> Aug 04 >>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue >>>>>>>>>>>>> Aug 04 >>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>> @@ -107,8 +107,8 @@ >>>>>>>>>>>>> >>>>>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>>>>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>>>> Rscratch); >>>>>>>>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>>>> +??? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>>>> Rscratch); >>>>>>>>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>> ??????? bne(CCR0, slow_case); >>>>>>>>>>>>> ????? } >>>>>>>>>>>>> >>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>> @@ -912,8 +912,8 @@ >>>>>>>>>>>>> >>>>>>>>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>> ????????? load_klass(tmp, object); >>>>>>>>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>>>>>> +????? testbitdi(CCR0, R0, tmp, >>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>> ????????? bne(CCR0, slow_case); >>>>>>>>>>>>> ??????? } >>>>>>>>>>>>> >>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>>>>>>>> >>>>>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>> ??????? load_klass(temp, oop); >>>>>>>>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>>>>>> +??? testbitdi(CCR0, R0, temp, >>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>> ??????? bne(CCR0, cont); >>>>>>>>>>>>> ????? } >>>>>>>>>>>>> >>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue >>>>>>>>>>>>> Aug 04 >>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue >>>>>>>>>>>>> Aug 04 >>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>> @@ -91,6 +91,12 @@ >>>>>>>>>>>>> ????? // Save object being locked into the BasicObjectLock... >>>>>>>>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>>>>>>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>> + z_btrue(slow_case); >>>>>>>>>>>>> +? } >>>>>>>>>>>>> + >>>>>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>>>>>>>> Z_R0_scratch, done, >>>>>>>>>>>> &slow_case); >>>>>>>>>>>>> ????? } >>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>> src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>>>>>>> 10:03:57 >>>>>>>>>>>> 2020 +0200 >>>>>>>>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>>>> 16:04:31 >>>>>>>>>>>> 2020 +0200 >>>>>>>>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>>>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>>>>>>>> ????? z_lg(displaced_header, >>>>>>>>>>>>> oopDesc::mark_offset_in_bytes(), object); >>>>>>>>>>>>> >>>>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>> + z_btrue(slow_case); >>>>>>>>>>>>> +? } >>>>>>>>>>>>> + >>>>>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>>>>> ??????? biased_locking_enter(object, displaced_header, >>>>>>>>>>>>> Z_R1, Z_R0, done, >>>>>>>>>>>> &slow_case); >>>>>>>>>>>>> ????? } >>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>>>>>>>> ????? // Load markWord from oop into mark. >>>>>>>>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>>>>>>>> >>>>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>> +??? z_btrue(done); >>>>>>>>>>>>> +? } >>>>>>>>>>>>> + >>>>>>>>>>>>> ????? if (try_bias) { >>>>>>>>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, >>>>>>>>>>>>> Z_R0, done); >>>>>>>>>>>>> ????? } >>>>>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From patricio.chilano.mateo at oracle.com Mon Aug 10 13:46:08 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Mon, 10 Aug 2020 10:46:08 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> Message-ID: <3aacdaa8-4ddb-0b2a-f8f8-6423453cf3af@oracle.com> Hi Martin, On 8/7/20 4:33 PM, Doerr, Martin wrote: > > Hi Patricio, > > thanks for incorporating my fixes. > > I don?t know if I can find time for measurements next week. I have a > lot of other things to do. > > Feel free to push it after you got enough reviews. > Ok, I'll do. Thanks for reviewing this! Patricio > > Best regards, > > Martin > > *From:*Patricio Chilano > *Sent:* Freitag, 7. August 2020 17:29 > *To:* Doerr, Martin ; > daniel.daugherty at oracle.com; David Holmes ; > hotspot-runtime-dev at openjdk.java.net; hotspot-jfr-dev at openjdk.java.net > *Subject:* Re: RFR 8242263: Diagnose synchronization on primitive wrappers > > Hi Martin, > > On 8/7/20 12:58 AM, Doerr, Martin wrote: > > Hi Patricio, > > thanks for the update. > > Sorry for causing trouble related to the tbnz instructions on aarch64. > > My concern was basically to use the correct number of Bytes for the load instructions. > > You have fixed that in webrev v3. Thanks. > > Unfortunately, I did the same mistake regarding the flag requirements on PPC64 and s390. And we have a wrong label on PPC64 (otherwise branch destination can be too far). > > Test duration is still terrible with the slow debug build. I still got timeouts. (Maybe I should better use fast debug builds.) > > I measured the time in my Mac (fastdebug build). The FatalTest takes ~ > less than a second to run, and we have 7*8=56 of those. The LogTest > takes between 1-4 seconds depending on the flag, and we have 7 of > those. Overall it takes 2-3 minutes to run. Maybe you can measure > those times too to see where its taking so long? > > > Fixes see below. > > Great, added the new fixes. Here is v4: > > Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v4/webrev/ > Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v4/inc/webrev/ > > > > Sometimes simple changes can require quite some time ... > > Yep, no problem? : ) > > Thanks! > > > Patricio > > Best regards, > > Martin > > diff -r f629202b3862 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp > > --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp???? Thu Aug 06 22:03:39 2020 +0200 > > +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp???? Thu Aug 06 23:26:02 2020 +0200 > > @@ -109,7 +109,7 @@ > > ???? load_klass(Rscratch, Roop); > > ???? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); > > ???? testbitdi(CCR0, R0, Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS)); > > -??? bne(CCR0, slow_case); > > +??? bne(CCR0, slow_int); > > ?? } > > ?? if (UseBiasedLocking) { > > diff -r f629202b3862 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp > > --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp??????? Thu Aug 06 22:03:39 2020 +0200 > > +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp??????? Thu Aug 06 23:26:02 2020 +0200 > > @@ -2839,8 +2839,8 @@ > > ?? if (DiagnoseSyncOnPrimitiveWrappers != 0) { > > ???? load_klass(temp, oop); > > ???? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); > > -??? testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); > > -??? bne(CCR0, cont); > > +??? testbitdi(flag, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); > > +??? bne(flag, cont); > > ?? } > > ?? if (try_bias) { > > diff -r f629202b3862 src/hotspot/cpu/s390/macroAssembler_s390.cpp > > --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp????? Thu Aug 06 22:03:39 2020 +0200 > > +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp????? Thu Aug 06 23:26:02 2020 +0200 > > @@ -3360,8 +3360,10 @@ > > ?? if (DiagnoseSyncOnPrimitiveWrappers != 0) { > > ???? load_klass(Z_R1_scratch, oop); > > -??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), exact_log2(JVM_ACC_IS_BOX_CLASS)); > > -??? z_btrue(done); > > +??? z_l(Z_R1_scratch, Address(Z_R1_scratch, Klass::access_flags_offset())); > > +??? assert((JVM_ACC_IS_BOX_CLASS & 0xFFFF) == 0, "or change following instruction"); > > +??? z_nilh(Z_R1_scratch, JVM_ACC_IS_BOX_CLASS >> 16); > > +??? z_brne(done); > > ?? } > > ?? if (try_bias) { > > diff -r f629202b3862 test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java > > --- a/test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java??????? Thu Aug 06 22:03:39 2020 +0200 > > +++ b/test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java??????? Thu Aug 06 23:26:02 2020 +0200 > > @@ -31,7 +31,7 @@ > > ? * @bug 8242263 > > ? * @summary Exercise DiagnoseSyncOnPrimitiveWrappers diagnostic flag > > ? * @library /test/lib > > - * @run driver SyncOnPrimitiveWrapperTest > > + * @run driver/timeout=180000 SyncOnPrimitiveWrapperTest > > ? */ > > public class SyncOnPrimitiveWrapperTest { > From alexander.scherbatiy at bell-sw.com Mon Aug 10 14:29:28 2020 From: alexander.scherbatiy at bell-sw.com (Alexander Scherbatiy) Date: Mon, 10 Aug 2020 17:29:28 +0300 Subject: RFR 8241053: Hotspot runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java test fails on Alpine Linux with debug build In-Reply-To: <5267bc12-ae35-4efa-b06a-bc98f69863a6@bell-sw.com> References: <87a32c5b-8e96-f323-1e20-8df181ab3ec0@bell-sw.com> <5b530109-bda2-820e-6070-d779f08b64f6@oracle.com> <64281c1f-970d-b792-eb42-63ae1488694b@bell-sw.com> <19c11cfa-480f-f0ea-1cd8-8431522de8b5@samersoff.net> <5267bc12-ae35-4efa-b06a-bc98f69863a6@bell-sw.com> Message-ID: <2e85bfdf-5fea-88fe-1e1a-426fbafb1db8@bell-sw.com> On 05.08.2020 11:41, Alexander Scherbatiy wrote: > On 05.08.2020 10:16, Dmitry Samersoff wrote: > >> Hello Alexander, >> >> Sorry for being later to the party. It might be better to write it as: >> >> if (status == EINVAL) { >> ????? // pthread_attr_setstacksize() function can fail >> ????? // if the stack size exceeds a system-imposed limit. >> ????? // >> ... >> >> ??? return false; >> } >> >> assert_status(status == 0, status, "pthread_attr_setstacksize"); >> >> I.e. just move "assert" after "if" in your initial fix. > > ? It looks like the behavior would be different when > pthread_attr_setstacksize() returns error other than EINVAL on > non-debug build. > ? In the former case there will be the thread cleanup and return from > the os::create_thread(). > ? In the later case the execution will be continued. > ? I am not sure which one is better. ? I would prefer to leave the current fix because it follows the fail fast principle. ? Thanks, ? Alexander. > > ? Thanks, > ? Alexander. > >> >> But I will not push for it. Fix looks good to me. >> >> -Dmitry >> >> On 04.08.2020 12:11, Alexander Scherbatiy wrote: >>> Could you review the updated fix: >>> ?? http://cr.openjdk.java.net/~alexsch/8241053/webrev.01 >>> >>> The assert_status() and the thread cleanup is moved under the "if >>> (status != 0)" check. >>> >>> Thanks, >>> Alexander. >>> >>> On 04.08.2020 01:51, David Holmes wrote: >>>> Hi Alexander, >>>> >>>> Thanks for fixing this. >>>> >>>> On 4/08/2020 1:33 am, Alexander Scherbatiy wrote: >>>>> Hello, >>>>> >>>>> Could you review the fix for the issue: >>>>> ?? Bug: https://bugs.openjdk.java.net/browse/JDK-8241053 >>>>> ?? Webrev: http://cr.openjdk.java.net/~alexsch/8241053/weberv.00 >>>> >>>> I would simplify this a little: >>>> >>>> ?? assert_status(status == 0 || status == EINVAL, status, >>>> "pthread_attr_setstacksize"); >>>> ?? // pthread_attr_setstacksize() function can fail >>>> ?? // if the stack size exceeds a system-imposed limit. >>>> ?? if (status == EINVAL) { >>>> >>>> to >>>> ?? if (status != 0) { >>>> ???? // pthread_attr_setstacksize() function can fail >>>> ???? // if the stack size exceeds a system-imposed limit. >>>> ???? assert_status(status == EINVAL, status, >>>> "pthread_attr_setstacksize"); >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>> >>>>> The following tests fail on Linux Alpine with musl libc: >>>>> test/hotspot/jtreg/runtime/Thread/TestThreadStackSizes.java >>>>> test/hotspot/jtreg/runtime/CommandLine/OptionsValidation/TestOptionsWithRanges.java >>>>> >>>>> >>>>> This is because muls pthread_attr_setstacksize(3) function >>>>> implementation [1] returns EINVAL when stack size exceeds the >>>>> certain limit. >>>>> >>>>> According to POSIX pthread_attr_setstacksize() function >>>>> description [2]: >>>>> ??The pthread_attr_setstacksize() function will fail if: >>>>> ???? [EINVAL]? The value of stacksize is less than >>>>> PTHREAD_STACK_MIN or exceeds a system-imposed limit. >>>>> >>>>> The proposed fix excludes EINVAL value from assert_status() and >>>>> for the EINVAL return value makes the same cleanup actions and >>>>> returns false from os::create_thread() function as it is done for >>>>> AIX[3]. >>>>> >>>>> To reproduce the issue it needs to build Portola from repository >>>>> [4] using instructions from JEP 386: Alpine Linux/x64 Port [5]. >>>>> After the fix both TestThreadStackSizes.java and >>>>> TestOptionsWithRanges.java tests pass on Alpine Linux with musl >>>>> libc and on Ubuntu 18.0.4. >>>>> >>>>> [1] >>>>> https://git.musl-libc.org/cgit/musl/tree/src/thread/pthread_attr_setstacksize.c >>>>> >>>>> [2] >>>>> https://pubs.opengroup.org/onlinepubs/007908799/xsh/pthread_attr_setstacksize.html >>>>> >>>>> [3] >>>>> https://hg.openjdk.java.net/jdk/jdk/file/c8102e6fc512/src/hotspot/os/aix/os_aix.cpp#l890 >>>>> >>>>> [4] https://github.com/openjdk/portola >>>>> [5] https://openjdk.java.net/jeps/386 >>>>> >>>>> Thanks, >>>>> Alexander. >>>>> >> From coleen.phillimore at oracle.com Mon Aug 10 17:52:01 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 10 Aug 2020 13:52:01 -0400 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <39f9b363-8ce1-563c-94b8-8525ff03ccc7@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> <2b030b14-0dc2-2de6-f01b-cc4f9ae80149@oracle.com> <41fd3f22-4266-de46-8d32-47ba6295218b@oracle.com> <39f9b363-8ce1-563c-94b8-8525ff03ccc7@oracle.com> Message-ID: On 8/10/20 9:44 AM, Patricio Chilano wrote: > Hi Coleen, > > On 8/7/20 12:58 PM, Coleen Phillimore wrote: >> >> >> On 8/7/20 11:49 AM, Patricio Chilano wrote: >>> Hi Coleen, >>> >>> On 8/7/20 10:53 AM, Coleen Phillimore wrote: >>>> >>>> >>>> On 8/6/20 9:45 PM, Patricio Chilano wrote: >>>>> Hi Coleen, >>>>> >>>>> Thanks for looking at this. >>>>> >>>>> On 8/6/20 7:17 PM, Coleen Phillimore wrote: >>>>>> >>>>>> Patricio,? One question: >>>>>> >>>>>> http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/src/hotspot/share/runtime/synchronizer.cpp.udiff.html >>>>>> >>>>>> >>>>>> + // adjust bcp to point back to monitorenter so that we print >>>>>> the correct line numbers >>>>>> >>>>>> >>>>>> How do the fatal and event print the correct line numbers here? I >>>>>> see the logging gets it from the stack trace. Should the abort >>>>>> message have more information in it?? You can get the source and >>>>>> line number information in the same way that print_stack_on() >>>>>> gets it. >>>>> For the fatal error case I'm not printing line numbers as I do >>>>> with the warning case or as you get with JFR events. But maybe I >>>>> should print the stack too and then exit the VM. You can get the >>>>> stack info and the line number of the monitorenter bytecode that >>>>> caused the crash from the generated hs_err file though. >>>> >>>> Can you send a sample, please? >>> At the top of the hs_err you will get: >>> >>> ?# >>> ?# A fatal error has been detected by the Java Runtime Environment: >>> ?# >>> ?#? Internal Error >>> (/xx/xxx/xxxx/open/src/hotspot/share/runtime/synchronizer.cpp:574), >>> pid=48810, tid=48811 >>> ?#? fatal error: Synchronizing on object 0x00000007ff0758a8 of klass >>> java.lang.Integer >>> ?# >>> >>> And then on the stacktrace you can see: >>> >>> Stack: [0x00007f982b6ce000,0x00007f982b7cf000], >>> sp=0x00007f982b7cd5c0,? free space=1021k >>> Native frames: (J=compiled Java code, A=aot compiled Java code, >>> j=interpreted, Vv=VM code, C=native code) >>> V? [libjvm.so+0x1662385] >>> ObjectSynchronizer::handle_sync_on_primitive_wrapper(Handle, >>> Thread*)+0x185 >>> V? [libjvm.so+0x16683ac]? ObjectSynchronizer::enter(Handle, >>> BasicLock*, Thread*)+0x21c >>> V? [libjvm.so+0xce88fa] >>> InterpreterRuntime::monitorenter(JavaThread*, BasicObjectLock*)+0x13a >>> j? SimpleTest.main([Ljava/lang/String;)V+24 >>> v? ~StubRoutines::call_stub >>> V? [libjvm.so+0xd0263a]? JavaCalls::call_helper(JavaValue*, >>> methodHandle const&, JavaCallArguments*, Thread*)+0x62a >>> V? [libjvm.so+0xe22c8e]? jni_invoke_static(JNIEnv_*, JavaValue*, >>> _jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, Thr >>> ?.constprop.1]+0x35e >>> V? [libjvm.so+0xe28e6b]? jni_CallStaticVoidMethod+0x21b >>> C? [libjli.so+0x496e]? JavaMain+0xc1e >>> C? [libjli.so+0x7759]? ThreadJavaMain+0x9 >>> >>> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) >>> j? SimpleTest.main([Ljava/lang/String;)V+24 >>> v? ~StubRoutines::call_stub >>> >>> Where +24 is the bci. But that would be with the interpreter, with >>> c1 that line will show as: >>> >>> J 576 c1 SimpleTest.main([Ljava/lang/String;)V (205 bytes) @ >>> 0x00007f97b5ae5dc7 >>> >>> You have the method where it failed but not the exact bci. So maybe >>> I should print the stack and then exit the VM. >>> >>>> I guess there isn't a handy function to show the source file name >>>> and line number in the fatal error message. >>> I'll see if there is an easy way to have it in the fatal error >>> message otherwise I can just print the stack as with the warning >>> case and exit. >> >> I don't know if we want the fatal message to print the whole stack. >> But an improvement would be: >> >> ?#? fatal error: Synchronizing on object 0x00000007ff0758a8 of klass >> java.lang.Integer in method->external_name()+bci >> >> which you have in the function and then people won't have to go to >> the hs_err file. > I tested two different changes. The first one gets the method and bci > and adds it to the fatal error message as you said: > > diff --git a/src/hotspot/share/runtime/synchronizer.cpp > b/src/hotspot/share/runtime/synchronizer.cpp > --- a/src/hotspot/share/runtime/synchronizer.cpp > +++ b/src/hotspot/share/runtime/synchronizer.cpp > @@ -572,5 +572,9 @@ > ?? if (DiagnoseSyncOnPrimitiveWrappers == FATAL_EXIT) { > ???? ResourceMark rm(self); > -??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass %s", > p2i(obj()), obj->klass()->external_name()); > +??? vframeStream vfst(self); > +??? Method* method = vfst.method(); > +??? int bci = vfst.bci(); > +??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass %s in > \"%s\" at bci:%d", > +????????? p2i(obj()), obj->klass()->external_name(), method ? > method->external_name() : "", bci); > ?? } > > Which produces the following output: > > #? Internal Error > (/scratch/pchilano/random7/open/src/hotspot/share/runtime/synchronizer.cpp:577), > pid=13759, tid=13785 > #? fatal error: Synchronizing on object 0x00000001013d72d8 of klass > java.lang.Float in "void SimpleTest.run()" at bci:24 > # > > In the second one I use the same print_stack_on() method I use for the > warning case but print on a stringStream object instead, and then I > print the first line of the buffer only. > > diff --git a/src/hotspot/share/runtime/synchronizer.cpp > b/src/hotspot/share/runtime/synchronizer.cpp > --- a/src/hotspot/share/runtime/synchronizer.cpp > +++ b/src/hotspot/share/runtime/synchronizer.cpp > @@ -572,5 +572,12 @@ > ?? if (DiagnoseSyncOnPrimitiveWrappers == FATAL_EXIT) { > ???? ResourceMark rm(self); > -??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass %s", > p2i(obj()), obj->klass()->external_name()); > +??? stringStream ss; > +??? self->print_stack_on(&ss); > +??? char* base = (char*)strstr(ss.base(), "at"); > +??? char* newline = (char*)strchr(ss.base(), '\n'); > +??? if (newline != NULL) { > +????? *newline = '\0'; > +??? } > +??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass %s %s", > p2i(obj()), obj->klass()->external_name(), base); > ?? } > > Which produces the following output: > > #? Internal Error > (/scratch/pchilano/random7/open/src/hotspot/share/runtime/synchronizer.cpp:581), > pid=9288, tid=9306 > #? fatal error: Synchronizing on object 0x00000001013d72d8 of klass > java.lang.Float at SimpleTest.run(SimpleTest.java:70) > # > > I think the second method is better since it provides the actual file > and line number, what do you think? Yes, this second output looks really nice! Coleen > > Thanks, > Patricio >> Thanks, >> Coleen >> >> >>> >>> >>> Thanks, >>> Patricio >>>> Thanks, >>>> Coleen >>>>> >>>>> I don't know which exact technique JFR uses to print line numbers >>>>> but it has to include reading the current bcp. Since when coming >>>>> from the interpreter for monitorenter the bcp is always already >>>>> pointing to the next instruction we need to decrement it to print >>>>> the correct line numbers. I tested it and if I don't fix the bcp, >>>>> as expected JFR too will print the next line number for the top >>>>> frame. >>>>> >>>>> >>>>> Patricio >>>>>> Otherwise, this looks good to me. >>>>>> Coleen >>>>>> >>>>>> >>>>>> On 8/6/20 2:48 PM, Patricio Chilano wrote: >>>>>>> Hi Dan, >>>>>>> >>>>>>> On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: >>>>>>>> I'm peeking ahead to the next webrev... :-) >>>>>>>> >>>>>>>> > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ >>>>>>> : ) >>>>>>> >>>>>>>> General comments: >>>>>>>> ? - check files for copyright year updates. >>>>>>> Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, >>>>>>> c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. >>>>>>> >>>>>>>> src/hotspot/share/runtime/synchronizer.hpp >>>>>>>> ??? No comments. >>>>>>>> >>>>>>>> src/hotspot/share/runtime/synchronizer.cpp >>>>>>>> ??? L507: ? const markWord mark = obj->mark(); >>>>>>>> ??? L508: >>>>>>>> ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && >>>>>>>> obj->klass()->is_box()) { >>>>>>>> ??? L510: ??? return false; >>>>>>>> ??? L511: ? } >>>>>>>> ??????? The new bailout on L509-511 can move above L507. >>>>>>> Moved. >>>>>>> >>>>>>>> L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of >>>>>>>> klass %s", p2i(obj()), obj->klass()->external_name()); >>>>>>>> ??????? This external_name() call does not have a ResourceMark. >>>>>>> Good catch! I had one in a previous version but then I changed >>>>>>> the conditionals and lost it for the fatal error case. The test >>>>>>> worked okay because for the main JavaThread there is a >>>>>>> ResourceMark in jni_invoke_static() (jni.cpp). >>>>>>> >>>>>>>> src/hotspot/share/logging/logTag.hpp >>>>>>>> ??? No comments. >>>>>>>> >>>>>>>> src/hotspot/share/oops/klass.hpp >>>>>>>> ??? No comments. >>>>>>>> >>>>>>>> src/hotspot/share/utilities/accessFlags.hpp >>>>>>>> ??? No comments. >>>>>>>> >>>>>>>> src/hotspot/share/runtime/globals.hpp >>>>>>>> ??? L814: ???????????? "0: off " >>>>>>>> ??????? Missing a ';' after "off". >>>>>>> Fixed. >>>>>>> >>>>>>>> L816: ???????????? "2: log message to stdout. >>>>>>>> ??????? Perhaps add "(by default)" after "stdout" or >>>>>>>> ??????? don't say where log output is at all. >>>>>>>> >>>>>>>> src/hotspot/share/runtime/arguments.cpp >>>>>>>> ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, >>>>>>>> true, LOG_TAGS(primitivewrappers)); >>>>>>>> ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp >>>>>>>> ??????? since it looks like logging output is configured to >>>>>>>> 'stdout'. >>>>>>>> ??????? I'm assuming that other log options to put output >>>>>>>> elsewhere >>>>>>>> ??????? are overridden by this code? >>>>>>> Right. So the logging is done under UL with the tag >>>>>>> primitivewrappers. If that tag is specified in Xlog then this >>>>>>> conditional will be skipped because !log_is_enabled(Info, >>>>>>> primitivewrappers) will be false. >>>>>>> >>>>>>>> src/hotspot/share/classfile/systemDictionary.cpp >>>>>>>> ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { >>>>>>>> ??????? nit - s/T_LONG+1/T_LONG + 1/ >>>>>>> Fixed. >>>>>>> >>>>>>>> L2191: >>>>>>>> _box_klasses[i]->set_prototype_header(markWord::prototype()); >>>>>>>> ??????? I assume we're keeping the prototype_header field when >>>>>>>> Biased Locking >>>>>>>> ??????? goes away? The reason I ask: >>>>>>>> >>>>>>>> ? ? ? ? static markWord prototype() { >>>>>>>> ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); >>>>>>>> ? ? ? ? } >>>>>>>> >>>>>>>> ??????? is because without Biased Locking, do we really need >>>>>>>> the prototype >>>>>>>> ??????? anymore? The initial markWord won't need possible >>>>>>>> variants... >>>>>>> Yes, I think it can go away unless somebody finds another use >>>>>>> for it. Maybe Valhalla. >>>>>>> >>>>>>>> src/hotspot/share/jfr/metadata/metadata.xml >>>>>>>> ??? L69: ? >>>>>>> ??????? Is the category "Java Application" because it's the >>>>>>>> application >>>>>>>> ??????? code that did something "wrong" here? Where >>>>>>>> "application" is loosely >>>>>>>> ??????? defined to include the possibility of auto generated >>>>>>>> code, library >>>>>>>> ??????? code and the like? Or perhaps "application" because >>>>>>>> something "above" >>>>>>>> ??????? the "Java Virtual Machine, Runtime" did the "wrong" >>>>>>>> thing here? >>>>>>> I don't know what the right category should be really. I saw the >>>>>>> events JavaMonitorEnter, JavaMonitorWait and JavaMonitorInflate >>>>>>> and thought this event should fall in the same category they do. >>>>>>> >>>>>>>> src/jdk.jfr/share/conf/jfr/default.jfc >>>>>>>> ??? No comments. >>>>>>>> >>>>>>>> src/jdk.jfr/share/conf/jfr/profile.jfc >>>>>>>> ??? No comments. >>>>>>>> >>>>>>>> test/lib/jdk/test/lib/jfr/EventNames.java >>>>>>>> ??? No comments. >>>>>>>> >>>>>>>> src/hotspot/cpu/aarch64/aarch64.ad >>>>>>>> ??? L3517: ????? __ tbnz(tmp, exact_log2(JVM_ACC_IS_BOX_CLASS), >>>>>>>> cont); >>>>>>>> >>>>>>>> ??? L3578: ??? __ bind(cont); >>>>>>>> ??? L3579: ??? // flag == EQ indicates success >>>>>>>> ??? L3580: ??? // flag == NE indicates failure >>>>>>>> ??????? If tbnz() branches to "cont" when we have a box class, >>>>>>>> what's >>>>>>>> ??????? the flag value set to (EQ or NE)? And what set that >>>>>>>> flag value? >>>>>>>> ??????? The reason I ask is I don't think tbnz() sets condition >>>>>>>> codes... >>>>>>> Right, it doesn't set condition codes, so I kept the version I >>>>>>> had (more on that below). >>>>>>> >>>>>>>> src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>>>> ??? No comments. >>>>>>>> >>>>>>>> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp >>>>>>>> ??? No comments. >>>>>>>> >>>>>>>> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp >>>>>>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>>>>>> Register obj_reg, >>>>>>>> Register swap_reg, >>>>>>>> Register tmp_reg, >>>>>>>> ?????????????????????????????????? ? ?? ???? bool >>>>>>>> swap_reg_contains_mark, >>>>>>>> Label& done, >>>>>>>> ??????????????????????????????????? ? ?? ??? Label* slow_case, >>>>>>>> BiasedLockingCounters* counters) { >>>>>>>> ??????? I think you've changed the only callers of >>>>>>>> biased_locking_enter() >>>>>>>> ??????? that cared about the return value with this changeset >>>>>>>> so it can >>>>>>>> ??????? be changed to a void function. >>>>>>> Ok, this is what I mentioned to David in a previous email. Done. >>>>>>> >>>>>>>> src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp >>>>>>>> ??? No comments. >>>>>>>> >>>>>>>> src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp >>>>>>>> ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), >>>>>>>> done); >>>>>>>> >>>>>>>> ??? L131: ? bind(done); >>>>>>>> ??? L132: >>>>>>>> ??? L133: ? // At this point flags are set as follows: >>>>>>>> ??? L134: ? //? EQ -> Success >>>>>>>> ??? L135: ? //? NE -> Failure, branch to slow path >>>>>>>> ??????? If tbnz() branches to "done" when we have a box class, >>>>>>>> what's >>>>>>>> ??????? the flag value set to (EQ or NE)? And what set that >>>>>>>> flag value? >>>>>>>> ??????? The reason I ask is I don't think tbnz() sets condition >>>>>>>> codes... >>>>>>> Right. Same as above, I kept the version I had. >>>>>>> >>>>>>>> src/hotspot/cpu/arm/interp_masm_arm.cpp >>>>>>>> ??? No comments. >>>>>>>> >>>>>>>> src/hotspot/cpu/arm/macroAssembler_arm.cpp >>>>>>>> ??? int MacroAssembler::biased_locking_enter(Register obj_reg, >>>>>>>> Register swap_reg, Register tmp_reg, >>>>>>>> ????????????????????? ? ?? ????????????????? bool >>>>>>>> swap_reg_contains_mark, >>>>>>>> Register tmp2, >>>>>>>> Label& done, Label& slow_case, >>>>>>>> BiasedLockingCounters* counters) { >>>>>>>> ??????? I think you've changed the only callers of >>>>>>>> biased_locking_enter() >>>>>>>> ??????? that cared about the return value with this changeset >>>>>>>> so it can >>>>>>>> ??????? be changed to a void function. >>>>>>> Done. >>>>>>> >>>>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>> ??? No comments on the PPC code. >>>>>>>> >>>>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>> src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>> ??? No comments on the S390 code. >>>>>>>> >>>>>>>> src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp >>>>>>>> ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); >>>>>>>> ??????? What will this do with a 'noreg' value? (I need a >>>>>>>> refresher.) >>>>>>> When not in 64 bit mode that register just won't be used. >>>>>>> >>>>>>>> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp >>>>>>>> ??? No comments. >>>>>>>> >>>>>>>> src/hotspot/cpu/x86/interp_masm_x86.cpp >>>>>>>> ??? No comments. >>>>>>>> >>>>>>>> src/hotspot/cpu/x86/macroAssembler_x86.cpp >>>>>>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>>>>>> Register obj_reg, >>>>>>>> Register swap_reg, >>>>>>>> Register tmp_reg, >>>>>>>> Register tmp_reg2, >>>>>>>> ?????????????????????????????????????? ? ? ? bool >>>>>>>> swap_reg_contains_mark, >>>>>>>> Label& done, >>>>>>>> ?????????????????????????????????? ? ?? ???? Label* slow_case, >>>>>>>> BiasedLockingCounters* counters) { >>>>>>>> ??????? I think you've changed the only caller of >>>>>>>> biased_locking_enter() >>>>>>>> ??????? that cared about the return value with this changeset >>>>>>>> so it can >>>>>>>> ??????? be changed to a void function. >>>>>>> Done. >>>>>>> >>>>>>>> test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >>>>>>>> ??? L30:? * @test SyncOnPrimitiveWrapperTest >>>>>>>> ??????? No parameter to @test directive. >>>>>>> Removed. >>>>>>> >>>>>>>> L136: ??????? private static long sharedCounter = 0L; >>>>>>>> ??????? Since you don't do anything with sharedCounter other >>>>>>>> than increment it, >>>>>>>> ??????? can the compilers optimize it away? If it can be >>>>>>>> optimized away, does >>>>>>>> ??????? that mean that: >>>>>>>> >>>>>>>> ??????????? L142: ??????????????? synchronized (obj) { >>>>>>>> >>>>>>>> ??????? can also be optimized away? >>>>>>>> >>>>>>>> ??????? I don't think that: >>>>>>>> >>>>>>>> ??????????? L161: ??????????????? synchronized (sharedLock1) { >>>>>>>> >>>>>>>> ??????? can be optimized away because it is shared by multiple >>>>>>>> threads. >>>>>>>> >>>>>>>> test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java >>>>>>>> >>>>>>>> ??? Similar questions about 'counter' being optimized away. >>>>>>>> ??? Similar question about "synchronized (obj) {" being >>>>>>>> optimized away. >>>>>>> I'm not sure if the compiler will optimize it away. Seems >>>>>>> unlikely given the counter we are incrementing is not just local >>>>>>> to some thread. >>>>>>> >>>>>>> >>>>>>> Ok, below is v3 which has the following changes: >>>>>>> - Use a 32 bit load for the _access_flags field, instead of 64 >>>>>>> - Martin's implementation for s390 and fix for PPC >>>>>>> - Faster LogTest version >>>>>>> - Above changes based on Dan review >>>>>>> >>>>>>> I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 >>>>>>> only) again with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I >>>>>>> checked it builds in arm32, ppc and s390. >>>>>>> >>>>>>> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev >>>>>>> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev >>>>>>> >>>>>>> >>>>>>> @Martin: >>>>>>> Please check if the test doesn't timeout for you now with the >>>>>>> changes I made to LogTest. >>>>>>> Also, I tried to use tbnz in aarch64 and arm32 instead of tst + >>>>>>> br (except for c2 since we actually need to set the condition >>>>>>> flags), but for c1 I was getting an assertion in the compiler >>>>>>> thread: >>>>>>> >>>>>>> guarantee(chk == -1 || chk == 0) failed: Field too big for insn >>>>>>> >>>>>>> This is the stack when that happens: >>>>>>> >>>>>>> MacroAssembler::pd_patch_instruction_size(unsigned char*, >>>>>>> unsigned char*)+0x398 >>>>>>> AbstractAssembler::bind(Label&)+0x118 >>>>>>> MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 >>>>>>> LIR_Assembler::emit_slow_case_stubs()+0x54 >>>>>>> Compilation::emit_code_body()+0x17c >>>>>>> >>>>>>> The change was simply: >>>>>>> >>>>>>> diff --git >>>>>>> a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>>> b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>>> --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>>> +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>>> @@ -78,7 +78,6 @@ >>>>>>> ?? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>> ???? load_klass(hdr, obj); >>>>>>> -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); >>>>>>> -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); >>>>>>> -??? br(Assembler::NE, slow_case); >>>>>>> +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); >>>>>>> +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); >>>>>>> ?? } >>>>>>> >>>>>>> So the issue must be related to where we want to jump. >>>>>>> >>>>>>> Thanks, >>>>>>> Patricio >>>>>>>> Dan >>>>>>>> >>>>>>>> >>>>>>>> On 8/5/20 9:01 AM, Patricio Chilano wrote: >>>>>>>>> Hi Martin, >>>>>>>>> >>>>>>>>> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>>>>>>>>> Hi Patricio, >>>>>>>>>> >>>>>>>>>> using 8 Byte load instructions for jint fields is a terrible >>>>>>>>>> coding style! >>>>>>>>>> Someone else may see it and use an 8 Byte store. Will result >>>>>>>>>> in great fun for debugging! >>>>>>>>>> >>>>>>>>>> And on Big Endian you will simply access the wrong bits. >>>>>>>>> Ah, of course! Those 32 bits will be in the wrong place when >>>>>>>>> doing the test. >>>>>>>>> >>>>>>>>>> Note that Big Endian Platforms are AIX, old linux ppc, s390, >>>>>>>>>> SPARC. I don't think you have tested on them. >>>>>>>>>> >>>>>>>>>>> We could remove the nested synchronized statements in the >>>>>>>>>>> run() method >>>>>>>>>>> so that we don't generate that much logging. We could also >>>>>>>>>>> lower >>>>>>>>>>> LOOP_COUNT. I think the issue is also because we are running >>>>>>>>>>> LogTest >>>>>>>>>>> with multiple flag combinations. Not sure what we should >>>>>>>>>>> touch first. >>>>>>>>>>> Maybe the synchronized statements, have only one or two and >>>>>>>>>>> test that >>>>>>>>>>> first? >>>>>>>>>> Sounds like helpful ideas. Please go ahead and strip things >>>>>>>>>> down. >>>>>>>>> Great, I will send v3 later with those changes. >>>>>>>>> >>>>>>>>> Thanks Martin! >>>>>>>>> >>>>>>>>> Patricio >>>>>>>>>> Thanks for taking care of it. >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> Martin >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: Patricio Chilano >>>>>>>>>>> Sent: Dienstag, 4. August 2020 20:26 >>>>>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>>>>> ; >>>>>>>>>>> hotspot-runtime-dev at openjdk.java.net; >>>>>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on >>>>>>>>>>> primitive wrappers >>>>>>>>>>> >>>>>>>>>>> Hi Martin, >>>>>>>>>>> >>>>>>>>>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>>>>>>>>> Hi Patricio, >>>>>>>>>>>> >>>>>>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>>>>>> sending v3. >>>>>>>>>>>> Thanks. I forgot to mention arm + aarch64. >>>>>>>>>>>> Aarch64 uses ldrw + tbnz. >>>>>>>>>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>>>>>>>>> >>>>>>>>>>>> I remember having seen the same mistake ?? >>>>>>>>>>>> And nobody noticed it on little Endian platforms. >>>>>>>>>>> Ok, I can use a tbnz instead of test and then a branch on >>>>>>>>>>> arm32 and aarch64. >>>>>>>>>>> Shouldn't a normal ldr in arm32 work fine? >>>>>>>>>>> Also in 64 bits (either x64 or aarch64) I don't see the >>>>>>>>>>> issue of using a >>>>>>>>>>> 64 bit load, besides the fact that we only care about the >>>>>>>>>>> first 32 bits. >>>>>>>>>>> Regardless of the endianness, aren't we masking out the >>>>>>>>>>> upper part when >>>>>>>>>>> we do AND/TEST or if we test a bit in the 0-31 bit range? >>>>>>>>>>> Otherwise it >>>>>>>>>>> seems it should have failed for one of those platforms in my >>>>>>>>>>> tests. >>>>>>>>>>> >>>>>>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe >>>>>>>>>>>>> you can >>>>>>>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>>>>>>> works and I >>>>>>>>>>>>> can add it to the test. >>>>>>>>>>>> That seems to have an effect. But now I'm not patient >>>>>>>>>>>> enough to wait for >>>>>>>>>>> the test to finish. >>>>>>>>>>>> Maybe the problem is that I'm using slow debug builds. >>>>>>>>>>>> But is there a chance to make the test quicker without >>>>>>>>>>>> losing coverage >>>>>>>>>>> significantly? >>>>>>>>>>>> I think the test is too slow this way to run it on a >>>>>>>>>>>> regular basis. We'd need to >>>>>>>>>>> spend dedicated machines for it. >>>>>>>>>>> We could remove the nested synchronized statements in the >>>>>>>>>>> run() method >>>>>>>>>>> so that we don't generate that much logging. We could also >>>>>>>>>>> lower >>>>>>>>>>> LOOP_COUNT. I think the issue is also because we are running >>>>>>>>>>> LogTest >>>>>>>>>>> with multiple flag combinations. Not sure what we should >>>>>>>>>>> touch first. >>>>>>>>>>> Maybe the synchronized statements, have only one or two and >>>>>>>>>>> test that >>>>>>>>>>> first? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Patricio >>>>>>>>>>>> Best regards, >>>>>>>>>>>> Martin >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>>> From: Patricio Chilano >>>>>>>>>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>>>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>>>>>>> ; >>>>>>>>>>>>> hotspot-runtime-dev at openjdk.java.net; >>>>>>>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on >>>>>>>>>>>>> primitive wrappers >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for fixing PPC and taking care of s390! >>>>>>>>>>>>> >>>>>>>>>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>>>>>>>>> Hi Patricio, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I suggest to use movl + testl for checking the access >>>>>>>>>>>>>> flag as for other >>>>>>>>>>> access >>>>>>>>>>>>> flags on x64. >>>>>>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>>>>>> sending v3. >>>>>>>>>>>>> >>>>>>>>>>>>>> New version for PPC64 and s390 see below. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>>>>>>>>>> expected. >>>>>>>>>>>>> However, I'm getting timeout issues: >>>>>>>>>>>>>> timed out (timeout set to 120000ms, elapsed time >>>>>>>>>>>>>> including timeout >>>>>>>>>>>>> handling was 122372ms) >>>>>>>>>>>>>> Can we provide more time? >>>>>>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe >>>>>>>>>>>>> you can >>>>>>>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>>>>>>> works and I >>>>>>>>>>>>> can add it to the test. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Patricio >>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>> Martin >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>>>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue >>>>>>>>>>>>>> Aug 04 >>>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue >>>>>>>>>>>>>> Aug 04 >>>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>>> @@ -107,8 +107,8 @@ >>>>>>>>>>>>>> >>>>>>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>>>>>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>>>>> Rscratch); >>>>>>>>>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>>>>> +??? lwz(Rscratch, >>>>>>>>>>>>>> in_bytes(Klass::access_flags_offset()), Rscratch); >>>>>>>>>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>>> ??????? bne(CCR0, slow_case); >>>>>>>>>>>>>> ????? } >>>>>>>>>>>>>> >>>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>>>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>>> @@ -912,8 +912,8 @@ >>>>>>>>>>>>>> >>>>>>>>>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>>> ????????? load_klass(tmp, object); >>>>>>>>>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>>>>> tmp); >>>>>>>>>>>>>> +????? testbitdi(CCR0, R0, tmp, >>>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>>> ????????? bne(CCR0, slow_case); >>>>>>>>>>>>>> ??????? } >>>>>>>>>>>>>> >>>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>>>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>>>>>>>>> >>>>>>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>>> ??????? load_klass(temp, oop); >>>>>>>>>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>>>>> temp); >>>>>>>>>>>>>> +??? testbitdi(CCR0, R0, temp, >>>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>>> ??????? bne(CCR0, cont); >>>>>>>>>>>>>> ????? } >>>>>>>>>>>>>> >>>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>>>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue >>>>>>>>>>>>>> Aug 04 >>>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue >>>>>>>>>>>>>> Aug 04 >>>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>>> @@ -91,6 +91,12 @@ >>>>>>>>>>>>>> ????? // Save object being locked into the >>>>>>>>>>>>>> BasicObjectLock... >>>>>>>>>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>>>>>>>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>>> + z_btrue(slow_case); >>>>>>>>>>>>>> +? } >>>>>>>>>>>>>> + >>>>>>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>>>>>>>>> Z_R0_scratch, done, >>>>>>>>>>>>> &slow_case); >>>>>>>>>>>>>> ????? } >>>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>>> src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>>>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug >>>>>>>>>>>>>> 04 10:03:57 >>>>>>>>>>>>> 2020 +0200 >>>>>>>>>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>>>>> 16:04:31 >>>>>>>>>>>>> 2020 +0200 >>>>>>>>>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>>>>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>>>>>>>>> ????? z_lg(displaced_header, >>>>>>>>>>>>>> oopDesc::mark_offset_in_bytes(), object); >>>>>>>>>>>>>> >>>>>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>>> + z_btrue(slow_case); >>>>>>>>>>>>>> +? } >>>>>>>>>>>>>> + >>>>>>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>>>>>> ??????? biased_locking_enter(object, displaced_header, >>>>>>>>>>>>>> Z_R1, Z_R0, done, >>>>>>>>>>>>> &slow_case); >>>>>>>>>>>>>> ????? } >>>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>>>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue >>>>>>>>>>>>>> Aug 04 >>>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue >>>>>>>>>>>>>> Aug 04 >>>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>>>>>>>>> ????? // Load markWord from oop into mark. >>>>>>>>>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>>>>>>>>> >>>>>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>>> + z_btrue(done); >>>>>>>>>>>>>> +? } >>>>>>>>>>>>>> + >>>>>>>>>>>>>> ????? if (try_bias) { >>>>>>>>>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, >>>>>>>>>>>>>> Z_R0, done); >>>>>>>>>>>>>> ????? } >>>>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From martin.doerr at sap.com Mon Aug 10 19:20:32 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 10 Aug 2020 19:20:32 +0000 Subject: RFR: 8250636: iso8601_time returns incorrect offset part on MacOS In-Reply-To: References: <777fc30a-45ef-aa45-8f04-0bbe80c5a83d@azul.com> <0e1ed1ae-2011-2e4a-4f58-d99b57ede74c@azul.com> <934a8a40-dd51-512f-f010-909ed312783d@oracle.com> Message-ID: Hi, unfortunately, tm_gmtoff seems to be a BSD and GNU extension. It's not POSIX and doesn't build on AIX: os.cpp:163:34: error: no member named 'tm_gmtoff' in 'tm' We'll need a follow up fix. Best regards, Martin > -----Original Message----- > From: hotspot-runtime-dev > On Behalf Of gerard ziemski > Sent: Dienstag, 4. August 2020 19:02 > To: Dmitry Cherepanov ; David Holmes > ; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR: 8250636: iso8601_time returns incorrect offset part on > MacOS > > Looks great! > > On 8/4/20 4:57 AM, Dmitry Cherepanov wrote: > > Hi David, > > > > On 03.08.2020 10:53, David Holmes wrote: > >> Hi Dmitry, > >> > >> On 31/07/2020 8:29 pm, Dmitry Cherepanov wrote: > >>> Hi Gerard, > >>> > >>> Thanks for reviewing this, moving the logic of get_timezone to > >>> iso8601_time looks good to me too. > >>> Here's an updated webrev > >>> http://cr.openjdk.java.net/~dcherepanov/8250636/webrev.v4/ > >> This version also looks good to me. But now I see all the code clearly in > one place I have to wonder why we are not using tm_gmtoff on all platforms > except Windows? That would simplify the logic even further. > >> > >> That goes beyond fixing the current actual bug though so I'll leave that > choice to you. > > Thanks for the suggestion. This change would not work on Solaris. But given > the removal of Solaris port, I also see no other reasons why not to use > tm_gmtoff on platforms except Windows. > > > > Updated webrev: > http://cr.openjdk.java.net/~dcherepanov/8250636/webrev.v5/ > > Jdk-submit testing passed with this change > > > > Thanks, > > > > Dmitry > > From patricio.chilano.mateo at oracle.com Mon Aug 10 20:03:16 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Mon, 10 Aug 2020 17:03:16 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> Message-ID: <29c3d48b-1875-8ea4-792a-157cdd07fe2c@oracle.com> Hi Erik, Thanks for looking into this and for the offline discussions. On 8/10/20 10:19 AM, Erik Gahlin wrote: > > Hi Patricio, > > I have tried to review the JFR changes, but I need more information on > how the feature is going to be used. > > Is this something temporary that will be used gain insight for the > Valhalla project, or do you think this will be needed longer term? If > it is longer term, the bar is higher on what can be accepted. > > If this is a temporary feature, we could mark the event as > experimental in metadata.xml, similar to what we did with GC events > during the development of ZGC. Events that are experimental do not > show up by default in visualization tools such as JMC and can be > removed when they are not needed, or we have a better solution. > Right, it's temporary so we can mark it as experimental. I also changed the category name to be "Java Virtual Machine, Diagnostics". > For events to be enabled in default.jfc, they should not cause more > than 1% overhead, not even in pathological cases. To me, it seems this > could happen if you make a loop where it is triggered all the time. > > For events in profile.jfc, the overhead should still be low (1-2%), > but the target is the typical application. For example, allocation > profiling is only enabled in profile.jfc. Some application that are > allocation intensive could cause a higher overhead than 1%, but that > is OK because that configuration is only to be used for a short period > of time. That said, there is still a budget on how much space in a > recording an event can take up, so it also depends on how important > the information is for the user. To me, it seems this event will not > be that important for the average user, which makes me think it should > be disabled by default. > After our off-list discussion I left it as enabled. I also removed the threshold setting and added the startTime=false in metadata.xml. > To complicate things, I noticed that aa command line flag is also used > to enable the event. This is something we have worked hard to avoid > when it comes to JFR events. All configuration should happen using > configuration files. We had problem with this in the past where users > gets confused why their events are not enabled. When we ported JFR to > Hotspot, we got rid of those of those flags/events and it is a much > better situation. Again, if this a temporary event to discover usage > pattern for a release or two, it might be OK, but if we believe this > event to stick around, I think we should look into alternatives, or > not use JFR at all for this. > Yes, I see your point. Since we want to diagnose synchronization on these classes but we don't want to affect performance the flag has to be specified at startup otherwise the diagnostic instrumentation is disabled, so I don't see a clear way to enable the events from JFR alone. In any case, this will be a temporary event. Here is v5. Let me know if you are okay with the changes. Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/webrev/ Thanks Erik! Patricio > > Thanks > > Erik > >> On 28 Jul 2020, at 21:16, Patricio Chilano >> > > wrote: >> >> Hi all, >> >> Please review the following change that adds diagnostic capabilities >> when synchronizing on primitive wrapper classes. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8242263 >> Webrev: http://cr.openjdk.java.net/~pchilanomate/8242263/v1/webrev/ >> >> The new flag allows to identify synchronization on these classes and >> to take one of the following actions: exit the VM with fatal error, >> log a warning message, or issue a JFR event. The implementation uses >> a simple approach where a check is added at the beginning of the >> monitorenter generated code when the flag is enabled to check whether >> the object is of a primitive wrapper class. If it is, we jump to the >> slow path, otherwise we just continue as always. The extra >> instructions will be: load the klass of the object, load the >> _access_flags field for that klass, AND with a constant, and branch >> based on the result. The code will only be generated whenever the new >> opt-in diagnostic flag is enabled so performance won't be affected >> when off. >> >> In addition to the purpose described in the description of the bug, >> this flag will also be useful when trying to diagnose possible >> synchronization issues if these classes ever become inline types as >> part of the Valhalla project. >> >> I added test SyncOnPrimitiveWrapperTest.java that tests for the exit >> and logging cases. I added test TestSyncOnPrimitiveWrapperEvent.java >> to test for the JFR event case. >> I tested the patch running tiers1-6 in mach5 with the flag set to >> DiagnoseSyncOnPrimitiveWrappers=2. >> I checked it builds with arm32 and ppc but can't run any tests on >> those platforms, so it would be good if somebody can run the new test >> included in the patch. >> >> Let me know if you think I should run or add any more tests. >> >> Thanks! >> >> Patricio > From patricio.chilano.mateo at oracle.com Mon Aug 10 20:05:33 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Mon, 10 Aug 2020 17:05:33 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> <2b030b14-0dc2-2de6-f01b-cc4f9ae80149@oracle.com> <41fd3f22-4266-de46-8d32-47ba6295218b@oracle.com> <39f9b363-8ce1-563c-94b8-8525ff03ccc7@oracle.com> Message-ID: Hi Coleen, On 8/10/20 2:52 PM, Coleen Phillimore wrote: > > > On 8/10/20 9:44 AM, Patricio Chilano wrote: >> Hi Coleen, >> >> On 8/7/20 12:58 PM, Coleen Phillimore wrote: >>> >>> >>> On 8/7/20 11:49 AM, Patricio Chilano wrote: >>>> Hi Coleen, >>>> >>>> On 8/7/20 10:53 AM, Coleen Phillimore wrote: >>>>> >>>>> >>>>> On 8/6/20 9:45 PM, Patricio Chilano wrote: >>>>>> Hi Coleen, >>>>>> >>>>>> Thanks for looking at this. >>>>>> >>>>>> On 8/6/20 7:17 PM, Coleen Phillimore wrote: >>>>>>> >>>>>>> Patricio,? One question: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/src/hotspot/share/runtime/synchronizer.cpp.udiff.html >>>>>>> >>>>>>> >>>>>>> + // adjust bcp to point back to monitorenter so that we print >>>>>>> the correct line numbers >>>>>>> >>>>>>> >>>>>>> How do the fatal and event print the correct line numbers here? >>>>>>> I see the logging gets it from the stack trace. Should the abort >>>>>>> message have more information in it?? You can get the source and >>>>>>> line number information in the same way that print_stack_on() >>>>>>> gets it. >>>>>> For the fatal error case I'm not printing line numbers as I do >>>>>> with the warning case or as you get with JFR events. But maybe I >>>>>> should print the stack too and then exit the VM. You can get the >>>>>> stack info and the line number of the monitorenter bytecode that >>>>>> caused the crash from the generated hs_err file though. >>>>> >>>>> Can you send a sample, please? >>>> At the top of the hs_err you will get: >>>> >>>> ?# >>>> ?# A fatal error has been detected by the Java Runtime Environment: >>>> ?# >>>> ?#? Internal Error >>>> (/xx/xxx/xxxx/open/src/hotspot/share/runtime/synchronizer.cpp:574), >>>> pid=48810, tid=48811 >>>> ?#? fatal error: Synchronizing on object 0x00000007ff0758a8 of >>>> klass java.lang.Integer >>>> ?# >>>> >>>> And then on the stacktrace you can see: >>>> >>>> Stack: [0x00007f982b6ce000,0x00007f982b7cf000], >>>> sp=0x00007f982b7cd5c0,? free space=1021k >>>> Native frames: (J=compiled Java code, A=aot compiled Java code, >>>> j=interpreted, Vv=VM code, C=native code) >>>> V? [libjvm.so+0x1662385] >>>> ObjectSynchronizer::handle_sync_on_primitive_wrapper(Handle, >>>> Thread*)+0x185 >>>> V? [libjvm.so+0x16683ac]? ObjectSynchronizer::enter(Handle, >>>> BasicLock*, Thread*)+0x21c >>>> V? [libjvm.so+0xce88fa] >>>> InterpreterRuntime::monitorenter(JavaThread*, BasicObjectLock*)+0x13a >>>> j? SimpleTest.main([Ljava/lang/String;)V+24 >>>> v? ~StubRoutines::call_stub >>>> V? [libjvm.so+0xd0263a]? JavaCalls::call_helper(JavaValue*, >>>> methodHandle const&, JavaCallArguments*, Thread*)+0x62a >>>> V? [libjvm.so+0xe22c8e]? jni_invoke_static(JNIEnv_*, JavaValue*, >>>> _jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, Thr >>>> ?.constprop.1]+0x35e >>>> V? [libjvm.so+0xe28e6b]? jni_CallStaticVoidMethod+0x21b >>>> C? [libjli.so+0x496e]? JavaMain+0xc1e >>>> C? [libjli.so+0x7759]? ThreadJavaMain+0x9 >>>> >>>> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) >>>> j? SimpleTest.main([Ljava/lang/String;)V+24 >>>> v? ~StubRoutines::call_stub >>>> >>>> Where +24 is the bci. But that would be with the interpreter, with >>>> c1 that line will show as: >>>> >>>> J 576 c1 SimpleTest.main([Ljava/lang/String;)V (205 bytes) @ >>>> 0x00007f97b5ae5dc7 >>>> >>>> You have the method where it failed but not the exact bci. So maybe >>>> I should print the stack and then exit the VM. >>>> >>>>> I guess there isn't a handy function to show the source file name >>>>> and line number in the fatal error message. >>>> I'll see if there is an easy way to have it in the fatal error >>>> message otherwise I can just print the stack as with the warning >>>> case and exit. >>> >>> I don't know if we want the fatal message to print the whole stack. >>> But an improvement would be: >>> >>> ?#? fatal error: Synchronizing on object 0x00000007ff0758a8 of klass >>> java.lang.Integer in method->external_name()+bci >>> >>> which you have in the function and then people won't have to go to >>> the hs_err file. >> I tested two different changes. The first one gets the method and bci >> and adds it to the fatal error message as you said: >> >> diff --git a/src/hotspot/share/runtime/synchronizer.cpp >> b/src/hotspot/share/runtime/synchronizer.cpp >> --- a/src/hotspot/share/runtime/synchronizer.cpp >> +++ b/src/hotspot/share/runtime/synchronizer.cpp >> @@ -572,5 +572,9 @@ >> ?? if (DiagnoseSyncOnPrimitiveWrappers == FATAL_EXIT) { >> ???? ResourceMark rm(self); >> -??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass %s", >> p2i(obj()), obj->klass()->external_name()); >> +??? vframeStream vfst(self); >> +??? Method* method = vfst.method(); >> +??? int bci = vfst.bci(); >> +??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass %s in >> \"%s\" at bci:%d", >> +????????? p2i(obj()), obj->klass()->external_name(), method ? >> method->external_name() : "", bci); >> ?? } >> >> Which produces the following output: >> >> #? Internal Error >> (/scratch/pchilano/random7/open/src/hotspot/share/runtime/synchronizer.cpp:577), >> pid=13759, tid=13785 >> #? fatal error: Synchronizing on object 0x00000001013d72d8 of klass >> java.lang.Float in "void SimpleTest.run()" at bci:24 >> # >> >> In the second one I use the same print_stack_on() method I use for >> the warning case but print on a stringStream object instead, and then >> I print the first line of the buffer only. >> >> diff --git a/src/hotspot/share/runtime/synchronizer.cpp >> b/src/hotspot/share/runtime/synchronizer.cpp >> --- a/src/hotspot/share/runtime/synchronizer.cpp >> +++ b/src/hotspot/share/runtime/synchronizer.cpp >> @@ -572,5 +572,12 @@ >> ?? if (DiagnoseSyncOnPrimitiveWrappers == FATAL_EXIT) { >> ???? ResourceMark rm(self); >> -??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass %s", >> p2i(obj()), obj->klass()->external_name()); >> +??? stringStream ss; >> +??? self->print_stack_on(&ss); >> +??? char* base = (char*)strstr(ss.base(), "at"); >> +??? char* newline = (char*)strchr(ss.base(), '\n'); >> +??? if (newline != NULL) { >> +????? *newline = '\0'; >> +??? } >> +??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass %s >> %s", p2i(obj()), obj->klass()->external_name(), base); >> ?? } >> >> Which produces the following output: >> >> #? Internal Error >> (/scratch/pchilano/random7/open/src/hotspot/share/runtime/synchronizer.cpp:581), >> pid=9288, tid=9306 >> #? fatal error: Synchronizing on object 0x00000001013d72d8 of klass >> java.lang.Float at SimpleTest.run(SimpleTest.java:70) >> # >> >> I think the second method is better since it provides the actual file >> and line number, what do you think? > > Yes, this second output looks really nice! Great, please check v5: Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/webrev/ Patricio > Coleen > >> >> Thanks, >> Patricio >>> Thanks, >>> Coleen >>> >>> >>>> >>>> >>>> Thanks, >>>> Patricio >>>>> Thanks, >>>>> Coleen >>>>>> >>>>>> I don't know which exact technique JFR uses to print line numbers >>>>>> but it has to include reading the current bcp. Since when coming >>>>>> from the interpreter for monitorenter the bcp is always already >>>>>> pointing to the next instruction we need to decrement it to print >>>>>> the correct line numbers. I tested it and if I don't fix the bcp, >>>>>> as expected JFR too will print the next line number for the top >>>>>> frame. >>>>>> >>>>>> >>>>>> Patricio >>>>>>> Otherwise, this looks good to me. >>>>>>> Coleen >>>>>>> >>>>>>> >>>>>>> On 8/6/20 2:48 PM, Patricio Chilano wrote: >>>>>>>> Hi Dan, >>>>>>>> >>>>>>>> On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: >>>>>>>>> I'm peeking ahead to the next webrev... :-) >>>>>>>>> >>>>>>>>> > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ >>>>>>>> : ) >>>>>>>> >>>>>>>>> General comments: >>>>>>>>> ? - check files for copyright year updates. >>>>>>>> Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, >>>>>>>> c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. >>>>>>>> >>>>>>>>> src/hotspot/share/runtime/synchronizer.hpp >>>>>>>>> ??? No comments. >>>>>>>>> >>>>>>>>> src/hotspot/share/runtime/synchronizer.cpp >>>>>>>>> ??? L507: ? const markWord mark = obj->mark(); >>>>>>>>> ??? L508: >>>>>>>>> ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && >>>>>>>>> obj->klass()->is_box()) { >>>>>>>>> ??? L510: ??? return false; >>>>>>>>> ??? L511: ? } >>>>>>>>> ??????? The new bailout on L509-511 can move above L507. >>>>>>>> Moved. >>>>>>>> >>>>>>>>> L573: fatal("Synchronizing on object " INTPTR_FORMAT " of >>>>>>>>> klass %s", p2i(obj()), obj->klass()->external_name()); >>>>>>>>> ??????? This external_name() call does not have a ResourceMark. >>>>>>>> Good catch! I had one in a previous version but then I changed >>>>>>>> the conditionals and lost it for the fatal error case. The test >>>>>>>> worked okay because for the main JavaThread there is a >>>>>>>> ResourceMark in jni_invoke_static() (jni.cpp). >>>>>>>> >>>>>>>>> src/hotspot/share/logging/logTag.hpp >>>>>>>>> ??? No comments. >>>>>>>>> >>>>>>>>> src/hotspot/share/oops/klass.hpp >>>>>>>>> ??? No comments. >>>>>>>>> >>>>>>>>> src/hotspot/share/utilities/accessFlags.hpp >>>>>>>>> ??? No comments. >>>>>>>>> >>>>>>>>> src/hotspot/share/runtime/globals.hpp >>>>>>>>> ??? L814: ???????????? "0: off " >>>>>>>>> ??????? Missing a ';' after "off". >>>>>>>> Fixed. >>>>>>>> >>>>>>>>> L816: ???????????? "2: log message to stdout. >>>>>>>>> ??????? Perhaps add "(by default)" after "stdout" or >>>>>>>>> ??????? don't say where log output is at all. >>>>>>>>> >>>>>>>>> src/hotspot/share/runtime/arguments.cpp >>>>>>>>> ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, >>>>>>>>> true, LOG_TAGS(primitivewrappers)); >>>>>>>>> ??????? Hmmm... maybe ignore my comments about L816 in >>>>>>>>> globals.hpp >>>>>>>>> ??????? since it looks like logging output is configured to >>>>>>>>> 'stdout'. >>>>>>>>> ??????? I'm assuming that other log options to put output >>>>>>>>> elsewhere >>>>>>>>> ??????? are overridden by this code? >>>>>>>> Right. So the logging is done under UL with the tag >>>>>>>> primitivewrappers. If that tag is specified in Xlog then this >>>>>>>> conditional will be skipped because !log_is_enabled(Info, >>>>>>>> primitivewrappers) will be false. >>>>>>>> >>>>>>>>> src/hotspot/share/classfile/systemDictionary.cpp >>>>>>>>> ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { >>>>>>>>> ??????? nit - s/T_LONG+1/T_LONG + 1/ >>>>>>>> Fixed. >>>>>>>> >>>>>>>>> L2191: >>>>>>>>> _box_klasses[i]->set_prototype_header(markWord::prototype()); >>>>>>>>> ??????? I assume we're keeping the prototype_header field when >>>>>>>>> Biased Locking >>>>>>>>> ??????? goes away? The reason I ask: >>>>>>>>> >>>>>>>>> ? ? ? ? static markWord prototype() { >>>>>>>>> ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); >>>>>>>>> ? ? ? ? } >>>>>>>>> >>>>>>>>> ??????? is because without Biased Locking, do we really need >>>>>>>>> the prototype >>>>>>>>> ??????? anymore? The initial markWord won't need possible >>>>>>>>> variants... >>>>>>>> Yes, I think it can go away unless somebody finds another use >>>>>>>> for it. Maybe Valhalla. >>>>>>>> >>>>>>>>> src/hotspot/share/jfr/metadata/metadata.xml >>>>>>>>> ??? L69: ? >>>>>>>> ??????? Is the category "Java Application" because it's the >>>>>>>>> application >>>>>>>>> ??????? code that did something "wrong" here? Where >>>>>>>>> "application" is loosely >>>>>>>>> ??????? defined to include the possibility of auto generated >>>>>>>>> code, library >>>>>>>>> ??????? code and the like? Or perhaps "application" because >>>>>>>>> something "above" >>>>>>>>> ??????? the "Java Virtual Machine, Runtime" did the "wrong" >>>>>>>>> thing here? >>>>>>>> I don't know what the right category should be really. I saw >>>>>>>> the events JavaMonitorEnter, JavaMonitorWait and >>>>>>>> JavaMonitorInflate and thought this event should fall in the >>>>>>>> same category they do. >>>>>>>> >>>>>>>>> src/jdk.jfr/share/conf/jfr/default.jfc >>>>>>>>> ??? No comments. >>>>>>>>> >>>>>>>>> src/jdk.jfr/share/conf/jfr/profile.jfc >>>>>>>>> ??? No comments. >>>>>>>>> >>>>>>>>> test/lib/jdk/test/lib/jfr/EventNames.java >>>>>>>>> ??? No comments. >>>>>>>>> >>>>>>>>> src/hotspot/cpu/aarch64/aarch64.ad >>>>>>>>> ??? L3517: ????? __ tbnz(tmp, >>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS), cont); >>>>>>>>> >>>>>>>>> ??? L3578: ??? __ bind(cont); >>>>>>>>> ??? L3579: ??? // flag == EQ indicates success >>>>>>>>> ??? L3580: ??? // flag == NE indicates failure >>>>>>>>> ??????? If tbnz() branches to "cont" when we have a box class, >>>>>>>>> what's >>>>>>>>> ??????? the flag value set to (EQ or NE)? And what set that >>>>>>>>> flag value? >>>>>>>>> ??????? The reason I ask is I don't think tbnz() sets >>>>>>>>> condition codes... >>>>>>>> Right, it doesn't set condition codes, so I kept the version I >>>>>>>> had (more on that below). >>>>>>>> >>>>>>>>> src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>>>>> ??? No comments. >>>>>>>>> >>>>>>>>> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp >>>>>>>>> ??? No comments. >>>>>>>>> >>>>>>>>> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp >>>>>>>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>>>>>>> Register obj_reg, >>>>>>>>> Register swap_reg, >>>>>>>>> Register tmp_reg, >>>>>>>>> ?????????????????????????????????? ? ?? ???? bool >>>>>>>>> swap_reg_contains_mark, >>>>>>>>> Label& done, >>>>>>>>> Label* slow_case, >>>>>>>>> BiasedLockingCounters* counters) { >>>>>>>>> ??????? I think you've changed the only callers of >>>>>>>>> biased_locking_enter() >>>>>>>>> ??????? that cared about the return value with this changeset >>>>>>>>> so it can >>>>>>>>> ??????? be changed to a void function. >>>>>>>> Ok, this is what I mentioned to David in a previous email. Done. >>>>>>>> >>>>>>>>> src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp >>>>>>>>> ??? No comments. >>>>>>>>> >>>>>>>>> src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp >>>>>>>>> ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), >>>>>>>>> done); >>>>>>>>> >>>>>>>>> ??? L131: ? bind(done); >>>>>>>>> ??? L132: >>>>>>>>> ??? L133: ? // At this point flags are set as follows: >>>>>>>>> ??? L134: ? //? EQ -> Success >>>>>>>>> ??? L135: ? //? NE -> Failure, branch to slow path >>>>>>>>> ??????? If tbnz() branches to "done" when we have a box class, >>>>>>>>> what's >>>>>>>>> ??????? the flag value set to (EQ or NE)? And what set that >>>>>>>>> flag value? >>>>>>>>> ??????? The reason I ask is I don't think tbnz() sets >>>>>>>>> condition codes... >>>>>>>> Right. Same as above, I kept the version I had. >>>>>>>> >>>>>>>>> src/hotspot/cpu/arm/interp_masm_arm.cpp >>>>>>>>> ??? No comments. >>>>>>>>> >>>>>>>>> src/hotspot/cpu/arm/macroAssembler_arm.cpp >>>>>>>>> ??? int MacroAssembler::biased_locking_enter(Register obj_reg, >>>>>>>>> Register swap_reg, Register tmp_reg, >>>>>>>>> ????????????????????? ? ?? ????????????????? bool >>>>>>>>> swap_reg_contains_mark, >>>>>>>>> Register tmp2, >>>>>>>>> Label& done, Label& slow_case, >>>>>>>>> BiasedLockingCounters* counters) { >>>>>>>>> ??????? I think you've changed the only callers of >>>>>>>>> biased_locking_enter() >>>>>>>>> ??????? that cared about the return value with this changeset >>>>>>>>> so it can >>>>>>>>> ??????? be changed to a void function. >>>>>>>> Done. >>>>>>>> >>>>>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>>> ??? No comments on the PPC code. >>>>>>>>> >>>>>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>>> src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>>> ??? No comments on the S390 code. >>>>>>>>> >>>>>>>>> src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp >>>>>>>>> ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); >>>>>>>>> ??????? What will this do with a 'noreg' value? (I need a >>>>>>>>> refresher.) >>>>>>>> When not in 64 bit mode that register just won't be used. >>>>>>>> >>>>>>>>> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp >>>>>>>>> ??? No comments. >>>>>>>>> >>>>>>>>> src/hotspot/cpu/x86/interp_masm_x86.cpp >>>>>>>>> ??? No comments. >>>>>>>>> >>>>>>>>> src/hotspot/cpu/x86/macroAssembler_x86.cpp >>>>>>>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>>>>>>> Register obj_reg, >>>>>>>>> Register swap_reg, >>>>>>>>> Register tmp_reg, >>>>>>>>> Register tmp_reg2, >>>>>>>>> ?????????????????????????????????????? ? ? ? bool >>>>>>>>> swap_reg_contains_mark, >>>>>>>>> Label& done, >>>>>>>>> Label* slow_case, >>>>>>>>> BiasedLockingCounters* counters) { >>>>>>>>> ??????? I think you've changed the only caller of >>>>>>>>> biased_locking_enter() >>>>>>>>> ??????? that cared about the return value with this changeset >>>>>>>>> so it can >>>>>>>>> ??????? be changed to a void function. >>>>>>>> Done. >>>>>>>> >>>>>>>>> test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >>>>>>>>> >>>>>>>>> ??? L30:? * @test SyncOnPrimitiveWrapperTest >>>>>>>>> ??????? No parameter to @test directive. >>>>>>>> Removed. >>>>>>>> >>>>>>>>> L136: ??????? private static long sharedCounter = 0L; >>>>>>>>> ??????? Since you don't do anything with sharedCounter other >>>>>>>>> than increment it, >>>>>>>>> ??????? can the compilers optimize it away? If it can be >>>>>>>>> optimized away, does >>>>>>>>> ??????? that mean that: >>>>>>>>> >>>>>>>>> ??????????? L142: ??????????????? synchronized (obj) { >>>>>>>>> >>>>>>>>> ??????? can also be optimized away? >>>>>>>>> >>>>>>>>> ??????? I don't think that: >>>>>>>>> >>>>>>>>> ??????????? L161: ??????????????? synchronized (sharedLock1) { >>>>>>>>> >>>>>>>>> ??????? can be optimized away because it is shared by multiple >>>>>>>>> threads. >>>>>>>>> >>>>>>>>> test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java >>>>>>>>> >>>>>>>>> ??? Similar questions about 'counter' being optimized away. >>>>>>>>> ??? Similar question about "synchronized (obj) {" being >>>>>>>>> optimized away. >>>>>>>> I'm not sure if the compiler will optimize it away. Seems >>>>>>>> unlikely given the counter we are incrementing is not just >>>>>>>> local to some thread. >>>>>>>> >>>>>>>> >>>>>>>> Ok, below is v3 which has the following changes: >>>>>>>> - Use a 32 bit load for the _access_flags field, instead of 64 >>>>>>>> - Martin's implementation for s390 and fix for PPC >>>>>>>> - Faster LogTest version >>>>>>>> - Above changes based on Dan review >>>>>>>> >>>>>>>> I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 >>>>>>>> only) again with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I >>>>>>>> checked it builds in arm32, ppc and s390. >>>>>>>> >>>>>>>> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev >>>>>>>> Inc: >>>>>>>> http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev >>>>>>>> >>>>>>>> >>>>>>>> @Martin: >>>>>>>> Please check if the test doesn't timeout for you now with the >>>>>>>> changes I made to LogTest. >>>>>>>> Also, I tried to use tbnz in aarch64 and arm32 instead of tst + >>>>>>>> br (except for c2 since we actually need to set the condition >>>>>>>> flags), but for c1 I was getting an assertion in the compiler >>>>>>>> thread: >>>>>>>> >>>>>>>> guarantee(chk == -1 || chk == 0) failed: Field too big for insn >>>>>>>> >>>>>>>> This is the stack when that happens: >>>>>>>> >>>>>>>> MacroAssembler::pd_patch_instruction_size(unsigned char*, >>>>>>>> unsigned char*)+0x398 >>>>>>>> AbstractAssembler::bind(Label&)+0x118 >>>>>>>> MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 >>>>>>>> LIR_Assembler::emit_slow_case_stubs()+0x54 >>>>>>>> Compilation::emit_code_body()+0x17c >>>>>>>> >>>>>>>> The change was simply: >>>>>>>> >>>>>>>> diff --git >>>>>>>> a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>>>> b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>>>> --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>>>> +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>>>>> @@ -78,7 +78,6 @@ >>>>>>>> ?? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>> ???? load_klass(hdr, obj); >>>>>>>> -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); >>>>>>>> -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); >>>>>>>> -??? br(Assembler::NE, slow_case); >>>>>>>> +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); >>>>>>>> +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); >>>>>>>> ?? } >>>>>>>> >>>>>>>> So the issue must be related to where we want to jump. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Patricio >>>>>>>>> Dan >>>>>>>>> >>>>>>>>> >>>>>>>>> On 8/5/20 9:01 AM, Patricio Chilano wrote: >>>>>>>>>> Hi Martin, >>>>>>>>>> >>>>>>>>>> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>>>>>>>>>> Hi Patricio, >>>>>>>>>>> >>>>>>>>>>> using 8 Byte load instructions for jint fields is a terrible >>>>>>>>>>> coding style! >>>>>>>>>>> Someone else may see it and use an 8 Byte store. Will result >>>>>>>>>>> in great fun for debugging! >>>>>>>>>>> >>>>>>>>>>> And on Big Endian you will simply access the wrong bits. >>>>>>>>>> Ah, of course! Those 32 bits will be in the wrong place when >>>>>>>>>> doing the test. >>>>>>>>>> >>>>>>>>>>> Note that Big Endian Platforms are AIX, old linux ppc, s390, >>>>>>>>>>> SPARC. I don't think you have tested on them. >>>>>>>>>>> >>>>>>>>>>>> We could remove the nested synchronized statements in the >>>>>>>>>>>> run() method >>>>>>>>>>>> so that we don't generate that much logging. We could also >>>>>>>>>>>> lower >>>>>>>>>>>> LOOP_COUNT. I think the issue is also because we are >>>>>>>>>>>> running LogTest >>>>>>>>>>>> with multiple flag combinations. Not sure what we should >>>>>>>>>>>> touch first. >>>>>>>>>>>> Maybe the synchronized statements, have only one or two and >>>>>>>>>>>> test that >>>>>>>>>>>> first? >>>>>>>>>>> Sounds like helpful ideas. Please go ahead and strip things >>>>>>>>>>> down. >>>>>>>>>> Great, I will send v3 later with those changes. >>>>>>>>>> >>>>>>>>>> Thanks Martin! >>>>>>>>>> >>>>>>>>>> Patricio >>>>>>>>>>> Thanks for taking care of it. >>>>>>>>>>> >>>>>>>>>>> Best regards, >>>>>>>>>>> Martin >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>> From: Patricio Chilano >>>>>>>>>>>> Sent: Dienstag, 4. August 2020 20:26 >>>>>>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>>>>>> ; >>>>>>>>>>>> hotspot-runtime-dev at openjdk.java.net; >>>>>>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on >>>>>>>>>>>> primitive wrappers >>>>>>>>>>>> >>>>>>>>>>>> Hi Martin, >>>>>>>>>>>> >>>>>>>>>>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>>>>>>>>>> Hi Patricio, >>>>>>>>>>>>> >>>>>>>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>>>>>>> sending v3. >>>>>>>>>>>>> Thanks. I forgot to mention arm + aarch64. >>>>>>>>>>>>> Aarch64 uses ldrw + tbnz. >>>>>>>>>>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>>>>>>>>>> >>>>>>>>>>>>> I remember having seen the same mistake ?? >>>>>>>>>>>>> And nobody noticed it on little Endian platforms. >>>>>>>>>>>> Ok, I can use a tbnz instead of test and then a branch on >>>>>>>>>>>> arm32 and aarch64. >>>>>>>>>>>> Shouldn't a normal ldr in arm32 work fine? >>>>>>>>>>>> Also in 64 bits (either x64 or aarch64) I don't see the >>>>>>>>>>>> issue of using a >>>>>>>>>>>> 64 bit load, besides the fact that we only care about the >>>>>>>>>>>> first 32 bits. >>>>>>>>>>>> Regardless of the endianness, aren't we masking out the >>>>>>>>>>>> upper part when >>>>>>>>>>>> we do AND/TEST or if we test a bit in the 0-31 bit range? >>>>>>>>>>>> Otherwise it >>>>>>>>>>>> seems it should have failed for one of those platforms in >>>>>>>>>>>> my tests. >>>>>>>>>>>> >>>>>>>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe >>>>>>>>>>>>>> you can >>>>>>>>>>>>>> specify that and see if that fixes it? Let me know if >>>>>>>>>>>>>> that works and I >>>>>>>>>>>>>> can add it to the test. >>>>>>>>>>>>> That seems to have an effect. But now I'm not patient >>>>>>>>>>>>> enough to wait for >>>>>>>>>>>> the test to finish. >>>>>>>>>>>>> Maybe the problem is that I'm using slow debug builds. >>>>>>>>>>>>> But is there a chance to make the test quicker without >>>>>>>>>>>>> losing coverage >>>>>>>>>>>> significantly? >>>>>>>>>>>>> I think the test is too slow this way to run it on a >>>>>>>>>>>>> regular basis. We'd need to >>>>>>>>>>>> spend dedicated machines for it. >>>>>>>>>>>> We could remove the nested synchronized statements in the >>>>>>>>>>>> run() method >>>>>>>>>>>> so that we don't generate that much logging. We could also >>>>>>>>>>>> lower >>>>>>>>>>>> LOOP_COUNT. I think the issue is also because we are >>>>>>>>>>>> running LogTest >>>>>>>>>>>> with multiple flag combinations. Not sure what we should >>>>>>>>>>>> touch first. >>>>>>>>>>>> Maybe the synchronized statements, have only one or two and >>>>>>>>>>>> test that >>>>>>>>>>>> first? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Patricio >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> Martin >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>>>> From: Patricio Chilano >>>>>>>>>>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>>>>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>>>>>>>> ; >>>>>>>>>>>>>> hotspot-runtime-dev at openjdk.java.net; >>>>>>>>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on >>>>>>>>>>>>>> primitive wrappers >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Martin, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for fixing PPC and taking care of s390! >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>>>>>>>>>> Hi Patricio, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I suggest to use movl + testl for checking the access >>>>>>>>>>>>>>> flag as for other >>>>>>>>>>>> access >>>>>>>>>>>>>> flags on x64. >>>>>>>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>>>>>>> sending v3. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> New version for PPC64 and s390 see below. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err >>>>>>>>>>>>>>> files as >>>>>>>>>>>> expected. >>>>>>>>>>>>>> However, I'm getting timeout issues: >>>>>>>>>>>>>>> timed out (timeout set to 120000ms, elapsed time >>>>>>>>>>>>>>> including timeout >>>>>>>>>>>>>> handling was 122372ms) >>>>>>>>>>>>>>> Can we provide more time? >>>>>>>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe >>>>>>>>>>>>>> you can >>>>>>>>>>>>>> specify that and see if that fixes it? Let me know if >>>>>>>>>>>>>> that works and I >>>>>>>>>>>>>> can add it to the test. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Patricio >>>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>>> Martin >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>>>>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue >>>>>>>>>>>>>>> Aug 04 >>>>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue >>>>>>>>>>>>>>> Aug 04 >>>>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>>>> @@ -107,8 +107,8 @@ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>>>>>>>>>> -??? ld(Rscratch, >>>>>>>>>>>>>>> in_bytes(Klass::access_flags_offset()), Rscratch); >>>>>>>>>>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>>>>>> +??? lwz(Rscratch, >>>>>>>>>>>>>>> in_bytes(Klass::access_flags_offset()), Rscratch); >>>>>>>>>>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>>>> ??????? bne(CCR0, slow_case); >>>>>>>>>>>>>>> ????? } >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>>>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>>>>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>>>> @@ -912,8 +912,8 @@ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>>>> ????????? load_klass(tmp, object); >>>>>>>>>>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>>>>>> tmp); >>>>>>>>>>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>>>>>> tmp); >>>>>>>>>>>>>>> +????? testbitdi(CCR0, R0, tmp, >>>>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>>>> ????????? bne(CCR0, slow_case); >>>>>>>>>>>>>>> ??????? } >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>>>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>>>>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>>>> ??????? load_klass(temp, oop); >>>>>>>>>>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>>>>>> temp); >>>>>>>>>>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>>>>>> temp); >>>>>>>>>>>>>>> +??? testbitdi(CCR0, R0, temp, >>>>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>>>> ??????? bne(CCR0, cont); >>>>>>>>>>>>>>> ????? } >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>>>>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>>>>>>>>> Tue Aug 04 >>>>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>>>>>>>>> Tue Aug 04 >>>>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>>>> @@ -91,6 +91,12 @@ >>>>>>>>>>>>>>> ????? // Save object being locked into the >>>>>>>>>>>>>>> BasicObjectLock... >>>>>>>>>>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>>>>>>>>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>>>> + z_btrue(slow_case); >>>>>>>>>>>>>>> +? } >>>>>>>>>>>>>>> + >>>>>>>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>>>>>>>>>> Z_R0_scratch, done, >>>>>>>>>>>>>> &slow_case); >>>>>>>>>>>>>>> ????? } >>>>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>>>> src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>>>>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug >>>>>>>>>>>>>>> 04 10:03:57 >>>>>>>>>>>>>> 2020 +0200 >>>>>>>>>>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>>>>>> 16:04:31 >>>>>>>>>>>>>> 2020 +0200 >>>>>>>>>>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>>>>>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>>>>>>>>>> ????? z_lg(displaced_header, >>>>>>>>>>>>>>> oopDesc::mark_offset_in_bytes(), object); >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>>>> + z_btrue(slow_case); >>>>>>>>>>>>>>> +? } >>>>>>>>>>>>>>> + >>>>>>>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>>>>>>> ??????? biased_locking_enter(object, displaced_header, >>>>>>>>>>>>>>> Z_R1, Z_R0, done, >>>>>>>>>>>>>> &slow_case); >>>>>>>>>>>>>>> ????? } >>>>>>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>>>>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>>>>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue >>>>>>>>>>>>>>> Aug 04 >>>>>>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue >>>>>>>>>>>>>>> Aug 04 >>>>>>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>>>>>>>>>> ????? // Load markWord from oop into mark. >>>>>>>>>>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>>>>>> + z_btrue(done); >>>>>>>>>>>>>>> +? } >>>>>>>>>>>>>>> + >>>>>>>>>>>>>>> ????? if (try_bias) { >>>>>>>>>>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, >>>>>>>>>>>>>>> Z_R0, done); >>>>>>>>>>>>>>> ????? } >>>>>>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > From dcherepanov at azul.com Mon Aug 10 20:12:15 2020 From: dcherepanov at azul.com (Dmitry Cherepanov) Date: Mon, 10 Aug 2020 23:12:15 +0300 Subject: RFR: 8250636: iso8601_time returns incorrect offset part on MacOS In-Reply-To: References: <777fc30a-45ef-aa45-8f04-0bbe80c5a83d@azul.com> <0e1ed1ae-2011-2e4a-4f58-d99b57ede74c@azul.com> <934a8a40-dd51-512f-f010-909ed312783d@oracle.com> Message-ID: <2b0273f3-3541-122b-4af1-56e7cd335e86@azul.com> Hi Martin, On 10.08.2020 22:20, Doerr, Martin wrote: > Hi, > > unfortunately, tm_gmtoff seems to be a BSD and GNU extension. > It's not POSIX and doesn't build on AIX: > os.cpp:163:34: error: no member named 'tm_gmtoff' in 'tm' > We'll need a follow up fix. Sorry for the broken build, I filed a new bug (https://bugs.openjdk.java.net/browse/JDK-8251365) and will send a review request soon. Dmitry From patricio.chilano.mateo at oracle.com Mon Aug 10 20:19:56 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Mon, 10 Aug 2020 17:19:56 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <6960c0f3-8cbe-fb80-3529-a1c7fb794f0d@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> <6960c0f3-8cbe-fb80-3529-a1c7fb794f0d@oracle.com> Message-ID: Hi David, On 8/10/20 1:12 AM, David Holmes wrote: > Hi Patricio, > > First, I confirmed with Valhalla folk that this change is fine to go > ahead as a stand-alone change independent of the proposed JEP in this > area. Great, thanks for that!? : ) > I've checked the incremental changes for v2, v3 and v4 and everything > seems good to me. > > I only have a couple of nits with the tests: > > ? 54???? private static void initTestObjects() { > ? 55???????? testObjects.add(Character.valueOf('H')); > ? 56???????? testObjects.add(Boolean.valueOf(true)); > ? 57???????? Byte byteT = 0x40; > ? 58???????? testObjects.add(byteT); > ? 59???????? Short shortT = 0x4000; > ? 60???????? testObjects.add(shortT); > ? 61???????? testObjects.add(Integer.valueOf(0x40000000)); > ? 62???????? testObjects.add(Long.valueOf(0x4000000000000000L)); > ? 63???????? testObjects.add(Float.valueOf(1.20f)); > ? 64???????? testObjects.add(Double.valueOf(1.2345)); > ? 65???? } > > Why are Byte and Short treated differently? If I do the same as with the other types, the argument passed to Byte.valueOf() and Short.valueOf() gets treated as an int and I get a compilation error: : error: no suitable method found for valueOf(int) ? testObjects.add(Byte.valueOf(0x40)); ???????????????????????????????? ^ ? method Byte.valueOf(byte) is not applicable ? (argument mismatch; possible lossy conversion from int to byte) ? method Byte.valueOf(String) is not applicable ? (argument mismatch; int cannot be converted to String) > ?74 fatalTests[index] = Stream.of(commonFatalTestsFlags, > specificFlags[i], new String[] > {"SyncOnPrimitiveWrapperTest$FatalTest", Integer.toString(j)}) > ? 75 .flatMap(Stream::of).toArray(String[]::new); > > The preferred Java style [1] for stream operations is: > > fatalTests[index] = Stream.of(commonFatalTestsFlags, specificFlags[i], > new String[] {"SyncOnPrimitiveWrapperTest$FatalTest", > Integer.toString(j)}) > ????????????????????????? .flatMap(Stream::of) > ????????????????????????? .toArray(String[]::new); Fixed. Here is v5: Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/webrev/ Thanks, Patricio > Thanks, > David > ----- > > [1] As demonstrated here (but widely used in JDK code): > http://cr.openjdk.java.net/~alundblad/styleguide/index-v6.html#toc-wrapping-lines > > On 7/08/2020 4:48 am, Patricio Chilano wrote: >> Hi Dan, >> >> On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: >>> I'm peeking ahead to the next webrev... :-) >>> >>> > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ >> : ) >> >>> General comments: >>> ? - check files for copyright year updates. >> Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, >> c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. >> >>> src/hotspot/share/runtime/synchronizer.hpp >>> ??? No comments. >>> >>> src/hotspot/share/runtime/synchronizer.cpp >>> ??? L507: ? const markWord mark = obj->mark(); >>> ??? L508: >>> ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && >>> obj->klass()->is_box()) { >>> ??? L510: ??? return false; >>> ??? L511: ? } >>> ??????? The new bailout on L509-511 can move above L507. >> Moved. >> >>> L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass >>> %s", p2i(obj()), obj->klass()->external_name()); >>> ??????? This external_name() call does not have a ResourceMark. >> Good catch! I had one in a previous version but then I changed the >> conditionals and lost it for the fatal error case. The test worked >> okay because for the main JavaThread there is a ResourceMark in >> jni_invoke_static() (jni.cpp). >> >>> src/hotspot/share/logging/logTag.hpp >>> ??? No comments. >>> >>> src/hotspot/share/oops/klass.hpp >>> ??? No comments. >>> >>> src/hotspot/share/utilities/accessFlags.hpp >>> ??? No comments. >>> >>> src/hotspot/share/runtime/globals.hpp >>> ??? L814: ???????????? "0: off " >>> ??????? Missing a ';' after "off". >> Fixed. >> >>> L816: ???????????? "2: log message to stdout. >>> ??????? Perhaps add "(by default)" after "stdout" or >>> ??????? don't say where log output is at all. >>> >>> src/hotspot/share/runtime/arguments.cpp >>> ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, true, >>> LOG_TAGS(primitivewrappers)); >>> ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp >>> ??????? since it looks like logging output is configured to 'stdout'. >>> ??????? I'm assuming that other log options to put output elsewhere >>> ??????? are overridden by this code? >> Right. So the logging is done under UL with the tag >> primitivewrappers. If that tag is specified in Xlog then this >> conditional will be skipped because !log_is_enabled(Info, >> primitivewrappers) will be false. >> >>> src/hotspot/share/classfile/systemDictionary.cpp >>> ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { >>> ??????? nit - s/T_LONG+1/T_LONG + 1/ >> Fixed. >> >>> L2191: _box_klasses[i]->set_prototype_header(markWord::prototype()); >>> ??????? I assume we're keeping the prototype_header field when >>> Biased Locking >>> ??????? goes away? The reason I ask: >>> >>> ? ? ? ? static markWord prototype() { >>> ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); >>> ? ? ? ? } >>> >>> ??????? is because without Biased Locking, do we really need the >>> prototype >>> ??????? anymore? The initial markWord won't need possible variants... >> Yes, I think it can go away unless somebody finds another use for it. >> Maybe Valhalla. >> >>> src/hotspot/share/jfr/metadata/metadata.xml >>> ??? L69: ? >> ??????? Is the category "Java Application" because it's the application >>> ??????? code that did something "wrong" here? Where "application" is >>> loosely >>> ??????? defined to include the possibility of auto generated code, >>> library >>> ??????? code and the like? Or perhaps "application" because >>> something "above" >>> ??????? the "Java Virtual Machine, Runtime" did the "wrong" thing here? >> I don't know what the right category should be really. I saw the >> events JavaMonitorEnter, JavaMonitorWait and JavaMonitorInflate and >> thought this event should fall in the same category they do. >> >>> src/jdk.jfr/share/conf/jfr/default.jfc >>> ??? No comments. >>> >>> src/jdk.jfr/share/conf/jfr/profile.jfc >>> ??? No comments. >>> >>> test/lib/jdk/test/lib/jfr/EventNames.java >>> ??? No comments. >>> >>> src/hotspot/cpu/aarch64/aarch64.ad >>> ??? L3517: ????? __ tbnz(tmp, exact_log2(JVM_ACC_IS_BOX_CLASS), cont); >>> >>> ??? L3578: ??? __ bind(cont); >>> ??? L3579: ??? // flag == EQ indicates success >>> ??? L3580: ??? // flag == NE indicates failure >>> ??????? If tbnz() branches to "cont" when we have a box class, what's >>> ??????? the flag value set to (EQ or NE)? And what set that flag value? >>> ??????? The reason I ask is I don't think tbnz() sets condition >>> codes... >> Right, it doesn't set condition codes, so I kept the version I had >> (more on that below). >> >>> src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp >>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>> ??????????? ? ?? ??????????????????????????? Register obj_reg, >>> ?????????????????????????????? ? ?? ???????? Register swap_reg, >>> ?????????????????????????????????? ? ?? ???? Register tmp_reg, >>> ?????????????????????????????????? ? ?? ???? bool >>> swap_reg_contains_mark, >>> ?????????????????????????????????? ? ?? ???? Label& done, >>> ??????????????????????????????????? ? ?? ??? Label* slow_case, >>> BiasedLockingCounters* counters) { >>> ??????? I think you've changed the only callers of >>> biased_locking_enter() >>> ??????? that cared about the return value with this changeset so it can >>> ??????? be changed to a void function. >> Ok, this is what I mentioned to David in a previous email. Done. >> >>> src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp >>> ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), done); >>> >>> ??? L131: ? bind(done); >>> ??? L132: >>> ??? L133: ? // At this point flags are set as follows: >>> ??? L134: ? //? EQ -> Success >>> ??? L135: ? //? NE -> Failure, branch to slow path >>> ??????? If tbnz() branches to "done" when we have a box class, what's >>> ??????? the flag value set to (EQ or NE)? And what set that flag value? >>> ??????? The reason I ask is I don't think tbnz() sets condition >>> codes... >> Right. Same as above, I kept the version I had. >> >>> src/hotspot/cpu/arm/interp_masm_arm.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/arm/macroAssembler_arm.cpp >>> ??? int MacroAssembler::biased_locking_enter(Register obj_reg, >>> Register swap_reg, Register tmp_reg, >>> ????????????????????? ? ?? ????????????????? bool >>> swap_reg_contains_mark, >>> ???????????????????????? ? ?? ?????????????? Register tmp2, >>> ????????????????????????? ? ?? ????????????? Label& done, Label& >>> slow_case, >>> BiasedLockingCounters* counters) { >>> ??????? I think you've changed the only callers of >>> biased_locking_enter() >>> ??????? that cared about the return value with this changeset so it can >>> ??????? be changed to a void function. >> Done. >> >>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>> ??? No comments on the PPC code. >>> >>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>> src/hotspot/cpu/s390/interp_masm_s390.cpp >>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>> ??? No comments on the S390 code. >>> >>> src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp >>> ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); >>> ??????? What will this do with a 'noreg' value? (I need a refresher.) >> When not in 64 bit mode that register just won't be used. >> >>> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/x86/interp_masm_x86.cpp >>> ??? No comments. >>> >>> src/hotspot/cpu/x86/macroAssembler_x86.cpp >>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>> ????????????????????????????????? ? ?? ????? Register obj_reg, >>> ????????????????????????????????? ? ?? ????? Register swap_reg, >>> ????????????????????????????????? ? ?? ????? Register tmp_reg, >>> ?????????????????????????????????? ? ?? ???? Register tmp_reg2, >>> ?????????????????????????????????????? ? ? ? bool >>> swap_reg_contains_mark, >>> ?????????????????????????????????? ? ?? ???? Label& done, >>> ?????????????????????????????????? ? ?? ???? Label* slow_case, >>> BiasedLockingCounters* counters) { >>> ??????? I think you've changed the only caller of >>> biased_locking_enter() >>> ??????? that cared about the return value with this changeset so it can >>> ??????? be changed to a void function. >> Done. >> >>> test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >>> ??? L30:? * @test SyncOnPrimitiveWrapperTest >>> ??????? No parameter to @test directive. >> Removed. >> >>> L136: ??????? private static long sharedCounter = 0L; >>> ??????? Since you don't do anything with sharedCounter other than >>> increment it, >>> ??????? can the compilers optimize it away? If it can be optimized >>> away, does >>> ??????? that mean that: >>> >>> ??????????? L142: ??????????????? synchronized (obj) { >>> >>> ??????? can also be optimized away? >>> >>> ??????? I don't think that: >>> >>> ??????????? L161: ??????????????? synchronized (sharedLock1) { >>> >>> ??????? can be optimized away because it is shared by multiple threads. >>> >>> test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java >>> ??? Similar questions about 'counter' being optimized away. >>> ??? Similar question about "synchronized (obj) {" being optimized away. >> I'm not sure if the compiler will optimize it away. Seems unlikely >> given the counter we are incrementing is not just local to some thread. >> >> >> Ok, below is v3 which has the following changes: >> - Use a 32 bit load for the _access_flags field, instead of 64 >> - Martin's implementation for s390 and fix for PPC >> - Faster LogTest version >> - Above changes based on Dan review >> >> I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 only) >> again with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I checked it builds >> in arm32, ppc and s390. >> >> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev >> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev >> >> >> @Martin: >> Please check if the test doesn't timeout for you now with the changes >> I made to LogTest. >> Also, I tried to use tbnz in aarch64 and arm32 instead of tst + br >> (except for c2 since we actually need to set the condition flags), >> but for c1 I was getting an assertion in the compiler thread: >> >> guarantee(chk == -1 || chk == 0) failed: Field too big for insn >> >> This is the stack when that happens: >> >> MacroAssembler::pd_patch_instruction_size(unsigned char*, unsigned >> char*)+0x398 >> AbstractAssembler::bind(Label&)+0x118 >> MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 >> LIR_Assembler::emit_slow_case_stubs()+0x54 >> Compilation::emit_code_body()+0x17c >> >> The change was simply: >> >> diff --git a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >> @@ -78,7 +78,6 @@ >> ??? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >> ????? load_klass(hdr, obj); >> -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); >> -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); >> -??? br(Assembler::NE, slow_case); >> +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); >> +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); >> ??? } >> >> So the issue must be related to where we want to jump. >> >> Thanks, >> Patricio >>> Dan >>> >>> >>> On 8/5/20 9:01 AM, Patricio Chilano wrote: >>>> Hi Martin, >>>> >>>> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>>>> Hi Patricio, >>>>> >>>>> using 8 Byte load instructions for jint fields is a terrible >>>>> coding style! >>>>> Someone else may see it and use an 8 Byte store. Will result in >>>>> great fun for debugging! >>>>> >>>>> And on Big Endian you will simply access the wrong bits. >>>> Ah, of course! Those 32 bits will be in the wrong place when doing >>>> the test. >>>> >>>>> Note that Big Endian Platforms are AIX, old linux ppc, s390, >>>>> SPARC. I don't think you have tested on them. >>>>> >>>>>> We could remove the nested synchronized statements in the run() >>>>>> method >>>>>> so that we don't generate that much logging. We could also lower >>>>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>>>> with multiple flag combinations. Not sure what we should touch >>>>>> first. >>>>>> Maybe the synchronized statements, have only one or two and test >>>>>> that >>>>>> first? >>>>> Sounds like helpful ideas. Please go ahead and strip things down. >>>> Great, I will send v3 later with those changes. >>>> >>>> Thanks Martin! >>>> >>>> Patricio >>>>> Thanks for taking care of it. >>>>> >>>>> Best regards, >>>>> Martin >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: Patricio Chilano >>>>>> Sent: Dienstag, 4. August 2020 20:26 >>>>>> To: Doerr, Martin ; David Holmes >>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>> wrappers >>>>>> >>>>>> Hi Martin, >>>>>> >>>>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>>>> Hi Patricio, >>>>>>> >>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>> sending v3. >>>>>>> Thanks. I forgot to mention arm + aarch64. >>>>>>> Aarch64 uses ldrw + tbnz. >>>>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>>>> >>>>>>> I remember having seen the same mistake ?? >>>>>>> And nobody noticed it on little Endian platforms. >>>>>> Ok, I can use a tbnz instead of test and then a branch on arm32 >>>>>> and aarch64. >>>>>> Shouldn't a normal ldr in arm32 work fine? >>>>>> Also in 64 bits (either x64 or aarch64) I don't see the issue of >>>>>> using a >>>>>> 64 bit load, besides the fact that we only care about the first >>>>>> 32 bits. >>>>>> Regardless of the endianness, aren't we masking out the upper >>>>>> part when >>>>>> we do AND/TEST or if we test a bit in the 0-31 bit range? >>>>>> Otherwise it >>>>>> seems it should have failed for one of those platforms in my tests. >>>>>> >>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>> works and I >>>>>>>> can add it to the test. >>>>>>> That seems to have an effect. But now I'm not patient enough to >>>>>>> wait for >>>>>> the test to finish. >>>>>>> Maybe the problem is that I'm using slow debug builds. >>>>>>> But is there a chance to make the test quicker without losing >>>>>>> coverage >>>>>> significantly? >>>>>>> I think the test is too slow this way to run it on a regular >>>>>>> basis. We'd need to >>>>>> spend dedicated machines for it. >>>>>> We could remove the nested synchronized statements in the run() >>>>>> method >>>>>> so that we don't generate that much logging. We could also lower >>>>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>>>> with multiple flag combinations. Not sure what we should touch >>>>>> first. >>>>>> Maybe the synchronized statements, have only one or two and test >>>>>> that >>>>>> first? >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Patricio >>>>>>> Best regards, >>>>>>> Martin >>>>>>> >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Patricio Chilano >>>>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>>>> wrappers >>>>>>>> >>>>>>>> Hi Martin, >>>>>>>> >>>>>>>> Thanks for fixing PPC and taking care of s390! >>>>>>>> >>>>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>>>> Hi Patricio, >>>>>>>>> >>>>>>>>> I suggest to use movl + testl for checking the access flag as >>>>>>>>> for other >>>>>> access >>>>>>>> flags on x64. >>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>> sending v3. >>>>>>>> >>>>>>>>> New version for PPC64 and s390 see below. >>>>>>>>> >>>>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>>>>> expected. >>>>>>>> However, I'm getting timeout issues: >>>>>>>>> timed out (timeout set to 120000ms, elapsed time including >>>>>>>>> timeout >>>>>>>> handling was 122372ms) >>>>>>>>> Can we provide more time? >>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>> works and I >>>>>>>> can add it to the test. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Patricio >>>>>>>>> Best regards, >>>>>>>>> Martin >>>>>>>>> >>>>>>>>> >>>>>>>>> diff -r 77852e129401 >>>>>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -107,8 +107,8 @@ >>>>>>>>> >>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>> Rscratch); >>>>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>>>> +??? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>> Rscratch); >>>>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> ??????? bne(CCR0, slow_case); >>>>>>>>> ????? } >>>>>>>>> >>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -912,8 +912,8 @@ >>>>>>>>> >>>>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> ????????? load_klass(tmp, object); >>>>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>> +????? testbitdi(CCR0, R0, tmp, >>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> ????????? bne(CCR0, slow_case); >>>>>>>>> ??????? } >>>>>>>>> >>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>>>> >>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> ??????? load_klass(temp, oop); >>>>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>> +??? testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> ??????? bne(CCR0, cont); >>>>>>>>> ????? } >>>>>>>>> >>>>>>>>> diff -r 77852e129401 >>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -91,6 +91,12 @@ >>>>>>>>> ????? // Save object being locked into the BasicObjectLock... >>>>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> +??? z_btrue(slow_case); >>>>>>>>> +? } >>>>>>>>> + >>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>>>> Z_R0_scratch, done, >>>>>>>> &slow_case); >>>>>>>>> ????? } >>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>>> 10:03:57 >>>>>>>> 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>> 16:04:31 >>>>>>>> 2020 +0200 >>>>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>>>> ????? z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), >>>>>>>>> object); >>>>>>>>> >>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> +??? z_btrue(slow_case); >>>>>>>>> +? } >>>>>>>>> + >>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>> ??????? biased_locking_enter(object, displaced_header, Z_R1, >>>>>>>>> Z_R0, done, >>>>>>>> &slow_case); >>>>>>>>> ????? } >>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>> 10:03:57 2020 +0200 >>>>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>> 16:04:31 2020 +0200 >>>>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>>>> ????? // Load markWord from oop into mark. >>>>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>>>> >>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>> +??? z_btrue(done); >>>>>>>>> +? } >>>>>>>>> + >>>>>>>>> ????? if (try_bias) { >>>>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, Z_R0, >>>>>>>>> done); >>>>>>>>> ????? } >>>>>>>>> >>>> >>> >> From dcherepanov at azul.com Mon Aug 10 20:27:14 2020 From: dcherepanov at azul.com (Dmitry Cherepanov) Date: Mon, 10 Aug 2020 23:27:14 +0300 Subject: RFR: 8251365: Build failure on AIX after 8250636 Message-ID: <48c0d4f2-ffa7-a242-bb5a-f53781eaa5ff@azul.com> Please review the fix for build failure on AIX. With this change, "tm_gmtoff" is used only if _ALLBSD_SOURCE or _GNU_SOURCE is set. In the case of AIX, we'll continue to use "timezone". I'll start jdk-submit testing for this change shortly, unfortunately I do not have an AIX machine at hand and would appreciate if someone (with access to AIX) could verify that the change fixes AIX build . Thanks in advance. JBS: https://bugs.openjdk.java.net/browse/JDK-8251365 Webrev: http://cr.openjdk.java.net/~dcherepanov/8251365/webrev.v1/ Thanks, Dmitry From daniel.daugherty at oracle.com Mon Aug 10 20:32:51 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 10 Aug 2020 16:32:51 -0400 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> Message-ID: I forgot to chime in on the v4 review. The link to "v4/inc" is actually a link to the full webrev so here's the correct incremental link: > http://cr.openjdk.java.net/~pchilanomate/8242263/v4/inc/webrev/ src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp src/hotspot/cpu/arm/macroAssembler_arm.cpp src/hotspot/cpu/arm/macroAssembler_arm.hpp ??? Thanks for fixing the indents. src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp src/hotspot/cpu/ppc/macroAssembler_ppc.cpp src/hotspot/cpu/s390/macroAssembler_s390.cpp ??? No comments on the PPC or S390 changes. src/hotspot/cpu/x86/macroAssembler_x86.cpp src/hotspot/cpu/x86/macroAssembler_x86.hpp ??? Thanks for fixing the indents. test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java ??? No comments. Thumbs up. Dan On 8/7/20 11:29 AM, Patricio Chilano wrote: > Hi Martin, > > On 8/7/20 12:58 AM, Doerr, Martin wrote: >> Hi Patricio, >> >> thanks for the update. >> >> Sorry for causing trouble related to the tbnz instructions on aarch64. >> My concern was basically to use the correct number of Bytes for the load instructions. >> You have fixed that in webrev v3. Thanks. >> >> Unfortunately, I did the same mistake regarding the flag requirements on PPC64 and s390. And we have a wrong label on PPC64 (otherwise branch destination can be too far). >> Test duration is still terrible with the slow debug build. I still got timeouts. (Maybe I should better use fast debug builds.) > I measured the time in my Mac (fastdebug build). The FatalTest takes ~ > less than a second to run, and we have 7*8=56 of those. The LogTest > takes between 1-4 seconds depending on the flag, and we have 7 of > those. Overall it takes 2-3 minutes to run. Maybe you can measure > those times too to see where its taking so long? > >> Fixes see below. > Great, added the new fixes. Here is v4: > > Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v4/webrev/ > Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v4/inc/webrev/ > > >> Sometimes simple changes can require quite some time ... > Yep, no problem? : ) > > Thanks! > > > Patricio >> Best regards, >> Martin >> >> >> diff -r f629202b3862 src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Thu Aug 06 22:03:39 2020 +0200 >> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Thu Aug 06 23:26:02 2020 +0200 >> @@ -109,7 +109,7 @@ >> load_klass(Rscratch, Roop); >> lwz(Rscratch, in_bytes(Klass::access_flags_offset()), Rscratch); >> testbitdi(CCR0, R0, Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS)); >> - bne(CCR0, slow_case); >> + bne(CCR0, slow_int); >> } >> >> if (UseBiasedLocking) { >> diff -r f629202b3862 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Thu Aug 06 22:03:39 2020 +0200 >> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Thu Aug 06 23:26:02 2020 +0200 >> @@ -2839,8 +2839,8 @@ >> if (DiagnoseSyncOnPrimitiveWrappers != 0) { >> load_klass(temp, oop); >> lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >> - testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >> - bne(CCR0, cont); >> + testbitdi(flag, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >> + bne(flag, cont); >> } >> >> if (try_bias) { >> diff -r f629202b3862 src/hotspot/cpu/s390/macroAssembler_s390.cpp >> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Thu Aug 06 22:03:39 2020 +0200 >> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Thu Aug 06 23:26:02 2020 +0200 >> @@ -3360,8 +3360,10 @@ >> >> if (DiagnoseSyncOnPrimitiveWrappers != 0) { >> load_klass(Z_R1_scratch, oop); >> - testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), exact_log2(JVM_ACC_IS_BOX_CLASS)); >> - z_btrue(done); >> + z_l(Z_R1_scratch, Address(Z_R1_scratch, Klass::access_flags_offset())); >> + assert((JVM_ACC_IS_BOX_CLASS & 0xFFFF) == 0, "or change following instruction"); >> + z_nilh(Z_R1_scratch, JVM_ACC_IS_BOX_CLASS >> 16); >> + z_brne(done); >> } >> >> if (try_bias) { >> diff -r f629202b3862 test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >> --- a/test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java Thu Aug 06 22:03:39 2020 +0200 >> +++ b/test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java Thu Aug 06 23:26:02 2020 +0200 >> @@ -31,7 +31,7 @@ >> * @bug 8242263 >> * @summary Exercise DiagnoseSyncOnPrimitiveWrappers diagnostic flag >> * @library /test/lib >> - * @run driver SyncOnPrimitiveWrapperTest >> + * @run driver/timeout=180000 SyncOnPrimitiveWrapperTest >> */ >> >> public class SyncOnPrimitiveWrapperTest { >> > From daniel.daugherty at oracle.com Mon Aug 10 20:39:41 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 10 Aug 2020 16:39:41 -0400 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <29c3d48b-1875-8ea4-792a-157cdd07fe2c@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <29c3d48b-1875-8ea4-792a-157cdd07fe2c@oracle.com> Message-ID: <4193d602-c04d-9664-07e1-f96f1e605adb@oracle.com> > Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ src/hotspot/share/jfr/metadata/metadata.xml ??? No comments. src/hotspot/share/runtime/synchronizer.cpp ??? L577: ??? char* newline = (char*)strchr(ss.base(), '\n'); ??? L578: ??? if (newline != NULL) { ??? L579: ????? *newline = '\0'; ??? L580: ??? } ??? L581: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass %s %s", p2i(obj()), obj->klass()->external_name(), base); ??????? Does this work as expected on Win* where is used? I don't ??????? remember where we convert => on Win*... src/jdk.jfr/share/conf/jfr/default.jfc ??? No comments. src/jdk.jfr/share/conf/jfr/profile.jfc ??? No comments. test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java ??? No comments. Thumbs up. No need to see a new webrev if you end up making changes for Win* above. Dan On 8/10/20 4:03 PM, Patricio Chilano wrote: > Hi Erik, > > Thanks for looking into this and for the offline discussions. > > On 8/10/20 10:19 AM, Erik Gahlin wrote: >> >> Hi Patricio, >> >> I have tried to review the JFR changes, but I need more information >> on how the feature is going to be used. >> >> Is this something temporary that will be used gain insight for the >> Valhalla project, or do you think this will be needed longer term? If >> it is longer term, the bar is higher on what can be accepted. >> >> If this is a temporary feature, we could mark the event as >> experimental in metadata.xml, similar to what we did with GC events >> during the development of ZGC. Events that are experimental do not >> show up by default in visualization tools such as JMC and can be >> removed when they are not needed, or we have a better solution. >> > Right, it's temporary so we can mark it as experimental. I also > changed the category name to be "Java Virtual Machine, Diagnostics". > >> For events to be enabled in default.jfc, they should not cause more >> than 1% overhead, not even in pathological cases. To me, it seems >> this could happen if you make a loop where it is triggered all the time. >> >> For events in profile.jfc, the overhead should still be low (1-2%), >> but the target is the typical application. For example, allocation >> profiling is only enabled in profile.jfc. Some application that are >> allocation intensive could cause a higher overhead than 1%, but that >> is OK because that configuration is only to be used for a short >> period of time. That said, there is still a budget on how much space >> in a recording an event can take up, so it also depends on how >> important the information is for the user. To me, it seems this event >> will not be that important for the average user, which makes me think >> it should be disabled by default. >> > After our off-list discussion I left it as enabled. I also removed the > threshold setting and added the startTime=false in metadata.xml. > >> To complicate things, I noticed that aa command line flag is also >> used to enable the event. This is something we have worked hard to >> avoid when it comes to JFR events. All configuration should happen >> using configuration files. We had problem with this in the past where >> users gets confused why their events are not enabled. When we ported >> JFR to Hotspot, we got rid of those of those flags/events and it is a >> much better situation. Again, if this a temporary event to discover >> usage pattern for a release or two, it might be OK, but if we believe >> this event to stick around, I think we should look into alternatives, >> or not use JFR at all for this. >> > Yes, I see your point. Since we want to diagnose synchronization on > these classes but we don't want to affect performance the flag has to > be specified at startup otherwise the diagnostic instrumentation is > disabled, so I don't see a clear way to enable the events from JFR > alone. In any case, this will be a temporary event. > > Here is v5. Let me know if you are okay with the changes. > > Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ > Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/webrev/ > > Thanks Erik! > > Patricio >> >> Thanks >> >> Erik >> >>> On 28 Jul 2020, at 21:16, Patricio Chilano >>> >> > wrote: >>> >>> Hi all, >>> >>> Please review the following change that adds diagnostic capabilities >>> when synchronizing on primitive wrapper classes. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8242263 >>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8242263/v1/webrev/ >>> >>> The new flag allows to identify synchronization on these classes and >>> to take one of the following actions: exit the VM with fatal error, >>> log a warning message, or issue a JFR event. The implementation uses >>> a simple approach where a check is added at the beginning of the >>> monitorenter generated code when the flag is enabled to check >>> whether the object is of a primitive wrapper class. If it is, we >>> jump to the slow path, otherwise we just continue as always. The >>> extra instructions will be: load the klass of the object, load the >>> _access_flags field for that klass, AND with a constant, and branch >>> based on the result. The code will only be generated whenever the >>> new opt-in diagnostic flag is enabled so performance won't be >>> affected when off. >>> >>> In addition to the purpose described in the description of the bug, >>> this flag will also be useful when trying to diagnose possible >>> synchronization issues if these classes ever become inline types as >>> part of the Valhalla project. >>> >>> I added test SyncOnPrimitiveWrapperTest.java that tests for the exit >>> and logging cases. I added test TestSyncOnPrimitiveWrapperEvent.java >>> to test for the JFR event case. >>> I tested the patch running tiers1-6 in mach5 with the flag set to >>> DiagnoseSyncOnPrimitiveWrappers=2. >>> I checked it builds with arm32 and ppc but can't run any tests on >>> those platforms, so it would be good if somebody can run the new >>> test included in the patch. >>> >>> Let me know if you think I should run or add any more tests. >>> >>> Thanks! >>> >>> Patricio >> > From david.holmes at oracle.com Mon Aug 10 23:05:04 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Aug 2020 09:05:04 +1000 Subject: RFR: 8250636: iso8601_time returns incorrect offset part on MacOS In-Reply-To: References: <777fc30a-45ef-aa45-8f04-0bbe80c5a83d@azul.com> <0e1ed1ae-2011-2e4a-4f58-d99b57ede74c@azul.com> <934a8a40-dd51-512f-f010-909ed312783d@oracle.com> Message-ID: <3f12c3a3-e1ad-d209-bf47-f5a87685c56a@oracle.com> On 11/08/2020 5:20 am, Doerr, Martin wrote: > Hi, > > unfortunately, tm_gmtoff seems to be a BSD and GNU extension. > It's not POSIX and doesn't build on AIX: > os.cpp:163:34: error: no member named 'tm_gmtoff' in 'tm' > We'll need a follow up fix. Sorry I'll have to take most of the responsibility for that as I suggested the Windows/non-Windows split. I completely forgot about AIX and didn't check that this was a standard vs simply being available. Dmitry, this means you'll need to back up to an earlier version of the fix that still used timezone. It may be simpler to allow Linux and AIX to continue to use timezone, and keep the old special casing just for BSD and Windows. David ----- > Best regards, > Martin > > >> -----Original Message----- >> From: hotspot-runtime-dev >> On Behalf Of gerard ziemski >> Sent: Dienstag, 4. August 2020 19:02 >> To: Dmitry Cherepanov ; David Holmes >> ; hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR: 8250636: iso8601_time returns incorrect offset part on >> MacOS >> >> Looks great! >> >> On 8/4/20 4:57 AM, Dmitry Cherepanov wrote: >>> Hi David, >>> >>> On 03.08.2020 10:53, David Holmes wrote: >>>> Hi Dmitry, >>>> >>>> On 31/07/2020 8:29 pm, Dmitry Cherepanov wrote: >>>>> Hi Gerard, >>>>> >>>>> Thanks for reviewing this, moving the logic of get_timezone to >>>>> iso8601_time looks good to me too. >>>>> Here's an updated webrev >>>>> http://cr.openjdk.java.net/~dcherepanov/8250636/webrev.v4/ >>>> This version also looks good to me. But now I see all the code clearly in >> one place I have to wonder why we are not using tm_gmtoff on all platforms >> except Windows? That would simplify the logic even further. >>>> >>>> That goes beyond fixing the current actual bug though so I'll leave that >> choice to you. >>> Thanks for the suggestion. This change would not work on Solaris. But given >> the removal of Solaris port, I also see no other reasons why not to use >> tm_gmtoff on platforms except Windows. >>> >>> Updated webrev: >> http://cr.openjdk.java.net/~dcherepanov/8250636/webrev.v5/ >>> Jdk-submit testing passed with this change >>> >>> Thanks, >>> >>> Dmitry >>> > From patricio.chilano.mateo at oracle.com Mon Aug 10 23:25:01 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Mon, 10 Aug 2020 20:25:01 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <4193d602-c04d-9664-07e1-f96f1e605adb@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <29c3d48b-1875-8ea4-792a-157cdd07fe2c@oracle.com> <4193d602-c04d-9664-07e1-f96f1e605adb@oracle.com> Message-ID: <5b65ec3d-b9ff-e726-8ed1-405a7eb47b64@oracle.com> Hi Dan, On 8/10/20 5:39 PM, Daniel D. Daugherty wrote: > > Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ > > src/hotspot/share/jfr/metadata/metadata.xml > ??? No comments. > > src/hotspot/share/runtime/synchronizer.cpp > ??? L577: ??? char* newline = (char*)strchr(ss.base(), '\n'); > ??? L578: ??? if (newline != NULL) { > ??? L579: ????? *newline = '\0'; > ??? L580: ??? } > ??? L581: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of > klass %s %s", p2i(obj()), obj->klass()->external_name(), base); > ??????? Does this work as expected on Win* where is used? I > don't > ??????? remember where we convert => on Win*... Yes, I tested it in Windows and the output is the same as in Linux and MacOS. I actually forgot about the '\r\n' vs '\n' difference so after you noticed it, I thought about it and I was in fact expecting the output to be slightly different. I thought the \r would make the cursor go back to the beginning of the line, so that the next line would be written there (which is just "#\r\n"), so effectively we would lose the separating line between the "fatal error: ... " line and the "JRE version: ..." line. Not a big deal. But the output is actually the same as in Linux. Looking at ostream.cpp I see '\n' hardcoded when adding or checking for new lines. I don't find references to '\r' in other files that would indicate they are used in stringStream (or any other outputStream object for that matter). The only references to os::line_separator() are in vmError.cpp. So it looks like any newlines in stringStream are in fact encoded using '\n'. > src/jdk.jfr/share/conf/jfr/default.jfc > ??? No comments. > > src/jdk.jfr/share/conf/jfr/profile.jfc > ??? No comments. > > test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java > ??? No comments. > > Thumbs up. No need to see a new webrev if you end up making changes > for Win* above. Great, thanks for reviewing this Dan! Patricio > Dan > > > On 8/10/20 4:03 PM, Patricio Chilano wrote: >> Hi Erik, >> >> Thanks for looking into this and for the offline discussions. >> >> On 8/10/20 10:19 AM, Erik Gahlin wrote: >>> >>> Hi Patricio, >>> >>> I have tried to review the JFR changes, but I need more information >>> on how the feature is going to be used. >>> >>> Is this something temporary that will be used gain insight for the >>> Valhalla project, or do you think this will be needed longer term? >>> If it is longer term, the bar is higher on what can be accepted. >>> >>> If this is a temporary feature, we could mark the event as >>> experimental in metadata.xml, similar to what we did with GC events >>> during the development of ZGC. Events that are experimental do not >>> show up by default in visualization tools such as JMC and can be >>> removed when they are not needed, or we have a better solution. >>> >> Right, it's temporary so we can mark it as experimental. I also >> changed the category name to be "Java Virtual Machine, Diagnostics". >> >>> For events to be enabled in default.jfc, they should not cause more >>> than 1% overhead, not even in pathological cases. To me, it seems >>> this could happen if you make a loop where it is triggered all the >>> time. >>> >>> For events in profile.jfc, the overhead should still be low (1-2%), >>> but the target is the typical application. For example, allocation >>> profiling is only enabled in profile.jfc. Some application that are >>> allocation intensive could cause a higher overhead than 1%, but that >>> is OK because that configuration is only to be used for a short >>> period of time. That said, there is still a budget on how much space >>> in a recording an event can take up, so it also depends on how >>> important the information is for the user. To me, it seems this >>> event will not be that important for the average user, which makes >>> me think it should be disabled by default. >>> >> After our off-list discussion I left it as enabled. I also removed >> the threshold setting and added the startTime=false in metadata.xml. >> >>> To complicate things, I noticed that aa command line flag is also >>> used to enable the event. This is something we have worked hard to >>> avoid when it comes to JFR events. All configuration should happen >>> using configuration files. We had problem with this in the past >>> where users gets confused why their events are not enabled. When we >>> ported JFR to Hotspot, we got rid of those of those flags/events and >>> it is a much better situation. Again, if this a temporary event to >>> discover usage pattern for a release or two, it might be OK, but if >>> we believe this event to stick around, I think we should look into >>> alternatives, or not use JFR at all for this. >>> >> Yes, I see your point. Since we want to diagnose synchronization on >> these classes but we don't want to affect performance the flag has to >> be specified at startup otherwise the diagnostic instrumentation is >> disabled, so I don't see a clear way to enable the events from JFR >> alone. In any case, this will be a temporary event. >> >> Here is v5. Let me know if you are okay with the changes. >> >> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ >> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/webrev/ >> >> Thanks Erik! >> >> Patricio >>> >>> Thanks >>> >>> Erik >>> >>>> On 28 Jul 2020, at 21:16, Patricio Chilano >>>> >>> > wrote: >>>> >>>> Hi all, >>>> >>>> Please review the following change that adds diagnostic >>>> capabilities when synchronizing on primitive wrapper classes. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8242263 >>>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8242263/v1/webrev/ >>>> >>>> The new flag allows to identify synchronization on these classes >>>> and to take one of the following actions: exit the VM with fatal >>>> error, log a warning message, or issue a JFR event. The >>>> implementation uses a simple approach where a check is added at the >>>> beginning of the monitorenter generated code when the flag is >>>> enabled to check whether the object is of a primitive wrapper >>>> class. If it is, we jump to the slow path, otherwise we just >>>> continue as always. The extra instructions will be: load the klass >>>> of the object, load the _access_flags field for that klass, AND >>>> with a constant, and branch based on the result. The code will only >>>> be generated whenever the new opt-in diagnostic flag is enabled so >>>> performance won't be affected when off. >>>> >>>> In addition to the purpose described in the description of the bug, >>>> this flag will also be useful when trying to diagnose possible >>>> synchronization issues if these classes ever become inline types as >>>> part of the Valhalla project. >>>> >>>> I added test SyncOnPrimitiveWrapperTest.java that tests for the >>>> exit and logging cases. I added test >>>> TestSyncOnPrimitiveWrapperEvent.java to test for the JFR event case. >>>> I tested the patch running tiers1-6 in mach5 with the flag set to >>>> DiagnoseSyncOnPrimitiveWrappers=2. >>>> I checked it builds with arm32 and ppc but can't run any tests on >>>> those platforms, so it would be good if somebody can run the new >>>> test included in the patch. >>>> >>>> Let me know if you think I should run or add any more tests. >>>> >>>> Thanks! >>>> >>>> Patricio >>> >> > From david.holmes at oracle.com Tue Aug 11 00:26:06 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Aug 2020 10:26:06 +1000 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> <6960c0f3-8cbe-fb80-3529-a1c7fb794f0d@oracle.com> Message-ID: One follow up ... I'll send separate mail for v5. On 11/08/2020 6:19 am, Patricio Chilano wrote: > Hi David, > > On 8/10/20 1:12 AM, David Holmes wrote: >> Hi Patricio, >> >> First, I confirmed with Valhalla folk that this change is fine to go >> ahead as a stand-alone change independent of the proposed JEP in this >> area. > Great, thanks for that!? : ) > >> I've checked the incremental changes for v2, v3 and v4 and everything >> seems good to me. >> >> I only have a couple of nits with the tests: >> >> ? 54???? private static void initTestObjects() { >> ? 55???????? testObjects.add(Character.valueOf('H')); >> ? 56???????? testObjects.add(Boolean.valueOf(true)); >> ? 57???????? Byte byteT = 0x40; >> ? 58???????? testObjects.add(byteT); >> ? 59???????? Short shortT = 0x4000; >> ? 60???????? testObjects.add(shortT); >> ? 61???????? testObjects.add(Integer.valueOf(0x40000000)); >> ? 62???????? testObjects.add(Long.valueOf(0x4000000000000000L)); >> ? 63???????? testObjects.add(Float.valueOf(1.20f)); >> ? 64???????? testObjects.add(Double.valueOf(1.2345)); >> ? 65???? } >> >> Why are Byte and Short treated differently? > If I do the same as with the other types, the argument passed to > Byte.valueOf() and Short.valueOf() gets treated as an int and I get a > compilation error: > > : error: no suitable method found for valueOf(int) > ? testObjects.add(Byte.valueOf(0x40)); Right - you'd need to cast the constant to byte/short: testObjects.add(Byte.valueOf((byte)0x40)); Cheers, David ???????????????????????????????? ^ > ? method Byte.valueOf(byte) is not applicable > ? (argument mismatch; possible lossy conversion from int to byte) > ? method Byte.valueOf(String) is not applicable > ? (argument mismatch; int cannot be converted to String) > >> ?74 fatalTests[index] = Stream.of(commonFatalTestsFlags, >> specificFlags[i], new String[] >> {"SyncOnPrimitiveWrapperTest$FatalTest", Integer.toString(j)}) >> ? 75 .flatMap(Stream::of).toArray(String[]::new); >> >> The preferred Java style [1] for stream operations is: >> >> fatalTests[index] = Stream.of(commonFatalTestsFlags, specificFlags[i], >> new String[] {"SyncOnPrimitiveWrapperTest$FatalTest", >> Integer.toString(j)}) >> ????????????????????????? .flatMap(Stream::of) >> ????????????????????????? .toArray(String[]::new); > Fixed. > > Here is v5: > Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ > Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/webrev/ > > Thanks, > Patricio >> Thanks, >> David >> ----- >> >> [1] As demonstrated here (but widely used in JDK code): >> http://cr.openjdk.java.net/~alundblad/styleguide/index-v6.html#toc-wrapping-lines >> >> >> On 7/08/2020 4:48 am, Patricio Chilano wrote: >>> Hi Dan, >>> >>> On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: >>>> I'm peeking ahead to the next webrev... :-) >>>> >>>> > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ >>> : ) >>> >>>> General comments: >>>> ? - check files for copyright year updates. >>> Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, >>> c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. >>> >>>> src/hotspot/share/runtime/synchronizer.hpp >>>> ??? No comments. >>>> >>>> src/hotspot/share/runtime/synchronizer.cpp >>>> ??? L507: ? const markWord mark = obj->mark(); >>>> ??? L508: >>>> ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && >>>> obj->klass()->is_box()) { >>>> ??? L510: ??? return false; >>>> ??? L511: ? } >>>> ??????? The new bailout on L509-511 can move above L507. >>> Moved. >>> >>>> L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of klass >>>> %s", p2i(obj()), obj->klass()->external_name()); >>>> ??????? This external_name() call does not have a ResourceMark. >>> Good catch! I had one in a previous version but then I changed the >>> conditionals and lost it for the fatal error case. The test worked >>> okay because for the main JavaThread there is a ResourceMark in >>> jni_invoke_static() (jni.cpp). >>> >>>> src/hotspot/share/logging/logTag.hpp >>>> ??? No comments. >>>> >>>> src/hotspot/share/oops/klass.hpp >>>> ??? No comments. >>>> >>>> src/hotspot/share/utilities/accessFlags.hpp >>>> ??? No comments. >>>> >>>> src/hotspot/share/runtime/globals.hpp >>>> ??? L814: ???????????? "0: off " >>>> ??????? Missing a ';' after "off". >>> Fixed. >>> >>>> L816: ???????????? "2: log message to stdout. >>>> ??????? Perhaps add "(by default)" after "stdout" or >>>> ??????? don't say where log output is at all. >>>> >>>> src/hotspot/share/runtime/arguments.cpp >>>> ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, true, >>>> LOG_TAGS(primitivewrappers)); >>>> ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp >>>> ??????? since it looks like logging output is configured to 'stdout'. >>>> ??????? I'm assuming that other log options to put output elsewhere >>>> ??????? are overridden by this code? >>> Right. So the logging is done under UL with the tag >>> primitivewrappers. If that tag is specified in Xlog then this >>> conditional will be skipped because !log_is_enabled(Info, >>> primitivewrappers) will be false. >>> >>>> src/hotspot/share/classfile/systemDictionary.cpp >>>> ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { >>>> ??????? nit - s/T_LONG+1/T_LONG + 1/ >>> Fixed. >>> >>>> L2191: _box_klasses[i]->set_prototype_header(markWord::prototype()); >>>> ??????? I assume we're keeping the prototype_header field when >>>> Biased Locking >>>> ??????? goes away? The reason I ask: >>>> >>>> ? ? ? ? static markWord prototype() { >>>> ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); >>>> ? ? ? ? } >>>> >>>> ??????? is because without Biased Locking, do we really need the >>>> prototype >>>> ??????? anymore? The initial markWord won't need possible variants... >>> Yes, I think it can go away unless somebody finds another use for it. >>> Maybe Valhalla. >>> >>>> src/hotspot/share/jfr/metadata/metadata.xml >>>> ??? L69: ? >>> ??????? Is the category "Java Application" because it's the application >>>> ??????? code that did something "wrong" here? Where "application" is >>>> loosely >>>> ??????? defined to include the possibility of auto generated code, >>>> library >>>> ??????? code and the like? Or perhaps "application" because >>>> something "above" >>>> ??????? the "Java Virtual Machine, Runtime" did the "wrong" thing here? >>> I don't know what the right category should be really. I saw the >>> events JavaMonitorEnter, JavaMonitorWait and JavaMonitorInflate and >>> thought this event should fall in the same category they do. >>> >>>> src/jdk.jfr/share/conf/jfr/default.jfc >>>> ??? No comments. >>>> >>>> src/jdk.jfr/share/conf/jfr/profile.jfc >>>> ??? No comments. >>>> >>>> test/lib/jdk/test/lib/jfr/EventNames.java >>>> ??? No comments. >>>> >>>> src/hotspot/cpu/aarch64/aarch64.ad >>>> ??? L3517: ????? __ tbnz(tmp, exact_log2(JVM_ACC_IS_BOX_CLASS), cont); >>>> >>>> ??? L3578: ??? __ bind(cont); >>>> ??? L3579: ??? // flag == EQ indicates success >>>> ??? L3580: ??? // flag == NE indicates failure >>>> ??????? If tbnz() branches to "cont" when we have a box class, what's >>>> ??????? the flag value set to (EQ or NE)? And what set that flag value? >>>> ??????? The reason I ask is I don't think tbnz() sets condition >>>> codes... >>> Right, it doesn't set condition codes, so I kept the version I had >>> (more on that below). >>> >>>> src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>> ??? No comments. >>>> >>>> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp >>>> ??? No comments. >>>> >>>> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp >>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>> ??????????? ? ?? ??????????????????????????? Register obj_reg, >>>> ?????????????????????????????? ? ?? ???????? Register swap_reg, >>>> ?????????????????????????????????? ? ?? ???? Register tmp_reg, >>>> ?????????????????????????????????? ? ?? ???? bool >>>> swap_reg_contains_mark, >>>> ?????????????????????????????????? ? ?? ???? Label& done, >>>> ??????????????????????????????????? ? ?? ??? Label* slow_case, >>>> BiasedLockingCounters* counters) { >>>> ??????? I think you've changed the only callers of >>>> biased_locking_enter() >>>> ??????? that cared about the return value with this changeset so it can >>>> ??????? be changed to a void function. >>> Ok, this is what I mentioned to David in a previous email. Done. >>> >>>> src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp >>>> ??? No comments. >>>> >>>> src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp >>>> ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), done); >>>> >>>> ??? L131: ? bind(done); >>>> ??? L132: >>>> ??? L133: ? // At this point flags are set as follows: >>>> ??? L134: ? //? EQ -> Success >>>> ??? L135: ? //? NE -> Failure, branch to slow path >>>> ??????? If tbnz() branches to "done" when we have a box class, what's >>>> ??????? the flag value set to (EQ or NE)? And what set that flag value? >>>> ??????? The reason I ask is I don't think tbnz() sets condition >>>> codes... >>> Right. Same as above, I kept the version I had. >>> >>>> src/hotspot/cpu/arm/interp_masm_arm.cpp >>>> ??? No comments. >>>> >>>> src/hotspot/cpu/arm/macroAssembler_arm.cpp >>>> ??? int MacroAssembler::biased_locking_enter(Register obj_reg, >>>> Register swap_reg, Register tmp_reg, >>>> ????????????????????? ? ?? ????????????????? bool >>>> swap_reg_contains_mark, >>>> ???????????????????????? ? ?? ?????????????? Register tmp2, >>>> ????????????????????????? ? ?? ????????????? Label& done, Label& >>>> slow_case, >>>> BiasedLockingCounters* counters) { >>>> ??????? I think you've changed the only callers of >>>> biased_locking_enter() >>>> ??????? that cared about the return value with this changeset so it can >>>> ??????? be changed to a void function. >>> Done. >>> >>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>> ??? No comments on the PPC code. >>>> >>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>> src/hotspot/cpu/s390/interp_masm_s390.cpp >>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>> ??? No comments on the S390 code. >>>> >>>> src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp >>>> ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); >>>> ??????? What will this do with a 'noreg' value? (I need a refresher.) >>> When not in 64 bit mode that register just won't be used. >>> >>>> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp >>>> ??? No comments. >>>> >>>> src/hotspot/cpu/x86/interp_masm_x86.cpp >>>> ??? No comments. >>>> >>>> src/hotspot/cpu/x86/macroAssembler_x86.cpp >>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>> ????????????????????????????????? ? ?? ????? Register obj_reg, >>>> ????????????????????????????????? ? ?? ????? Register swap_reg, >>>> ????????????????????????????????? ? ?? ????? Register tmp_reg, >>>> ?????????????????????????????????? ? ?? ???? Register tmp_reg2, >>>> ?????????????????????????????????????? ? ? ? bool >>>> swap_reg_contains_mark, >>>> ?????????????????????????????????? ? ?? ???? Label& done, >>>> ?????????????????????????????????? ? ?? ???? Label* slow_case, >>>> BiasedLockingCounters* counters) { >>>> ??????? I think you've changed the only caller of >>>> biased_locking_enter() >>>> ??????? that cared about the return value with this changeset so it can >>>> ??????? be changed to a void function. >>> Done. >>> >>>> test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >>>> ??? L30:? * @test SyncOnPrimitiveWrapperTest >>>> ??????? No parameter to @test directive. >>> Removed. >>> >>>> L136: ??????? private static long sharedCounter = 0L; >>>> ??????? Since you don't do anything with sharedCounter other than >>>> increment it, >>>> ??????? can the compilers optimize it away? If it can be optimized >>>> away, does >>>> ??????? that mean that: >>>> >>>> ??????????? L142: ??????????????? synchronized (obj) { >>>> >>>> ??????? can also be optimized away? >>>> >>>> ??????? I don't think that: >>>> >>>> ??????????? L161: ??????????????? synchronized (sharedLock1) { >>>> >>>> ??????? can be optimized away because it is shared by multiple threads. >>>> >>>> test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java >>>> ??? Similar questions about 'counter' being optimized away. >>>> ??? Similar question about "synchronized (obj) {" being optimized away. >>> I'm not sure if the compiler will optimize it away. Seems unlikely >>> given the counter we are incrementing is not just local to some thread. >>> >>> >>> Ok, below is v3 which has the following changes: >>> - Use a 32 bit load for the _access_flags field, instead of 64 >>> - Martin's implementation for s390 and fix for PPC >>> - Faster LogTest version >>> - Above changes based on Dan review >>> >>> I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 only) >>> again with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I checked it builds >>> in arm32, ppc and s390. >>> >>> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev >>> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev >>> >>> >>> @Martin: >>> Please check if the test doesn't timeout for you now with the changes >>> I made to LogTest. >>> Also, I tried to use tbnz in aarch64 and arm32 instead of tst + br >>> (except for c2 since we actually need to set the condition flags), >>> but for c1 I was getting an assertion in the compiler thread: >>> >>> guarantee(chk == -1 || chk == 0) failed: Field too big for insn >>> >>> This is the stack when that happens: >>> >>> MacroAssembler::pd_patch_instruction_size(unsigned char*, unsigned >>> char*)+0x398 >>> AbstractAssembler::bind(Label&)+0x118 >>> MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 >>> LIR_Assembler::emit_slow_case_stubs()+0x54 >>> Compilation::emit_code_body()+0x17c >>> >>> The change was simply: >>> >>> diff --git a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>> b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>> --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>> +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>> @@ -78,7 +78,6 @@ >>> ??? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>> ????? load_klass(hdr, obj); >>> -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); >>> -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); >>> -??? br(Assembler::NE, slow_case); >>> +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); >>> +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); >>> ??? } >>> >>> So the issue must be related to where we want to jump. >>> >>> Thanks, >>> Patricio >>>> Dan >>>> >>>> >>>> On 8/5/20 9:01 AM, Patricio Chilano wrote: >>>>> Hi Martin, >>>>> >>>>> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>>>>> Hi Patricio, >>>>>> >>>>>> using 8 Byte load instructions for jint fields is a terrible >>>>>> coding style! >>>>>> Someone else may see it and use an 8 Byte store. Will result in >>>>>> great fun for debugging! >>>>>> >>>>>> And on Big Endian you will simply access the wrong bits. >>>>> Ah, of course! Those 32 bits will be in the wrong place when doing >>>>> the test. >>>>> >>>>>> Note that Big Endian Platforms are AIX, old linux ppc, s390, >>>>>> SPARC. I don't think you have tested on them. >>>>>> >>>>>>> We could remove the nested synchronized statements in the run() >>>>>>> method >>>>>>> so that we don't generate that much logging. We could also lower >>>>>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>>>>> with multiple flag combinations. Not sure what we should touch >>>>>>> first. >>>>>>> Maybe the synchronized statements, have only one or two and test >>>>>>> that >>>>>>> first? >>>>>> Sounds like helpful ideas. Please go ahead and strip things down. >>>>> Great, I will send v3 later with those changes. >>>>> >>>>> Thanks Martin! >>>>> >>>>> Patricio >>>>>> Thanks for taking care of it. >>>>>> >>>>>> Best regards, >>>>>> Martin >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Patricio Chilano >>>>>>> Sent: Dienstag, 4. August 2020 20:26 >>>>>>> To: Doerr, Martin ; David Holmes >>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>>> wrappers >>>>>>> >>>>>>> Hi Martin, >>>>>>> >>>>>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>>>>> Hi Patricio, >>>>>>>> >>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>> sending v3. >>>>>>>> Thanks. I forgot to mention arm + aarch64. >>>>>>>> Aarch64 uses ldrw + tbnz. >>>>>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>>>>> >>>>>>>> I remember having seen the same mistake ?? >>>>>>>> And nobody noticed it on little Endian platforms. >>>>>>> Ok, I can use a tbnz instead of test and then a branch on arm32 >>>>>>> and aarch64. >>>>>>> Shouldn't a normal ldr in arm32 work fine? >>>>>>> Also in 64 bits (either x64 or aarch64) I don't see the issue of >>>>>>> using a >>>>>>> 64 bit load, besides the fact that we only care about the first >>>>>>> 32 bits. >>>>>>> Regardless of the endianness, aren't we masking out the upper >>>>>>> part when >>>>>>> we do AND/TEST or if we test a bit in the 0-31 bit range? >>>>>>> Otherwise it >>>>>>> seems it should have failed for one of those platforms in my tests. >>>>>>> >>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>>> works and I >>>>>>>>> can add it to the test. >>>>>>>> That seems to have an effect. But now I'm not patient enough to >>>>>>>> wait for >>>>>>> the test to finish. >>>>>>>> Maybe the problem is that I'm using slow debug builds. >>>>>>>> But is there a chance to make the test quicker without losing >>>>>>>> coverage >>>>>>> significantly? >>>>>>>> I think the test is too slow this way to run it on a regular >>>>>>>> basis. We'd need to >>>>>>> spend dedicated machines for it. >>>>>>> We could remove the nested synchronized statements in the run() >>>>>>> method >>>>>>> so that we don't generate that much logging. We could also lower >>>>>>> LOOP_COUNT. I think the issue is also because we are running LogTest >>>>>>> with multiple flag combinations. Not sure what we should touch >>>>>>> first. >>>>>>> Maybe the synchronized statements, have only one or two and test >>>>>>> that >>>>>>> first? >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Patricio >>>>>>>> Best regards, >>>>>>>> Martin >>>>>>>> >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Patricio Chilano >>>>>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>>>>> wrappers >>>>>>>>> >>>>>>>>> Hi Martin, >>>>>>>>> >>>>>>>>> Thanks for fixing PPC and taking care of s390! >>>>>>>>> >>>>>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>>>>> Hi Patricio, >>>>>>>>>> >>>>>>>>>> I suggest to use movl + testl for checking the access flag as >>>>>>>>>> for other >>>>>>> access >>>>>>>>> flags on x64. >>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>> sending v3. >>>>>>>>> >>>>>>>>>> New version for PPC64 and s390 see below. >>>>>>>>>> >>>>>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>>>>>> expected. >>>>>>>>> However, I'm getting timeout issues: >>>>>>>>>> timed out (timeout set to 120000ms, elapsed time including >>>>>>>>>> timeout >>>>>>>>> handling was 122372ms) >>>>>>>>>> Can we provide more time? >>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you can >>>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>>> works and I >>>>>>>>> can add it to the test. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Patricio >>>>>>>>>> Best regards, >>>>>>>>>> Martin >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> diff -r 77852e129401 >>>>>>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>> @@ -107,8 +107,8 @@ >>>>>>>>>> >>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>>> Rscratch); >>>>>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>> +??? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>>> Rscratch); >>>>>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>> ??????? bne(CCR0, slow_case); >>>>>>>>>> ????? } >>>>>>>>>> >>>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>> @@ -912,8 +912,8 @@ >>>>>>>>>> >>>>>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>> ????????? load_klass(tmp, object); >>>>>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>>> +????? testbitdi(CCR0, R0, tmp, >>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>> ????????? bne(CCR0, slow_case); >>>>>>>>>> ??????? } >>>>>>>>>> >>>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>>>>> >>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>> ??????? load_klass(temp, oop); >>>>>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>>> +??? testbitdi(CCR0, R0, temp, exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>> ??????? bne(CCR0, cont); >>>>>>>>>> ????? } >>>>>>>>>> >>>>>>>>>> diff -r 77852e129401 >>>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue Aug 04 >>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>> @@ -91,6 +91,12 @@ >>>>>>>>>> ????? // Save object being locked into the BasicObjectLock... >>>>>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>>>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>> +??? z_btrue(slow_case); >>>>>>>>>> +? } >>>>>>>>>> + >>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>>>>> Z_R0_scratch, done, >>>>>>>>> &slow_case); >>>>>>>>>> ????? } >>>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>>>> 10:03:57 >>>>>>>>> 2020 +0200 >>>>>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>> 16:04:31 >>>>>>>>> 2020 +0200 >>>>>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>>>>> ????? z_lg(displaced_header, oopDesc::mark_offset_in_bytes(), >>>>>>>>>> object); >>>>>>>>>> >>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>> +??? z_btrue(slow_case); >>>>>>>>>> +? } >>>>>>>>>> + >>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>> ??????? biased_locking_enter(object, displaced_header, Z_R1, >>>>>>>>>> Z_R0, done, >>>>>>>>> &slow_case); >>>>>>>>>> ????? } >>>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>>>>> ????? // Load markWord from oop into mark. >>>>>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>>>>> >>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>>>>> +??? testbit(Address(Z_R1_scratch, Klass::access_flags_offset()), >>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>> +??? z_btrue(done); >>>>>>>>>> +? } >>>>>>>>>> + >>>>>>>>>> ????? if (try_bias) { >>>>>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, Z_R0, >>>>>>>>>> done); >>>>>>>>>> ????? } >>>>>>>>>> >>>>> >>>> >>> > From david.holmes at oracle.com Tue Aug 11 00:40:12 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Aug 2020 10:40:12 +1000 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <29c3d48b-1875-8ea4-792a-157cdd07fe2c@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <29c3d48b-1875-8ea4-792a-157cdd07fe2c@oracle.com> Message-ID: <5f3a48b0-274f-c0f6-57a1-8fbb2bebf823@oracle.com> Hi Patricio, On 11/08/2020 6:03 am, Patricio Chilano wrote: > Hi Erik, > > Thanks for looking into this and for the offline discussions. If there are still issues with this regarding JFR then I would suggest we simply drop the use of JFR for reporting this. Meanwhile looking at v5 incremental that all seems fine to me. Thanks, David ----- > On 8/10/20 10:19 AM, Erik Gahlin wrote: >> >> Hi Patricio, >> >> I have tried to review the JFR changes, but I need more information on >> how the feature is going to be used. >> >> Is this something temporary that will be used gain insight for the >> Valhalla project, or do you think this will be needed longer term? If >> it is longer term, the bar is higher on what can be accepted. >> >> If this is a temporary feature, we could mark the event as >> experimental in metadata.xml, similar to what we did with GC events >> during the development of ZGC. Events that are experimental do not >> show up by default in visualization tools such as JMC and can be >> removed when they are not needed, or we have a better solution. >> > Right, it's temporary so we can mark it as experimental. I also changed > the category name to be "Java Virtual Machine, Diagnostics". > >> For events to be enabled in default.jfc, they should not cause more >> than 1% overhead, not even in pathological cases. To me, it seems this >> could happen if you make a loop where it is triggered all the time. >> >> For events in profile.jfc, the overhead should still be low (1-2%), >> but the target is the typical application. For example, allocation >> profiling is only enabled in profile.jfc. Some application that are >> allocation intensive could cause a higher overhead than 1%, but that >> is OK because that configuration is only to be used for a short period >> of time. That said, there is still a budget on how much space in a >> recording an event can take up, so it also depends on how important >> the information is for the user. To me, it seems this event will not >> be that important for the average user, which makes me think it should >> be disabled by default. >> > After our off-list discussion I left it as enabled. I also removed the > threshold setting and added the startTime=false in metadata.xml. > >> To complicate things, I noticed that aa command line flag is also used >> to enable the event. This is something we have worked hard to avoid >> when it comes to JFR events. All configuration should happen using >> configuration files. We had problem with this in the past where users >> gets confused why their events are not enabled. When we ported JFR to >> Hotspot, we got rid of those of those flags/events and it is a much >> better situation. Again, if this a temporary event to discover usage >> pattern for a release or two, it might be OK, but if we believe this >> event to stick around, I think we should look into alternatives, or >> not use JFR at all for this. >> > Yes, I see your point. Since we want to diagnose synchronization on > these classes but we don't want to affect performance the flag has to be > specified at startup otherwise the diagnostic instrumentation is > disabled, so I don't see a clear way to enable the events from JFR > alone. In any case, this will be a temporary event. > > Here is v5. Let me know if you are okay with the changes. > > Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ > Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/webrev/ > > Thanks Erik! > > Patricio >> >> Thanks >> >> Erik >> >>> On 28 Jul 2020, at 21:16, Patricio Chilano >>> >> > wrote: >>> >>> Hi all, >>> >>> Please review the following change that adds diagnostic capabilities >>> when synchronizing on primitive wrapper classes. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8242263 >>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8242263/v1/webrev/ >>> >>> The new flag allows to identify synchronization on these classes and >>> to take one of the following actions: exit the VM with fatal error, >>> log a warning message, or issue a JFR event. The implementation uses >>> a simple approach where a check is added at the beginning of the >>> monitorenter generated code when the flag is enabled to check whether >>> the object is of a primitive wrapper class. If it is, we jump to the >>> slow path, otherwise we just continue as always. The extra >>> instructions will be: load the klass of the object, load the >>> _access_flags field for that klass, AND with a constant, and branch >>> based on the result. The code will only be generated whenever the new >>> opt-in diagnostic flag is enabled so performance won't be affected >>> when off. >>> >>> In addition to the purpose described in the description of the bug, >>> this flag will also be useful when trying to diagnose possible >>> synchronization issues if these classes ever become inline types as >>> part of the Valhalla project. >>> >>> I added test SyncOnPrimitiveWrapperTest.java that tests for the exit >>> and logging cases. I added test TestSyncOnPrimitiveWrapperEvent.java >>> to test for the JFR event case. >>> I tested the patch running tiers1-6 in mach5 with the flag set to >>> DiagnoseSyncOnPrimitiveWrappers=2. >>> I checked it builds with arm32 and ppc but can't run any tests on >>> those platforms, so it would be good if somebody can run the new test >>> included in the patch. >>> >>> Let me know if you think I should run or add any more tests. >>> >>> Thanks! >>> >>> Patricio >> > From david.holmes at oracle.com Tue Aug 11 01:54:51 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Aug 2020 11:54:51 +1000 Subject: RFR: 8251365: Build failure on AIX after 8250636 In-Reply-To: <48c0d4f2-ffa7-a242-bb5a-f53781eaa5ff@azul.com> References: <48c0d4f2-ffa7-a242-bb5a-f53781eaa5ff@azul.com> Message-ID: Hi Dmitry, I didn't see this when replying to the other thread. On 11/08/2020 6:27 am, Dmitry Cherepanov wrote: > Please review the fix for build failure on AIX. With this change, "tm_gmtoff" is used only if _ALLBSD_SOURCE or _GNU_SOURCE is set. In the case of AIX, we'll continue to use "timezone". > > I'll start jdk-submit testing for this change shortly, unfortunately I do not have an AIX machine at hand and would appreciate if someone (with access to AIX) could verify that the change fixes AIX build . Thanks in advance. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251365 > Webrev: http://cr.openjdk.java.net/~dcherepanov/8251365/webrev.v1/ This seems okay to me. All the ifdefs are a bit jarring (makes it seem like there should be a pd_xxx method underpinning this) but acceptable. Thanks, David > Thanks, > > Dmitry > From ioi.lam at oracle.com Tue Aug 11 05:29:06 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 10 Aug 2020 22:29:06 -0700 Subject: RFR(S) 8249276 CDS archived objects must have "neutral" markwords In-Reply-To: <11f16292-909d-6d61-d353-cfa9d7054506@oracle.com> References: <5d5b4b17-a40f-b0f4-fa50-fdd783def9d4@oracle.com> <37245599-dfb8-0481-6c74-89cffdc5b457@oracle.com> <7a81547c-8221-da80-2d1a-edb42f4078fc@oracle.com> <11f16292-909d-6d61-d353-cfa9d7054506@oracle.com> Message-ID: On 8/6/20 5:55 PM, David Holmes wrote: > Hi Ioi, > > On 6/08/2020 8:26 am, Ioi Lam wrote: >> Hi David & Coleen, >> >> Thanks for the review. I have created a new version: >> >> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ >> > > Functional changes are good. > > For the test, as per our offline chat, the Helper class is not being > archived. This line: > > ?66???????????????? TestCommon.testDump(appJar, > TestCommon.list(LockDuringDumpApp.class.getName()), > > should be > > ?66???????????????? TestCommon.testDump(appJar, > TestCommon.list(appClasses), > > Hi David, (The Helper class was actually archived, because it was loaded by LockDuringDumpAgent during dump time, so it was not necessary to explicitly include it in the class list. ....) Anyway, it wasn't clear to me whether any code (now or in the future) in the JVM might synchronize on the java mirrors. So I've updated the test case to synchronize on a literal string instead. This way I know that only the test, and no one else, will synchronize on it. I also added comments to explain how the test works. http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03/ Thanks - Ioi >> >> I created a new test case (LockDuringDump), and incorporated David's >> fix for JDK-8250606 [1] during my testing: >> >> + Without my patch, LockDuringDumpApp would crash at runtime, because >> ?? we have an invalid markword for the Helper.class object. >> >> + Without the JDK-8250606 patch, we would assert during CDS dump time. >> >> I also removed the assert in HeapShared::oop_hash() since it's >> irrelevant after JDK-8250606. >> >> I intend to push my patch after JDK-8250606 is integrated. >> >> >> ------- >> >> More comments below >> >> On 8/3/20 6:13 PM, David Holmes wrote: >>> Hi Ioi, >>> >>> On 27/07/2020 3:20 pm, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8249276 >>>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v01/ >>>> >>>> >>>> Please review this change (initial patch provided by David Holmes; >>>> I rearranged it a little and added more comments). >>>> >>>> We reinitialize the markWord of all archived objects to remove any >>>> side effect of GC/locking/etc during "java -Xshare:dump". >>> >>> Changes look good to me (naturally :) ). A couple of nits: >>> >>> ?? int hash_archived = archived_oop->identity_hash(); >>> ?? assert(hash_original == hash_archived, "Different hash codes: >>> original %x, archived %x", hash_original, hash_archived); >>> >>> the has_archived setting should be in ifdef DEBUG. >>> >> >> Fixed >> >> >>> 42 Object.class.wait(); >>> ? 43???????????????? } >>> ? 44???????????????? System.out.println("Huh?? notified??"); >>> >>> That could be a spurious wakeup. You could just use Thread.sleep >>> with a suitably long sleep time. that said I'm not sure how the test >>> is intended to operate. If premain returns after starting the thread >>> there is no guarantee the thread will start executing and lock the >>> object before the dump actually commences. Even the sighting of >>> "Let's wait ....." is not sufficient to guarantee that unless you >>> move it inside the sync block. >>> >> >> I have fixed the test case so that the "Let's hold the lock" message >> is printed after the lock is held, and the lock is never released >> afterwards. >> >> I also added synchronization to make sure the premain() method >> doesn't return until the child thread has held the lock. The log file >> shows that theses messages are printed before CDS tries to load >> classes from the classlist: >> >> [0.005s][info][cds] Allocated shared space: 3221225472 bytes at >> 0x0000000800000000 >> inside LockDuringDumpAgent: >> jdk.internal.loader.ClassLoaders$AppClassLoader at 707f7052 >> Helper class is initialized >> Class = class LockDuringDumpApp$Helper >> Let's hold the lock on Helper.class forever ..... >> Thread has started >> [0.145s][info][cds] Loading classes to share ... >> >>>> (David mentioned in the bug comments about assertions inside the >>>> identity_hash() call, but maybe this should be fixed in a different >>>> bug?) >>> >>> Yes those assertions are being removed under: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8250606 >>> >> >> Thanks >> - Ioi >> >> ------- >> >> [1] >> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/040985.html >> >> >> From martin.doerr at sap.com Tue Aug 11 07:46:46 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 11 Aug 2020 07:46:46 +0000 Subject: RFR: 8250636: iso8601_time returns incorrect offset part on MacOS In-Reply-To: <2b0273f3-3541-122b-4af1-56e7cd335e86@azul.com> References: <777fc30a-45ef-aa45-8f04-0bbe80c5a83d@azul.com> <0e1ed1ae-2011-2e4a-4f58-d99b57ede74c@azul.com> <934a8a40-dd51-512f-f010-909ed312783d@oracle.com> <2b0273f3-3541-122b-4af1-56e7cd335e86@azul.com> Message-ID: Hi Dmitry, no problem. Such things happen. Thanks a lot for taking care of it. Best regards, Martin > -----Original Message----- > From: Dmitry Cherepanov > Sent: Montag, 10. August 2020 22:12 > To: Doerr, Martin ; gerard ziemski > ; David Holmes ; > hotspot-runtime-dev at openjdk.java.net; Stuefe, Thomas > > Subject: Re: RFR: 8250636: iso8601_time returns incorrect offset part on > MacOS > > Hi Martin, > > On 10.08.2020 22:20, Doerr, Martin wrote: > > Hi, > > > > unfortunately, tm_gmtoff seems to be a BSD and GNU extension. > > It's not POSIX and doesn't build on AIX: > > os.cpp:163:34: error: no member named 'tm_gmtoff' in 'tm' > > We'll need a follow up fix. > > Sorry for the broken build, I filed a new bug > (https://bugs.openjdk.java.net/browse/JDK-8251365) and will send a review > request soon. > > Dmitry > From martin.doerr at sap.com Tue Aug 11 07:47:30 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 11 Aug 2020 07:47:30 +0000 Subject: RFR: 8250636: iso8601_time returns incorrect offset part on MacOS In-Reply-To: <3f12c3a3-e1ad-d209-bf47-f5a87685c56a@oracle.com> References: <777fc30a-45ef-aa45-8f04-0bbe80c5a83d@azul.com> <0e1ed1ae-2011-2e4a-4f58-d99b57ede74c@azul.com> <934a8a40-dd51-512f-f010-909ed312783d@oracle.com> <3f12c3a3-e1ad-d209-bf47-f5a87685c56a@oracle.com> Message-ID: Hi David, no problem. I'm glad Dmitry is fixing it. Best regards, Martin > -----Original Message----- > From: David Holmes > Sent: Dienstag, 11. August 2020 01:05 > To: Doerr, Martin ; gerard ziemski > ; Dmitry Cherepanov > ; hotspot-runtime-dev at openjdk.java.net; > Stuefe, Thomas > Subject: Re: RFR: 8250636: iso8601_time returns incorrect offset part on > MacOS > > On 11/08/2020 5:20 am, Doerr, Martin wrote: > > Hi, > > > > unfortunately, tm_gmtoff seems to be a BSD and GNU extension. > > It's not POSIX and doesn't build on AIX: > > os.cpp:163:34: error: no member named 'tm_gmtoff' in 'tm' > > We'll need a follow up fix. > > Sorry I'll have to take most of the responsibility for that as I > suggested the Windows/non-Windows split. I completely forgot about AIX > and didn't check that this was a standard vs simply being available. > > Dmitry, this means you'll need to back up to an earlier version of the > fix that still used timezone. It may be simpler to allow Linux and AIX > to continue to use timezone, and keep the old special casing just for > BSD and Windows. > > David > ----- > > > Best regards, > > Martin > > > > > >> -----Original Message----- > >> From: hotspot-runtime-dev retn at openjdk.java.net> > >> On Behalf Of gerard ziemski > >> Sent: Dienstag, 4. August 2020 19:02 > >> To: Dmitry Cherepanov ; David Holmes > >> ; hotspot-runtime-dev at openjdk.java.net > >> Subject: Re: RFR: 8250636: iso8601_time returns incorrect offset part on > >> MacOS > >> > >> Looks great! > >> > >> On 8/4/20 4:57 AM, Dmitry Cherepanov wrote: > >>> Hi David, > >>> > >>> On 03.08.2020 10:53, David Holmes wrote: > >>>> Hi Dmitry, > >>>> > >>>> On 31/07/2020 8:29 pm, Dmitry Cherepanov wrote: > >>>>> Hi Gerard, > >>>>> > >>>>> Thanks for reviewing this, moving the logic of get_timezone to > >>>>> iso8601_time looks good to me too. > >>>>> Here's an updated webrev > >>>>> http://cr.openjdk.java.net/~dcherepanov/8250636/webrev.v4/ > >>>> This version also looks good to me. But now I see all the code clearly in > >> one place I have to wonder why we are not using tm_gmtoff on all > platforms > >> except Windows? That would simplify the logic even further. > >>>> > >>>> That goes beyond fixing the current actual bug though so I'll leave that > >> choice to you. > >>> Thanks for the suggestion. This change would not work on Solaris. But > given > >> the removal of Solaris port, I also see no other reasons why not to use > >> tm_gmtoff on platforms except Windows. > >>> > >>> Updated webrev: > >> http://cr.openjdk.java.net/~dcherepanov/8250636/webrev.v5/ > >>> Jdk-submit testing passed with this change > >>> > >>> Thanks, > >>> > >>> Dmitry > >>> > > From david.holmes at oracle.com Tue Aug 11 09:17:31 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Aug 2020 19:17:31 +1000 Subject: RFR (s) 8251383: Disable Event::log from linux_mprotect when processing the assertion poison page Message-ID: <4f20d795-fea6-282b-cc96-2a91e0665a53@oracle.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8251383 webrev: http://cr.openjdk.java.net/~dholmes/8251383/webrev/ When the assertion poison page is enabled (Linux only and on by default) the signal handler will call os::protect_memory to change the page protection bits. That will call linux_mprotect which will call Events::log(NULL, "Protecting memory [" INTPTR_FORMAT "," INTPTR_FORMAT "] with protection modes %x", p2i(bottom), p2i(bottom+size), prot); Event logging in turn can use Mutexes and other VM facilities - all of which are now being executed in a signal handling context (which is inherently unsafe). It also means that there cannot be any other failing assertions on that path as you will re-trigger the poison page pagefault and abort with no hs_err file. Hence, as happened to me, a failing assertion in the mutex code triggers this problem. The issue can be worked-around by setting -XX:-ShowRegistersOnAssert (once you realise what is happening). The simple fix is to skip the logging if the faulting address is the poison page address. This only affects debug builds of course. Testing: - runtime/ErrorHandling - tier 1-3 Thanks, David From thomas.stuefe at gmail.com Tue Aug 11 09:28:44 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 11 Aug 2020 11:28:44 +0200 Subject: RFR (s) 8251383: Disable Event::log from linux_mprotect when processing the assertion poison page In-Reply-To: <4f20d795-fea6-282b-cc96-2a91e0665a53@oracle.com> References: <4f20d795-fea6-282b-cc96-2a91e0665a53@oracle.com> Message-ID: Hi David, ugh. Sorry for the complexity. Our compiler guys really liked this feature, but I still have the nagging feeling this was not my best development ever (or, more precisely, that the added complexity may not be worth the information). As for the patch, you should use CAN_SHOW_REGISTERS_ON_ASSERT instead of ASSERT/DEBUG since this may break zero. See debug.hpp. Cheers, Thomas On Tue, Aug 11, 2020 at 11:17 AM David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8251383 > webrev: http://cr.openjdk.java.net/~dholmes/8251383/webrev/ > > When the assertion poison page is enabled (Linux only and on by default) > the signal handler will call os::protect_memory to change the page > protection bits. That will call linux_mprotect which will call > > Events::log(NULL, "Protecting memory [" INTPTR_FORMAT "," INTPTR_FORMAT > "] with protection modes %x", p2i(bottom), p2i(bottom+size), prot); > > Event logging in turn can use Mutexes and other VM facilities - all of > which are now being executed in a signal handling context (which is > inherently unsafe). It also means that there cannot be any other failing > assertions on that path as you will re-trigger the poison page pagefault > and abort with no hs_err file. Hence, as happened to me, a failing > assertion in the mutex code triggers this problem. > > The issue can be worked-around by setting -XX:-ShowRegistersOnAssert > (once you realise what is happening). > > The simple fix is to skip the logging if the faulting address is the > poison page address. > > This only affects debug builds of course. > > Testing: > - runtime/ErrorHandling > - tier 1-3 > > Thanks, > David > From david.holmes at oracle.com Tue Aug 11 09:31:31 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Aug 2020 19:31:31 +1000 Subject: RFR(S) 8249276 CDS archived objects must have "neutral" markwords In-Reply-To: References: <5d5b4b17-a40f-b0f4-fa50-fdd783def9d4@oracle.com> <37245599-dfb8-0481-6c74-89cffdc5b457@oracle.com> <7a81547c-8221-da80-2d1a-edb42f4078fc@oracle.com> <11f16292-909d-6d61-d353-cfa9d7054506@oracle.com> Message-ID: <352a476d-685c-4530-994b-0e89593ef261@oracle.com> Hi Ioi, Okay nothing further from me on the test. Thanks, David On 11/08/2020 3:29 pm, Ioi Lam wrote: > > > On 8/6/20 5:55 PM, David Holmes wrote: >> Hi Ioi, >> >> On 6/08/2020 8:26 am, Ioi Lam wrote: >>> Hi David & Coleen, >>> >>> Thanks for the review. I have created a new version: >>> >>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ >>> >> >> Functional changes are good. >> >> For the test, as per our offline chat, the Helper class is not being >> archived. This line: >> >> ?66???????????????? TestCommon.testDump(appJar, >> TestCommon.list(LockDuringDumpApp.class.getName()), >> >> should be >> >> ?66???????????????? TestCommon.testDump(appJar, >> TestCommon.list(appClasses), >> >> > > Hi David, > > (The Helper class was actually archived, because it was loaded by > LockDuringDumpAgent during dump time, so it was not necessary to > explicitly include it in the class list. ....) > > Anyway, it wasn't clear to me whether any code (now or in the future) in > the JVM might synchronize on the java mirrors. So I've updated the test > case to synchronize on a literal string instead. This way I know that > only the test, and no one else, will synchronize on it. > > I also added comments to explain how the test works. > > http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03/ > > > Thanks > - Ioi > > >>> >>> I created a new test case (LockDuringDump), and incorporated David's >>> fix for JDK-8250606 [1] during my testing: >>> >>> + Without my patch, LockDuringDumpApp would crash at runtime, because >>> ?? we have an invalid markword for the Helper.class object. >>> >>> + Without the JDK-8250606 patch, we would assert during CDS dump time. >>> >>> I also removed the assert in HeapShared::oop_hash() since it's >>> irrelevant after JDK-8250606. >>> >>> I intend to push my patch after JDK-8250606 is integrated. >>> >>> >>> ------- >>> >>> More comments below >>> >>> On 8/3/20 6:13 PM, David Holmes wrote: >>>> Hi Ioi, >>>> >>>> On 27/07/2020 3:20 pm, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8249276 >>>>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v01/ >>>>> >>>>> >>>>> Please review this change (initial patch provided by David Holmes; >>>>> I rearranged it a little and added more comments). >>>>> >>>>> We reinitialize the markWord of all archived objects to remove any >>>>> side effect of GC/locking/etc during "java -Xshare:dump". >>>> >>>> Changes look good to me (naturally :) ). A couple of nits: >>>> >>>> ?? int hash_archived = archived_oop->identity_hash(); >>>> ?? assert(hash_original == hash_archived, "Different hash codes: >>>> original %x, archived %x", hash_original, hash_archived); >>>> >>>> the has_archived setting should be in ifdef DEBUG. >>>> >>> >>> Fixed >>> >>> >>>> 42 Object.class.wait(); >>>> ? 43???????????????? } >>>> ? 44???????????????? System.out.println("Huh?? notified??"); >>>> >>>> That could be a spurious wakeup. You could just use Thread.sleep >>>> with a suitably long sleep time. that said I'm not sure how the test >>>> is intended to operate. If premain returns after starting the thread >>>> there is no guarantee the thread will start executing and lock the >>>> object before the dump actually commences. Even the sighting of >>>> "Let's wait ....." is not sufficient to guarantee that unless you >>>> move it inside the sync block. >>>> >>> >>> I have fixed the test case so that the "Let's hold the lock" message >>> is printed after the lock is held, and the lock is never released >>> afterwards. >>> >>> I also added synchronization to make sure the premain() method >>> doesn't return until the child thread has held the lock. The log file >>> shows that theses messages are printed before CDS tries to load >>> classes from the classlist: >>> >>> [0.005s][info][cds] Allocated shared space: 3221225472 bytes at >>> 0x0000000800000000 >>> inside LockDuringDumpAgent: >>> jdk.internal.loader.ClassLoaders$AppClassLoader at 707f7052 >>> Helper class is initialized >>> Class = class LockDuringDumpApp$Helper >>> Let's hold the lock on Helper.class forever ..... >>> Thread has started >>> [0.145s][info][cds] Loading classes to share ... >>> >>>>> (David mentioned in the bug comments about assertions inside the >>>>> identity_hash() call, but maybe this should be fixed in a different >>>>> bug?) >>>> >>>> Yes those assertions are being removed under: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8250606 >>>> >>> >>> Thanks >>> - Ioi >>> >>> ------- >>> >>> [1] >>> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/040985.html >>> >>> >>> > From david.holmes at oracle.com Tue Aug 11 09:39:07 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Aug 2020 19:39:07 +1000 Subject: RFR (s) 8251383: Disable Event::log from linux_mprotect when processing the assertion poison page In-Reply-To: References: <4f20d795-fea6-282b-cc96-2a91e0665a53@oracle.com> Message-ID: <78b26855-cd57-8ca2-ecf7-342cd3c6571c@oracle.com> Hi Thomas, On 11/08/2020 7:28 pm, Thomas St?fe wrote: > Hi David, > > ugh. Sorry for the complexity. Our compiler guys really liked this > feature, but I still have the nagging feeling this was not my best > development ever (or, more precisely, that the added complexity may not > be worth the information). I had a painful time figuring out what was going wrong as I was looking for a problem with my assert condition, not something in the assertion code itself. That said I didn't pay enough attention to the gdb stack either so mea culpa. :( > As for the patch, you should use?CAN_SHOW_REGISTERS_ON_ASSERT instead of > ASSERT/DEBUG since this may break zero. See debug.hpp. Ah right. That is a bit uglier. webrev updated in place. Thanks, David ----- > Cheers, Thomas > > > On Tue, Aug 11, 2020 at 11:17 AM David Holmes > wrote: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8251383 > webrev: http://cr.openjdk.java.net/~dholmes/8251383/webrev/ > > When the assertion poison page is enabled (Linux only and on by > default) > the signal handler will call os::protect_memory to change the page > protection bits. That will call linux_mprotect which will call > > Events::log(NULL, "Protecting memory [" INTPTR_FORMAT "," INTPTR_FORMAT > "] with protection modes %x", p2i(bottom), p2i(bottom+size), prot); > > Event logging in turn can use Mutexes and other VM facilities - all of > which are now being executed in a signal handling context (which is > inherently unsafe). It also means that there cannot be any other > failing > assertions on that path as you will re-trigger the poison page > pagefault > and abort with no hs_err file. Hence, as happened to me, a failing > assertion in the mutex code triggers this problem. > > The issue can be worked-around by setting -XX:-ShowRegistersOnAssert > (once you realise what is happening). > > The simple fix is to skip the logging if the faulting address is the > poison page address. > > This only affects debug builds of course. > > Testing: > ? ?- runtime/ErrorHandling > ? ?- tier 1-3 > > Thanks, > David > From thomas.stuefe at gmail.com Tue Aug 11 09:52:38 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 11 Aug 2020 11:52:38 +0200 Subject: RFR (s) 8251383: Disable Event::log from linux_mprotect when processing the assertion poison page In-Reply-To: <78b26855-cd57-8ca2-ecf7-342cd3c6571c@oracle.com> References: <4f20d795-fea6-282b-cc96-2a91e0665a53@oracle.com> <78b26855-cd57-8ca2-ecf7-342cd3c6571c@oracle.com> Message-ID: Looks good now. Thanks for fixing. ...Thomas On Tue, Aug 11, 2020 at 11:41 AM David Holmes wrote: > Hi Thomas, > > On 11/08/2020 7:28 pm, Thomas St?fe wrote: > > Hi David, > > > > ugh. Sorry for the complexity. Our compiler guys really liked this > > feature, but I still have the nagging feeling this was not my best > > development ever (or, more precisely, that the added complexity may not > > be worth the information). > > I had a painful time figuring out what was going wrong as I was looking > for a problem with my assert condition, not something in the assertion > code itself. That said I didn't pay enough attention to the gdb stack > either so mea culpa. :( > > > As for the patch, you should use CAN_SHOW_REGISTERS_ON_ASSERT instead of > > ASSERT/DEBUG since this may break zero. See debug.hpp. > > Ah right. That is a bit uglier. webrev updated in place. > > Thanks, > David > ----- > > > Cheers, Thomas > > > > > > On Tue, Aug 11, 2020 at 11:17 AM David Holmes > > wrote: > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8251383 > > webrev: http://cr.openjdk.java.net/~dholmes/8251383/webrev/ > > > > When the assertion poison page is enabled (Linux only and on by > > default) > > the signal handler will call os::protect_memory to change the page > > protection bits. That will call linux_mprotect which will call > > > > Events::log(NULL, "Protecting memory [" INTPTR_FORMAT "," > INTPTR_FORMAT > > "] with protection modes %x", p2i(bottom), p2i(bottom+size), prot); > > > > Event logging in turn can use Mutexes and other VM facilities - all > of > > which are now being executed in a signal handling context (which is > > inherently unsafe). It also means that there cannot be any other > > failing > > assertions on that path as you will re-trigger the poison page > > pagefault > > and abort with no hs_err file. Hence, as happened to me, a failing > > assertion in the mutex code triggers this problem. > > > > The issue can be worked-around by setting -XX:-ShowRegistersOnAssert > > (once you realise what is happening). > > > > The simple fix is to skip the logging if the faulting address is the > > poison page address. > > > > This only affects debug builds of course. > > > > Testing: > > - runtime/ErrorHandling > > - tier 1-3 > > > > Thanks, > > David > > > From david.holmes at oracle.com Tue Aug 11 10:30:31 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Aug 2020 20:30:31 +1000 Subject: RFR (s) 8251383: Disable Event::log from linux_mprotect when processing the assertion poison page In-Reply-To: References: <4f20d795-fea6-282b-cc96-2a91e0665a53@oracle.com> <78b26855-cd57-8ca2-ecf7-342cd3c6571c@oracle.com> Message-ID: <2c5c83ae-d83a-61b3-dbdf-fdb55ea85cfa@oracle.com> Thanks for the review! David On 11/08/2020 7:52 pm, Thomas St?fe wrote: > Looks good now. Thanks for fixing. > > ...Thomas > > On Tue, Aug 11, 2020 at 11:41 AM David Holmes > wrote: > > Hi Thomas, > > On 11/08/2020 7:28 pm, Thomas St?fe wrote: > > Hi David, > > > > ugh. Sorry for the complexity. Our compiler guys really liked this > > feature, but I still have the nagging feeling this was not my best > > development ever (or, more precisely, that the added complexity > may not > > be worth the information). > > I had a painful time figuring out what was going wrong as I was looking > for a problem with my assert condition, not something in the assertion > code itself. That said I didn't pay enough attention to the gdb stack > either so mea culpa. :( > > > As for the patch, you should use?CAN_SHOW_REGISTERS_ON_ASSERT > instead of > > ASSERT/DEBUG since this may break zero. See debug.hpp. > > Ah right. That is a bit uglier. webrev updated in place. > > Thanks, > David > ----- > > > Cheers, Thomas > > > > > > On Tue, Aug 11, 2020 at 11:17 AM David Holmes > > > >> wrote: > > > >? ? ?Bug: https://bugs.openjdk.java.net/browse/JDK-8251383 > >? ? ?webrev: http://cr.openjdk.java.net/~dholmes/8251383/webrev/ > > > >? ? ?When the assertion poison page is enabled (Linux only and on by > >? ? ?default) > >? ? ?the signal handler will call os::protect_memory to change the > page > >? ? ?protection bits. That will call linux_mprotect which will call > > > >? ? ?Events::log(NULL, "Protecting memory [" INTPTR_FORMAT "," > INTPTR_FORMAT > >? ? ?"] with protection modes %x", p2i(bottom), p2i(bottom+size), > prot); > > > >? ? ?Event logging in turn can use Mutexes and other VM facilities > - all of > >? ? ?which are now being executed in a signal handling context > (which is > >? ? ?inherently unsafe). It also means that there cannot be any other > >? ? ?failing > >? ? ?assertions on that path as you will re-trigger the poison page > >? ? ?pagefault > >? ? ?and abort with no hs_err file. Hence, as happened to me, a > failing > >? ? ?assertion in the mutex code triggers this problem. > > > >? ? ?The issue can be worked-around by setting > -XX:-ShowRegistersOnAssert > >? ? ?(once you realise what is happening). > > > >? ? ?The simple fix is to skip the logging if the faulting address > is the > >? ? ?poison page address. > > > >? ? ?This only affects debug builds of course. > > > >? ? ?Testing: > >? ? ? ? ?- runtime/ErrorHandling > >? ? ? ? ?- tier 1-3 > > > >? ? ?Thanks, > >? ? ?David > > > From david.holmes at oracle.com Tue Aug 11 10:32:43 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 11 Aug 2020 20:32:43 +1000 Subject: RFR: 8251365: Build failure on AIX after 8250636 In-Reply-To: References: <48c0d4f2-ffa7-a242-bb5a-f53781eaa5ff@azul.com> Message-ID: <3804285b-6188-9af6-a420-d09a0dd9c258@oracle.com> Hi Dmitry, This needed a second review before pushing. At a minimum Martin needed a chance to verify it on AIX. David On 11/08/2020 11:54 am, David Holmes wrote: > Hi Dmitry, > > I didn't see this when replying to the other thread. > > On 11/08/2020 6:27 am, Dmitry Cherepanov wrote: >> Please review the fix for build failure on AIX. With this change, >> "tm_gmtoff" is used only if _ALLBSD_SOURCE or _GNU_SOURCE is set. In >> the case of AIX, we'll continue to use "timezone". >> >> I'll start jdk-submit testing for this change shortly, unfortunately I >> do not have an AIX machine at hand and would appreciate if someone >> (with access to AIX) could verify that the change fixes AIX build . >> Thanks in advance. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8251365 >> Webrev: http://cr.openjdk.java.net/~dcherepanov/8251365/webrev.v1/ > > This seems okay to me. All the ifdefs are a bit jarring (makes it seem > like there should be a pd_xxx method underpinning this) but acceptable. > > Thanks, > David > >> Thanks, >> >> Dmitry >> From dcherepanov at azul.com Tue Aug 11 11:06:11 2020 From: dcherepanov at azul.com (Dmitry Cherepanov) Date: Tue, 11 Aug 2020 14:06:11 +0300 Subject: RFR: 8251365: Build failure on AIX after 8250636 In-Reply-To: <3804285b-6188-9af6-a420-d09a0dd9c258@oracle.com> References: <48c0d4f2-ffa7-a242-bb5a-f53781eaa5ff@azul.com> <3804285b-6188-9af6-a420-d09a0dd9c258@oracle.com> Message-ID: <849bd937-ae60-1fa0-2756-9733155cac1b@azul.com> Hi David, I'm sorry for pushing this without second review. I will keep that in mind in the future. In the bug report (https://bugs.openjdk.java.net/browse/JDK-8251365) Martin confirmed that AIX now builds with this change. Thanks, Dmitry On 11.08.2020 13:32, David Holmes wrote: > Hi Dmitry, > > This needed a second review before pushing. At a minimum Martin needed a chance to verify it on AIX. > > David > > On 11/08/2020 11:54 am, David Holmes wrote: >> Hi Dmitry, >> >> I didn't see this when replying to the other thread. >> >> On 11/08/2020 6:27 am, Dmitry Cherepanov wrote: >>> Please review the fix for build failure on AIX. With this change, "tm_gmtoff" is used only if _ALLBSD_SOURCE or _GNU_SOURCE is set. In the case of AIX, we'll continue to use "timezone". >>> >>> I'll start jdk-submit testing for this change shortly, unfortunately I do not have an AIX machine at hand and would appreciate if someone (with access to AIX) could verify that the change fixes AIX build . Thanks in advance. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251365 >>> Webrev: http://cr.openjdk.java.net/~dcherepanov/8251365/webrev.v1/ >> >> This seems okay to me. All the ifdefs are a bit jarring (makes it seem like there should be a pd_xxx method underpinning this) but acceptable. >> >> Thanks, >> David >> >>> Thanks, >>> >>> Dmitry >>> From martin.doerr at sap.com Tue Aug 11 12:11:36 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 11 Aug 2020 12:11:36 +0000 Subject: RFR: 8251365: Build failure on AIX after 8250636 In-Reply-To: <849bd937-ae60-1fa0-2756-9733155cac1b@azul.com> References: <48c0d4f2-ffa7-a242-bb5a-f53781eaa5ff@azul.com> <3804285b-6188-9af6-a420-d09a0dd9c258@oracle.com> <849bd937-ae60-1fa0-2756-9733155cac1b@azul.com> Message-ID: Hi Dmitry, I have a post submission review: Looks good to me. Thanks for fixing it quickly. Best regards, Martin > -----Original Message----- > From: Dmitry Cherepanov > Sent: Dienstag, 11. August 2020 13:06 > To: David Holmes ; hotspot-runtime- > dev at openjdk.java.net; Doerr, Martin > Subject: Re: RFR: 8251365: Build failure on AIX after 8250636 > > Hi David, > > I'm sorry for pushing this without second review. I will keep that in mind in > the future. > In the bug report (https://bugs.openjdk.java.net/browse/JDK-8251365) > Martin confirmed that AIX now builds with this change. > > Thanks, > > Dmitry > > On 11.08.2020 13:32, David Holmes wrote: > > Hi Dmitry, > > > > This needed a second review before pushing. At a minimum Martin > needed a chance to verify it on AIX. > > > > David > > > > On 11/08/2020 11:54 am, David Holmes wrote: > >> Hi Dmitry, > >> > >> I didn't see this when replying to the other thread. > >> > >> On 11/08/2020 6:27 am, Dmitry Cherepanov wrote: > >>> Please review the fix for build failure on AIX. With this change, > "tm_gmtoff" is used only if _ALLBSD_SOURCE or _GNU_SOURCE is set. In the > case of AIX, we'll continue to use "timezone". > >>> > >>> I'll start jdk-submit testing for this change shortly, unfortunately I do not > have an AIX machine at hand and would appreciate if someone (with access > to AIX) could verify that the change fixes AIX build . Thanks in advance. > >>> > >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251365 > >>> Webrev: > http://cr.openjdk.java.net/~dcherepanov/8251365/webrev.v1/ > >> > >> This seems okay to me. All the ifdefs are a bit jarring (makes it seem like > there should be a pd_xxx method underpinning this) but acceptable. > >> > >> Thanks, > >> David > >> > >>> Thanks, > >>> > >>> Dmitry > >>> From yumin.qi at oracle.com Tue Aug 11 14:44:34 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 11 Aug 2020 07:44:34 -0700 Subject: RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS Message-ID: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> Hi, Please reivew ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ ? Summary: CDS does not archive pre-generated lambda form classes for method handles: [0.142s][info][class,load] java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: __JVM_LookupDefineClass__ The method handle resolution if not found in the holder class, a class with the method will be generated and loaded as vm internal created class and not archived like above. Those class names are mangled as combination of the class name and the class instance address. In this patch, collect those method holder class names and their associated methods' signatures when user specify DumpLoadedClassList, and record them in the log file in a format similar to the output format from traced by -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use another name for CDS: -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The line prefix also changed from "[LF_RESOLVE]" to "@lambda-invoke-handle".? At dump time, regenerate the holder class with those methods and replace the existing holder class and archived it. At runtime, the resolution of the holder class which contains those methods are loaded from CDS archive so avoid regenerating them again. The patch reorganized the code for Jlink plugin, so the new added InvokerBytecodeGeneratorHelper can be shared both by JLink plugin and CDS code. Also change made to the mangled generated class name at static dump, the class name combining the class name and a sequentially ordered number instead of the class instance address, which looks like a random. To use this feature, one can do the dump/run just like done for a static dump/run, so no extra steps required. ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp ????? DumpLoadedClassList will turn on -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist -XX:SharedArchiveFile=my.jsa JavaApp ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp ? The performance on javac HelloWorld.java (javac-benchmark), no patch vs patch: ?? 1:?? 2689285002? 2641821474 (-47463528)????? ----???? 391.720 382.990 ( -8.730)????? ---- ?? 2:?? 2687495085? 2632969688 (-54525397)????? ----???? 391.030 381.480 ( -9.550)????? ----- ?? 3:?? 2694664066? 2636523114 (-58140952)????? -----??? 390.610 382.550 ( -8.060)????? ---- ?? 4:?? 2686554164? 2639355233 (-47198931)????? ----???? 390.700 383.390 ( -7.310)????? --- ?? 5:?? 2691072338? 2633016687 (-58055651)????? -----??? 388.990 382.360 ( -6.630)????? --- ?? 6:?? 2684448174? 2644191854 (-40256320)????? ---????? 389.450 382.990 ( -6.460)????? --- ?? 7:?? 2694921227? 2630505090 (-64416137)????? -----??? 389.300 383.160 ( -6.140)????? --- ?? 8:?? 2685209712? 2639334320 (-45875392)????? ----???? 388.370 381.060 ( -7.310)????? --- ?? 9:?? 2695885942? 2640618655 (-55267287)????? ----???? 389.560 381.100 ( -8.460)????? ---- ? 10:?? 2689162942? 2635658943 (-53503999)????? ----???? 389.690 379.110 (-10.580)????? ----- ============================================================ ??????? 2689866989? 2637396210 (-52470778)????? ----???? 389.941 382.017 ( -7.924)????? ---- instr delta =??? -52470778??? -1.9507% time? delta =?????? -7.924 ms -2.0321% ? It will show much more improvement if compare with the default archive: ?????????? ? ? ? no patch??????????????????????? patch instr ??? ? ? 3,996,847,605??????? 3,320,928,995 time(s).???? 0.686835162 ? ? ? ?? 0.474933546 ? Tests: 1)jtreg on jdk/tools and hotspot/runtime local. 2) mach5 ter1,2 Thanks Yumin ? ` From gerard.ziemski at oracle.com Tue Aug 11 14:48:10 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Tue, 11 Aug 2020 09:48:10 -0500 Subject: RFR (S) 8237591: Mac: include OS X version in hs_err_pid crash log file In-Reply-To: References: <49AB6201-C2C2-4862-A019-B60EEE44E515@me.com> <74c08f37-673c-84ae-a512-6f5afbe08050@oracle.com> <70bb6f74-e626-bd54-ddf0-568bebe933e9@oracle.com> <56bb935f-1277-e97e-aeb2-3f488bc825a7@oracle.com> Message-ID: <8bc72743-c173-b1a8-c027-2488b72deaa7@oracle.com> ping Can I please have a 2nd review? David Holmes already reviewed it. bug link at https://bugs.openjdk.java.net/browse/JDK-8237591 open webrev at http://cr.openjdk.java.net/~gziemski/8237591_rev1 testing passes Mach5 hs_tier1,2,3,4,5 cheers On 7/30/20 1:10 AM, David Holmes wrote: > On 30/07/2020 4:51 am, gerard ziemski wrote: >> On 7/27/20 6:21 PM, David Holmes wrote: >>> On 28/07/2020 2:12 am, gerard ziemski wrote: >>>> Thank you David for taking a look. >>>> >>>> >>>> On 7/19/20 11:37 PM, David Holmes wrote: >>>>> Hi Gerard, >>>>> >>>>> On 18/07/2020 5:19 am, gerard ziemski wrote: >>>>>> Hi all, >>>>>> >>>>>> Please review this small fix that adds the OS version and the OS >>>>>> build number to the hs_err_pidXXX.log output in the ?Summary? >>>>>> section for Mac platform (it?s easier to use for developers than >>>>>> the Darwin kernel version that we display right now). >>>>>> >>>>>> This is how things used to look: >>>>>> >>>>>> >>>>>> --------------- S U M M A R Y ------------ >>>>>> >>>>>> Command Line: Crasher >>>>>> >>>>>> Host: Gerards-MBP-16, MacBookPro16,1 x86_64 2600 MHz, 12 cores, >>>>>> 32G, Darwin 19.5.0 >>>>>> Time: Thu Jul 16 14:01:46 2020 CDT elapsed time: 1.089465 seconds >>>>>> (0d 0h 0m 1s) >>>>>> >>>>>> >>>>>> And this is how the ?Summary? section looks like with the >>>>>> proposed change: >>>>>> >>>>>> >>>>>> --------------- S U M M A R Y ------------ >>>>>> >>>>>> Command Line: Crasher >>>>>> >>>>>> Host: Gerards-MBP-16, MacBookPro16,1 x86_64 2600 MHz, 12 cores, >>>>>> 32G, Darwin 19.5.0, macOS 10.15.5 (19F101) >>>>>> Time: Thu Jul 16 14:02:29 2020 CDT elapsed time: 0.360881 seconds >>>>>> (0d 0h 0m 0s) >>>>>> >>>>>> >>>>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8237591 >>>>>> open webrev at http://cr.openjdk.java.net/~gziemski/8237591_rev1 >>>>>> testing Mach5 hs_tier1,2,3,4,5 in progress >>>>> >>>>> Just to be clear, the changes prior to: >>>>> >>>>> 1555 #ifdef __APPLE__ >>>>> >>>>> are just fixing up existing indentation errors - correct? >>>> >>>> Yes, hope that's OK, as this was the only spot in the function that >>>> stood out with inconsistent indentation. >>> >>> Yes that is fine. >>> >>>>> >>>>> The actual change seems okay, just one query: >>>>> >>>>> 1562???? int mib_build[] = { CTL_KERN, KERN_OSVERSION }; >>>>> >>>>> I couldn't find KERN_OSVERSION documented for sysctl - is it a >>>>> "recent" addition? >>>> >>>> Yes it is. Apple added it back in 2018 (see bug comments or this >>>> link >>>> https://github.com/apple/darwin-xnu/commit/5bbb823c13f3ab1ab58878f96b35433a29882676?diff=split#diff-6651b0c84a045f400bc45faa9f61c9e1 >>>> ) >>> >>> That link shows the addition of sysctl_osproductversion which I >>> assume underpins "kern.osproductversion". But my question was on >>> KERN_OSVERSION. That definition seems to already exist prior to the >>> change you link. My concern is whether it was also fairly recently >>> introduced and so referring to it would require a minimum macOS >>> version on the build machine? >> >> Sorry, I thought you meant "kern.osproductversion", not >> KERN_OSVERSION, but that's a valid question. >> >> I found Apple using KERN_OSVERSION in its own code since macOS 10.7, >> i.e. >> https://opensource.apple.com/source/Libc/Libc-763.11/gen/assumes.c.auto.html >> , though I could not find any documentation of it either. > > I found it in sysctl.h from 10.5 dev kit as well, so that looks fine > to use. > > Thanks for checking. > David > ----- > >> >> cheers >> From coleen.phillimore at oracle.com Tue Aug 11 16:09:27 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 11 Aug 2020 12:09:27 -0400 Subject: RFR(S) 8249276 CDS archived objects must have "neutral" markwords In-Reply-To: References: <5d5b4b17-a40f-b0f4-fa50-fdd783def9d4@oracle.com> <37245599-dfb8-0481-6c74-89cffdc5b457@oracle.com> <7a81547c-8221-da80-2d1a-edb42f4078fc@oracle.com> <11f16292-909d-6d61-d353-cfa9d7054506@oracle.com> Message-ID: http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03/test/hotspot/jtreg/runtime/cds/appcds/javaldr/LockDuringDumpAgent.mf.html This doesn't have a copyright.? There are tests like test/hotspot/jtreg/serviceability/jvmti/GetObjectSizeClass.java that create the manifest file inside the .java source code so you don't need this file, which I think you can do here: http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03/test/hotspot/jtreg/runtime/cds/appcds/javaldr/LockDuringDump.java.html Coleen On 8/11/20 1:29 AM, Ioi Lam wrote: > > > On 8/6/20 5:55 PM, David Holmes wrote: >> Hi Ioi, >> >> On 6/08/2020 8:26 am, Ioi Lam wrote: >>> Hi David & Coleen, >>> >>> Thanks for the review. I have created a new version: >>> >>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ >>> >> >> Functional changes are good. >> >> For the test, as per our offline chat, the Helper class is not being >> archived. This line: >> >> ?66???????????????? TestCommon.testDump(appJar, >> TestCommon.list(LockDuringDumpApp.class.getName()), >> >> should be >> >> ?66???????????????? TestCommon.testDump(appJar, >> TestCommon.list(appClasses), >> >> > > Hi David, > > (The Helper class was actually archived, because it was loaded by > LockDuringDumpAgent during dump time, so it was not necessary to > explicitly include it in the class list. ....) > > Anyway, it wasn't clear to me whether any code (now or in the future) > in the JVM might synchronize on the java mirrors. So I've updated the > test case to synchronize on a literal string instead. This way I know > that only the test, and no one else, will synchronize on it. > > I also added comments to explain how the test works. > > http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03/ > > > Thanks > - Ioi > > >>> >>> I created a new test case (LockDuringDump), and incorporated David's >>> fix for JDK-8250606 [1] during my testing: >>> >>> + Without my patch, LockDuringDumpApp would crash at runtime, because >>> ?? we have an invalid markword for the Helper.class object. >>> >>> + Without the JDK-8250606 patch, we would assert during CDS dump time. >>> >>> I also removed the assert in HeapShared::oop_hash() since it's >>> irrelevant after JDK-8250606. >>> >>> I intend to push my patch after JDK-8250606 is integrated. >>> >>> >>> ------- >>> >>> More comments below >>> >>> On 8/3/20 6:13 PM, David Holmes wrote: >>>> Hi Ioi, >>>> >>>> On 27/07/2020 3:20 pm, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8249276 >>>>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v01/ >>>>> >>>>> >>>>> Please review this change (initial patch provided by David Holmes; >>>>> I rearranged it a little and added more comments). >>>>> >>>>> We reinitialize the markWord of all archived objects to remove any >>>>> side effect of GC/locking/etc during "java -Xshare:dump". >>>> >>>> Changes look good to me (naturally :) ). A couple of nits: >>>> >>>> ?? int hash_archived = archived_oop->identity_hash(); >>>> ?? assert(hash_original == hash_archived, "Different hash codes: >>>> original %x, archived %x", hash_original, hash_archived); >>>> >>>> the has_archived setting should be in ifdef DEBUG. >>>> >>> >>> Fixed >>> >>> >>>> 42 Object.class.wait(); >>>> ? 43???????????????? } >>>> ? 44???????????????? System.out.println("Huh?? notified??"); >>>> >>>> That could be a spurious wakeup. You could just use Thread.sleep >>>> with a suitably long sleep time. that said I'm not sure how the >>>> test is intended to operate. If premain returns after starting the >>>> thread there is no guarantee the thread will start executing and >>>> lock the object before the dump actually commences. Even the >>>> sighting of "Let's wait ....." is not sufficient to guarantee that >>>> unless you move it inside the sync block. >>>> >>> >>> I have fixed the test case so that the "Let's hold the lock" message >>> is printed after the lock is held, and the lock is never released >>> afterwards. >>> >>> I also added synchronization to make sure the premain() method >>> doesn't return until the child thread has held the lock. The log >>> file shows that theses messages are printed before CDS tries to load >>> classes from the classlist: >>> >>> [0.005s][info][cds] Allocated shared space: 3221225472 bytes at >>> 0x0000000800000000 >>> inside LockDuringDumpAgent: >>> jdk.internal.loader.ClassLoaders$AppClassLoader at 707f7052 >>> Helper class is initialized >>> Class = class LockDuringDumpApp$Helper >>> Let's hold the lock on Helper.class forever ..... >>> Thread has started >>> [0.145s][info][cds] Loading classes to share ... >>> >>>>> (David mentioned in the bug comments about assertions inside the >>>>> identity_hash() call, but maybe this should be fixed in a >>>>> different bug?) >>>> >>>> Yes those assertions are being removed under: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8250606 >>>> >>> >>> Thanks >>> - Ioi >>> >>> ------- >>> >>> [1] >>> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/040985.html >>> >>> >>> > From daniel.daugherty at oracle.com Tue Aug 11 16:09:52 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 11 Aug 2020 12:09:52 -0400 Subject: RFR (S) 8237591: Mac: include OS X version in hs_err_pid crash log file In-Reply-To: References: <49AB6201-C2C2-4862-A019-B60EEE44E515@me.com> <74c08f37-673c-84ae-a512-6f5afbe08050@oracle.com> <70bb6f74-e626-bd54-ddf0-568bebe933e9@oracle.com> <56bb935f-1277-e97e-aeb2-3f488bc825a7@oracle.com> Message-ID: > open webrev at http://cr.openjdk.java.net/~gziemski/8237591_rev1 src/hotspot/os/bsd/os_bsd.cpp ??? No comments. Thumbs up. Thanks for including the before and after snippets in the review request. Dan On 7/30/20 12:54 PM, gerard ziemski wrote: > Thank you for the review. > > May I please have a 2nd reviewer? (or would this be considered trivial?) > > > cheers > > On 7/30/20 1:10 AM, David Holmes wrote: >> On 30/07/2020 4:51 am, gerard ziemski wrote: >>> On 7/27/20 6:21 PM, David Holmes wrote: >>>> On 28/07/2020 2:12 am, gerard ziemski wrote: >>>>> Thank you David for taking a look. >>>>> >>>>> >>>>> On 7/19/20 11:37 PM, David Holmes wrote: >>>>>> Hi Gerard, >>>>>> >>>>>> On 18/07/2020 5:19 am, gerard ziemski wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> Please review this small fix that adds the OS version and the OS >>>>>>> build number to the hs_err_pidXXX.log output in the ?Summary? >>>>>>> section for Mac platform (it?s easier to use for developers than >>>>>>> the Darwin kernel version that we display right now). >>>>>>> >>>>>>> This is how things used to look: >>>>>>> >>>>>>> >>>>>>> --------------- S U M M A R Y ------------ >>>>>>> >>>>>>> Command Line: Crasher >>>>>>> >>>>>>> Host: Gerards-MBP-16, MacBookPro16,1 x86_64 2600 MHz, 12 cores, >>>>>>> 32G, Darwin 19.5.0 >>>>>>> Time: Thu Jul 16 14:01:46 2020 CDT elapsed time: 1.089465 >>>>>>> seconds (0d 0h 0m 1s) >>>>>>> >>>>>>> >>>>>>> And this is how the ?Summary? section looks like with the >>>>>>> proposed change: >>>>>>> >>>>>>> >>>>>>> --------------- S U M M A R Y ------------ >>>>>>> >>>>>>> Command Line: Crasher >>>>>>> >>>>>>> Host: Gerards-MBP-16, MacBookPro16,1 x86_64 2600 MHz, 12 cores, >>>>>>> 32G, Darwin 19.5.0, macOS 10.15.5 (19F101) >>>>>>> Time: Thu Jul 16 14:02:29 2020 CDT elapsed time: 0.360881 >>>>>>> seconds (0d 0h 0m 0s) >>>>>>> >>>>>>> >>>>>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8237591 >>>>>>> open webrev at http://cr.openjdk.java.net/~gziemski/8237591_rev1 >>>>>>> testing Mach5 hs_tier1,2,3,4,5 in progress >>>>>> >>>>>> Just to be clear, the changes prior to: >>>>>> >>>>>> 1555 #ifdef __APPLE__ >>>>>> >>>>>> are just fixing up existing indentation errors - correct? >>>>> >>>>> Yes, hope that's OK, as this was the only spot in the function >>>>> that stood out with inconsistent indentation. >>>> >>>> Yes that is fine. >>>> >>>>>> >>>>>> The actual change seems okay, just one query: >>>>>> >>>>>> 1562???? int mib_build[] = { CTL_KERN, KERN_OSVERSION }; >>>>>> >>>>>> I couldn't find KERN_OSVERSION documented for sysctl - is it a >>>>>> "recent" addition? >>>>> >>>>> Yes it is. Apple added it back in 2018 (see bug comments or this >>>>> link >>>>> https://github.com/apple/darwin-xnu/commit/5bbb823c13f3ab1ab58878f96b35433a29882676?diff=split#diff-6651b0c84a045f400bc45faa9f61c9e1 >>>>> ) >>>> >>>> That link shows the addition of sysctl_osproductversion which I >>>> assume underpins "kern.osproductversion". But my question was on >>>> KERN_OSVERSION. That definition seems to already exist prior to the >>>> change you link. My concern is whether it was also fairly recently >>>> introduced and so referring to it would require a minimum macOS >>>> version on the build machine? >>> >>> Sorry, I thought you meant "kern.osproductversion", not >>> KERN_OSVERSION, but that's a valid question. >>> >>> I found Apple using KERN_OSVERSION in its own code since macOS 10.7, >>> i.e. >>> https://opensource.apple.com/source/Libc/Libc-763.11/gen/assumes.c.auto.html >>> , though I could not find any documentation of it either. >> >> I found it in sysctl.h from 10.5 dev kit as well, so that looks fine >> to use. >> >> Thanks for checking. >> David >> ----- > From daniel.daugherty at oracle.com Tue Aug 11 16:22:51 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 11 Aug 2020 12:22:51 -0400 Subject: RFR (s) 8251383: Disable Event::log from linux_mprotect when processing the assertion poison page In-Reply-To: <4f20d795-fea6-282b-cc96-2a91e0665a53@oracle.com> References: <4f20d795-fea6-282b-cc96-2a91e0665a53@oracle.com> Message-ID: <780d9243-67d5-d70e-bda3-ace37316eb7f@oracle.com> On 8/11/20 5:17 AM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8251383 > webrev: http://cr.openjdk.java.net/~dholmes/8251383/webrev/ src/hotspot/os/linux/os_linux.cpp ??? No comments. Thumbs up. Dan > > When the assertion poison page is enabled (Linux only and on by > default) the signal handler will call os::protect_memory to change the > page protection bits. That will call linux_mprotect which will call > > Events::log(NULL, "Protecting memory [" INTPTR_FORMAT "," > INTPTR_FORMAT "] with protection modes %x", p2i(bottom), > p2i(bottom+size), prot); > > Event logging in turn can use Mutexes and other VM facilities - all of > which are now being executed in a signal handling context (which is > inherently unsafe). It also means that there cannot be any other > failing assertions on that path as you will re-trigger the poison page > pagefault and abort with no hs_err file. Hence, as happened to me, a > failing assertion in the mutex code triggers this problem. > > The issue can be worked-around by setting -XX:-ShowRegistersOnAssert > (once you realise what is happening). > > The simple fix is to skip the logging if the faulting address is the > poison page address. > > This only affects debug builds of course. > > Testing: > ? - runtime/ErrorHandling > ? - tier 1-3 > > Thanks, > David From ioi.lam at oracle.com Tue Aug 11 16:32:01 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 11 Aug 2020 09:32:01 -0700 Subject: RFR(S) 8249276 CDS archived objects must have "neutral" markwords In-Reply-To: References: <5d5b4b17-a40f-b0f4-fa50-fdd783def9d4@oracle.com> <37245599-dfb8-0481-6c74-89cffdc5b457@oracle.com> <7a81547c-8221-da80-2d1a-edb42f4078fc@oracle.com> <11f16292-909d-6d61-d353-cfa9d7054506@oracle.com> Message-ID: <635d46e6-c91a-bb56-52ea-f3407bb7b4df@oracle.com> Hi Coleen, Thanks for the review. I've changed the code to the following, and removed the LockDuringDumpAgent.mf file +??? private static final String MANIFEST = +??????? "Manifest-Version: 1.0\nPremain-Class: LockDuringDumpAgent\n"; + ???? public static void main(String[] args) throws Throwable { ???????? String agentJar = ClassFileInstaller.writeJar("LockDuringDumpAgent.jar", - ClassFileInstaller.Manifest.fromSourceFile("LockDuringDumpAgent.mf"), + ClassFileInstaller.Manifest.fromString(MANIFEST), ???????????????????????????????????????? agentClasses); Thanks - Ioi On 8/11/20 9:09 AM, Coleen Phillimore wrote: > > http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03/test/hotspot/jtreg/runtime/cds/appcds/javaldr/LockDuringDumpAgent.mf.html > > > This doesn't have a copyright.? There are tests like > test/hotspot/jtreg/serviceability/jvmti/GetObjectSizeClass.java that > create the manifest file inside the .java source code so you don't > need this file, which I think you can do here: > > http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03/test/hotspot/jtreg/runtime/cds/appcds/javaldr/LockDuringDump.java.html > > > Coleen > > On 8/11/20 1:29 AM, Ioi Lam wrote: >> >> >> On 8/6/20 5:55 PM, David Holmes wrote: >>> Hi Ioi, >>> >>> On 6/08/2020 8:26 am, Ioi Lam wrote: >>>> Hi David & Coleen, >>>> >>>> Thanks for the review. I have created a new version: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ >>>> >>> >>> Functional changes are good. >>> >>> For the test, as per our offline chat, the Helper class is not being >>> archived. This line: >>> >>> ?66???????????????? TestCommon.testDump(appJar, >>> TestCommon.list(LockDuringDumpApp.class.getName()), >>> >>> should be >>> >>> ?66???????????????? TestCommon.testDump(appJar, >>> TestCommon.list(appClasses), >>> >>> >> >> Hi David, >> >> (The Helper class was actually archived, because it was loaded by >> LockDuringDumpAgent during dump time, so it was not necessary to >> explicitly include it in the class list. ....) >> >> Anyway, it wasn't clear to me whether any code (now or in the future) >> in the JVM might synchronize on the java mirrors. So I've updated the >> test case to synchronize on a literal string instead. This way I know >> that only the test, and no one else, will synchronize on it. >> >> I also added comments to explain how the test works. >> >> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03/ >> >> >> Thanks >> - Ioi >> >> >>>> >>>> I created a new test case (LockDuringDump), and incorporated >>>> David's fix for JDK-8250606 [1] during my testing: >>>> >>>> + Without my patch, LockDuringDumpApp would crash at runtime, because >>>> ?? we have an invalid markword for the Helper.class object. >>>> >>>> + Without the JDK-8250606 patch, we would assert during CDS dump time. >>>> >>>> I also removed the assert in HeapShared::oop_hash() since it's >>>> irrelevant after JDK-8250606. >>>> >>>> I intend to push my patch after JDK-8250606 is integrated. >>>> >>>> >>>> ------- >>>> >>>> More comments below >>>> >>>> On 8/3/20 6:13 PM, David Holmes wrote: >>>>> Hi Ioi, >>>>> >>>>> On 27/07/2020 3:20 pm, Ioi Lam wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8249276 >>>>>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v01/ >>>>>> >>>>>> >>>>>> Please review this change (initial patch provided by David >>>>>> Holmes; I rearranged it a little and added more comments). >>>>>> >>>>>> We reinitialize the markWord of all archived objects to remove >>>>>> any side effect of GC/locking/etc during "java -Xshare:dump". >>>>> >>>>> Changes look good to me (naturally :) ). A couple of nits: >>>>> >>>>> ?? int hash_archived = archived_oop->identity_hash(); >>>>> ?? assert(hash_original == hash_archived, "Different hash codes: >>>>> original %x, archived %x", hash_original, hash_archived); >>>>> >>>>> the has_archived setting should be in ifdef DEBUG. >>>>> >>>> >>>> Fixed >>>> >>>> >>>>> 42 Object.class.wait(); >>>>> ? 43???????????????? } >>>>> ? 44???????????????? System.out.println("Huh?? notified??"); >>>>> >>>>> That could be a spurious wakeup. You could just use Thread.sleep >>>>> with a suitably long sleep time. that said I'm not sure how the >>>>> test is intended to operate. If premain returns after starting the >>>>> thread there is no guarantee the thread will start executing and >>>>> lock the object before the dump actually commences. Even the >>>>> sighting of "Let's wait ....." is not sufficient to guarantee that >>>>> unless you move it inside the sync block. >>>>> >>>> >>>> I have fixed the test case so that the "Let's hold the lock" >>>> message is printed after the lock is held, and the lock is never >>>> released afterwards. >>>> >>>> I also added synchronization to make sure the premain() method >>>> doesn't return until the child thread has held the lock. The log >>>> file shows that theses messages are printed before CDS tries to >>>> load classes from the classlist: >>>> >>>> [0.005s][info][cds] Allocated shared space: 3221225472 bytes at >>>> 0x0000000800000000 >>>> inside LockDuringDumpAgent: >>>> jdk.internal.loader.ClassLoaders$AppClassLoader at 707f7052 >>>> Helper class is initialized >>>> Class = class LockDuringDumpApp$Helper >>>> Let's hold the lock on Helper.class forever ..... >>>> Thread has started >>>> [0.145s][info][cds] Loading classes to share ... >>>> >>>>>> (David mentioned in the bug comments about assertions inside the >>>>>> identity_hash() call, but maybe this should be fixed in a >>>>>> different bug?) >>>>> >>>>> Yes those assertions are being removed under: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8250606 >>>>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> ------- >>>> >>>> [1] >>>> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/040985.html >>>> >>>> >>>> >> > From coleen.phillimore at oracle.com Tue Aug 11 16:37:39 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 11 Aug 2020 12:37:39 -0400 Subject: RFR(S) 8249276 CDS archived objects must have "neutral" markwords In-Reply-To: <635d46e6-c91a-bb56-52ea-f3407bb7b4df@oracle.com> References: <5d5b4b17-a40f-b0f4-fa50-fdd783def9d4@oracle.com> <37245599-dfb8-0481-6c74-89cffdc5b457@oracle.com> <7a81547c-8221-da80-2d1a-edb42f4078fc@oracle.com> <11f16292-909d-6d61-d353-cfa9d7054506@oracle.com> <635d46e6-c91a-bb56-52ea-f3407bb7b4df@oracle.com> Message-ID: <612bc5cc-e634-b9e8-fd46-5d2c393b41bf@oracle.com> On 8/11/20 12:32 PM, Ioi Lam wrote: > Hi Coleen, > > Thanks for the review. I've changed the code to the following, and > removed the LockDuringDumpAgent.mf file > > +??? private static final String MANIFEST = > +??????? "Manifest-Version: 1.0\nPremain-Class: LockDuringDumpAgent\n"; > + > ???? public static void main(String[] args) throws Throwable { > ???????? String agentJar = > ClassFileInstaller.writeJar("LockDuringDumpAgent.jar", > - ClassFileInstaller.Manifest.fromSourceFile("LockDuringDumpAgent.mf"), > + ClassFileInstaller.Manifest.fromString(MANIFEST), > ???????????????????????????????????????? agentClasses); > > Great!? Thank you.? Looks good. Coleen > Thanks > - Ioi > > On 8/11/20 9:09 AM, Coleen Phillimore wrote: >> >> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03/test/hotspot/jtreg/runtime/cds/appcds/javaldr/LockDuringDumpAgent.mf.html >> >> >> This doesn't have a copyright.? There are tests like >> test/hotspot/jtreg/serviceability/jvmti/GetObjectSizeClass.java that >> create the manifest file inside the .java source code so you don't >> need this file, which I think you can do here: >> >> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03/test/hotspot/jtreg/runtime/cds/appcds/javaldr/LockDuringDump.java.html >> >> >> Coleen >> >> On 8/11/20 1:29 AM, Ioi Lam wrote: >>> >>> >>> On 8/6/20 5:55 PM, David Holmes wrote: >>>> Hi Ioi, >>>> >>>> On 6/08/2020 8:26 am, Ioi Lam wrote: >>>>> Hi David & Coleen, >>>>> >>>>> Thanks for the review. I have created a new version: >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ >>>>> >>>> >>>> Functional changes are good. >>>> >>>> For the test, as per our offline chat, the Helper class is not >>>> being archived. This line: >>>> >>>> ?66???????????????? TestCommon.testDump(appJar, >>>> TestCommon.list(LockDuringDumpApp.class.getName()), >>>> >>>> should be >>>> >>>> ?66???????????????? TestCommon.testDump(appJar, >>>> TestCommon.list(appClasses), >>>> >>>> >>> >>> Hi David, >>> >>> (The Helper class was actually archived, because it was loaded by >>> LockDuringDumpAgent during dump time, so it was not necessary to >>> explicitly include it in the class list. ....) >>> >>> Anyway, it wasn't clear to me whether any code (now or in the >>> future) in the JVM might synchronize on the java mirrors. So I've >>> updated the test case to synchronize on a literal string instead. >>> This way I know that only the test, and no one else, will >>> synchronize on it. >>> >>> I also added comments to explain how the test works. >>> >>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03/ >>> >>> >>> Thanks >>> - Ioi >>> >>> >>>>> >>>>> I created a new test case (LockDuringDump), and incorporated >>>>> David's fix for JDK-8250606 [1] during my testing: >>>>> >>>>> + Without my patch, LockDuringDumpApp would crash at runtime, because >>>>> ?? we have an invalid markword for the Helper.class object. >>>>> >>>>> + Without the JDK-8250606 patch, we would assert during CDS dump >>>>> time. >>>>> >>>>> I also removed the assert in HeapShared::oop_hash() since it's >>>>> irrelevant after JDK-8250606. >>>>> >>>>> I intend to push my patch after JDK-8250606 is integrated. >>>>> >>>>> >>>>> ------- >>>>> >>>>> More comments below >>>>> >>>>> On 8/3/20 6:13 PM, David Holmes wrote: >>>>>> Hi Ioi, >>>>>> >>>>>> On 27/07/2020 3:20 pm, Ioi Lam wrote: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8249276 >>>>>>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v01/ >>>>>>> >>>>>>> >>>>>>> Please review this change (initial patch provided by David >>>>>>> Holmes; I rearranged it a little and added more comments). >>>>>>> >>>>>>> We reinitialize the markWord of all archived objects to remove >>>>>>> any side effect of GC/locking/etc during "java -Xshare:dump". >>>>>> >>>>>> Changes look good to me (naturally :) ). A couple of nits: >>>>>> >>>>>> ?? int hash_archived = archived_oop->identity_hash(); >>>>>> ?? assert(hash_original == hash_archived, "Different hash codes: >>>>>> original %x, archived %x", hash_original, hash_archived); >>>>>> >>>>>> the has_archived setting should be in ifdef DEBUG. >>>>>> >>>>> >>>>> Fixed >>>>> >>>>> >>>>>> 42 Object.class.wait(); >>>>>> ? 43???????????????? } >>>>>> ? 44???????????????? System.out.println("Huh?? notified??"); >>>>>> >>>>>> That could be a spurious wakeup. You could just use Thread.sleep >>>>>> with a suitably long sleep time. that said I'm not sure how the >>>>>> test is intended to operate. If premain returns after starting >>>>>> the thread there is no guarantee the thread will start executing >>>>>> and lock the object before the dump actually commences. Even the >>>>>> sighting of "Let's wait ....." is not sufficient to guarantee >>>>>> that unless you move it inside the sync block. >>>>>> >>>>> >>>>> I have fixed the test case so that the "Let's hold the lock" >>>>> message is printed after the lock is held, and the lock is never >>>>> released afterwards. >>>>> >>>>> I also added synchronization to make sure the premain() method >>>>> doesn't return until the child thread has held the lock. The log >>>>> file shows that theses messages are printed before CDS tries to >>>>> load classes from the classlist: >>>>> >>>>> [0.005s][info][cds] Allocated shared space: 3221225472 bytes at >>>>> 0x0000000800000000 >>>>> inside LockDuringDumpAgent: >>>>> jdk.internal.loader.ClassLoaders$AppClassLoader at 707f7052 >>>>> Helper class is initialized >>>>> Class = class LockDuringDumpApp$Helper >>>>> Let's hold the lock on Helper.class forever ..... >>>>> Thread has started >>>>> [0.145s][info][cds] Loading classes to share ... >>>>> >>>>>>> (David mentioned in the bug comments about assertions inside the >>>>>>> identity_hash() call, but maybe this should be fixed in a >>>>>>> different bug?) >>>>>> >>>>>> Yes those assertions are being removed under: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8250606 >>>>>> >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> ------- >>>>> >>>>> [1] >>>>> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/040985.html >>>>> >>>>> >>>>> >>> >> > From xxinliu at amazon.com Tue Aug 11 17:09:11 2020 From: xxinliu at amazon.com (Liu, Xin) Date: Tue, 11 Aug 2020 17:09:11 +0000 Subject: RFR(S): 8247732: validate user-input intrinsic_ids in ControlIntrinsic In-Reply-To: <1596523192072.15354@amazon.com> References: <821e3d29-c95b-aafc-8ee5-6e49a1bdde82@amazon.com> <9b324805-eb86-27e1-5dcb-96a823f8495b@amazon.com> <82cba5e4-2020-ce0a-4576-e8e0cc2e5ae5@oracle.com> <1595401959932.33284@amazon.com> <1595520162373.22868@amazon.com>, <916b3a4a-5617-941d-6161-840f3ea900bd@oracle.com>, <1596523192072.15354@amazon.com> Message-ID: <1597165750921.4285@amazon.com> Hi, Reviewers, May I gently ping this? I stuck because I don't know which error handling is appropriate. If we do nothing, current hotspot ignores wrong intrinsic Ids in the cmdline. This patch aborts hotspot when it detects any invalid intrinsic id. thanks, --lx ________________________________________ From: Liu, Xin Sent: Monday, August 3, 2020 11:39 PM To: Tobias Hartmann; Nils Eliasson; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev Subject: Re: [EXTERNAL] RFR(S): 8247732: validate user-input intrinsic_ids in ControlIntrinsic hi, Nils, Tobias would like to keep the parser behavior consistency. I think it means that the hotspot need to suppress the warning if the intrinsic_id doesn't exists in compiler directive. eg. -XX:CompileCommand=option,,ControlIntrinsic=-_nonexist. What do you think about it? Here is the latest webrev: http://cr.openjdk.java.net/~xliu/8247732/01/webrev/ thanks, --lx ________________________________________ From: Tobias Hartmann Sent: Friday, July 24, 2020 2:52 AM To: Liu, Xin; Nils Eliasson; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev Subject: RE: [EXTERNAL] RFR(S): 8247732: validate user-input intrinsic_ids in ControlIntrinsic CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hi Liu, On 23.07.20 18:02, Liu, Xin wrote: > That is my intention too, but CompilerOracle doesn't exit JVM when it encounters parsing errors. > It just exacts information from CompileCommand as many as possible. That makes sense because compiler "directives" are supposed to be optional for program execution. > > I do put the error message in parser's errorbuf. I set a flag "exit_on_error" to quit JVM after it dumps parser errors. yes, I treat undefined intrinsics as fatal errors. > This behavior is from Nils comment: "I want to see an error on startup if the user has specified unknown intrinsic names." It is also consistent with JVM option -XX:ControlIntrinsic=. Okay, thanks for the explanation! I would prefer consistency in error handling of compiler directives, i.e., handle all parser failures the same way. But I leave it to Nils to decide. Best regards, Tobias From yumin.qi at oracle.com Tue Aug 11 17:36:27 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 11 Aug 2020 10:36:27 -0700 Subject: Fwd: RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS In-Reply-To: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> Message-ID: <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> Forget to send to @core-lib-dev, the patch changed jdk code. Thanks Yumin -------- Forwarded Message -------- Subject: RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS Date: Tue, 11 Aug 2020 07:44:34 -0700 From: Yumin Qi To: hotspot-runtime-dev at openjdk.java.net Hi, Please reivew ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ ? Summary: CDS does not archive pre-generated lambda form classes for method handles: [0.142s][info][class,load] java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: __JVM_LookupDefineClass__ The method handle resolution if not found in the holder class, a class with the method will be generated and loaded as vm internal created class and not archived like above. Those class names are mangled as combination of the class name and the class instance address. In this patch, collect those method holder class names and their associated methods' signatures when user specify DumpLoadedClassList, and record them in the log file in a format similar to the output format from traced by -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use another name for CDS: -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The line prefix also changed from "[LF_RESOLVE]" to "@lambda-invoke-handle".? At dump time, regenerate the holder class with those methods and replace the existing holder class and archived it. At runtime, the resolution of the holder class which contains those methods are loaded from CDS archive so avoid regenerating them again. The patch reorganized the code for Jlink plugin, so the new added InvokerBytecodeGeneratorHelper can be shared both by JLink plugin and CDS code. Also change made to the mangled generated class name at static dump, the class name combining the class name and a sequentially ordered number instead of the class instance address, which looks like a random. To use this feature, one can do the dump/run just like done for a static dump/run, so no extra steps required. ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp ????? DumpLoadedClassList will turn on -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist -XX:SharedArchiveFile=my.jsa JavaApp ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp ? The performance on javac HelloWorld.java (javac-benchmark), no patch vs patch: ?? 1:?? 2689285002? 2641821474 (-47463528)????? ----???? 391.720 382.990 ( -8.730)????? ---- ?? 2:?? 2687495085? 2632969688 (-54525397)????? ----???? 391.030 381.480 ( -9.550)????? ----- ?? 3:?? 2694664066? 2636523114 (-58140952)????? -----??? 390.610 382.550 ( -8.060)????? ---- ?? 4:?? 2686554164? 2639355233 (-47198931)????? ----???? 390.700 383.390 ( -7.310)????? --- ?? 5:?? 2691072338? 2633016687 (-58055651)????? -----??? 388.990 382.360 ( -6.630)????? --- ?? 6:?? 2684448174? 2644191854 (-40256320)????? ---????? 389.450 382.990 ( -6.460)????? --- ?? 7:?? 2694921227? 2630505090 (-64416137)????? -----??? 389.300 383.160 ( -6.140)????? --- ?? 8:?? 2685209712? 2639334320 (-45875392)????? ----???? 388.370 381.060 ( -7.310)????? --- ?? 9:?? 2695885942? 2640618655 (-55267287)????? ----???? 389.560 381.100 ( -8.460)????? ---- ? 10:?? 2689162942? 2635658943 (-53503999)????? ----???? 389.690 379.110 (-10.580)????? ----- ============================================================ ??????? 2689866989? 2637396210 (-52470778)????? ----???? 389.941 382.017 ( -7.924)????? ---- instr delta =??? -52470778??? -1.9507% time? delta =?????? -7.924 ms -2.0321% ? It will show much more improvement if compare with the default archive: ?????????? ? ? ? no patch??????????????????????? patch instr ??? ? ? 3,996,847,605??????? 3,320,928,995 time(s).???? 0.686835162 ? ? ? ?? 0.474933546 ? Tests: 1)jtreg on jdk/tools and hotspot/runtime local. 2) mach5 ter1,2 Thanks Yumin ? ` From patricio.chilano.mateo at oracle.com Tue Aug 11 18:11:48 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Tue, 11 Aug 2020 15:11:48 -0300 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark Message-ID: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> Hi all, Please review the following small fix which involves removing the added HandleMark in BiasedLocking::preserve_marks(): Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ I've inspected the callers of BiasedLocking::preserve_marks() and they all have an assert that the current thread is the VMThread. Since the VMThread creates a HandleMark object before executing a VM operation the extra HandleMark added in 8249192 is not needed. I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without the patch I get several crashes in BiasedLocking::restore_marks(). With the patch tests completed successfully. Thanks, Patricio From harold.seigel at oracle.com Tue Aug 11 19:11:21 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 11 Aug 2020 15:11:21 -0400 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> References: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> Message-ID: <7bcc8162-cf8b-b20e-d3ea-9b90ab5d196a@oracle.com> Hi Patricio, The fix looks good.? Could you also add a test based on the failure discussed in the bug description? Thanks, Harold On 8/11/2020 2:11 PM, Patricio Chilano wrote: > Hi all, > > Please review the following small fix which involves removing the > added HandleMark in BiasedLocking::preserve_marks(): > > Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 > Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ > > I've inspected the callers of BiasedLocking::preserve_marks() and they > all have an assert that the current thread is the VMThread. Since the > VMThread creates a HandleMark object before executing a VM operation > the extra HandleMark added in 8249192 is not needed. > I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without the > patch I get several crashes in BiasedLocking::restore_marks(). With > the patch tests completed successfully. > > Thanks, > Patricio > From coleen.phillimore at oracle.com Tue Aug 11 19:27:44 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 11 Aug 2020 15:27:44 -0400 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: <7bcc8162-cf8b-b20e-d3ea-9b90ab5d196a@oracle.com> References: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> <7bcc8162-cf8b-b20e-d3ea-9b90ab5d196a@oracle.com> Message-ID: On 8/11/20 3:11 PM, Harold Seigel wrote: > Hi Patricio, > > The fix looks good.? Could you also add a test based on the failure > discussed in the bug description? +1.? Thank you for careful examination of the callers. Coleen > > Thanks, Harold > > On 8/11/2020 2:11 PM, Patricio Chilano wrote: >> Hi all, >> >> Please review the following small fix which involves removing the >> added HandleMark in BiasedLocking::preserve_marks(): >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 >> Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ >> >> I've inspected the callers of BiasedLocking::preserve_marks() and >> they all have an assert that the current thread is the VMThread. >> Since the VMThread creates a HandleMark object before executing a VM >> operation the extra HandleMark added in 8249192 is not needed. >> I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without the >> patch I get several crashes in BiasedLocking::restore_marks(). With >> the patch tests completed successfully. >> >> Thanks, >> Patricio >> From daniel.daugherty at oracle.com Tue Aug 11 19:53:36 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 11 Aug 2020 15:53:36 -0400 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> References: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> Message-ID: <3c9ffc2b-51e8-272c-f03b-d5faf5036bf6@oracle.com> On 8/11/20 2:11 PM, Patricio Chilano wrote: > Hi all, > > Please review the following small fix which involves removing the > added HandleMark in BiasedLocking::preserve_marks(): > > Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 > Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ src/hotspot/share/runtime/biasedLocking.cpp ??? No comments. As mentioned by Harold, adding a new test case to: ??? test/hotspot/jtreg/runtime/7158988/FieldMonitor.java that enables '-XX:+UseBiasedLocking' is a good idea. Also, Coleen added this note to your bug report: > Also in the patch for JDK-8249192 > , vframe.hpp > shouldn't include > handles.inline.hpp (only cpp and inline.hpp files can include > inline.hpp). It doesn't look like it needs handles.inline.hpp, > only handles.hpp. Were you planning on resolving that left over in this bug? Dan > > I've inspected the callers of BiasedLocking::preserve_marks() and they > all have an assert that the current thread is the VMThread. Since the > VMThread creates a HandleMark object before executing a VM operation > the extra HandleMark added in 8249192 is not needed. > I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without the > patch I get several crashes in BiasedLocking::restore_marks(). With > the patch tests completed successfully. > > Thanks, > Patricio > From patricio.chilano.mateo at oracle.com Tue Aug 11 20:11:21 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Tue, 11 Aug 2020 17:11:21 -0300 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: <7bcc8162-cf8b-b20e-d3ea-9b90ab5d196a@oracle.com> References: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> <7bcc8162-cf8b-b20e-d3ea-9b90ab5d196a@oracle.com> Message-ID: <3b5e7234-8b2b-49d8-ffaf-93be472a07d6@oracle.com> Hi Harold, Thanks for looking at this. On 8/11/20 4:11 PM, Harold Seigel wrote: > Hi Patricio, > > The fix looks good.? Could you also add a test based on the failure > discussed in the bug description? Sure. From the mach5 job I run without the patch I see that test/hotspot/jtreg/gc/TestFullGCALot.java fails pretty much consistently on all platforms and tiers 1-3. What do you think about adding a new run on that test with -XX:+UseBiasedLocking? Change would be: diff --git a/test/hotspot/jtreg/gc/TestFullGCALot.java b/test/hotspot/jtreg/gc/TestFullGCALot.java --- a/test/hotspot/jtreg/gc/TestFullGCALot.java +++ b/test/hotspot/jtreg/gc/TestFullGCALot.java @@ -26,8 +26,9 @@ ?/* ? * @test TestFullGCALot - * @bug 4187687 8187819 + * @bug 4187687 8187819 8251118 ? * @summary Ensure no access violation when using FullGCALot ? * @requires vm.debug ? * @run main/othervm -XX:NewSize=10m -XX:+FullGCALot -XX:FullGCALotInterval=120 gc.TestFullGCALot + * @run main/othervm -XX:NewSize=10m -XX:+FullGCALot -XX:FullGCALotInterval=120 -XX:+UseBiasedLocking gc.TestFullGCALot ? */ I would suggest adding a new run to FieldMonitor.java with -XX:+UseBiasedLocking instead, but it doesn't fail as consistently. Alternatively a new test would be pretty much like TestFullGCALot.java. The only change I would make is to explicitly add a synchronized statement: /* ?* @test TestFullGCOnSync ?* @bug 8251118 ?* @run main/othervm -XX:NewSize=10m -XX:+FullGCALot -XX:FullGCALotInterval=120 -XX:+UseBiasedLocking TestFullGCOnSync ?*/ public class TestFullGCOnSync { ? public static void main(String args[]) throws Exception { ??? Integer x = new Integer(10); ??? synchronized (x) { ????? System.gc(); ????? System.out.println("Hello world!"); ??? } ? } } Thanks, Patricio > Thanks, Harold > > On 8/11/2020 2:11 PM, Patricio Chilano wrote: >> Hi all, >> >> Please review the following small fix which involves removing the >> added HandleMark in BiasedLocking::preserve_marks(): >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 >> Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ >> >> I've inspected the callers of BiasedLocking::preserve_marks() and >> they all have an assert that the current thread is the VMThread. >> Since the VMThread creates a HandleMark object before executing a VM >> operation the extra HandleMark added in 8249192 is not needed. >> I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without the >> patch I get several crashes in BiasedLocking::restore_marks(). With >> the patch tests completed successfully. >> >> Thanks, >> Patricio >> From patricio.chilano.mateo at oracle.com Tue Aug 11 20:13:07 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Tue, 11 Aug 2020 17:13:07 -0300 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: References: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> <7bcc8162-cf8b-b20e-d3ea-9b90ab5d196a@oracle.com> Message-ID: <55b0cdf4-5324-bb9e-9003-76926e65a7d1@oracle.com> On 8/11/20 4:27 PM, Coleen Phillimore wrote: > > > On 8/11/20 3:11 PM, Harold Seigel wrote: >> Hi Patricio, >> >> The fix looks good.? Could you also add a test based on the failure >> discussed in the bug description? > > +1.? Thank you for careful examination of the callers. Thanks for reviewing this Coleen! Patricio > Coleen >> >> Thanks, Harold >> >> On 8/11/2020 2:11 PM, Patricio Chilano wrote: >>> Hi all, >>> >>> Please review the following small fix which involves removing the >>> added HandleMark in BiasedLocking::preserve_marks(): >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 >>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ >>> >>> I've inspected the callers of BiasedLocking::preserve_marks() and >>> they all have an assert that the current thread is the VMThread. >>> Since the VMThread creates a HandleMark object before executing a VM >>> operation the extra HandleMark added in 8249192 is not needed. >>> I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without >>> the patch I get several crashes in BiasedLocking::restore_marks(). >>> With the patch tests completed successfully. >>> >>> Thanks, >>> Patricio >>> > From patricio.chilano.mateo at oracle.com Tue Aug 11 20:27:46 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Tue, 11 Aug 2020 17:27:46 -0300 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: <3c9ffc2b-51e8-272c-f03b-d5faf5036bf6@oracle.com> References: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> <3c9ffc2b-51e8-272c-f03b-d5faf5036bf6@oracle.com> Message-ID: <51f4bbe4-0a19-b09c-c544-5681afb2aa1d@oracle.com> Hi Dan, Thanks for looking at this. On 8/11/20 4:53 PM, Daniel D. Daugherty wrote: > On 8/11/20 2:11 PM, Patricio Chilano wrote: >> Hi all, >> >> Please review the following small fix which involves removing the >> added HandleMark in BiasedLocking::preserve_marks(): >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 >> Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ > > src/hotspot/share/runtime/biasedLocking.cpp > ??? No comments. > > As mentioned by Harold, adding a new test case to: > > ??? test/hotspot/jtreg/runtime/7158988/FieldMonitor.java > > that enables '-XX:+UseBiasedLocking' is a good idea. I suggested adding it to test TestFullGCALot.java instead because it fails much more consistently. > Also, Coleen added this note to your bug report: > >> Also in the patch for JDK-8249192 >> , vframe.hpp >> shouldn't include >> handles.inline.hpp (only cpp and inline.hpp files can include >> inline.hpp). It doesn't look like it needs handles.inline.hpp, >> only handles.hpp. > > Were you planning on resolving that left over in this bug? Not really. I can change it here if you are okay though. Patricio > Dan > > >> >> I've inspected the callers of BiasedLocking::preserve_marks() and >> they all have an assert that the current thread is the VMThread. >> Since the VMThread creates a HandleMark object before executing a VM >> operation the extra HandleMark added in 8249192 is not needed. >> I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without the >> patch I get several crashes in BiasedLocking::restore_marks(). With >> the patch tests completed successfully. >> >> Thanks, >> Patricio >> > From harold.seigel at oracle.com Tue Aug 11 20:29:07 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 11 Aug 2020 16:29:07 -0400 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: <3b5e7234-8b2b-49d8-ffaf-93be472a07d6@oracle.com> References: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> <7bcc8162-cf8b-b20e-d3ea-9b90ab5d196a@oracle.com> <3b5e7234-8b2b-49d8-ffaf-93be472a07d6@oracle.com> Message-ID: <0928e0cd-28c9-f805-1499-ae791576ca4e@oracle.com> Hi Patricio, I think your modifications of TestFullGCALot.java are fine for testing your fix.? No need for a brand new test. Thanks, Harold On 8/11/2020 4:11 PM, Patricio Chilano wrote: > Hi Harold, > > Thanks for looking at this. > > On 8/11/20 4:11 PM, Harold Seigel wrote: >> Hi Patricio, >> >> The fix looks good.? Could you also add a test based on the failure >> discussed in the bug description? > Sure. From the mach5 job I run without the patch I see that > test/hotspot/jtreg/gc/TestFullGCALot.java fails pretty much > consistently on all platforms and tiers 1-3. What do you think about > adding a new run on that test with -XX:+UseBiasedLocking? Change would > be: > > diff --git a/test/hotspot/jtreg/gc/TestFullGCALot.java > b/test/hotspot/jtreg/gc/TestFullGCALot.java > --- a/test/hotspot/jtreg/gc/TestFullGCALot.java > +++ b/test/hotspot/jtreg/gc/TestFullGCALot.java > @@ -26,8 +26,9 @@ > ?/* > ? * @test TestFullGCALot > - * @bug 4187687 8187819 > + * @bug 4187687 8187819 8251118 > ? * @summary Ensure no access violation when using FullGCALot > ? * @requires vm.debug > ? * @run main/othervm -XX:NewSize=10m -XX:+FullGCALot > -XX:FullGCALotInterval=120 gc.TestFullGCALot > + * @run main/othervm -XX:NewSize=10m -XX:+FullGCALot > -XX:FullGCALotInterval=120 -XX:+UseBiasedLocking gc.TestFullGCALot > ? */ > > I would suggest adding a new run to FieldMonitor.java with > -XX:+UseBiasedLocking instead, but it doesn't fail as consistently. > > Alternatively a new test would be pretty much like > TestFullGCALot.java. The only change I would make is to explicitly add > a synchronized statement: > > /* > ?* @test TestFullGCOnSync > ?* @bug 8251118 > ?* @run main/othervm -XX:NewSize=10m -XX:+FullGCALot > -XX:FullGCALotInterval=120 -XX:+UseBiasedLocking TestFullGCOnSync > ?*/ > > public class TestFullGCOnSync { > ? public static void main(String args[]) throws Exception { > ??? Integer x = new Integer(10); > ??? synchronized (x) { > ????? System.gc(); > ????? System.out.println("Hello world!"); > ??? } > ? } > } > > Thanks, > Patricio >> Thanks, Harold >> >> On 8/11/2020 2:11 PM, Patricio Chilano wrote: >>> Hi all, >>> >>> Please review the following small fix which involves removing the >>> added HandleMark in BiasedLocking::preserve_marks(): >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 >>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ >>> >>> I've inspected the callers of BiasedLocking::preserve_marks() and >>> they all have an assert that the current thread is the VMThread. >>> Since the VMThread creates a HandleMark object before executing a VM >>> operation the extra HandleMark added in 8249192 is not needed. >>> I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without >>> the patch I get several crashes in BiasedLocking::restore_marks(). >>> With the patch tests completed successfully. >>> >>> Thanks, >>> Patricio >>> > From daniel.daugherty at oracle.com Tue Aug 11 20:34:11 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 11 Aug 2020 16:34:11 -0400 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: <51f4bbe4-0a19-b09c-c544-5681afb2aa1d@oracle.com> References: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> <3c9ffc2b-51e8-272c-f03b-d5faf5036bf6@oracle.com> <51f4bbe4-0a19-b09c-c544-5681afb2aa1d@oracle.com> Message-ID: On 8/11/20 4:27 PM, Patricio Chilano wrote: > Hi Dan, > > Thanks for looking at this. > > On 8/11/20 4:53 PM, Daniel D. Daugherty wrote: >> On 8/11/20 2:11 PM, Patricio Chilano wrote: >>> Hi all, >>> >>> Please review the following small fix which involves removing the >>> added HandleMark in BiasedLocking::preserve_marks(): >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 >>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ >> >> src/hotspot/share/runtime/biasedLocking.cpp >> ??? No comments. >> >> As mentioned by Harold, adding a new test case to: >> >> ??? test/hotspot/jtreg/runtime/7158988/FieldMonitor.java >> >> that enables '-XX:+UseBiasedLocking' is a good idea. > I suggested adding it to test TestFullGCALot.java instead because it > fails much more consistently. I'm good with that also. > >> Also, Coleen added this note to your bug report: >> >>> Also in the patch for JDK-8249192 >>> , vframe.hpp >>> shouldn't include >>> handles.inline.hpp (only cpp and inline.hpp files can include >>> inline.hpp). It doesn't look like it needs handles.inline.hpp, >>> only handles.hpp. >> >> Were you planning on resolving that left over in this bug? > Not really. I can change it here if you are okay though. Your call on whether to fix it here or with a new bug. Seems like fixing it here is less work... this bug is a follow-on to the original and that comment is a follow-on to the original... Dan > > Patricio >> Dan >> >> >>> >>> I've inspected the callers of BiasedLocking::preserve_marks() and >>> they all have an assert that the current thread is the VMThread. >>> Since the VMThread creates a HandleMark object before executing a VM >>> operation the extra HandleMark added in 8249192 is not needed. >>> I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without >>> the patch I get several crashes in BiasedLocking::restore_marks(). >>> With the patch tests completed successfully. >>> >>> Thanks, >>> Patricio >>> >> > From patricio.chilano.mateo at oracle.com Tue Aug 11 20:37:05 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Tue, 11 Aug 2020 17:37:05 -0300 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: <0928e0cd-28c9-f805-1499-ae791576ca4e@oracle.com> References: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> <7bcc8162-cf8b-b20e-d3ea-9b90ab5d196a@oracle.com> <3b5e7234-8b2b-49d8-ffaf-93be472a07d6@oracle.com> <0928e0cd-28c9-f805-1499-ae791576ca4e@oracle.com> Message-ID: On 8/11/20 5:29 PM, Harold Seigel wrote: > Hi Patricio, > > I think your modifications of TestFullGCALot.java are fine for testing > your fix.? No need for a brand new test. Great, thanks for reviewing this Harold! Patricio > Thanks, Harold > > On 8/11/2020 4:11 PM, Patricio Chilano wrote: >> Hi Harold, >> >> Thanks for looking at this. >> >> On 8/11/20 4:11 PM, Harold Seigel wrote: >>> Hi Patricio, >>> >>> The fix looks good.? Could you also add a test based on the failure >>> discussed in the bug description? >> Sure. From the mach5 job I run without the patch I see that >> test/hotspot/jtreg/gc/TestFullGCALot.java fails pretty much >> consistently on all platforms and tiers 1-3. What do you think about >> adding a new run on that test with -XX:+UseBiasedLocking? Change >> would be: >> >> diff --git a/test/hotspot/jtreg/gc/TestFullGCALot.java >> b/test/hotspot/jtreg/gc/TestFullGCALot.java >> --- a/test/hotspot/jtreg/gc/TestFullGCALot.java >> +++ b/test/hotspot/jtreg/gc/TestFullGCALot.java >> @@ -26,8 +26,9 @@ >> ?/* >> ? * @test TestFullGCALot >> - * @bug 4187687 8187819 >> + * @bug 4187687 8187819 8251118 >> ? * @summary Ensure no access violation when using FullGCALot >> ? * @requires vm.debug >> ? * @run main/othervm -XX:NewSize=10m -XX:+FullGCALot >> -XX:FullGCALotInterval=120 gc.TestFullGCALot >> + * @run main/othervm -XX:NewSize=10m -XX:+FullGCALot >> -XX:FullGCALotInterval=120 -XX:+UseBiasedLocking gc.TestFullGCALot >> ? */ >> >> I would suggest adding a new run to FieldMonitor.java with >> -XX:+UseBiasedLocking instead, but it doesn't fail as consistently. >> >> Alternatively a new test would be pretty much like >> TestFullGCALot.java. The only change I would make is to explicitly >> add a synchronized statement: >> >> /* >> ?* @test TestFullGCOnSync >> ?* @bug 8251118 >> ?* @run main/othervm -XX:NewSize=10m -XX:+FullGCALot >> -XX:FullGCALotInterval=120 -XX:+UseBiasedLocking TestFullGCOnSync >> ?*/ >> >> public class TestFullGCOnSync { >> ? public static void main(String args[]) throws Exception { >> ??? Integer x = new Integer(10); >> ??? synchronized (x) { >> ????? System.gc(); >> ????? System.out.println("Hello world!"); >> ??? } >> ? } >> } >> >> Thanks, >> Patricio >>> Thanks, Harold >>> >>> On 8/11/2020 2:11 PM, Patricio Chilano wrote: >>>> Hi all, >>>> >>>> Please review the following small fix which involves removing the >>>> added HandleMark in BiasedLocking::preserve_marks(): >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 >>>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ >>>> >>>> I've inspected the callers of BiasedLocking::preserve_marks() and >>>> they all have an assert that the current thread is the VMThread. >>>> Since the VMThread creates a HandleMark object before executing a >>>> VM operation the extra HandleMark added in 8249192 is not needed. >>>> I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without >>>> the patch I get several crashes in BiasedLocking::restore_marks(). >>>> With the patch tests completed successfully. >>>> >>>> Thanks, >>>> Patricio >>>> >> From patricio.chilano.mateo at oracle.com Tue Aug 11 20:45:00 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Tue, 11 Aug 2020 17:45:00 -0300 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: References: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> <3c9ffc2b-51e8-272c-f03b-d5faf5036bf6@oracle.com> <51f4bbe4-0a19-b09c-c544-5681afb2aa1d@oracle.com> Message-ID: <8f0f8423-f46b-9b78-1544-24f02a8606f7@oracle.com> On 8/11/20 5:34 PM, Daniel D. Daugherty wrote: > On 8/11/20 4:27 PM, Patricio Chilano wrote: >> Hi Dan, >> >> Thanks for looking at this. >> >> On 8/11/20 4:53 PM, Daniel D. Daugherty wrote: >>> On 8/11/20 2:11 PM, Patricio Chilano wrote: >>>> Hi all, >>>> >>>> Please review the following small fix which involves removing the >>>> added HandleMark in BiasedLocking::preserve_marks(): >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 >>>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ >>> >>> src/hotspot/share/runtime/biasedLocking.cpp >>> ??? No comments. >>> >>> As mentioned by Harold, adding a new test case to: >>> >>> ??? test/hotspot/jtreg/runtime/7158988/FieldMonitor.java >>> >>> that enables '-XX:+UseBiasedLocking' is a good idea. >> I suggested adding it to test TestFullGCALot.java instead because it >> fails much more consistently. > > I'm good with that also. > > >> >>> Also, Coleen added this note to your bug report: >>> >>>> Also in the patch for JDK-8249192 >>>> , vframe.hpp >>>> shouldn't include >>>> handles.inline.hpp (only cpp and inline.hpp files can include >>>> inline.hpp). It doesn't look like it needs handles.inline.hpp, >>>> only handles.hpp. >>> >>> Were you planning on resolving that left over in this bug? >> Not really. I can change it here if you are okay though. > > Your call on whether to fix it here or with a new bug. Seems > like fixing it here is less work... this bug is a follow-on to > the original and that comment is a follow-on to the original... Yeah, sounds good to me to remove the extra header in this change. I'll check it builds fine on all platforms first. Below is v2 in case you wanted to see it: Full: http://cr.openjdk.java.net/~pchilanomate/8251118/v2/webrev/ Inc: http://cr.openjdk.java.net/~pchilanomate/8251118/v2/inc/webrev/ Thanks Dan! Patricio > Dan > > >> >> Patricio >>> Dan >>> >>> >>>> >>>> I've inspected the callers of BiasedLocking::preserve_marks() and >>>> they all have an assert that the current thread is the VMThread. >>>> Since the VMThread creates a HandleMark object before executing a >>>> VM operation the extra HandleMark added in 8249192 is not needed. >>>> I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without >>>> the patch I get several crashes in BiasedLocking::restore_marks(). >>>> With the patch tests completed successfully. >>>> >>>> Thanks, >>>> Patricio >>>> >>> >> > From thomas.schatzl at oracle.com Tue Aug 11 20:47:06 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Tue, 11 Aug 2020 20:47:06 +0000 (UTC) Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark Message-ID: <44ad98e3-584d-4ae0-bcf7-4afe2339a7f1@default> Hi, v2 looks good to me. Thanks for fixing this, Thomas ----- Original Message ----- From: patricio.chilano.mateo at oracle.com To: daniel.daugherty at oracle.com, hotspot-runtime-dev at openjdk.java.net Sent: Tuesday, August 11, 2020 10:45:28 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: Re: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark On 8/11/20 5:34 PM, Daniel D. Daugherty wrote: > On 8/11/20 4:27 PM, Patricio Chilano wrote: >> Hi Dan, >> >> Thanks for looking at this. >> >> On 8/11/20 4:53 PM, Daniel D. Daugherty wrote: >>> On 8/11/20 2:11 PM, Patricio Chilano wrote: >>>> Hi all, >>>> >>>> Please review the following small fix which involves removing the >>>> added HandleMark in BiasedLocking::preserve_marks(): >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 >>>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ >>> >>> src/hotspot/share/runtime/biasedLocking.cpp >>> ??? No comments. >>> >>> As mentioned by Harold, adding a new test case to: >>> >>> ??? test/hotspot/jtreg/runtime/7158988/FieldMonitor.java >>> >>> that enables '-XX:+UseBiasedLocking' is a good idea. >> I suggested adding it to test TestFullGCALot.java instead because it >> fails much more consistently. > > I'm good with that also. > > >> >>> Also, Coleen added this note to your bug report: >>> >>>> Also in the patch for JDK-8249192 >>>> , vframe.hpp >>>> shouldn't include >>>> handles.inline.hpp (only cpp and inline.hpp files can include >>>> inline.hpp). It doesn't look like it needs handles.inline.hpp, >>>> only handles.hpp. >>> >>> Were you planning on resolving that left over in this bug? >> Not really. I can change it here if you are okay though. > > Your call on whether to fix it here or with a new bug. Seems > like fixing it here is less work... this bug is a follow-on to > the original and that comment is a follow-on to the original... Yeah, sounds good to me to remove the extra header in this change. I'll check it builds fine on all platforms first. Below is v2 in case you wanted to see it: Full: http://cr.openjdk.java.net/~pchilanomate/8251118/v2/webrev/ Inc: http://cr.openjdk.java.net/~pchilanomate/8251118/v2/inc/webrev/ Thanks Dan! Patricio > Dan > > >> >> Patricio >>> Dan >>> >>> >>>> >>>> I've inspected the callers of BiasedLocking::preserve_marks() and >>>> they all have an assert that the current thread is the VMThread. >>>> Since the VMThread creates a HandleMark object before executing a >>>> VM operation the extra HandleMark added in 8249192 is not needed. >>>> I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without >>>> the patch I get several crashes in BiasedLocking::restore_marks(). >>>> With the patch tests completed successfully. >>>> >>>> Thanks, >>>> Patricio >>>> >>> >> > From coleen.phillimore at oracle.com Tue Aug 11 20:51:24 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 11 Aug 2020 16:51:24 -0400 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: <8f0f8423-f46b-9b78-1544-24f02a8606f7@oracle.com> References: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> <3c9ffc2b-51e8-272c-f03b-d5faf5036bf6@oracle.com> <51f4bbe4-0a19-b09c-c544-5681afb2aa1d@oracle.com> <8f0f8423-f46b-9b78-1544-24f02a8606f7@oracle.com> Message-ID: <750e129d-40c7-d205-5352-2f4582475c83@oracle.com> On 8/11/20 4:45 PM, Patricio Chilano wrote: > > On 8/11/20 5:34 PM, Daniel D. Daugherty wrote: >> On 8/11/20 4:27 PM, Patricio Chilano wrote: >>> Hi Dan, >>> >>> Thanks for looking at this. >>> >>> On 8/11/20 4:53 PM, Daniel D. Daugherty wrote: >>>> On 8/11/20 2:11 PM, Patricio Chilano wrote: >>>>> Hi all, >>>>> >>>>> Please review the following small fix which involves removing the >>>>> added HandleMark in BiasedLocking::preserve_marks(): >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 >>>>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ >>>> >>>> src/hotspot/share/runtime/biasedLocking.cpp >>>> ??? No comments. >>>> >>>> As mentioned by Harold, adding a new test case to: >>>> >>>> ??? test/hotspot/jtreg/runtime/7158988/FieldMonitor.java >>>> >>>> that enables '-XX:+UseBiasedLocking' is a good idea. >>> I suggested adding it to test TestFullGCALot.java instead because it >>> fails much more consistently. >> >> I'm good with that also. Me too, even better than FieldMonitor.java if it's more reproduceable. >> >> >>> >>>> Also, Coleen added this note to your bug report: >>>> >>>>> Also in the patch for JDK-8249192 >>>>> , vframe.hpp >>>>> shouldn't include >>>>> handles.inline.hpp (only cpp and inline.hpp files can include >>>>> inline.hpp). It doesn't look like it needs handles.inline.hpp, >>>>> only handles.hpp. >>>> >>>> Were you planning on resolving that left over in this bug? >>> Not really. I can change it here if you are okay though. >> >> Your call on whether to fix it here or with a new bug. Seems >> like fixing it here is less work... this bug is a follow-on to >> the original and that comment is a follow-on to the original... > Yeah, sounds good to me to remove the extra header in this change. > I'll check it builds fine on all platforms first. > Below is v2 in case you wanted to see it: > > Full: http://cr.openjdk.java.net/~pchilanomate/8251118/v2/webrev/ > Inc: http://cr.openjdk.java.net/~pchilanomate/8251118/v2/inc/webrev/ I think vframe.hpp should include handles.hpp because there are Handle types in it, just in case something changes and it's not transiently included anymore. Otherwise, everything looks good.? I don't need to see another webrev. thanks! Coleen > > Thanks Dan! > > Patricio >> Dan >> >> >>> >>> Patricio >>>> Dan >>>> >>>> >>>>> >>>>> I've inspected the callers of BiasedLocking::preserve_marks() and >>>>> they all have an assert that the current thread is the VMThread. >>>>> Since the VMThread creates a HandleMark object before executing a >>>>> VM operation the extra HandleMark added in 8249192 is not needed. >>>>> I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without >>>>> the patch I get several crashes in BiasedLocking::restore_marks(). >>>>> With the patch tests completed successfully. >>>>> >>>>> Thanks, >>>>> Patricio >>>>> >>>> >>> >> > From patricio.chilano.mateo at oracle.com Tue Aug 11 20:52:22 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Tue, 11 Aug 2020 17:52:22 -0300 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: <44ad98e3-584d-4ae0-bcf7-4afe2339a7f1@default> References: <44ad98e3-584d-4ae0-bcf7-4afe2339a7f1@default> Message-ID: <9a608319-ab04-e5ef-a589-e5d46297f12e@oracle.com> Hi Thomas, On 8/11/20 5:47 PM, Thomas Schatzl wrote: > Hi, > > v2 looks good to me. Great, thanks for reviewing this! Patricio > Thanks for fixing this, > Thomas > > ----- Original Message ----- > From: patricio.chilano.mateo at oracle.com > To: daniel.daugherty at oracle.com, hotspot-runtime-dev at openjdk.java.net > Sent: Tuesday, August 11, 2020 10:45:28 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna > Subject: Re: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark > > > On 8/11/20 5:34 PM, Daniel D. Daugherty wrote: >> On 8/11/20 4:27 PM, Patricio Chilano wrote: >>> Hi Dan, >>> >>> Thanks for looking at this. >>> >>> On 8/11/20 4:53 PM, Daniel D. Daugherty wrote: >>>> On 8/11/20 2:11 PM, Patricio Chilano wrote: >>>>> Hi all, >>>>> >>>>> Please review the following small fix which involves removing the >>>>> added HandleMark in BiasedLocking::preserve_marks(): >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 >>>>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ >>>> src/hotspot/share/runtime/biasedLocking.cpp >>>> ??? No comments. >>>> >>>> As mentioned by Harold, adding a new test case to: >>>> >>>> ??? test/hotspot/jtreg/runtime/7158988/FieldMonitor.java >>>> >>>> that enables '-XX:+UseBiasedLocking' is a good idea. >>> I suggested adding it to test TestFullGCALot.java instead because it >>> fails much more consistently. >> I'm good with that also. >> >> >>>> Also, Coleen added this note to your bug report: >>>> >>>>> Also in the patch for JDK-8249192 >>>>> , vframe.hpp >>>>> shouldn't include >>>>> handles.inline.hpp (only cpp and inline.hpp files can include >>>>> inline.hpp). It doesn't look like it needs handles.inline.hpp, >>>>> only handles.hpp. >>>> Were you planning on resolving that left over in this bug? >>> Not really. I can change it here if you are okay though. >> Your call on whether to fix it here or with a new bug. Seems >> like fixing it here is less work... this bug is a follow-on to >> the original and that comment is a follow-on to the original... > Yeah, sounds good to me to remove the extra header in this change. I'll > check it builds fine on all platforms first. > Below is v2 in case you wanted to see it: > > Full: http://cr.openjdk.java.net/~pchilanomate/8251118/v2/webrev/ > Inc: http://cr.openjdk.java.net/~pchilanomate/8251118/v2/inc/webrev/ > > Thanks Dan! > > Patricio >> Dan >> >> >>> Patricio >>>> Dan >>>> >>>> >>>>> I've inspected the callers of BiasedLocking::preserve_marks() and >>>>> they all have an assert that the current thread is the VMThread. >>>>> Since the VMThread creates a HandleMark object before executing a >>>>> VM operation the extra HandleMark added in 8249192 is not needed. >>>>> I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without >>>>> the patch I get several crashes in BiasedLocking::restore_marks(). >>>>> With the patch tests completed successfully. >>>>> >>>>> Thanks, >>>>> Patricio >>>>> From ioi.lam at oracle.com Tue Aug 11 22:09:12 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 11 Aug 2020 15:09:12 -0700 Subject: Fwd: RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS In-Reply-To: <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> Message-ID: <3f59d2fe-14ab-7b45-5fea-85128354ce09@oracle.com> Hi Yumin, This looks good! Here are my comments: [1] classListParser.cpp: #define LAMBDA_FOMRM_INVOKER "@lambda-form-invoker" I think this should be moved to classListParser.hpp: ??? const char* lambda_form_invoker_tag() { ??????? return "@lambda-form-invoker"; ??? } Also, instead of also duplicating this string in InvokerBytecodeGenerator.java, the Java side of the code can be changed to ? ?? String outLine = holder.getName() + " " + name + " " + .... and JVM_CDSTraceResolve() can be changed to ??? classlist_file->print_cr("%s %s", ClassListParser::lambda_form_invoker_tag(), c_line); [2] metaspaceShared.cpp: This file is getting too big. I think we should move the new functionality to a new file ?? share/memory/lambdaFormInvokers.cpp [3] MetaspaceShared::preload_classes() 2080 if (parser.is_lambda_format()) This name is too generic. How about "parser.is_lambda_form_invoker()"? 2109?????? if (lambda_list == NULL) { 2110???????? lambda_list = new GrowableArray(8); 2111?????? } 2112?????? lambda_list->append(parser.current_line()); These can be changed to LambdaFormInvokers::append(parser.current_line()); [4] I think the follow code that changes the system dictionary is kind of scary. Maybe we can avoid it. 2060?? // replace with the new created klass. 2061?? { 2062???? MutexLocker lock(THREAD, SystemDictionary_lock); 2063???? InstanceKlass* old = cld->replace_class(sym, result); 2064???? SystemDictionaryShared::set_excluded(old); 2065???? log_info(cds)("Replace class %s, old: %p? new: %p", class_name, old, result); 2066?? } How about just do this: ??? InstanceKlass* old = cld->find_class(sym); ??? SystemDictionaryShared::set_excluded(old); This will prevent the old class from being added to CollectClassesClosure. We actually don't need to add the new class into cld->dictionary() -- we don't want to "use" the new class in any way other than writing it into the archive. We can do this to add the new class to the CollectClassesClosure: + GrowableArray* LambdaFormInvokers::replacement_holder_classes(); ? ... ? _global_klass_objects = new GrowableArray(1000); ? CollectClassesClosure collect_classes; ? ClassLoaderDataGraph::loaded_classes_do(&collect_classes); + collect_classes.add_extra_classes(LambdaFormInvokers::replacement_holder_classes()); ? _global_klass_objects->sort(global_klass_compare); [5] InvokerGenerateBytesException.java This class cannot be public, as it's not part of the java.lang.invoke package specification. Anyway, we don't seem to actually need this type, as it's not used outside of InvokerBytecodeGeneratorHelper. How about replacing all use of ??? throw new InvokerGenerateBytesException(...); to ??? throw new RuntimeException(...); [6] This adds duplicated code for processing the option: 2846???? // -XX:DumpLoadedClassList 2847???? } else if (match_option(option, "-XX:DumpLoadedClassList=", &tail)) { 2848?????? if (tail != NULL) { 2849 add_property("java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true"); 2850???????? DumpLoadedClassList = os::strdup_check_oom(tail); 2851?????? } else { 2852???????? warning("Bad option for -XX:DumpLoadedClassList="); 2853???????? return JNI_EINVAL; 2854?????? } I think it's better to move the code to here: jint Arguments::parse(const JavaVMInitArgs* initial_cmd_args) { .... - #if !INCLUDE_CDS + #if INCLUDE_CDS +?? if (DumpLoadedClassList != NULL) { +????? if (!add_property("java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true")) { +??????? return JNI_ENOMEM; +????? } +?? } + #else [7] For testing, we should have different kinds of invalid input for the @lambda-form-invoker to make sure that the errors are handled properly. [8] Some variable names should be changed to be more descriptive. Here is an example: // k - the class full name // v - the class bytes void MetaspaceShared::reload_class(Handle k, Handle v, TRAPS) { should be changed class_name, class_bytes. Thanks - Ioi On 8/11/20 10:36 AM, Yumin Qi wrote: > > Hi, Please reivew > > ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 > ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ > > ? Summary: CDS does not archive pre-generated lambda form classes for > method handles: > > [0.142s][info][class,load] > java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: > __JVM_LookupDefineClass__ > > The method handle resolution if not found in the holder class, a class > with the method will be generated and loaded as vm internal created > class and not archived like above. Those class names are mangled as > combination of the class name and the class instance address. > > In this patch, collect those method holder class names and their > associated methods' signatures when user specify DumpLoadedClassList, > and record them in the log file in a format similar to the output > format from traced by > -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use > another name for CDS: > -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The > line prefix also changed from "[LF_RESOLVE]" to > "@lambda-invoke-handle".? At dump time, regenerate the holder class > with those methods and replace the existing holder class and archived > it. At runtime, the resolution of the holder class which contains > those methods are loaded from CDS archive so avoid regenerating them > again. The patch reorganized the code for Jlink plugin, so the new > added InvokerBytecodeGeneratorHelper can be shared both by JLink > plugin and CDS code. Also change made to the mangled generated class > name at static dump, the class name combining the class name and a > sequentially ordered number instead of the class instance address, > which looks like a random. > > > To use this feature, one can do the dump/run just like done for a > static dump/run, so no extra steps required. > > ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp > > ????? DumpLoadedClassList will turn on > -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true > > ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist > -XX:SharedArchiveFile=my.jsa JavaApp > > ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp > > ? The performance on javac HelloWorld.java (javac-benchmark), no patch > vs patch: > > ?? 1:?? 2689285002? 2641821474 (-47463528)????? ---- 391.720 382.990 ( > -8.730)????? ---- > ?? 2:?? 2687495085? 2632969688 (-54525397)????? ---- 391.030 381.480 ( > -9.550)????? ----- > ?? 3:?? 2694664066? 2636523114 (-58140952)????? ----- 390.610 382.550 > ( -8.060)????? ---- > ?? 4:?? 2686554164? 2639355233 (-47198931)????? ---- 390.700 383.390 ( > -7.310)????? --- > ?? 5:?? 2691072338? 2633016687 (-58055651)????? ----- 388.990 382.360 > ( -6.630)????? --- > ?? 6:?? 2684448174? 2644191854 (-40256320)????? --- 389.450 382.990 ( > -6.460)????? --- > ?? 7:?? 2694921227? 2630505090 (-64416137)????? ----- 389.300 383.160 > ( -6.140)????? --- > ?? 8:?? 2685209712? 2639334320 (-45875392)????? ---- 388.370 381.060 ( > -7.310)????? --- > ?? 9:?? 2695885942? 2640618655 (-55267287)????? ---- 389.560 381.100 ( > -8.460)????? ---- > ? 10:?? 2689162942? 2635658943 (-53503999)????? ---- 389.690 379.110 > (-10.580)????? ----- > ============================================================ > ??????? 2689866989? 2637396210 (-52470778)????? ---- 389.941 382.017 ( > -7.924)????? ---- > > instr delta =??? -52470778??? -1.9507% > > time? delta =?????? -7.924 ms -2.0321% > > ? It will show much more improvement if compare with the default archive: > > ?????????? ? ? ? no patch??????????????????????? patch > > instr ??? ? ? 3,996,847,605??????? 3,320,928,995 > > time(s).???? 0.686835162 ? ? ? ?? 0.474933546 > > > ? Tests: > > 1)jtreg on jdk/tools and hotspot/runtime local. > > 2) mach5 ter1,2 > > > Thanks > > Yumin > > ? ` From patricio.chilano.mateo at oracle.com Tue Aug 11 22:10:43 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Tue, 11 Aug 2020 19:10:43 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <512b4000-52d1-d465-81cf-57097d5deaec@oracle.com> <858d3665-6121-81fe-530c-c73d00d49de0@oracle.com> <789b1116-9aba-c567-df7f-fa016aab766f@oracle.com> <31e7d577-863c-440d-02d6-34edc9a4b540@oracle.com> <903166d7-5680-886e-5bcc-3ba31864b179@oracle.com> <6960c0f3-8cbe-fb80-3529-a1c7fb794f0d@oracle.com> Message-ID: <9e44c166-3583-2ad1-dff6-b70475bff7c7@oracle.com> Hi David, On 8/10/20 9:26 PM, David Holmes wrote: > One follow up ... I'll send separate mail for v5. > > On 11/08/2020 6:19 am, Patricio Chilano wrote: >> Hi David, >> >> On 8/10/20 1:12 AM, David Holmes wrote: >>> Hi Patricio, >>> >>> First, I confirmed with Valhalla folk that this change is fine to go >>> ahead as a stand-alone change independent of the proposed JEP in >>> this area. >> Great, thanks for that!? : ) >> >>> I've checked the incremental changes for v2, v3 and v4 and >>> everything seems good to me. >>> >>> I only have a couple of nits with the tests: >>> >>> ? 54???? private static void initTestObjects() { >>> ? 55???????? testObjects.add(Character.valueOf('H')); >>> ? 56???????? testObjects.add(Boolean.valueOf(true)); >>> ? 57???????? Byte byteT = 0x40; >>> ? 58???????? testObjects.add(byteT); >>> ? 59???????? Short shortT = 0x4000; >>> ? 60???????? testObjects.add(shortT); >>> ? 61???????? testObjects.add(Integer.valueOf(0x40000000)); >>> ? 62 testObjects.add(Long.valueOf(0x4000000000000000L)); >>> ? 63???????? testObjects.add(Float.valueOf(1.20f)); >>> ? 64???????? testObjects.add(Double.valueOf(1.2345)); >>> ? 65???? } >>> >>> Why are Byte and Short treated differently? >> If I do the same as with the other types, the argument passed to >> Byte.valueOf() and Short.valueOf() gets treated as an int and I get a >> compilation error: >> >> : error: no suitable method found for valueOf(int) >> ?? testObjects.add(Byte.valueOf(0x40)); > > Right - you'd need to cast the constant to byte/short: > > testObjects.add(Byte.valueOf((byte)0x40)); Ok, I changed it in both SyncOnPrimitiveWrapperTest.java and TestSyncOnPrimitiveWrapperEvent.java: diff --git a/test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java b/test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java --- a/test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java +++ b/test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java @@ -55,8 +55,6 @@ ???????? testObjects.add(Character.valueOf('H')); ???????? testObjects.add(Boolean.valueOf(true)); -??????? Byte byteT = 0x40; -??????? testObjects.add(byteT); -??????? Short shortT = 0x4000; -??????? testObjects.add(shortT); +??????? testObjects.add(Byte.valueOf((byte)0x40)); +??????? testObjects.add(Short.valueOf((short)0x4000)); ???????? testObjects.add(Integer.valueOf(0x40000000)); ???????? testObjects.add(Long.valueOf(0x4000000000000000L)); diff --git a/test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java b/test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java --- a/test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java +++ b/test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java @@ -51,8 +51,6 @@ ???????? testObjects.add(Character.valueOf('H')); ???????? testObjects.add(Boolean.valueOf(true)); -??????? Byte byteT = 0x40; -??????? testObjects.add(byteT); -??????? Short shortT = 0x4000; -??????? testObjects.add(shortT); +??????? testObjects.add(Byte.valueOf((byte)0x40)); +??????? testObjects.add(Short.valueOf((short)0x4000)); ???????? testObjects.add(Integer.valueOf(0x40000000)); ???????? testObjects.add(Long.valueOf(0x4000000000000000L)); Thanks, Patricio > Cheers, > David > ? ???????????????????????????????? ^ >> ?? method Byte.valueOf(byte) is not applicable >> ?? (argument mismatch; possible lossy conversion from int to byte) >> ?? method Byte.valueOf(String) is not applicable >> ?? (argument mismatch; int cannot be converted to String) >> >>> ?74 fatalTests[index] = Stream.of(commonFatalTestsFlags, >>> specificFlags[i], new String[] >>> {"SyncOnPrimitiveWrapperTest$FatalTest", Integer.toString(j)}) >>> ? 75 .flatMap(Stream::of).toArray(String[]::new); >>> >>> The preferred Java style [1] for stream operations is: >>> >>> fatalTests[index] = Stream.of(commonFatalTestsFlags, >>> specificFlags[i], new String[] >>> {"SyncOnPrimitiveWrapperTest$FatalTest", Integer.toString(j)}) >>> ????????????????????????? .flatMap(Stream::of) >>> ????????????????????????? .toArray(String[]::new); >> Fixed. >> >> Here is v5: >> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ >> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/webrev/ >> >> Thanks, >> Patricio >>> Thanks, >>> David >>> ----- >>> >>> [1] As demonstrated here (but widely used in JDK code): >>> http://cr.openjdk.java.net/~alundblad/styleguide/index-v6.html#toc-wrapping-lines >>> >>> >>> On 7/08/2020 4:48 am, Patricio Chilano wrote: >>>> Hi Dan, >>>> >>>> On 8/5/20 6:47 PM, Daniel D. Daugherty wrote: >>>>> I'm peeking ahead to the next webrev... :-) >>>>> >>>>> > http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev/ >>>> : ) >>>> >>>>> General comments: >>>>> ? - check files for copyright year updates. >>>> Updated accessFlags.hpp, c1_MacroAssembler_aarch64.cpp, >>>> c1_MacroAssembler_arm.cpp and c1_MacroAssembler_x86.cpp. >>>> >>>>> src/hotspot/share/runtime/synchronizer.hpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/share/runtime/synchronizer.cpp >>>>> ??? L507: ? const markWord mark = obj->mark(); >>>>> ??? L508: >>>>> ??? L509: ? if (DiagnoseSyncOnPrimitiveWrappers != 0 && >>>>> obj->klass()->is_box()) { >>>>> ??? L510: ??? return false; >>>>> ??? L511: ? } >>>>> ??????? The new bailout on L509-511 can move above L507. >>>> Moved. >>>> >>>>> L573: ??? fatal("Synchronizing on object " INTPTR_FORMAT " of >>>>> klass %s", p2i(obj()), obj->klass()->external_name()); >>>>> ??????? This external_name() call does not have a ResourceMark. >>>> Good catch! I had one in a previous version but then I changed the >>>> conditionals and lost it for the fatal error case. The test worked >>>> okay because for the main JavaThread there is a ResourceMark in >>>> jni_invoke_static() (jni.cpp). >>>> >>>>> src/hotspot/share/logging/logTag.hpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/share/oops/klass.hpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/share/utilities/accessFlags.hpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/share/runtime/globals.hpp >>>>> ??? L814: ???????????? "0: off " >>>>> ??????? Missing a ';' after "off". >>>> Fixed. >>>> >>>>> L816: ???????????? "2: log message to stdout. >>>>> ??????? Perhaps add "(by default)" after "stdout" or >>>>> ??????? don't say where log output is at all. >>>>> >>>>> src/hotspot/share/runtime/arguments.cpp >>>>> ??? L4197: LogConfiguration::configure_stdout(LogLevel::Info, >>>>> true, LOG_TAGS(primitivewrappers)); >>>>> ??????? Hmmm... maybe ignore my comments about L816 in globals.hpp >>>>> ??????? since it looks like logging output is configured to 'stdout'. >>>>> ??????? I'm assuming that other log options to put output elsewhere >>>>> ??????? are overridden by this code? >>>> Right. So the logging is done under UL with the tag >>>> primitivewrappers. If that tag is specified in Xlog then this >>>> conditional will be skipped because !log_is_enabled(Info, >>>> primitivewrappers) will be false. >>>> >>>>> src/hotspot/share/classfile/systemDictionary.cpp >>>>> ??? L2188: ??? for (int i = T_BOOLEAN; i < T_LONG+1; i++) { >>>>> ??????? nit - s/T_LONG+1/T_LONG + 1/ >>>> Fixed. >>>> >>>>> L2191: _box_klasses[i]->set_prototype_header(markWord::prototype()); >>>>> ??????? I assume we're keeping the prototype_header field when >>>>> Biased Locking >>>>> ??????? goes away? The reason I ask: >>>>> >>>>> ? ? ? ? static markWord prototype() { >>>>> ? ? ? ? ? return markWord( no_hash_in_place | no_lock_in_place ); >>>>> ? ? ? ? } >>>>> >>>>> ??????? is because without Biased Locking, do we really need the >>>>> prototype >>>>> ??????? anymore? The initial markWord won't need possible variants... >>>> Yes, I think it can go away unless somebody finds another use for >>>> it. Maybe Valhalla. >>>> >>>>> src/hotspot/share/jfr/metadata/metadata.xml >>>>> ??? L69: ? >>>> ??????? Is the category "Java Application" because it's the >>>>> application >>>>> ??????? code that did something "wrong" here? Where "application" >>>>> is loosely >>>>> ??????? defined to include the possibility of auto generated code, >>>>> library >>>>> ??????? code and the like? Or perhaps "application" because >>>>> something "above" >>>>> ??????? the "Java Virtual Machine, Runtime" did the "wrong" thing >>>>> here? >>>> I don't know what the right category should be really. I saw the >>>> events JavaMonitorEnter, JavaMonitorWait and JavaMonitorInflate and >>>> thought this event should fall in the same category they do. >>>> >>>>> src/jdk.jfr/share/conf/jfr/default.jfc >>>>> ??? No comments. >>>>> >>>>> src/jdk.jfr/share/conf/jfr/profile.jfc >>>>> ??? No comments. >>>>> >>>>> test/lib/jdk/test/lib/jfr/EventNames.java >>>>> ??? No comments. >>>>> >>>>> src/hotspot/cpu/aarch64/aarch64.ad >>>>> ??? L3517: ????? __ tbnz(tmp, exact_log2(JVM_ACC_IS_BOX_CLASS), >>>>> cont); >>>>> >>>>> ??? L3578: ??? __ bind(cont); >>>>> ??? L3579: ??? // flag == EQ indicates success >>>>> ??? L3580: ??? // flag == NE indicates failure >>>>> ??????? If tbnz() branches to "cont" when we have a box class, what's >>>>> ??????? the flag value set to (EQ or NE)? And what set that flag >>>>> value? >>>>> ??????? The reason I ask is I don't think tbnz() sets condition >>>>> codes... >>>> Right, it doesn't set condition codes, so I kept the version I had >>>> (more on that below). >>>> >>>>> src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp >>>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>>> ??????????? ? ?? ??????????????????????????? Register obj_reg, >>>>> ?????????????????????????????? ? ?? ???????? Register swap_reg, >>>>> ?????????????????????????????????? ? ?? ???? Register tmp_reg, >>>>> ?????????????????????????????????? ? ?? ???? bool >>>>> swap_reg_contains_mark, >>>>> ?????????????????????????????????? ? ?? ???? Label& done, >>>>> ??????????????????????????????????? ? ?? ??? Label* slow_case, >>>>> BiasedLockingCounters* counters) { >>>>> ??????? I think you've changed the only callers of >>>>> biased_locking_enter() >>>>> ??????? that cared about the return value with this changeset so >>>>> it can >>>>> ??????? be changed to a void function. >>>> Ok, this is what I mentioned to David in a previous email. Done. >>>> >>>>> src/hotspot/cpu/arm/c1_MacroAssembler_arm.cpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/cpu/arm/c2_MacroAssembler_arm.cpp >>>>> ??? L96:? ??? tbnz(Rscratch, exact_log2(JVM_ACC_IS_BOX_CLASS), done); >>>>> >>>>> ??? L131: ? bind(done); >>>>> ??? L132: >>>>> ??? L133: ? // At this point flags are set as follows: >>>>> ??? L134: ? //? EQ -> Success >>>>> ??? L135: ? //? NE -> Failure, branch to slow path >>>>> ??????? If tbnz() branches to "done" when we have a box class, what's >>>>> ??????? the flag value set to (EQ or NE)? And what set that flag >>>>> value? >>>>> ??????? The reason I ask is I don't think tbnz() sets condition >>>>> codes... >>>> Right. Same as above, I kept the version I had. >>>> >>>>> src/hotspot/cpu/arm/interp_masm_arm.cpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/cpu/arm/macroAssembler_arm.cpp >>>>> ??? int MacroAssembler::biased_locking_enter(Register obj_reg, >>>>> Register swap_reg, Register tmp_reg, >>>>> ????????????????????? ? ?? ????????????????? bool >>>>> swap_reg_contains_mark, >>>>> ???????????????????????? ? ?? ?????????????? Register tmp2, >>>>> ????????????????????????? ? ?? ????????????? Label& done, Label& >>>>> slow_case, >>>>> BiasedLockingCounters* counters) { >>>>> ??????? I think you've changed the only callers of >>>>> biased_locking_enter() >>>>> ??????? that cared about the return value with this changeset so >>>>> it can >>>>> ??????? be changed to a void function. >>>> Done. >>>> >>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>> ??? No comments on the PPC code. >>>>> >>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>> src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>> ??? No comments on the S390 code. >>>>> >>>>> src/hotspot/cpu/x86/c1_MacroAssembler_x86.cpp >>>>> ??? L58: ??? load_klass(hdr, obj, rklass_decode_tmp); >>>>> ??????? What will this do with a 'noreg' value? (I need a refresher.) >>>> When not in 64 bit mode that register just won't be used. >>>> >>>>> src/hotspot/cpu/x86/c2_MacroAssembler_x86.cpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/cpu/x86/interp_masm_x86.cpp >>>>> ??? No comments. >>>>> >>>>> src/hotspot/cpu/x86/macroAssembler_x86.cpp >>>>> ??? int MacroAssembler::biased_locking_enter(Register lock_reg, >>>>> ????????????????????????????????? ? ?? ????? Register obj_reg, >>>>> ????????????????????????????????? ? ?? ????? Register swap_reg, >>>>> ????????????????????????????????? ? ?? ????? Register tmp_reg, >>>>> ?????????????????????????????????? ? ?? ???? Register tmp_reg2, >>>>> ?????????????????????????????????????? ? ? ? bool >>>>> swap_reg_contains_mark, >>>>> ?????????????????????????????????? ? ?? ???? Label& done, >>>>> ?????????????????????????????????? ? ?? ???? Label* slow_case, >>>>> BiasedLockingCounters* counters) { >>>>> ??????? I think you've changed the only caller of >>>>> biased_locking_enter() >>>>> ??????? that cared about the return value with this changeset so >>>>> it can >>>>> ??????? be changed to a void function. >>>> Done. >>>> >>>>> test/hotspot/jtreg/runtime/Monitor/SyncOnPrimitiveWrapperTest.java >>>>> ??? L30:? * @test SyncOnPrimitiveWrapperTest >>>>> ??????? No parameter to @test directive. >>>> Removed. >>>> >>>>> L136: ??????? private static long sharedCounter = 0L; >>>>> ??????? Since you don't do anything with sharedCounter other than >>>>> increment it, >>>>> ??????? can the compilers optimize it away? If it can be optimized >>>>> away, does >>>>> ??????? that mean that: >>>>> >>>>> ??????????? L142: ??????????????? synchronized (obj) { >>>>> >>>>> ??????? can also be optimized away? >>>>> >>>>> ??????? I don't think that: >>>>> >>>>> ??????????? L161: ??????????????? synchronized (sharedLock1) { >>>>> >>>>> ??????? can be optimized away because it is shared by multiple >>>>> threads. >>>>> >>>>> test/jdk/jdk/jfr/event/runtime/TestSyncOnPrimitiveWrapperEvent.java >>>>> ??? Similar questions about 'counter' being optimized away. >>>>> ??? Similar question about "synchronized (obj) {" being optimized >>>>> away. >>>> I'm not sure if the compiler will optimize it away. Seems unlikely >>>> given the counter we are incrementing is not just local to some >>>> thread. >>>> >>>> >>>> Ok, below is v3 which has the following changes: >>>> - Use a 32 bit load for the _access_flags field, instead of 64 >>>> - Martin's implementation for s390 and fix for PPC >>>> - Faster LogTest version >>>> - Above changes based on Dan review >>>> >>>> I'm retesting in mach5 tiers1-6 (which tests x64 and aarch64 only) >>>> again with -XX:DiagnoseSyncOnPrimitiveWrappers=2. I checked it >>>> builds in arm32, ppc and s390. >>>> >>>> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/webrev >>>> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v3/inc/webrev >>>> >>>> >>>> @Martin: >>>> Please check if the test doesn't timeout for you now with the >>>> changes I made to LogTest. >>>> Also, I tried to use tbnz in aarch64 and arm32 instead of tst + br >>>> (except for c2 since we actually need to set the condition flags), >>>> but for c1 I was getting an assertion in the compiler thread: >>>> >>>> guarantee(chk == -1 || chk == 0) failed: Field too big for insn >>>> >>>> This is the stack when that happens: >>>> >>>> MacroAssembler::pd_patch_instruction_size(unsigned char*, unsigned >>>> char*)+0x398 >>>> AbstractAssembler::bind(Label&)+0x118 >>>> MonitorEnterStub::emit_code(LIR_Assembler*)+0x28 >>>> LIR_Assembler::emit_slow_case_stubs()+0x54 >>>> Compilation::emit_code_body()+0x17c >>>> >>>> The change was simply: >>>> >>>> diff --git a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>> b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>> --- a/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>> +++ b/src/hotspot/cpu/aarch64/c1_MacroAssembler_aarch64.cpp >>>> @@ -78,7 +78,6 @@ >>>> ??? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>> ????? load_klass(hdr, obj); >>>> -??? ldr(hdr, Address(hdr, Klass::access_flags_offset())); >>>> -??? tst(hdr, JVM_ACC_IS_BOX_CLASS); >>>> -??? br(Assembler::NE, slow_case); >>>> +??? ldrw(hdr, Address(hdr, Klass::access_flags_offset())); >>>> +??? tbnz(hdr, exact_log2(JVM_ACC_IS_BOX_CLASS), slow_case); >>>> ??? } >>>> >>>> So the issue must be related to where we want to jump. >>>> >>>> Thanks, >>>> Patricio >>>>> Dan >>>>> >>>>> >>>>> On 8/5/20 9:01 AM, Patricio Chilano wrote: >>>>>> Hi Martin, >>>>>> >>>>>> On 8/5/20 4:47 AM, Doerr, Martin wrote: >>>>>>> Hi Patricio, >>>>>>> >>>>>>> using 8 Byte load instructions for jint fields is a terrible >>>>>>> coding style! >>>>>>> Someone else may see it and use an 8 Byte store. Will result in >>>>>>> great fun for debugging! >>>>>>> >>>>>>> And on Big Endian you will simply access the wrong bits. >>>>>> Ah, of course! Those 32 bits will be in the wrong place when >>>>>> doing the test. >>>>>> >>>>>>> Note that Big Endian Platforms are AIX, old linux ppc, s390, >>>>>>> SPARC. I don't think you have tested on them. >>>>>>> >>>>>>>> We could remove the nested synchronized statements in the run() >>>>>>>> method >>>>>>>> so that we don't generate that much logging. We could also lower >>>>>>>> LOOP_COUNT. I think the issue is also because we are running >>>>>>>> LogTest >>>>>>>> with multiple flag combinations. Not sure what we should touch >>>>>>>> first. >>>>>>>> Maybe the synchronized statements, have only one or two and >>>>>>>> test that >>>>>>>> first? >>>>>>> Sounds like helpful ideas. Please go ahead and strip things down. >>>>>> Great, I will send v3 later with those changes. >>>>>> >>>>>> Thanks Martin! >>>>>> >>>>>> Patricio >>>>>>> Thanks for taking care of it. >>>>>>> >>>>>>> Best regards, >>>>>>> Martin >>>>>>> >>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Patricio Chilano >>>>>>>> Sent: Dienstag, 4. August 2020 20:26 >>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >>>>>>>> wrappers >>>>>>>> >>>>>>>> Hi Martin, >>>>>>>> >>>>>>>> On 8/4/20 1:35 PM, Doerr, Martin wrote: >>>>>>>>> Hi Patricio, >>>>>>>>> >>>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>>> sending v3. >>>>>>>>> Thanks. I forgot to mention arm + aarch64. >>>>>>>>> Aarch64 uses ldrw + tbnz. >>>>>>>>> Arm 32 bit uses ldr_u32 + tbnz. >>>>>>>>> >>>>>>>>> I remember having seen the same mistake ?? >>>>>>>>> And nobody noticed it on little Endian platforms. >>>>>>>> Ok, I can use a tbnz instead of test and then a branch on arm32 >>>>>>>> and aarch64. >>>>>>>> Shouldn't a normal ldr in arm32 work fine? >>>>>>>> Also in 64 bits (either x64 or aarch64) I don't see the issue >>>>>>>> of using a >>>>>>>> 64 bit load, besides the fact that we only care about the first >>>>>>>> 32 bits. >>>>>>>> Regardless of the endianness, aren't we masking out the upper >>>>>>>> part when >>>>>>>> we do AND/TEST or if we test a bit in the 0-31 bit range? >>>>>>>> Otherwise it >>>>>>>> seems it should have failed for one of those platforms in my >>>>>>>> tests. >>>>>>>> >>>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you >>>>>>>>>> can >>>>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>>>> works and I >>>>>>>>>> can add it to the test. >>>>>>>>> That seems to have an effect. But now I'm not patient enough >>>>>>>>> to wait for >>>>>>>> the test to finish. >>>>>>>>> Maybe the problem is that I'm using slow debug builds. >>>>>>>>> But is there a chance to make the test quicker without losing >>>>>>>>> coverage >>>>>>>> significantly? >>>>>>>>> I think the test is too slow this way to run it on a regular >>>>>>>>> basis. We'd need to >>>>>>>> spend dedicated machines for it. >>>>>>>> We could remove the nested synchronized statements in the run() >>>>>>>> method >>>>>>>> so that we don't generate that much logging. We could also lower >>>>>>>> LOOP_COUNT. I think the issue is also because we are running >>>>>>>> LogTest >>>>>>>> with multiple flag combinations. Not sure what we should touch >>>>>>>> first. >>>>>>>> Maybe the synchronized statements, have only one or two and >>>>>>>> test that >>>>>>>> first? >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Patricio >>>>>>>>> Best regards, >>>>>>>>> Martin >>>>>>>>> >>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Patricio Chilano >>>>>>>>>> Sent: Dienstag, 4. August 2020 17:47 >>>>>>>>>> To: Doerr, Martin ; David Holmes >>>>>>>>>> ; hotspot-runtime-dev at openjdk.java.net; >>>>>>>>>> hotspot-jfr-dev at openjdk.java.net >>>>>>>>>> Subject: Re: RFR 8242263: Diagnose synchronization on >>>>>>>>>> primitive wrappers >>>>>>>>>> >>>>>>>>>> Hi Martin, >>>>>>>>>> >>>>>>>>>> Thanks for fixing PPC and taking care of s390! >>>>>>>>>> >>>>>>>>>> On 8/4/20 11:18 AM, Doerr, Martin wrote: >>>>>>>>>>> Hi Patricio, >>>>>>>>>>> >>>>>>>>>>> I suggest to use movl + testl for checking the access flag >>>>>>>>>>> as for other >>>>>>>> access >>>>>>>>>> flags on x64. >>>>>>>>>> Ok, I'll change it to movl + testl and test it out before >>>>>>>>>> sending v3. >>>>>>>>>> >>>>>>>>>>> New version for PPC64 and s390 see below. >>>>>>>>>>> >>>>>>>>>>> The test SyncOnPrimitiveWrapperTest produces hs_err files as >>>>>>>> expected. >>>>>>>>>> However, I'm getting timeout issues: >>>>>>>>>>> timed out (timeout set to 120000ms, elapsed time including >>>>>>>>>>> timeout >>>>>>>>>> handling was 122372ms) >>>>>>>>>>> Can we provide more time? >>>>>>>>>> I see that some tests use @run driver/timeout=xxxx. Maybe you >>>>>>>>>> can >>>>>>>>>> specify that and see if that fixes it? Let me know if that >>>>>>>>>> works and I >>>>>>>>>> can add it to the test. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Patricio >>>>>>>>>>> Best regards, >>>>>>>>>>> Martin >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>> src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp >>>>>>>>>>> --- a/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/c1_MacroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>> @@ -107,8 +107,8 @@ >>>>>>>>>>> >>>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>> ??????? load_klass(Rscratch, Roop); >>>>>>>>>>> -??? ld(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>> Rscratch); >>>>>>>>>>> -??? andi(Rscratch, Rscratch, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>> +??? lwz(Rscratch, in_bytes(Klass::access_flags_offset()), >>>>>>>>>>> Rscratch); >>>>>>>>>>> +??? testbitdi(CCR0, R0, Rscratch, >>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>> ??????? bne(CCR0, slow_case); >>>>>>>>>>> ????? } >>>>>>>>>>> >>>>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp >>>>>>>>>>> --- a/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp Tue Aug 04 >>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>> @@ -912,8 +912,8 @@ >>>>>>>>>>> >>>>>>>>>>> ??????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>> ????????? load_klass(tmp, object); >>>>>>>>>>> -????? ld(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>>>> -????? andi(tmp, tmp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>> +????? lwz(tmp, in_bytes(Klass::access_flags_offset()), tmp); >>>>>>>>>>> +????? testbitdi(CCR0, R0, tmp, >>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>> ????????? bne(CCR0, slow_case); >>>>>>>>>>> ??????? } >>>>>>>>>>> >>>>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/ppc/macroAssembler_ppc.cpp >>>>>>>>>>> --- a/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>> +++ b/src/hotspot/cpu/ppc/macroAssembler_ppc.cpp Tue Aug 04 >>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>> @@ -2838,8 +2838,8 @@ >>>>>>>>>>> >>>>>>>>>>> ????? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>> ??????? load_klass(temp, oop); >>>>>>>>>>> -??? ld(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>>>> -??? andi(temp, temp, JVM_ACC_IS_BOX_CLASS); >>>>>>>>>>> +??? lwz(temp, in_bytes(Klass::access_flags_offset()), temp); >>>>>>>>>>> +??? testbitdi(CCR0, R0, temp, >>>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>> ??????? bne(CCR0, cont); >>>>>>>>>>> ????? } >>>>>>>>>>> >>>>>>>>>>> diff -r 77852e129401 >>>>>>>> src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp >>>>>>>>>>> --- a/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue >>>>>>>>>>> Aug 04 >>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>> +++ b/src/hotspot/cpu/s390/c1_MacroAssembler_s390.cpp Tue >>>>>>>>>>> Aug 04 >>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>> @@ -91,6 +91,12 @@ >>>>>>>>>>> ????? // Save object being locked into the BasicObjectLock... >>>>>>>>>>> ????? z_stg(obj, Address(disp_hdr, >>>>>>>> BasicObjectLock::obj_offset_in_bytes())); >>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>> +??? load_klass(Z_R1_scratch, obj); >>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>> +??? z_btrue(slow_case); >>>>>>>>>>> +? } >>>>>>>>>>> + >>>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>>> ??????? biased_locking_enter(obj, hdr, Z_R1_scratch, >>>>>>>>>>> Z_R0_scratch, done, >>>>>>>>>> &slow_case); >>>>>>>>>>> ????? } >>>>>>>>>>> diff -r 77852e129401 src/hotspot/cpu/s390/interp_masm_s390.cpp >>>>>>>>>>> --- a/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>>>>> 10:03:57 >>>>>>>>>> 2020 +0200 >>>>>>>>>>> +++ b/src/hotspot/cpu/s390/interp_masm_s390.cpp Tue Aug 04 >>>>>>>> 16:04:31 >>>>>>>>>> 2020 +0200 >>>>>>>>>>> @@ -1000,6 +1000,12 @@ >>>>>>>>>>> ????? // Load markWord from object into displaced_header. >>>>>>>>>>> ????? z_lg(displaced_header, >>>>>>>>>>> oopDesc::mark_offset_in_bytes(), object); >>>>>>>>>>> >>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>> +??? load_klass(Z_R1_scratch, object); >>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>> +??? z_btrue(slow_case); >>>>>>>>>>> +? } >>>>>>>>>>> + >>>>>>>>>>> ????? if (UseBiasedLocking) { >>>>>>>>>>> ??????? biased_locking_enter(object, displaced_header, Z_R1, >>>>>>>>>>> Z_R0, done, >>>>>>>>>> &slow_case); >>>>>>>>>>> ????? } >>>>>>>>>>> diff -r 77852e129401 >>>>>>>>>>> src/hotspot/cpu/s390/macroAssembler_s390.cpp >>>>>>>>>>> --- a/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>>>> 10:03:57 2020 +0200 >>>>>>>>>>> +++ b/src/hotspot/cpu/s390/macroAssembler_s390.cpp Tue Aug 04 >>>>>>>>>> 16:04:31 2020 +0200 >>>>>>>>>>> @@ -3358,6 +3358,12 @@ >>>>>>>>>>> ????? // Load markWord from oop into mark. >>>>>>>>>>> ????? z_lg(displacedHeader, 0, oop); >>>>>>>>>>> >>>>>>>>>>> +? if (DiagnoseSyncOnPrimitiveWrappers != 0) { >>>>>>>>>>> +??? load_klass(Z_R1_scratch, oop); >>>>>>>>>>> +??? testbit(Address(Z_R1_scratch, >>>>>>>>>>> Klass::access_flags_offset()), >>>>>>>>>> exact_log2(JVM_ACC_IS_BOX_CLASS)); >>>>>>>>>>> +??? z_btrue(done); >>>>>>>>>>> +? } >>>>>>>>>>> + >>>>>>>>>>> ????? if (try_bias) { >>>>>>>>>>> ??????? biased_locking_enter(oop, displacedHeader, temp, >>>>>>>>>>> Z_R0, done); >>>>>>>>>>> ????? } >>>>>>>>>>> >>>>>> >>>>> >>>> >> From patricio.chilano.mateo at oracle.com Tue Aug 11 22:13:43 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Tue, 11 Aug 2020 19:13:43 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <5f3a48b0-274f-c0f6-57a1-8fbb2bebf823@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <29c3d48b-1875-8ea4-792a-157cdd07fe2c@oracle.com> <5f3a48b0-274f-c0f6-57a1-8fbb2bebf823@oracle.com> Message-ID: <9e6a0427-8d56-3bab-33f6-b2aa64268099@oracle.com> On 8/10/20 9:40 PM, David Holmes wrote: > Hi Patricio, > > On 11/08/2020 6:03 am, Patricio Chilano wrote: >> Hi Erik, >> >> Thanks for looking into this and for the offline discussions. > > If there are still issues with this regarding JFR then I would suggest > we simply drop the use of JFR for reporting this. Ok, I'll wait to see about the JFR changes. > Meanwhile looking at v5 incremental that all seems fine to me. Great, thanks for looking into this David! Thanks, Patricio > Thanks, > David > ----- > >> On 8/10/20 10:19 AM, Erik Gahlin wrote: >>> >>> Hi Patricio, >>> >>> I have tried to review the JFR changes, but I need more information >>> on how the feature is going to be used. >>> >>> Is this something temporary that will be used gain insight for the >>> Valhalla project, or do you think this will be needed longer term? >>> If it is longer term, the bar is higher on what can be accepted. >>> >>> If this is a temporary feature, we could mark the event as >>> experimental in metadata.xml, similar to what we did with GC events >>> during the development of ZGC. Events that are experimental do not >>> show up by default in visualization tools such as JMC and can be >>> removed when they are not needed, or we have a better solution. >>> >> Right, it's temporary so we can mark it as experimental. I also >> changed the category name to be "Java Virtual Machine, Diagnostics". >> >>> For events to be enabled in default.jfc, they should not cause more >>> than 1% overhead, not even in pathological cases. To me, it seems >>> this could happen if you make a loop where it is triggered all the >>> time. >>> >>> For events in profile.jfc, the overhead should still be low (1-2%), >>> but the target is the typical application. For example, allocation >>> profiling is only enabled in profile.jfc. Some application that are >>> allocation intensive could cause a higher overhead than 1%, but that >>> is OK because that configuration is only to be used for a short >>> period of time. That said, there is still a budget on how much space >>> in a recording an event can take up, so it also depends on how >>> important the information is for the user. To me, it seems this >>> event will not be that important for the average user, which makes >>> me think it should be disabled by default. >>> >> After our off-list discussion I left it as enabled. I also removed >> the threshold setting and added the startTime=false in metadata.xml. >> >>> To complicate things, I noticed that aa command line flag is also >>> used to enable the event. This is something we have worked hard to >>> avoid when it comes to JFR events. All configuration should happen >>> using configuration files. We had problem with this in the past >>> where users gets confused why their events are not enabled. When we >>> ported JFR to Hotspot, we got rid of those of those flags/events and >>> it is a much better situation. Again, if this a temporary event to >>> discover usage pattern for a release or two, it might be OK, but if >>> we believe this event to stick around, I think we should look into >>> alternatives, or not use JFR at all for this. >>> >> Yes, I see your point. Since we want to diagnose synchronization on >> these classes but we don't want to affect performance the flag has to >> be specified at startup otherwise the diagnostic instrumentation is >> disabled, so I don't see a clear way to enable the events from JFR >> alone. In any case, this will be a temporary event. >> >> Here is v5. Let me know if you are okay with the changes. >> >> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ >> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/webrev/ >> >> Thanks Erik! >> >> Patricio >>> >>> Thanks >>> >>> Erik >>> >>>> On 28 Jul 2020, at 21:16, Patricio Chilano >>>> >>> > wrote: >>>> >>>> Hi all, >>>> >>>> Please review the following change that adds diagnostic >>>> capabilities when synchronizing on primitive wrapper classes. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8242263 >>>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8242263/v1/webrev/ >>>> >>>> The new flag allows to identify synchronization on these classes >>>> and to take one of the following actions: exit the VM with fatal >>>> error, log a warning message, or issue a JFR event. The >>>> implementation uses a simple approach where a check is added at the >>>> beginning of the monitorenter generated code when the flag is >>>> enabled to check whether the object is of a primitive wrapper >>>> class. If it is, we jump to the slow path, otherwise we just >>>> continue as always. The extra instructions will be: load the klass >>>> of the object, load the _access_flags field for that klass, AND >>>> with a constant, and branch based on the result. The code will only >>>> be generated whenever the new opt-in diagnostic flag is enabled so >>>> performance won't be affected when off. >>>> >>>> In addition to the purpose described in the description of the bug, >>>> this flag will also be useful when trying to diagnose possible >>>> synchronization issues if these classes ever become inline types as >>>> part of the Valhalla project. >>>> >>>> I added test SyncOnPrimitiveWrapperTest.java that tests for the >>>> exit and logging cases. I added test >>>> TestSyncOnPrimitiveWrapperEvent.java to test for the JFR event case. >>>> I tested the patch running tiers1-6 in mach5 with the flag set to >>>> DiagnoseSyncOnPrimitiveWrappers=2. >>>> I checked it builds with arm32 and ppc but can't run any tests on >>>> those platforms, so it would be good if somebody can run the new >>>> test included in the patch. >>>> >>>> Let me know if you think I should run or add any more tests. >>>> >>>> Thanks! >>>> >>>> Patricio >>> >> From david.holmes at oracle.com Tue Aug 11 22:54:03 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Aug 2020 08:54:03 +1000 Subject: RFR (s) 8251383: Disable Event::log from linux_mprotect when processing the assertion poison page In-Reply-To: <780d9243-67d5-d70e-bda3-ace37316eb7f@oracle.com> References: <4f20d795-fea6-282b-cc96-2a91e0665a53@oracle.com> <780d9243-67d5-d70e-bda3-ace37316eb7f@oracle.com> Message-ID: <220c599e-ff29-d4c6-085b-13057b892175@oracle.com> Thanks for the review Dan! David On 12/08/2020 2:22 am, Daniel D. Daugherty wrote: > On 8/11/20 5:17 AM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8251383 >> webrev: http://cr.openjdk.java.net/~dholmes/8251383/webrev/ > > src/hotspot/os/linux/os_linux.cpp > ??? No comments. > > Thumbs up. > > Dan > >> >> When the assertion poison page is enabled (Linux only and on by >> default) the signal handler will call os::protect_memory to change the >> page protection bits. That will call linux_mprotect which will call >> >> Events::log(NULL, "Protecting memory [" INTPTR_FORMAT "," >> INTPTR_FORMAT "] with protection modes %x", p2i(bottom), >> p2i(bottom+size), prot); >> >> Event logging in turn can use Mutexes and other VM facilities - all of >> which are now being executed in a signal handling context (which is >> inherently unsafe). It also means that there cannot be any other >> failing assertions on that path as you will re-trigger the poison page >> pagefault and abort with no hs_err file. Hence, as happened to me, a >> failing assertion in the mutex code triggers this problem. >> >> The issue can be worked-around by setting -XX:-ShowRegistersOnAssert >> (once you realise what is happening). >> >> The simple fix is to skip the logging if the faulting address is the >> poison page address. >> >> This only affects debug builds of course. >> >> Testing: >> ? - runtime/ErrorHandling >> ? - tier 1-3 >> >> Thanks, >> David > From david.holmes at oracle.com Wed Aug 12 00:36:01 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Aug 2020 10:36:01 +1000 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: <750e129d-40c7-d205-5352-2f4582475c83@oracle.com> References: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> <3c9ffc2b-51e8-272c-f03b-d5faf5036bf6@oracle.com> <51f4bbe4-0a19-b09c-c544-5681afb2aa1d@oracle.com> <8f0f8423-f46b-9b78-1544-24f02a8606f7@oracle.com> <750e129d-40c7-d205-5352-2f4582475c83@oracle.com> Message-ID: +1 on Coleen's comment about still including handles.hpp +1 on changes looking good. Thanks, David On 12/08/2020 6:51 am, Coleen Phillimore wrote: > > > On 8/11/20 4:45 PM, Patricio Chilano wrote: >> >> On 8/11/20 5:34 PM, Daniel D. Daugherty wrote: >>> On 8/11/20 4:27 PM, Patricio Chilano wrote: >>>> Hi Dan, >>>> >>>> Thanks for looking at this. >>>> >>>> On 8/11/20 4:53 PM, Daniel D. Daugherty wrote: >>>>> On 8/11/20 2:11 PM, Patricio Chilano wrote: >>>>>> Hi all, >>>>>> >>>>>> Please review the following small fix which involves removing the >>>>>> added HandleMark in BiasedLocking::preserve_marks(): >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 >>>>>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ >>>>> >>>>> src/hotspot/share/runtime/biasedLocking.cpp >>>>> ??? No comments. >>>>> >>>>> As mentioned by Harold, adding a new test case to: >>>>> >>>>> ??? test/hotspot/jtreg/runtime/7158988/FieldMonitor.java >>>>> >>>>> that enables '-XX:+UseBiasedLocking' is a good idea. >>>> I suggested adding it to test TestFullGCALot.java instead because it >>>> fails much more consistently. >>> >>> I'm good with that also. > > Me too, even better than FieldMonitor.java if it's more reproduceable. > >>> >>> >>>> >>>>> Also, Coleen added this note to your bug report: >>>>> >>>>>> Also in the patch for JDK-8249192 >>>>>> , vframe.hpp >>>>>> shouldn't include >>>>>> handles.inline.hpp (only cpp and inline.hpp files can include >>>>>> inline.hpp). It doesn't look like it needs handles.inline.hpp, >>>>>> only handles.hpp. >>>>> >>>>> Were you planning on resolving that left over in this bug? >>>> Not really. I can change it here if you are okay though. >>> >>> Your call on whether to fix it here or with a new bug. Seems >>> like fixing it here is less work... this bug is a follow-on to >>> the original and that comment is a follow-on to the original... >> Yeah, sounds good to me to remove the extra header in this change. >> I'll check it builds fine on all platforms first. >> Below is v2 in case you wanted to see it: >> >> Full: http://cr.openjdk.java.net/~pchilanomate/8251118/v2/webrev/ >> Inc: http://cr.openjdk.java.net/~pchilanomate/8251118/v2/inc/webrev/ > > I think vframe.hpp should include handles.hpp because there are Handle > types in it, just in case something changes and it's not transiently > included anymore. > > Otherwise, everything looks good.? I don't need to see another webrev. > > thanks! > Coleen >> >> Thanks Dan! >> >> Patricio >>> Dan >>> >>> >>>> >>>> Patricio >>>>> Dan >>>>> >>>>> >>>>>> >>>>>> I've inspected the callers of BiasedLocking::preserve_marks() and >>>>>> they all have an assert that the current thread is the VMThread. >>>>>> Since the VMThread creates a HandleMark object before executing a >>>>>> VM operation the extra HandleMark added in 8249192 is not needed. >>>>>> I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without >>>>>> the patch I get several crashes in BiasedLocking::restore_marks(). >>>>>> With the patch tests completed successfully. >>>>>> >>>>>> Thanks, >>>>>> Patricio >>>>>> >>>>> >>>> >>> >> > From suenaga at oss.nttdata.com Wed Aug 12 00:44:33 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 12 Aug 2020 09:44:33 +0900 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> Message-ID: <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> PING: Could you review this change? JBS: https://bugs.openjdk.java.net/browse/JDK-8250598 webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/ Build change has been reviewed by Erik. Thanks, Yasumasa On 2020/08/05 18:30, Yasumasa Suenaga wrote: > Hi Matthias, > (CC'ed build-dev due to makefile change) > > I uploaded new webrev. Could you review it? > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/ > > On 2020/08/05 16:26, Baesken, Matthias wrote: >> Hello, >> >>>> I also checked the? hserr on another Hyper-V guest and still get the expected " HyperV virtualization detected"?? (which is good ?? ). >>>> >>>> >>>> However on a host, I would still prefer to get some output (probably different than the guest output now) . >>> >>> What message do you expect? How about "Hyper-V role detected" ? >> >> Thats okay with me? (I think some people here say "Hyper-V host" so that might be an idea as well). > > I said "Hyper-V host" in previous mails, but it is not strictly correct. In Hyper-V, it should be "root partision". However it's a little difficult to understand, so I choose "role" in this place. > > >>>> Regarding the usage of >>>> >>>> #pragma comment(lib, "wbemuuid.lib") >>>> >>>> This seem to add an additional lib dependency ,? should some people from build/deploy? area comment on this maybe ? >>>> (not saying it is a very bad thing ) > > I modified libraries.m4. > wbemuuid.lib is needed for WMI call. > > > Thanks, > > Yasumasa > > >>> We add wbemuuid.lib to make/autoconf/libraries.m4 . >>> I will change it in next webrev, and I will CCing to build-dev on next turn. >> >> >> Let's see what they have to say. >> >> Best regards, Matthias >> >> From david.holmes at oracle.com Wed Aug 12 02:20:34 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Aug 2020 12:20:34 +1000 Subject: RFR(S) 8249276 CDS archived objects must have "neutral" markwords In-Reply-To: <352a476d-685c-4530-994b-0e89593ef261@oracle.com> References: <5d5b4b17-a40f-b0f4-fa50-fdd783def9d4@oracle.com> <37245599-dfb8-0481-6c74-89cffdc5b457@oracle.com> <7a81547c-8221-da80-2d1a-edb42f4078fc@oracle.com> <11f16292-909d-6d61-d353-cfa9d7054506@oracle.com> <352a476d-685c-4530-994b-0e89593ef261@oracle.com> Message-ID: Hi Ioi, But one further follow up in relation to biased-locking code. This assertion is invalid: 104 unsigned HeapShared::oop_hash(oop const& p) { 105 assert(!p->mark().has_bias_pattern(), 106 "this object should never have been locked"); // so identity_hash won't safepoin 107 unsigned hash = (unsigned)p->identity_hash(); 108 return hash; 109 } As I discovered having the "bias pattern" doesn't mean you have been locked, it can mean you're eligible to be biased-locked (ie you are anonymously biased). So if you enable BL for CDS dump (and fix the asserts I'm working on) you will still hit this assertion. Cheers, David On 11/08/2020 7:31 pm, David Holmes wrote: > Hi Ioi, > > Okay nothing further from me on the test. > > Thanks, > David > > On 11/08/2020 3:29 pm, Ioi Lam wrote: >> >> >> On 8/6/20 5:55 PM, David Holmes wrote: >>> Hi Ioi, >>> >>> On 6/08/2020 8:26 am, Ioi Lam wrote: >>>> Hi David & Coleen, >>>> >>>> Thanks for the review. I have created a new version: >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ >>>> >>> >>> Functional changes are good. >>> >>> For the test, as per our offline chat, the Helper class is not being >>> archived. This line: >>> >>> ?66???????????????? TestCommon.testDump(appJar, >>> TestCommon.list(LockDuringDumpApp.class.getName()), >>> >>> should be >>> >>> ?66???????????????? TestCommon.testDump(appJar, >>> TestCommon.list(appClasses), >>> >>> >> >> Hi David, >> >> (The Helper class was actually archived, because it was loaded by >> LockDuringDumpAgent during dump time, so it was not necessary to >> explicitly include it in the class list. ....) >> >> Anyway, it wasn't clear to me whether any code (now or in the future) >> in the JVM might synchronize on the java mirrors. So I've updated the >> test case to synchronize on a literal string instead. This way I know >> that only the test, and no one else, will synchronize on it. >> >> I also added comments to explain how the test works. >> >> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03/ >> >> >> Thanks >> - Ioi >> >> >>>> >>>> I created a new test case (LockDuringDump), and incorporated David's >>>> fix for JDK-8250606 [1] during my testing: >>>> >>>> + Without my patch, LockDuringDumpApp would crash at runtime, because >>>> ?? we have an invalid markword for the Helper.class object. >>>> >>>> + Without the JDK-8250606 patch, we would assert during CDS dump time. >>>> >>>> I also removed the assert in HeapShared::oop_hash() since it's >>>> irrelevant after JDK-8250606. >>>> >>>> I intend to push my patch after JDK-8250606 is integrated. >>>> >>>> >>>> ------- >>>> >>>> More comments below >>>> >>>> On 8/3/20 6:13 PM, David Holmes wrote: >>>>> Hi Ioi, >>>>> >>>>> On 27/07/2020 3:20 pm, Ioi Lam wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8249276 >>>>>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v01/ >>>>>> >>>>>> >>>>>> Please review this change (initial patch provided by David Holmes; >>>>>> I rearranged it a little and added more comments). >>>>>> >>>>>> We reinitialize the markWord of all archived objects to remove any >>>>>> side effect of GC/locking/etc during "java -Xshare:dump". >>>>> >>>>> Changes look good to me (naturally :) ). A couple of nits: >>>>> >>>>> ?? int hash_archived = archived_oop->identity_hash(); >>>>> ?? assert(hash_original == hash_archived, "Different hash codes: >>>>> original %x, archived %x", hash_original, hash_archived); >>>>> >>>>> the has_archived setting should be in ifdef DEBUG. >>>>> >>>> >>>> Fixed >>>> >>>> >>>>> 42 Object.class.wait(); >>>>> ? 43???????????????? } >>>>> ? 44???????????????? System.out.println("Huh?? notified??"); >>>>> >>>>> That could be a spurious wakeup. You could just use Thread.sleep >>>>> with a suitably long sleep time. that said I'm not sure how the >>>>> test is intended to operate. If premain returns after starting the >>>>> thread there is no guarantee the thread will start executing and >>>>> lock the object before the dump actually commences. Even the >>>>> sighting of "Let's wait ....." is not sufficient to guarantee that >>>>> unless you move it inside the sync block. >>>>> >>>> >>>> I have fixed the test case so that the "Let's hold the lock" message >>>> is printed after the lock is held, and the lock is never released >>>> afterwards. >>>> >>>> I also added synchronization to make sure the premain() method >>>> doesn't return until the child thread has held the lock. The log >>>> file shows that theses messages are printed before CDS tries to load >>>> classes from the classlist: >>>> >>>> [0.005s][info][cds] Allocated shared space: 3221225472 bytes at >>>> 0x0000000800000000 >>>> inside LockDuringDumpAgent: >>>> jdk.internal.loader.ClassLoaders$AppClassLoader at 707f7052 >>>> Helper class is initialized >>>> Class = class LockDuringDumpApp$Helper >>>> Let's hold the lock on Helper.class forever ..... >>>> Thread has started >>>> [0.145s][info][cds] Loading classes to share ... >>>> >>>>>> (David mentioned in the bug comments about assertions inside the >>>>>> identity_hash() call, but maybe this should be fixed in a >>>>>> different bug?) >>>>> >>>>> Yes those assertions are being removed under: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8250606 >>>>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> ------- >>>> >>>> [1] >>>> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/040985.html >>>> >>>> >>>> >> From david.holmes at oracle.com Wed Aug 12 04:23:40 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Aug 2020 14:23:40 +1000 Subject: RFR (s): 8251460: Fix the biased-locking code in ObjectSynchronizer::FastHashCode Message-ID: <797db6f1-e080-3a33-fefa-34d045a9c681@oracle.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8251460 webrev: http://cr.openjdk.java.net/~dholmes/8251460/webrev/ During the review for JDK-8250606 it was noted that the biased-locking section of FastHashCode contained a similar assertion regarding execution at a safepoint: // Relaxing assertion for bug 6320749. assert(Universe::verify_in_progress() || !SafepointSynchronize::is_at_safepoint(), "biases should not be seen by VM thread here"); which gives the impression that it is okay to execute this code block at a safepoint if verify_in_progress() is true. But if we examine the next line of code: BiasedLocking::revoke(hobj, JavaThread::current()); we find that at a safepoint: a) JavaThread::current() will assert because the current thread will be the VMThread; and b) BiasedLocking::revoke itself contains an assertion that it is not invoked at a safepoint. In practice there is no issue with revoking the bias at a safepoint, we just have to use the right code to do so (as done elsewhere in the same file): if (SafepointSynchronize::is_at_safepoint()) { BiasedLocking::revoke_at_safepoint(hobj); } else { BiasedLocking::revoke(hobj, self); } Thanks, David From ioi.lam at oracle.com Wed Aug 12 05:08:04 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 11 Aug 2020 22:08:04 -0700 Subject: RFR(S) 8249276 CDS archived objects must have "neutral" markwords In-Reply-To: References: <5d5b4b17-a40f-b0f4-fa50-fdd783def9d4@oracle.com> <37245599-dfb8-0481-6c74-89cffdc5b457@oracle.com> <7a81547c-8221-da80-2d1a-edb42f4078fc@oracle.com> <11f16292-909d-6d61-d353-cfa9d7054506@oracle.com> <352a476d-685c-4530-994b-0e89593ef261@oracle.com> Message-ID: Hi David, As we discussed off-line, I pushed the fix as-it. I'll fix the assert you mentioned below after 8251460 (Fix the biased-locking code in ObjectSynchronizer::FastHashCode) is integrated. Thanks - Ioi On 8/11/20 7:20 PM, David Holmes wrote: > Hi Ioi, > > But one further follow up in relation to biased-locking code. This > assertion is invalid: > > ?104 unsigned HeapShared::oop_hash(oop const& p) { > ?105?? assert(!p->mark().has_bias_pattern(), > ?106????????? "this object should never have been locked");? // so > identity_hash won't safepoin > ?107?? unsigned hash = (unsigned)p->identity_hash(); > ?108?? return hash; > ?109 } > > As I discovered having the "bias pattern" doesn't mean you have been > locked, it can mean you're eligible to be biased-locked (ie you are > anonymously biased). So if you enable BL for CDS dump (and fix the > asserts I'm working on) you will still hit this assertion. > > Cheers, > David > > On 11/08/2020 7:31 pm, David Holmes wrote: >> Hi Ioi, >> >> Okay nothing further from me on the test. >> >> Thanks, >> David >> >> On 11/08/2020 3:29 pm, Ioi Lam wrote: >>> >>> >>> On 8/6/20 5:55 PM, David Holmes wrote: >>>> Hi Ioi, >>>> >>>> On 6/08/2020 8:26 am, Ioi Lam wrote: >>>>> Hi David & Coleen, >>>>> >>>>> Thanks for the review. I have created a new version: >>>>> >>>>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v02/ >>>>> >>>> >>>> Functional changes are good. >>>> >>>> For the test, as per our offline chat, the Helper class is not >>>> being archived. This line: >>>> >>>> ?66???????????????? TestCommon.testDump(appJar, >>>> TestCommon.list(LockDuringDumpApp.class.getName()), >>>> >>>> should be >>>> >>>> ?66???????????????? TestCommon.testDump(appJar, >>>> TestCommon.list(appClasses), >>>> >>>> >>> >>> Hi David, >>> >>> (The Helper class was actually archived, because it was loaded by >>> LockDuringDumpAgent during dump time, so it was not necessary to >>> explicitly include it in the class list. ....) >>> >>> Anyway, it wasn't clear to me whether any code (now or in the >>> future) in the JVM might synchronize on the java mirrors. So I've >>> updated the test case to synchronize on a literal string instead. >>> This way I know that only the test, and no one else, will >>> synchronize on it. >>> >>> I also added comments to explain how the test works. >>> >>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v03/ >>> >>> >>> Thanks >>> - Ioi >>> >>> >>>>> >>>>> I created a new test case (LockDuringDump), and incorporated >>>>> David's fix for JDK-8250606 [1] during my testing: >>>>> >>>>> + Without my patch, LockDuringDumpApp would crash at runtime, because >>>>> ?? we have an invalid markword for the Helper.class object. >>>>> >>>>> + Without the JDK-8250606 patch, we would assert during CDS dump >>>>> time. >>>>> >>>>> I also removed the assert in HeapShared::oop_hash() since it's >>>>> irrelevant after JDK-8250606. >>>>> >>>>> I intend to push my patch after JDK-8250606 is integrated. >>>>> >>>>> >>>>> ------- >>>>> >>>>> More comments below >>>>> >>>>> On 8/3/20 6:13 PM, David Holmes wrote: >>>>>> Hi Ioi, >>>>>> >>>>>> On 27/07/2020 3:20 pm, Ioi Lam wrote: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8249276 >>>>>>> http://cr.openjdk.java.net/~iklam/jdk16/8249276-reset-archive-obj-headers.v01/ >>>>>>> >>>>>>> >>>>>>> Please review this change (initial patch provided by David >>>>>>> Holmes; I rearranged it a little and added more comments). >>>>>>> >>>>>>> We reinitialize the markWord of all archived objects to remove >>>>>>> any side effect of GC/locking/etc during "java -Xshare:dump". >>>>>> >>>>>> Changes look good to me (naturally :) ). A couple of nits: >>>>>> >>>>>> ?? int hash_archived = archived_oop->identity_hash(); >>>>>> ?? assert(hash_original == hash_archived, "Different hash codes: >>>>>> original %x, archived %x", hash_original, hash_archived); >>>>>> >>>>>> the has_archived setting should be in ifdef DEBUG. >>>>>> >>>>> >>>>> Fixed >>>>> >>>>> >>>>>> 42 Object.class.wait(); >>>>>> ? 43???????????????? } >>>>>> ? 44???????????????? System.out.println("Huh?? notified??"); >>>>>> >>>>>> That could be a spurious wakeup. You could just use Thread.sleep >>>>>> with a suitably long sleep time. that said I'm not sure how the >>>>>> test is intended to operate. If premain returns after starting >>>>>> the thread there is no guarantee the thread will start executing >>>>>> and lock the object before the dump actually commences. Even the >>>>>> sighting of "Let's wait ....." is not sufficient to guarantee >>>>>> that unless you move it inside the sync block. >>>>>> >>>>> >>>>> I have fixed the test case so that the "Let's hold the lock" >>>>> message is printed after the lock is held, and the lock is never >>>>> released afterwards. >>>>> >>>>> I also added synchronization to make sure the premain() method >>>>> doesn't return until the child thread has held the lock. The log >>>>> file shows that theses messages are printed before CDS tries to >>>>> load classes from the classlist: >>>>> >>>>> [0.005s][info][cds] Allocated shared space: 3221225472 bytes at >>>>> 0x0000000800000000 >>>>> inside LockDuringDumpAgent: >>>>> jdk.internal.loader.ClassLoaders$AppClassLoader at 707f7052 >>>>> Helper class is initialized >>>>> Class = class LockDuringDumpApp$Helper >>>>> Let's hold the lock on Helper.class forever ..... >>>>> Thread has started >>>>> [0.145s][info][cds] Loading classes to share ... >>>>> >>>>>>> (David mentioned in the bug comments about assertions inside the >>>>>>> identity_hash() call, but maybe this should be fixed in a >>>>>>> different bug?) >>>>>> >>>>>> Yes those assertions are being removed under: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8250606 >>>>>> >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> ------- >>>>> >>>>> [1] >>>>> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/040985.html >>>>> >>>>> >>>>> >>> From nils.eliasson at oracle.com Wed Aug 12 08:21:58 2020 From: nils.eliasson at oracle.com (Nils Eliasson) Date: Wed, 12 Aug 2020 10:21:58 +0200 Subject: RFR(S): 8247732: validate user-input intrinsic_ids in ControlIntrinsic In-Reply-To: <1597165750921.4285@amazon.com> References: <821e3d29-c95b-aafc-8ee5-6e49a1bdde82@amazon.com> <9b324805-eb86-27e1-5dcb-96a823f8495b@amazon.com> <82cba5e4-2020-ce0a-4576-e8e0cc2e5ae5@oracle.com> <1595401959932.33284@amazon.com> <1595520162373.22868@amazon.com> <916b3a4a-5617-941d-6161-840f3ea900bd@oracle.com> <1596523192072.15354@amazon.com> <1597165750921.4285@amazon.com> Message-ID: Hi, Sorry for the delay. About the error handling: For CompilerDirectivesFile there are two scenarios: 1) If a file containing bad contents is passed on the commandline - the VM prints an descriptive error and refuses to start. 2) If a file containing bad contents is passed through jcmd - the VM prints and error on the jcmd stream and continues to run (ignoring the command). This is achieved by letting the parser just register any parsing error, and defer to the caller to decide how to handle the situation. Regards, Nils Eliasson On 2020-08-11 19:09, Liu, Xin wrote: > Hi, Reviewers, > > May I gently ping this? > > I stuck because I don't know which error handling is appropriate. > > If we do nothing, current hotspot ignores wrong intrinsic Ids in the cmdline. > This patch aborts hotspot when it detects any invalid intrinsic id. > > thanks, > --lx > > > ________________________________________ > From: Liu, Xin > Sent: Monday, August 3, 2020 11:39 PM > To: Tobias Hartmann; Nils Eliasson; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev > Subject: Re: [EXTERNAL] RFR(S): 8247732: validate user-input intrinsic_ids in ControlIntrinsic > > hi, Nils, > > Tobias would like to keep the parser behavior consistency. I think it means that the hotspot need to suppress the warning if the intrinsic_id doesn't exists in compiler directive. > eg. -XX:CompileCommand=option,,ControlIntrinsic=-_nonexist. > > What do you think about it? > > Here is the latest webrev: > http://cr.openjdk.java.net/~xliu/8247732/01/webrev/ > > thanks, > --lx > > ________________________________________ > From: Tobias Hartmann > Sent: Friday, July 24, 2020 2:52 AM > To: Liu, Xin; Nils Eliasson; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev > Subject: RE: [EXTERNAL] RFR(S): 8247732: validate user-input intrinsic_ids in ControlIntrinsic > > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. > > > > Hi Liu, > > On 23.07.20 18:02, Liu, Xin wrote: >> That is my intention too, but CompilerOracle doesn't exit JVM when it encounters parsing errors. >> It just exacts information from CompileCommand as many as possible. That makes sense because compiler "directives" are supposed to be optional for program execution. >> >> I do put the error message in parser's errorbuf. I set a flag "exit_on_error" to quit JVM after it dumps parser errors. yes, I treat undefined intrinsics as fatal errors. >> This behavior is from Nils comment: "I want to see an error on startup if the user has specified unknown intrinsic names." It is also consistent with JVM option -XX:ControlIntrinsic=. > Okay, thanks for the explanation! I would prefer consistency in error handling of compiler > directives, i.e., handle all parser failures the same way. But I leave it to Nils to decide. > > Best regards, > Tobias From matthias.baesken at sap.com Wed Aug 12 10:51:49 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 12 Aug 2020 10:51:49 +0000 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> Message-ID: Hi Yasumasa , I'm more or less fine with the change . But still not fully convinced that removing the iteration is a good thing . http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/src/hotspot/cpu/x86/vm_version_x86.cpp.frames.html 1827 for (base = 0x40000000; base < 0x40010000; base += 0x100) { 1828 check_virt_cpuid(base, registers); I think just checking "0x40000000" should work in most cases but if I remember correctly sometimes it was not enough . See also some references about Xen/KVM : https://lists.linuxfoundation.org/pipermail/virtualization/2012-May/019974.html "If compat mode for another h/v is enabled then those leaves will appear at 0x40000000 and Xen's will be bumped up, so a fully Xen aware set of drivers (or detection routine, etc) should check at 0x100 intervals until 0x40010000 " and https://lore.kernel.org/patchwork/patch/394371/ And not so happy about the WMI usage (called in early JVM startup) : http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/src/hotspot/os_cpu/windows_x86/vm_version_windows_x86.cpp.frames.html bool VM_Version::is_in_VM() { ... } ... } But if noone else complains about it, I guess it's okay . Best regards, Matthias >PING: Could you review this change? > > JBS: https://bugs.openjdk.java.net/browse/JDK-8250598 > webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/ > >Build change has been reviewed by Erik. > From coleen.phillimore at oracle.com Wed Aug 12 11:58:27 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 12 Aug 2020 07:58:27 -0400 Subject: RFR (s): 8251460: Fix the biased-locking code in ObjectSynchronizer::FastHashCode In-Reply-To: <797db6f1-e080-3a33-fefa-34d045a9c681@oracle.com> References: <797db6f1-e080-3a33-fefa-34d045a9c681@oracle.com> Message-ID: Looks good! Coleen On 8/12/20 12:23 AM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8251460 > webrev: http://cr.openjdk.java.net/~dholmes/8251460/webrev/ > > During the review for JDK-8250606 it was noted that the biased-locking > section of FastHashCode contained a similar assertion regarding > execution at a safepoint: > > ????? // Relaxing assertion for bug 6320749. > ????? assert(Universe::verify_in_progress() || > ???????????? !SafepointSynchronize::is_at_safepoint(), > ???????????? "biases should not be seen by VM thread here"); > > which gives the impression that it is okay to execute this code block > at a safepoint if verify_in_progress() is true. But if we examine the > next line of code: > > ? BiasedLocking::revoke(hobj, JavaThread::current()); > > we find that at a safepoint: > a) JavaThread::current() will assert because the current thread will > be the VMThread; and > b) BiasedLocking::revoke itself contains an assertion that it is not > invoked at a safepoint. > > In practice there is no issue with revoking the bias at a safepoint, > we just have to use the right code to do so (as done elsewhere in the > same file): > > if (SafepointSynchronize::is_at_safepoint()) { > ? BiasedLocking::revoke_at_safepoint(hobj); > } else { > ? BiasedLocking::revoke(hobj, self); > } > > Thanks, > David From david.holmes at oracle.com Wed Aug 12 12:12:23 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Aug 2020 22:12:23 +1000 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> Message-ID: <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> On 12/08/2020 8:51 pm, Baesken, Matthias wrote: > Hi Yasumasa , I'm more or less fine with the change . > But still not fully convinced that removing the iteration is a good thing . > > http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/src/hotspot/cpu/x86/vm_version_x86.cpp.frames.html > > 1827 for (base = 0x40000000; base < 0x40010000; base += 0x100) { > 1828 check_virt_cpuid(base, registers); > > I think just checking "0x40000000" should work in most cases but if I remember correctly sometimes it was not enough . > See also some references about Xen/KVM : > > https://lists.linuxfoundation.org/pipermail/virtualization/2012-May/019974.html > > "If compat mode for another h/v is enabled then those leaves will appear > at 0x40000000 and Xen's will be bumped up, so a fully Xen aware set of > drivers (or detection routine, etc) should check at 0x100 intervals > until 0x40010000 " > > and > > https://lore.kernel.org/patchwork/patch/394371/ > > > > And not so happy about the WMI usage (called in early JVM startup) : > > http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/src/hotspot/os_cpu/windows_x86/vm_version_windows_x86.cpp.frames.html > > bool VM_Version::is_in_VM() { ... } > ... > } > > But if noone else complains about it, I guess it's okay . I haven't reviewed this because I don't understand any of the code. But all that extra stuff during VM initialization doesn't look good to me. We do not want to pay a startup penalty just because we want a more accurate hypervisor listing in an error report! Can we not determine this only if needed? (Granted doing all this stuff during crash reporting may also be an issue.) David ----- > > Best regards, Matthias > > >> PING: Could you review this change? >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8250598 >> webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/ >> >> Build change has been reviewed by Erik. >> > From david.holmes at oracle.com Wed Aug 12 12:16:20 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Aug 2020 22:16:20 +1000 Subject: RFR (s): 8251460: Fix the biased-locking code in ObjectSynchronizer::FastHashCode In-Reply-To: References: <797db6f1-e080-3a33-fefa-34d045a9c681@oracle.com> Message-ID: Thanks Coleen! I forgot to mention that the testing for this was to reenable (in arguments.cpp) biased-locking for CDS (and fix an errant assert that Ioi is going to fix) and run the CDS tests. None of those changes will be pushed of course. David On 12/08/2020 9:58 pm, Coleen Phillimore wrote: > Looks good! > Coleen > > On 8/12/20 12:23 AM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8251460 >> webrev: http://cr.openjdk.java.net/~dholmes/8251460/webrev/ >> >> During the review for JDK-8250606 it was noted that the biased-locking >> section of FastHashCode contained a similar assertion regarding >> execution at a safepoint: >> >> ????? // Relaxing assertion for bug 6320749. >> ????? assert(Universe::verify_in_progress() || >> ???????????? !SafepointSynchronize::is_at_safepoint(), >> ???????????? "biases should not be seen by VM thread here"); >> >> which gives the impression that it is okay to execute this code block >> at a safepoint if verify_in_progress() is true. But if we examine the >> next line of code: >> >> ? BiasedLocking::revoke(hobj, JavaThread::current()); >> >> we find that at a safepoint: >> a) JavaThread::current() will assert because the current thread will >> be the VMThread; and >> b) BiasedLocking::revoke itself contains an assertion that it is not >> invoked at a safepoint. >> >> In practice there is no issue with revoking the bias at a safepoint, >> we just have to use the right code to do so (as done elsewhere in the >> same file): >> >> if (SafepointSynchronize::is_at_safepoint()) { >> ? BiasedLocking::revoke_at_safepoint(hobj); >> } else { >> ? BiasedLocking::revoke(hobj, self); >> } >> >> Thanks, >> David > From coleen.phillimore at oracle.com Wed Aug 12 12:19:56 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 12 Aug 2020 08:19:56 -0400 Subject: RFR (s): 8251460: Fix the biased-locking code in ObjectSynchronizer::FastHashCode In-Reply-To: References: <797db6f1-e080-3a33-fefa-34d045a9c681@oracle.com> Message-ID: <587d84f4-2579-b70e-0ce4-4891d3df7147@oracle.com> On 8/12/20 8:16 AM, David Holmes wrote: > Thanks Coleen! > > I forgot to mention that the testing for this was to reenable (in > arguments.cpp) biased-locking for CDS (and fix an errant assert that > Ioi is going to fix) and run the CDS tests. None of those changes will > be pushed of course. Yes, it's good to know that CDS dumping could turn on biased locking, not that it would ever need to. Coleen > > David > > On 12/08/2020 9:58 pm, Coleen Phillimore wrote: >> Looks good! >> Coleen >> >> On 8/12/20 12:23 AM, David Holmes wrote: >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8251460 >>> webrev: http://cr.openjdk.java.net/~dholmes/8251460/webrev/ >>> >>> During the review for JDK-8250606 it was noted that the >>> biased-locking section of FastHashCode contained a similar assertion >>> regarding execution at a safepoint: >>> >>> ????? // Relaxing assertion for bug 6320749. >>> ????? assert(Universe::verify_in_progress() || >>> ???????????? !SafepointSynchronize::is_at_safepoint(), >>> ???????????? "biases should not be seen by VM thread here"); >>> >>> which gives the impression that it is okay to execute this code >>> block at a safepoint if verify_in_progress() is true. But if we >>> examine the next line of code: >>> >>> ? BiasedLocking::revoke(hobj, JavaThread::current()); >>> >>> we find that at a safepoint: >>> a) JavaThread::current() will assert because the current thread will >>> be the VMThread; and >>> b) BiasedLocking::revoke itself contains an assertion that it is not >>> invoked at a safepoint. >>> >>> In practice there is no issue with revoking the bias at a safepoint, >>> we just have to use the right code to do so (as done elsewhere in >>> the same file): >>> >>> if (SafepointSynchronize::is_at_safepoint()) { >>> ? BiasedLocking::revoke_at_safepoint(hobj); >>> } else { >>> ? BiasedLocking::revoke(hobj, self); >>> } >>> >>> Thanks, >>> David >> From suenaga at oss.nttdata.com Wed Aug 12 12:56:08 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 12 Aug 2020 21:56:08 +0900 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> Message-ID: <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> Hi Matthias, David, On 2020/08/12 21:12, David Holmes wrote: > On 12/08/2020 8:51 pm, Baesken, Matthias wrote: >> Hi Yasumasa ,? I'm more or less fine with the change . >> But still not fully convinced? that removing? the iteration is a good thing . >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/src/hotspot/cpu/x86/vm_version_x86.cpp.frames.html >> >> 1827?? for (base = 0x40000000; base < 0x40010000; base += 0x100) { >> 1828???? check_virt_cpuid(base, registers); >> >> I think just checking? "0x40000000" should work in most cases but if I remember correctly sometimes it was not enough . >> See also? some references about Xen/KVM : >> >> https://lists.linuxfoundation.org/pipermail/virtualization/2012-May/019974.html >> >> "If compat mode for another h/v is enabled then those leaves will appear >> at 0x40000000 and Xen's will be bumped up, so a fully Xen aware set of >> drivers (or detection routine, etc) should check at 0x100 intervals >> until 0x40010000 " >> >> and >> >> https://lore.kernel.org/patchwork/patch/394371/ I understand that if the process runs on Xen on other hypervisor (e.g. KVM), information for Xen would be set between 0x40000100 and 0x40010000. Ok, I will not remove the loop in new webrev, and will add comment about it. >> And not so happy about the WMI? usage (called in early JVM startup) : >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/src/hotspot/os_cpu/windows_x86/vm_version_windows_x86.cpp.frames.html >> >> bool VM_Version::is_in_VM() { ... } >> ? ... >> } >> >> ? But if noone else complains about it, I guess it's okay . > > I haven't reviewed this because I don't understand any of the code. But all that extra stuff during VM initialization doesn't look good to me. We do not want to pay a startup penalty just because we want a more accurate hypervisor listing in an error report! > > Can we not determine this only if needed? (Granted doing all this stuff during crash reporting may also be an issue.) This value would be used in hs_err and VM.info dcmd, so we can determine it when it is needed. However the change would be bigger, and it is not the original scope. Can we fix them at the same time? Thanks, Yasumasa > David > ----- > >> >> Best regards, Matthias >> >> >>> PING: Could you review this change? >>> >>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8250598 >>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/ >>> >>> Build change has been reviewed by Erik. >>> >> From david.holmes at oracle.com Wed Aug 12 13:04:41 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 12 Aug 2020 23:04:41 +1000 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> Message-ID: Hi Yasumasa, On 12/08/2020 10:56 pm, Yasumasa Suenaga wrote: > Hi Matthias, David, > > On 2020/08/12 21:12, David Holmes wrote: >> On 12/08/2020 8:51 pm, Baesken, Matthias wrote: >>> Hi Yasumasa ,? I'm more or less fine with the change . >>> But still not fully convinced? that removing? the iteration is a good >>> thing . >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/src/hotspot/cpu/x86/vm_version_x86.cpp.frames.html >>> >>> >>> 1827?? for (base = 0x40000000; base < 0x40010000; base += 0x100) { >>> 1828???? check_virt_cpuid(base, registers); >>> >>> I think just checking? "0x40000000" should work in most cases but if >>> I remember correctly sometimes it was not enough . >>> See also? some references about Xen/KVM : >>> >>> https://lists.linuxfoundation.org/pipermail/virtualization/2012-May/019974.html >>> >>> >>> "If compat mode for another h/v is enabled then those leaves will appear >>> at 0x40000000 and Xen's will be bumped up, so a fully Xen aware set of >>> drivers (or detection routine, etc) should check at 0x100 intervals >>> until 0x40010000 " >>> >>> and >>> >>> https://lore.kernel.org/patchwork/patch/394371/ > > I understand that if the process runs on Xen on other hypervisor (e.g. > KVM), information for Xen would be set between 0x40000100 and 0x40010000. > Ok, I will not remove the loop in new webrev, and will add comment about > it. > > >>> And not so happy about the WMI? usage (called in early JVM startup) : >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/src/hotspot/os_cpu/windows_x86/vm_version_windows_x86.cpp.frames.html >>> >>> >>> bool VM_Version::is_in_VM() { ... } >>> ? ... >>> } >>> >>> ? But if noone else complains about it, I guess it's okay . >> >> I haven't reviewed this because I don't understand any of the code. >> But all that extra stuff during VM initialization doesn't look good to >> me. We do not want to pay a startup penalty just because we want a >> more accurate hypervisor listing in an error report! >> >> Can we not determine this only if needed? (Granted doing all this >> stuff during crash reporting may also be an issue.) > > This value would be used in hs_err and VM.info dcmd, so we can determine > it when it is needed. > However the change would be bigger, and it is not the original scope. > Can we fix them at the same time? Are you able to run startup benchmarks to see if there is any regression? David ----- > > Thanks, > > Yasumasa > > >> David >> ----- >> >>> >>> Best regards, Matthias >>> >>> >>>> PING: Could you review this change? >>>> >>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8250598 >>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.01/ >>>> >>>> Build change has been reviewed by Erik. >>>> >>> From martin.doerr at sap.com Wed Aug 12 13:40:09 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 12 Aug 2020 13:40:09 +0000 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> Message-ID: Hi David, thanks for your good questions. I had also started to look at it and had similar thoughts. If we want to reduce overhead, inline assembler / intrisics will be better than creating a stub in the code cache. I think the existing inline assembler implementation for linux should also work on 32 bit linux. Maybe also on BSD (not sure). And on Windows, there's a __cpuid intrinsic available. (I have never tried it.) These implementations have no significant overhead. Maybe we can use them instead? Unfortunately, I'm not familiar with the code in vm_version_windows_x86.cpp, either. And I don't know if it's always fine to 0x40000000 as input to CPUID. Seems like Intel doesn't support that in the real instruction (returns all zero). I this value caught by all virtualization environments? Maybe we can find a little more precise name than "is_in_VM" because I usually think about the Java Thread state "_thread_in_vm". Nevertheless, I appreciate improvements in virtualization detection. Sometimes, problems or crashes are related to the virtualization layer, so having precise information is helpful. Best regards, Martin > -----Original Message----- > From: David Holmes > Sent: Mittwoch, 12. August 2020 15:05 > To: Yasumasa Suenaga ; Baesken, Matthias > ; hotspot-runtime-dev at openjdk.java.net; > build-dev at openjdk.java.net > Cc: Doerr, Martin > Subject: Re: PING: RFR: 8250598: Hyper-V is detected in spite of running on > host OS > > Hi Yasumasa, > > On 12/08/2020 10:56 pm, Yasumasa Suenaga wrote: > > Hi Matthias, David, > > > > On 2020/08/12 21:12, David Holmes wrote: > >> On 12/08/2020 8:51 pm, Baesken, Matthias wrote: > >>> Hi Yasumasa ,? I'm more or less fine with the change . > >>> But still not fully convinced? that removing? the iteration is a good > >>> thing . > >>> > >>> http://cr.openjdk.java.net/~ysuenaga/JDK- > 8250598/webrev.01/src/hotspot/cpu/x86/vm_version_x86.cpp.frames.html > >>> > >>> > >>> 1827?? for (base = 0x40000000; base < 0x40010000; base += 0x100) { > >>> 1828???? check_virt_cpuid(base, registers); > >>> > >>> I think just checking? "0x40000000" should work in most cases but if > >>> I remember correctly sometimes it was not enough . > >>> See also? some references about Xen/KVM : > >>> > >>> https://lists.linuxfoundation.org/pipermail/virtualization/2012- > May/019974.html > >>> > >>> > >>> "If compat mode for another h/v is enabled then those leaves will > appear > >>> at 0x40000000 and Xen's will be bumped up, so a fully Xen aware set of > >>> drivers (or detection routine, etc) should check at 0x100 intervals > >>> until 0x40010000 " > >>> > >>> and > >>> > >>> https://lore.kernel.org/patchwork/patch/394371/ > > > > I understand that if the process runs on Xen on other hypervisor (e.g. > > KVM), information for Xen would be set between 0x40000100 and > 0x40010000. > > Ok, I will not remove the loop in new webrev, and will add comment about > > it. > > > > > >>> And not so happy about the WMI? usage (called in early JVM startup) : > >>> > >>> http://cr.openjdk.java.net/~ysuenaga/JDK- > 8250598/webrev.01/src/hotspot/os_cpu/windows_x86/vm_version_windo > ws_x86.cpp.frames.html > >>> > >>> > >>> bool VM_Version::is_in_VM() { ... } > >>> ? ... > >>> } > >>> > >>> ? But if noone else complains about it, I guess it's okay . > >> > >> I haven't reviewed this because I don't understand any of the code. > >> But all that extra stuff during VM initialization doesn't look good to > >> me. We do not want to pay a startup penalty just because we want a > >> more accurate hypervisor listing in an error report! > >> > >> Can we not determine this only if needed? (Granted doing all this > >> stuff during crash reporting may also be an issue.) > > > > This value would be used in hs_err and VM.info dcmd, so we can > determine > > it when it is needed. > > However the change would be bigger, and it is not the original scope. > > Can we fix them at the same time? > > Are you able to run startup benchmarks to see if there is any regression? > > David > ----- > > > > > Thanks, > > > > Yasumasa > > > > > >> David > >> ----- > >> > >>> > >>> Best regards, Matthias > >>> > >>> > >>>> PING: Could you review this change? > >>>> > >>>> ?? JBS: https://bugs.openjdk.java.net/browse/JDK-8250598 > >>>> ?? webrev: http://cr.openjdk.java.net/~ysuenaga/JDK- > 8250598/webrev.01/ > >>>> > >>>> Build change has been reviewed by Erik. > >>>> > >>> From matthias.baesken at sap.com Wed Aug 12 14:45:22 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 12 Aug 2020 14:45:22 +0000 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> Message-ID: > And on Windows, there's a __cpuid intrinsic available. (I have never tried it.) Hi Martin, we use this intrinsic in "our" JVM so yes it works . > Nevertheless, I appreciate improvements in virtualization detection. Sometimes, problems or crashes are related to the virtualization layer, so having precise information is helpful. Our support colleagues often use that info . Best regards, Matthias From matthias.baesken at sap.com Wed Aug 12 15:05:35 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Wed, 12 Aug 2020 15:05:35 +0000 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> Message-ID: >I understand that if the process runs on Xen on other hypervisor (e.g. KVM), information for Xen would be set between 0x40000100 and 0x40010000. >Ok, I will not remove the loop in new webrev, and will add comment about it. Hi Yasumasa , thanks ! Regarding the WMI overhead , if you could get some more info on this it would be better to judge if it is perfectly okay or concerning . (I remember that I looked into it a couple of years ago , but decided not to use it in early VM startup ; but have to confess this was just based on a "gut feeling", No micro benchmarks ) Best regards, Matthias From patricio.chilano.mateo at oracle.com Wed Aug 12 15:17:39 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Wed, 12 Aug 2020 12:17:39 -0300 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: <750e129d-40c7-d205-5352-2f4582475c83@oracle.com> References: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> <3c9ffc2b-51e8-272c-f03b-d5faf5036bf6@oracle.com> <51f4bbe4-0a19-b09c-c544-5681afb2aa1d@oracle.com> <8f0f8423-f46b-9b78-1544-24f02a8606f7@oracle.com> <750e129d-40c7-d205-5352-2f4582475c83@oracle.com> Message-ID: Hi Coleen, On 8/11/20 5:51 PM, Coleen Phillimore wrote: > > > On 8/11/20 4:45 PM, Patricio Chilano wrote: >> >> On 8/11/20 5:34 PM, Daniel D. Daugherty wrote: >>> On 8/11/20 4:27 PM, Patricio Chilano wrote: >>>> Hi Dan, >>>> >>>> Thanks for looking at this. >>>> >>>> On 8/11/20 4:53 PM, Daniel D. Daugherty wrote: >>>>> On 8/11/20 2:11 PM, Patricio Chilano wrote: >>>>>> Hi all, >>>>>> >>>>>> Please review the following small fix which involves removing the >>>>>> added HandleMark in BiasedLocking::preserve_marks(): >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 >>>>>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ >>>>> >>>>> src/hotspot/share/runtime/biasedLocking.cpp >>>>> ??? No comments. >>>>> >>>>> As mentioned by Harold, adding a new test case to: >>>>> >>>>> ??? test/hotspot/jtreg/runtime/7158988/FieldMonitor.java >>>>> >>>>> that enables '-XX:+UseBiasedLocking' is a good idea. >>>> I suggested adding it to test TestFullGCALot.java instead because >>>> it fails much more consistently. >>> >>> I'm good with that also. > > Me too, even better than FieldMonitor.java if it's more reproduceable. > >>> >>> >>>> >>>>> Also, Coleen added this note to your bug report: >>>>> >>>>>> Also in the patch for JDK-8249192 >>>>>> , vframe.hpp >>>>>> shouldn't include >>>>>> handles.inline.hpp (only cpp and inline.hpp files can include >>>>>> inline.hpp). It doesn't look like it needs handles.inline.hpp, >>>>>> only handles.hpp. >>>>> >>>>> Were you planning on resolving that left over in this bug? >>>> Not really. I can change it here if you are okay though. >>> >>> Your call on whether to fix it here or with a new bug. Seems >>> like fixing it here is less work... this bug is a follow-on to >>> the original and that comment is a follow-on to the original... >> Yeah, sounds good to me to remove the extra header in this change. >> I'll check it builds fine on all platforms first. >> Below is v2 in case you wanted to see it: >> >> Full: http://cr.openjdk.java.net/~pchilanomate/8251118/v2/webrev/ >> Inc: http://cr.openjdk.java.net/~pchilanomate/8251118/v2/inc/webrev/ > > I think vframe.hpp should include handles.hpp because there are Handle > types in it, just in case something changes and it's not transiently > included anymore. > > Otherwise, everything looks good.? I don't need to see another webrev. Ok, I removed handles.inline.hpp and included handles.hpp instead. Thanks for looking into this! Patricio > thanks! > Coleen >> >> Thanks Dan! >> >> Patricio >>> Dan >>> >>> >>>> >>>> Patricio >>>>> Dan >>>>> >>>>> >>>>>> >>>>>> I've inspected the callers of BiasedLocking::preserve_marks() and >>>>>> they all have an assert that the current thread is the VMThread. >>>>>> Since the VMThread creates a HandleMark object before executing a >>>>>> VM operation the extra HandleMark added in 8249192 is not needed. >>>>>> I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and without >>>>>> the patch I get several crashes in >>>>>> BiasedLocking::restore_marks(). With the patch tests completed >>>>>> successfully. >>>>>> >>>>>> Thanks, >>>>>> Patricio >>>>>> >>>>> >>>> >>> >> > From patricio.chilano.mateo at oracle.com Wed Aug 12 15:20:09 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Wed, 12 Aug 2020 12:20:09 -0300 Subject: RFR 8251118: BiasedLocking::preserve_marks should not have a HandleMark In-Reply-To: References: <2b987fc3-d9f6-8670-ebd0-45a1802e0780@oracle.com> <3c9ffc2b-51e8-272c-f03b-d5faf5036bf6@oracle.com> <51f4bbe4-0a19-b09c-c544-5681afb2aa1d@oracle.com> <8f0f8423-f46b-9b78-1544-24f02a8606f7@oracle.com> <750e129d-40c7-d205-5352-2f4582475c83@oracle.com> Message-ID: On 8/11/20 9:36 PM, David Holmes wrote: > +1 on Coleen's comment about still including handles.hpp > > +1 on changes looking good. Thanks for the review David! Patricio > Thanks, > David > > On 12/08/2020 6:51 am, Coleen Phillimore wrote: >> >> >> On 8/11/20 4:45 PM, Patricio Chilano wrote: >>> >>> On 8/11/20 5:34 PM, Daniel D. Daugherty wrote: >>>> On 8/11/20 4:27 PM, Patricio Chilano wrote: >>>>> Hi Dan, >>>>> >>>>> Thanks for looking at this. >>>>> >>>>> On 8/11/20 4:53 PM, Daniel D. Daugherty wrote: >>>>>> On 8/11/20 2:11 PM, Patricio Chilano wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> Please review the following small fix which involves removing >>>>>>> the added HandleMark in BiasedLocking::preserve_marks(): >>>>>>> >>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8251118 >>>>>>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8251118/v1/webrev/ >>>>>> >>>>>> src/hotspot/share/runtime/biasedLocking.cpp >>>>>> ??? No comments. >>>>>> >>>>>> As mentioned by Harold, adding a new test case to: >>>>>> >>>>>> ??? test/hotspot/jtreg/runtime/7158988/FieldMonitor.java >>>>>> >>>>>> that enables '-XX:+UseBiasedLocking' is a good idea. >>>>> I suggested adding it to test TestFullGCALot.java instead because >>>>> it fails much more consistently. >>>> >>>> I'm good with that also. >> >> Me too, even better than FieldMonitor.java if it's more reproduceable. >> >>>> >>>> >>>>> >>>>>> Also, Coleen added this note to your bug report: >>>>>> >>>>>>> Also in the patch for JDK-8249192 >>>>>>> , vframe.hpp >>>>>>> shouldn't include >>>>>>> handles.inline.hpp (only cpp and inline.hpp files can include >>>>>>> inline.hpp). It doesn't look like it needs handles.inline.hpp, >>>>>>> only handles.hpp. >>>>>> >>>>>> Were you planning on resolving that left over in this bug? >>>>> Not really. I can change it here if you are okay though. >>>> >>>> Your call on whether to fix it here or with a new bug. Seems >>>> like fixing it here is less work... this bug is a follow-on to >>>> the original and that comment is a follow-on to the original... >>> Yeah, sounds good to me to remove the extra header in this change. >>> I'll check it builds fine on all platforms first. >>> Below is v2 in case you wanted to see it: >>> >>> Full: http://cr.openjdk.java.net/~pchilanomate/8251118/v2/webrev/ >>> Inc: http://cr.openjdk.java.net/~pchilanomate/8251118/v2/inc/webrev/ >> >> I think vframe.hpp should include handles.hpp because there are >> Handle types in it, just in case something changes and it's not >> transiently included anymore. >> >> Otherwise, everything looks good.? I don't need to see another webrev. >> >> thanks! >> Coleen >>> >>> Thanks Dan! >>> >>> Patricio >>>> Dan >>>> >>>> >>>>> >>>>> Patricio >>>>>> Dan >>>>>> >>>>>> >>>>>>> >>>>>>> I've inspected the callers of BiasedLocking::preserve_marks() >>>>>>> and they all have an assert that the current thread is the >>>>>>> VMThread. Since the VMThread creates a HandleMark object before >>>>>>> executing a VM operation the extra HandleMark added in 8249192 >>>>>>> is not needed. >>>>>>> I've run tiers1-3 in mach5 with -XX:+UseBiasedLocking and >>>>>>> without the patch I get several crashes in >>>>>>> BiasedLocking::restore_marks(). With the patch tests completed >>>>>>> successfully. >>>>>>> >>>>>>> Thanks, >>>>>>> Patricio >>>>>>> >>>>>> >>>>> >>>> >>> >> From daniel.daugherty at oracle.com Wed Aug 12 15:26:15 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Wed, 12 Aug 2020 11:26:15 -0400 Subject: RFR (s): 8251460: Fix the biased-locking code in ObjectSynchronizer::FastHashCode In-Reply-To: References: <797db6f1-e080-3a33-fefa-34d045a9c681@oracle.com> Message-ID: <07130e08-6b75-fdc6-a5e6-393493c3391c@oracle.com> > webrev: http://cr.openjdk.java.net/~dholmes/8251460/webrev/ src/hotspot/share/runtime/synchronizer.cpp ??? No comments. Thumbs up! Thanks for the info about how this was tested. Any Mach5 testing done yet? Dan On 8/12/20 8:16 AM, David Holmes wrote: > Thanks Coleen! > > I forgot to mention that the testing for this was to reenable (in > arguments.cpp) biased-locking for CDS (and fix an errant assert that > Ioi is going to fix) and run the CDS tests. None of those changes will > be pushed of course. > > David > > On 12/08/2020 9:58 pm, Coleen Phillimore wrote: >> Looks good! >> Coleen >> >> On 8/12/20 12:23 AM, David Holmes wrote: >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8251460 >>> webrev: http://cr.openjdk.java.net/~dholmes/8251460/webrev/ >>> >>> During the review for JDK-8250606 it was noted that the >>> biased-locking section of FastHashCode contained a similar assertion >>> regarding execution at a safepoint: >>> >>> ????? // Relaxing assertion for bug 6320749. >>> ????? assert(Universe::verify_in_progress() || >>> ???????????? !SafepointSynchronize::is_at_safepoint(), >>> ???????????? "biases should not be seen by VM thread here"); >>> >>> which gives the impression that it is okay to execute this code >>> block at a safepoint if verify_in_progress() is true. But if we >>> examine the next line of code: >>> >>> ? BiasedLocking::revoke(hobj, JavaThread::current()); >>> >>> we find that at a safepoint: >>> a) JavaThread::current() will assert because the current thread will >>> be the VMThread; and >>> b) BiasedLocking::revoke itself contains an assertion that it is not >>> invoked at a safepoint. >>> >>> In practice there is no issue with revoking the bias at a safepoint, >>> we just have to use the right code to do so (as done elsewhere in >>> the same file): >>> >>> if (SafepointSynchronize::is_at_safepoint()) { >>> ? BiasedLocking::revoke_at_safepoint(hobj); >>> } else { >>> ? BiasedLocking::revoke(hobj, self); >>> } >>> >>> Thanks, >>> David >> From coleen.phillimore at oracle.com Wed Aug 12 15:28:26 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 12 Aug 2020 11:28:26 -0400 Subject: RFR (T) 8251489: universe.cpp includes too many headers Message-ID: <2b1d8a9e-126c-48c9-5eaa-4e53e5f6ad8e@oracle.com> See bug for description. Tested with tier1 on 4 platforms, built with --no-precompiled-headers. open webrev at http://cr.openjdk.java.net/~coleenp/2020/8251489.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8251489 Thanks, Coleen From lois.foltan at oracle.com Wed Aug 12 16:05:53 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 12 Aug 2020 12:05:53 -0400 Subject: RFR (T) 8251489: universe.cpp includes too many headers In-Reply-To: <2b1d8a9e-126c-48c9-5eaa-4e53e5f6ad8e@oracle.com> References: <2b1d8a9e-126c-48c9-5eaa-4e53e5f6ad8e@oracle.com> Message-ID: <8d1dd76c-e410-9d55-ca81-f0649f4aeeae@oracle.com> Looks good & trivial. Lois On 8/12/2020 11:28 AM, Coleen Phillimore wrote: > See bug for description. Tested with tier1 on 4 platforms, built with > --no-precompiled-headers. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8251489.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8251489 > > Thanks, > Coleen > From coleen.phillimore at oracle.com Wed Aug 12 16:22:36 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 12 Aug 2020 12:22:36 -0400 Subject: RFR (T) 8251489: universe.cpp includes too many headers In-Reply-To: <8d1dd76c-e410-9d55-ca81-f0649f4aeeae@oracle.com> References: <2b1d8a9e-126c-48c9-5eaa-4e53e5f6ad8e@oracle.com> <8d1dd76c-e410-9d55-ca81-f0649f4aeeae@oracle.com> Message-ID: <56e1e3fe-0f55-fc09-b981-b405e4495952@oracle.com> Thanks Lois! Coleen On 8/12/20 12:05 PM, Lois Foltan wrote: > Looks good & trivial. > Lois > > On 8/12/2020 11:28 AM, Coleen Phillimore wrote: >> See bug for description. Tested with tier1 on 4 platforms, built with >> --no-precompiled-headers. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8251489.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8251489 >> >> Thanks, >> Coleen >> > From thomas.stuefe at gmail.com Wed Aug 12 16:34:32 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 12 Aug 2020 18:34:32 +0200 Subject: RFR (T) 8251489: universe.cpp includes too many headers In-Reply-To: <2b1d8a9e-126c-48c9-5eaa-4e53e5f6ad8e@oracle.com> References: <2b1d8a9e-126c-48c9-5eaa-4e53e5f6ad8e@oracle.com> Message-ID: This looks good. I wish we could do this automatically. Thanks, Thomas On Wed, Aug 12, 2020 at 5:33 PM Coleen Phillimore < coleen.phillimore at oracle.com> wrote: > See bug for description. Tested with tier1 on 4 platforms, built with > --no-precompiled-headers. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8251489.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8251489 > > Thanks, > Coleen > > From coleen.phillimore at oracle.com Wed Aug 12 16:36:11 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 12 Aug 2020 12:36:11 -0400 Subject: RFR (T) 8251489: universe.cpp includes too many headers In-Reply-To: References: <2b1d8a9e-126c-48c9-5eaa-4e53e5f6ad8e@oracle.com> Message-ID: <27921cdb-10a3-c3ba-469c-a7410b0d667f@oracle.com> On 8/12/20 12:34 PM, Thomas St?fe wrote: > This looks good. I wish we could do this automatically. Thanks, Thomas, I agree! Coleen > > Thanks, Thomas > > On Wed, Aug 12, 2020 at 5:33 PM Coleen Phillimore > > > wrote: > > See bug for description. Tested with tier1 on 4 platforms, built with > --no-precompiled-headers. > > open webrev at > http://cr.openjdk.java.net/~coleenp/2020/8251489.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8251489 > > Thanks, > Coleen > From thomas.stuefe at gmail.com Wed Aug 12 17:59:58 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 12 Aug 2020 19:59:58 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace Message-ID: Dear all, I would like to start the review for the implementation of JEP 387 "Elastic Metaspace". The relevant JBS items are: JEP: https://openjdk.java.net/jeps/387 Implementation Issue (pretty much only a placeholder currently): https://bugs.openjdk.java.net/browse/JDK-8251158 -- Reminder of why we do this: 1. The new metaspace saves memory. How much depends on context: if it is not a problem today it will continue to be none. But in cases where today we face large metaspace consumption we may get reductions, sometimes drastic ones. These reductions are caused by two facts: - after mass class unloading we release memory more promptly to the OS - even without class unloading chunk allocation is just plain smarter, which reduces the overhead per class loader. This is especially true for scenarios involving masses of small loaders which only load very few classes. As an example, see [1] where two VMs - one stock, one patched - run the same test program which creates tons of small loaders. The second run shows a much reduced memory footprint and increased elasticity. 2. The rewritten metaspace code base got cleaner and less complex and thus should be much easier to maintain and to extend. It also would be possible to easily reuse the allocator for different parts of the VM, since it is less tightly tied to the notion of just storing class metadata. -- In preparation of this review I prepared a guide [2], [3], which gives an architectural overview and should be the starting point for an interested Reviewer. The prototype has been tested extensively for quite some time in SAP's CI system. We regularly run JCK test, JTReg tests and a whole battery of SAP internal tests on 8 platforms. Tests are also currently ongoing at Oracle and Red Hat. -- The full webrev [4] is somewhat large. In order to make this more bearable I broke the patch up into three parts: 1) The core parts [5] This is the heart of the metaspace, mostly rewritten from scratch. I propose any Reviewer should not look at the diff but rather just examine the new implementation. One possible exception is metaspace.hpp, which is the outside interface to metaspace. There are several ways to get a feeling for this code (after reading at least the concept part of the provided guide [2]). The central, most "beefy" classes are: - VirtualSpaceNode - does all the work of reserving, committing, uncommitting memory - RootChunkArea - does the grunt work of chunk splitting and merging - ChunkManager - which takes care of the chunk freelists, of directing chunk splits and merges, of enlarging chunks in place - MetaspaceArena - which takes care of fine granular allocation on behalf of a CLD, and of managing deallocated blocks. One way to review could be bottom up: starting at VirtualSpaceNode/CommitMask/RootChunkArea(LUT), working your way up to ChunkManager and Metachunk; finally up to to MetaspaceArena while taking a side stroll to FreeBlocks/BinList/BlockTree. Another way would be to follow typical paths: Allocation path: Starting at MetaspaceArena::allocate() -> Metachunk::allocate() (note the committing-on-demand code path starting here) -> ChunkManager::get_chunk() -> VirtualSpaceList/Node->allocate_root_chunk(). Destruction: ~MetaspaceArena() -> ChunkManager::return_chunk() -> (merging chunks) -> (uncommitting chunks) Premature deallocation: -> MetaspaceArena::deallocate() -> see freeblock list handling 2) The tests [6] This part of the patch contains all the new tests. There are a lot; the test coverage of the new metaspace is very thorough. New gtests have been added under 'test/hotspot/gtest/metaspace'. New jtreg have been added under 'test/hotspot/jtreg/runtime/Metaspace/elastic' and under 'test/hotspot/jtreg/gtest/MetaspaceGtests.java'. 4) Miscellaneous diffs [7] This is the part of the patch which is neither considered core nor test. These changes are small, often uninteresting, and can be reviewed via diff. --- Out of scope for this patch is revamping outside metaspace statistics: monitoring of metaspace statistics is mostly left untouched, beyond the work needed to keep existing tests green. I wanted to keep those changes separate from the core changes for JEP387. They are tracked in JDK-8251392 [8] and JDK-8251342 [9], respectively. --- Code complexity: Numerically, lines of code went ever so slightly down with this patch: Before: (cloc hotspot/share/memory) ------------------------------------------------------------------------------- C++ 36 2533 3097 12735 C/C++ Header 54 1728 2867 6278 ------------------------------------------------------------------------------- SUM: 90 4261 5964 19013 After: ------------------------------------------------------------------------------- C++ 50 3048 3774 13127 C/C++ Header 63 2006 3575 5738 ------------------------------------------------------------------------------- SUM: 113 5054 7349 18865 But since the new implementation is more powerful than the old one - doing things like committing/uncommitting on demand, guarding allocated blocks etc - one could argue that the new solution does quite a lot more with slightly less code, which I think is a good result. --- Idle musing: it would simplify matters quite a bit if we were to unify class space and non-class metaspace. If we keep the current narrow Klass pointer encoding scheme, we would still need to keep the space they are stored in contiguous. But we could use the class space for everything, in effect dropping the "class" moniker from the name. It would have to be larger than it is currently, of course. Cons: - we would have to abandon the notion of "limitless metaspace" - but if we run with class space this has never been true anyway since the number of classes is limited by the size of the compressed class space. - virtual process size would go up since it now needs to be as large as the total projected metaspace size. - we may need to expand narrow Klass pointer encoding to include shifting, if 4G are not enough to hold all metaspace data. Pros: - this would simplify a lot of code. - this would save real (committed) memory, since we save quite a bit of overhead. - we could narrow-pointer-encode other metadata too, not only Klass pointers, should that ever be interesting ... but that is not part of this JEP. --- With this patch, we have a much stronger separation of concerns, and it should be easy to reuse the metaspace allocator in other hotspot areas as well. For instance, ResourceAreas and friends could be replaced by MetaspaceArenas. They do almost the same thing. And as we have seen in the past at SAP, the C-heap backed hotspot Arenas can be a pain since spikes in Arena usage lingers forever as process footprint (we even once rewrote parts of the arena code to use mmap, which is just on a more primitive level what Metaspace does). --- I know this is short notice, but there will be a call for interested people tomorrow at 11AM ET. In case any potential Reviewers are interested just drop me a short note. --- Thank you, Thomas [1] http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/reduction-small-loaders.pdf [2] http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.pdf [3] http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.html [4] http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-all/webrev/ [5] http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-core/webrev/ [6] http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-test/webrev/ [7] http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-misc/webrev/ [8] https://bugs.openjdk.java.net/browse/JDK-8251342 [9] https://bugs.openjdk.java.net/browse/JDK-8251392 From harold.seigel at oracle.com Wed Aug 12 18:14:48 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Wed, 12 Aug 2020 14:14:48 -0400 Subject: RFR 8251414: Add test that invokeinterface of a protected method in java.lang.Object throws NoSuchMethodError Message-ID: Hi, Please review this change to add test cases to test hotspot/jtreg/runtime/8026394/InterfaceObjectTest.java to check for NoSuchMethodError exceptions when calling Object protected methods clone() and finalize() using invokeinterface. Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8251414/webrev/index.html JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251414 The fix was tested by running the modified test on multiple platforms, including Mac OS, Windows, and Linux. Thanks, Harold From patricio.chilano.mateo at oracle.com Wed Aug 12 18:42:04 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Wed, 12 Aug 2020 15:42:04 -0300 Subject: RFR (s): 8251460: Fix the biased-locking code in ObjectSynchronizer::FastHashCode In-Reply-To: <797db6f1-e080-3a33-fefa-34d045a9c681@oracle.com> References: <797db6f1-e080-3a33-fefa-34d045a9c681@oracle.com> Message-ID: <13acfb8f-4a58-4560-91ed-c7e7b494fec3@oracle.com> Hi David, Changes look good to me. Thanks, Patricio On 8/12/20 1:23 AM, David Holmes wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8251460 > webrev: http://cr.openjdk.java.net/~dholmes/8251460/webrev/ > > During the review for JDK-8250606 it was noted that the biased-locking > section of FastHashCode contained a similar assertion regarding > execution at a safepoint: > > ????? // Relaxing assertion for bug 6320749. > ????? assert(Universe::verify_in_progress() || > ???????????? !SafepointSynchronize::is_at_safepoint(), > ???????????? "biases should not be seen by VM thread here"); > > which gives the impression that it is okay to execute this code block > at a safepoint if verify_in_progress() is true. But if we examine the > next line of code: > > ? BiasedLocking::revoke(hobj, JavaThread::current()); > > we find that at a safepoint: > a) JavaThread::current() will assert because the current thread will > be the VMThread; and > b) BiasedLocking::revoke itself contains an assertion that it is not > invoked at a safepoint. > > In practice there is no issue with revoking the bias at a safepoint, > we just have to use the right code to do so (as done elsewhere in the > same file): > > if (SafepointSynchronize::is_at_safepoint()) { > ? BiasedLocking::revoke_at_safepoint(hobj); > } else { > ? BiasedLocking::revoke(hobj, self); > } > > Thanks, > David From calvin.cheung at oracle.com Wed Aug 12 18:54:27 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Wed, 12 Aug 2020 11:54:27 -0700 Subject: Fwd: RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS In-Reply-To: <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> Message-ID: <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> Hi Yumin, I reviewed mostly the native code. Below are my comments: 1) classListParser.hpp 71?? bool??????????????? _lambda_format; The above name is too generic. How about _lambda_form or _is_lambda_form? If you rename the above, please also rename the function which returns the above bool. 2) jvm.cpp 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line)) ignore -> ignored 3) jvm.hpp 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); Same comment as for jvm.cpp 4) metaspaceShared.cpp 2017?? size_t i = 0; 2018?? while (i < size) { 2019???? full_name[i++] = *start++; 2020?? } Could the above be simplified to the following? ??? strncpy(full_name, start, size - 1); 2029?? char* class_name = get_full_class_name(path_name); Should os::free(class_name) be called before the function returns? 1870 static GrowableArray* lambda_list = NULL; The name lambda_list is a bit generic, how about lambda_form_list? 2112?????? lambda_list->append(parser.current_line()); Since parser.current_line() does a strdup, do those buffer need to be freed after its use? (i.e. after the call to regenerate_holder_classes()?) In MetaspaceShared::regenerate_holder_classes, before calling up to java, I think it's better to strip the prefix "@lambda-form-invoker" from the strings. So that the java InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need to handle it: ?143???????????????????? case "[LF_RESOLVE]": ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: 5) DumpSymbolAndStringTable.java 37???? private static final String my_string = "DumpSymbolAndStringTable"; Unused variable? thanks, Calvin On 8/11/20 10:36 AM, Yumin Qi wrote: > Forget to send to @core-lib-dev, the patch changed jdk code. > > > Thanks > > Yumin > > > > -------- Forwarded Message -------- > Subject:???? RFR: 8247536: Support pre-generated MethodHandle lambda > forms in CDS > Date:???? Tue, 11 Aug 2020 07:44:34 -0700 > From:???? Yumin Qi > To:???? hotspot-runtime-dev at openjdk.java.net > > > > Hi, Please reivew > > ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 > ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ > > ? Summary: CDS does not archive pre-generated lambda form classes for > method handles: > > [0.142s][info][class,load] > java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: > __JVM_LookupDefineClass__ > > The method handle resolution if not found in the holder class, a class > with the method will be generated and loaded as vm internal created > class and not archived like above. Those class names are mangled as > combination of the class name and the class instance address. > > In this patch, collect those method holder class names and their > associated methods' signatures when user specify DumpLoadedClassList, > and record them in the log file in a format similar to the output > format from traced by > -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use > another name for CDS: > -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The > line prefix also changed from "[LF_RESOLVE]" to > "@lambda-invoke-handle".? At dump time, regenerate the holder class > with those methods and replace the existing holder class and archived > it. At runtime, the resolution of the holder class which contains > those methods are loaded from CDS archive so avoid regenerating them > again. The patch reorganized the code for Jlink plugin, so the new > added InvokerBytecodeGeneratorHelper can be shared both by JLink > plugin and CDS code. Also change made to the mangled generated class > name at static dump, the class name combining the class name and a > sequentially ordered number instead of the class instance address, > which looks like a random. > > > To use this feature, one can do the dump/run just like done for a > static dump/run, so no extra steps required. > > ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp > > ????? DumpLoadedClassList will turn on > -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true > > ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist > -XX:SharedArchiveFile=my.jsa JavaApp > > ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp > > ? The performance on javac HelloWorld.java (javac-benchmark), no patch > vs patch: > > ?? 1:?? 2689285002? 2641821474 (-47463528)????? ----???? 391.720 > 382.990 ( -8.730)????? ---- > ?? 2:?? 2687495085? 2632969688 (-54525397)????? ----???? 391.030 > 381.480 ( -9.550)????? ----- > ?? 3:?? 2694664066? 2636523114 (-58140952)????? -----??? 390.610 > 382.550 ( -8.060)????? ---- > ?? 4:?? 2686554164? 2639355233 (-47198931)????? ----???? 390.700 > 383.390 ( -7.310)????? --- > ?? 5:?? 2691072338? 2633016687 (-58055651)????? -----??? 388.990 > 382.360 ( -6.630)????? --- > ?? 6:?? 2684448174? 2644191854 (-40256320)????? ---????? 389.450 > 382.990 ( -6.460)????? --- > ?? 7:?? 2694921227? 2630505090 (-64416137)????? -----??? 389.300 > 383.160 ( -6.140)????? --- > ?? 8:?? 2685209712? 2639334320 (-45875392)????? ----???? 388.370 > 381.060 ( -7.310)????? --- > ?? 9:?? 2695885942? 2640618655 (-55267287)????? ----???? 389.560 > 381.100 ( -8.460)????? ---- > ? 10:?? 2689162942? 2635658943 (-53503999)????? ----???? 389.690 > 379.110 (-10.580)????? ----- > ============================================================ > ??????? 2689866989? 2637396210 (-52470778)????? ----???? 389.941 > 382.017 ( -7.924)????? ---- > > instr delta =??? -52470778??? -1.9507% > > time? delta =?????? -7.924 ms -2.0321% > > ? It will show much more improvement if compare with the default archive: > > ?????????? ? ? ? no patch??????????????????????? patch > > instr ??? ? ? 3,996,847,605??????? 3,320,928,995 > > time(s).???? 0.686835162 ? ? ? ?? 0.474933546 > > > ? Tests: > > 1)jtreg on jdk/tools and hotspot/runtime local. > > 2) mach5 ter1,2 > > > Thanks > > Yumin > > ? ` From ioi.lam at oracle.com Wed Aug 12 22:06:35 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 12 Aug 2020 15:06:35 -0700 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: <9e6b0043-65a1-dd97-a3d2-33679c8048d4@oracle.com> References: <9e6b0043-65a1-dd97-a3d2-33679c8048d4@oracle.com> Message-ID: Hi Lois, Thanks for the comments. I have an updated webrev http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/ http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02.delta/ Here are the general notes on the changes. Replies to your questions are in-line below. (1) Integrated updates in the Java code from Alan Bateman. Here are Alan's ? ? notes: ??? The archive of the boot layer is as before, except that archiving is ??? skipped if there are split packages or incubator modules. Incubating ??? modules aren't resolved by default so we shouldn't have them in the ??? boot layer by default anyway. ??? I've dropped the module finders from the boot layer archive. I've ??? left the IllegalAccessLogger.Builder in the acrhive for now (even ??? though it is not the boot layer). We should be able to remove that ??? once the JEP to disallow illegal access by default is in. ??? Related is that I don't like the archive for the module graph ??? (ArchivedModuleGraph, pre-dates this RFE) including the set of ??? packages to export/open for illegal access as they aren't part ??? of the module graph. I've left it for now but we can also remove ??? that once we disallow illegal access by default (as those sets ??? will be empty). ??? The archive of built-in class loaders is now in one object ??? jdk.internal.loader.ArchivedClassLoaders which I think will be a ??? bit more maintainable. I've also drop the ucp field from the ??? AppClassLoader as the changes to BuiltinClassLoader means is no ??? longer needs to duplicated. ??? There's one remaining issue in ModuleBootstrap.boot where it has fix ??? an app class loader value (ModuleLayer.CLV). Ideally the initialization ??? of the built-in class loaders would do this but we are kinda forced ??? to separate the archiving of the built-in class loaders from the boot ??? layer. I might look at this again some time. (2) Moved code from classLoaderData.cpp -> classLoaderSharedData.cpp (3) Reverted unnecessary changes in JavaClasses::compute_offset (4) Minor clean up to use QuickSort::sort() instead of qsort() (5) Moved the C-side of module initialization for platform/system ? ? loaders to inside java.lang.System::initPhase2(), so this happens ??? at the same time as without full module archiving. (6) Moved the use of Module_lock to so all modules in a class loader ??? are restored atomically. See ArchivedClassLoaderData::restore() ??? This fixed a bug where test/jdk/com/sun/jdi/ModulesTest.java ? ? would fail as it sees a partially restored set of modules. On 8/7/20 12:06 PM, Lois Foltan wrote: > Hi Ioi, > > Overall looks promising.? I have some review comments below, but not a > complete review.? I concentrated on the JVM side primarily how the > archived module graph is read in, how the ModuleEntry and PackageEntry > tables are created from the archive for the 3 builtin loaders and > potential timing issues during start up. > > On 7/22/2020 3:36 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8244778 >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/ >> >> >> Please review this patch that stores the full module graph in the CDS >> archive heap. This reduces the initialization time of the basic JVM by >> about 22%: >> >> $ perf stat -r 100 bin/java -version >> before: 98,219,329 instructions 0.03971 secs elapsed (+- 0.44%) >> after:? 55,835,815 instructions 0.03109 secs elapsed (+- 0.65%) >> >> [1] Start with ModuleBootstrap.java. The current implementation is >> ??? quite restrictive: the archived module graph is used only when no >> ??? module options are specified. >> >> ??? See ModuleBootstrap.mayUseArchivedBootLayer(). >> >> ??? We can probably support options such as main module and module path >> ??? in a future RFE. > > Yes, I noticed the restrictions.? The JBS issue discusses the > possibility of using the dumped module graph in the event that the > value of the options are the same.? I think for this implementation to > be viable and have a positive impact you should consider comparing at > least the options --add-modules, --add-exports and --add-reads. I agree. I want to keep the changes minimal in this RFE, and will add more support for other use cases in follow-on RFEs. Instead of requiring these options to be unspecified, we can relax the restriction such that these options must be the same between archive dump time and run time. > > >> >> [2] In the current JDK implementation, there is no single object >> ??? that represents "the module graph". Most of the information >> ??? is stored in the archive bootLayer object, but a few additional >> ??? restoration operations need to be performed: >> >> ??? + See ModuleBootstrap.getArchivedBootLayer() >> ??? + Some static fields need to be archived/restored in >> ????? Module.java, BuiltinClassLoader.java, ClassLoaders.java >> ????? and BootLoader.java >> >> [3] I ran into a complication with two loader instances of >> ??? PlatformClassLoader and AppClassLoader. They are stored in >> ??? multiple tables inside the module graph (e.g., >> ??? BuiltinClassLoader$LoadedModule) so I cannot easily recreate >> ??? them at runtime. >> >> ??? However, these two loaders contain information specific to the >> ??? dump time VM lifecycle (such as the classes that were loaded >> ??? during CDS dumping) that need to be scrubbed. I couldn't find an >> ??? elegant way of doing this, so I added a private >> "resetArchivedStates" >> ??? method to a few classes. They are called inside >> ??? HeapShared::reset_archived_object_states(). >> >> [4] Related native data structures (PackageEntry and ModuleEntry) >> ??? are also archived. Start with classLoaderData.cpp >> >> Passes mach5 tiers 1-4. I will test with additional tiers. >> >> Thanks >> - Ioi > > classfile/classLoader.cpp > - line #1644 pulling the javabase_moduleEntry() check outside of the > Module_lock maybe problematic if after you check it another > ? thread initializes in the time between the check and the obtaining > of the Module_lock on line #1645. Fixed. > > classfile/classLoader.hpp > - somewhere in ArchivedClassLoaderData there should be an assert to > make sure that the CLD, whose package entries and module entries are > being archived is not a "_has_class_mirror_holder" CLD for a weakly > defined hidden class.? Those dedicated CLDs should never have package > entries or module entries. > I added ArchivedClassLoaderData::assert_valid() > classfile/moduleEntry.cpp > - line #400, typo "conver" --> "convert" > - line #500, maybe sort if n is greater than 1 instead of 0 (same > comment for packageEntry.cpp line #270) > Fixed > classfile/systemDictionary.cpp > - It looks like the PackageEntry and ModuleEntry tables for the system > and platform class loaders are? added within > SystemDictionary::compute_java_loaders() which is called from > Thread::create_vm() but after the call in Thread::create_vm() to > call_initPhase2 which is when Universe::_module_initialized is set to > true.? Did I read this correctly?? If yes, then I think you have to be > sure that Universe::_module_initialized is not set until the module > graph for the 3 builtin loaders is loaded from the archive. > I moved the code to be called by ModuleBootstrap::boot() so it should now happen inside call_initPhase2. > memory/filemap.cpp > - line #237 typo "modue" --> "module" > Fixed > - Since the CDS support for capturing the module graph does not > archive unnamed modules, I assume > Modules::set_bootloader_unnamed_module() is still called.? Is this > true or does the unnamed module for the boot loader get archived? The unnamed module for the boot loader is not archived. Modules::set_bootloader_unnamed_module()? wasn't called in my last webrev. Thanks for catching it. I added a call to BootLoader.getUnnamedModule() in ModuleBootstrap::boot()? to trigger of BootLoader, which will call into the VM for Modules::set_bootloader_unnamed_module(). > Clarification/timing questions: > Here's an overall problem I am facing: The module graph is archived after the module system has fully started up. This means that for the boot loader, we only know the full set of modules/packages, but we don't know which part is the subset that was initialized during early VM bootstraping (e.g., when ModuleEntryTable::javabase_defined() == false). So the behavior is a bit different during the early bootstrap phase (all the way before we reach ModuleBootstrap::boot()): ClassLoaderData::the_null_class_loader_data()->modules() and ClassLoaderData::the_null_class_loader_data()->packages() are already populated before a single class is loaded. This difference doesn't seem to make a practical difference. E.g., none of our code seems to assume that "before any classes in the java.util package is loaded, ClassLoaderData::the_null_class_loader_data()->packages() must not contain an entry for java.util". I think we have two choices when the archived module graph is used: [1] We require that the state of the module system must, at every step of VM initialization, be identical to that of a VM that doesn't use an archived module graph. [2] We make sure that the VM/JDK bootstrap code can tolerate the existence of module/packages that are added earlier than a VM that doesn't use an archived module graph. I tried doing a version of [1] and found that to be too difficult. [2] seems much simpler and is the approach I am using now. > oops/instanceKlass.cpp > - line #2545, comment "TO DO? -- point it to the archived PackageEntry > (JDK-8249262)" > are you thinking that since the module graph is read in ahead of time > that it can be set when an InstanceKlass is created?? There is a point > before java.base is defined that InstanceKlass::set_package takes into > account that could be a timing issue. > > I think it will work as if another class in the same package has already been defined. > - There are some checks in modules.cpp that are valuable to have > during bootstrapping.? For example, packages that are encountered > during bootstrapping before java.base is defined are put into the > java.base module but then verified after java.base is defined via > verify_javabase_packages.? So at what point in the bootstrapping > process does the boot loader's archived module's become known? Could > classes have been defined to the VM before this point?? Then their > packages will have to be verified to be sure that they are indeed > packages within java.base.? Consider looking at other checks that > occur within classfile/modules.cpp as well. > As mentioned above, calling verify_javabase_packages() at run time will fail, as we have loaded all packages for the boot loader, not just those for java.base. Well, verify_javabase_packages() was called once and it succeeded, but that was during CDS dump time. So we know at least we have verified this once :-) Thanks - Ioi > I may have more review comments as I continue to look at this! > > Thanks, > Lois > > > > > From ioi.lam at oracle.com Wed Aug 12 22:11:49 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 12 Aug 2020 15:11:49 -0700 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: References: Message-ID: Hi Coleen, Thanks for the comments. I have an updated webrev http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/ http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02.delta/ On 8/4/20 7:49 AM, Coleen Phillimore wrote: > This is sort of massive, so I'll only make a couple initial comments. > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html > > > All this new code in ClassLoaderData should go somewhere else, like > maybe classLoaderDataShared.hpp/cpp ?? I don't see how it needs to be > owned by CLD, except maybe the serialized function. > > Fixed > + if (loader_data) { > > > There are many checks that should be != NULL here. > Fixed and added comments > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/javaClasses.cpp.udiff.html > > > I don't see why compute_offset/s() has been exported outside > javaClasses.cpp. > It was left-over code that shouldn't be there. I removed it. > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/moduleEntry.cpp.udiff.html > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/packageEntry.cpp.udiff.html > > > These look like the same code to me.? Maybe they can be generalized > and put together in a moduleEntryShared.cpp/hpp ? > They look similar but also do different things. Any suggestion for how to generalize them? > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/systemDictionary.cpp.udiff.html > > > Why do you register the loaders here? > It's to force the creation of the ClassLoaderData if a loader oop doesn't have one yet. Also, in the new version, I moved the code to Modules::define_archived_modules(). Thanks - Ioi > I don't know enough about the Java code to review that. > > I assume that this going to be rebased on other changes, so this is > only an incomplete review. > > Coleen > > > On 7/22/20 3:36 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8244778 >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/ >> >> >> Please review this patch that stores the full module graph in the CDS >> archive heap. This reduces the initialization time of the basic JVM by >> about 22%: >> >> $ perf stat -r 100 bin/java -version >> before: 98,219,329 instructions 0.03971 secs elapsed (+- 0.44%) >> after:? 55,835,815 instructions 0.03109 secs elapsed (+- 0.65%) >> >> [1] Start with ModuleBootstrap.java. The current implementation is >> ??? quite restrictive: the archived module graph is used only when no >> ??? module options are specified. >> >> ??? See ModuleBootstrap.mayUseArchivedBootLayer(). >> >> ??? We can probably support options such as main module and module path >> ??? in a future RFE. >> >> [2] In the current JDK implementation, there is no single object >> ??? that represents "the module graph". Most of the information >> ??? is stored in the archive bootLayer object, but a few additional >> ??? restoration operations need to be performed: >> >> ??? + See ModuleBootstrap.getArchivedBootLayer() >> ??? + Some static fields need to be archived/restored in >> ????? Module.java, BuiltinClassLoader.java, ClassLoaders.java >> ????? and BootLoader.java >> >> [3] I ran into a complication with two loader instances of >> ??? PlatformClassLoader and AppClassLoader. They are stored in >> ??? multiple tables inside the module graph (e.g., >> ??? BuiltinClassLoader$LoadedModule) so I cannot easily recreate >> ??? them at runtime. >> >> ??? However, these two loaders contain information specific to the >> ??? dump time VM lifecycle (such as the classes that were loaded >> ??? during CDS dumping) that need to be scrubbed. I couldn't find an >> ??? elegant way of doing this, so I added a private >> "resetArchivedStates" >> ??? method to a few classes. They are called inside >> ??? HeapShared::reset_archived_object_states(). >> >> [4] Related native data structures (PackageEntry and ModuleEntry) >> ??? are also archived. Start with classLoaderData.cpp >> >> Passes mach5 tiers 1-4. I will test with additional tiers. >> >> Thanks >> - Ioi > From david.holmes at oracle.com Wed Aug 12 22:17:09 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Aug 2020 08:17:09 +1000 Subject: RFR (s): 8251460: Fix the biased-locking code in ObjectSynchronizer::FastHashCode In-Reply-To: <07130e08-6b75-fdc6-a5e6-393493c3391c@oracle.com> References: <797db6f1-e080-3a33-fefa-34d045a9c681@oracle.com> <07130e08-6b75-fdc6-a5e6-393493c3391c@oracle.com> Message-ID: <97f7bcfb-ea71-83ec-894f-df85922a247e@oracle.com> Hi Dan, On 13/08/2020 1:26 am, Daniel D. Daugherty wrote: >> webrev: http://cr.openjdk.java.net/~dholmes/8251460/webrev/ > > src/hotspot/share/runtime/synchronizer.cpp > ??? No comments. > > Thumbs up! Thanks for the review. > Thanks for the info about how this was tested. Any Mach5 testing > done yet? Putting it through t1-3 now. Will push once that passes. Thanks, David ----- > Dan > > > On 8/12/20 8:16 AM, David Holmes wrote: >> Thanks Coleen! >> >> I forgot to mention that the testing for this was to reenable (in >> arguments.cpp) biased-locking for CDS (and fix an errant assert that >> Ioi is going to fix) and run the CDS tests. None of those changes will >> be pushed of course. >> >> David >> >> On 12/08/2020 9:58 pm, Coleen Phillimore wrote: >>> Looks good! >>> Coleen >>> >>> On 8/12/20 12:23 AM, David Holmes wrote: >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8251460 >>>> webrev: http://cr.openjdk.java.net/~dholmes/8251460/webrev/ >>>> >>>> During the review for JDK-8250606 it was noted that the >>>> biased-locking section of FastHashCode contained a similar assertion >>>> regarding execution at a safepoint: >>>> >>>> ????? // Relaxing assertion for bug 6320749. >>>> ????? assert(Universe::verify_in_progress() || >>>> ???????????? !SafepointSynchronize::is_at_safepoint(), >>>> ???????????? "biases should not be seen by VM thread here"); >>>> >>>> which gives the impression that it is okay to execute this code >>>> block at a safepoint if verify_in_progress() is true. But if we >>>> examine the next line of code: >>>> >>>> ? BiasedLocking::revoke(hobj, JavaThread::current()); >>>> >>>> we find that at a safepoint: >>>> a) JavaThread::current() will assert because the current thread will >>>> be the VMThread; and >>>> b) BiasedLocking::revoke itself contains an assertion that it is not >>>> invoked at a safepoint. >>>> >>>> In practice there is no issue with revoking the bias at a safepoint, >>>> we just have to use the right code to do so (as done elsewhere in >>>> the same file): >>>> >>>> if (SafepointSynchronize::is_at_safepoint()) { >>>> ? BiasedLocking::revoke_at_safepoint(hobj); >>>> } else { >>>> ? BiasedLocking::revoke(hobj, self); >>>> } >>>> >>>> Thanks, >>>> David >>> > From david.holmes at oracle.com Wed Aug 12 22:17:56 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Aug 2020 08:17:56 +1000 Subject: RFR (s): 8251460: Fix the biased-locking code in ObjectSynchronizer::FastHashCode In-Reply-To: <13acfb8f-4a58-4560-91ed-c7e7b494fec3@oracle.com> References: <797db6f1-e080-3a33-fefa-34d045a9c681@oracle.com> <13acfb8f-4a58-4560-91ed-c7e7b494fec3@oracle.com> Message-ID: <6bbf8599-214f-f75a-f45b-47d029197add@oracle.com> Thanks Patricio! David On 13/08/2020 4:42 am, Patricio Chilano wrote: > Hi David, > > Changes look good to me. > > Thanks, > Patricio > On 8/12/20 1:23 AM, David Holmes wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8251460 >> webrev: http://cr.openjdk.java.net/~dholmes/8251460/webrev/ >> >> During the review for JDK-8250606 it was noted that the biased-locking >> section of FastHashCode contained a similar assertion regarding >> execution at a safepoint: >> >> ????? // Relaxing assertion for bug 6320749. >> ????? assert(Universe::verify_in_progress() || >> ???????????? !SafepointSynchronize::is_at_safepoint(), >> ???????????? "biases should not be seen by VM thread here"); >> >> which gives the impression that it is okay to execute this code block >> at a safepoint if verify_in_progress() is true. But if we examine the >> next line of code: >> >> ? BiasedLocking::revoke(hobj, JavaThread::current()); >> >> we find that at a safepoint: >> a) JavaThread::current() will assert because the current thread will >> be the VMThread; and >> b) BiasedLocking::revoke itself contains an assertion that it is not >> invoked at a safepoint. >> >> In practice there is no issue with revoking the bias at a safepoint, >> we just have to use the right code to do so (as done elsewhere in the >> same file): >> >> if (SafepointSynchronize::is_at_safepoint()) { >> ? BiasedLocking::revoke_at_safepoint(hobj); >> } else { >> ? BiasedLocking::revoke(hobj, self); >> } >> >> Thanks, >> David > From suenaga at oss.nttdata.com Thu Aug 13 01:12:29 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 13 Aug 2020 10:12:29 +0900 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> Message-ID: Hi Matthias, David, I measured startup benchmarks with `Measure-Command {.\jdk\build\windows-x86_64-server-release\images\jdk\bin\java.exe --version}` on PowerShell. * PC: Ryzen 3 3300X, 16GB memory * OS: Windows 10 x64 (May 2020 Update) * Java: jdk/jdk revision 60537 (Compiled by VS 2019 (16.7.0)) * without patch TotalMilliseconds : 70.2124 TotalMilliseconds : 64.4465 TotalMilliseconds : 59.0854 TotalMilliseconds : 68.0255 TotalMilliseconds : 72.6467 average : 66.8833 * with webrev.01 TotalMilliseconds : 81.7185 TotalMilliseconds : 68.539 TotalMilliseconds : 85.7226 TotalMilliseconds : 72.6584 TotalMilliseconds : 75.6091 average : 76.84952 Overhead of WMI seems to be +10ms in this case. Yasumasa On 2020/08/13 0:05, Baesken, Matthias wrote: >> I understand that if the process runs on Xen on other hypervisor (e.g. KVM), information for Xen would be set between 0x40000100 and 0x40010000. >> Ok, I will not remove the loop in new webrev, and will add comment about it. > > Hi Yasumasa , thanks ! > > Regarding the WMI overhead , if you could get some more info on this it would be better to judge if it is perfectly okay or concerning . > > (I remember that I looked into it a couple of years ago , but decided not to use it in early VM startup ; but have to confess this was just based on a "gut feeling", > No micro benchmarks ) > > Best regards, Matthias > > From david.holmes at oracle.com Thu Aug 13 02:54:57 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Aug 2020 12:54:57 +1000 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> Message-ID: <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> On 13/08/2020 11:12 am, Yasumasa Suenaga wrote: > Hi Matthias, David, > > I measured startup benchmarks with `Measure-Command > {.\jdk\build\windows-x86_64-server-release\images\jdk\bin\java.exe > --version}` on PowerShell. > > * PC: Ryzen 3 3300X, 16GB memory > * OS: Windows 10 x64 (May 2020 Update) > * Java: jdk/jdk revision 60537 > ??? (Compiled by VS 2019 (16.7.0)) > > * without patch > TotalMilliseconds : 70.2124 > TotalMilliseconds : 64.4465 > TotalMilliseconds : 59.0854 > TotalMilliseconds : 68.0255 > TotalMilliseconds : 72.6467 > average : 66.8833 > > * with webrev.01 > TotalMilliseconds : 81.7185 > TotalMilliseconds : 68.539 > TotalMilliseconds : 85.7226 > TotalMilliseconds : 72.6584 > TotalMilliseconds : 75.6091 > average : 76.84952 > > Overhead of WMI seems to be +10ms in this case. Which is nearly 15%! Sorry but I just know Claes will be very unhappy if this were to go in as-is. David ----- > > Yasumasa > > > On 2020/08/13 0:05, Baesken, Matthias wrote: >>> I understand that if the process runs on Xen on other hypervisor >>> (e.g. KVM), information for Xen would be set between 0x40000100 and >>> 0x40010000. >>> Ok, I will not remove the loop in new webrev, and will add comment >>> about it. >> >> Hi Yasumasa? , thanks ! >> >> Regarding the WMI overhead , if you could? get some more info on this >> it would be better to judge if it is perfectly okay or? concerning . >> >> (I remember that I looked into it a couple of years ago , but decided >> not to use it in early VM startup ;? but have to confess this was just >> based on a "gut feeling", >> ? No micro benchmarks? ) >> >> Best regards, Matthias >> >> From suenaga at oss.nttdata.com Thu Aug 13 04:15:16 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 13 Aug 2020 13:15:16 +0900 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> Message-ID: On 2020/08/13 11:54, David Holmes wrote: > On 13/08/2020 11:12 am, Yasumasa Suenaga wrote: >> Hi Matthias, David, >> >> I measured startup benchmarks with `Measure-Command {.\jdk\build\windows-x86_64-server-release\images\jdk\bin\java.exe --version}` on PowerShell. >> >> * PC: Ryzen 3 3300X, 16GB memory >> * OS: Windows 10 x64 (May 2020 Update) >> * Java: jdk/jdk revision 60537 >> ???? (Compiled by VS 2019 (16.7.0)) >> >> * without patch >> TotalMilliseconds : 70.2124 >> TotalMilliseconds : 64.4465 >> TotalMilliseconds : 59.0854 >> TotalMilliseconds : 68.0255 >> TotalMilliseconds : 72.6467 >> average : 66.8833 >> >> * with webrev.01 >> TotalMilliseconds : 81.7185 >> TotalMilliseconds : 68.539 >> TotalMilliseconds : 85.7226 >> TotalMilliseconds : 72.6584 >> TotalMilliseconds : 75.6091 >> average : 76.84952 >> >> Overhead of WMI seems to be +10ms in this case. > > Which is nearly 15%! Sorry but I just know Claes will be very unhappy if this were to go in as-is. Should we make the change to determine just before it is needed (e.g. VM.info or hs_err log) at first? It is out of scope of this change, so I want to work as another issue if it is needed. Yasumasa > David > ----- > >> >> Yasumasa >> >> >> On 2020/08/13 0:05, Baesken, Matthias wrote: >>>> I understand that if the process runs on Xen on other hypervisor (e.g. KVM), information for Xen would be set between 0x40000100 and 0x40010000. >>>> Ok, I will not remove the loop in new webrev, and will add comment about it. >>> >>> Hi Yasumasa? , thanks ! >>> >>> Regarding the WMI overhead , if you could? get some more info on this it would be better to judge if it is perfectly okay or? concerning . >>> >>> (I remember that I looked into it a couple of years ago , but decided not to use it in early VM startup ;? but have to confess this was just based on a "gut feeling", >>> ? No micro benchmarks? ) >>> >>> Best regards, Matthias >>> >>> From david.holmes at oracle.com Thu Aug 13 06:20:09 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Aug 2020 16:20:09 +1000 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> Message-ID: <88d9af48-025e-9ae2-47f8-8e5e6cef3685@oracle.com> On 13/08/2020 2:15 pm, Yasumasa Suenaga wrote: > On 2020/08/13 11:54, David Holmes wrote: >> On 13/08/2020 11:12 am, Yasumasa Suenaga wrote: >>> Hi Matthias, David, >>> >>> I measured startup benchmarks with `Measure-Command >>> {.\jdk\build\windows-x86_64-server-release\images\jdk\bin\java.exe >>> --version}` on PowerShell. >>> >>> * PC: Ryzen 3 3300X, 16GB memory >>> * OS: Windows 10 x64 (May 2020 Update) >>> * Java: jdk/jdk revision 60537 >>> ???? (Compiled by VS 2019 (16.7.0)) >>> >>> * without patch >>> TotalMilliseconds : 70.2124 >>> TotalMilliseconds : 64.4465 >>> TotalMilliseconds : 59.0854 >>> TotalMilliseconds : 68.0255 >>> TotalMilliseconds : 72.6467 >>> average : 66.8833 >>> >>> * with webrev.01 >>> TotalMilliseconds : 81.7185 >>> TotalMilliseconds : 68.539 >>> TotalMilliseconds : 85.7226 >>> TotalMilliseconds : 72.6584 >>> TotalMilliseconds : 75.6091 >>> average : 76.84952 >>> >>> Overhead of WMI seems to be +10ms in this case. >> >> Which is nearly 15%! Sorry but I just know Claes will be very unhappy >> if this were to go in as-is. > > Should we make the change to determine just before it is needed (e.g. > VM.info or hs_err log) at first? > It is out of scope of this change, so I want to work as another issue if > it is needed. Okay use another issue to do deferred initialization of detailed version info that may not be used. I'm not sure exactly what that covers. Concurrent initialization may be an issue to consider. Thanks, David P.S I have a long weekend coming up. > > Yasumasa > > >> David >> ----- >> >>> >>> Yasumasa >>> >>> >>> On 2020/08/13 0:05, Baesken, Matthias wrote: >>>>> I understand that if the process runs on Xen on other hypervisor >>>>> (e.g. KVM), information for Xen would be set between 0x40000100 and >>>>> 0x40010000. >>>>> Ok, I will not remove the loop in new webrev, and will add comment >>>>> about it. >>>> >>>> Hi Yasumasa? , thanks ! >>>> >>>> Regarding the WMI overhead , if you could? get some more info on >>>> this it would be better to judge if it is perfectly okay or >>>> concerning . >>>> >>>> (I remember that I looked into it a couple of years ago , but >>>> decided not to use it in early VM startup ;? but have to confess >>>> this was just based on a "gut feeling", >>>> ? No micro benchmarks? ) >>>> >>>> Best regards, Matthias >>>> >>>> From matthias.baesken at sap.com Thu Aug 13 06:52:28 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Thu, 13 Aug 2020 06:52:28 +0000 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <05f4a122-6b17-37f9-5ee4-228704308a6f@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> Message-ID: > >Should we make the change to determine just before it is needed (e.g. VM.info or hs_err log) at first? >It is out of scope of this change, so I want to work as another issue if it is needed. > Hi , I think we better do not call into WMI , when writing an hs_err file . Best regards, Matthias -----Original Message----- From: Yasumasa Suenaga Sent: Donnerstag, 13. August 2020 06:15 To: David Holmes ; Baesken, Matthias ; hotspot-runtime-dev at openjdk.java.net; build-dev at openjdk.java.net Cc: Doerr, Martin Subject: Re: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS On 2020/08/13 11:54, David Holmes wrote: > On 13/08/2020 11:12 am, Yasumasa Suenaga wrote: >> Hi Matthias, David, >> >> I measured startup benchmarks with `Measure-Command {.\jdk\build\windows-x86_64-server-release\images\jdk\bin\java.exe --version}` on PowerShell. >> >> * PC: Ryzen 3 3300X, 16GB memory >> * OS: Windows 10 x64 (May 2020 Update) >> * Java: jdk/jdk revision 60537 >> ???? (Compiled by VS 2019 (16.7.0)) >> >> * without patch >> TotalMilliseconds : 70.2124 >> TotalMilliseconds : 64.4465 >> TotalMilliseconds : 59.0854 >> TotalMilliseconds : 68.0255 >> TotalMilliseconds : 72.6467 >> average : 66.8833 >> >> * with webrev.01 >> TotalMilliseconds : 81.7185 >> TotalMilliseconds : 68.539 >> TotalMilliseconds : 85.7226 >> TotalMilliseconds : 72.6584 >> TotalMilliseconds : 75.6091 >> average : 76.84952 >> >> Overhead of WMI seems to be +10ms in this case. > > Which is nearly 15%! Sorry but I just know Claes will be very unhappy if this were to go in as-is. Should we make the change to determine just before it is needed (e.g. VM.info or hs_err log) at first? It is out of scope of this change, so I want to work as another issue if it is needed. Yasumasa > David > ----- > >> >> Yasumasa >> >> >> On 2020/08/13 0:05, Baesken, Matthias wrote: >>>> I understand that if the process runs on Xen on other hypervisor (e.g. KVM), information for Xen would be set between 0x40000100 and 0x40010000. >>>> Ok, I will not remove the loop in new webrev, and will add comment about it. >>> >>> Hi Yasumasa? , thanks ! >>> >>> Regarding the WMI overhead , if you could? get some more info on this it would be better to judge if it is perfectly okay or? concerning . >>> >>> (I remember that I looked into it a couple of years ago , but decided not to use it in early VM startup ;? but have to confess this was just based on a "gut feeling", >>> ? No micro benchmarks? ) >>> >>> Best regards, Matthias >>> >>> From suenaga at oss.nttdata.com Thu Aug 13 07:39:46 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Thu, 13 Aug 2020 16:39:46 +0900 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <247113e2-ca18-85c2-4743-e73ddd7dca14@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> Message-ID: Hi Matthias, David, On 2020/08/13 15:52, Baesken, Matthias wrote: > >> >> Should we make the change to determine just before it is needed (e.g. VM.info or hs_err log) at first? >> It is out of scope of this change, so I want to work as another issue if it is needed. >> > > Hi , > I think we better do not call into WMI , when writing an hs_err file . I found out from Hypervisor Top Level Functional Specification [1] section 2.4.8. That says CPUID leaf 0x40000007 is available to the root partition only. I changed VM_Version::is_in_VM() as below, and it works fine. (I use __cpuid intrinsic in here because this code would be compiled by cl.exe only.) ``` #include #define EAX 0 #define EBX 1 #define ECX 2 #define EDX 3 bool VM_Version::is_in_VM() { int regs[4]; __cpuid(regs, 0x40000007); // CPUID leaf 0x40000007 is available to the root partition only. return (regs[EAX] == 0x0) && (regs[EBX] == 0x0) && (regs[ECX] == 0x0) && (regs[EDX] == 0x0); } ``` Also startup performance is better than WMI call. TotalMilliseconds : 66.7863 TotalMilliseconds : 56.8123 TotalMilliseconds : 57.0015 TotalMilliseconds : 62.4755 TotalMilliseconds : 67.7632 average : 62.16776 If you are ok this change, I will update webrev, and will send review request. (new webrev will include both loop for CPUID leaf and renaming is_in_VM) Thanks, Yasumasa [1] https://github.com/MicrosoftDocs/Virtualization-Documentation/raw/master/tlfs/Hypervisor%20Top%20Level%20Functional%20Specification%20v6.0b.pdf > Best regards, Matthias > > > > -----Original Message----- > From: Yasumasa Suenaga > Sent: Donnerstag, 13. August 2020 06:15 > To: David Holmes ; Baesken, Matthias ; hotspot-runtime-dev at openjdk.java.net; build-dev at openjdk.java.net > Cc: Doerr, Martin > Subject: Re: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS > > On 2020/08/13 11:54, David Holmes wrote: >> On 13/08/2020 11:12 am, Yasumasa Suenaga wrote: >>> Hi Matthias, David, >>> >>> I measured startup benchmarks with `Measure-Command {.\jdk\build\windows-x86_64-server-release\images\jdk\bin\java.exe --version}` on PowerShell. >>> >>> * PC: Ryzen 3 3300X, 16GB memory >>> * OS: Windows 10 x64 (May 2020 Update) >>> * Java: jdk/jdk revision 60537 >>> ???? (Compiled by VS 2019 (16.7.0)) >>> >>> * without patch >>> TotalMilliseconds : 70.2124 >>> TotalMilliseconds : 64.4465 >>> TotalMilliseconds : 59.0854 >>> TotalMilliseconds : 68.0255 >>> TotalMilliseconds : 72.6467 >>> average : 66.8833 >>> >>> * with webrev.01 >>> TotalMilliseconds : 81.7185 >>> TotalMilliseconds : 68.539 >>> TotalMilliseconds : 85.7226 >>> TotalMilliseconds : 72.6584 >>> TotalMilliseconds : 75.6091 >>> average : 76.84952 >>> >>> Overhead of WMI seems to be +10ms in this case. >> >> Which is nearly 15%! Sorry but I just know Claes will be very unhappy if this were to go in as-is. > > Should we make the change to determine just before it is needed (e.g. VM.info or hs_err log) at first? > It is out of scope of this change, so I want to work as another issue if it is needed. > > > Yasumasa > > >> David >> ----- >> >>> >>> Yasumasa >>> >>> >>> On 2020/08/13 0:05, Baesken, Matthias wrote: >>>>> I understand that if the process runs on Xen on other hypervisor (e.g. KVM), information for Xen would be set between 0x40000100 and 0x40010000. >>>>> Ok, I will not remove the loop in new webrev, and will add comment about it. >>>> >>>> Hi Yasumasa? , thanks ! >>>> >>>> Regarding the WMI overhead , if you could? get some more info on this it would be better to judge if it is perfectly okay or? concerning . >>>> >>>> (I remember that I looked into it a couple of years ago , but decided not to use it in early VM startup ;? but have to confess this was just based on a "gut feeling", >>>> ? No micro benchmarks? ) >>>> >>>> Best regards, Matthias >>>> >>>> From magnus.ihse.bursie at oracle.com Thu Aug 13 10:02:17 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 13 Aug 2020 12:02:17 +0200 Subject: RFR (T) 8251489: universe.cpp includes too many headers In-Reply-To: <27921cdb-10a3-c3ba-469c-a7410b0d667f@oracle.com> References: <2b1d8a9e-126c-48c9-5eaa-4e53e5f6ad8e@oracle.com> <27921cdb-10a3-c3ba-469c-a7410b0d667f@oracle.com> Message-ID: <975e6daa-4798-ef74-9135-d3ee8ba1c420@oracle.com> On 2020-08-12 18:36, Coleen Phillimore wrote: > > > On 8/12/20 12:34 PM, Thomas St?fe wrote: >> This looks good. I wish we could do this automatically. > > Thanks, Thomas, I agree! I can probably add something to the makefiles along the lines of this https://stackoverflow.com/a/7135530/432034. But when should it be run? Regularly, or just whenever someone feels there's too many headers again? Doing this as part of a schedule is likely to be a much harder problem, so for one-shot runs it's probably fairly trivial to solve. /Magnus > Coleen >> >> Thanks, Thomas >> >> On Wed, Aug 12, 2020 at 5:33 PM Coleen Phillimore >> > >> wrote: >> >> ??? See bug for description. Tested with tier1 on 4 platforms, built >> with >> ??? --no-precompiled-headers. >> >> ??? open webrev at >> ??? http://cr.openjdk.java.net/~coleenp/2020/8251489.01/webrev >> ??? bug link https://bugs.openjdk.java.net/browse/JDK-8251489 >> >> ??? Thanks, >> ??? Coleen >> > From david.holmes at oracle.com Thu Aug 13 11:39:49 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 13 Aug 2020 21:39:49 +1000 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> Message-ID: <08584797-0841-945e-ec6d-7050a12326b4@oracle.com> On 13/08/2020 5:39 pm, Yasumasa Suenaga wrote: > Hi Matthias, David, > > On 2020/08/13 15:52, Baesken, Matthias wrote: >> >>> >>> Should we make the change to determine just before it is needed (e.g. >>> VM.info or hs_err log) at first? >>> It is out of scope of this change, so I want to work as another issue >>> if it is needed. >>> >> >> Hi , >> I think we better do not call into WMI , when? writing an hs_err file?? . > > I found out from Hypervisor Top Level Functional Specification [1] > section 2.4.8. > That says CPUID leaf 0x40000007 is available to the root partition only. > > I changed VM_Version::is_in_VM() as below, and it works fine. > (I use __cpuid intrinsic in here because this code would be compiled by > cl.exe only.) > > ``` > #include > > #define EAX 0 > #define EBX 1 > #define ECX 2 > #define EDX 3 > > bool VM_Version::is_in_VM() { > ? int regs[4]; > ? __cpuid(regs, 0x40000007); > > ? // CPUID leaf 0x40000007 is available to the root partition only. > ? return (regs[EAX] == 0x0) && (regs[EBX] == 0x0) && (regs[ECX] == 0x0) > && (regs[EDX] == 0x0); > } > ``` > > Also startup performance is better than WMI call. > > TotalMilliseconds : 66.7863 > TotalMilliseconds : 56.8123 > TotalMilliseconds : 57.0015 > TotalMilliseconds : 62.4755 > TotalMilliseconds : 67.7632 > average : 62.16776 > > If you are ok this change, I will update webrev, and will send review > request. > (new webrev will include both loop for CPUID leaf and renaming is_in_VM) If it works and addresses the startup hit then I'm fine with it - but still can't actually review the code. Thanks, David > > Thanks, > > Yasumasa > > > [1] > https://github.com/MicrosoftDocs/Virtualization-Documentation/raw/master/tlfs/Hypervisor%20Top%20Level%20Functional%20Specification%20v6.0b.pdf > > > >> Best regards, Matthias >> >> >> >> -----Original Message----- >> From: Yasumasa Suenaga >> Sent: Donnerstag, 13. August 2020 06:15 >> To: David Holmes ; Baesken, Matthias >> ; hotspot-runtime-dev at openjdk.java.net; >> build-dev at openjdk.java.net >> Cc: Doerr, Martin >> Subject: Re: PING: RFR: 8250598: Hyper-V is detected in spite of >> running on host OS >> >> On 2020/08/13 11:54, David Holmes wrote: >>> On 13/08/2020 11:12 am, Yasumasa Suenaga wrote: >>>> Hi Matthias, David, >>>> >>>> I measured startup benchmarks with `Measure-Command >>>> {.\jdk\build\windows-x86_64-server-release\images\jdk\bin\java.exe >>>> --version}` on PowerShell. >>>> >>>> * PC: Ryzen 3 3300X, 16GB memory >>>> * OS: Windows 10 x64 (May 2020 Update) >>>> * Java: jdk/jdk revision 60537 >>>> ????? (Compiled by VS 2019 (16.7.0)) >>>> >>>> * without patch >>>> TotalMilliseconds : 70.2124 >>>> TotalMilliseconds : 64.4465 >>>> TotalMilliseconds : 59.0854 >>>> TotalMilliseconds : 68.0255 >>>> TotalMilliseconds : 72.6467 >>>> average : 66.8833 >>>> >>>> * with webrev.01 >>>> TotalMilliseconds : 81.7185 >>>> TotalMilliseconds : 68.539 >>>> TotalMilliseconds : 85.7226 >>>> TotalMilliseconds : 72.6584 >>>> TotalMilliseconds : 75.6091 >>>> average : 76.84952 >>>> >>>> Overhead of WMI seems to be +10ms in this case. >>> >>> Which is nearly 15%! Sorry but I just know Claes will be very unhappy >>> if this were to go in as-is. >> >> Should we make the change to determine just before it is needed (e.g. >> VM.info or hs_err log) at first? >> It is out of scope of this change, so I want to work as another issue >> if it is needed. >> >> >> Yasumasa >> >> >>> David >>> ----- >>> >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/08/13 0:05, Baesken, Matthias wrote: >>>>>> I understand that if the process runs on Xen on other hypervisor >>>>>> (e.g. KVM), information for Xen would be set between 0x40000100 >>>>>> and 0x40010000. >>>>>> Ok, I will not remove the loop in new webrev, and will add comment >>>>>> about it. >>>>> >>>>> Hi Yasumasa? , thanks ! >>>>> >>>>> Regarding the WMI overhead , if you could? get some more info on >>>>> this it would be better to judge if it is perfectly okay or >>>>> concerning . >>>>> >>>>> (I remember that I looked into it a couple of years ago , but >>>>> decided not to use it in early VM startup ;? but have to confess >>>>> this was just based on a "gut feeling", >>>>> ?? No micro benchmarks? ) >>>>> >>>>> Best regards, Matthias >>>>> >>>>> From coleen.phillimore at oracle.com Thu Aug 13 13:19:59 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 13 Aug 2020 09:19:59 -0400 Subject: RFR (T) 8251489: universe.cpp includes too many headers In-Reply-To: <975e6daa-4798-ef74-9135-d3ee8ba1c420@oracle.com> References: <2b1d8a9e-126c-48c9-5eaa-4e53e5f6ad8e@oracle.com> <27921cdb-10a3-c3ba-469c-a7410b0d667f@oracle.com> <975e6daa-4798-ef74-9135-d3ee8ba1c420@oracle.com> Message-ID: On 8/13/20 6:02 AM, Magnus Ihse Bursie wrote: > On 2020-08-12 18:36, Coleen Phillimore wrote: >> >> >> On 8/12/20 12:34 PM, Thomas St?fe wrote: >>> This looks good. I wish we could do this automatically. >> >> Thanks, Thomas, I agree! > I can probably add something to the makefiles along the lines of this > https://stackoverflow.com/a/7135530/432034. But when should it be run? > Regularly, or just whenever someone feels there's too many headers > again? Doing this as part of a schedule is likely to be a much harder > problem, so for one-shot runs it's probably fairly trivial to solve. Actually, I agree in principle that it would be nice if this could be done automatically, but only as a one-shot thing by an individual who wants to clean this up.? This falls into the nice-to-do list when someone has time and energy for it.? I'd hate to see it permanent because there could be false positives as someone writes in the stackoverflow article and for most cpp files, the cost of the extra header includes is quite low. Coleen > > /Magnus > > > >> Coleen >>> >>> Thanks, Thomas >>> >>> On Wed, Aug 12, 2020 at 5:33 PM Coleen Phillimore >>> > >>> wrote: >>> >>> ??? See bug for description. Tested with tier1 on 4 platforms, built >>> with >>> ??? --no-precompiled-headers. >>> >>> ??? open webrev at >>> ??? http://cr.openjdk.java.net/~coleenp/2020/8251489.01/webrev >>> ??? bug link https://bugs.openjdk.java.net/browse/JDK-8251489 >>> >>> ??? Thanks, >>> ??? Coleen >>> >> > From daniel.daugherty at oracle.com Thu Aug 13 14:58:11 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 13 Aug 2020 10:58:11 -0400 Subject: RFR(T): 8251543: add mention of INFLATING() to share/oops/markWord.hpp header comment Message-ID: <1a61123a-a289-c88a-de31-de0f8d0d9eda@oracle.com> Greetings, I have a trivial comment addition to src/hotspot/share/oops/markWord.hpp. Here's the bug: ??? JDK-8251543 add mention of INFLATING() to share/oops/markWord.hpp header comment ??? https://bugs.openjdk.java.net/browse/JDK-8251543 And here's the proposed fix: $ hg diff diff -r 73778bfbffe3 src/hotspot/share/oops/markWord.hpp --- a/src/hotspot/share/oops/markWord.hpp??? Thu Aug 13 10:02:35 2020 -0400 +++ b/src/hotspot/share/oops/markWord.hpp??? Thu Aug 13 10:51:44 2020 -0400 @@ -86,6 +86,10 @@ ?//??? [ptr???????????? | 11]? marked???????????? used to mark an object ?// ?//??? We assume that stack/thread pointers have the lowest two bits cleared. +// +//? - INFLATING() is a distinguished markword value that is used when +//??? inflating an existing stack lock into an ObjectMonitor. See below +//??? for is_being_inflated() and INFLATING(). ?class BasicLock; ?class ObjectMonitor; @@ -226,7 +230,7 @@ ?? bool is_being_inflated() const { return (value() == 0); } ?? // Distinguished markword value - used when inflating over -? // an existing stacklock.? 0 indicates the markword is "BUSY". +? // an existing stack lock.? 0 indicates the markword is "BUSY". ?? // Lockword mutators that use a LD...CAS idiom should always ?? // check for and avoid overwriting a 0 value installed by some ?? // other thread.? (They should spin or block instead.? The 0 value Thanks, in advance, for any comments, questions or suggestions. Dan From thomas.stuefe at gmail.com Thu Aug 13 17:30:09 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 13 Aug 2020 19:30:09 +0200 Subject: RFR (T) 8251489: universe.cpp includes too many headers In-Reply-To: References: <2b1d8a9e-126c-48c9-5eaa-4e53e5f6ad8e@oracle.com> <27921cdb-10a3-c3ba-469c-a7410b0d667f@oracle.com> <975e6daa-4798-ef74-9135-d3ee8ba1c420@oracle.com> Message-ID: +1 Having such a script for manual execution would be very welcome. Something to do occasionally, when bored. Or in the course of a large patch. It would be nice if the script could run optionally only on files which have been changed, or are part of a patch set. Thanks, Thomas On Thu, Aug 13, 2020, 15:20 Coleen Phillimore wrote: > > > On 8/13/20 6:02 AM, Magnus Ihse Bursie wrote: > > On 2020-08-12 18:36, Coleen Phillimore wrote: > >> > >> > >> On 8/12/20 12:34 PM, Thomas St?fe wrote: > >>> This looks good. I wish we could do this automatically. > >> > >> Thanks, Thomas, I agree! > > I can probably add something to the makefiles along the lines of this > > https://stackoverflow.com/a/7135530/432034. But when should it be run? > > Regularly, or just whenever someone feels there's too many headers > > again? Doing this as part of a schedule is likely to be a much harder > > problem, so for one-shot runs it's probably fairly trivial to solve. > Actually, I agree in principle that it would be nice if this could be > done automatically, but only as a one-shot thing by an individual who > wants to clean this up. This falls into the nice-to-do list when > someone has time and energy for it. I'd hate to see it permanent > because there could be false positives as someone writes in the > stackoverflow article and for most cpp files, the cost of the extra > header includes is quite low. > > Coleen > > > > > /Magnus > > > > > > > >> Coleen > >>> > >>> Thanks, Thomas > >>> > >>> On Wed, Aug 12, 2020 at 5:33 PM Coleen Phillimore > >>> > > >>> wrote: > >>> > >>> See bug for description. Tested with tier1 on 4 platforms, built > >>> with > >>> --no-precompiled-headers. > >>> > >>> open webrev at > >>> http://cr.openjdk.java.net/~coleenp/2020/8251489.01/webrev > >>> bug link https://bugs.openjdk.java.net/browse/JDK-8251489 > >>> > >>> Thanks, > >>> Coleen > >>> > >> > > > > From peter.tribble at gmail.com Thu Aug 13 18:49:00 2020 From: peter.tribble at gmail.com (Peter Tribble) Date: Thu, 13 Aug 2020 19:49:00 +0100 Subject: RFR: 8251365: Build failure on AIX after 8250636 In-Reply-To: <48c0d4f2-ffa7-a242-bb5a-f53781eaa5ff@azul.com> References: <48c0d4f2-ffa7-a242-bb5a-f53781eaa5ff@azul.com> Message-ID: On Mon, Aug 10, 2020 at 9:27 PM Dmitry Cherepanov wrote: > Please review the fix for build failure on AIX. With this change, > "tm_gmtoff" is used only if _ALLBSD_SOURCE or _GNU_SOURCE is set. In the > case of AIX, we'll continue to use "timezone". > > I'll start jdk-submit testing for this change shortly, unfortunately I do > not have an AIX machine at hand and would appreciate if someone (with > access to AIX) could verify that the change fixes AIX build . Thanks in > advance. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251365 > Webrev: http://cr.openjdk.java.net/~dcherepanov/8251365/webrev.v1/ > Apologies for butting in at this late stage, but this doesn't look quite right. It still breaks on illumos/Solaris (not supported, but I'm continuing with an external gcc-based port). The reason is that guarding with _GNU_SOURCE isn't the correct discriminant. I think this is set up in this block of make/autoconf/flags-cflags.m4 #### TOOLCHAIN DEFINES if test "x$TOOLCHAIN_TYPE" = xgcc; then ALWAYS_DEFINES_JVM="-D_GNU_SOURCE -D_REENTRANT" ALWAYS_DEFINES_JDK="-D_GNU_SOURCE -D_REENTRANT -D_LARGEFILE64_SOURCE" elif test "x$TOOLCHAIN_TYPE" = xclang; then ALWAYS_DEFINES_JVM="-D_GNU_SOURCE" elif test "x$TOOLCHAIN_TYPE" = xxlc; then ALWAYS_DEFINES_JVM="-D_REENTRANT" ALWAYS_DEFINES_JDK="-D_GNU_SOURCE -D_REENTRANT -D_LARGEFILE64_SOURCE -DSTDC" elif test "x$TOOLCHAIN_TYPE" = xmicrosoft; then ALWAYS_DEFINES_JDK="-DWIN32_LEAN_AND_MEAN -D_CRT_SECURE_NO_DEPRECATE \ -D_CRT_NONSTDC_NO_DEPRECATE -DWIN32 -DIAL" ALWAYS_DEFINES_JVM="-DNOMINMAX -DWIN32_LEAN_AND_MEAN" fi In other words, you get _GNU_SOURCE defined if you're using gcc or clang; the fix works on AIX by accident, as a side-effect of using the xlc toolchain there. I'll work out a way of patching it for my own work, but it may be worth looking at a proper fix. Thanks, -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ From calvin.cheung at oracle.com Thu Aug 13 19:12:33 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Thu, 13 Aug 2020 12:12:33 -0700 Subject: RFR(M) 8250990 Consolidate object copying code for CDS static/dynamic archive dumping In-Reply-To: References: <07b2ee18-0a5f-0839-0a6c-b54d788e4cd2@oracle.com> <39423dce-d769-bb92-4d22-9dc848d1086e@oracle.com> Message-ID: <89530161-c6c2-0e53-d2e7-38b046290621@oracle.com> Hi Ioi, This looks pretty good. One question on archiveBuilder.cpp: ?246 int ArchiveBuilder::compare_symbols_by_address(Symbol** a, Symbol** b) { ?247?? if (a[0] < b[0]) { ?248???? return -1; ?249?? } else { ?250???? assert(a[0] > b[0], "Duplicated symbol %s unexpected", (*a)->as_C_string()); ?251???? return 1; ?252?? } ?253 } It seems the (a[0] == b[0]) case and a ResourceMark are missing. The old code in metaSpaceShared.cpp has the following: ?669 static int compare_symbols_by_address(Symbol** a, Symbol** b) { ?670?? if (a[0] < b[0]) { ?671???? return -1; ?672?? } else if (a[0] == b[0]) { ?673???? ResourceMark rm; ?674???? log_warning(cds)("Duplicated symbol %s unexpected", (*a)->as_C_string()); ?675???? return 0; ?676?? } else { ?677???? return 1; ?678?? } ?679 } thanks, Calvin On 8/3/20 10:59 PM, Ioi Lam wrote: > Here's an updated version after offline discussion with Coleen: > > http://cr.openjdk.java.net/~iklam/jdk16/8250990-consolidate-static-dynamic-cds-objcopy.v02/ > > http://cr.openjdk.java.net/~iklam/jdk16/8250990-consolidate-static-dynamic-cds-objcopy.v02-delta/ > > > - Moved DumpAllocStats into its own cpp/hpp files > - Removed the use of inline.hpp files from archiveBuilder.hpp > - Changed comments about the old ArchiveCompactor class to > ? the new AcrchiveBuilder class. > - Removed CDSAddSymbol_lock which is no longer used. > > Thanks > - Ioi > > On 8/3/20 3:11 PM, Ioi Lam wrote: >> There turns out to be a lot of duplication in the static/dynamic >> dumping code. To keep the changes manageable, I've reorganized the >> REF into a few sub-tasks. This particular review is for: >> >> https://bugs.openjdk.java.net/browse/JDK-8250990 >> http://cr.openjdk.java.net/~iklam/jdk16/8250990-consolidate-static-dynamic-cds-objcopy.v01/ >> >> https://wiki.openjdk.java.net/display/HotSpot/How+CDS+Copies+Class+Metadata+into+the+Archive >> >> >> I've also updated the e-mail subject to use the new JBS ID >> (JDK-8250990). >> >> This patch is an intermediate step. Some code might look like they >> are in the wrong place -- for example: _estimated_metsapceobj_bytes >> has been moved to archiveBuilder.hpp, but _estimated_hashtable_bytes >> is still in dynamicArchive.cpp. The latter will eventually be moved >> as part of the second step (JDK-8250989 >> Consolidate buffer allocation code for CDS static/dynamic archive >> dumping). >> >> (Coleen has given me feedback off-line. I'll post a new version soon >> to limit the API in archiveBuilder.hpp) >> >> Thanks >> - Ioi >> >> On 7/31/20 11:36 PM, Ioi Lam wrote: >>> >>> Currently there are two very similar implementations for static CDS >>> archive >>> dumping (metaspaceShared.cpp) and dynamic CDS archive dumping >>> (dynamicArchive.cpp). >>> >>> I've merged the two versions and moved to a new file, >>> archiveBuilder.cpp. >>> >>> I also improved the copying algorithm to speed up dumping. Times for >>> dumping >>> 20000+ classes: >>> >>> ? ?? ????? static??????? dynamic >>> ??? Old??? 42.655 sec??? 67.014 sec >>> ??? New??? 37.027 sec??? 34.974 sec >>> >>> Normally I would not try to optimize when refactoring code. I made an >>> earlier, naive attempt to use the old dynamic dumper for static dumping >>> (because dynamic dumping is essentially a superset of static dumping). >>> However, that caused significant regression in static dumping speed. >>> >>> Hopefully the new algorithm is both faster and easier to understand. >>> Please see the wiki document for more information. >>> >>> Tested with mach5 tiers 1-4. >>> >>> Thanks >>> - Ioi >> > From ioi.lam at oracle.com Thu Aug 13 20:49:54 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 13 Aug 2020 13:49:54 -0700 Subject: RFR(M) 8250990 Consolidate object copying code for CDS static/dynamic archive dumping In-Reply-To: <89530161-c6c2-0e53-d2e7-38b046290621@oracle.com> References: <07b2ee18-0a5f-0839-0a6c-b54d788e4cd2@oracle.com> <39423dce-d769-bb92-4d22-9dc848d1086e@oracle.com> <89530161-c6c2-0e53-d2e7-38b046290621@oracle.com> Message-ID: <18c3cbbe-e040-068b-077f-cecddb683a6d@oracle.com> Hi Calvin, Thanks for the review. It's not possible for a[0] == b[0] because the array is built using UniqueMetaspaceClosure, which ensures that we will not visit see two Symbols with the same address. This condition is caught by the assert(a[0] > b[0]). When this function is called, we are already inside a ResourceMark, so (*a)->as_C_string() will be able to allocate from the resource area. Such an allocation will happen only if a[0] == b[0], and afterwards the VM will abort, so there's no need (and it's cleaner) to not use a ResourceMark here. Thanks - Ioi On 8/13/20 12:12 PM, calvin.cheung at oracle.com wrote: > Hi Ioi, > > This looks pretty good. > > One question on archiveBuilder.cpp: > > ?246 int ArchiveBuilder::compare_symbols_by_address(Symbol** a, > Symbol** b) { > ?247?? if (a[0] < b[0]) { > ?248???? return -1; > ?249?? } else { > ?250???? assert(a[0] > b[0], "Duplicated symbol %s unexpected", > (*a)->as_C_string()); > ?251???? return 1; > ?252?? } > ?253 } > > It seems the (a[0] == b[0]) case and a ResourceMark are missing. The > old code in metaSpaceShared.cpp has the following: > > ?669 static int compare_symbols_by_address(Symbol** a, Symbol** b) { > ?670?? if (a[0] < b[0]) { > ?671???? return -1; > ?672?? } else if (a[0] == b[0]) { > ?673???? ResourceMark rm; > ?674???? log_warning(cds)("Duplicated symbol %s unexpected", > (*a)->as_C_string()); > ?675???? return 0; > ?676?? } else { > ?677???? return 1; > ?678?? } > ?679 } > > thanks, > > Calvin > > On 8/3/20 10:59 PM, Ioi Lam wrote: >> Here's an updated version after offline discussion with Coleen: >> >> http://cr.openjdk.java.net/~iklam/jdk16/8250990-consolidate-static-dynamic-cds-objcopy.v02/ >> >> http://cr.openjdk.java.net/~iklam/jdk16/8250990-consolidate-static-dynamic-cds-objcopy.v02-delta/ >> >> >> - Moved DumpAllocStats into its own cpp/hpp files >> - Removed the use of inline.hpp files from archiveBuilder.hpp >> - Changed comments about the old ArchiveCompactor class to >> ? the new AcrchiveBuilder class. >> - Removed CDSAddSymbol_lock which is no longer used. >> >> Thanks >> - Ioi >> >> On 8/3/20 3:11 PM, Ioi Lam wrote: >>> There turns out to be a lot of duplication in the static/dynamic >>> dumping code. To keep the changes manageable, I've reorganized the >>> REF into a few sub-tasks. This particular review is for: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8250990 >>> http://cr.openjdk.java.net/~iklam/jdk16/8250990-consolidate-static-dynamic-cds-objcopy.v01/ >>> >>> https://wiki.openjdk.java.net/display/HotSpot/How+CDS+Copies+Class+Metadata+into+the+Archive >>> >>> >>> I've also updated the e-mail subject to use the new JBS ID >>> (JDK-8250990). >>> >>> This patch is an intermediate step. Some code might look like they >>> are in the wrong place -- for example: _estimated_metsapceobj_bytes >>> has been moved to archiveBuilder.hpp, but _estimated_hashtable_bytes >>> is still in dynamicArchive.cpp. The latter will eventually be moved >>> as part of the second step (JDK-8250989 >>> Consolidate buffer allocation code for CDS static/dynamic archive >>> dumping). >>> >>> (Coleen has given me feedback off-line. I'll post a new version soon >>> to limit the API in archiveBuilder.hpp) >>> >>> Thanks >>> - Ioi >>> >>> On 7/31/20 11:36 PM, Ioi Lam wrote: >>>> >>>> Currently there are two very similar implementations for static CDS >>>> archive >>>> dumping (metaspaceShared.cpp) and dynamic CDS archive dumping >>>> (dynamicArchive.cpp). >>>> >>>> I've merged the two versions and moved to a new file, >>>> archiveBuilder.cpp. >>>> >>>> I also improved the copying algorithm to speed up dumping. Times >>>> for dumping >>>> 20000+ classes: >>>> >>>> ? ?? ????? static??????? dynamic >>>> ??? Old??? 42.655 sec??? 67.014 sec >>>> ??? New??? 37.027 sec??? 34.974 sec >>>> >>>> Normally I would not try to optimize when refactoring code. I made an >>>> earlier, naive attempt to use the old dynamic dumper for static >>>> dumping >>>> (because dynamic dumping is essentially a superset of static dumping). >>>> However, that caused significant regression in static dumping speed. >>>> >>>> Hopefully the new algorithm is both faster and easier to understand. >>>> Please see the wiki document for more information. >>>> >>>> Tested with mach5 tiers 1-4. >>>> >>>> Thanks >>>> - Ioi >>> >> From daniel.daugherty at oracle.com Thu Aug 13 21:16:45 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 13 Aug 2020 17:16:45 -0400 Subject: RFR(T): 8251543: add mention of INFLATING() to share/oops/markWord.hpp header comment In-Reply-To: <1a61123a-a289-c88a-de31-de0f8d0d9eda@oracle.com> References: <1a61123a-a289-c88a-de31-de0f8d0d9eda@oracle.com> Message-ID: In a previous cleanup bug, we normalized src/share/runtime/synchronizer.cpp to use "stack-lock" instead of "stack lock" and other variants. I'm going to change the two "stack lock" uses to "stack-lock" below. Dan On 8/13/20 10:58 AM, Daniel D. Daugherty wrote: > Greetings, > > I have a trivial comment addition to src/hotspot/share/oops/markWord.hpp. > > Here's the bug: > > ??? JDK-8251543 add mention of INFLATING() to share/oops/markWord.hpp > header comment > ??? https://bugs.openjdk.java.net/browse/JDK-8251543 > > And here's the proposed fix: > > $ hg diff > diff -r 73778bfbffe3 src/hotspot/share/oops/markWord.hpp > --- a/src/hotspot/share/oops/markWord.hpp??? Thu Aug 13 10:02:35 2020 > -0400 > +++ b/src/hotspot/share/oops/markWord.hpp??? Thu Aug 13 10:51:44 2020 > -0400 > @@ -86,6 +86,10 @@ > ?//??? [ptr???????????? | 11]? marked???????????? used to mark an object > ?// > ?//??? We assume that stack/thread pointers have the lowest two bits > cleared. > +// > +//? - INFLATING() is a distinguished markword value that is used when > +//??? inflating an existing stack lock into an ObjectMonitor. See below > +//??? for is_being_inflated() and INFLATING(). > > ?class BasicLock; > ?class ObjectMonitor; > @@ -226,7 +230,7 @@ > ?? bool is_being_inflated() const { return (value() == 0); } > > ?? // Distinguished markword value - used when inflating over > -? // an existing stacklock.? 0 indicates the markword is "BUSY". > +? // an existing stack lock.? 0 indicates the markword is "BUSY". > ?? // Lockword mutators that use a LD...CAS idiom should always > ?? // check for and avoid overwriting a 0 value installed by some > ?? // other thread.? (They should spin or block instead.? The 0 value > > > Thanks, in advance, for any comments, questions or suggestions. > > Dan From ioi.lam at oracle.com Fri Aug 14 01:55:31 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 13 Aug 2020 18:55:31 -0700 Subject: RFR(T) 8251559 Remove empty file utilities/sizes.cpp Message-ID: <4482d141-de71-c2c9-359d-307c61433c39@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8251559 Please review this trivial fix -- This file has no contents other than #includes, so it should be removed. (no webrev since it's just simple removal) Thanks - Ioi From suenaga at oss.nttdata.com Fri Aug 14 02:36:26 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Fri, 14 Aug 2020 11:36:26 +0900 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: <08584797-0841-945e-ec6d-7050a12326b4@oracle.com> References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> <08584797-0841-945e-ec6d-7050a12326b4@oracle.com> Message-ID: <0facc9ee-eb47-a40c-4064-75d648b48eab@oss.nttdata.com> Hi David, Matthias, Martin, I uploaded new webrev. Could you review again? http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.02/ It generates runtime stub for hypervisor detection, and it can distinguish between Hyper-V host (root partition) and guest. And also it works on 32bit platforms. I measured startup performance (java --version) with Measure-Command on PowerShell, this change is faster than current implementation. TotalMilliseconds : 57.8196 TotalMilliseconds : 66.8379 TotalMilliseconds : 64.7693 TotalMilliseconds : 55.6546 TotalMilliseconds : 63.848 average : 61.78588 This change has passed tests on submit repo (mach5-one-ysuenaga-JDK-8250598-20200814-0119-13424118), and also it works fine on following environments: - Hyper-V host (Windows 10 x64) - Hyper-V guest (Windows 10 x64) - Hyper-V guest (Linux x64) - Hyper-V guest (32bit JDK on Linux x64) Thanks, Yasumasa On 2020/08/13 20:39, David Holmes wrote: > On 13/08/2020 5:39 pm, Yasumasa Suenaga wrote: >> Hi Matthias, David, >> >> On 2020/08/13 15:52, Baesken, Matthias wrote: >>> >>>> >>>> Should we make the change to determine just before it is needed (e.g. VM.info or hs_err log) at first? >>>> It is out of scope of this change, so I want to work as another issue if it is needed. >>>> >>> >>> Hi , >>> I think we better do not call into WMI , when? writing an hs_err file?? . >> >> I found out from Hypervisor Top Level Functional Specification [1] section 2.4.8. >> That says CPUID leaf 0x40000007 is available to the root partition only. >> >> I changed VM_Version::is_in_VM() as below, and it works fine. >> (I use __cpuid intrinsic in here because this code would be compiled by cl.exe only.) >> >> ``` >> #include >> >> #define EAX 0 >> #define EBX 1 >> #define ECX 2 >> #define EDX 3 >> >> bool VM_Version::is_in_VM() { >> ?? int regs[4]; >> ?? __cpuid(regs, 0x40000007); >> >> ?? // CPUID leaf 0x40000007 is available to the root partition only. >> ?? return (regs[EAX] == 0x0) && (regs[EBX] == 0x0) && (regs[ECX] == 0x0) && (regs[EDX] == 0x0); >> } >> ``` >> >> Also startup performance is better than WMI call. >> >> TotalMilliseconds : 66.7863 >> TotalMilliseconds : 56.8123 >> TotalMilliseconds : 57.0015 >> TotalMilliseconds : 62.4755 >> TotalMilliseconds : 67.7632 >> average : 62.16776 >> >> If you are ok this change, I will update webrev, and will send review request. >> (new webrev will include both loop for CPUID leaf and renaming is_in_VM) > > If it works and addresses the startup hit then I'm fine with it - but still can't actually review the code. > > Thanks, > David > >> >> Thanks, >> >> Yasumasa >> >> >> [1] https://github.com/MicrosoftDocs/Virtualization-Documentation/raw/master/tlfs/Hypervisor%20Top%20Level%20Functional%20Specification%20v6.0b.pdf >> >> >>> Best regards, Matthias >>> >>> >>> >>> -----Original Message----- >>> From: Yasumasa Suenaga >>> Sent: Donnerstag, 13. August 2020 06:15 >>> To: David Holmes ; Baesken, Matthias ; hotspot-runtime-dev at openjdk.java.net; build-dev at openjdk.java.net >>> Cc: Doerr, Martin >>> Subject: Re: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS >>> >>> On 2020/08/13 11:54, David Holmes wrote: >>>> On 13/08/2020 11:12 am, Yasumasa Suenaga wrote: >>>>> Hi Matthias, David, >>>>> >>>>> I measured startup benchmarks with `Measure-Command {.\jdk\build\windows-x86_64-server-release\images\jdk\bin\java.exe --version}` on PowerShell. >>>>> >>>>> * PC: Ryzen 3 3300X, 16GB memory >>>>> * OS: Windows 10 x64 (May 2020 Update) >>>>> * Java: jdk/jdk revision 60537 >>>>> ????? (Compiled by VS 2019 (16.7.0)) >>>>> >>>>> * without patch >>>>> TotalMilliseconds : 70.2124 >>>>> TotalMilliseconds : 64.4465 >>>>> TotalMilliseconds : 59.0854 >>>>> TotalMilliseconds : 68.0255 >>>>> TotalMilliseconds : 72.6467 >>>>> average : 66.8833 >>>>> >>>>> * with webrev.01 >>>>> TotalMilliseconds : 81.7185 >>>>> TotalMilliseconds : 68.539 >>>>> TotalMilliseconds : 85.7226 >>>>> TotalMilliseconds : 72.6584 >>>>> TotalMilliseconds : 75.6091 >>>>> average : 76.84952 >>>>> >>>>> Overhead of WMI seems to be +10ms in this case. >>>> >>>> Which is nearly 15%! Sorry but I just know Claes will be very unhappy if this were to go in as-is. >>> >>> Should we make the change to determine just before it is needed (e.g. VM.info or hs_err log) at first? >>> It is out of scope of this change, so I want to work as another issue if it is needed. >>> >>> >>> Yasumasa >>> >>> >>>> David >>>> ----- >>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2020/08/13 0:05, Baesken, Matthias wrote: >>>>>>> I understand that if the process runs on Xen on other hypervisor (e.g. KVM), information for Xen would be set between 0x40000100 and 0x40010000. >>>>>>> Ok, I will not remove the loop in new webrev, and will add comment about it. >>>>>> >>>>>> Hi Yasumasa? , thanks ! >>>>>> >>>>>> Regarding the WMI overhead , if you could? get some more info on this it would be better to judge if it is perfectly okay or concerning . >>>>>> >>>>>> (I remember that I looked into it a couple of years ago , but decided not to use it in early VM startup ;? but have to confess this was just based on a "gut feeling", >>>>>> ?? No micro benchmarks? ) >>>>>> >>>>>> Best regards, Matthias >>>>>> >>>>>> From calvin.cheung at oracle.com Fri Aug 14 05:07:15 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Thu, 13 Aug 2020 22:07:15 -0700 Subject: RFR(T) 8251559 Remove empty file utilities/sizes.cpp In-Reply-To: <4482d141-de71-c2c9-359d-307c61433c39@oracle.com> References: <4482d141-de71-c2c9-359d-307c61433c39@oracle.com> Message-ID: Hi Ioi, The removal of the file seems fine. thanks, Calvin On 8/13/20 6:55 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8251559 > > Please review this trivial fix -- This file has no contents other than > #includes, so it should be removed. > > (no webrev since it's just simple removal) > > Thanks > - Ioi From lois.foltan at oracle.com Fri Aug 14 15:09:50 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Fri, 14 Aug 2020 11:09:50 -0400 Subject: RFR 8251414: Add test that invokeinterface of a protected method in java.lang.Object throws NoSuchMethodError In-Reply-To: References: Message-ID: On 8/12/2020 2:14 PM, Harold Seigel wrote: > Hi, > > Please review this change to add test cases to test > hotspot/jtreg/runtime/8026394/InterfaceObjectTest.java to check for > NoSuchMethodError exceptions when calling Object protected methods > clone() and finalize() using invokeinterface. > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8251414/webrev/index.html > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251414 > > The fix was tested by running the modified test on multiple platforms, > including Mac OS, Windows, and Linux. > > Thanks, Harold Looks good Harold.? Don't forget to update the copyright on file InterfaceObjectTest.java. Thanks, Lois From harold.seigel at oracle.com Fri Aug 14 15:16:09 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Fri, 14 Aug 2020 11:16:09 -0400 Subject: RFR 8251414: Add test that invokeinterface of a protected method in java.lang.Object throws NoSuchMethodError In-Reply-To: References: Message-ID: <9021047d-c134-6496-6beb-170d26565d1b@oracle.com> Thanks Lois! Harold On 8/14/2020 11:09 AM, Lois Foltan wrote: > On 8/12/2020 2:14 PM, Harold Seigel wrote: >> Hi, >> >> Please review this change to add test cases to test >> hotspot/jtreg/runtime/8026394/InterfaceObjectTest.java to check for >> NoSuchMethodError exceptions when calling Object protected methods >> clone() and finalize() using invokeinterface. >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8251414/webrev/index.html >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251414 >> >> The fix was tested by running the modified test on multiple >> platforms, including Mac OS, Windows, and Linux. >> >> Thanks, Harold > > Looks good Harold.? Don't forget to update the copyright on file > InterfaceObjectTest.java. > > Thanks, > Lois From coleen.phillimore at oracle.com Fri Aug 14 15:25:33 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 14 Aug 2020 11:25:33 -0400 Subject: RFR 8251414: Add test that invokeinterface of a protected method in java.lang.Object throws NoSuchMethodError In-Reply-To: References: Message-ID: This looks good.? Can you hg mv this test into the linkResolver directory and out of 8026394 before checking it in? Thanks, Coleen On 8/12/20 2:14 PM, Harold Seigel wrote: > Hi, > > Please review this change to add test cases to test > hotspot/jtreg/runtime/8026394/InterfaceObjectTest.java to check for > NoSuchMethodError exceptions when calling Object protected methods > clone() and finalize() using invokeinterface. > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8251414/webrev/index.html > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251414 > > The fix was tested by running the modified test on multiple platforms, > including Mac OS, Windows, and Linux. > > Thanks, Harold From harold.seigel at oracle.com Fri Aug 14 15:27:54 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Fri, 14 Aug 2020 11:27:54 -0400 Subject: RFR 8251414: Add test that invokeinterface of a protected method in java.lang.Object throws NoSuchMethodError In-Reply-To: References: Message-ID: <1f1a29ea-3c02-0ae7-07f6-6321f8ed79e8@oracle.com> Thanks Coleen! I'll move the test before pushing the change. Harold On 8/14/2020 11:25 AM, Coleen Phillimore wrote: > > This looks good.? Can you hg mv this test into the linkResolver > directory and out of 8026394 before checking it in? > Thanks, > Coleen > > On 8/12/20 2:14 PM, Harold Seigel wrote: >> Hi, >> >> Please review this change to add test cases to test >> hotspot/jtreg/runtime/8026394/InterfaceObjectTest.java to check for >> NoSuchMethodError exceptions when calling Object protected methods >> clone() and finalize() using invokeinterface. >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8251414/webrev/index.html >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251414 >> >> The fix was tested by running the modified test on multiple >> platforms, including Mac OS, Windows, and Linux. >> >> Thanks, Harold > From martin.doerr at sap.com Fri Aug 14 15:56:52 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 14 Aug 2020 15:56:52 +0000 Subject: RFR(T): 8251846: Replace __linux which is no longer defined Message-ID: Hi, we are observing lots of "error occurred during error reporting" problems on linux on PPC64 in JDK16. Reason: debug.cpp uses "#if defined(__linux)". It is no longer defined in our latest build environment. Hence, fixup code for copy of uncontext is missing. Bug: https://bugs.openjdk.java.net/browse/JDK-8251846 Please review this trivial fix: http://cr.openjdk.java.net/~mdoerr/8251846_linux_ppc_context_copy_fix/webrev.00/ Best regards, Martin From calvin.cheung at oracle.com Fri Aug 14 16:10:02 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Fri, 14 Aug 2020 09:10:02 -0700 Subject: RFR(S): 8247529: Crash in runtime/cds/appcds/dynamicArchive/LambdaProxyCallerIsHidden.java with Graal Message-ID: bug: https://bugs.openjdk.java.net/browse/JDK-8247529 webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8247529/webrev.00/ This proposed change includes: 1) Avoid inserting InstanceKlass into the DumpTimeSharedClassTable after dynamic CDS dumping has started. This is done by adding a _dump_in_progress bool which will be set at the beginning of dynamic CDS dumping. In DumpTimeSharedClassTable::find_or_allocate_info_for, if the _dump_in_progress is set, it will not create a new entry in the table. 2) Acquire the DumpTimeTable_lock before iterations on _dumptime_table to ensure memory order correctness. Testing: passed tier1 - 4 tests. thanks, Calvin From ioi.lam at oracle.com Fri Aug 14 16:54:29 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 14 Aug 2020 09:54:29 -0700 Subject: RFR(S) 8251560 Remove excessive header file inclusion from systemDictionary.hpp and others Message-ID: <8ad94328-1ccd-b7c5-001b-1f5ec4a6ba90@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8251560 http://cr.openjdk.java.net/~iklam/jdk16/8251560-remove-includes-in-system-dict-etc.v01/ Summary: I removed a few unnecessary #includes from systemDictionary.hpp, signature.hpp, altHashing.hpp and klass.inline.hpp Two rarely used classes are moved to a new header file classLoadInfo.hpp Some non-critical inline functions were moved to cpp files. Some cpp files needed to be updated (because they didn't explicitly declare dependencies on header files that were transitively included by other header files). Tested with mach5 tier1 and also local Linux builds (no-pch, minimal VM, etc). Thanks - Ioi From coleen.phillimore at oracle.com Fri Aug 14 18:35:12 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 14 Aug 2020 14:35:12 -0400 Subject: RFR(S) 8251560 Remove excessive header file inclusion from systemDictionary.hpp and others In-Reply-To: <8ad94328-1ccd-b7c5-001b-1f5ec4a6ba90@oracle.com> References: <8ad94328-1ccd-b7c5-001b-1f5ec4a6ba90@oracle.com> Message-ID: <1055715e-92f0-2a02-55f7-b01ecb52d0e3@oracle.com> I really like this change.? It looks like you need a classLoadInfo.hpp file with the constructors. Thanks, Coleen On 8/14/20 12:54 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8251560 > http://cr.openjdk.java.net/~iklam/jdk16/8251560-remove-includes-in-system-dict-etc.v01/ > > > Summary: > > I removed a few unnecessary #includes from systemDictionary.hpp, > signature.hpp, altHashing.hpp and klass.inline.hpp > > Two rarely used classes are moved to a new header file classLoadInfo.hpp > > Some non-critical inline functions were moved to cpp files. > > Some cpp files needed to be updated (because they didn't explicitly > declare dependencies on header files that were transitively included > by other header files). > > Tested with mach5 tier1 and also local Linux builds (no-pch, minimal > VM, etc). > Thanks > - Ioi From coleen.phillimore at oracle.com Fri Aug 14 18:38:43 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 14 Aug 2020 14:38:43 -0400 Subject: RFR(S) 8251560 Remove excessive header file inclusion from systemDictionary.hpp and others In-Reply-To: <1055715e-92f0-2a02-55f7-b01ecb52d0e3@oracle.com> References: <8ad94328-1ccd-b7c5-001b-1f5ec4a6ba90@oracle.com> <1055715e-92f0-2a02-55f7-b01ecb52d0e3@oracle.com> Message-ID: On 8/14/20 2:35 PM, Coleen Phillimore wrote: > > I really like this change.? It looks like you need a classLoadInfo.hpp > file with the constructors. Oops, I mean classLoadInfo.cpp file. > > Thanks, > Coleen > > On 8/14/20 12:54 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8251560 >> http://cr.openjdk.java.net/~iklam/jdk16/8251560-remove-includes-in-system-dict-etc.v01/ >> >> >> Summary: >> >> I removed a few unnecessary #includes from systemDictionary.hpp, >> signature.hpp, altHashing.hpp and klass.inline.hpp >> >> Two rarely used classes are moved to a new header file classLoadInfo.hpp >> >> Some non-critical inline functions were moved to cpp files. >> >> Some cpp files needed to be updated (because they didn't explicitly >> declare dependencies on header files that were transitively included >> by other header files). >> >> Tested with mach5 tier1 and also local Linux builds (no-pch, minimal >> VM, etc). >> Thanks >> - Ioi > From t-maana at microsoft.com Fri Aug 14 22:11:30 2020 From: t-maana at microsoft.com (Ana Marsh) Date: Fri, 14 Aug 2020 22:11:30 +0000 Subject: Timing Differences Between JFR and GC Logs Message-ID: Hi everyone, My name is Ana Marsh and I am interning at Microsoft on the Java Engineering Team (under Martijn Verburg) and this summer I have been diving into JFR in the JVM and analyzing how it gathers information on garbage collection events. I have noticed over the past two months of digging that the start time of a garbage collection event returned by JFR varies widely from the start time of the same event given with Unified Logging. I decided to explore this further by looking at the specifics of OpenJDK code and found some interesting points to highlight. 1. Logging decides to log the time of events as it prints and not necessarily as the events happen. That being said, it appears that JFR collects the time directly at the beginning of an event making it appear as a better marker of the start time of an event. 1. Summary: Garbage collection timing is measured in Ticks which holds values from two different clocks when using an x86 machine. These clocks are called "os" (ElapsedCounterSource) and "rdtsc" (FastUnorderedElapsedCounterSource) and one value of Ticks will hold a relevant time for each clock. When a GC event begins (specifically Parallel GC because that is what I have been looking at) it registers the start in Ticks by calling now(). This is what now() looks like: jdk/src/hotspot/share/utilities/ticks.cpp CompositeElapsedCounterSource::Type CompositeElapsedCounterSource::now() { CompositeTime ct; ct.val1 = ElapsedCounterSource::now(); #if defined(X86) && !defined(ZERO) static bool initialized = false; static bool valid_rdtsc = false; if (!initialized) { valid_rdtsc = Rdtsc::initialize(); initialized = true; } if (valid_rdtsc) { ct.val2 = Rdtsc::elapsed_counter(); } #endif return ct; } ElapsedCounterSource::Type ElapsedCounterSource::now() { return os::elapsed_counter(); } For JFR files, the start time is set to the RDTSC value while in GC logs it uses the OS clock as it prints out the log. This is obviously a further point of concern for timing differences, so I wanted to see how much of a contrast there was in the times these clocks reported. Experimentation with Results: To find the differences between the two I added this to the code: jdk/src/hotspot/share/utilities/ticks.cpp CompositeElapsedCounterSource::Type CompositeElapsedCounterSource::now() { CompositeTime ct; ct.val1 = ElapsedCounterSource::now(); #if defined(X86) && !defined(ZERO) static bool initialized = false; static bool valid_rdtsc = false; if (!initialized) { valid_rdtsc = Rdtsc::initialize(); initialized = true; } if (valid_rdtsc) { ct.val2 = Rdtsc::elapsed_counter(); } #endif std::cout << "I got ticks!!\n"; std::cout << "OS time: "; std::cout << ElapsedCounterSource::seconds(ct.val1) << " seconds\n"; std::cout<< "RDTSC time: "; std::cout << FastUnorderedElapsedCounterSource::seconds(ct.val2) << " seconds\n\n"; return ct; } I ran this with a test application that forces garbage collection events and I also had a JFR file and GC log being recorded during the run as well. For reference I am running on a Windows 10 machine but using WSL with Ubuntu. Here are a few outputs from the beginning of the application: I got ticks!! OS time: 2.0739 seconds RDTSC time: 1.16455 seconds I got ticks!! OS time: 2.07512 seconds RDTSC time: 1.16591 seconds I got ticks!! OS time: 2.07549 seconds RDTSC time: 1.16631 seconds Here are a few outputs from ~40 seconds into the application: I got ticks!! OS time: 43.0537 seconds RDTSC time: 46.717 seconds I got ticks!! OS time: 43.0562 seconds RDTSC time: 46.7198 seconds I got ticks!! OS time: 43.0562 seconds RDTSC time: 46.7198 seconds Here are a few outputs from ~200 seconds into the application: I got ticks!! OS time: 191.654 seconds RDTSC time: 211.898 seconds I got ticks!! OS time: 191.654 seconds RDTSC time: 211.898 seconds I got ticks!! OS time: 191.654 seconds RDTSC time: 211.898 seconds Observations on Results: Both ElapsedCounterSource (OS) and FastUnorderedElapsedCounterSource (RDTSC) have their own way of determining seconds so there could be some issues in there as well. Additionally in this example the clocks are not being set instantaneously causing some numbers to be off, however at the beginning of my test the OS clock ends up reporting a later time even though it is called first. Over time though RDTSC seems to begin reporting later times. Overall these clocks get far very off, in degrees of seconds which seems big in the clock world. Any insight or thoughts on this matter would be helpful. Thank you! Best, Ana Marsh From ioi.lam at oracle.com Sat Aug 15 01:21:29 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 14 Aug 2020 18:21:29 -0700 Subject: RFR(S) 8251560 Remove excessive header file inclusion from systemDictionary.hpp and others In-Reply-To: References: <8ad94328-1ccd-b7c5-001b-1f5ec4a6ba90@oracle.com> <1055715e-92f0-2a02-55f7-b01ecb52d0e3@oracle.com> Message-ID: <35e11229-6397-6539-3acf-330160b30303@oracle.com> Hi Coleen, Thanks for the review. Since the constructors are pretty simple, I moved them from systemDictionary.cpp into classLoadInfo.hpp. I also removed the default constructor which doesn't seem to be used. http://cr.openjdk.java.net/~iklam/jdk16/8251560-remove-includes-in-system-dict-etc.v02-delta/ Thanks - Ioi On 8/14/20 11:38 AM, Coleen Phillimore wrote: > > > On 8/14/20 2:35 PM, Coleen Phillimore wrote: >> >> I really like this change.? It looks like you need a >> classLoadInfo.hpp file with the constructors. > Oops, I mean classLoadInfo.cpp file. >> >> Thanks, >> Coleen >> >> On 8/14/20 12:54 PM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8251560 >>> http://cr.openjdk.java.net/~iklam/jdk16/8251560-remove-includes-in-system-dict-etc.v01/ >>> >>> >>> Summary: >>> >>> I removed a few unnecessary #includes from systemDictionary.hpp, >>> signature.hpp, altHashing.hpp and klass.inline.hpp >>> >>> Two rarely used classes are moved to a new header file >>> classLoadInfo.hpp >>> >>> Some non-critical inline functions were moved to cpp files. >>> >>> Some cpp files needed to be updated (because they didn't explicitly >>> declare dependencies on header files that were transitively included >>> by other header files). >>> >>> Tested with mach5 tier1 and also local Linux builds (no-pch, minimal >>> VM, etc). >>> Thanks >>> - Ioi >> > From kim.barrett at oracle.com Sat Aug 15 02:29:11 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 14 Aug 2020 22:29:11 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation Message-ID: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> Please review this simplification of the constructors for ResourceMark, accomplished using delegating constructors. Also made the constructors "explicit" and the class non-copyable. CR: https://bugs.openjdk.java.net/browse/JDK-8251850 Webrev: https://cr.openjdk.java.net/~kbarrett/8251850/open.00/ Testing: mach5 tier1 From david.holmes at oracle.com Sat Aug 15 02:56:58 2020 From: david.holmes at oracle.com (David Holmes) Date: Sat, 15 Aug 2020 12:56:58 +1000 Subject: Timing Differences Between JFR and GC Logs In-Reply-To: References: Message-ID: Hi Ana, Simple answer is that JFR insists on trying to use rdtsc if it thinks the machine has a safe/stable tsc value to read. There is some risk in doing this as some machines lie about their TSC capabilities. That is one reason why the FastUnorderedElapsedCounterSource flag exists. Trying to correlate/synchronise a raw clock using rdtsc and the OS reported time is a very difficult problem and not one we are trying to solve. The JFR folk (cc'd) are the best ones to discuss the details of JFR timestamps, but I don't think anyone should be trying to correlate the JFR event information with the UL event information. That is probably not a very satisfactory answer :) All the best for your internship! Cheers, David Holmes On 15/08/2020 8:11 am, Ana Marsh wrote: > Hi everyone, > > My name is Ana Marsh and I am interning at Microsoft on the Java Engineering Team (under Martijn Verburg) and this summer I have been diving into JFR in the JVM and analyzing how it gathers information on garbage collection events. I have noticed over the past two months of digging that the start time of a garbage collection event returned by JFR varies widely from the start time of the same event given with Unified Logging. I decided to explore this further by looking at the specifics of OpenJDK code and found some interesting points to highlight. > > > 1. Logging decides to log the time of events as it prints and not necessarily as the events happen. That being said, it appears that JFR collects the time directly at the beginning of an event making it appear as a better marker of the start time of an event. > > > 1. Summary: > Garbage collection timing is measured in Ticks which holds values from two different clocks when using an x86 machine. These clocks are called "os" (ElapsedCounterSource) and "rdtsc" (FastUnorderedElapsedCounterSource) and one value of Ticks will hold a relevant time for each clock. When a GC event begins (specifically Parallel GC because that is what I have been looking at) it registers the start in Ticks by calling now(). This is what now() looks like: > jdk/src/hotspot/share/utilities/ticks.cpp > CompositeElapsedCounterSource::Type CompositeElapsedCounterSource::now() { > CompositeTime ct; > ct.val1 = ElapsedCounterSource::now(); > #if defined(X86) && !defined(ZERO) > static bool initialized = false; > static bool valid_rdtsc = false; > if (!initialized) { > valid_rdtsc = Rdtsc::initialize(); > initialized = true; > } > if (valid_rdtsc) { > ct.val2 = Rdtsc::elapsed_counter(); > } > #endif > return ct; > } > > ElapsedCounterSource::Type ElapsedCounterSource::now() { > return os::elapsed_counter(); > } > > For JFR files, the start time is set to the RDTSC value while in GC logs it uses the OS clock as it prints out the log. This is obviously a further point of concern for timing differences, so I wanted to see how much of a contrast there was in the times these clocks reported. > > Experimentation with Results: > To find the differences between the two I added this to the code: > jdk/src/hotspot/share/utilities/ticks.cpp > CompositeElapsedCounterSource::Type CompositeElapsedCounterSource::now() { > CompositeTime ct; > ct.val1 = ElapsedCounterSource::now(); > #if defined(X86) && !defined(ZERO) > static bool initialized = false; > static bool valid_rdtsc = false; > if (!initialized) { > valid_rdtsc = Rdtsc::initialize(); > initialized = true; > } > if (valid_rdtsc) { > ct.val2 = Rdtsc::elapsed_counter(); > } > #endif > std::cout << "I got ticks!!\n"; > std::cout << "OS time: "; > std::cout << ElapsedCounterSource::seconds(ct.val1) << " seconds\n"; > std::cout<< "RDTSC time: "; > std::cout << FastUnorderedElapsedCounterSource::seconds(ct.val2) << " seconds\n\n"; > return ct; > } > > I ran this with a test application that forces garbage collection events and I also had a JFR file and GC log being recorded during the run as well. For reference I am running on a Windows 10 machine but using WSL with Ubuntu. > > Here are a few outputs from the beginning of the application: > I got ticks!! > OS time: 2.0739 seconds > RDTSC time: 1.16455 seconds > > I got ticks!! > OS time: 2.07512 seconds > RDTSC time: 1.16591 seconds > > I got ticks!! > OS time: 2.07549 seconds > RDTSC time: 1.16631 seconds > > Here are a few outputs from ~40 seconds into the application: > I got ticks!! > OS time: 43.0537 seconds > RDTSC time: 46.717 seconds > > I got ticks!! > OS time: 43.0562 seconds > RDTSC time: 46.7198 seconds > > I got ticks!! > OS time: 43.0562 seconds > RDTSC time: 46.7198 seconds > > Here are a few outputs from ~200 seconds into the application: > I got ticks!! > OS time: 191.654 seconds > RDTSC time: 211.898 seconds > > I got ticks!! > OS time: 191.654 seconds > RDTSC time: 211.898 seconds > > I got ticks!! > OS time: 191.654 seconds > RDTSC time: 211.898 seconds > > > Observations on Results: > Both ElapsedCounterSource (OS) and FastUnorderedElapsedCounterSource (RDTSC) have their own way of determining seconds so there could be some issues in there as well. Additionally in this example the clocks are not being set instantaneously causing some numbers to be off, however at the beginning of my test the OS clock ends up reporting a later time even though it is called first. Over time though RDTSC seems to begin reporting later times. Overall these clocks get far very off, in degrees of seconds which seems big in the clock world. > > Any insight or thoughts on this matter would be helpful. Thank you! > > Best, > Ana Marsh > From ioi.lam at oracle.com Sat Aug 15 03:27:02 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 14 Aug 2020 20:27:02 -0700 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> Message-ID: Hi Kim, This looks good overall. I found the following function a little confusing: ?108?? static Thread* current_thread_or_null() { ?109???? return DEBUG_ONLY(Thread::current_or_null()) NOT_DEBUG(nullptr); In usual HotSpot code, xxx_or_null() means "if there's no xxx, use". However, in this case, we do have a current thread, but we are just not using it in product build. I think it's clearer to changed the constructor ? 88?? ResourceMark(ResourceArea* area, Thread* thread) : to ? 88?? ResourceMark(NOT_PRODUCT_ARG(Thread* thread) ResourceArea* area) : and then use ?119?? explicit ResourceMark(ResourceArea* area) ?120???? : ResourceMark(NOT_PRODUCT_ARG(Thread::current()) area) {} Also, these can be changed to NOT_PRODUCT_ARG -- but you need to move the corresponding field declarations up a few lines ? 94???? DEBUG_ONLY(COMMA _thread(thread)) ? 95???? DEBUG_ONLY(COMMA _previous_resource_mark(nullptr)) Thanks - Ioi On 8/14/20 7:29 PM, Kim Barrett wrote: > Please review this simplification of the constructors for ResourceMark, > accomplished using delegating constructors. > > Also made the constructors "explicit" and the class non-copyable. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8251850 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8251850/open.00/ > > Testing: > mach5 tier1 > From david.holmes at oracle.com Sat Aug 15 04:56:14 2020 From: david.holmes at oracle.com (David Holmes) Date: Sat, 15 Aug 2020 14:56:14 +1000 Subject: RFR(T): 8251846: Replace __linux which is no longer defined In-Reply-To: References: Message-ID: <49746dc4-b8ae-2498-224d-30637a1c3a2e@oracle.com> Looks fine and trivial. Thanks, David On 15/08/2020 1:56 am, Doerr, Martin wrote: > Hi, > > we are observing lots of "error occurred during error reporting" problems on linux on PPC64 in JDK16. > > Reason: > debug.cpp uses "#if defined(__linux)". It is no longer defined in our latest build environment. > Hence, fixup code for copy of uncontext is missing. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8251846 > > Please review this trivial fix: > http://cr.openjdk.java.net/~mdoerr/8251846_linux_ppc_context_copy_fix/webrev.00/ > > Best regards, > Martin > From david.holmes at oracle.com Sat Aug 15 05:11:56 2020 From: david.holmes at oracle.com (David Holmes) Date: Sat, 15 Aug 2020 15:11:56 +1000 Subject: RFR(T) 8251559 Remove empty file utilities/sizes.cpp In-Reply-To: <4482d141-de71-c2c9-359d-307c61433c39@oracle.com> References: <4482d141-de71-c2c9-359d-307c61433c39@oracle.com> Message-ID: On 14/08/2020 11:55 am, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8251559 > > Please review this trivial fix -- This file has no contents other than > #includes, so it should be removed. Bizarre! It was created that way in June 2000 and has remained unnoticed for 20 years :) Removal is fine. Cheers, David > (no webrev since it's just simple removal) > > Thanks > - Ioi From thomas.stuefe at gmail.com Sat Aug 15 07:01:05 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 15 Aug 2020 09:01:05 +0200 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> Message-ID: Hi Kim, This is a good cleanup. I looked over the old version and there are a number of inconsistencies, most of which go away with this patch. Remarks: -- RM can be made immutable now. All members can be made const (also _thread and _previous_resource_mark, if you reformulate the initializer a bit with a '?' expression). As well as all member functions. AFAICS none of the members get modified after construction, which makes sense considering what RM is. Some of the pointer members could be made destination-const too, at least _previous_resource_mark. With RM being immutable there would be no reason ever to have a non-const RM*. I know constness can snowball a bit, but I think it's worth the work since it makes the code more clearer and prevents accidents. -- In constructor: + _area->_nesting++; + assert(_area->_nesting > 0, "must stack allocate RMs" ); Not your patch,but I do not understand this assert. Is it - as the comment says - to make sure RMs are allocated on the stack, as in StackObj? Would deriving from StackObj not be enough? How does this assert help then? Or is it to ensure that all allocations happen under a RM? That should be checked in Arena::allocate(), and it is. Checking here is pointless since we just established an RM, so there is nothing to check. --- small nit: I like that you made protected private, now you could just omit the qualifier altogether. --- Since you are here ... I'd prefer it if free_malloced_objects() would be DEBUG_ONLY() (so, not an empty prototype but completely removed) which would be consistent with other debug only code in this class. Side question, what is the difference between ASSERT and !PRODUCT? Thank you, Thomas On Sat, Aug 15, 2020 at 4:30 AM Kim Barrett wrote: > Please review this simplification of the constructors for ResourceMark, > accomplished using delegating constructors. > > Also made the constructors "explicit" and the class non-copyable. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8251850 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8251850/open.00/ > > Testing: > mach5 tier1 > > From kim.barrett at oracle.com Sat Aug 15 11:42:30 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 15 Aug 2020 07:42:30 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> Message-ID: <6DF4EB82-3B66-4896-9F6B-2E3C7F13AC05@oracle.com> > On Aug 14, 2020, at 11:27 PM, Ioi Lam wrote: > > Hi Kim, > > This looks good overall. Thanks. > I found the following function a little confusing: > > 108 static Thread* current_thread_or_null() { > 109 return DEBUG_ONLY(Thread::current_or_null()) NOT_DEBUG(nullptr); > > In usual HotSpot code, xxx_or_null() means "if there's no xxx, use". However, in this case, we do have a current thread, but we are just not using it in product build. Maybe a better name would be something like "current_thread_for_debug"? The point is we don't need the actual current thread at all in a non-debug (not non-product, which also includes optimize) build, a null pointer of the appropriate type will do. And even in a debug build the call might occur early, before threading has been set up, so we might not have a current thread. > I think it's clearer to changed the constructor > > 88 ResourceMark(ResourceArea* area, Thread* thread) : > > to > > 88 ResourceMark(NOT_PRODUCT_ARG(Thread* thread) ResourceArea* area) : That doesn't work. (Besides missing a COMMA.) That private constructor needs to always have the Thread* argument in order to distinguish it from the public (ResourceArea*) overload. Also, PRODUCT is the wrong flag to be testing for any of this stuff. All of the conditionalization here was, and should still be, based on ASSERT, not on PRODUCT. > and then use > > 119 explicit ResourceMark(ResourceArea* area) > 120 : ResourceMark(NOT_PRODUCT_ARG(Thread::current()) area) {} That would be defining a constructor that delegates to itself in a PRODUCT build. See above. (Besides also missing a COMMA here.) > Also, these can be changed to NOT_PRODUCT_ARG -- but you need to move the corresponding field declarations up a few lines > > 94 DEBUG_ONLY(COMMA _thread(thread)) > 95 DEBUG_ONLY(COMMA _previous_resource_mark(nullptr)) I don't think I understand what you were getting at here with the "move the corresponding field declarations", but PRODUCT is the wrong flag to be testing. From kim.barrett at oracle.com Sat Aug 15 12:31:06 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 15 Aug 2020 08:31:06 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> Message-ID: > On Aug 14, 2020, at 10:29 PM, Kim Barrett wrote: > > Please review this simplification of the constructors for ResourceMark, > accomplished using delegating constructors. > > Also made the constructors "explicit" and the class non-copyable. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8251850 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8251850/open.00/ > > Testing: > mach5 tier1 I should have looked further along in resourceArea.hpp. All the changes I made to ResourceMark? DeoptResourceMark should have similar changes made to it. Well, I have a first round of comments from the ResourceMark changes. I can take those into account for DeoptResourceMark. From kim.barrett at oracle.com Sat Aug 15 12:41:00 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 15 Aug 2020 08:41:00 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> Message-ID: > On Aug 15, 2020, at 3:01 AM, Thomas St?fe wrote: > > Hi Kim, > > This is a good cleanup. I looked over the old version and there are a number of inconsistencies, most of which go away with this patch. Thanks. > RM can be made immutable now. All members can be made const (also _thread and _previous_resource_mark, if you reformulate the initializer a bit with a '?' expression). As well as all member functions. AFAICS none of the members get modified after construction, which makes sense considering what RM is. > > Some of the pointer members could be made destination-const too, at least _previous_resource_mark. With RM being immutable there would be no reason ever to have a non-const RM*. I know constness can snowball a bit, but I think it's worth the work since it makes the code more clearer and prevents accidents. That could be done, but it's stylistically almost never done in HotSpot code, or even in the wider C++ community that I've ever seen. I think changing the pointers (other than than _area) to pointer to const isn't really an improvement, since what they are pointing at isn't actually const, just happens to not be modified through that specific copy of the pointer. > In constructor: > > + _area->_nesting++; > + assert(_area->_nesting > 0, "must stack allocate RMs" ); > > Not your patch,but I do not understand this assert. > > Is it - as the comment says - to make sure RMs are allocated on the stack, as in StackObj? Would deriving from StackObj not be enough? How does this assert help then? > > Or is it to ensure that all allocations happen under a RM? That should be checked in Arena::allocate(), and it is. Checking here is pointless since we just established an RM, so there is nothing to check. I didn't try to figure out what that assert is checking. Looking at it more carefully now, it might be doing a couple of things: (1) Verifying that _nesting started from a good point, that it had been properly initialized and there hadn't been some underflow somewhere. (2) Verifying that the increment of _nesting didn't overflow. But it doesn't actually do that, since such overflow would be UB and the compiler is free to treat it as impossible and delete the test. The error message for the assert doesn't clarify things at all. I think making any changes to that assert is a different problem than what this change is doing. I could file a new RFE to try to figure out what to do about it. (Certainly something should be done, since I agree it's quite unclear what it's trying to verify.) > small nit: I like that you made protected private, now you could just omit the qualifier altogether. HotSpot code varies on that; some folks prefer the consistency of always having it be explicit. I don't care that much either way. > Since you are here ... > > I'd prefer it if free_malloced_objects() would be DEBUG_ONLY() (so, not an empty prototype but completely removed) which would be consistent with other debug only code in this class. Side question, what is the difference between ASSERT and !PRODUCT? Changing it to a DEBUG_ONLY declaration would be incorrect. That would require making the call DEBUG_ONLY too. But the condition for calling is UseMallcOnly, which is a develop flag, which is a PRODUCT distinction, not a debug (ASSERT) distinction. The difference between ASSERT and !PRODUCT is that the case of (!PRODUCT && !ASSERT) is the so-called "optimized" build. There are periodic calls for eliminating this category of builds (JDK-8183287, for example), and it often gets broken because most people don't use or test it, but it has fans and so has never gone away. The purpose of "optimized" builds (as explained to me by one of its long-time proponents) is to have the performance characteristics of a release build (so no extra checks that affect performance or timing), but provide additional tools and data (printers, names, &etc) that we want to exclude from a release build for reasons of saving space or whatever. There's certainly lots of confusion around it though. From daniel.daugherty at oracle.com Sat Aug 15 14:04:14 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Sat, 15 Aug 2020 10:04:14 -0400 Subject: RFR(T): 8251543: add mention of INFLATING() to share/oops/markWord.hpp header comment In-Reply-To: References: <1a61123a-a289-c88a-de31-de0f8d0d9eda@oracle.com> Message-ID: <2627c417-4034-8709-3947-92921896a0d1@oracle.com> Tap, tap, tap... is this thing working? Any takers for a trivial review? Dan On 8/13/20 5:16 PM, Daniel D. Daugherty wrote: > In a previous cleanup bug, we normalized > src/share/runtime/synchronizer.cpp to > use "stack-lock" instead of "stack lock" and other variants. I'm going > to change > the two "stack lock" uses to "stack-lock" below. > > Dan > > > On 8/13/20 10:58 AM, Daniel D. Daugherty wrote: >> Greetings, >> >> I have a trivial comment addition to >> src/hotspot/share/oops/markWord.hpp. >> >> Here's the bug: >> >> ??? JDK-8251543 add mention of INFLATING() to share/oops/markWord.hpp >> header comment >> ??? https://bugs.openjdk.java.net/browse/JDK-8251543 >> >> And here's the proposed fix: >> >> $ hg diff >> diff -r 73778bfbffe3 src/hotspot/share/oops/markWord.hpp >> --- a/src/hotspot/share/oops/markWord.hpp??? Thu Aug 13 10:02:35 2020 >> -0400 >> +++ b/src/hotspot/share/oops/markWord.hpp??? Thu Aug 13 10:51:44 2020 >> -0400 >> @@ -86,6 +86,10 @@ >> ?//??? [ptr???????????? | 11]? marked???????????? used to mark an object >> ?// >> ?//??? We assume that stack/thread pointers have the lowest two bits >> cleared. >> +// >> +//? - INFLATING() is a distinguished markword value that is used when >> +//??? inflating an existing stack lock into an ObjectMonitor. See below >> +//??? for is_being_inflated() and INFLATING(). >> >> ?class BasicLock; >> ?class ObjectMonitor; >> @@ -226,7 +230,7 @@ >> ?? bool is_being_inflated() const { return (value() == 0); } >> >> ?? // Distinguished markword value - used when inflating over >> -? // an existing stacklock.? 0 indicates the markword is "BUSY". >> +? // an existing stack lock.? 0 indicates the markword is "BUSY". >> ?? // Lockword mutators that use a LD...CAS idiom should always >> ?? // check for and avoid overwriting a 0 value installed by some >> ?? // other thread.? (They should spin or block instead.? The 0 value >> >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan > From thomas.stuefe at gmail.com Sat Aug 15 17:12:30 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 15 Aug 2020 19:12:30 +0200 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> Message-ID: On Sat, Aug 15, 2020 at 2:43 PM Kim Barrett wrote: > > On Aug 15, 2020, at 3:01 AM, Thomas St?fe > wrote: > > > > Hi Kim, > > > > This is a good cleanup. I looked over the old version and there are a > number of inconsistencies, most of which go away with this patch. > > Thanks. > > > RM can be made immutable now. All members can be made const (also > _thread and _previous_resource_mark, if you reformulate the initializer a > bit with a '?' expression). As well as all member functions. AFAICS none of > the members get modified after construction, which makes sense considering > what RM is. > > > > Some of the pointer members could be made destination-const too, at > least _previous_resource_mark. With RM being immutable there would be no > reason ever to have a non-const RM*. I know constness can snowball a bit, > but I think it's worth the work since it makes the code more clearer and > prevents accidents. > > That could be done, but it's stylistically almost never done in > HotSpot code, or even in the wider C++ community that I've ever seen. > > I agree that constness is underused in hotspot code. I use it a lot, so the "almost never" part is not true, but some developers don't. I disagree about the wider community. And neither is really a good argument - if it is useful, we should use it. Constness is a very useful instrument of the language. It clearly communicates intent and prevents accidents. Why would we deny ourselves such a tool? I think changing the pointers (other than than _area) to pointer to > const isn't really an improvement, since what they are pointing at > isn't actually const, just happens to not be modified through that > specific copy of the pointer. > Maybe I miscommunicated. RM is a bookmark into an Arena; a savepoint, a frozen snapshot. Once taken it must never change because that would be wrong. This is a good example for an immutable object. So why not express that in code? As it is, one has to read the whole code to find out that none of the members are ever modified after construction. But one glance at a "const" member would make this immediately clear, and would also show that this is no accident but a deliberate design decision. So, the frozen state is expressed by (_area+_chunk+_hwm+_max+_size_in_bytes) (bit excessive, but that is another topic). None of those must be changed after initialization so all of them should be const. For the pointers, I mean the pointer value to be const of course. As for pointer destinations: without letting changes ripple out too much, at least _previous_resource_mark could be both const and destination const (should be made so in Thread as well of course) since no-one will ever change that mark if it is immutable. _area cannot be destination-const since we use it to modify the internal members of the arena when resetting (which is a bit eww but maybe it is just me). Same goes for _chunk. _hwm and _max could in theory be destination const, when done in a wider cleanup. --- But seriously, if I were to redesign Arena and ResouceMark, I would define some opaque public type in arena as state, and let ResourceMark store a const version of that state, and hand it back to the Arena for it to reset itself, like this: Arena { public: struct state_t { ... hwm, max, whatever }; state_t get_state() const; void reset_to_state(const state_t& state); } ResourceMark { Arena* const _arena; const state_t _state; public: ResourceMark(Arena* a) _state(a->get_state()) {} void reset() const { _arena->reset_to_state(_state); } } That way RM can be simple and dumb and we don't need to expose any RA innards to the outside world. (if returning state by value is not to your taste we could have a "void Arena::fill_state(state_t&) const" member, and would not be able to make the state const in RM). > > In constructor: > > > > + _area->_nesting++; > > + assert(_area->_nesting > 0, "must stack allocate RMs" ); > > > > Not your patch,but I do not understand this assert. > > > > Is it - as the comment says - to make sure RMs are allocated on the > stack, as in StackObj? Would deriving from StackObj not be enough? How does > this assert help then? > > > > Or is it to ensure that all allocations happen under a RM? That should > be checked in Arena::allocate(), and it is. Checking here is pointless > since we just established an RM, so there is nothing to check. > > I didn't try to figure out what that assert is checking. Looking at it > more carefully now, it might be doing a couple of things: > > (1) Verifying that _nesting started from a good point, that it had > been properly initialized and there hadn't been some underflow > somewhere. > > (2) Verifying that the increment of _nesting didn't overflow. But it > doesn't actually do that, since such overflow would be UB and the > compiler is free to treat it as impossible and delete the test. > > The error message for the assert doesn't clarify things at all. > > I think making any changes to that assert is a different problem than > what this change is doing. I could file a new RFE to try to figure > out what to do about it. (Certainly something should be done, since > I agree it's quite unclear what it's trying to verify.) > I remember staring at this code in frustration a long time ago - before OpenJDK, when there was no community to ask, and no way to change things upstream. I would not be surprised if no one at Oracle remembers :) > > > small nit: I like that you made protected private, now you could just > omit the qualifier altogether. > > HotSpot code varies on that; some folks prefer the consistency of > always having it be explicit. I don't care that much either way. > Okay, no strong emotions either, please do what you think makes sense. > > Since you are here ... > > > > I'd prefer it if free_malloced_objects() would be DEBUG_ONLY() (so, not > an empty prototype but completely removed) which would be consistent with > other debug only code in this class. Side question, what is the difference > between ASSERT and !PRODUCT? > > Changing it to a DEBUG_ONLY declaration would be incorrect. That > would require making the call DEBUG_ONLY too. But the condition for > calling is UseMallcOnly, which is a develop flag, which is a PRODUCT > distinction, not a debug (ASSERT) distinction. > > The difference between ASSERT and !PRODUCT is that the case of > (!PRODUCT && !ASSERT) is the so-called "optimized" build. There are > periodic calls for eliminating this category of builds (JDK-8183287, > for example), and it often gets broken because most people don't use > or test it, but it has fans and so has never gone away. > > The purpose of "optimized" builds (as explained to me by one of its > long-time proponents) is to have the performance characteristics of a > release build (so no extra checks that affect performance or timing), > but provide additional tools and data (printers, names, &etc) that we > want to exclude from a release build for reasons of saving space or > whatever. There's certainly lots of confusion around it though. > > I see. Thank you for the explanation. Now that I read this I remember I ran into this myself at least once: https://bugs.openjdk.java.net/browse/JDK-8183228 and seems I promptly forgot. This is a sign that things are too complicated :) The issue seems stalled, let's hope it moves forward. Cheers, Thomas From kim.barrett at oracle.com Sat Aug 15 19:06:08 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 15 Aug 2020 15:06:08 -0400 Subject: RFR(T): 8251543: add mention of INFLATING() to share/oops/markWord.hpp header comment In-Reply-To: References: <1a61123a-a289-c88a-de31-de0f8d0d9eda@oracle.com> Message-ID: > On Aug 13, 2020, at 5:16 PM, Daniel D. Daugherty wrote: > > In a previous cleanup bug, we normalized src/share/runtime/synchronizer.cpp to > use "stack-lock" instead of "stack lock" and other variants. I'm going to change > the two "stack lock" uses to "stack-lock" below. > > Dan Looks good, and trivial. > > > On 8/13/20 10:58 AM, Daniel D. Daugherty wrote: >> Greetings, >> >> I have a trivial comment addition to src/hotspot/share/oops/markWord.hpp. >> >> Here's the bug: >> >> JDK-8251543 add mention of INFLATING() to share/oops/markWord.hpp header comment >> https://bugs.openjdk.java.net/browse/JDK-8251543 >> >> And here's the proposed fix: >> >> $ hg diff >> diff -r 73778bfbffe3 src/hotspot/share/oops/markWord.hpp >> --- a/src/hotspot/share/oops/markWord.hpp Thu Aug 13 10:02:35 2020 -0400 >> +++ b/src/hotspot/share/oops/markWord.hpp Thu Aug 13 10:51:44 2020 -0400 >> @@ -86,6 +86,10 @@ >> // [ptr | 11] marked used to mark an object >> // >> // We assume that stack/thread pointers have the lowest two bits cleared. >> +// >> +// - INFLATING() is a distinguished markword value that is used when >> +// inflating an existing stack lock into an ObjectMonitor. See below >> +// for is_being_inflated() and INFLATING(). >> >> class BasicLock; >> class ObjectMonitor; >> @@ -226,7 +230,7 @@ >> bool is_being_inflated() const { return (value() == 0); } >> >> // Distinguished markword value - used when inflating over >> - // an existing stacklock. 0 indicates the markword is "BUSY". >> + // an existing stack lock. 0 indicates the markword is "BUSY". >> // Lockword mutators that use a LD...CAS idiom should always >> // check for and avoid overwriting a 0 value installed by some >> // other thread. (They should spin or block instead. The 0 value >> >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan From daniel.daugherty at oracle.com Sat Aug 15 19:56:33 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Sat, 15 Aug 2020 15:56:33 -0400 Subject: RFR(T): 8251543: add mention of INFLATING() to share/oops/markWord.hpp header comment In-Reply-To: References: <1a61123a-a289-c88a-de31-de0f8d0d9eda@oracle.com> Message-ID: <9d185849-56f5-538b-1d9d-aa54e77fe853@oracle.com> On 8/15/20 3:06 PM, Kim Barrett wrote: >> On Aug 13, 2020, at 5:16 PM, Daniel D. Daugherty wrote: >> >> In a previous cleanup bug, we normalized src/share/runtime/synchronizer.cpp to >> use "stack-lock" instead of "stack lock" and other variants. I'm going to change >> the two "stack lock" uses to "stack-lock" below. >> >> Dan > Looks good, and trivial. Thanks for the review Kim. I'll push shortly. Dan > >> >> On 8/13/20 10:58 AM, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> I have a trivial comment addition to src/hotspot/share/oops/markWord.hpp. >>> >>> Here's the bug: >>> >>> JDK-8251543 add mention of INFLATING() to share/oops/markWord.hpp header comment >>> https://bugs.openjdk.java.net/browse/JDK-8251543 >>> >>> And here's the proposed fix: >>> >>> $ hg diff >>> diff -r 73778bfbffe3 src/hotspot/share/oops/markWord.hpp >>> --- a/src/hotspot/share/oops/markWord.hpp Thu Aug 13 10:02:35 2020 -0400 >>> +++ b/src/hotspot/share/oops/markWord.hpp Thu Aug 13 10:51:44 2020 -0400 >>> @@ -86,6 +86,10 @@ >>> // [ptr | 11] marked used to mark an object >>> // >>> // We assume that stack/thread pointers have the lowest two bits cleared. >>> +// >>> +// - INFLATING() is a distinguished markword value that is used when >>> +// inflating an existing stack lock into an ObjectMonitor. See below >>> +// for is_being_inflated() and INFLATING(). >>> >>> class BasicLock; >>> class ObjectMonitor; >>> @@ -226,7 +230,7 @@ >>> bool is_being_inflated() const { return (value() == 0); } >>> >>> // Distinguished markword value - used when inflating over >>> - // an existing stacklock. 0 indicates the markword is "BUSY". >>> + // an existing stack lock. 0 indicates the markword is "BUSY". >>> // Lockword mutators that use a LD...CAS idiom should always >>> // check for and avoid overwriting a 0 value installed by some >>> // other thread. (They should spin or block instead. The 0 value >>> >>> >>> Thanks, in advance, for any comments, questions or suggestions. >>> >>> Dan > From ioi.lam at oracle.com Sun Aug 16 01:14:53 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 15 Aug 2020 18:14:53 -0700 Subject: RFR(T) 8251559 Remove empty file utilities/sizes.cpp In-Reply-To: References: <4482d141-de71-c2c9-359d-307c61433c39@oracle.com> Message-ID: David & Calvin, Thanks for the review. Pushed. - Ioi On 8/14/20 10:11 PM, David Holmes wrote: > On 14/08/2020 11:55 am, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8251559 >> >> Please review this trivial fix -- This file has no contents other >> than #includes, so it should be removed. > > Bizarre! It was created that way in June 2000 and has remained > unnoticed for 20 years :) > > Removal is fine. > > Cheers, > David > >> (no webrev since it's just simple removal) >> >> Thanks >> - Ioi From ioi.lam at oracle.com Sun Aug 16 01:19:55 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 15 Aug 2020 18:19:55 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> Message-ID: To better capture what we're trying to do in this RFE, I've changed the RFE title (and the subject of this email thread) to https://bugs.openjdk.java.net/browse/JDK-8247536 Support for pre-generated java.lang.invoke classes in CDS static archive I also update the RFE Description to give an overview of the problem and the solution. The design document is also updated to reflect Yumin's latest implementation https://wiki.openjdk.java.net/display/HotSpot/JDK-8247536+Support+pre-generated+MethodHandle+LambdaForms+in+CDS Thanks - Ioi On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: > Hi Yumin, > > I reviewed mostly the native code. Below are my comments: > > 1) classListParser.hpp > > 71?? bool??????????????? _lambda_format; > > The above name is too generic. How about _lambda_form or _is_lambda_form? > If you rename the above, please also rename the function which returns > the above bool. > > 2) jvm.cpp > > 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, > jstring line)) > > ignore -> ignored > > 3) jvm.hpp > > 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); > > Same comment as for jvm.cpp > > 4) metaspaceShared.cpp > > 2017?? size_t i = 0; > 2018?? while (i < size) { > 2019???? full_name[i++] = *start++; > 2020?? } > > Could the above be simplified to the following? > > ??? strncpy(full_name, start, size - 1); > > 2029?? char* class_name = get_full_class_name(path_name); > > Should os::free(class_name) be called before the function returns? > > 1870 static GrowableArray* lambda_list = NULL; > > The name lambda_list is a bit generic, how about lambda_form_list? > > 2112?????? lambda_list->append(parser.current_line()); > > Since parser.current_line() does a strdup, do those buffer need to be > freed after its use? (i.e. after the call to > regenerate_holder_classes()?) > > In MetaspaceShared::regenerate_holder_classes, before calling up to > java, I think it's better to strip the prefix "@lambda-form-invoker" > from the strings. So that the java > InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need to > handle it: > > ?143???????????????????? case "[LF_RESOLVE]": > ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: > > 5) DumpSymbolAndStringTable.java > > 37???? private static final String my_string = > "DumpSymbolAndStringTable"; > > Unused variable? > > thanks, > > Calvin > > On 8/11/20 10:36 AM, Yumin Qi wrote: >> Forget to send to @core-lib-dev, the patch changed jdk code. >> >> >> Thanks >> >> Yumin >> >> >> >> -------- Forwarded Message -------- >> Subject:???? RFR: 8247536: Support pre-generated MethodHandle lambda >> forms in CDS >> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >> From:???? Yumin Qi >> To:???? hotspot-runtime-dev at openjdk.java.net >> >> >> >> Hi, Please reivew >> >> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >> >> ? Summary: CDS does not archive pre-generated lambda form classes for >> method handles: >> >> [0.142s][info][class,load] >> java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: >> __JVM_LookupDefineClass__ >> >> The method handle resolution if not found in the holder class, a >> class with the method will be generated and loaded as vm internal >> created class and not archived like above. Those class names are >> mangled as combination of the class name and the class instance address. >> >> In this patch, collect those method holder class names and their >> associated methods' signatures when user specify DumpLoadedClassList, >> and record them in the log file in a format similar to the output >> format from traced by >> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use >> another name for CDS: >> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The >> line prefix also changed from "[LF_RESOLVE]" to >> "@lambda-invoke-handle".? At dump time, regenerate the holder class >> with those methods and replace the existing holder class and archived >> it. At runtime, the resolution of the holder class which contains >> those methods are loaded from CDS archive so avoid regenerating them >> again. The patch reorganized the code for Jlink plugin, so the new >> added InvokerBytecodeGeneratorHelper can be shared both by JLink >> plugin and CDS code. Also change made to the mangled generated class >> name at static dump, the class name combining the class name and a >> sequentially ordered number instead of the class instance address, >> which looks like a random. >> >> >> To use this feature, one can do the dump/run just like done for a >> static dump/run, so no extra steps required. >> >> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >> >> ????? DumpLoadedClassList will turn on >> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >> >> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist >> -XX:SharedArchiveFile=my.jsa JavaApp >> >> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >> >> ? The performance on javac HelloWorld.java (javac-benchmark), no >> patch vs patch: >> >> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ----???? 391.720 >> 382.990 ( -8.730)????? ---- >> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ----???? 391.030 >> 381.480 ( -9.550)????? ----- >> ?? 3:?? 2694664066? 2636523114 (-58140952)????? -----??? 390.610 >> 382.550 ( -8.060)????? ---- >> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ----???? 390.700 >> 383.390 ( -7.310)????? --- >> ?? 5:?? 2691072338? 2633016687 (-58055651)????? -----??? 388.990 >> 382.360 ( -6.630)????? --- >> ?? 6:?? 2684448174? 2644191854 (-40256320)????? ---????? 389.450 >> 382.990 ( -6.460)????? --- >> ?? 7:?? 2694921227? 2630505090 (-64416137)????? -----??? 389.300 >> 383.160 ( -6.140)????? --- >> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ----???? 388.370 >> 381.060 ( -7.310)????? --- >> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ----???? 389.560 >> 381.100 ( -8.460)????? ---- >> ? 10:?? 2689162942? 2635658943 (-53503999)????? ----???? 389.690 >> 379.110 (-10.580)????? ----- >> ============================================================ >> ??????? 2689866989? 2637396210 (-52470778)????? ----???? 389.941 >> 382.017 ( -7.924)????? ---- >> >> instr delta =??? -52470778??? -1.9507% >> >> time? delta =?????? -7.924 ms -2.0321% >> >> ? It will show much more improvement if compare with the default >> archive: >> >> ?????????? ? ? ? no patch??????????????????????? patch >> >> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >> >> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >> >> >> ? Tests: >> >> 1)jtreg on jdk/tools and hotspot/runtime local. >> >> 2) mach5 ter1,2 >> >> >> Thanks >> >> Yumin >> >> ? ` From ioi.lam at oracle.com Sun Aug 16 01:27:38 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 15 Aug 2020 18:27:38 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> Message-ID: <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> On 8/15/20 6:19 PM, Ioi Lam wrote: > To better capture what we're trying to do in this RFE, I've changed > the RFE title (and the subject of this email thread) to > > https://bugs.openjdk.java.net/browse/JDK-8247536 > Support for pre-generated java.lang.invoke classes in CDS static archive > > I also update the RFE Description to give an overview of the problem > and the solution. > > The design document is also updated to reflect Yumin's latest > implementation > Oops, sent too quickly. The design doc's title is also updated: https://wiki.openjdk.java.net/display/HotSpot/Support+for+pre-generated+java.lang.invoke+classes+in+CDS+archive > Thanks > - Ioi > > On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: >> Hi Yumin, >> >> I reviewed mostly the native code. Below are my comments: >> >> 1) classListParser.hpp >> >> 71?? bool??????????????? _lambda_format; >> >> The above name is too generic. How about _lambda_form or >> _is_lambda_form? >> If you rename the above, please also rename the function which >> returns the above bool. >> >> 2) jvm.cpp >> >> 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, >> jstring line)) >> >> ignore -> ignored >> >> 3) jvm.hpp >> >> 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); >> >> Same comment as for jvm.cpp >> >> 4) metaspaceShared.cpp >> >> 2017?? size_t i = 0; >> 2018?? while (i < size) { >> 2019???? full_name[i++] = *start++; >> 2020?? } >> >> Could the above be simplified to the following? >> >> ??? strncpy(full_name, start, size - 1); >> >> 2029?? char* class_name = get_full_class_name(path_name); >> >> Should os::free(class_name) be called before the function returns? >> >> 1870 static GrowableArray* lambda_list = NULL; >> >> The name lambda_list is a bit generic, how about lambda_form_list? >> >> 2112?????? lambda_list->append(parser.current_line()); >> >> Since parser.current_line() does a strdup, do those buffer need to be >> freed after its use? (i.e. after the call to >> regenerate_holder_classes()?) >> >> In MetaspaceShared::regenerate_holder_classes, before calling up to >> java, I think it's better to strip the prefix "@lambda-form-invoker" >> from the strings. So that the java >> InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need to >> handle it: >> >> ?143???????????????????? case "[LF_RESOLVE]": >> ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: >> >> 5) DumpSymbolAndStringTable.java >> >> 37???? private static final String my_string = >> "DumpSymbolAndStringTable"; >> >> Unused variable? >> >> thanks, >> >> Calvin >> >> On 8/11/20 10:36 AM, Yumin Qi wrote: >>> Forget to send to @core-lib-dev, the patch changed jdk code. >>> >>> >>> Thanks >>> >>> Yumin >>> >>> >>> >>> -------- Forwarded Message -------- >>> Subject:???? RFR: 8247536: Support pre-generated MethodHandle lambda >>> forms in CDS >>> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >>> From:???? Yumin Qi >>> To:???? hotspot-runtime-dev at openjdk.java.net >>> >>> >>> >>> Hi, Please reivew >>> >>> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >>> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >>> >>> ? Summary: CDS does not archive pre-generated lambda form classes >>> for method handles: >>> >>> [0.142s][info][class,load] >>> java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: >>> __JVM_LookupDefineClass__ >>> >>> The method handle resolution if not found in the holder class, a >>> class with the method will be generated and loaded as vm internal >>> created class and not archived like above. Those class names are >>> mangled as combination of the class name and the class instance >>> address. >>> >>> In this patch, collect those method holder class names and their >>> associated methods' signatures when user specify >>> DumpLoadedClassList, and record them in the log file in a format >>> similar to the output format from traced by >>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use >>> another name for CDS: >>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The >>> line prefix also changed from "[LF_RESOLVE]" to >>> "@lambda-invoke-handle".? At dump time, regenerate the holder class >>> with those methods and replace the existing holder class and >>> archived it. At runtime, the resolution of the holder class which >>> contains those methods are loaded from CDS archive so avoid >>> regenerating them again. The patch reorganized the code for Jlink >>> plugin, so the new added InvokerBytecodeGeneratorHelper can be >>> shared both by JLink plugin and CDS code. Also change made to the >>> mangled generated class name at static dump, the class name >>> combining the class name and a sequentially ordered number instead >>> of the class instance address, which looks like a random. >>> >>> >>> To use this feature, one can do the dump/run just like done for a >>> static dump/run, so no extra steps required. >>> >>> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >>> >>> ????? DumpLoadedClassList will turn on >>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >>> >>> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist >>> -XX:SharedArchiveFile=my.jsa JavaApp >>> >>> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >>> >>> ? The performance on javac HelloWorld.java (javac-benchmark), no >>> patch vs patch: >>> >>> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ---- 391.720 382.990 >>> ( -8.730)????? ---- >>> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ---- 391.030 381.480 >>> ( -9.550)????? ----- >>> ?? 3:?? 2694664066? 2636523114 (-58140952)????? ----- 390.610 >>> 382.550 ( -8.060)????? ---- >>> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ---- 390.700 383.390 >>> ( -7.310)????? --- >>> ?? 5:?? 2691072338? 2633016687 (-58055651)????? ----- 388.990 >>> 382.360 ( -6.630)????? --- >>> ?? 6:?? 2684448174? 2644191854 (-40256320)????? --- 389.450 382.990 >>> ( -6.460)????? --- >>> ?? 7:?? 2694921227? 2630505090 (-64416137)????? ----- 389.300 >>> 383.160 ( -6.140)????? --- >>> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ---- 388.370 381.060 >>> ( -7.310)????? --- >>> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ---- 389.560 381.100 >>> ( -8.460)????? ---- >>> ? 10:?? 2689162942? 2635658943 (-53503999)????? ---- 389.690 379.110 >>> (-10.580)????? ----- >>> ============================================================ >>> ??????? 2689866989? 2637396210 (-52470778)????? ---- 389.941 382.017 >>> ( -7.924)????? ---- >>> >>> instr delta =??? -52470778??? -1.9507% >>> >>> time? delta =?????? -7.924 ms -2.0321% >>> >>> ? It will show much more improvement if compare with the default >>> archive: >>> >>> ?????????? ? ? ? no patch??????????????????????? patch >>> >>> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >>> >>> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >>> >>> >>> ? Tests: >>> >>> 1)jtreg on jdk/tools and hotspot/runtime local. >>> >>> 2) mach5 ter1,2 >>> >>> >>> Thanks >>> >>> Yumin >>> >>> ? ` > From ioi.lam at oracle.com Sun Aug 16 05:25:30 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 15 Aug 2020 22:25:30 -0700 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <6DF4EB82-3B66-4896-9F6B-2E3C7F13AC05@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <6DF4EB82-3B66-4896-9F6B-2E3C7F13AC05@oracle.com> Message-ID: <37aad7e4-7793-13a2-07e5-df73d7c3b624@oracle.com> On 8/15/20 4:42 AM, Kim Barrett wrote: >> On Aug 14, 2020, at 11:27 PM, Ioi Lam wrote: >> >> Hi Kim, >> >> This looks good overall. > Thanks. > >> I found the following function a little confusing: >> >> 108 static Thread* current_thread_or_null() { >> 109 return DEBUG_ONLY(Thread::current_or_null()) NOT_DEBUG(nullptr); >> >> In usual HotSpot code, xxx_or_null() means "if there's no xxx, use". However, in this case, we do have a current thread, but we are just not using it in product build. > Maybe a better name would be something like "current_thread_for_debug"? > The point is we don't need the actual current thread at all in a > non-debug (not non-product, which also includes optimize) build, a > null pointer of the appropriate type will do. And even in a debug > build the call might occur early, before threading has been set up, > so we might not have a current thread. > >> I think it's clearer to changed the constructor >> >> 88 ResourceMark(ResourceArea* area, Thread* thread) : >> >> to >> >> 88 ResourceMark(NOT_PRODUCT_ARG(Thread* thread) ResourceArea* area) : > That doesn't work. (Besides missing a COMMA.) That private > constructor needs to always have the Thread* argument in order to > distinguish it from the public (ResourceArea*) overload. Hi Kim, Thanks for the explanation. Since the new function current_thread_or_null() is used in only one place, I think it's better to get rid of it, and change this constructor as: ? explicit ResourceMark(ResourceArea* area) ??? : ResourceMark(area, DEBUG_ONLY(Thread::current_or_null()) NOT_DEBUG(nullptr)) {} Thanks - Ioi PS, NOT_PRODUCT_ARG does come with a comma: #define NOT_PRODUCT_ARG(arg) arg, > Also, PRODUCT is the wrong flag to be testing for any of this stuff. > All of the conditionalization here was, and should still be, based on > ASSERT, not on PRODUCT. > >> and then use >> >> 119 explicit ResourceMark(ResourceArea* area) >> 120 : ResourceMark(NOT_PRODUCT_ARG(Thread::current()) area) {} > That would be defining a constructor that delegates to itself in a > PRODUCT build. See above. (Besides also missing a COMMA here.) > >> Also, these can be changed to NOT_PRODUCT_ARG -- but you need to move the corresponding field declarations up a few lines >> >> 94 DEBUG_ONLY(COMMA _thread(thread)) >> 95 DEBUG_ONLY(COMMA _previous_resource_mark(nullptr)) > I don't think I understand what you were getting at here with the > "move the corresponding field declarations", but PRODUCT is the wrong > flag to be testing. > From goetz.lindenmaier at sap.com Sun Aug 16 06:25:00 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Sun, 16 Aug 2020 06:25:00 +0000 Subject: RFR(T): 8251846: Replace __linux which is no longer defined In-Reply-To: References: Message-ID: Hi Martin, Looks good to me! Best regards, Goetz -----Original Message----- From: hotspot-runtime-dev On Behalf Of Doerr, Martin Sent: Freitag, 14. August 2020 18:57 To: hotspot-runtime-dev at openjdk.java.net Cc: Stuefe, Thomas Subject: [CAUTION] RFR(T): 8251846: Replace __linux which is no longer defined Hi, we are observing lots of "error occurred during error reporting" problems on linux on PPC64 in JDK16. Reason: debug.cpp uses "#if defined(__linux)". It is no longer defined in our latest build environment. Hence, fixup code for copy of uncontext is missing. Bug: https://bugs.openjdk.java.net/browse/JDK-8251846 Please review this trivial fix: http://cr.openjdk.java.net/~mdoerr/8251846_linux_ppc_context_copy_fix/webrev.00/ Best regards, Martin From kim.barrett at oracle.com Mon Aug 17 00:59:01 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 16 Aug 2020 20:59:01 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <37aad7e4-7793-13a2-07e5-df73d7c3b624@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <6DF4EB82-3B66-4896-9F6B-2E3C7F13AC05@oracle.com> <37aad7e4-7793-13a2-07e5-df73d7c3b624@oracle.com> Message-ID: <8F0D4BE8-AE56-440C-9EB9-6DA5D8D1B7F8@oracle.com> > On Aug 16, 2020, at 1:25 AM, Ioi Lam wrote: > Since the new function current_thread_or_null() is used in only one place, I think it's better to get rid of it, and change this constructor as: > > explicit ResourceMark(ResourceArea* area) > : ResourceMark(area, DEBUG_ONLY(Thread::current_or_null()) NOT_DEBUG(nullptr)) {} I?ll do that. > PS, NOT_PRODUCT_ARG does come with a comma: > > #define NOT_PRODUCT_ARG(arg) arg, Oh, I?ve never had cause to use one of those. I hardly ever write anything conditional on PRODUCT. From david.holmes at oracle.com Mon Aug 17 02:06:04 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 17 Aug 2020 12:06:04 +1000 Subject: RFR 8251414: Add test that invokeinterface of a protected method in java.lang.Object throws NoSuchMethodError In-Reply-To: References: Message-ID: Hi Harold, On 13/08/2020 4:14 am, Harold Seigel wrote: > Hi, > > Please review this change to add test cases to test > hotspot/jtreg/runtime/8026394/InterfaceObjectTest.java to check for > NoSuchMethodError exceptions when calling Object protected methods > clone() and finalize() using invokeinterface. > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8251414/webrev/index.html Can you add a comment explaining why we need to use jasm and reflection in this new case. Thanks, David > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251414 > > The fix was tested by running the modified test on multiple platforms, > including Mac OS, Windows, and Linux. > > Thanks, Harold From felix.yang at huawei.com Mon Aug 17 02:07:26 2020 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Mon, 17 Aug 2020 02:07:26 +0000 Subject: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic References: <654e8a2e-87a1-9c3d-4c58-36b4215bb0ad@redhat.com> <5350e745-87af-ce34-8711-2b652a38e323@redhat.com> <4d1f7179-2cad-620d-f833-949e8b39ee06@redhat.com> <7cedaced-9628-32b3-6d01-beae760fc964@redhat.com> Message-ID: P.S., I witnessed some random assembler warnings when executing aarch64-asmtest.py script. $ python ./src/hotspot/cpu/aarch64/aarch64-asmtest.py | grep Warning aarch64ops.s: Assembler messages: aarch64ops.s:424: Warning: unpredictable load of register pair -- `ldpsw x10,x10,[x9,#-32]' aarch64ops.s:428: Warning: unpredictable transfer with writeback -- `stp w30,w12,[x30,#-256]!' aarch64ops.s:435: Warning: unpredictable transfer with writeback -- `ldp w4,w23,[x4],#-112' aarch64ops.s:221: Warning: unpredictable transfer with writeback -- `str x6,[x6,11]!' aarch64ops.s:223: Warning: unpredictable transfer with writeback -- `strb w0,[x0,-8]!' I think we should avoid this kind of noise. This could be fixed by the following fix: diff -r ff84f0491003 src/hotspot/cpu/aarch64/aarch64-asmtest.py --- a/src/hotspot/cpu/aarch64/aarch64-asmtest.py Sat Aug 15 18:13:49 2020 -0700 +++ b/src/hotspot/cpu/aarch64/aarch64-asmtest.py Mon Aug 17 09:40:12 2020 +0800 @@ -741,6 +741,11 @@ regMode = FloatRegister if isFloat else GeneralRegister self.reg = regMode().generate() + + kindStr = Address.kindToStr(self.kind); + if (not isFloat) and (kindStr is "pre" or kindStr is "post"): + (self.reg.number, self.adr.base.number) = random.sample(range(31), 2) + return self def cstr(self): @@ -777,6 +782,16 @@ self.reg = [OperandFactory.create(self.mode).generate() for i in range(self.numRegs)] self.base = OperandFactory.create('x').generate() + + kindStr = Address.kindToStr(self.kind); + if kindStr is "pre" or kindStr is "post": + if self._name.startswith("ld"): + (self.reg[0].number, self.reg[1].number, self.base.number) = random.sample(range(31), 3) + if self._name.startswith("st"): + self.base.number = random.choice(list(set(range(31)) - set([self.reg[0].number, self.reg[1].number]))) + elif self._name.startswith("ld"): + (self.reg[0].number, self.reg[1].number) = random.sample(range(31), 2) + return self def astr(self): > -----Original Message----- > From: Yangfei (Felix) > Sent: Friday, August 7, 2020 2:20 PM > To: 'Andrew Haley' ; hotspot-runtime- > dev at openjdk.java.net > Cc: aarch64-port-dev at openjdk.java.net > Subject: RE: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic > > Hi, > > > -----Original Message----- > > From: Andrew Haley [mailto:aph at redhat.com] > > Sent: Thursday, August 6, 2020 7:42 PM > > To: Yangfei (Felix) ; hotspot-runtime- > > dev at openjdk.java.net > > Cc: aarch64-port-dev at openjdk.java.net > > Subject: Re: RFR: 8165404: AArch64: Implement SHA512 > > accelerator/intrinsic > > > > On 8/6/20 2:13 AM, Yangfei (Felix) wrote: > > > I have added tests for sha512 crypto instructions in this script and > > regenerated the code. > > > Tier1-3 tested with an aarch64 fastdebug build on my aarch64 linux > > > host to > > cover this part. > > > Updated webrev: > http://cr.openjdk.java.net/~fyang/8165404/webrev.02/ > > > Is it OK? > > > > Yes, thanks. > > > > It's a bit annoying that the indentation seems to change so > > frequently. I'm wondering if perhaps some people have been running > > this on Windows, which has different tab sizes, but that seems very > > unlikely, given that this is a Linux port. But never mind, that's not your > problem. > > I see the output of the python script contains tabs which is not allowed by > jcheck. > So I guess it may depends on the way in which people substitute these tabs. > I used vim editor to do the replacement, which is simple. > Open the file which contains the script output and do: > > :set ts=8 > :set expandtab > :%retab! > > Looks like the indentation is more consistent this way. Hope that helps. > Pushed as: https://hg.openjdk.java.net/jdk/jdk/rev/09ad5b67a099 > > Thanks, > Felix From david.holmes at oracle.com Mon Aug 17 02:15:42 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 17 Aug 2020 12:15:42 +1000 Subject: RFR: 8251365: Build failure on AIX after 8250636 In-Reply-To: References: <48c0d4f2-ffa7-a242-bb5a-f53781eaa5ff@azul.com> Message-ID: <0111efcd-4075-640a-8eaa-02c2de356e59@oracle.com> Hi Peter, On 14/08/2020 4:49 am, Peter Tribble wrote: > On Mon, Aug 10, 2020 at 9:27 PM Dmitry Cherepanov > wrote: > >> Please review the fix for build failure on AIX. With this change, >> "tm_gmtoff" is used only if _ALLBSD_SOURCE or _GNU_SOURCE is set. In the >> case of AIX, we'll continue to use "timezone". >> >> I'll start jdk-submit testing for this change shortly, unfortunately I do >> not have an AIX machine at hand and would appreciate if someone (with >> access to AIX) could verify that the change fixes AIX build . Thanks in >> advance. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8251365 >> Webrev: http://cr.openjdk.java.net/~dcherepanov/8251365/webrev.v1/ >> > > Apologies for butting in at this late stage, but this doesn't look quite > right. > > It still breaks on illumos/Solaris (not supported, but I'm continuing with > an external gcc-based port). > > The reason is that guarding with _GNU_SOURCE isn't the correct > discriminant. I think this > is set up in this block of make/autoconf/flags-cflags.m4 > > #### TOOLCHAIN DEFINES > > if test "x$TOOLCHAIN_TYPE" = xgcc; then > ALWAYS_DEFINES_JVM="-D_GNU_SOURCE -D_REENTRANT" > ALWAYS_DEFINES_JDK="-D_GNU_SOURCE -D_REENTRANT -D_LARGEFILE64_SOURCE" > elif test "x$TOOLCHAIN_TYPE" = xclang; then > ALWAYS_DEFINES_JVM="-D_GNU_SOURCE" > elif test "x$TOOLCHAIN_TYPE" = xxlc; then > ALWAYS_DEFINES_JVM="-D_REENTRANT" > ALWAYS_DEFINES_JDK="-D_GNU_SOURCE -D_REENTRANT -D_LARGEFILE64_SOURCE > -DSTDC" > elif test "x$TOOLCHAIN_TYPE" = xmicrosoft; then > ALWAYS_DEFINES_JDK="-DWIN32_LEAN_AND_MEAN -D_CRT_SECURE_NO_DEPRECATE \ > -D_CRT_NONSTDC_NO_DEPRECATE -DWIN32 -DIAL" > ALWAYS_DEFINES_JVM="-DNOMINMAX -DWIN32_LEAN_AND_MEAN" > fi > > In other words, you get _GNU_SOURCE defined if you're using gcc or clang; > the fix > works on AIX by accident, as a side-effect of using the xlc toolchain there. > > I'll work out a way of patching it for my own work, but it may be worth > looking at a proper fix. So IIUC tm_gmtoff is a GNU libc extension, so we should be checking for __GLIBC__ rather than _GNU_SOURCE (for non-BSD). David ----- > > Thanks, > From kim.barrett at oracle.com Mon Aug 17 02:24:10 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sun, 16 Aug 2020 22:24:10 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> Message-ID: <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> > On Aug 15, 2020, at 8:31 AM, Kim Barrett wrote: > >> On Aug 14, 2020, at 10:29 PM, Kim Barrett wrote: >> >> Please review this simplification of the constructors for ResourceMark, >> accomplished using delegating constructors. >> >> Also made the constructors "explicit" and the class non-copyable. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8251850 >> >> Webrev: >> https://cr.openjdk.java.net/~kbarrett/8251850/open.00/ >> >> Testing: >> mach5 tier1 > > I should have looked further along in resourceArea.hpp. All the > changes I made to ResourceMark? DeoptResourceMark should have similar > changes made to it. > > Well, I have a first round of comments from the ResourceMark changes. I can > take those into account for DeoptResourceMark. This ended up engendering a fairly substantial revision. I've taken at least some of Thomas's comments into account along the way. Changed ResourceMark and DeoptResourceMark to share ResourceMarkImpl, which provides most of the implementation of both, eliminating the copy-paste-modify "reuse" that used to be there. Moved all of the state capture and rollback support into ResourceArea. The mark objects include a captured state object and pass that back to the ResourceArea when doing rollbacks. Cleaned up the _nesting checking. Removed nonsensical ResourceArea::_warned. Maybe I should change the bug summary from "Simplify ResourceMark constructors using delegation" to something like "Refactor ResourceMark and DeoptResourceMark for better code sharing" Note: ResourceArea's handling of UseMallocOnly probably doesn't work in an optimize build. ("probably" because I haven't actually tested it.) The allocation side of it is #ifdef ASSERT, because functions it calls from Arena are debug-only. But the free side is not conditionalized, so could be used in any non-product build when the option is enabled. But maybe it's not really a problem because the free side won't find any work. But this conditionalization inconsistency is odd. New webrevs: full: https://cr.openjdk.java.net/~kbarrett/8251850/open.01/ incr: https://cr.openjdk.java.net/~kbarrett/8251850/open.01.inc/ Testing: mach5 tier1-3 local hotspot:tier1 with -XX:+UseMallocOnly local hotspot:tier1 with -XX:+ZapResourceArea Note that compiler/loopopts/TestDeepGraphVerifyIterativeGVN.java times out when run with -XX:+UseMallocOnly, with or without these changes. From david.holmes at oracle.com Mon Aug 17 02:42:33 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 17 Aug 2020 12:42:33 +1000 Subject: RFR(T): 8251543: add mention of INFLATING() to share/oops/markWord.hpp header comment In-Reply-To: <2627c417-4034-8709-3947-92921896a0d1@oracle.com> References: <1a61123a-a289-c88a-de31-de0f8d0d9eda@oracle.com> <2627c417-4034-8709-3947-92921896a0d1@oracle.com> Message-ID: Hi Dan, On 16/08/2020 12:04 am, Daniel D. Daugherty wrote: > Tap, tap, tap... is this thing working? Any takers for a trivial review? I had a long weekend. :( I see you pushed this but it isn't really what I was looking for. Specifically I wanted the fact that an all-zero header meant "inflating" would be clear from the added text. At the moment if I see: // [ptr | 00] locked ptr points to real header on stack // [header | 0 | 01] unlocked regular object header // [ptr | 10] monitor inflated lock (header is wapped out) // [ptr | 11] marked used to mark an object then it appears that checking for 00 in lsb is sufficient to identify a locked object. But that isn't true because all zeros means inflating. I had hoped to see that explicitly added e.g. // [0 ... 0| 00] inflating inflation in progress then your new text. At a minimum I would have liked the new text to actually state that the INFLATING value is all zeroes. Sorry that my expectations were not clear enough. David ----- > Dan > > > On 8/13/20 5:16 PM, Daniel D. Daugherty wrote: >> In a previous cleanup bug, we normalized >> src/share/runtime/synchronizer.cpp to >> use "stack-lock" instead of "stack lock" and other variants. I'm going >> to change >> the two "stack lock" uses to "stack-lock" below. >> >> Dan >> >> >> On 8/13/20 10:58 AM, Daniel D. Daugherty wrote: >>> Greetings, >>> >>> I have a trivial comment addition to >>> src/hotspot/share/oops/markWord.hpp. >>> >>> Here's the bug: >>> >>> ??? JDK-8251543 add mention of INFLATING() to share/oops/markWord.hpp >>> header comment >>> ??? https://bugs.openjdk.java.net/browse/JDK-8251543 >>> >>> And here's the proposed fix: >>> >>> $ hg diff >>> diff -r 73778bfbffe3 src/hotspot/share/oops/markWord.hpp >>> --- a/src/hotspot/share/oops/markWord.hpp??? Thu Aug 13 10:02:35 2020 >>> -0400 >>> +++ b/src/hotspot/share/oops/markWord.hpp??? Thu Aug 13 10:51:44 2020 >>> -0400 >>> @@ -86,6 +86,10 @@ >>> ?//??? [ptr???????????? | 11]? marked???????????? used to mark an object >>> ?// >>> ?//??? We assume that stack/thread pointers have the lowest two bits >>> cleared. >>> +// >>> +//? - INFLATING() is a distinguished markword value that is used when >>> +//??? inflating an existing stack lock into an ObjectMonitor. See below >>> +//??? for is_being_inflated() and INFLATING(). >>> >>> ?class BasicLock; >>> ?class ObjectMonitor; >>> @@ -226,7 +230,7 @@ >>> ?? bool is_being_inflated() const { return (value() == 0); } >>> >>> ?? // Distinguished markword value - used when inflating over >>> -? // an existing stacklock.? 0 indicates the markword is "BUSY". >>> +? // an existing stack lock.? 0 indicates the markword is "BUSY". >>> ?? // Lockword mutators that use a LD...CAS idiom should always >>> ?? // check for and avoid overwriting a 0 value installed by some >>> ?? // other thread.? (They should spin or block instead.? The 0 value >>> >>> >>> Thanks, in advance, for any comments, questions or suggestions. >>> >>> Dan >> > From thomas.stuefe at gmail.com Mon Aug 17 07:31:15 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 17 Aug 2020 09:31:15 +0200 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> Message-ID: Hi Kim, More remarks, but I slowly enter bikeshedding territory, please decide what you take from this. Apart from the typo this patch looks already fine in its current form. --- activate_state, deactivate_state Matter of taste, I would define those inside an ASSERT block and wrap calls in DEBUG_ONLY, rather than have empty implementations in release code. --- In theory one could make the _area pointer part of the state instead of keeping it as an extra member of RM, since a state is bound to a specific area. And we nowhere test that the area we rollback to is the original one we took the state from. But it's not important. --- typo // Roll back the allocation state to the indicates state values. s/indicates/indicated --- I was surprised to find out that RMs can be triggered prematurely, and also multiple times, by manually calling reset_to_mark(). It's done in two places in the compiler, PhaseCFG::global_code_motion()) and PhaseChaitin::Register_Allocate(). The latter case is the only one I found which resets a mark multiple times. In those two mentioned cases, the mark also fires one additional time when the object goes out of scope, since the mark never gets disarmed. I guess harmless, but surprising and at least unnecessary (in both cases the arena goes out of scope right after). One future improvement may be to split this functionality into two classes, one which does auto resetting in destructor and does not expose "reset_to_mark", one which exposes "reset_to_mark" and has an empty destructor. "AutoResourceMark" vs "ManualResourceMark" maybe? --- I did not find gtests for all that. Maybe we should write some. --- Finally, I wonder whether we could just merge Arena and ResourceArea, for simplicity sake. Move the ability to save/restore state up to Arena and get rid of ResourceArea. There is HandleArena, derived from Arena, and HandleMark, which looks similar to ResourceMark. They would also benefit from inheriting that ability. Thanks, Thomas On Mon, Aug 17, 2020 at 4:24 AM Kim Barrett wrote: > > On Aug 15, 2020, at 8:31 AM, Kim Barrett wrote: > > > >> On Aug 14, 2020, at 10:29 PM, Kim Barrett > wrote: > >> > >> Please review this simplification of the constructors for ResourceMark, > >> accomplished using delegating constructors. > >> > >> Also made the constructors "explicit" and the class non-copyable. > >> > >> CR: > >> https://bugs.openjdk.java.net/browse/JDK-8251850 > >> > >> Webrev: > >> https://cr.openjdk.java.net/~kbarrett/8251850/open.00/ > >> > >> Testing: > >> mach5 tier1 > > > > I should have looked further along in resourceArea.hpp. All the > > changes I made to ResourceMark? DeoptResourceMark should have similar > > changes made to it. > > > > Well, I have a first round of comments from the ResourceMark changes. I > can > > take those into account for DeoptResourceMark. > > This ended up engendering a fairly substantial revision. I've taken at > least some of Thomas's comments into account along the way. > > Changed ResourceMark and DeoptResourceMark to share ResourceMarkImpl, which > provides most of the implementation of both, eliminating the > copy-paste-modify > "reuse" that used to be there. > > Moved all of the state capture and rollback support into ResourceArea. The > mark objects include a captured state object and pass that back to the > ResourceArea when doing rollbacks. > > Cleaned up the _nesting checking. > > Removed nonsensical ResourceArea::_warned. > > Maybe I should change the bug summary from > "Simplify ResourceMark constructors using delegation" > to something like > "Refactor ResourceMark and DeoptResourceMark for better code sharing" > > Note: ResourceArea's handling of UseMallocOnly probably doesn't work in an > optimize build. ("probably" because I haven't actually tested it.) The > allocation side of it is #ifdef ASSERT, because functions it calls from > Arena are debug-only. But the free side is not conditionalized, so could be > used in any non-product build when the option is enabled. But maybe it's > not > really a problem because the free side won't find any work. But this > conditionalization inconsistency is odd. > > New webrevs: > full: https://cr.openjdk.java.net/~kbarrett/8251850/open.01/ > incr: https://cr.openjdk.java.net/~kbarrett/8251850/open.01.inc/ > > Testing: > mach5 tier1-3 > local hotspot:tier1 with -XX:+UseMallocOnly > local hotspot:tier1 with -XX:+ZapResourceArea > > Note that compiler/loopopts/TestDeepGraphVerifyIterativeGVN.java times out > when run with -XX:+UseMallocOnly, with or without these changes. > > > From aph at redhat.com Mon Aug 17 08:32:41 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 17 Aug 2020 09:32:41 +0100 Subject: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic In-Reply-To: References: <654e8a2e-87a1-9c3d-4c58-36b4215bb0ad@redhat.com> <5350e745-87af-ce34-8711-2b652a38e323@redhat.com> <4d1f7179-2cad-620d-f833-949e8b39ee06@redhat.com> <7cedaced-9628-32b3-6d01-beae760fc964@redhat.com> Message-ID: <9e733fb0-5916-2342-7962-ee1ec2f6f076@redhat.com> On 17/08/2020 03:07, Yangfei (Felix) wrote: > P.S., I witnessed some random assembler warnings when executing aarch64-asmtest.py script. > > $ python ./src/hotspot/cpu/aarch64/aarch64-asmtest.py | grep Warning > aarch64ops.s: Assembler messages: > aarch64ops.s:424: Warning: unpredictable load of register pair -- `ldpsw x10,x10,[x9,#-32]' > aarch64ops.s:428: Warning: unpredictable transfer with writeback -- `stp w30,w12,[x30,#-256]!' > aarch64ops.s:435: Warning: unpredictable transfer with writeback -- `ldp w4,w23,[x4],#-112' > aarch64ops.s:221: Warning: unpredictable transfer with writeback -- `str x6,[x6,11]!' > aarch64ops.s:223: Warning: unpredictable transfer with writeback -- `strb w0,[x0,-8]!' > > I think we should avoid this kind of noise. This could be fixed by the following fix: Looks good. If I were editing the Python script I'd also add random.seed(0) so that the generated code doesn't change every time. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From martin.doerr at sap.com Mon Aug 17 08:58:14 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 17 Aug 2020 08:58:14 +0000 Subject: RFR(T): 8251846: Replace __linux which is no longer defined In-Reply-To: <49746dc4-b8ae-2498-224d-30637a1c3a2e@oracle.com> References: <49746dc4-b8ae-2498-224d-30637a1c3a2e@oracle.com> Message-ID: Hi David and G?tz, thanks for the reviews. Pushed. Best regards, Martin > -----Original Message----- > From: David Holmes > Sent: Samstag, 15. August 2020 06:56 > To: Doerr, Martin ; hotspot-runtime- > dev at openjdk.java.net > Cc: Stuefe, Thomas > Subject: Re: RFR(T): 8251846: Replace __linux which is no longer defined > > Looks fine and trivial. > > Thanks, > David > > On 15/08/2020 1:56 am, Doerr, Martin wrote: > > Hi, > > > > we are observing lots of "error occurred during error reporting" problems > on linux on PPC64 in JDK16. > > > > Reason: > > debug.cpp uses "#if defined(__linux)". It is no longer defined in our latest > build environment. > > Hence, fixup code for copy of uncontext is missing. > > > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8251846 > > > > Please review this trivial fix: > > > http://cr.openjdk.java.net/~mdoerr/8251846_linux_ppc_context_copy_fix/ > webrev.00/ > > > > Best regards, > > Martin > > From felix.yang at huawei.com Mon Aug 17 12:01:45 2020 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Mon, 17 Aug 2020 12:01:45 +0000 Subject: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic In-Reply-To: <9e733fb0-5916-2342-7962-ee1ec2f6f076@redhat.com> References: <654e8a2e-87a1-9c3d-4c58-36b4215bb0ad@redhat.com> <5350e745-87af-ce34-8711-2b652a38e323@redhat.com> <4d1f7179-2cad-620d-f833-949e8b39ee06@redhat.com> <7cedaced-9628-32b3-6d01-beae760fc964@redhat.com> <9e733fb0-5916-2342-7962-ee1ec2f6f076@redhat.com> Message-ID: Hi, > -----Original Message----- > From: Andrew Haley [mailto:aph at redhat.com] > Sent: Monday, August 17, 2020 4:33 PM > To: Yangfei (Felix) ; hotspot-runtime- > dev at openjdk.java.net > Cc: aarch64-port-dev at openjdk.java.net > Subject: Re: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic > > On 17/08/2020 03:07, Yangfei (Felix) wrote: > > P.S., I witnessed some random assembler warnings when executing > aarch64-asmtest.py script. > > > > $ python ./src/hotspot/cpu/aarch64/aarch64-asmtest.py | grep Warning > > aarch64ops.s: Assembler messages: > > aarch64ops.s:424: Warning: unpredictable load of register pair -- `ldpsw > x10,x10,[x9,#-32]' > > aarch64ops.s:428: Warning: unpredictable transfer with writeback -- `stp > w30,w12,[x30,#-256]!' > > aarch64ops.s:435: Warning: unpredictable transfer with writeback -- `ldp > w4,w23,[x4],#-112' > > aarch64ops.s:221: Warning: unpredictable transfer with writeback -- `str > x6,[x6,11]!' > > aarch64ops.s:223: Warning: unpredictable transfer with writeback -- `strb > w0,[x0,-8]!' > > > > I think we should avoid this kind of noise. This could be fixed by the > following fix: > > Looks good. If I were editing the Python script I'd also add > random.seed(0) so that the generated code doesn't change every time. Thanks for the quick reply. Do you mean we need any more change? For the purpose for testing the instructions, it might not be a good idea to add random.seed(0). This will limit the randomness of the register numbers used for each run. Best regards, Felix From kim.barrett at oracle.com Mon Aug 17 13:15:01 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 17 Aug 2020 09:15:01 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> Message-ID: <1A8063BF-5C2C-4A8E-9491-E1D3B6B31870@oracle.com> > On Aug 17, 2020, at 3:31 AM, Thomas St?fe wrote: > > Hi Kim, > > More remarks, but I slowly enter bikeshedding territory, please decide what you take from this. Apart from the typo this patch looks already fine in its current form. Thanks. I've fixed the typo locally. > activate_state, deactivate_state > > Matter of taste, I would define those inside an ASSERT block and wrap calls in DEBUG_ONLY, rather than have empty implementations in release code. I prefer having the DEBUG_ONLY close to the code that cares, i.e. ResourceArea has DEBUG_ONLY _nested, and ResourceArea should be responsible for dealing with that. That these functions end up being nops in release builds is an implementation detail of ResourceArea. > In theory one could make the _area pointer part of the state instead of keeping it as an extra member of RM, since a state is bound to a specific area. And we nowhere test that the area we rollback to is the original one we took the state from. But it's not important. We could, but that didn?t seem important. > typo > > // Roll back the allocation state to the indicates state values. > > s/indicates/indicated Fixed locally. > I was surprised to find out that RMs can be triggered prematurely, and also multiple times, by manually calling reset_to_mark(). It's done in two places in the compiler, PhaseCFG::global_code_motion()) and PhaseChaitin::Register_Allocate(). The latter case is the only one I found which resets a mark multiple times. I was surprised by this too. I spent a little bit of time trying to figure out why, and whether those uses could instead just use several scoped ResourceMarks, but eventually decided I just wasn't sufficiently familiar with that code, and this change had already gotten a bit larger than I'd been expecting. > In those two mentioned cases, the mark also fires one additional time when the object goes out of scope, since the mark never gets disarmed. I guess harmless, but surprising and at least unnecessary (in both cases the arena goes out of scope right after). I don't think the extra firing is really a problem, though presumably the calling code could be changed to eliminate an unnecessary reset that shortly preceeds the destructor. OTOH being explicit about where the reset is safe might make future modifications of those callers a little easier. > One future improvement may be to split this functionality into two classes, one which does auto resetting in destructor and does not expose "reset_to_mark", one which exposes "reset_to_mark" and has an empty destructor. "AutoResourceMark" vs "ManualResourceMark" maybe? I don't know if that distinction would really carry its weight. And I think it would be better to first see if the manual cases can be eliminated. > I did not find gtests for all that. Maybe we should write some. Yeah, someone should probably do that. :) > Finally, I wonder whether we could just merge Arena and ResourceArea, for simplicity sake. Move the ability to save/restore state up to Arena and get rid of ResourceArea. There is HandleArena, derived from Arena, and HandleMark, which looks similar to ResourceMark. They would also benefit from inheriting that ability. Yeah, you are right that HandleMark and ResourceMark are very similar, and probably similarly mergeable. I think I will leave that to someone else though. From daniel.daugherty at oracle.com Mon Aug 17 13:26:29 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Mon, 17 Aug 2020 09:26:29 -0400 Subject: RFR(T): 8251543: add mention of INFLATING() to share/oops/markWord.hpp header comment In-Reply-To: References: <1a61123a-a289-c88a-de31-de0f8d0d9eda@oracle.com> <2627c417-4034-8709-3947-92921896a0d1@oracle.com> Message-ID: <74653a0f-6c4d-7169-20ff-c3dfa473cabd@oracle.com> On 8/16/20 10:42 PM, David Holmes wrote: > Hi Dan, > > On 16/08/2020 12:04 am, Daniel D. Daugherty wrote: >> Tap, tap, tap... is this thing working? Any takers for a trivial review? > > I had a long weekend. :( No worries. > I see you pushed this but it isn't really what I was looking for. > Specifically I wanted the fact that an all-zero header meant > "inflating" would be clear from the added text. At the moment if I see: > > //??? [ptr???????????? | 00]? locked???????????? ptr points to real > header on stack > > //??? [header????? | 0 | 01]? unlocked?????????? regular object header > > //??? [ptr???????????? | 10]? monitor??????????? inflated lock (header > is wapped out) > > //??? [ptr???????????? | 11]? marked???????????? used to mark an object > > then it appears that checking for 00 in lsb is sufficient to identify > a locked object. But that isn't true because all zeros means > inflating. I had hoped to see that explicitly added e.g. > > //??? [0????? ...??? 0| 00]? inflating???????????? inflation in progress > > then your new text. At a minimum I would have liked the new text to > actually state that the INFLATING value is all zeroes. When I first wrote the comment, I mentioned that INFLATING was all zeroes. Later I went back and reread the comment and guessed that you wouldn't like that I mentioned that level of detail. Kind of like when I add comments that detail underlying memory barrier ops and you've asked me to be less specific. I can file a new bug to add the following to that "table": //??? [0????? ...??? 0| 00]? inflating???????????? inflation in progress > Sorry that my expectations were not clear enough. No worries. I probably should have waited to hear from you since you asked for the comment. Dan > > David > ----- > >> Dan >> >> >> On 8/13/20 5:16 PM, Daniel D. Daugherty wrote: >>> In a previous cleanup bug, we normalized >>> src/share/runtime/synchronizer.cpp to >>> use "stack-lock" instead of "stack lock" and other variants. I'm >>> going to change >>> the two "stack lock" uses to "stack-lock" below. >>> >>> Dan >>> >>> >>> On 8/13/20 10:58 AM, Daniel D. Daugherty wrote: >>>> Greetings, >>>> >>>> I have a trivial comment addition to >>>> src/hotspot/share/oops/markWord.hpp. >>>> >>>> Here's the bug: >>>> >>>> ??? JDK-8251543 add mention of INFLATING() to >>>> share/oops/markWord.hpp header comment >>>> ??? https://bugs.openjdk.java.net/browse/JDK-8251543 >>>> >>>> And here's the proposed fix: >>>> >>>> $ hg diff >>>> diff -r 73778bfbffe3 src/hotspot/share/oops/markWord.hpp >>>> --- a/src/hotspot/share/oops/markWord.hpp??? Thu Aug 13 10:02:35 >>>> 2020 -0400 >>>> +++ b/src/hotspot/share/oops/markWord.hpp??? Thu Aug 13 10:51:44 >>>> 2020 -0400 >>>> @@ -86,6 +86,10 @@ >>>> ?//??? [ptr???????????? | 11]? marked???????????? used to mark an >>>> object >>>> ?// >>>> ?//??? We assume that stack/thread pointers have the lowest two >>>> bits cleared. >>>> +// >>>> +//? - INFLATING() is a distinguished markword value that is used when >>>> +//??? inflating an existing stack lock into an ObjectMonitor. See >>>> below >>>> +//??? for is_being_inflated() and INFLATING(). >>>> >>>> ?class BasicLock; >>>> ?class ObjectMonitor; >>>> @@ -226,7 +230,7 @@ >>>> ?? bool is_being_inflated() const { return (value() == 0); } >>>> >>>> ?? // Distinguished markword value - used when inflating over >>>> -? // an existing stacklock.? 0 indicates the markword is "BUSY". >>>> +? // an existing stack lock.? 0 indicates the markword is "BUSY". >>>> ?? // Lockword mutators that use a LD...CAS idiom should always >>>> ?? // check for and avoid overwriting a 0 value installed by some >>>> ?? // other thread.? (They should spin or block instead. The 0 value >>>> >>>> >>>> Thanks, in advance, for any comments, questions or suggestions. >>>> >>>> Dan >>> >> From thomas.stuefe at gmail.com Mon Aug 17 13:30:47 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 17 Aug 2020 15:30:47 +0200 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <1A8063BF-5C2C-4A8E-9491-E1D3B6B31870@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <1A8063BF-5C2C-4A8E-9491-E1D3B6B31870@oracle.com> Message-ID: On Mon, Aug 17, 2020 at 3:15 PM Kim Barrett wrote: > > On Aug 17, 2020, at 3:31 AM, Thomas St?fe > wrote: > > > > Hi Kim, > > > > More remarks, but I slowly enter bikeshedding territory, please decide > what you take from this. Apart from the typo this patch looks already fine > in its current form. > > Thanks. I've fixed the typo locally. > > > activate_state, deactivate_state > > > > Matter of taste, I would define those inside an ASSERT block and wrap > calls in DEBUG_ONLY, rather than have empty implementations in release code. > > I prefer having the DEBUG_ONLY close to the code that cares, i.e. > ResourceArea has DEBUG_ONLY _nested, and ResourceArea should be responsible > for dealing with that. That these functions end up being nops in release > builds is an implementation detail of ResourceArea. > > > In theory one could make the _area pointer part of the state instead of > keeping it as an extra member of RM, since a state is bound to a specific > area. And we nowhere test that the area we rollback to is the original one > we took the state from. But it's not important. > > We could, but that didn?t seem important. > > > typo > > > > // Roll back the allocation state to the indicates state values. > > > > s/indicates/indicated > > Fixed locally. > > > I was surprised to find out that RMs can be triggered prematurely, and > also multiple times, by manually calling reset_to_mark(). It's done in two > places in the compiler, PhaseCFG::global_code_motion()) and > PhaseChaitin::Register_Allocate(). The latter case is the only one I found > which resets a mark multiple times. > > I was surprised by this too. I spent a little bit of time trying to figure > out why, and whether those uses could instead just use several scoped > ResourceMarks, but eventually decided I just wasn't sufficiently familiar > with that code, and this change had already gotten a bit larger than I'd > been expecting. > > > In those two mentioned cases, the mark also fires one additional time > when the object goes out of scope, since the mark never gets disarmed. I > guess harmless, but surprising and at least unnecessary (in both cases the > arena goes out of scope right after). > > I don't think the extra firing is really a problem, though presumably the > calling code could be changed to eliminate an unnecessary reset that > shortly > preceeds the destructor. OTOH being explicit about where the reset is safe > might make future modifications of those callers a little easier. > > > One future improvement may be to split this functionality into two > classes, one which does auto resetting in destructor and does not expose > "reset_to_mark", one which exposes "reset_to_mark" and has an empty > destructor. "AutoResourceMark" vs "ManualResourceMark" maybe? > > I don't know if that distinction would really carry its weight. And I think > it would be better to first see if the manual cases can be eliminated. > > > I did not find gtests for all that. Maybe we should write some. > > Yeah, someone should probably do that. :) > > > Finally, I wonder whether we could just merge Arena and ResourceArea, > for simplicity sake. Move the ability to save/restore state up to Arena and > get rid of ResourceArea. There is HandleArena, derived from Arena, and > HandleMark, which looks similar to ResourceMark. They would also benefit > from inheriting that ability. > > Yeah, you are right that HandleMark and ResourceMark are very similar, and > probably similarly mergeable. I think I will leave that to someone else > though. > > All good then. To me this looks fine, don't need another webrev. Thanks, Thomas From thomas.schatzl at oracle.com Mon Aug 17 13:45:05 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 17 Aug 2020 15:45:05 +0200 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> Message-ID: <6020d90d-5413-997a-a689-d6a817fb7b7e@oracle.com> Hi, On 17.08.20 04:24, Kim Barrett wrote: >> On Aug 15, 2020, at 8:31 AM, Kim Barrett wrote: >> >>> On Aug 14, 2020, at 10:29 PM, Kim Barrett wrote: >>> >>> Please review this simplification of the constructors for ResourceMark, >>> accomplished using delegating constructors. >>> >>> Also made the constructors "explicit" and the class non-copyable. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8251850 >>> >>> Webrev: >>> https://cr.openjdk.java.net/~kbarrett/8251850/open.00/ >>> >>> Testing: >>> mach5 tier1 >> >> I should have looked further along in resourceArea.hpp. All the >> changes I made to ResourceMark? DeoptResourceMark should have similar >> changes made to it. >> >> Well, I have a first round of comments from the ResourceMark changes. I can >> take those into account for DeoptResourceMark. > > This ended up engendering a fairly substantial revision. I've taken at > least some of Thomas's comments into account along the way. > > Changed ResourceMark and DeoptResourceMark to share ResourceMarkImpl, which > provides most of the implementation of both, eliminating the copy-paste-modify > "reuse" that used to be there. > > Moved all of the state capture and rollback support into ResourceArea. The > mark objects include a captured state object and pass that back to the > ResourceArea when doing rollbacks. > > Cleaned up the _nesting checking. > > Removed nonsensical ResourceArea::_warned. > > Maybe I should change the bug summary from > "Simplify ResourceMark constructors using delegation" > to something like > "Refactor ResourceMark and DeoptResourceMark for better code sharing" > Please do. > New webrevs: > full: https://cr.openjdk.java.net/~kbarrett/8251850/open.01/ > incr: https://cr.openjdk.java.net/~kbarrett/8251850/open.01.inc/ > > Testing: > mach5 tier1-3 > local hotspot:tier1 with -XX:+UseMallocOnly > local hotspot:tier1 with -XX:+ZapResourceArea > > Note that compiler/loopopts/TestDeepGraphVerifyIterativeGVN.java times out > when run with -XX:+UseMallocOnly, with or without these changes. > > looks good. Thomas From harold.seigel at oracle.com Mon Aug 17 14:26:34 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Mon, 17 Aug 2020 10:26:34 -0400 Subject: RFR 8251414: Add test that invokeinterface of a protected method in java.lang.Object throws NoSuchMethodError In-Reply-To: References: Message-ID: <97328605-aa99-51db-25eb-e1465919b8db@oracle.com> Hi David, This change got pushed before I saw your comments.? Let me know if I should enter a new RFE to add the comments that you request below. Thanks, Harold On 8/16/2020 10:06 PM, David Holmes wrote: > Hi Harold, > > On 13/08/2020 4:14 am, Harold Seigel wrote: >> Hi, >> >> Please review this change to add test cases to test >> hotspot/jtreg/runtime/8026394/InterfaceObjectTest.java to check for >> NoSuchMethodError exceptions when calling Object protected methods >> clone() and finalize() using invokeinterface. >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8251414/webrev/index.html > > Can you add a comment explaining why we need to use jasm and > reflection in this new case. > > Thanks, > David > >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251414 >> >> The fix was tested by running the modified test on multiple >> platforms, including Mac OS, Windows, and Linux. >> >> Thanks, Harold From kim.barrett at oracle.com Mon Aug 17 14:48:59 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 17 Aug 2020 10:48:59 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <1A8063BF-5C2C-4A8E-9491-E1D3B6B31870@oracle.com> Message-ID: <52C5E056-6AF9-4981-A0DC-49B813FF2348@oracle.com> > On Aug 17, 2020, at 9:30 AM, Thomas St?fe wrote: > > > > All good then. To me this looks fine, don't need another webrev. > > Thanks, Thomas > > Thanks. From kim.barrett at oracle.com Mon Aug 17 14:52:45 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 17 Aug 2020 10:52:45 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <6020d90d-5413-997a-a689-d6a817fb7b7e@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <6020d90d-5413-997a-a689-d6a817fb7b7e@oracle.com> Message-ID: <32A717E4-687E-4B06-A696-0CBA037E9676@oracle.com> > On Aug 17, 2020, at 9:45 AM, Thomas Schatzl wrote: > > Hi, > > On 17.08.20 04:24, Kim Barrett wrote: >>> On Aug 15, 2020, at 8:31 AM, Kim Barrett wrote: >>> >>>> On Aug 14, 2020, at 10:29 PM, Kim Barrett wrote: >>>> >>>> Please review this simplification of the constructors for ResourceMark, >>>> accomplished using delegating constructors. >>>> >>>> Also made the constructors "explicit" and the class non-copyable. >>>> >>>> CR: >>>> https://bugs.openjdk.java.net/browse/JDK-8251850 >>>> >>>> Webrev: >>>> https://cr.openjdk.java.net/~kbarrett/8251850/open.00/ >>>> >>>> Testing: >>>> mach5 tier1 >>> >>> I should have looked further along in resourceArea.hpp. All the >>> changes I made to ResourceMark? DeoptResourceMark should have similar >>> changes made to it. >>> >>> Well, I have a first round of comments from the ResourceMark changes. I can >>> take those into account for DeoptResourceMark. >> This ended up engendering a fairly substantial revision. I've taken at >> least some of Thomas's comments into account along the way. >> Changed ResourceMark and DeoptResourceMark to share ResourceMarkImpl, which >> provides most of the implementation of both, eliminating the copy-paste-modify >> "reuse" that used to be there. >> Moved all of the state capture and rollback support into ResourceArea. The >> mark objects include a captured state object and pass that back to the >> ResourceArea when doing rollbacks. >> Cleaned up the _nesting checking. >> Removed nonsensical ResourceArea::_warned. >> Maybe I should change the bug summary from >> "Simplify ResourceMark constructors using delegation" >> to something like >> "Refactor ResourceMark and DeoptResourceMark for better code sharing" > > Please do. Done. > >> New webrevs: >> full: https://cr.openjdk.java.net/~kbarrett/8251850/open.01/ >> incr: https://cr.openjdk.java.net/~kbarrett/8251850/open.01.inc/ >> Testing: >> mach5 tier1-3 >> local hotspot:tier1 with -XX:+UseMallocOnly >> local hotspot:tier1 with -XX:+ZapResourceArea >> Note that compiler/loopopts/TestDeepGraphVerifyIterativeGVN.java times out >> when run with -XX:+UseMallocOnly, with or without these changes. > > looks good. > > Thomas Thanks. From gerard.ziemski at oracle.com Mon Aug 17 15:28:43 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Mon, 17 Aug 2020 10:28:43 -0500 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> Message-ID: <95b46d0f-8a81-10c6-78bb-a3abbd7cb955@oracle.com> Thank you David and Igor for the feedback! I rewrote the test again, this time basing it off test/hotspot/jtreg/runtime/Thread/* tests. For macOS and Linux we create threads until we hit a process limit thread count. Windows, however, allows unlimited (?) number of threads, though the process gradually slows down as the count grows, so I had to add time limit and I set it to 5 sec. I chose 5 sec because the value is well below jtreg time out (i.e. 120s), but still quite above how long my laptop needs to hit the process thread count limit (i.e. my Mac laptop needs about 1 sec to hit that limit). Otherwise there are no other hardwired values or assumptions (I hope). What do you think of the test now? bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev2 testing: Mach5 hs-tier1,2,3,4,5 in progress... On 8/7/20 2:41 PM, Igor Ignatyev wrote: > > >> On Aug 6, 2020, at 12:33 AM, David Holmes > > wrote: >> >> Basing the new test around the existing thread stress tests was not >> unreasonable but I don't think we're actively adding to those tests. >> I would check with Igor whether it is better to just add this under >> runtime/thread rather than vmTestBase. > > Hi Gerard, > > David is right, we are trying to get rid of vmTestbase directory and > move all tests into corresponding component directories cleaning them > up at the same time. so runtime/thread directory sound like a much > better location for a new test. > > I don't think you need to cary all this baggage of 'VM testbase > keywords', 'converted from ...' and the rest, I'd suggest you just use > the text from 'DESCRIPTION' as @summary. I'd also remove System.exit > call and would just make thread009::run to return bool. it's also > unclear to me if someone is ever going to change THREADS_EXPECTED , > TIMEOUT, and DEBUG_MODE?values, I'd just hardcode them in the test and > get rid of parsing. it seems that debug output won't clutter output, > so you might want to always print them and remove DEBUG_MODE field. > > using `assert` in test code isn't a good idea as it will work only if > someone runs test w/ `-ea` which not all people do. it's better to > replace w/ `if (p) throw new Error(...)` ?or use jdk.test.lib.Asserts > class. > > Cheers, > -- Igor From aph at redhat.com Mon Aug 17 15:53:48 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 17 Aug 2020 16:53:48 +0100 Subject: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic In-Reply-To: References: <654e8a2e-87a1-9c3d-4c58-36b4215bb0ad@redhat.com> <5350e745-87af-ce34-8711-2b652a38e323@redhat.com> <4d1f7179-2cad-620d-f833-949e8b39ee06@redhat.com> <7cedaced-9628-32b3-6d01-beae760fc964@redhat.com> <9e733fb0-5916-2342-7962-ee1ec2f6f076@redhat.com> Message-ID: <80b588a5-e07f-9d8e-1079-c14530570549@redhat.com> On 17/08/2020 13:01, Yangfei (Felix) wrote: >> Looks good. If I were editing the Python script I'd also add >> random.seed(0) so that the generated code doesn't change every time. > Thanks for the quick reply. Do you mean we need any more change? > For the purpose for testing the instructions, it might not be a good > idea to add random.seed(0). This will limit the randomness of the > register numbers used for each run. Not really, because we paste the output into assembler.cpp so that we use the same values every time it runs. I don't think it helps anyone that we use different random offsets every time the test code is regenerated. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From IOI.LAM at ORACLE.COM Mon Aug 17 16:43:06 2020 From: IOI.LAM at ORACLE.COM (Ioi Lam) Date: Mon, 17 Aug 2020 09:43:06 -0700 Subject: RFR(S): 8247529: Crash in runtime/cds/appcds/dynamicArchive/LambdaProxyCallerIsHidden.java with Graal In-Reply-To: References: Message-ID: <41100C06-748F-4A78-B9AA-FB3CF6042EBE@ORACLE.COM> Hi Calvin, This looks good to me. Just a small nit: the NULL assignment is not needed here since p is assigned immediately below 06 DumpTimeSharedClassInfo* p = NULL; Thanks Ioi > On Aug 14, 2020, at 9:10 AM, calvin.cheung at oracle.com wrote: > > ?bug: https://bugs.openjdk.java.net/browse/JDK-8247529 > webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8247529/webrev.00/ > > This proposed change includes: > > 1) Avoid inserting InstanceKlass into the DumpTimeSharedClassTable after dynamic CDS dumping has started. > > This is done by adding a _dump_in_progress bool which will be set at the beginning of dynamic CDS dumping. In DumpTimeSharedClassTable::find_or_allocate_info_for, if the _dump_in_progress is set, it will not create a new entry in the table. > > 2) Acquire the DumpTimeTable_lock before iterations on _dumptime_table to ensure memory order correctness. > > Testing: passed tier1 - 4 tests. > > thanks, > > Calvin > > > From xxinliu at amazon.com Mon Aug 17 16:52:56 2020 From: xxinliu at amazon.com (Liu, Xin) Date: Mon, 17 Aug 2020 16:52:56 +0000 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> Message-ID: <1597683176006.22884@amazon.com> Hi, Kim, You patch looks good to me. I have a relevant question about constructor. In opto, there's another candidate to use delegating constructor. https://hg.openjdk.java.net/jdk/jdk/file/b6475f411b98/src/hotspot/share/opto/node.cpp#l361 It's pretty annoying when I attempt to add a new member variable to Node. To well initialize it, I have to modify all different constructors. needless to say, it's error-prone because C++ can't promise you that all constructors are in a same file. I have two options here. 1) use delegate constructor like you did here. 2) use default initializers for member variables(c++11) eg. http://cr.openjdk.java.net/~phh/8251464/webrev.00/src/hotspot/share/opto/node.hpp.udiff.html I am not sure if the second option is endorsed. Quote from hotspot coding style (https://hg.openjdk.java.net/jdk/jdk/raw-file/b6475f411b98/doc/hotspot-style.html) "Initialize all variables and data structures to a known state. If a class has a constructor, initialize it there." My understanding is if multiple constructors exist, prefer to use delegate constructor. Use default initializer if and only if a class doesn't have any custom constructor. is that correct? If it's correct, do we have a style for default initializer? for scalar variables, both assignment and {} work. eg. int _depth = 0; or int _depth {0}; thanks, --lx _______________________________________ From: hotspot-runtime-dev on behalf of Kim Barrett Sent: Friday, August 14, 2020 7:29 PM To: hotspot-runtime-dev at openjdk.java.net runtime Subject: [EXTERNAL] RFR: 8251850: Simplify ResourceMark constructors using delegation CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Please review this simplification of the constructors for ResourceMark, accomplished using delegating constructors. Also made the constructors "explicit" and the class non-copyable. CR: https://bugs.openjdk.java.net/browse/JDK-8251850 Webrev: https://cr.openjdk.java.net/~kbarrett/8251850/open.00/ Testing: mach5 tier1 From yumin.qi at oracle.com Mon Aug 17 17:53:41 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Mon, 17 Aug 2020 10:53:41 -0700 Subject: RFR(S): 8247529: Crash in runtime/cds/appcds/dynamicArchive/LambdaProxyCallerIsHidden.java with Graal In-Reply-To: <41100C06-748F-4A78-B9AA-FB3CF6042EBE@ORACLE.COM> References: <41100C06-748F-4A78-B9AA-FB3CF6042EBE@ORACLE.COM> Message-ID: <33b68a3d-da9f-cc6b-72e0-caac6b497c91@oracle.com> +1 Thanks Yumin On 8/17/20 9:43 AM, Ioi Lam wrote: > Hi Calvin, > > This looks good to me. Just a small nit: the NULL assignment is not needed here since p is assigned immediately below > > 06 DumpTimeSharedClassInfo* p = NULL; > > Thanks > Ioi >> On Aug 14, 2020, at 9:10 AM, calvin.cheung at oracle.com wrote: >> >> ?bug: https://bugs.openjdk.java.net/browse/JDK-8247529 >> webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8247529/webrev.00/ >> >> This proposed change includes: >> >> 1) Avoid inserting InstanceKlass into the DumpTimeSharedClassTable after dynamic CDS dumping has started. >> >> This is done by adding a _dump_in_progress bool which will be set at the beginning of dynamic CDS dumping. In DumpTimeSharedClassTable::find_or_allocate_info_for, if the _dump_in_progress is set, it will not create a new entry in the table. >> >> 2) Acquire the DumpTimeTable_lock before iterations on _dumptime_table to ensure memory order correctness. >> >> Testing: passed tier1 - 4 tests. >> >> thanks, >> >> Calvin >> >> >> From calvin.cheung at oracle.com Mon Aug 17 17:56:58 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Mon, 17 Aug 2020 10:56:58 -0700 Subject: RFR(S): 8247529: Crash in runtime/cds/appcds/dynamicArchive/LambdaProxyCallerIsHidden.java with Graal In-Reply-To: <41100C06-748F-4A78-B9AA-FB3CF6042EBE@ORACLE.COM> References: <41100C06-748F-4A78-B9AA-FB3CF6042EBE@ORACLE.COM> Message-ID: <6eb133d9-1646-89e2-c095-9f321a1b6612@oracle.com> Hi Ioi, Thanks for your review! I'll make the change before push. thanks, Calvin On 8/17/20 9:43 AM, Ioi Lam wrote: > Hi Calvin, > > This looks good to me. Just a small nit: the NULL assignment is not > needed here since p is assigned immediately below > > 06 DumpTimeSharedClassInfo* p = NULL; > Thanks > Ioi >> On Aug 14, 2020, at 9:10 AM, calvin.cheung at oracle.com wrote: >> >> ?bug: https://bugs.openjdk.java.net/browse/JDK-8247529 >> webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8247529/webrev.00/ >> >> This proposed change includes: >> >> 1) Avoid inserting InstanceKlass into the DumpTimeSharedClassTable >> after dynamic CDS dumping has started. >> >> This is done by adding a _dump_in_progress bool which will be set at >> the beginning of dynamic CDS dumping. In >> DumpTimeSharedClassTable::find_or_allocate_info_for, if the >> _dump_in_progress is set, it will not create a new entry in the table. >> >> 2) Acquire the DumpTimeTable_lock before iterations on >> _dumptime_table to ensure memory order correctness. >> >> Testing: passed tier1 - 4 tests. >> >> thanks, >> >> Calvin >> >> >> From calvin.cheung at oracle.com Mon Aug 17 17:57:56 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Mon, 17 Aug 2020 10:57:56 -0700 Subject: RFR(S): 8247529: Crash in runtime/cds/appcds/dynamicArchive/LambdaProxyCallerIsHidden.java with Graal In-Reply-To: <33b68a3d-da9f-cc6b-72e0-caac6b497c91@oracle.com> References: <41100C06-748F-4A78-B9AA-FB3CF6042EBE@ORACLE.COM> <33b68a3d-da9f-cc6b-72e0-caac6b497c91@oracle.com> Message-ID: <3fc03b42-88be-5a03-9b61-684130898942@oracle.com> Thanks, Yumin! On 8/17/20 10:53 AM, Yumin Qi wrote: > +1 > > > Thanks > > Yumin > > On 8/17/20 9:43 AM, Ioi Lam wrote: >> Hi Calvin, >> >> This looks good to me. Just a small nit: the NULL assignment is not >> needed here since p is assigned immediately below >> >> 06???? DumpTimeSharedClassInfo* p = NULL; >> >> Thanks >> Ioi >>> On Aug 14, 2020, at 9:10 AM, calvin.cheung at oracle.com wrote: >>> >>> ?bug: https://bugs.openjdk.java.net/browse/JDK-8247529 >>> webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8247529/webrev.00/ >>> >>> This proposed change includes: >>> >>> 1) Avoid inserting InstanceKlass into the DumpTimeSharedClassTable >>> after dynamic CDS dumping has started. >>> >>> This is done by adding a _dump_in_progress bool which will be set at >>> the beginning of dynamic CDS dumping. In >>> DumpTimeSharedClassTable::find_or_allocate_info_for, if the >>> _dump_in_progress is set, it will not create a new entry in the table. >>> >>> 2) Acquire the DumpTimeTable_lock before iterations on >>> _dumptime_table to ensure memory order correctness. >>> >>> Testing: passed tier1 - 4 tests. >>> >>> thanks, >>> >>> Calvin >>> >>> >>> From mandy.chung at oracle.com Mon Aug 17 19:29:31 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 17 Aug 2020 12:29:31 -0700 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: References: <9e6b0043-65a1-dd97-a3d2-33679c8048d4@oracle.com> Message-ID: <57711fdd-4c94-d4dc-de1e-f0be6dce4fc6@oracle.com> On 8/12/20 3:06 PM, Ioi Lam wrote: > Hi Lois, > > Thanks for the comments. I have an updated webrev > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/ > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02.delta/ > > This is very good work.?? I reviewed the Java code change that looks okay to me.?? This version is cleaner.? I skimmed through the VM code which is a huge change. ? I see Coleen and Lois reviewing and nothing to add. Mandy From yumin.qi at oracle.com Mon Aug 17 19:37:31 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Mon, 17 Aug 2020 12:37:31 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> Message-ID: <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> Hi, Ioi ? Thanks for review/suggestion. I have updated the webrev at the following link: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ ? Following changes done in this patch: ? 1) move regenerating holder classes into new added file: lambdaFormInvokers.[ch]pp "@lambda-form-invoker" tag only observed at jvm side. ? 2) remove InvokerBytecodeGeneratorHelper.java, move related functions to existing class, GenerateJLIClassesHelper.java ? 3) add tracing for SPECIES_RESOLVE, so archive more classes. ? 4) remove InvokerGenerateBytesException.java, using RuntimeException instead. ? 5) Changed make file to exclude the new added lamdaFormInvokers.cpp from the non-cds building list. ? 6) There is a bug in previous patch, jobject (typeArrayOop) should not be used directly for loading class since GC will move the java object. Fixed by copying the bytes from handle to C heap. ? 7) Tested with tier1,2 Thanks Yumin On 8/15/20 6:27 PM, Ioi Lam wrote: > On 8/15/20 6:19 PM, Ioi Lam wrote: >> To better capture what we're trying to do in this RFE, I've changed the RFE title (and the subject of this email thread) to >> >> https://bugs.openjdk.java.net/browse/JDK-8247536 >> Support for pre-generated java.lang.invoke classes in CDS static archive >> >> I also update the RFE Description to give an overview of the problem and the solution. >> >> The design document is also updated to reflect Yumin's latest implementation >> > > Oops, sent too quickly. The design doc's title is also updated: > > https://wiki.openjdk.java.net/display/HotSpot/Support+for+pre-generated+java.lang.invoke+classes+in+CDS+archive > >> Thanks >> - Ioi >> >> On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: >>> Hi Yumin, >>> >>> I reviewed mostly the native code. Below are my comments: >>> >>> 1) classListParser.hpp >>> >>> 71?? bool??????????????? _lambda_format; >>> >>> The above name is too generic. How about _lambda_form or _is_lambda_form? >>> If you rename the above, please also rename the function which returns the above bool. >>> >>> 2) jvm.cpp >>> >>> 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line)) >>> >>> ignore -> ignored >>> >>> 3) jvm.hpp >>> >>> 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); >>> >>> Same comment as for jvm.cpp >>> >>> 4) metaspaceShared.cpp >>> >>> 2017?? size_t i = 0; >>> 2018?? while (i < size) { >>> 2019???? full_name[i++] = *start++; >>> 2020?? } >>> >>> Could the above be simplified to the following? >>> >>> ??? strncpy(full_name, start, size - 1); >>> >>> 2029?? char* class_name = get_full_class_name(path_name); >>> >>> Should os::free(class_name) be called before the function returns? >>> >>> 1870 static GrowableArray* lambda_list = NULL; >>> >>> The name lambda_list is a bit generic, how about lambda_form_list? >>> >>> 2112?????? lambda_list->append(parser.current_line()); >>> >>> Since parser.current_line() does a strdup, do those buffer need to be freed after its use? (i.e. after the call to regenerate_holder_classes()?) >>> >>> In MetaspaceShared::regenerate_holder_classes, before calling up to java, I think it's better to strip the prefix "@lambda-form-invoker" from the strings. So that the java InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need to handle it: >>> >>> ?143???????????????????? case "[LF_RESOLVE]": >>> ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: >>> >>> 5) DumpSymbolAndStringTable.java >>> >>> 37???? private static final String my_string = "DumpSymbolAndStringTable"; >>> >>> Unused variable? >>> >>> thanks, >>> >>> Calvin >>> >>> On 8/11/20 10:36 AM, Yumin Qi wrote: >>>> Forget to send to @core-lib-dev, the patch changed jdk code. >>>> >>>> >>>> Thanks >>>> >>>> Yumin >>>> >>>> >>>> >>>> -------- Forwarded Message -------- >>>> Subject:???? RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS >>>> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >>>> From:???? Yumin Qi >>>> To:???? hotspot-runtime-dev at openjdk.java.net >>>> >>>> >>>> >>>> Hi, Please reivew >>>> >>>> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >>>> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >>>> >>>> ? Summary: CDS does not archive pre-generated lambda form classes for method handles: >>>> >>>> [0.142s][info][class,load] java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: __JVM_LookupDefineClass__ >>>> >>>> The method handle resolution if not found in the holder class, a class with the method will be generated and loaded as vm internal created class and not archived like above. Those class names are mangled as combination of the class name and the class instance address. >>>> >>>> In this patch, collect those method holder class names and their associated methods' signatures when user specify DumpLoadedClassList, and record them in the log file in a format similar to the output format from traced by -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use another name for CDS: -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The line prefix also changed from "[LF_RESOLVE]" to "@lambda-invoke-handle".? At dump time, regenerate the holder class with those methods and replace the existing holder class and archived it. At runtime, the resolution of the holder class which contains those methods are loaded from CDS archive so avoid regenerating them again. The patch reorganized the code for Jlink plugin, so the new added InvokerBytecodeGeneratorHelper can be shared both by JLink plugin and CDS code. Also change made to the mangled generated class name at static dump, the class name combining the class name and a sequentially >>>> ordered number instead of the class instance address, which looks like a random. >>>> >>>> >>>> To use this feature, one can do the dump/run just like done for a static dump/run, so no extra steps required. >>>> >>>> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >>>> >>>> ????? DumpLoadedClassList will turn on -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >>>> >>>> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist -XX:SharedArchiveFile=my.jsa JavaApp >>>> >>>> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >>>> >>>> ? The performance on javac HelloWorld.java (javac-benchmark), no patch vs patch: >>>> >>>> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ---- 391.720 382.990 ( -8.730)????? ---- >>>> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ---- 391.030 381.480 ( -9.550)????? ----- >>>> ?? 3:?? 2694664066? 2636523114 (-58140952)????? ----- 390.610 382.550 ( -8.060)????? ---- >>>> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ---- 390.700 383.390 ( -7.310)????? --- >>>> ?? 5:?? 2691072338? 2633016687 (-58055651)????? ----- 388.990 382.360 ( -6.630)????? --- >>>> ?? 6:?? 2684448174? 2644191854 (-40256320)????? --- 389.450 382.990 ( -6.460)????? --- >>>> ?? 7:?? 2694921227? 2630505090 (-64416137)????? ----- 389.300 383.160 ( -6.140)????? --- >>>> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ---- 388.370 381.060 ( -7.310)????? --- >>>> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ---- 389.560 381.100 ( -8.460)????? ---- >>>> ? 10:?? 2689162942? 2635658943 (-53503999)????? ---- 389.690 379.110 (-10.580)????? ----- >>>> ============================================================ >>>> ??????? 2689866989? 2637396210 (-52470778)????? ---- 389.941 382.017 ( -7.924)????? ---- >>>> >>>> instr delta =??? -52470778??? -1.9507% >>>> >>>> time? delta =?????? -7.924 ms -2.0321% >>>> >>>> ? It will show much more improvement if compare with the default archive: >>>> >>>> ?????????? ? ? ? no patch??????????????????????? patch >>>> >>>> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >>>> >>>> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >>>> >>>> >>>> ? Tests: >>>> >>>> 1)jtreg on jdk/tools and hotspot/runtime local. >>>> >>>> 2) mach5 ter1,2 >>>> >>>> >>>> Thanks >>>> >>>> Yumin >>>> >>>> ? ` >> > From david.holmes at oracle.com Mon Aug 17 21:53:54 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Aug 2020 07:53:54 +1000 Subject: RFR 8251414: Add test that invokeinterface of a protected method in java.lang.Object throws NoSuchMethodError In-Reply-To: <97328605-aa99-51db-25eb-e1465919b8db@oracle.com> References: <97328605-aa99-51db-25eb-e1465919b8db@oracle.com> Message-ID: <6173e896-dd66-a8ba-f524-02e9e350b654@oracle.com> Hi Harold, On 18/08/2020 12:26 am, Harold Seigel wrote: > Hi David, > > This change got pushed before I saw your comments.? Let me know if I > should enter a new RFE to add the comments that you request below. No that's okay. But can you tell me why it needed to use jasm and reflection? :) Thanks, David > Thanks, Harold > > On 8/16/2020 10:06 PM, David Holmes wrote: >> Hi Harold, >> >> On 13/08/2020 4:14 am, Harold Seigel wrote: >>> Hi, >>> >>> Please review this change to add test cases to test >>> hotspot/jtreg/runtime/8026394/InterfaceObjectTest.java to check for >>> NoSuchMethodError exceptions when calling Object protected methods >>> clone() and finalize() using invokeinterface. >>> >>> Open Webrev: >>> http://cr.openjdk.java.net/~hseigel/bug_8251414/webrev/index.html >> >> Can you add a comment explaining why we need to use jasm and >> reflection in this new case. >> >> Thanks, >> David >> >>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251414 >>> >>> The fix was tested by running the modified test on multiple >>> platforms, including Mac OS, Windows, and Linux. >>> >>> Thanks, Harold From martin.doerr at sap.com Mon Aug 17 21:57:06 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 17 Aug 2020 21:57:06 +0000 Subject: FW: [11u] RFR(S): 8241234: Unify monitor enter/exit runtime entries. In-Reply-To: References: Message-ID: Forwarding to hotspot-runtime. Hi, I'd like to backport https://bugs.openjdk.java.net/browse/JDK-8241234 to JDK11u. Original JDK15 patch (https://hg.openjdk.java.net/jdk/jdk/rev/87c506c8be63) doesn't fit to JDK11u because the locking code has been reworked by https://bugs.openjdk.java.net/browse/JDK-8229844 As mentioned by Vladimir, there's already a GraalVM version available which consists of 2 patches (original + addon) and which can be applied: https://github.com/graalvm/labs-openjdk-11/commit/6c162cb15262e6aa77e36eb3a268320ef0a206a4 https://github.com/graalvm/labs-openjdk-11/commit/6a28a618cdbe595f9a3993e0eb63c01ccae1a528 Only JVMCI part from GraalVM doesn't apply automatically. The version of this file from JDK15 is very simple and fits perfectly. Please review the JDK11u backport webrev: http://cr.openjdk.java.net/~mdoerr/8241234_monitorenterexit_11u/webrev.00/ Thanks and best regards, Martin From david.holmes at oracle.com Tue Aug 18 01:42:45 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Aug 2020 11:42:45 +1000 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: <0facc9ee-eb47-a40c-4064-75d648b48eab@oss.nttdata.com> References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <37c30cea-c4f7-f329-c8dd-4e719cad608e@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> <08584797-0841-945e-ec6d-7050a12326b4@oracle.com> <0facc9ee-eb47-a40c-4064-75d648b48eab@oss.nttdata.com> Message-ID: <145b716c-54ee-b32a-d008-a4ca3e7359db@oracle.com> Hi Yasumasa, On 14/08/2020 12:36 pm, Yasumasa Suenaga wrote: > Hi David, Matthias, Martin, > > I uploaded new webrev. Could you review again? > > ? http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.02/ > > It generates runtime stub for hypervisor detection, and it can > distinguish between Hyper-V host (root partition) and guest. And also it > works on 32bit platforms. > > I measured startup performance (java --version) with Measure-Command on > PowerShell, this change is faster than current implementation. > > TotalMilliseconds : 57.8196 > TotalMilliseconds : 66.8379 > TotalMilliseconds : 64.7693 > TotalMilliseconds : 55.6546 > TotalMilliseconds : 63.848 > average : 61.78588 This seems good to me, but again I can't review the details. My main concern is how you are testing this across a range of virtualized hosts? But if it fixes your Hyper-V issue and doesn't break anything else then I am fine with it. I will leave the technical review to Martin and Matthias. FYI I'm heading out on a weeks vacation after today so won't be able to follow up further. Thanks, David > This change has passed tests on submit repo > (mach5-one-ysuenaga-JDK-8250598-20200814-0119-13424118), and also it > works fine on following environments: > > ? - Hyper-V host (Windows 10 x64) > ? - Hyper-V guest (Windows 10 x64) > ? - Hyper-V guest (Linux x64) > ? - Hyper-V guest (32bit JDK on Linux x64) > > > Thanks, > > Yasumasa > > > On 2020/08/13 20:39, David Holmes wrote: >> On 13/08/2020 5:39 pm, Yasumasa Suenaga wrote: >>> Hi Matthias, David, >>> >>> On 2020/08/13 15:52, Baesken, Matthias wrote: >>>> >>>>> >>>>> Should we make the change to determine just before it is needed >>>>> (e.g. VM.info or hs_err log) at first? >>>>> It is out of scope of this change, so I want to work as another >>>>> issue if it is needed. >>>>> >>>> >>>> Hi , >>>> I think we better do not call into WMI , when? writing an hs_err >>>> file?? . >>> >>> I found out from Hypervisor Top Level Functional Specification [1] >>> section 2.4.8. >>> That says CPUID leaf 0x40000007 is available to the root partition only. >>> >>> I changed VM_Version::is_in_VM() as below, and it works fine. >>> (I use __cpuid intrinsic in here because this code would be compiled >>> by cl.exe only.) >>> >>> ``` >>> #include >>> >>> #define EAX 0 >>> #define EBX 1 >>> #define ECX 2 >>> #define EDX 3 >>> >>> bool VM_Version::is_in_VM() { >>> ?? int regs[4]; >>> ?? __cpuid(regs, 0x40000007); >>> >>> ?? // CPUID leaf 0x40000007 is available to the root partition only. >>> ?? return (regs[EAX] == 0x0) && (regs[EBX] == 0x0) && (regs[ECX] == >>> 0x0) && (regs[EDX] == 0x0); >>> } >>> ``` >>> >>> Also startup performance is better than WMI call. >>> >>> TotalMilliseconds : 66.7863 >>> TotalMilliseconds : 56.8123 >>> TotalMilliseconds : 57.0015 >>> TotalMilliseconds : 62.4755 >>> TotalMilliseconds : 67.7632 >>> average : 62.16776 >>> >>> If you are ok this change, I will update webrev, and will send review >>> request. >>> (new webrev will include both loop for CPUID leaf and renaming is_in_VM) >> >> If it works and addresses the startup hit then I'm fine with it - but >> still can't actually review the code. >> >> Thanks, >> David >> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> [1] >>> https://github.com/MicrosoftDocs/Virtualization-Documentation/raw/master/tlfs/Hypervisor%20Top%20Level%20Functional%20Specification%20v6.0b.pdf >>> >>> >>> >>>> Best regards, Matthias >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: Yasumasa Suenaga >>>> Sent: Donnerstag, 13. August 2020 06:15 >>>> To: David Holmes ; Baesken, Matthias >>>> ; hotspot-runtime-dev at openjdk.java.net; >>>> build-dev at openjdk.java.net >>>> Cc: Doerr, Martin >>>> Subject: Re: PING: RFR: 8250598: Hyper-V is detected in spite of >>>> running on host OS >>>> >>>> On 2020/08/13 11:54, David Holmes wrote: >>>>> On 13/08/2020 11:12 am, Yasumasa Suenaga wrote: >>>>>> Hi Matthias, David, >>>>>> >>>>>> I measured startup benchmarks with `Measure-Command >>>>>> {.\jdk\build\windows-x86_64-server-release\images\jdk\bin\java.exe >>>>>> --version}` on PowerShell. >>>>>> >>>>>> * PC: Ryzen 3 3300X, 16GB memory >>>>>> * OS: Windows 10 x64 (May 2020 Update) >>>>>> * Java: jdk/jdk revision 60537 >>>>>> ????? (Compiled by VS 2019 (16.7.0)) >>>>>> >>>>>> * without patch >>>>>> TotalMilliseconds : 70.2124 >>>>>> TotalMilliseconds : 64.4465 >>>>>> TotalMilliseconds : 59.0854 >>>>>> TotalMilliseconds : 68.0255 >>>>>> TotalMilliseconds : 72.6467 >>>>>> average : 66.8833 >>>>>> >>>>>> * with webrev.01 >>>>>> TotalMilliseconds : 81.7185 >>>>>> TotalMilliseconds : 68.539 >>>>>> TotalMilliseconds : 85.7226 >>>>>> TotalMilliseconds : 72.6584 >>>>>> TotalMilliseconds : 75.6091 >>>>>> average : 76.84952 >>>>>> >>>>>> Overhead of WMI seems to be +10ms in this case. >>>>> >>>>> Which is nearly 15%! Sorry but I just know Claes will be very >>>>> unhappy if this were to go in as-is. >>>> >>>> Should we make the change to determine just before it is needed >>>> (e.g. VM.info or hs_err log) at first? >>>> It is out of scope of this change, so I want to work as another >>>> issue if it is needed. >>>> >>>> >>>> Yasumasa >>>> >>>> >>>>> David >>>>> ----- >>>>> >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2020/08/13 0:05, Baesken, Matthias wrote: >>>>>>>> I understand that if the process runs on Xen on other hypervisor >>>>>>>> (e.g. KVM), information for Xen would be set between 0x40000100 >>>>>>>> and 0x40010000. >>>>>>>> Ok, I will not remove the loop in new webrev, and will add >>>>>>>> comment about it. >>>>>>> >>>>>>> Hi Yasumasa? , thanks ! >>>>>>> >>>>>>> Regarding the WMI overhead , if you could? get some more info on >>>>>>> this it would be better to judge if it is perfectly okay or >>>>>>> concerning . >>>>>>> >>>>>>> (I remember that I looked into it a couple of years ago , but >>>>>>> decided not to use it in early VM startup ;? but have to confess >>>>>>> this was just based on a "gut feeling", >>>>>>> ?? No micro benchmarks? ) >>>>>>> >>>>>>> Best regards, Matthias >>>>>>> >>>>>>> From sundararajan.athijegannathan at oracle.com Tue Aug 18 02:25:21 2020 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Tue, 18 Aug 2020 07:55:21 +0530 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> Message-ID: Not a full review of fresh changes. But couple of comments: * src/hotspot/share/memory/lambdaFormInvokers.cpp and src/hotspot/share/memory/lambdaFormInvokers.hpp miss "Classpath exception" clause in the copyright header * I had suggested a change GenerateJLIClassesPlugin.java in last round of review. That's still applicable. https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068160.html -Sundar On 18/08/20 1:07 am, Yumin Qi wrote: > Hi, Ioi > > ? Thanks for review/suggestion. I have updated the webrev at the > following link: > > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ > > ? Following changes done in this patch: > > ? 1) move regenerating holder classes into new added file: > lambdaFormInvokers.[ch]pp > > ? "@lambda-form-invoker" tag only observed at jvm side. > > ? 2) remove InvokerBytecodeGeneratorHelper.java, move related > functions to existing class, GenerateJLIClassesHelper.java > > ? 3) add tracing for SPECIES_RESOLVE, so archive more classes. > > ? 4) remove InvokerGenerateBytesException.java, using RuntimeException > instead. > > ? 5) Changed make file to exclude the new added lamdaFormInvokers.cpp > from the non-cds building list. > > ? 6) There is a bug in previous patch, jobject (typeArrayOop) should > not be used directly for loading class since GC will move the java > object. Fixed by copying the bytes from handle to C heap. > > ? 7) Tested with tier1,2 > > > Thanks > > Yumin > > > > On 8/15/20 6:27 PM, Ioi Lam wrote: >> On 8/15/20 6:19 PM, Ioi Lam wrote: >>> To better capture what we're trying to do in this RFE, I've changed >>> the RFE title (and the subject of this email thread) to >>> >>> https://bugs.openjdk.java.net/browse/JDK-8247536 >>> Support for pre-generated java.lang.invoke classes in CDS static >>> archive >>> >>> I also update the RFE Description to give an overview of the problem >>> and the solution. >>> >>> The design document is also updated to reflect Yumin's latest >>> implementation >>> >> >> Oops, sent too quickly. The design doc's title is also updated: >> >> https://wiki.openjdk.java.net/display/HotSpot/Support+for+pre-generated+java.lang.invoke+classes+in+CDS+archive >> >> >>> Thanks >>> - Ioi >>> >>> On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: >>>> Hi Yumin, >>>> >>>> I reviewed mostly the native code. Below are my comments: >>>> >>>> 1) classListParser.hpp >>>> >>>> 71?? bool??????????????? _lambda_format; >>>> >>>> The above name is too generic. How about _lambda_form or >>>> _is_lambda_form? >>>> If you rename the above, please also rename the function which >>>> returns the above bool. >>>> >>>> 2) jvm.cpp >>>> >>>> 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass >>>> ignore, jstring line)) >>>> >>>> ignore -> ignored >>>> >>>> 3) jvm.hpp >>>> >>>> 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); >>>> >>>> Same comment as for jvm.cpp >>>> >>>> 4) metaspaceShared.cpp >>>> >>>> 2017?? size_t i = 0; >>>> 2018?? while (i < size) { >>>> 2019???? full_name[i++] = *start++; >>>> 2020?? } >>>> >>>> Could the above be simplified to the following? >>>> >>>> ??? strncpy(full_name, start, size - 1); >>>> >>>> 2029?? char* class_name = get_full_class_name(path_name); >>>> >>>> Should os::free(class_name) be called before the function returns? >>>> >>>> 1870 static GrowableArray* lambda_list = NULL; >>>> >>>> The name lambda_list is a bit generic, how about lambda_form_list? >>>> >>>> 2112?????? lambda_list->append(parser.current_line()); >>>> >>>> Since parser.current_line() does a strdup, do those buffer need to >>>> be freed after its use? (i.e. after the call to >>>> regenerate_holder_classes()?) >>>> >>>> In MetaspaceShared::regenerate_holder_classes, before calling up to >>>> java, I think it's better to strip the prefix >>>> "@lambda-form-invoker" from the strings. So that the java >>>> InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need >>>> to handle it: >>>> >>>> ?143???????????????????? case "[LF_RESOLVE]": >>>> ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: >>>> >>>> 5) DumpSymbolAndStringTable.java >>>> >>>> 37???? private static final String my_string = >>>> "DumpSymbolAndStringTable"; >>>> >>>> Unused variable? >>>> >>>> thanks, >>>> >>>> Calvin >>>> >>>> On 8/11/20 10:36 AM, Yumin Qi wrote: >>>>> Forget to send to @core-lib-dev, the patch changed jdk code. >>>>> >>>>> >>>>> Thanks >>>>> >>>>> Yumin >>>>> >>>>> >>>>> >>>>> -------- Forwarded Message -------- >>>>> Subject:???? RFR: 8247536: Support pre-generated MethodHandle >>>>> lambda forms in CDS >>>>> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >>>>> From:???? Yumin Qi >>>>> To:???? hotspot-runtime-dev at openjdk.java.net >>>>> >>>>> >>>>> >>>>> Hi, Please reivew >>>>> >>>>> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >>>>> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >>>>> >>>>> ? Summary: CDS does not archive pre-generated lambda form classes >>>>> for method handles: >>>>> >>>>> [0.142s][info][class,load] >>>>> java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: >>>>> __JVM_LookupDefineClass__ >>>>> >>>>> The method handle resolution if not found in the holder class, a >>>>> class with the method will be generated and loaded as vm internal >>>>> created class and not archived like above. Those class names are >>>>> mangled as combination of the class name and the class instance >>>>> address. >>>>> >>>>> In this patch, collect those method holder class names and their >>>>> associated methods' signatures when user specify >>>>> DumpLoadedClassList, and record them in the log file in a format >>>>> similar to the output format from traced by >>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use >>>>> another name for CDS: >>>>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. >>>>> The line prefix also changed from "[LF_RESOLVE]" to >>>>> "@lambda-invoke-handle".? At dump time, regenerate the holder >>>>> class with those methods and replace the existing holder class and >>>>> archived it. At runtime, the resolution of the holder class which >>>>> contains those methods are loaded from CDS archive so avoid >>>>> regenerating them again. The patch reorganized the code for Jlink >>>>> plugin, so the new added InvokerBytecodeGeneratorHelper can be >>>>> shared both by JLink plugin and CDS code. Also change made to the >>>>> mangled generated class name at static dump, the class name >>>>> combining the class name and a sequentially ordered number instead >>>>> of the class instance address, which looks like a random. >>>>> >>>>> >>>>> To use this feature, one can do the dump/run just like done for a >>>>> static dump/run, so no extra steps required. >>>>> >>>>> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >>>>> >>>>> ????? DumpLoadedClassList will turn on >>>>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >>>>> >>>>> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist >>>>> -XX:SharedArchiveFile=my.jsa JavaApp >>>>> >>>>> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >>>>> >>>>> ? The performance on javac HelloWorld.java (javac-benchmark), no >>>>> patch vs patch: >>>>> >>>>> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ---- 391.720 >>>>> 382.990 ( -8.730)????? ---- >>>>> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ---- 391.030 >>>>> 381.480 ( -9.550)????? ----- >>>>> ?? 3:?? 2694664066? 2636523114 (-58140952)????? ----- 390.610 >>>>> 382.550 ( -8.060)????? ---- >>>>> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ---- 390.700 >>>>> 383.390 ( -7.310)????? --- >>>>> ?? 5:?? 2691072338? 2633016687 (-58055651)????? ----- 388.990 >>>>> 382.360 ( -6.630)????? --- >>>>> ?? 6:?? 2684448174? 2644191854 (-40256320)????? --- 389.450 >>>>> 382.990 ( -6.460)????? --- >>>>> ?? 7:?? 2694921227? 2630505090 (-64416137)????? ----- 389.300 >>>>> 383.160 ( -6.140)????? --- >>>>> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ---- 388.370 >>>>> 381.060 ( -7.310)????? --- >>>>> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ---- 389.560 >>>>> 381.100 ( -8.460)????? ---- >>>>> ? 10:?? 2689162942? 2635658943 (-53503999)????? ---- 389.690 >>>>> 379.110 (-10.580)????? ----- >>>>> ============================================================ >>>>> ??????? 2689866989? 2637396210 (-52470778)????? ---- 389.941 >>>>> 382.017 ( -7.924)????? ---- >>>>> >>>>> instr delta =??? -52470778??? -1.9507% >>>>> >>>>> time? delta =?????? -7.924 ms -2.0321% >>>>> >>>>> ? It will show much more improvement if compare with the default >>>>> archive: >>>>> >>>>> ?????????? ? ? ? no patch??????????????????????? patch >>>>> >>>>> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >>>>> >>>>> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >>>>> >>>>> >>>>> ? Tests: >>>>> >>>>> 1)jtreg on jdk/tools and hotspot/runtime local. >>>>> >>>>> 2) mach5 ter1,2 >>>>> >>>>> >>>>> Thanks >>>>> >>>>> Yumin >>>>> >>>>> ? ` >>> >> From david.holmes at oracle.com Tue Aug 18 02:34:34 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Aug 2020 12:34:34 +1000 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> Message-ID: <2fc105ce-adaa-be64-fddf-c2f4b6293c54@oracle.com> Hi Sundar, On 18/08/2020 12:25 pm, sundararajan.athijegannathan at oracle.com wrote: > Not a full review of fresh changes. But couple of comments: > > * src/hotspot/share/memory/lambdaFormInvokers.cpp and > src/hotspot/share/memory/lambdaFormInvokers.hpp miss "Classpath > exception" clause in the copyright header Hotspot files do not use the Classpath exception. That is for .java sources that will be"linked against" by applications. Cheers, David > * I had suggested a change GenerateJLIClassesPlugin.java in last round > of review. That's still applicable. > > https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068160.html > > > -Sundar > > On 18/08/20 1:07 am, Yumin Qi wrote: >> Hi, Ioi >> >> ? Thanks for review/suggestion. I have updated the webrev at the >> following link: >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >> >> ? Following changes done in this patch: >> >> ? 1) move regenerating holder classes into new added file: >> lambdaFormInvokers.[ch]pp >> >> ? "@lambda-form-invoker" tag only observed at jvm side. >> >> ? 2) remove InvokerBytecodeGeneratorHelper.java, move related >> functions to existing class, GenerateJLIClassesHelper.java >> >> ? 3) add tracing for SPECIES_RESOLVE, so archive more classes. >> >> ? 4) remove InvokerGenerateBytesException.java, using RuntimeException >> instead. >> >> ? 5) Changed make file to exclude the new added lamdaFormInvokers.cpp >> from the non-cds building list. >> >> ? 6) There is a bug in previous patch, jobject (typeArrayOop) should >> not be used directly for loading class since GC will move the java >> object. Fixed by copying the bytes from handle to C heap. >> >> ? 7) Tested with tier1,2 >> >> >> Thanks >> >> Yumin >> >> >> >> On 8/15/20 6:27 PM, Ioi Lam wrote: >>> On 8/15/20 6:19 PM, Ioi Lam wrote: >>>> To better capture what we're trying to do in this RFE, I've changed >>>> the RFE title (and the subject of this email thread) to >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8247536 >>>> Support for pre-generated java.lang.invoke classes in CDS static >>>> archive >>>> >>>> I also update the RFE Description to give an overview of the problem >>>> and the solution. >>>> >>>> The design document is also updated to reflect Yumin's latest >>>> implementation >>>> >>> >>> Oops, sent too quickly. The design doc's title is also updated: >>> >>> https://wiki.openjdk.java.net/display/HotSpot/Support+for+pre-generated+java.lang.invoke+classes+in+CDS+archive >>> >>> >>>> Thanks >>>> - Ioi >>>> >>>> On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: >>>>> Hi Yumin, >>>>> >>>>> I reviewed mostly the native code. Below are my comments: >>>>> >>>>> 1) classListParser.hpp >>>>> >>>>> 71?? bool??????????????? _lambda_format; >>>>> >>>>> The above name is too generic. How about _lambda_form or >>>>> _is_lambda_form? >>>>> If you rename the above, please also rename the function which >>>>> returns the above bool. >>>>> >>>>> 2) jvm.cpp >>>>> >>>>> 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass >>>>> ignore, jstring line)) >>>>> >>>>> ignore -> ignored >>>>> >>>>> 3) jvm.hpp >>>>> >>>>> 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); >>>>> >>>>> Same comment as for jvm.cpp >>>>> >>>>> 4) metaspaceShared.cpp >>>>> >>>>> 2017?? size_t i = 0; >>>>> 2018?? while (i < size) { >>>>> 2019???? full_name[i++] = *start++; >>>>> 2020?? } >>>>> >>>>> Could the above be simplified to the following? >>>>> >>>>> ??? strncpy(full_name, start, size - 1); >>>>> >>>>> 2029?? char* class_name = get_full_class_name(path_name); >>>>> >>>>> Should os::free(class_name) be called before the function returns? >>>>> >>>>> 1870 static GrowableArray* lambda_list = NULL; >>>>> >>>>> The name lambda_list is a bit generic, how about lambda_form_list? >>>>> >>>>> 2112?????? lambda_list->append(parser.current_line()); >>>>> >>>>> Since parser.current_line() does a strdup, do those buffer need to >>>>> be freed after its use? (i.e. after the call to >>>>> regenerate_holder_classes()?) >>>>> >>>>> In MetaspaceShared::regenerate_holder_classes, before calling up to >>>>> java, I think it's better to strip the prefix >>>>> "@lambda-form-invoker" from the strings. So that the java >>>>> InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need >>>>> to handle it: >>>>> >>>>> ?143???????????????????? case "[LF_RESOLVE]": >>>>> ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: >>>>> >>>>> 5) DumpSymbolAndStringTable.java >>>>> >>>>> 37???? private static final String my_string = >>>>> "DumpSymbolAndStringTable"; >>>>> >>>>> Unused variable? >>>>> >>>>> thanks, >>>>> >>>>> Calvin >>>>> >>>>> On 8/11/20 10:36 AM, Yumin Qi wrote: >>>>>> Forget to send to @core-lib-dev, the patch changed jdk code. >>>>>> >>>>>> >>>>>> Thanks >>>>>> >>>>>> Yumin >>>>>> >>>>>> >>>>>> >>>>>> -------- Forwarded Message -------- >>>>>> Subject:???? RFR: 8247536: Support pre-generated MethodHandle >>>>>> lambda forms in CDS >>>>>> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >>>>>> From:???? Yumin Qi >>>>>> To:???? hotspot-runtime-dev at openjdk.java.net >>>>>> >>>>>> >>>>>> >>>>>> Hi, Please reivew >>>>>> >>>>>> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >>>>>> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >>>>>> >>>>>> ? Summary: CDS does not archive pre-generated lambda form classes >>>>>> for method handles: >>>>>> >>>>>> [0.142s][info][class,load] >>>>>> java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: >>>>>> __JVM_LookupDefineClass__ >>>>>> >>>>>> The method handle resolution if not found in the holder class, a >>>>>> class with the method will be generated and loaded as vm internal >>>>>> created class and not archived like above. Those class names are >>>>>> mangled as combination of the class name and the class instance >>>>>> address. >>>>>> >>>>>> In this patch, collect those method holder class names and their >>>>>> associated methods' signatures when user specify >>>>>> DumpLoadedClassList, and record them in the log file in a format >>>>>> similar to the output format from traced by >>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use >>>>>> another name for CDS: >>>>>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. >>>>>> The line prefix also changed from "[LF_RESOLVE]" to >>>>>> "@lambda-invoke-handle".? At dump time, regenerate the holder >>>>>> class with those methods and replace the existing holder class and >>>>>> archived it. At runtime, the resolution of the holder class which >>>>>> contains those methods are loaded from CDS archive so avoid >>>>>> regenerating them again. The patch reorganized the code for Jlink >>>>>> plugin, so the new added InvokerBytecodeGeneratorHelper can be >>>>>> shared both by JLink plugin and CDS code. Also change made to the >>>>>> mangled generated class name at static dump, the class name >>>>>> combining the class name and a sequentially ordered number instead >>>>>> of the class instance address, which looks like a random. >>>>>> >>>>>> >>>>>> To use this feature, one can do the dump/run just like done for a >>>>>> static dump/run, so no extra steps required. >>>>>> >>>>>> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >>>>>> >>>>>> ????? DumpLoadedClassList will turn on >>>>>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >>>>>> >>>>>> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist >>>>>> -XX:SharedArchiveFile=my.jsa JavaApp >>>>>> >>>>>> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >>>>>> >>>>>> ? The performance on javac HelloWorld.java (javac-benchmark), no >>>>>> patch vs patch: >>>>>> >>>>>> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ---- 391.720 >>>>>> 382.990 ( -8.730)????? ---- >>>>>> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ---- 391.030 >>>>>> 381.480 ( -9.550)????? ----- >>>>>> ?? 3:?? 2694664066? 2636523114 (-58140952)????? ----- 390.610 >>>>>> 382.550 ( -8.060)????? ---- >>>>>> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ---- 390.700 >>>>>> 383.390 ( -7.310)????? --- >>>>>> ?? 5:?? 2691072338? 2633016687 (-58055651)????? ----- 388.990 >>>>>> 382.360 ( -6.630)????? --- >>>>>> ?? 6:?? 2684448174? 2644191854 (-40256320)????? --- 389.450 >>>>>> 382.990 ( -6.460)????? --- >>>>>> ?? 7:?? 2694921227? 2630505090 (-64416137)????? ----- 389.300 >>>>>> 383.160 ( -6.140)????? --- >>>>>> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ---- 388.370 >>>>>> 381.060 ( -7.310)????? --- >>>>>> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ---- 389.560 >>>>>> 381.100 ( -8.460)????? ---- >>>>>> ? 10:?? 2689162942? 2635658943 (-53503999)????? ---- 389.690 >>>>>> 379.110 (-10.580)????? ----- >>>>>> ============================================================ >>>>>> ??????? 2689866989? 2637396210 (-52470778)????? ---- 389.941 >>>>>> 382.017 ( -7.924)????? ---- >>>>>> >>>>>> instr delta =??? -52470778??? -1.9507% >>>>>> >>>>>> time? delta =?????? -7.924 ms -2.0321% >>>>>> >>>>>> ? It will show much more improvement if compare with the default >>>>>> archive: >>>>>> >>>>>> ?????????? ? ? ? no patch??????????????????????? patch >>>>>> >>>>>> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >>>>>> >>>>>> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >>>>>> >>>>>> >>>>>> ? Tests: >>>>>> >>>>>> 1)jtreg on jdk/tools and hotspot/runtime local. >>>>>> >>>>>> 2) mach5 ter1,2 >>>>>> >>>>>> >>>>>> Thanks >>>>>> >>>>>> Yumin >>>>>> >>>>>> ? ` >>>> >>> From sundararajan.athijegannathan at oracle.com Tue Aug 18 02:39:24 2020 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Tue, 18 Aug 2020 08:09:24 +0530 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <2fc105ce-adaa-be64-fddf-c2f4b6293c54@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <2fc105ce-adaa-be64-fddf-c2f4b6293c54@oracle.com> Message-ID: <9f1452b2-796a-1bf2-ab61-f0019d3192d3@oracle.com> Hi David. Thanks. -Sundar On 18/08/20 8:04 am, David Holmes wrote: > Hi Sundar, > > On 18/08/2020 12:25 pm, sundararajan.athijegannathan at oracle.com wrote: >> Not a full review of fresh changes. But couple of comments: >> >> * src/hotspot/share/memory/lambdaFormInvokers.cpp and >> src/hotspot/share/memory/lambdaFormInvokers.hpp miss "Classpath >> exception" clause in the copyright header > > Hotspot files do not use the Classpath exception. That is for .java > sources that will be"linked against" by applications. > > Cheers, > David > >> * I had suggested a change GenerateJLIClassesPlugin.java in last >> round of review. That's still applicable. >> >> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068160.html >> >> >> -Sundar >> >> On 18/08/20 1:07 am, Yumin Qi wrote: >>> Hi, Ioi >>> >>> ? Thanks for review/suggestion. I have updated the webrev at the >>> following link: >>> >>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>> >>> ? Following changes done in this patch: >>> >>> ? 1) move regenerating holder classes into new added file: >>> lambdaFormInvokers.[ch]pp >>> >>> ? "@lambda-form-invoker" tag only observed at jvm side. >>> >>> ? 2) remove InvokerBytecodeGeneratorHelper.java, move related >>> functions to existing class, GenerateJLIClassesHelper.java >>> >>> ? 3) add tracing for SPECIES_RESOLVE, so archive more classes. >>> >>> ? 4) remove InvokerGenerateBytesException.java, using >>> RuntimeException instead. >>> >>> ? 5) Changed make file to exclude the new added >>> lamdaFormInvokers.cpp from the non-cds building list. >>> >>> ? 6) There is a bug in previous patch, jobject (typeArrayOop) should >>> not be used directly for loading class since GC will move the java >>> object. Fixed by copying the bytes from handle to C heap. >>> >>> ? 7) Tested with tier1,2 >>> >>> >>> Thanks >>> >>> Yumin >>> >>> >>> >>> On 8/15/20 6:27 PM, Ioi Lam wrote: >>>> On 8/15/20 6:19 PM, Ioi Lam wrote: >>>>> To better capture what we're trying to do in this RFE, I've >>>>> changed the RFE title (and the subject of this email thread) to >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8247536 >>>>> Support for pre-generated java.lang.invoke classes in CDS static >>>>> archive >>>>> >>>>> I also update the RFE Description to give an overview of the >>>>> problem and the solution. >>>>> >>>>> The design document is also updated to reflect Yumin's latest >>>>> implementation >>>>> >>>> >>>> Oops, sent too quickly. The design doc's title is also updated: >>>> >>>> https://wiki.openjdk.java.net/display/HotSpot/Support+for+pre-generated+java.lang.invoke+classes+in+CDS+archive >>>> >>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: >>>>>> Hi Yumin, >>>>>> >>>>>> I reviewed mostly the native code. Below are my comments: >>>>>> >>>>>> 1) classListParser.hpp >>>>>> >>>>>> 71?? bool??????????????? _lambda_format; >>>>>> >>>>>> The above name is too generic. How about _lambda_form or >>>>>> _is_lambda_form? >>>>>> If you rename the above, please also rename the function which >>>>>> returns the above bool. >>>>>> >>>>>> 2) jvm.cpp >>>>>> >>>>>> 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass >>>>>> ignore, jstring line)) >>>>>> >>>>>> ignore -> ignored >>>>>> >>>>>> 3) jvm.hpp >>>>>> >>>>>> 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); >>>>>> >>>>>> Same comment as for jvm.cpp >>>>>> >>>>>> 4) metaspaceShared.cpp >>>>>> >>>>>> 2017?? size_t i = 0; >>>>>> 2018?? while (i < size) { >>>>>> 2019???? full_name[i++] = *start++; >>>>>> 2020?? } >>>>>> >>>>>> Could the above be simplified to the following? >>>>>> >>>>>> ??? strncpy(full_name, start, size - 1); >>>>>> >>>>>> 2029?? char* class_name = get_full_class_name(path_name); >>>>>> >>>>>> Should os::free(class_name) be called before the function returns? >>>>>> >>>>>> 1870 static GrowableArray* lambda_list = NULL; >>>>>> >>>>>> The name lambda_list is a bit generic, how about lambda_form_list? >>>>>> >>>>>> 2112 lambda_list->append(parser.current_line()); >>>>>> >>>>>> Since parser.current_line() does a strdup, do those buffer need >>>>>> to be freed after its use? (i.e. after the call to >>>>>> regenerate_holder_classes()?) >>>>>> >>>>>> In MetaspaceShared::regenerate_holder_classes, before calling up >>>>>> to java, I think it's better to strip the prefix >>>>>> "@lambda-form-invoker" from the strings. So that the java >>>>>> InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't >>>>>> need to handle it: >>>>>> >>>>>> ?143???????????????????? case "[LF_RESOLVE]": >>>>>> ?144???????????????????? case >>>>>> InvokerBytecodeGenerator.CDS_LOG_PREFIX: >>>>>> >>>>>> 5) DumpSymbolAndStringTable.java >>>>>> >>>>>> 37???? private static final String my_string = >>>>>> "DumpSymbolAndStringTable"; >>>>>> >>>>>> Unused variable? >>>>>> >>>>>> thanks, >>>>>> >>>>>> Calvin >>>>>> >>>>>> On 8/11/20 10:36 AM, Yumin Qi wrote: >>>>>>> Forget to send to @core-lib-dev, the patch changed jdk code. >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Yumin >>>>>>> >>>>>>> >>>>>>> >>>>>>> -------- Forwarded Message -------- >>>>>>> Subject:???? RFR: 8247536: Support pre-generated MethodHandle >>>>>>> lambda forms in CDS >>>>>>> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >>>>>>> From:???? Yumin Qi >>>>>>> To:???? hotspot-runtime-dev at openjdk.java.net >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hi, Please reivew >>>>>>> >>>>>>> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >>>>>>> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >>>>>>> >>>>>>> ? Summary: CDS does not archive pre-generated lambda form >>>>>>> classes for method handles: >>>>>>> >>>>>>> [0.142s][info][class,load] >>>>>>> java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: >>>>>>> __JVM_LookupDefineClass__ >>>>>>> >>>>>>> The method handle resolution if not found in the holder class, a >>>>>>> class with the method will be generated and loaded as vm >>>>>>> internal created class and not archived like above. Those class >>>>>>> names are mangled as combination of the class name and the class >>>>>>> instance address. >>>>>>> >>>>>>> In this patch, collect those method holder class names and their >>>>>>> associated methods' signatures when user specify >>>>>>> DumpLoadedClassList, and record them in the log file in a format >>>>>>> similar to the output format from traced by >>>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use >>>>>>> another name for CDS: >>>>>>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. >>>>>>> The line prefix also changed from "[LF_RESOLVE]" to >>>>>>> "@lambda-invoke-handle".? At dump time, regenerate the holder >>>>>>> class with those methods and replace the existing holder class >>>>>>> and archived it. At runtime, the resolution of the holder class >>>>>>> which contains those methods are loaded from CDS archive so >>>>>>> avoid regenerating them again. The patch reorganized the code >>>>>>> for Jlink plugin, so the new added >>>>>>> InvokerBytecodeGeneratorHelper can be shared both by JLink >>>>>>> plugin and CDS code. Also change made to the mangled generated >>>>>>> class name at static dump, the class name combining the class >>>>>>> name and a sequentially ordered number instead of the class >>>>>>> instance address, which looks like a random. >>>>>>> >>>>>>> >>>>>>> To use this feature, one can do the dump/run just like done for >>>>>>> a static dump/run, so no extra steps required. >>>>>>> >>>>>>> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >>>>>>> >>>>>>> ????? DumpLoadedClassList will turn on >>>>>>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >>>>>>> >>>>>>> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist >>>>>>> -XX:SharedArchiveFile=my.jsa JavaApp >>>>>>> >>>>>>> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >>>>>>> >>>>>>> ? The performance on javac HelloWorld.java (javac-benchmark), no >>>>>>> patch vs patch: >>>>>>> >>>>>>> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ---- 391.720 >>>>>>> 382.990 ( -8.730)????? ---- >>>>>>> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ---- 391.030 >>>>>>> 381.480 ( -9.550)????? ----- >>>>>>> ?? 3:?? 2694664066? 2636523114 (-58140952)????? ----- 390.610 >>>>>>> 382.550 ( -8.060)????? ---- >>>>>>> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ---- 390.700 >>>>>>> 383.390 ( -7.310)????? --- >>>>>>> ?? 5:?? 2691072338? 2633016687 (-58055651)????? ----- 388.990 >>>>>>> 382.360 ( -6.630)????? --- >>>>>>> ?? 6:?? 2684448174? 2644191854 (-40256320)????? --- 389.450 >>>>>>> 382.990 ( -6.460)????? --- >>>>>>> ?? 7:?? 2694921227? 2630505090 (-64416137)????? ----- 389.300 >>>>>>> 383.160 ( -6.140)????? --- >>>>>>> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ---- 388.370 >>>>>>> 381.060 ( -7.310)????? --- >>>>>>> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ---- 389.560 >>>>>>> 381.100 ( -8.460)????? ---- >>>>>>> ? 10:?? 2689162942? 2635658943 (-53503999)????? ---- 389.690 >>>>>>> 379.110 (-10.580)????? ----- >>>>>>> ============================================================ >>>>>>> ??????? 2689866989? 2637396210 (-52470778)????? ---- 389.941 >>>>>>> 382.017 ( -7.924)????? ---- >>>>>>> >>>>>>> instr delta =??? -52470778??? -1.9507% >>>>>>> >>>>>>> time? delta =?????? -7.924 ms -2.0321% >>>>>>> >>>>>>> ? It will show much more improvement if compare with the default >>>>>>> archive: >>>>>>> >>>>>>> ?????????? ? ? ? no patch??????????????????????? patch >>>>>>> >>>>>>> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >>>>>>> >>>>>>> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >>>>>>> >>>>>>> >>>>>>> ? Tests: >>>>>>> >>>>>>> 1)jtreg on jdk/tools and hotspot/runtime local. >>>>>>> >>>>>>> 2) mach5 ter1,2 >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Yumin >>>>>>> >>>>>>> ? ` >>>>> >>>> From david.holmes at oracle.com Tue Aug 18 02:44:20 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Aug 2020 12:44:20 +1000 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> Message-ID: <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> Hi Kim, Not a full review - sorry. Have you tested what happens if a resource leak is introduced? The _warned variable was used, IIUC, to avoid hitting recursive errors during error reporting. Having flagged that I may not be around to see your response as I'm on vacation after today - sorry. Cheers, David On 17/08/2020 12:24 pm, Kim Barrett wrote: >> On Aug 15, 2020, at 8:31 AM, Kim Barrett wrote: >> >>> On Aug 14, 2020, at 10:29 PM, Kim Barrett wrote: >>> >>> Please review this simplification of the constructors for ResourceMark, >>> accomplished using delegating constructors. >>> >>> Also made the constructors "explicit" and the class non-copyable. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8251850 >>> >>> Webrev: >>> https://cr.openjdk.java.net/~kbarrett/8251850/open.00/ >>> >>> Testing: >>> mach5 tier1 >> >> I should have looked further along in resourceArea.hpp. All the >> changes I made to ResourceMark? DeoptResourceMark should have similar >> changes made to it. >> >> Well, I have a first round of comments from the ResourceMark changes. I can >> take those into account for DeoptResourceMark. > > This ended up engendering a fairly substantial revision. I've taken at > least some of Thomas's comments into account along the way. > > Changed ResourceMark and DeoptResourceMark to share ResourceMarkImpl, which > provides most of the implementation of both, eliminating the copy-paste-modify > "reuse" that used to be there. > > Moved all of the state capture and rollback support into ResourceArea. The > mark objects include a captured state object and pass that back to the > ResourceArea when doing rollbacks. > > Cleaned up the _nesting checking. > > Removed nonsensical ResourceArea::_warned. > > Maybe I should change the bug summary from > "Simplify ResourceMark constructors using delegation" > to something like > "Refactor ResourceMark and DeoptResourceMark for better code sharing" > > Note: ResourceArea's handling of UseMallocOnly probably doesn't work in an > optimize build. ("probably" because I haven't actually tested it.) The > allocation side of it is #ifdef ASSERT, because functions it calls from > Arena are debug-only. But the free side is not conditionalized, so could be > used in any non-product build when the option is enabled. But maybe it's not > really a problem because the free side won't find any work. But this > conditionalization inconsistency is odd. > > New webrevs: > full: https://cr.openjdk.java.net/~kbarrett/8251850/open.01/ > incr: https://cr.openjdk.java.net/~kbarrett/8251850/open.01.inc/ > > Testing: > mach5 tier1-3 > local hotspot:tier1 with -XX:+UseMallocOnly > local hotspot:tier1 with -XX:+ZapResourceArea > > Note that compiler/loopopts/TestDeepGraphVerifyIterativeGVN.java times out > when run with -XX:+UseMallocOnly, with or without these changes. > > From suenaga at oss.nttdata.com Tue Aug 18 03:02:47 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Tue, 18 Aug 2020 12:02:47 +0900 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: <145b716c-54ee-b32a-d008-a4ca3e7359db@oracle.com> References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> <08584797-0841-945e-ec6d-7050a12326b4@oracle.com> <0facc9ee-eb47-a40c-4064-75d648b48eab@oss.nttdata.com> <145b716c-54ee-b32a-d008-a4ca3e7359db@oracle.com> Message-ID: On 2020/08/18 10:42, David Holmes wrote: > Hi Yasumasa, > > On 14/08/2020 12:36 pm, Yasumasa Suenaga wrote: >> Hi David, Matthias, Martin, >> >> I uploaded new webrev. Could you review again? >> >> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.02/ >> >> It generates runtime stub for hypervisor detection, and it can distinguish between Hyper-V host (root partition) and guest. And also it works on 32bit platforms. >> >> I measured startup performance (java --version) with Measure-Command on PowerShell, this change is faster than current implementation. >> >> TotalMilliseconds : 57.8196 >> TotalMilliseconds : 66.8379 >> TotalMilliseconds : 64.7693 >> TotalMilliseconds : 55.6546 >> TotalMilliseconds : 63.848 >> average : 61.78588 > > This seems good to me, but again I can't review the details. My main concern is how you are testing this across a range of virtualized hosts? But if it fixes your Hyper-V issue and doesn't break anything else then I am fine with it. Thanks David! I don't have any other virtualized hosts excepting Hyper-V, so I cannot test any more. However detection logic is equivalent from current implementation, so I believe it does not break anything else if it works fine on Hyper-V. And also Matthias has queued this change to internal tests in SAP. I guess he will share the result next week. > I will leave the technical review to Martin and Matthias. > > FYI I'm heading out on a weeks vacation after today so won't be able to follow up further. Have a nice vacation! Thanks, Yasumasa > Thanks, > David > >> This change has passed tests on submit repo (mach5-one-ysuenaga-JDK-8250598-20200814-0119-13424118), and also it works fine on following environments: >> >> ?? - Hyper-V host (Windows 10 x64) >> ?? - Hyper-V guest (Windows 10 x64) >> ?? - Hyper-V guest (Linux x64) >> ?? - Hyper-V guest (32bit JDK on Linux x64) >> >> >> Thanks, >> >> Yasumasa >> >> >> On 2020/08/13 20:39, David Holmes wrote: >>> On 13/08/2020 5:39 pm, Yasumasa Suenaga wrote: >>>> Hi Matthias, David, >>>> >>>> On 2020/08/13 15:52, Baesken, Matthias wrote: >>>>> >>>>>> >>>>>> Should we make the change to determine just before it is needed (e.g. VM.info or hs_err log) at first? >>>>>> It is out of scope of this change, so I want to work as another issue if it is needed. >>>>>> >>>>> >>>>> Hi , >>>>> I think we better do not call into WMI , when? writing an hs_err file?? . >>>> >>>> I found out from Hypervisor Top Level Functional Specification [1] section 2.4.8. >>>> That says CPUID leaf 0x40000007 is available to the root partition only. >>>> >>>> I changed VM_Version::is_in_VM() as below, and it works fine. >>>> (I use __cpuid intrinsic in here because this code would be compiled by cl.exe only.) >>>> >>>> ``` >>>> #include >>>> >>>> #define EAX 0 >>>> #define EBX 1 >>>> #define ECX 2 >>>> #define EDX 3 >>>> >>>> bool VM_Version::is_in_VM() { >>>> ?? int regs[4]; >>>> ?? __cpuid(regs, 0x40000007); >>>> >>>> ?? // CPUID leaf 0x40000007 is available to the root partition only. >>>> ?? return (regs[EAX] == 0x0) && (regs[EBX] == 0x0) && (regs[ECX] == 0x0) && (regs[EDX] == 0x0); >>>> } >>>> ``` >>>> >>>> Also startup performance is better than WMI call. >>>> >>>> TotalMilliseconds : 66.7863 >>>> TotalMilliseconds : 56.8123 >>>> TotalMilliseconds : 57.0015 >>>> TotalMilliseconds : 62.4755 >>>> TotalMilliseconds : 67.7632 >>>> average : 62.16776 >>>> >>>> If you are ok this change, I will update webrev, and will send review request. >>>> (new webrev will include both loop for CPUID leaf and renaming is_in_VM) >>> >>> If it works and addresses the startup hit then I'm fine with it - but still can't actually review the code. >>> >>> Thanks, >>> David >>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> [1] https://github.com/MicrosoftDocs/Virtualization-Documentation/raw/master/tlfs/Hypervisor%20Top%20Level%20Functional%20Specification%20v6.0b.pdf >>>> >>>> >>>>> Best regards, Matthias >>>>> >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: Yasumasa Suenaga >>>>> Sent: Donnerstag, 13. August 2020 06:15 >>>>> To: David Holmes ; Baesken, Matthias ; hotspot-runtime-dev at openjdk.java.net; build-dev at openjdk.java.net >>>>> Cc: Doerr, Martin >>>>> Subject: Re: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS >>>>> >>>>> On 2020/08/13 11:54, David Holmes wrote: >>>>>> On 13/08/2020 11:12 am, Yasumasa Suenaga wrote: >>>>>>> Hi Matthias, David, >>>>>>> >>>>>>> I measured startup benchmarks with `Measure-Command {.\jdk\build\windows-x86_64-server-release\images\jdk\bin\java.exe --version}` on PowerShell. >>>>>>> >>>>>>> * PC: Ryzen 3 3300X, 16GB memory >>>>>>> * OS: Windows 10 x64 (May 2020 Update) >>>>>>> * Java: jdk/jdk revision 60537 >>>>>>> ????? (Compiled by VS 2019 (16.7.0)) >>>>>>> >>>>>>> * without patch >>>>>>> TotalMilliseconds : 70.2124 >>>>>>> TotalMilliseconds : 64.4465 >>>>>>> TotalMilliseconds : 59.0854 >>>>>>> TotalMilliseconds : 68.0255 >>>>>>> TotalMilliseconds : 72.6467 >>>>>>> average : 66.8833 >>>>>>> >>>>>>> * with webrev.01 >>>>>>> TotalMilliseconds : 81.7185 >>>>>>> TotalMilliseconds : 68.539 >>>>>>> TotalMilliseconds : 85.7226 >>>>>>> TotalMilliseconds : 72.6584 >>>>>>> TotalMilliseconds : 75.6091 >>>>>>> average : 76.84952 >>>>>>> >>>>>>> Overhead of WMI seems to be +10ms in this case. >>>>>> >>>>>> Which is nearly 15%! Sorry but I just know Claes will be very unhappy if this were to go in as-is. >>>>> >>>>> Should we make the change to determine just before it is needed (e.g. VM.info or hs_err log) at first? >>>>> It is out of scope of this change, so I want to work as another issue if it is needed. >>>>> >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2020/08/13 0:05, Baesken, Matthias wrote: >>>>>>>>> I understand that if the process runs on Xen on other hypervisor (e.g. KVM), information for Xen would be set between 0x40000100 and 0x40010000. >>>>>>>>> Ok, I will not remove the loop in new webrev, and will add comment about it. >>>>>>>> >>>>>>>> Hi Yasumasa? , thanks ! >>>>>>>> >>>>>>>> Regarding the WMI overhead , if you could? get some more info on this it would be better to judge if it is perfectly okay or concerning . >>>>>>>> >>>>>>>> (I remember that I looked into it a couple of years ago , but decided not to use it in early VM startup ;? but have to confess this was just based on a "gut feeling", >>>>>>>> ?? No micro benchmarks? ) >>>>>>>> >>>>>>>> Best regards, Matthias >>>>>>>> >>>>>>>> From david.holmes at oracle.com Tue Aug 18 05:15:02 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Aug 2020 15:15:02 +1000 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: <95b46d0f-8a81-10c6-78bb-a3abbd7cb955@oracle.com> References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> <95b46d0f-8a81-10c6-78bb-a3abbd7cb955@oracle.com> Message-ID: <55ae5763-476b-562c-c7cc-a913992d304c@oracle.com> Hi Gerard, On 18/08/2020 1:28 am, gerard ziemski wrote: > Thank you David and Igor for the feedback! > > I rewrote the test again, this time basing it off > test/hotspot/jtreg/runtime/Thread/* tests. > > For macOS and Linux we create threads until we hit a process limit > thread count. Windows, however, allows unlimited (?) number of threads, > though the process gradually slows down as the count grows, so I had to > add time limit and I set it to 5 sec. I chose 5 sec because the value is > well below jtreg time out (i.e. 120s), but still quite above how long my > laptop needs to hit the process thread count limit (i.e. my Mac laptop > needs about 1 sec to hit that limit). > > Otherwise there are no other hardwired values or assumptions (I hope). > What do you think of the test now? Not unreasonable :) It could still have tried for an upper limit of 10000 instead of using the time-limit, but it makes no real difference. A few nits/suggestions: Please add an @bug line to the test comment. 49 if ((this.index % 250) == 0) { Outside of the constructor it is poor style to use this.x to refer to fields methods (unless needed due to local variable shadowing). This applies throughout the test. 54 doWork(); Not sure why you factored things out the way you did - everything could go in run(). 61 setName(""+this.index); This could go in constructor, but I presume it adds something to the workload? Is the use of ""+index deliberate in that regard (generate string concatenation work), rather than using String.valueOf(index)? 65 assert(this.index == Integer.parseInt(getName())); We never run tests with Java asserts enabled so this is "dead" code. 87 // Windows path Windows path or system with very large ulimit? 92 } catch (OutOfMemoryError e) { I suggest checking this is actually the "unable to create native thread" OOME rather than a general Java OOME (if run with a small heap for example you might get OOME growing the ArrayList). 110 String cmd = System.getProperty("sun.java.command"); 111 if (cmd != null && !cmd.startsWith("com.sun.javatest.regtest.agent.MainWrapper")) { 112 // Exit with success in a non-JavaTest environment: 113 System.exit(0); 114 } I never noticed when this was used in the little set of runtime/Thread tests! I don't know what the rationale was but it is completely unnecessary as a normal return from main would be the equivalent of exit(0) - so the tests work fine without System.exit when run standalone. Please don't emulate this part of the existing tests. I'll file a RFE to remove it from the other tests. I won't be around for any further follow up for a week, so feel free to proceed once Igor is okay with things. Thanks, David ----- > bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 > open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev2 > testing: Mach5 hs-tier1,2,3,4,5 in progress... > > > > > On 8/7/20 2:41 PM, Igor Ignatyev wrote: >> >> >>> On Aug 6, 2020, at 12:33 AM, David Holmes >> > wrote: >>> >>> Basing the new test around the existing thread stress tests was not >>> unreasonable but I don't think we're actively adding to those tests. >>> I would check with Igor whether it is better to just add this under >>> runtime/thread rather than vmTestBase. >> >> Hi Gerard, >> >> David is right, we are trying to get rid of vmTestbase directory and >> move all tests into corresponding component directories cleaning them >> up at the same time. so runtime/thread directory sound like a much >> better location for a new test. >> >> I don't think you need to cary all this baggage of 'VM testbase >> keywords', 'converted from ...' and the rest, I'd suggest you just use >> the text from 'DESCRIPTION' as @summary. I'd also remove System.exit >> call and would just make thread009::run to return bool. it's also >> unclear to me if someone is ever going to change THREADS_EXPECTED , >> TIMEOUT, and DEBUG_MODE?values, I'd just hardcode them in the test and >> get rid of parsing. it seems that debug output won't clutter output, >> so you might want to always print them and remove DEBUG_MODE field. >> >> using `assert` in test code isn't a good idea as it will work only if >> someone runs test w/ `-ea` which not all people do. it's better to >> replace w/ `if (p) throw new Error(...)` ?or use jdk.test.lib.Asserts >> class. >> >> Cheers, >> -- Igor > From felix.yang at huawei.com Tue Aug 18 06:27:35 2020 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Tue, 18 Aug 2020 06:27:35 +0000 Subject: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic References: <654e8a2e-87a1-9c3d-4c58-36b4215bb0ad@redhat.com> <5350e745-87af-ce34-8711-2b652a38e323@redhat.com> <4d1f7179-2cad-620d-f833-949e8b39ee06@redhat.com> <7cedaced-9628-32b3-6d01-beae760fc964@redhat.com> <9e733fb0-5916-2342-7962-ee1ec2f6f076@redhat.com> <80b588a5-e07f-9d8e-1079-c14530570549@redhat.com> Message-ID: CCing to the mailing lists. Sorry for forgetting that. > -----Original Message----- > From: Yangfei (Felix) > Sent: Tuesday, August 18, 2020 2:17 PM > To: 'Andrew Haley' > Subject: RE: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic > > Hi, > > > -----Original Message----- > > From: Andrew Haley [mailto:aph at redhat.com] > > Sent: Monday, August 17, 2020 11:54 PM > > To: Yangfei (Felix) ; hotspot-runtime- > > dev at openjdk.java.net > > Cc: aarch64-port-dev at openjdk.java.net > > Subject: Re: RFR: 8165404: AArch64: Implement SHA512 > > accelerator/intrinsic > > > > On 17/08/2020 13:01, Yangfei (Felix) wrote: > > >> Looks good. If I were editing the Python script I'd also add > > >> random.seed(0) so that the generated code doesn't change every time. > > > > > Thanks for the quick reply. Do you mean we need any more change? > > > For the purpose for testing the instructions, it might not be a good > > > idea to add random.seed(0). This will limit the randomness of the > > > register numbers used for each run. > > > > Not really, because we paste the output into assembler.cpp so that we > > use the same values every time it runs. I don't think it helps anyone > > that we use different random offsets every time the test code is > regenerated. > > Make sense to me in that respect. > > I created a new issue and prepared a webrev for this: > Bug: https://bugs.openjdk.java.net/browse/JDK-8251885 > Webrev: http://cr.openjdk.java.net/~fyang/8251885/webrev.01 > > This added random.seed(0) to aarch64-asmtest.py and regenerate the code > as a baseline. > Passed the smoke test for assembler in assembler_aarch64.cpp with a > slowdebug build. > Is it good to go? > > Thanks, > Felix From richard.reingruber at sap.com Tue Aug 18 07:43:51 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Tue, 18 Aug 2020 07:43:51 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Goetz, I have collected the changes based on your feedback in a new webrev: Webrev.7: http://cr.openjdk.java.net/~rrich/webrevs/8227745/webrev.7/ Delta: http://cr.openjdk.java.net/~rrich/webrevs/8227745/webrev.7.inc/ Most of the changes are renamings, commenting, and reformatting. Besides that ... - I converted the native agent of the test IterateHeapWithEscapeAnalysisEnabled from C to C++, because this seems to be preferred by serviceability developers. I also re-indented the file, but excluded this from the delta webrev. - I had to adapt test/jdk/com/sun/jdi/EATests.java to the fact that background compilation (-Xbatch) cannot be reliably disabled for JVMCI compilers. E.g. the compile broker will compile in the background if JVMCI is not yet fully initialized. Therefore it is possible that test cases are executed before the main test method is compiled on the highest level and then the test case fails. The higher the system load the higher the probability for this to happen. In webrev.7 I skip the compilation level check if the vm is configured to use the JVMCI compiler. I also answered you inline below. Thanks, Richard. -----Original Message----- From: Lindenmaier, Goetz Sent: Donnerstag, 23. Juli 2020 16:20 To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi Richard, Thanks for your two further explanations in the other thread. That made the points clear to me. > > I was not that happy with the names saying not_global_escape > > and similar. I now agreed you have to use the terms of the escape > > analysis (NoEscape ArgEscape= throughout the runtime code. I'm still not happy with > > the 'not' in the term, I always try to expand the name to some > > sentence with a negated verb, but it makes no sense. > > For example, "has_not_global_escape_in_scope" expands to > > "Hasn't a global escape in its scope." in my thinking, which makes > > no sense. You probably mean > > "Has not-global escape in its scope." or "Has {ArgEscape|NoEscape} > > in its scope." > > > C2 is using the word "non" in this context, e.g., here > > alloc->is_non_escaping. > > There is also ConnectionGraph::not_global_escape() That talks about a single node that represents a single Object. An object has a single state wrt. ea. You use the term for safepoint which tracks a set of objects. Here, has_not_global_excape can mean 1. None of the several objects does escape globaly. 2. There is at least one object that escapes globaly. > > non obviously negates the adjective 'global', > > non-global or nonglobal even is a English term I find in the > > net. > > So what about "has_non_global_escape_in_scope?" > > And what about has_ea_local_in_scope? That's good. Please document somewhere that Ea_local == ArgEscape | NoEscape. That's what it is, right? > > Does jvmti specify that the same limits are used ...? > > ok on your side. > > I don't know and didn't find anything in a quick search. Ok, not your business. > > > jvmtiEnvBase.cpp ok > > jvmtiImpl.h|cpp ok > > jvmtiTagMap.cpp ok > > whitebox.cpp ok > > > deoptimization.cpp > > > line 177: Please break line > > line 246, 281: Please break line > > 1578, 1583, 1589, 1632, 1649, 1651 Break line > > > 1651: You use 'non'-terms, too: non-escaping :) > > I know :) At least here it is wrong I'd say. "...has to be a not escaping obj..." > sounds better > (hopefully not only to my german ears). I thought the term non-escpaing makes it quite clear. I just wanted to point out that using non above would be similar to the wording here. > > IterateHeapWithEscapeAnalysisEnabled.java > > > line 415: > > msg("wait until target thread has set testMethod_result"); > > while (testMethod_result == 0) { > > Thread.sleep(50); > > } > > Might the test run into timeouts at this place? > > The field is volatile, i.e. it will be reloaded > > in each iteration. But will dontinline_testMethod > > write it back to main memory in time? > > You mean, the test could hang in that loop for a couple of minutes? I don't > think so. There are cache coherence protocols in place which will invalidate > stale data very timely. Ok, anyways, it would only be a hanging test. > > Ok. I've removed quite a lot of the occurrances. > > > Also, I like full sentences in comments. > > Especially for me as foreign speaker, this makes > > things much more clear. I.e., I try to make it > > a real sentence with articles, capitalized and a > > dot at the end if there is a subject and a verb > > in first place. > > E.g., jvmtiEnvBase.cpp:1327 > > Are you referring to the following? > (from > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6/src/hots > pot/share/prims/jvmtiEnvBase.cpp.frames.html) > > 1326 > 1327 // If the frame is a compiled one, need to deoptimize it. > 1328 if (vf->is_compiled_frame()) { > > This line 1327 is preexisting. Sorry, wrong line number again. I think I meant 1333 // eagerly reallocate scalar replaced objects. But I must admit, the subject is missing. It's one of these imperative sentences where the subject is left out, which are used throughout documentation. Bad example, but still a correct sentence, so qualifies for punctuation? Best regards, Goetz. From kim.barrett at oracle.com Tue Aug 18 07:49:39 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 18 Aug 2020 03:49:39 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> Message-ID: > On Aug 17, 2020, at 10:44 PM, David Holmes wrote: > > Hi Kim, > > Not a full review - sorry. Have you tested what happens if a resource leak is introduced? The _warned variable was used, IIUC, to avoid hitting recursive errors during error reporting. OH! The recursive case hadn?t occurred to me! Poorly named and poorly commented variable from before the beginning of the mercurial age, so hard to track down the changeset that introduced it to see if that shed some light. Good thing we have some institutional memory :) I will do something about that, and then try to figure out how to trigger it. From kim.barrett at oracle.com Tue Aug 18 08:05:37 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 18 Aug 2020 04:05:37 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <1597683176006.22884@amazon.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <1597683176006.22884@amazon.com> Message-ID: > On Aug 17, 2020, at 12:52 PM, Liu, Xin wrote: > > Hi, Kim, > > You patch looks good to me. Thanks. > I have a relevant question about constructor. > > In opto, there's another candidate to use delegating constructor. > https://hg.openjdk.java.net/jdk/jdk/file/b6475f411b98/src/hotspot/share/opto/node.cpp#l361 > > It's pretty annoying when I attempt to add a new member variable to Node. > To well initialize it, I have to modify all different constructors. needless to say, it's error-prone because C++ can't promise you that all constructors are in a same file. > > I have two options here. > 1) use delegate constructor like you did here. > 2) use default initializers for member variables(c++11) > eg. http://cr.openjdk.java.net/~phh/8251464/webrev.00/src/hotspot/share/opto/node.hpp.udiff.html > > I am not sure if the second option is endorsed. Quote from hotspot coding style (https://hg.openjdk.java.net/jdk/jdk/raw-file/b6475f411b98/doc/hotspot-style.html) > "Initialize all variables and data structures to a known state. If a class has a constructor, initialize it there.? 2) default initializers for member variables is not (yet) an approved feature. That could change. I?m not sure it really helps here anyway though. But I think this particular case is more likely a candidate for a variadic template as the constructor, rather than constructor delegation. I?ve not worked through the details to see what that would look like, but I suspect it would be a lot simpler. From aph at redhat.com Tue Aug 18 09:34:00 2020 From: aph at redhat.com (Andrew Haley) Date: Tue, 18 Aug 2020 10:34:00 +0100 Subject: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic In-Reply-To: References: <654e8a2e-87a1-9c3d-4c58-36b4215bb0ad@redhat.com> <5350e745-87af-ce34-8711-2b652a38e323@redhat.com> <4d1f7179-2cad-620d-f833-949e8b39ee06@redhat.com> <7cedaced-9628-32b3-6d01-beae760fc964@redhat.com> <9e733fb0-5916-2342-7962-ee1ec2f6f076@redhat.com> <80b588a5-e07f-9d8e-1079-c14530570549@redhat.com> Message-ID: On 18/08/2020 07:27, Yangfei (Felix) wrote: > I created a new issue and prepared a webrev for this: > Bug: https://bugs.openjdk.java.net/browse/JDK-8251885 > Webrev: http://cr.openjdk.java.net/~fyang/8251885/webrev.01 > > This added random.seed(0) to aarch64-asmtest.py and regenerate the code > as a baseline. > Passed the smoke test for assembler in assembler_aarch64.cpp with a > slowdebug build. > Is it good to go? Super! Thank you for your patience and great work. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From thomas.stuefe at gmail.com Tue Aug 18 09:40:16 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 18 Aug 2020 11:40:16 +0200 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> Message-ID: On Tue, Aug 18, 2020 at 9:52 AM Kim Barrett wrote: > > On Aug 17, 2020, at 10:44 PM, David Holmes > wrote: > > > > Hi Kim, > > > > Not a full review - sorry. Have you tested what happens if a resource > leak is introduced? The _warned variable was used, IIUC, to avoid hitting > recursive errors during error reporting. > > OH! The recursive case hadn?t occurred to me! > > Poorly named and poorly commented variable from before the beginning of > the mercurial age, > so hard to track down the changeset that introduced it to see if that shed > some light. Good > thing we have some institutional memory :) > > I will do something about that, and then try to figure out how to trigger > it. > You could rig up something using -XX:ErrorHandlerTest (VMError::controlled_crash) and allocate from the same ResourceArea again inside the error handler. There are tests for similar things, e.g. SafeFetch, see hotspot/jtreg/runtime/ErrorReporting. But I am not sure this is worthwhile. IMHO it would be better to fix the places where ResourceAreas are used in error reporting instead of muting this warning for recursive calls. Because ideally we do not want allocations to happen at all during error reporting. BTW at SAP we have generic tests which scan hs-err files for secondary errors, but they are high maintenance. Cheers, Thomas From david.holmes at oracle.com Tue Aug 18 09:58:54 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Aug 2020 19:58:54 +1000 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> Message-ID: <742c3e2a-a6ca-2727-8977-942e88746362@oracle.com> On 18/08/2020 7:40 pm, Thomas St?fe wrote: > On Tue, Aug 18, 2020 at 9:52 AM Kim Barrett > wrote: > > > On Aug 17, 2020, at 10:44 PM, David Holmes > > wrote: > > > > Hi Kim, > > > > Not a full review - sorry. Have you tested what happens if a > resource leak is introduced? The _warned variable was used, IIUC, to > avoid hitting recursive errors during error reporting. > > OH!? The recursive case hadn?t occurred to me! > > Poorly named and poorly commented variable from before the beginning > of the mercurial age, > so hard to track down the changeset that introduced it to see if > that shed some light.? Good > thing we have some institutional memory :) > > I will do something about that, and then try to figure out how to > trigger it. > > > You could rig up something using -XX:ErrorHandlerTest > (VMError::controlled_crash) and allocate from the same ResourceArea > again inside the error handler. There are tests for similar things, e.g. > SafeFetch, see hotspot/jtreg/runtime/ErrorReporting. > > But I am not sure this is worthwhile. IMHO it would be better to fix the > places where ResourceAreas are used in error reporting instead of muting > this warning for recursive calls. Because ideally we do not want > allocations to happen at all during error reporting. I think that is impractical given all the information we try to print as part of the error report. I expect a lot of it - e.g register and memory contents - to require ResourceMarks (e.g. to print class names). David ----- > BTW at SAP we have generic tests which scan hs-err files for secondary > errors, but they are high maintenance. > > Cheers, Thomas > From thomas.stuefe at gmail.com Tue Aug 18 10:38:07 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 18 Aug 2020 12:38:07 +0200 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <742c3e2a-a6ca-2727-8977-942e88746362@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> <742c3e2a-a6ca-2727-8977-942e88746362@oracle.com> Message-ID: On Tue, Aug 18, 2020 at 11:59 AM David Holmes wrote: > On 18/08/2020 7:40 pm, Thomas St?fe wrote: > > On Tue, Aug 18, 2020 at 9:52 AM Kim Barrett > > wrote: > > > > > On Aug 17, 2020, at 10:44 PM, David Holmes > > > wrote: > > > > > > Hi Kim, > > > > > > Not a full review - sorry. Have you tested what happens if a > > resource leak is introduced? The _warned variable was used, IIUC, to > > avoid hitting recursive errors during error reporting. > > > > OH! The recursive case hadn?t occurred to me! > > > > Poorly named and poorly commented variable from before the beginning > > of the mercurial age, > > so hard to track down the changeset that introduced it to see if > > that shed some light. Good > > thing we have some institutional memory :) > > > > I will do something about that, and then try to figure out how to > > trigger it. > > > > > > You could rig up something using -XX:ErrorHandlerTest > > (VMError::controlled_crash) and allocate from the same ResourceArea > > again inside the error handler. There are tests for similar things, e.g. > > SafeFetch, see hotspot/jtreg/runtime/ErrorReporting. > > > > But I am not sure this is worthwhile. IMHO it would be better to fix the > > places where ResourceAreas are used in error reporting instead of muting > > this warning for recursive calls. Because ideally we do not want > > allocations to happen at all during error reporting. > > I think that is impractical given all the information we try to print as > part of the error report. I expect a lot of it - e.g register and memory > contents - to require ResourceMarks (e.g. to print class names). > > David > ----- > > All those cases feed from the current Thread's ResourceArea, no? So we could in theory just place a single ResourceMark at the beginning of VMError::report() to avoid all those errors (only if we have a current Thread, of course)? Thomas From david.holmes at oracle.com Tue Aug 18 12:58:09 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Aug 2020 22:58:09 +1000 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> <742c3e2a-a6ca-2727-8977-942e88746362@oracle.com> Message-ID: On 18/08/2020 8:38 pm, Thomas St?fe wrote: > On Tue, Aug 18, 2020 at 11:59 AM David Holmes > wrote: > > On 18/08/2020 7:40 pm, Thomas St?fe wrote: > > On Tue, Aug 18, 2020 at 9:52 AM Kim Barrett > > > >> > wrote: > > > >? ? ? > On Aug 17, 2020, at 10:44 PM, David Holmes > >? ? ? > >> > wrote: > >? ? ? > > >? ? ? > Hi Kim, > >? ? ? > > >? ? ? > Not a full review - sorry. Have you tested what happens if a > >? ? ?resource leak is introduced? The _warned variable was used, > IIUC, to > >? ? ?avoid hitting recursive errors during error reporting. > > > >? ? ?OH!? The recursive case hadn?t occurred to me! > > > >? ? ?Poorly named and poorly commented variable from before the > beginning > >? ? ?of the mercurial age, > >? ? ?so hard to track down the changeset that introduced it to see if > >? ? ?that shed some light.? Good > >? ? ?thing we have some institutional memory :) > > > >? ? ?I will do something about that, and then try to figure out how to > >? ? ?trigger it. > > > > > > You could rig up something using -XX:ErrorHandlerTest > > (VMError::controlled_crash) and allocate from the same ResourceArea > > again inside the error handler. There are tests for similar > things, e.g. > > SafeFetch, see hotspot/jtreg/runtime/ErrorReporting. > > > > But I am not sure this is worthwhile. IMHO it would be better to > fix the > > places where ResourceAreas are used in error reporting instead of > muting > > this warning for recursive calls. Because ideally we do not want > > allocations to happen at all during error reporting. > > I think that is impractical given all the information we try to > print as > part of the error report. I expect a lot of it - e.g register and > memory > contents - to require ResourceMarks (e.g. to print class names). > > David > ----- > > > All those cases feed from the current Thread's ResourceArea, no? So we > could in theory just place a single ResourceMark at the beginning of > VMError::report() to avoid all those errors (only if we have a current > Thread, of course)? But if the current thread has encountered a missing RM and aborts, the error code will be trying to use a RM in a thread where the RMs are already broken. Will that work or will it just abort again? And there may still be nested RMs in the code called from the error code. This is getting a long way away from just fixing up some constructor issues! And with that I'm off on vacation - sorry. :) Cheers, David ----- > Thomas From daniel.daugherty at oracle.com Tue Aug 18 13:31:58 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 18 Aug 2020 09:31:58 -0400 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: <55ae5763-476b-562c-c7cc-a913992d304c@oracle.com> References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> <95b46d0f-8a81-10c6-78bb-a3abbd7cb955@oracle.com> <55ae5763-476b-562c-c7cc-a913992d304c@oracle.com> Message-ID: This caught my eye: > We never run tests with Java asserts enabled so this is "dead" code. I'm pretty certain that I've seen test tasks in Mach5 with enable assertion options passed. I don't have an example at my fingertips though... Dan On 8/18/20 1:15 AM, David Holmes wrote: > Hi Gerard, > > On 18/08/2020 1:28 am, gerard ziemski wrote: >> Thank you David and Igor for the feedback! >> >> I rewrote the test again, this time basing it off >> test/hotspot/jtreg/runtime/Thread/* tests. >> >> For macOS and Linux we create threads until we hit a process limit >> thread count. Windows, however, allows unlimited (?) number of >> threads, though the process gradually slows down as the count grows, >> so I had to add time limit and I set it to 5 sec. I chose 5 sec >> because the value is well below jtreg time out (i.e. 120s), but still >> quite above how long my laptop needs to hit the process thread count >> limit (i.e. my Mac laptop needs about 1 sec to hit that limit). >> >> Otherwise there are no other hardwired values or assumptions (I >> hope). What do you think of the test now? > > Not unreasonable :) It could still have tried for an upper limit of > 10000 instead of using the time-limit, but it makes no real difference. > > A few nits/suggestions: > > Please add an @bug line to the test comment. > > ?49???????? if ((this.index % 250) == 0) { > > Outside of the constructor it is poor style to use this.x to refer to > fields methods (unless needed due to local variable shadowing). This > applies throughout the test. > > ?54???????? doWork(); > > Not sure why you factored things out the way you did - everything > could go in run(). > > ?61?????? setName(""+this.index); > > This could go in constructor, but I presume it adds something to the > workload? Is the use of ""+index deliberate in that regard (generate > string concatenation work), rather than using String.valueOf(index)? > > ?65?????? assert(this.index == Integer.parseInt(getName())); > > We never run tests with Java asserts enabled so this is "dead" code. > > ?87?????????? // Windows path > > Windows path or system with very large ulimit? > > ? 92???? } catch (OutOfMemoryError e) { > > I suggest checking this is actually the "unable to create native > thread" OOME rather than a general Java OOME (if run with a small heap > for example you might get OOME growing the ArrayList). > > ?110???? String cmd = System.getProperty("sun.java.command"); > ?111???? if (cmd != null && > !cmd.startsWith("com.sun.javatest.regtest.agent.MainWrapper")) { > ?112?????? // Exit with success in a non-JavaTest environment: > ?113?????? System.exit(0); > ?114???? } > > I never noticed when this was used in the little set of runtime/Thread > tests! I don't know what the rationale was but it is completely > unnecessary as a normal return from main would be the equivalent of > exit(0) - so the tests work fine without System.exit when run > standalone. Please don't emulate this part of the existing tests. I'll > file a RFE to remove it from the other tests. > > I won't be around for any further follow up for a week, so feel free > to proceed once Igor is okay with things. > > Thanks, > David > ----- > >> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev2 >> testing: Mach5 hs-tier1,2,3,4,5 in progress... >> >> >> >> >> On 8/7/20 2:41 PM, Igor Ignatyev wrote: >>> >>> >>>> On Aug 6, 2020, at 12:33 AM, David Holmes >>> > wrote: >>>> >>>> Basing the new test around the existing thread stress tests was not >>>> unreasonable but I don't think we're actively adding to those >>>> tests. I would check with Igor whether it is better to just add >>>> this under runtime/thread rather than vmTestBase. >>> >>> Hi Gerard, >>> >>> David is right, we are trying to get rid of vmTestbase directory and >>> move all tests into corresponding component directories cleaning >>> them up at the same time. so runtime/thread directory sound like a >>> much better location for a new test. >>> >>> I don't think you need to cary all this baggage of 'VM testbase >>> keywords', 'converted from ...' and the rest, I'd suggest you just >>> use the text from 'DESCRIPTION' as @summary. I'd also remove >>> System.exit call and would just make thread009::run to return bool. >>> it's also unclear to me if someone is ever going to change >>> THREADS_EXPECTED , TIMEOUT, and DEBUG_MODE?values, I'd just hardcode >>> them in the test and get rid of parsing. it seems that debug output >>> won't clutter output, so you might want to always print them and >>> remove DEBUG_MODE field. >>> >>> using `assert` in test code isn't a good idea as it will work only >>> if someone runs test w/ `-ea` which not all people do. it's better >>> to replace w/ `if (p) throw new Error(...)` ?or use >>> jdk.test.lib.Asserts class. >>> >>> Cheers, >>> -- Igor >> From gerard.ziemski at oracle.com Tue Aug 18 14:38:11 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Tue, 18 Aug 2020 09:38:11 -0500 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: <55ae5763-476b-562c-c7cc-a913992d304c@oracle.com> References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> <95b46d0f-8a81-10c6-78bb-a3abbd7cb955@oracle.com> <55ae5763-476b-562c-c7cc-a913992d304c@oracle.com> Message-ID: <4b9dfeda-c6f7-3ee5-fb23-85cf58f1130a@oracle.com> Thank you David for the review! I incorporated all your feedback. I'm surprised that "assert" is not available by default in our tests though - what is a reason for that? Of all the places where asserts should be available, a testing suite would seem to be it to me. bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev3 testing: passes locally, Mach5 hs-tier1,2,3,4,5 in progress... cheers On 8/18/20 12:15 AM, David Holmes wrote: > Hi Gerard, > > On 18/08/2020 1:28 am, gerard ziemski wrote: >> Thank you David and Igor for the feedback! >> >> I rewrote the test again, this time basing it off >> test/hotspot/jtreg/runtime/Thread/* tests. >> >> For macOS and Linux we create threads until we hit a process limit >> thread count. Windows, however, allows unlimited (?) number of >> threads, though the process gradually slows down as the count grows, >> so I had to add time limit and I set it to 5 sec. I chose 5 sec >> because the value is well below jtreg time out (i.e. 120s), but still >> quite above how long my laptop needs to hit the process thread count >> limit (i.e. my Mac laptop needs about 1 sec to hit that limit). >> >> Otherwise there are no other hardwired values or assumptions (I >> hope). What do you think of the test now? > > Not unreasonable :) It could still have tried for an upper limit of > 10000 instead of using the time-limit, but it makes no real difference. > > A few nits/suggestions: > > Please add an @bug line to the test comment. > > ?49???????? if ((this.index % 250) == 0) { > > Outside of the constructor it is poor style to use this.x to refer to > fields methods (unless needed due to local variable shadowing). This > applies throughout the test. > > ?54???????? doWork(); > > Not sure why you factored things out the way you did - everything > could go in run(). > > ?61?????? setName(""+this.index); > > This could go in constructor, but I presume it adds something to the > workload? Is the use of ""+index deliberate in that regard (generate > string concatenation work), rather than using String.valueOf(index)? > > ?65?????? assert(this.index == Integer.parseInt(getName())); > > We never run tests with Java asserts enabled so this is "dead" code. > > ?87?????????? // Windows path > > Windows path or system with very large ulimit? > > ? 92???? } catch (OutOfMemoryError e) { > > I suggest checking this is actually the "unable to create native > thread" OOME rather than a general Java OOME (if run with a small heap > for example you might get OOME growing the ArrayList). > > ?110???? String cmd = System.getProperty("sun.java.command"); > ?111???? if (cmd != null && > !cmd.startsWith("com.sun.javatest.regtest.agent.MainWrapper")) { > ?112?????? // Exit with success in a non-JavaTest environment: > ?113?????? System.exit(0); > ?114???? } > > I never noticed when this was used in the little set of runtime/Thread > tests! I don't know what the rationale was but it is completely > unnecessary as a normal return from main would be the equivalent of > exit(0) - so the tests work fine without System.exit when run > standalone. Please don't emulate this part of the existing tests. I'll > file a RFE to remove it from the other tests. > > I won't be around for any further follow up for a week, so feel free > to proceed once Igor is okay with things. > > Thanks, > David > ----- > >> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev2 >> testing: Mach5 hs-tier1,2,3,4,5 in progress... >> >> >> >> >> On 8/7/20 2:41 PM, Igor Ignatyev wrote: >>> >>> >>>> On Aug 6, 2020, at 12:33 AM, David Holmes >>> > wrote: >>>> >>>> Basing the new test around the existing thread stress tests was not >>>> unreasonable but I don't think we're actively adding to those >>>> tests. I would check with Igor whether it is better to just add >>>> this under runtime/thread rather than vmTestBase. >>> >>> Hi Gerard, >>> >>> David is right, we are trying to get rid of vmTestbase directory and >>> move all tests into corresponding component directories cleaning >>> them up at the same time. so runtime/thread directory sound like a >>> much better location for a new test. >>> >>> I don't think you need to cary all this baggage of 'VM testbase >>> keywords', 'converted from ...' and the rest, I'd suggest you just >>> use the text from 'DESCRIPTION' as @summary. I'd also remove >>> System.exit call and would just make thread009::run to return bool. >>> it's also unclear to me if someone is ever going to change >>> THREADS_EXPECTED , TIMEOUT, and DEBUG_MODE?values, I'd just hardcode >>> them in the test and get rid of parsing. it seems that debug output >>> won't clutter output, so you might want to always print them and >>> remove DEBUG_MODE field. >>> >>> using `assert` in test code isn't a good idea as it will work only >>> if someone runs test w/ `-ea` which not all people do. it's better >>> to replace w/ `if (p) throw new Error(...)` ?or use >>> jdk.test.lib.Asserts class. >>> >>> Cheers, >>> -- Igor >> From martin.doerr at sap.com Tue Aug 18 14:39:20 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 18 Aug 2020 14:39:20 +0000 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> <08584797-0841-945e-ec6d-7050a12326b4@oracle.com> <0facc9ee-eb47-a40c-4064-75d648b48eab@oss.nttdata.com> <145b716c-54ee-b32a-d008-a4ca3e7359db@oracle.com> Message-ID: Hi Yasumasa, thanks a lot for measuring and improving the overhead. This version looks much better. I still wonder what the reason was for using a stub. There was probably a reason, but I don't remember. Hopefully, Matthias can find time for another review when he's back next week. Thanks and best regards, Martin > -----Original Message----- > From: Yasumasa Suenaga > Sent: Dienstag, 18. August 2020 05:03 > To: David Holmes ; Baesken, Matthias > ; hotspot-runtime-dev at openjdk.java.net; > build-dev at openjdk.java.net; Doerr, Martin > Subject: Re: PING: RFR: 8250598: Hyper-V is detected in spite of running on > host OS > > On 2020/08/18 10:42, David Holmes wrote: > > Hi Yasumasa, > > > > On 14/08/2020 12:36 pm, Yasumasa Suenaga wrote: > >> Hi David, Matthias, Martin, > >> > >> I uploaded new webrev. Could you review again? > >> > >> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.02/ > >> > >> It generates runtime stub for hypervisor detection, and it can distinguish > between Hyper-V host (root partition) and guest. And also it works on 32bit > platforms. > >> > >> I measured startup performance (java --version) with Measure-Command > on PowerShell, this change is faster than current implementation. > >> > >> TotalMilliseconds : 57.8196 > >> TotalMilliseconds : 66.8379 > >> TotalMilliseconds : 64.7693 > >> TotalMilliseconds : 55.6546 > >> TotalMilliseconds : 63.848 > >> average : 61.78588 > > > > This seems good to me, but again I can't review the details. My main > concern is how you are testing this across a range of virtualized hosts? But if it > fixes your Hyper-V issue and doesn't break anything else then I am fine with > it. > > Thanks David! > > I don't have any other virtualized hosts excepting Hyper-V, so I cannot test > any more. > However detection logic is equivalent from current implementation, so I > believe it does not break anything else if it works fine on Hyper-V. > > And also Matthias has queued this change to internal tests in SAP. I guess he > will share the result next week. > > > > I will leave the technical review to Martin and Matthias. > > > > FYI I'm heading out on a weeks vacation after today so won't be able to > follow up further. > > Have a nice vacation! > > > Thanks, > > Yasumasa > > > > Thanks, > > David > > > >> This change has passed tests on submit repo (mach5-one-ysuenaga-JDK- > 8250598-20200814-0119-13424118), and also it works fine on following > environments: > >> > >> ?? - Hyper-V host (Windows 10 x64) > >> ?? - Hyper-V guest (Windows 10 x64) > >> ?? - Hyper-V guest (Linux x64) > >> ?? - Hyper-V guest (32bit JDK on Linux x64) > >> > >> > >> Thanks, > >> > >> Yasumasa > >> > >> > >> On 2020/08/13 20:39, David Holmes wrote: > >>> On 13/08/2020 5:39 pm, Yasumasa Suenaga wrote: > >>>> Hi Matthias, David, > >>>> > >>>> On 2020/08/13 15:52, Baesken, Matthias wrote: > >>>>> > >>>>>> > >>>>>> Should we make the change to determine just before it is needed > (e.g. VM.info or hs_err log) at first? > >>>>>> It is out of scope of this change, so I want to work as another issue if > it is needed. > >>>>>> > >>>>> > >>>>> Hi , > >>>>> I think we better do not call into WMI , when? writing an hs_err file?? . > >>>> > >>>> I found out from Hypervisor Top Level Functional Specification [1] > section 2.4.8. > >>>> That says CPUID leaf 0x40000007 is available to the root partition only. > >>>> > >>>> I changed VM_Version::is_in_VM() as below, and it works fine. > >>>> (I use __cpuid intrinsic in here because this code would be compiled by > cl.exe only.) > >>>> > >>>> ``` > >>>> #include > >>>> > >>>> #define EAX 0 > >>>> #define EBX 1 > >>>> #define ECX 2 > >>>> #define EDX 3 > >>>> > >>>> bool VM_Version::is_in_VM() { > >>>> ?? int regs[4]; > >>>> ?? __cpuid(regs, 0x40000007); > >>>> > >>>> ?? // CPUID leaf 0x40000007 is available to the root partition only. > >>>> ?? return (regs[EAX] == 0x0) && (regs[EBX] == 0x0) && (regs[ECX] == > 0x0) && (regs[EDX] == 0x0); > >>>> } > >>>> ``` > >>>> > >>>> Also startup performance is better than WMI call. > >>>> > >>>> TotalMilliseconds : 66.7863 > >>>> TotalMilliseconds : 56.8123 > >>>> TotalMilliseconds : 57.0015 > >>>> TotalMilliseconds : 62.4755 > >>>> TotalMilliseconds : 67.7632 > >>>> average : 62.16776 > >>>> > >>>> If you are ok this change, I will update webrev, and will send review > request. > >>>> (new webrev will include both loop for CPUID leaf and renaming > is_in_VM) > >>> > >>> If it works and addresses the startup hit then I'm fine with it - but still > can't actually review the code. > >>> > >>> Thanks, > >>> David > >>> > >>>> > >>>> Thanks, > >>>> > >>>> Yasumasa > >>>> > >>>> > >>>> [1] https://github.com/MicrosoftDocs/Virtualization- > Documentation/raw/master/tlfs/Hypervisor%20Top%20Level%20Functional > %20Specification%20v6.0b.pdf > >>>> > >>>> > >>>>> Best regards, Matthias > >>>>> > >>>>> > >>>>> > >>>>> -----Original Message----- > >>>>> From: Yasumasa Suenaga > >>>>> Sent: Donnerstag, 13. August 2020 06:15 > >>>>> To: David Holmes ; Baesken, Matthias > ; hotspot-runtime-dev at openjdk.java.net; > build-dev at openjdk.java.net > >>>>> Cc: Doerr, Martin > >>>>> Subject: Re: PING: RFR: 8250598: Hyper-V is detected in spite of > running on host OS > >>>>> > >>>>> On 2020/08/13 11:54, David Holmes wrote: > >>>>>> On 13/08/2020 11:12 am, Yasumasa Suenaga wrote: > >>>>>>> Hi Matthias, David, > >>>>>>> > >>>>>>> I measured startup benchmarks with `Measure-Command > {.\jdk\build\windows-x86_64-server-release\images\jdk\bin\java.exe -- > version}` on PowerShell. > >>>>>>> > >>>>>>> * PC: Ryzen 3 3300X, 16GB memory > >>>>>>> * OS: Windows 10 x64 (May 2020 Update) > >>>>>>> * Java: jdk/jdk revision 60537 > >>>>>>> ????? (Compiled by VS 2019 (16.7.0)) > >>>>>>> > >>>>>>> * without patch > >>>>>>> TotalMilliseconds : 70.2124 > >>>>>>> TotalMilliseconds : 64.4465 > >>>>>>> TotalMilliseconds : 59.0854 > >>>>>>> TotalMilliseconds : 68.0255 > >>>>>>> TotalMilliseconds : 72.6467 > >>>>>>> average : 66.8833 > >>>>>>> > >>>>>>> * with webrev.01 > >>>>>>> TotalMilliseconds : 81.7185 > >>>>>>> TotalMilliseconds : 68.539 > >>>>>>> TotalMilliseconds : 85.7226 > >>>>>>> TotalMilliseconds : 72.6584 > >>>>>>> TotalMilliseconds : 75.6091 > >>>>>>> average : 76.84952 > >>>>>>> > >>>>>>> Overhead of WMI seems to be +10ms in this case. > >>>>>> > >>>>>> Which is nearly 15%! Sorry but I just know Claes will be very unhappy > if this were to go in as-is. > >>>>> > >>>>> Should we make the change to determine just before it is needed > (e.g. VM.info or hs_err log) at first? > >>>>> It is out of scope of this change, so I want to work as another issue if it > is needed. > >>>>> > >>>>> > >>>>> Yasumasa > >>>>> > >>>>> > >>>>>> David > >>>>>> ----- > >>>>>> > >>>>>>> > >>>>>>> Yasumasa > >>>>>>> > >>>>>>> > >>>>>>> On 2020/08/13 0:05, Baesken, Matthias wrote: > >>>>>>>>> I understand that if the process runs on Xen on other hypervisor > (e.g. KVM), information for Xen would be set between 0x40000100 and > 0x40010000. > >>>>>>>>> Ok, I will not remove the loop in new webrev, and will add > comment about it. > >>>>>>>> > >>>>>>>> Hi Yasumasa? , thanks ! > >>>>>>>> > >>>>>>>> Regarding the WMI overhead , if you could? get some more info > on this it would be better to judge if it is perfectly okay or concerning . > >>>>>>>> > >>>>>>>> (I remember that I looked into it a couple of years ago , but > decided not to use it in early VM startup ;? but have to confess this was just > based on a "gut feeling", > >>>>>>>> ?? No micro benchmarks? ) > >>>>>>>> > >>>>>>>> Best regards, Matthias > >>>>>>>> > >>>>>>>> From vladimir.x.ivanov at oracle.com Tue Aug 18 16:13:31 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 18 Aug 2020 19:13:31 +0300 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> Message-ID: <49aec8be-2fc6-ab07-542d-a4d0177f8b5c@oracle.com> > full: https://cr.openjdk.java.net/~kbarrett/8251850/open.01/ > incr: https://cr.openjdk.java.net/~kbarrett/8251850/open.01.inc/ Nice cleanups, Kim. Looks good. Best regards, Vladimir Ivanov From igor.ignatyev at oracle.com Tue Aug 18 17:04:58 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 18 Aug 2020 10:04:58 -0700 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> <95b46d0f-8a81-10c6-78bb-a3abbd7cb955@oracle.com> <55ae5763-476b-562c-c7cc-a913992d304c@oracle.com> Message-ID: <585124BE-BDE7-496B-8A1F-B4E345DC5411@oracle.com> Dan/Gerard, we use '-ea -esa' in some of our configurations, yet not in all configurations and not all people use these configurations when they run tests. also some people run tests directly thru jtreg (or even w/o jtreg), hence adding it into make won't really help. therefore the tests shouldn't depend on -ea/-esa. generally speaking, `assert` makes sense only for product code where you might want some checks to be disabled in production runs, for tests you really want all checks to be always done. -- Igor > On Aug 18, 2020, at 6:31 AM, Daniel D. Daugherty wrote: > > This caught my eye: > > > We never run tests with Java asserts enabled so this is "dead" code. > > I'm pretty certain that I've seen test tasks in Mach5 with enable assertion > options passed. I don't have an example at my fingertips though... > > Dan > > > On 8/18/20 1:15 AM, David Holmes wrote: >> Hi Gerard, >> >> On 18/08/2020 1:28 am, gerard ziemski wrote: >>> Thank you David and Igor for the feedback! >>> >>> I rewrote the test again, this time basing it off test/hotspot/jtreg/runtime/Thread/* tests. >>> >>> For macOS and Linux we create threads until we hit a process limit thread count. Windows, however, allows unlimited (?) number of threads, though the process gradually slows down as the count grows, so I had to add time limit and I set it to 5 sec. I chose 5 sec because the value is well below jtreg time out (i.e. 120s), but still quite above how long my laptop needs to hit the process thread count limit (i.e. my Mac laptop needs about 1 sec to hit that limit). >>> >>> Otherwise there are no other hardwired values or assumptions (I hope). What do you think of the test now? >> >> Not unreasonable :) It could still have tried for an upper limit of 10000 instead of using the time-limit, but it makes no real difference. >> >> A few nits/suggestions: >> >> Please add an @bug line to the test comment. >> >> 49 if ((this.index % 250) == 0) { >> >> Outside of the constructor it is poor style to use this.x to refer to fields methods (unless needed due to local variable shadowing). This applies throughout the test. >> >> 54 doWork(); >> >> Not sure why you factored things out the way you did - everything could go in run(). >> >> 61 setName(""+this.index); >> >> This could go in constructor, but I presume it adds something to the workload? Is the use of ""+index deliberate in that regard (generate string concatenation work), rather than using String.valueOf(index)? >> >> 65 assert(this.index == Integer.parseInt(getName())); >> >> We never run tests with Java asserts enabled so this is "dead" code. >> >> 87 // Windows path >> >> Windows path or system with very large ulimit? >> >> 92 } catch (OutOfMemoryError e) { >> >> I suggest checking this is actually the "unable to create native thread" OOME rather than a general Java OOME (if run with a small heap for example you might get OOME growing the ArrayList). >> >> 110 String cmd = System.getProperty("sun.java.command"); >> 111 if (cmd != null && !cmd.startsWith("com.sun.javatest.regtest.agent.MainWrapper")) { >> 112 // Exit with success in a non-JavaTest environment: >> 113 System.exit(0); >> 114 } >> >> I never noticed when this was used in the little set of runtime/Thread tests! I don't know what the rationale was but it is completely unnecessary as a normal return from main would be the equivalent of exit(0) - so the tests work fine without System.exit when run standalone. Please don't emulate this part of the existing tests. I'll file a RFE to remove it from the other tests. >> >> I won't be around for any further follow up for a week, so feel free to proceed once Igor is okay with things. >> >> Thanks, >> David >> ----- >> >>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev2 >>> testing: Mach5 hs-tier1,2,3,4,5 in progress... >>> >>> >>> >>> >>> On 8/7/20 2:41 PM, Igor Ignatyev wrote: >>>> >>>> >>>>> On Aug 6, 2020, at 12:33 AM, David Holmes > wrote: >>>>> >>>>> Basing the new test around the existing thread stress tests was not unreasonable but I don't think we're actively adding to those tests. I would check with Igor whether it is better to just add this under runtime/thread rather than vmTestBase. >>>> >>>> Hi Gerard, >>>> >>>> David is right, we are trying to get rid of vmTestbase directory and move all tests into corresponding component directories cleaning them up at the same time. so runtime/thread directory sound like a much better location for a new test. >>>> >>>> I don't think you need to cary all this baggage of 'VM testbase keywords', 'converted from ...' and the rest, I'd suggest you just use the text from 'DESCRIPTION' as @summary. I'd also remove System.exit call and would just make thread009::run to return bool. it's also unclear to me if someone is ever going to change THREADS_EXPECTED , TIMEOUT, and DEBUG_MODE values, I'd just hardcode them in the test and get rid of parsing. it seems that debug output won't clutter output, so you might want to always print them and remove DEBUG_MODE field. >>>> >>>> using `assert` in test code isn't a good idea as it will work only if someone runs test w/ `-ea` which not all people do. it's better to replace w/ `if (p) throw new Error(...)` or use jdk.test.lib.Asserts class. >>>> >>>> Cheers, >>>> -- Igor >>> > From igor.ignatyev at oracle.com Tue Aug 18 17:18:35 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 18 Aug 2020 10:18:35 -0700 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: <4b9dfeda-c6f7-3ee5-fb23-85cf58f1130a@oracle.com> References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> <95b46d0f-8a81-10c6-78bb-a3abbd7cb955@oracle.com> <55ae5763-476b-562c-c7cc-a913992d304c@oracle.com> <4b9dfeda-c6f7-3ee5-fb23-85cf58f1130a@oracle.com> Message-ID: <2BAD9BBF-B8F9-45E7-8378-E69622856C7A@oracle.com> Hi Gerard, looks be 1. > 26 * @bug 8222671 @bug tag should refer to a product bug which can be reproduced w/ this test, not the bug id used to push a test. 2. > 28 * @run main/othervm ThreadCountLimit what's the reason to run in othervm mode? 3. > 34 public class ThreadCountLimit extends Object { you don't need to explicitly extend Object. 4. in ThreadCountLimit::run() > 67 } catch (InterruptedException ex) { > 68 } shouldn't this mark the test as failed? Thanks, -- Igor > On Aug 18, 2020, at 7:38 AM, gerard ziemski wrote: > > Thank you David for the review! > > I incorporated all your feedback. > > I'm surprised that "assert" is not available by default in our tests though - what is a reason for that? Of all the places where asserts should be available, a testing suite would seem to be it to me. > > bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 > open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev3 > testing: passes locally, Mach5 hs-tier1,2,3,4,5 in progress... > > > cheers > > On 8/18/20 12:15 AM, David Holmes wrote: >> Hi Gerard, >> >> On 18/08/2020 1:28 am, gerard ziemski wrote: >>> Thank you David and Igor for the feedback! >>> >>> I rewrote the test again, this time basing it off test/hotspot/jtreg/runtime/Thread/* tests. >>> >>> For macOS and Linux we create threads until we hit a process limit thread count. Windows, however, allows unlimited (?) number of threads, though the process gradually slows down as the count grows, so I had to add time limit and I set it to 5 sec. I chose 5 sec because the value is well below jtreg time out (i.e. 120s), but still quite above how long my laptop needs to hit the process thread count limit (i.e. my Mac laptop needs about 1 sec to hit that limit). >>> >>> Otherwise there are no other hardwired values or assumptions (I hope). What do you think of the test now? >> >> Not unreasonable :) It could still have tried for an upper limit of 10000 instead of using the time-limit, but it makes no real difference. >> >> A few nits/suggestions: >> >> Please add an @bug line to the test comment. >> >> 49 if ((this.index % 250) == 0) { >> >> Outside of the constructor it is poor style to use this.x to refer to fields methods (unless needed due to local variable shadowing). This applies throughout the test. >> >> 54 doWork(); >> >> Not sure why you factored things out the way you did - everything could go in run(). >> >> 61 setName(""+this.index); >> >> This could go in constructor, but I presume it adds something to the workload? Is the use of ""+index deliberate in that regard (generate string concatenation work), rather than using String.valueOf(index)? >> >> 65 assert(this.index == Integer.parseInt(getName())); >> >> We never run tests with Java asserts enabled so this is "dead" code. >> >> 87 // Windows path >> >> Windows path or system with very large ulimit? >> >> 92 } catch (OutOfMemoryError e) { >> >> I suggest checking this is actually the "unable to create native thread" OOME rather than a general Java OOME (if run with a small heap for example you might get OOME growing the ArrayList). >> >> 110 String cmd = System.getProperty("sun.java.command"); >> 111 if (cmd != null && !cmd.startsWith("com.sun.javatest.regtest.agent.MainWrapper")) { >> 112 // Exit with success in a non-JavaTest environment: >> 113 System.exit(0); >> 114 } >> >> I never noticed when this was used in the little set of runtime/Thread tests! I don't know what the rationale was but it is completely unnecessary as a normal return from main would be the equivalent of exit(0) - so the tests work fine without System.exit when run standalone. Please don't emulate this part of the existing tests. I'll file a RFE to remove it from the other tests. >> >> I won't be around for any further follow up for a week, so feel free to proceed once Igor is okay with things. >> >> Thanks, >> David >> ----- >> >>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev2 >>> testing: Mach5 hs-tier1,2,3,4,5 in progress... >>> >>> >>> >>> >>> On 8/7/20 2:41 PM, Igor Ignatyev wrote: >>>> >>>> >>>>> On Aug 6, 2020, at 12:33 AM, David Holmes > wrote: >>>>> >>>>> Basing the new test around the existing thread stress tests was not unreasonable but I don't think we're actively adding to those tests. I would check with Igor whether it is better to just add this under runtime/thread rather than vmTestBase. >>>> >>>> Hi Gerard, >>>> >>>> David is right, we are trying to get rid of vmTestbase directory and move all tests into corresponding component directories cleaning them up at the same time. so runtime/thread directory sound like a much better location for a new test. >>>> >>>> I don't think you need to cary all this baggage of 'VM testbase keywords', 'converted from ...' and the rest, I'd suggest you just use the text from 'DESCRIPTION' as @summary. I'd also remove System.exit call and would just make thread009::run to return bool. it's also unclear to me if someone is ever going to change THREADS_EXPECTED , TIMEOUT, and DEBUG_MODE values, I'd just hardcode them in the test and get rid of parsing. it seems that debug output won't clutter output, so you might want to always print them and remove DEBUG_MODE field. >>>> >>>> using `assert` in test code isn't a good idea as it will work only if someone runs test w/ `-ea` which not all people do. it's better to replace w/ `if (p) throw new Error(...)` or use jdk.test.lib.Asserts class. >>>> >>>> Cheers, >>>> -- Igor >>> > From harold.seigel at oracle.com Tue Aug 18 17:19:11 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 18 Aug 2020 13:19:11 -0400 Subject: RFR 8251490: [TESTBUG] The Java thread stack size specified is too small for nsk/stress/stack. Specify at least 448k Message-ID: <9f12fc32-646c-1aff-86fa-003db2641c91@oracle.com> Hi, Please review this small change to fix five vmTestbase/nsk/stress/stack tests that are failing in tier5 on Linux AArch64.? The change increases the stack size for three of the tests.? Increasing the stack size of the other two tests could result in timeouts on other platforms.? So, those tests were changed to no longer run on AArch64. Open Webrev: http://cr.openjdk.java.net/~hseigel/bug_8251490/webrev/index.html JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251490 The change was tested by running the tests as part of mach5 tier5 on Windows, Mac, Linux x64, and Linux AArch64. Thanks, Harold From harold.seigel at oracle.com Tue Aug 18 17:36:12 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 18 Aug 2020 13:36:12 -0400 Subject: RFR 8251414: Add test that invokeinterface of a protected method in java.lang.Object throws NoSuchMethodError In-Reply-To: <6173e896-dd66-a8ba-f524-02e9e350b654@oracle.com> References: <97328605-aa99-51db-25eb-e1465919b8db@oracle.com> <6173e896-dd66-a8ba-f524-02e9e350b654@oracle.com> Message-ID: <64ae2e82-c08c-66ac-2a8d-13b0afbf42ab@oracle.com> Hi David, The test was adapted from an existing test that used reflection. It probably doesn't need to use it. If test file InterfaceObj.jasm is written in java, as described in the file's comments, then its compilation fails with the following errors: javac I.java InterfaceObj.java InterfaceObjectTest.java InterfaceObj.java:3: error: cannot find symbol ??????? I.finalize(); ???????? ^ ? symbol:?? method finalize() ? location: interface I InterfaceObj.java:9: error: cannot find symbol ??????? I.clone(); ???????? ^ ? symbol:?? method clone() ? location: interface I Thanks, Harold On 8/17/2020 5:53 PM, David Holmes wrote: > Hi Harold, > > On 18/08/2020 12:26 am, Harold Seigel wrote: >> Hi David, >> >> This change got pushed before I saw your comments.? Let me know if I >> should enter a new RFE to add the comments that you request below. > > No that's okay. But can you tell me why it needed to use jasm and > reflection? :) > > Thanks, > David > >> Thanks, Harold >> >> On 8/16/2020 10:06 PM, David Holmes wrote: >>> Hi Harold, >>> >>> On 13/08/2020 4:14 am, Harold Seigel wrote: >>>> Hi, >>>> >>>> Please review this change to add test cases to test >>>> hotspot/jtreg/runtime/8026394/InterfaceObjectTest.java to check for >>>> NoSuchMethodError exceptions when calling Object protected methods >>>> clone() and finalize() using invokeinterface. >>>> >>>> Open Webrev: >>>> http://cr.openjdk.java.net/~hseigel/bug_8251414/webrev/index.html >>> >>> Can you add a comment explaining why we need to use jasm and >>> reflection in this new case. >>> >>> Thanks, >>> David >>> >>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251414 >>>> >>>> The fix was tested by running the modified test on multiple >>>> platforms, including Mac OS, Windows, and Linux. >>>> >>>> Thanks, Harold From gerard.ziemski at oracle.com Tue Aug 18 18:06:01 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Tue, 18 Aug 2020 13:06:01 -0500 Subject: RFR 8251490: [TESTBUG] The Java thread stack size specified is too small for nsk/stress/stack. Specify at least 448k In-Reply-To: <9f12fc32-646c-1aff-86fa-003db2641c91@oracle.com> References: <9f12fc32-646c-1aff-86fa-003db2641c91@oracle.com> Message-ID: <11880712-97f7-efee-2cd3-fb22edf89af9@oracle.com> hi Harold, Thank you for fixing this. On 8/18/20 12:19 PM, Harold Seigel wrote: > Hi, > > Please review this small change to fix five > vmTestbase/nsk/stress/stack tests that are failing in tier5 on Linux > AArch64.? The change increases the stack size for three of the tests.? > Increasing the stack size of the other two tests could result in > timeouts on other platforms.? So, those tests were changed to no > longer run on AArch64. > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8251490/webrev/index.html > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251490 > > The change was tested by running the tests as part of mach5 tier5 on > Windows, Mac, Linux x64, and Linux AArch64. > > Thanks, Harold > It looks reasonable to me, but I have some questions: #1 Does "-eager" really make much difference here? Do you have any timing numbers you can provide with and without that flag, just so that we know how much it helps? #2 How did you determine the the 900s timeout? How long does it take for a regular, middle of the road machine to execute each test? Do we use any slow/low end machines in our testing that could take longer than this time out? (I'm worried about that hardcoded time out value if you can tell) cheers From harold.seigel at oracle.com Tue Aug 18 19:54:39 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 18 Aug 2020 15:54:39 -0400 Subject: RFR 8251490: [TESTBUG] The Java thread stack size specified is too small for nsk/stress/stack. Specify at least 448k In-Reply-To: <11880712-97f7-efee-2cd3-fb22edf89af9@oracle.com> References: <9f12fc32-646c-1aff-86fa-003db2641c91@oracle.com> <11880712-97f7-efee-2cd3-fb22edf89af9@oracle.com> Message-ID: <1e0f8292-5929-2356-d1b8-638443e9a4fd@oracle.com> Hi Gerard, Thanks for looking at this change.? Please see inline comments. On 8/18/2020 2:06 PM, gerard ziemski wrote: > hi Harold, > > Thank you for fixing this. > > > On 8/18/20 12:19 PM, Harold Seigel wrote: >> Hi, >> >> Please review this small change to fix five >> vmTestbase/nsk/stress/stack tests that are failing in tier5 on Linux >> AArch64.? The change increases the stack size for three of the >> tests.? Increasing the stack size of the other two tests could result >> in timeouts on other platforms.? So, those tests were changed to no >> longer run on AArch64. >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8251490/webrev/index.html >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251490 >> >> The change was tested by running the tests as part of mach5 tier5 on >> Windows, Mac, Linux x64, and Linux AArch64. >> >> Thanks, Harold >> > It looks reasonable to me, but I have some questions: > > #1 Does "-eager" really make much difference here? Do you have any > timing numbers you can provide with and without that flag, just so > that we know how much it helps? This change does not affect the use of "-eager".? "-eager" has been part of the tests at least as far back as JDK-11.? The only affect of running with "-eager" is to prevent the creation of a Terminator thread that waits a designated amount of time before calling System.exit().? Its purpose is described here in Terminator.java: /** ?* Terminator is used to terminate a stress test with PASS exit status ?* before the test is terminated as timed out (and so failed). ?* ?* Terminator class holds a thread which sleeps for the given amount ?* of time, and then wakes up and executes System.exit() ?* with the given exit status. That thread is daemon, so it doesn't ?* prevent application from exiting once all its threads finish ?* before it's time for termination. Appointing terminator in zero ?* delay implies immediate exit(). ?* ?* There is a limitation: you may appoint no more than one terminator ?* per application. ?*/ > > #2 How did you determine the the 900s timeout? How long does it take > for a regular, middle of the road machine to execute each test? Do we > use any slow/low end machines in our testing that could take longer > than this time out? (I'm worried about that hardcoded time out value > if you can tell) I did not determine the 900s timeout.? That also has been in the tests for at least as far back as JDK-11.? When working on JDK-8139875, I ran the tests many times with different stack sizes both locally and with mach5 and looked at the run times for the tests, making sure they were all significantly below the timeout. So far, the tests have run as part of tier5 for almost two weeks without timing out. The purpose of this proposed change is provide stack size values that will work on AArch64.? Do you think a new bug is needed to address the use of -eager and the 900s timeout? Thanks, Harold > > > cheers > From coleen.phillimore at oracle.com Tue Aug 18 22:17:19 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 18 Aug 2020 18:17:19 -0400 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: Message-ID: <8bd366a3-a40b-c2d6-8e19-1abf7a4c7e04@oracle.com> Hi Thomas, I've read through the new implementation files.? This looks really good!? These are my initial comments.? There aren't many and nothing of substance really.? Some files need more in-depth review though. I haven't looked at the tests, which are more difficult.? I'll do that tomorrow. General comment:? Some files have a lot of vertical white space, like two or more blank lines together: eg. metachunk.cpp at the end.? More about that below. === counter.hpp This seems like a general utility that should be in utilities so that it can be generalized for use in other places.? I didn't review it carefully to suggest it go there right now, but we should file another RFE to do that.? Kim has some ideas about it. === freeBlocks.hpp I like how the small blocks and tree of freed blocks are together (except see below). This has a really good comment. #ifndef SHARE_MEMORY_METASPACE_LEFTOVERBINS_HPP #define SHARE_MEMORY_METASPACE_LEFTOVERBINS_HPP Wrong ifdef guard names. #endif // SHARE_MEMORY_METASPACE_CHUNKMANAGER_HPP === binList.hpp ? block_t* _v[num_lists]; Some slightly longer name would be better than _v.? Maybe _small_blocks or something like that? // These aren't used typedef BinListImpl<2, 8>? BinList8; typedef BinListImpl<2, 16> BinList16; typedef BinListImpl<2, 64> BinList64; This file should be binList.hpp rather than binlist.hpp. === blockTree.hpp/cpp I didn't review this code.? Someone suggested using std::map instead but we're not using std:: yet. blocktree.hpp/cpp should be blockTree.hpp/cpp Since for class metaspace, the reclaimed InstanceKlass are generally bigger than the largest BinList size, maybe class type MetaspaceArenas should only have a blockTree and not a SmallBlocks, so not the whole freeList.hpp structure.? I think we could do some footprint analysis with NMT and see if this might help.? It's fairly rare unless you have a lot of redefinition.? I don't think this should be changed in this change though. === metaspaceContext.cpp #define LOGFMT???????? "SpcMgr @" PTR_FORMAT " (%s)" #define LOGFMT_ARGS??? p2i(this), this->_name These don't seem to be used and is SpcMgr a reference to the old SpaceManager?? Correction, I see how it's used but you should change it to not refer to Space manager. // Destroys the context: deletes chunkmanager and virtualspacelist. // If this is a non-expandable context over an existing space, that space remains // untouched, otherwise all memory is unmapped. MetaspaceContext::~MetaspaceContext() { ? delete _cm; ? delete _vslist; } Why would you ever do this?? I thought the two contexts were global.? If it's for the tests, add a comment here. === metachunk.cpp ??? class UnsortedMetachunkList : public AbstractMetachunkList { This class is declared in a strange place (in a function) and appears unused. check_pattern() appears unused which is good because it's a lot of code. === metaspace_test.hpp/cpp I keep thinking this file is in the wrong place.? Can you rename metaspaceTestHelper? or some name like that? === internStats.hpp Indentation is inconstent and too much vertical white space at the end. Should be renamed internalStats.hpp/cpp to match the class name.? This looks like a nice improvement. One of the things that made metaspace code hard to read was that there was a lot of code for statistics mixed in everywhere. === arenaGrowthPolicy.hpp/cpp I like this! === chunkLevel.hpp/cpp I'm fine with the nested namespace chunklevel, rather than an AllStatic class ChunkLevel.? The names of the files should probably be changed to chunklevel.hpp/cpp though. === metaspaceEnums.hpp/cpp This set of files seems like they are misnamed and should be encorporated into metaspace.hpp, since they are used outside of the memory/metaspace directory. === settings.hpp Include guards are wrong names. #ifndef SHARE_MEMORY_METASPACE_CONSTANTS_HPP #define SHARE_MEMORY_METASPACE_CONSTANTS_HPP #endif // SHARE_MEMORY_METASPACE_BLOCKFREELIST_HPP I actually took the allocation path when looking at this.? Thank you for keeping many of the function names because that helped. thanks, Coleen On 8/12/20 1:59 PM, Thomas St?fe wrote: > Dear all, > > I would like to start the review for the implementation of JEP 387 "Elastic > Metaspace". > > The relevant JBS items are: > > JEP: https://openjdk.java.net/jeps/387 > > Implementation Issue (pretty much only a placeholder currently): > https://bugs.openjdk.java.net/browse/JDK-8251158 > > -- > > Reminder of why we do this: > > 1. The new metaspace saves memory. How much depends on context: if it is > not a problem today it will continue to be none. But in cases where today > we face large metaspace consumption we may get reductions, sometimes > drastic ones. These reductions are caused by two facts: > - after mass class unloading we release memory more promptly to the OS > - even without class unloading chunk allocation is just plain smarter, > which reduces the overhead per class loader. This is especially true for > scenarios involving masses of small loaders which only load very few > classes. > > As an example, see [1] where two VMs - one stock, one patched - run the > same test program which creates tons of small loaders. The second run shows > a much reduced memory footprint and increased elasticity. > > 2. The rewritten metaspace code base got cleaner and less complex and thus > should be much easier to maintain and to extend. It also would be possible > to easily reuse the allocator for different parts of the VM, since it is > less tightly tied to the notion of just storing class metadata. > > -- > > In preparation of this review I prepared a guide [2], [3], which gives an > architectural overview and should be the starting point for an interested > Reviewer. > > The prototype has been tested extensively for quite some time in SAP's CI > system. We regularly run JCK test, JTReg tests and a whole battery of SAP > internal tests on 8 platforms. Tests are also currently ongoing at Oracle > and Red Hat. > > -- > > The full webrev [4] is somewhat large. In order to make this more bearable > I broke the patch up into three parts: > > 1) The core parts [5] > > This is the heart of the metaspace, mostly rewritten from scratch. I > propose any Reviewer should not look at the diff but rather just examine > the new implementation. One possible exception is metaspace.hpp, which is > the outside interface to metaspace. > > There are several ways to get a feeling for this code (after reading at > least the concept part of the provided guide [2]). The central, most > "beefy" classes are: > > - VirtualSpaceNode - does all the work of reserving, committing, > uncommitting memory > - RootChunkArea - does the grunt work of chunk splitting and merging > - ChunkManager - which takes care of the chunk freelists, of directing > chunk splits and merges, of enlarging chunks in place > - MetaspaceArena - which takes care of fine granular allocation on behalf > of a CLD, and of managing deallocated blocks. > > One way to review could be bottom up: starting at > VirtualSpaceNode/CommitMask/RootChunkArea(LUT), working your way up to > ChunkManager and Metachunk; finally up to to MetaspaceArena while taking a > side stroll to FreeBlocks/BinList/BlockTree. > > Another way would be to follow typical paths: > > Allocation path: Starting at MetaspaceArena::allocate() -> > Metachunk::allocate() (note the committing-on-demand code path starting > here) -> ChunkManager::get_chunk() -> > VirtualSpaceList/Node->allocate_root_chunk(). > > Destruction: ~MetaspaceArena() -> ChunkManager::return_chunk() -> (merging > chunks) -> (uncommitting chunks) > > Premature deallocation: -> MetaspaceArena::deallocate() -> see freeblock > list handling > > 2) The tests [6] > > This part of the patch contains all the new tests. There are a lot; the > test coverage of the new metaspace is very thorough. > > New gtests have been added under 'test/hotspot/gtest/metaspace'. > New jtreg have been added under > 'test/hotspot/jtreg/runtime/Metaspace/elastic' and under > 'test/hotspot/jtreg/gtest/MetaspaceGtests.java'. > > 4) Miscellaneous diffs [7] > > This is the part of the patch which is neither considered core nor test. > These changes are small, often uninteresting, and can be reviewed via diff. > > --- > > Out of scope for this patch is revamping outside metaspace statistics: > monitoring of metaspace statistics is mostly left untouched, beyond the > work needed to keep existing tests green. I wanted to keep those changes > separate from the core changes for JEP387. They are tracked in JDK-8251392 > [8] and JDK-8251342 [9], respectively. > > --- > > Code complexity: > > Numerically, lines of code went ever so slightly down with this patch: > > Before: (cloc hotspot/share/memory) > ------------------------------------------------------------------------------- > C++ 36 2533 3097 12735 > C/C++ Header 54 1728 2867 6278 > ------------------------------------------------------------------------------- > SUM: 90 4261 5964 19013 > > After: > ------------------------------------------------------------------------------- > C++ 50 3048 3774 13127 > C/C++ Header 63 2006 3575 5738 > ------------------------------------------------------------------------------- > SUM: 113 5054 7349 18865 > > But since the new implementation is more powerful than the old one - doing > things like committing/uncommitting on demand, guarding allocated blocks > etc - one could argue that the new solution does quite a lot more with > slightly less code, which I think is a good result. > > --- > > Idle musing: it would simplify matters quite a bit if we were to unify > class space and non-class metaspace. If we keep the current narrow Klass > pointer encoding scheme, we would still need to keep the space they are > stored in contiguous. But we could use the class space for everything, in > effect dropping the "class" moniker from the name. It would have to be > larger than it is currently, of course. > > Cons: > - we would have to abandon the notion of "limitless metaspace" - but if we > run with class space this has never been true anyway since the number of > classes is limited by the size of the compressed class space. > - virtual process size would go up since it now needs to be as large as the > total projected metaspace size. > - we may need to expand narrow Klass pointer encoding to include shifting, > if 4G are not enough to hold all metaspace data. > > Pros: > - this would simplify a lot of code. > - this would save real (committed) memory, since we save quite a bit of > overhead. > - we could narrow-pointer-encode other metadata too, not only Klass > pointers, should that ever be interesting > > ... but that is not part of this JEP. > > --- > > With this patch, we have a much stronger separation of concerns, and it > should be easy to reuse the metaspace allocator in other hotspot areas as > well. For instance, ResourceAreas and friends could be replaced by > MetaspaceArenas. They do almost the same thing. And as we have seen in the > past at SAP, the C-heap backed hotspot Arenas can be a pain since spikes in > Arena usage lingers forever as process footprint (we even once rewrote > parts of the arena code to use mmap, which is just on a more primitive > level what Metaspace does). > > --- > > I know this is short notice, but there will be a call for interested people > tomorrow at 11AM ET. In case any potential Reviewers are interested just > drop me a short note. > > --- > > Thank you, Thomas > > > [1] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/reduction-small-loaders.pdf > [2] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.pdf > [3] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.html > [4] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-all/webrev/ > [5] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-core/webrev/ > [6] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-test/webrev/ > [7] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-misc/webrev/ > [8] https://bugs.openjdk.java.net/browse/JDK-8251342 > [9] https://bugs.openjdk.java.net/browse/JDK-8251392 From igor.ignatyev at oracle.com Tue Aug 18 23:42:19 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 18 Aug 2020 16:42:19 -0700 Subject: RFR(T) : 8252005 : narrow disabling of allowSmartActionArgs in vmTestbase Message-ID: <4E6FECE6-9103-46ED-84B2-79DBA0123ED9@oracle.com> http://cr.openjdk.java.net/~iignatyev//8252005/webrev.00/ > 0 lines changed: 0 ins; 0 del; 0 mod; Hi all, could you please review this trivial (and apparently empty) patch which sets allowSmartActionArgs to false only in subdirectories of vmTestbase which currently use PropertyResolvingWrapper? (it's hard to tell from webrev or patch, but test/hotspot/jtreg/vmTestbase/TEST.properties is effectively removed) webrev: http://cr.openjdk.java.net/~iignatyev//8252005/webrev.00/ JBS: https://bugs.openjdk.java.net/browse/JDK-8252005 Thanks, -- Igor From felix.yang at huawei.com Wed Aug 19 00:58:42 2020 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Wed, 19 Aug 2020 00:58:42 +0000 Subject: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic In-Reply-To: References: <654e8a2e-87a1-9c3d-4c58-36b4215bb0ad@redhat.com> <5350e745-87af-ce34-8711-2b652a38e323@redhat.com> <4d1f7179-2cad-620d-f833-949e8b39ee06@redhat.com> <7cedaced-9628-32b3-6d01-beae760fc964@redhat.com> <9e733fb0-5916-2342-7962-ee1ec2f6f076@redhat.com> <80b588a5-e07f-9d8e-1079-c14530570549@redhat.com> Message-ID: > -----Original Message----- > From: Andrew Haley [mailto:aph at redhat.com] > Sent: Tuesday, August 18, 2020 5:34 PM > To: Yangfei (Felix) > Cc: hotspot-runtime-dev at openjdk.java.net; aarch64-port- > dev at openjdk.java.net > Subject: Re: RFR: 8165404: AArch64: Implement SHA512 accelerator/intrinsic > > On 18/08/2020 07:27, Yangfei (Felix) wrote: > > I created a new issue and prepared a webrev for this: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8251885 > > Webrev: http://cr.openjdk.java.net/~fyang/8251885/webrev.01 > > > > This added random.seed(0) to aarch64-asmtest.py and regenerate the > > code as a baseline. > > Passed the smoke test for assembler in assembler_aarch64.cpp with a > > slowdebug build. > > Is it good to go? > > Super! Thank you for your patience and great work. Thanks for reviewing this. Pushed as: https://hg.openjdk.java.net/jdk/jdk/rev/06a0743e84d9 Felix From suenaga at oss.nttdata.com Wed Aug 19 01:33:11 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Wed, 19 Aug 2020 10:33:11 +0900 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> <08584797-0841-945e-ec6d-7050a12326b4@oracle.com> <0facc9ee-eb47-a40c-4064-75d648b48eab@oss.nttdata.com> <145b716c-54ee-b32a-d008-a4ca3e7359db@oracle.com> Message-ID: <17f9e7d8-d5fa-d4c6-8846-7c57f026cacb@oss.nttdata.com> Hi Martin, On 2020/08/18 23:39, Doerr, Martin wrote: > Hi Yasumasa, > > thanks a lot for measuring and improving the overhead. This version looks much better. > > I still wonder what the reason was for using a stub. There was probably a reason, but I don't remember. It might be called in many times, so I made change would generate stub in first. VM_Version::check_virtualizations() would call CPUID 256 times (0x40000000 to 0x40010000 step 0x100), and also I changed it is used to call CPUID with EAX = 0x40000007 in Hyper-V. > Hopefully, Matthias can find time for another review when he's back next week. Yeah, I want to commit this change after his review. Thanks, Yasumasa > Thanks and best regards, > Martin > > >> -----Original Message----- >> From: Yasumasa Suenaga >> Sent: Dienstag, 18. August 2020 05:03 >> To: David Holmes ; Baesken, Matthias >> ; hotspot-runtime-dev at openjdk.java.net; >> build-dev at openjdk.java.net; Doerr, Martin >> Subject: Re: PING: RFR: 8250598: Hyper-V is detected in spite of running on >> host OS >> >> On 2020/08/18 10:42, David Holmes wrote: >>> Hi Yasumasa, >>> >>> On 14/08/2020 12:36 pm, Yasumasa Suenaga wrote: >>>> Hi David, Matthias, Martin, >>>> >>>> I uploaded new webrev. Could you review again? >>>> >>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.02/ >>>> >>>> It generates runtime stub for hypervisor detection, and it can distinguish >> between Hyper-V host (root partition) and guest. And also it works on 32bit >> platforms. >>>> >>>> I measured startup performance (java --version) with Measure-Command >> on PowerShell, this change is faster than current implementation. >>>> >>>> TotalMilliseconds : 57.8196 >>>> TotalMilliseconds : 66.8379 >>>> TotalMilliseconds : 64.7693 >>>> TotalMilliseconds : 55.6546 >>>> TotalMilliseconds : 63.848 >>>> average : 61.78588 >>> >>> This seems good to me, but again I can't review the details. My main >> concern is how you are testing this across a range of virtualized hosts? But if it >> fixes your Hyper-V issue and doesn't break anything else then I am fine with >> it. >> >> Thanks David! >> >> I don't have any other virtualized hosts excepting Hyper-V, so I cannot test >> any more. >> However detection logic is equivalent from current implementation, so I >> believe it does not break anything else if it works fine on Hyper-V. >> >> And also Matthias has queued this change to internal tests in SAP. I guess he >> will share the result next week. >> >> >>> I will leave the technical review to Martin and Matthias. >>> >>> FYI I'm heading out on a weeks vacation after today so won't be able to >> follow up further. >> >> Have a nice vacation! >> >> >> Thanks, >> >> Yasumasa >> >> >>> Thanks, >>> David >>> >>>> This change has passed tests on submit repo (mach5-one-ysuenaga-JDK- >> 8250598-20200814-0119-13424118), and also it works fine on following >> environments: >>>> >>>> ?? - Hyper-V host (Windows 10 x64) >>>> ?? - Hyper-V guest (Windows 10 x64) >>>> ?? - Hyper-V guest (Linux x64) >>>> ?? - Hyper-V guest (32bit JDK on Linux x64) >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2020/08/13 20:39, David Holmes wrote: >>>>> On 13/08/2020 5:39 pm, Yasumasa Suenaga wrote: >>>>>> Hi Matthias, David, >>>>>> >>>>>> On 2020/08/13 15:52, Baesken, Matthias wrote: >>>>>>> >>>>>>>> >>>>>>>> Should we make the change to determine just before it is needed >> (e.g. VM.info or hs_err log) at first? >>>>>>>> It is out of scope of this change, so I want to work as another issue if >> it is needed. >>>>>>>> >>>>>>> >>>>>>> Hi , >>>>>>> I think we better do not call into WMI , when? writing an hs_err file?? . >>>>>> >>>>>> I found out from Hypervisor Top Level Functional Specification [1] >> section 2.4.8. >>>>>> That says CPUID leaf 0x40000007 is available to the root partition only. >>>>>> >>>>>> I changed VM_Version::is_in_VM() as below, and it works fine. >>>>>> (I use __cpuid intrinsic in here because this code would be compiled by >> cl.exe only.) >>>>>> >>>>>> ``` >>>>>> #include >>>>>> >>>>>> #define EAX 0 >>>>>> #define EBX 1 >>>>>> #define ECX 2 >>>>>> #define EDX 3 >>>>>> >>>>>> bool VM_Version::is_in_VM() { >>>>>> ?? int regs[4]; >>>>>> ?? __cpuid(regs, 0x40000007); >>>>>> >>>>>> ?? // CPUID leaf 0x40000007 is available to the root partition only. >>>>>> ?? return (regs[EAX] == 0x0) && (regs[EBX] == 0x0) && (regs[ECX] == >> 0x0) && (regs[EDX] == 0x0); >>>>>> } >>>>>> ``` >>>>>> >>>>>> Also startup performance is better than WMI call. >>>>>> >>>>>> TotalMilliseconds : 66.7863 >>>>>> TotalMilliseconds : 56.8123 >>>>>> TotalMilliseconds : 57.0015 >>>>>> TotalMilliseconds : 62.4755 >>>>>> TotalMilliseconds : 67.7632 >>>>>> average : 62.16776 >>>>>> >>>>>> If you are ok this change, I will update webrev, and will send review >> request. >>>>>> (new webrev will include both loop for CPUID leaf and renaming >> is_in_VM) >>>>> >>>>> If it works and addresses the startup hit then I'm fine with it - but still >> can't actually review the code. >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> [1] https://github.com/MicrosoftDocs/Virtualization- >> Documentation/raw/master/tlfs/Hypervisor%20Top%20Level%20Functional >> %20Specification%20v6.0b.pdf >>>>>> >>>>>> >>>>>>> Best regards, Matthias >>>>>>> >>>>>>> >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Yasumasa Suenaga >>>>>>> Sent: Donnerstag, 13. August 2020 06:15 >>>>>>> To: David Holmes ; Baesken, Matthias >> ; hotspot-runtime-dev at openjdk.java.net; >> build-dev at openjdk.java.net >>>>>>> Cc: Doerr, Martin >>>>>>> Subject: Re: PING: RFR: 8250598: Hyper-V is detected in spite of >> running on host OS >>>>>>> >>>>>>> On 2020/08/13 11:54, David Holmes wrote: >>>>>>>> On 13/08/2020 11:12 am, Yasumasa Suenaga wrote: >>>>>>>>> Hi Matthias, David, >>>>>>>>> >>>>>>>>> I measured startup benchmarks with `Measure-Command >> {.\jdk\build\windows-x86_64-server-release\images\jdk\bin\java.exe -- >> version}` on PowerShell. >>>>>>>>> >>>>>>>>> * PC: Ryzen 3 3300X, 16GB memory >>>>>>>>> * OS: Windows 10 x64 (May 2020 Update) >>>>>>>>> * Java: jdk/jdk revision 60537 >>>>>>>>> ????? (Compiled by VS 2019 (16.7.0)) >>>>>>>>> >>>>>>>>> * without patch >>>>>>>>> TotalMilliseconds : 70.2124 >>>>>>>>> TotalMilliseconds : 64.4465 >>>>>>>>> TotalMilliseconds : 59.0854 >>>>>>>>> TotalMilliseconds : 68.0255 >>>>>>>>> TotalMilliseconds : 72.6467 >>>>>>>>> average : 66.8833 >>>>>>>>> >>>>>>>>> * with webrev.01 >>>>>>>>> TotalMilliseconds : 81.7185 >>>>>>>>> TotalMilliseconds : 68.539 >>>>>>>>> TotalMilliseconds : 85.7226 >>>>>>>>> TotalMilliseconds : 72.6584 >>>>>>>>> TotalMilliseconds : 75.6091 >>>>>>>>> average : 76.84952 >>>>>>>>> >>>>>>>>> Overhead of WMI seems to be +10ms in this case. >>>>>>>> >>>>>>>> Which is nearly 15%! Sorry but I just know Claes will be very unhappy >> if this were to go in as-is. >>>>>>> >>>>>>> Should we make the change to determine just before it is needed >> (e.g. VM.info or hs_err log) at first? >>>>>>> It is out of scope of this change, so I want to work as another issue if it >> is needed. >>>>>>> >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>>> David >>>>>>>> ----- >>>>>>>> >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2020/08/13 0:05, Baesken, Matthias wrote: >>>>>>>>>>> I understand that if the process runs on Xen on other hypervisor >> (e.g. KVM), information for Xen would be set between 0x40000100 and >> 0x40010000. >>>>>>>>>>> Ok, I will not remove the loop in new webrev, and will add >> comment about it. >>>>>>>>>> >>>>>>>>>> Hi Yasumasa? , thanks ! >>>>>>>>>> >>>>>>>>>> Regarding the WMI overhead , if you could? get some more info >> on this it would be better to judge if it is perfectly okay or concerning . >>>>>>>>>> >>>>>>>>>> (I remember that I looked into it a couple of years ago , but >> decided not to use it in early VM startup ;? but have to confess this was just >> based on a "gut feeling", >>>>>>>>>> ?? No micro benchmarks? ) >>>>>>>>>> >>>>>>>>>> Best regards, Matthias >>>>>>>>>> >>>>>>>>>> From yumin.qi at oracle.com Wed Aug 19 05:14:00 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 18 Aug 2020 22:14:00 -0700 Subject: Fwd: RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS In-Reply-To: <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> Message-ID: Hi, Calvin ? I have updated the webrev at http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: > Hi Yumin, > > I reviewed mostly the native code. Below are my comments: > > 1) classListParser.hpp > > 71?? bool??????????????? _lambda_format; > No longer? used in new patch. > The above name is too generic. How about _lambda_form or _is_lambda_form? > If you rename the above, please also rename the function which returns the above bool. > > 2) jvm.cpp > > 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line)) > > ignore -> ignored done. > > 3) jvm.hpp > > 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); > > Same comment as for jvm.cpp > done > 4) metaspaceShared.cpp > > 2017?? size_t i = 0; > 2018?? while (i < size) { > 2019???? full_name[i++] = *start++; > 2020?? } > > Could the above be simplified to the following? > > ??? strncpy(full_name, start, size - 1); > I could use strncpy, but think it is not as efficient as this version since it does the same thing, and call c library function: The call chain for strcpy -> memcpy -> above code. > 2029 char* class_name = get_full_class_name(path_name); > > Should os::free(class_name) be called before the function returns? > In fact it does not matter, since we exit after dump. > 1870 static GrowableArray* lambda_list = NULL; > > The name lambda_list is a bit generic, how about lambda_form_list? > > 2112?????? lambda_list->append(parser.current_line()); > > Since parser.current_line() does a strdup, do those buffer need to be freed after its use? (i.e. after the call to regenerate_holder_classes()?) > > In MetaspaceShared::regenerate_holder_classes, before calling up to java, I think it's better to strip the prefix "@lambda-form-invoker" from the strings. So that the java InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need to handle it: > > ?143???????????????????? case "[LF_RESOLVE]": > ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: > See new version in the new patch. No call for free due to the same reason: there about 100+ allocations, not a big allocation, and after dump we exit. > 5) DumpSymbolAndStringTable.java > > 37???? private static final String my_string = "DumpSymbolAndStringTable"; > > Unused variable? > It is used, the string will be in stringTable and output should contain it. Thanks Yumin > thanks, > > Calvin > > On 8/11/20 10:36 AM, Yumin Qi wrote: >> Forget to send to @core-lib-dev, the patch changed jdk code. >> >> >> Thanks >> >> Yumin >> >> >> >> -------- Forwarded Message -------- >> Subject:???? RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS >> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >> From:???? Yumin Qi >> To:???? hotspot-runtime-dev at openjdk.java.net >> >> >> >> Hi, Please reivew >> >> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >> >> ? Summary: CDS does not archive pre-generated lambda form classes for method handles: >> >> [0.142s][info][class,load] java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: __JVM_LookupDefineClass__ >> >> The method handle resolution if not found in the holder class, a class with the method will be generated and loaded as vm internal created class and not archived like above. Those class names are mangled as combination of the class name and the class instance address. >> >> In this patch, collect those method holder class names and their associated methods' signatures when user specify DumpLoadedClassList, and record them in the log file in a format similar to the output format from traced by -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use another name for CDS: -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The line prefix also changed from "[LF_RESOLVE]" to "@lambda-invoke-handle".? At dump time, regenerate the holder class with those methods and replace the existing holder class and archived it. At runtime, the resolution of the holder class which contains those methods are loaded from CDS archive so avoid regenerating them again. The patch reorganized the code for Jlink plugin, so the new added InvokerBytecodeGeneratorHelper can be shared both by JLink plugin and CDS code. Also change made to the mangled generated class name at static dump, the class name combining the class name and a sequentially ordered >> number instead of the class instance address, which looks like a random. >> >> >> To use this feature, one can do the dump/run just like done for a static dump/run, so no extra steps required. >> >> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >> >> ????? DumpLoadedClassList will turn on -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >> >> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist -XX:SharedArchiveFile=my.jsa JavaApp >> >> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >> >> ? The performance on javac HelloWorld.java (javac-benchmark), no patch vs patch: >> >> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ----???? 391.720 382.990 ( -8.730)????? ---- >> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ----???? 391.030 381.480 ( -9.550)????? ----- >> ?? 3:?? 2694664066? 2636523114 (-58140952)????? -----??? 390.610 382.550 ( -8.060)????? ---- >> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ----???? 390.700 383.390 ( -7.310)????? --- >> ?? 5:?? 2691072338? 2633016687 (-58055651)????? -----??? 388.990 382.360 ( -6.630)????? --- >> ?? 6:?? 2684448174? 2644191854 (-40256320)????? ---????? 389.450 382.990 ( -6.460)????? --- >> ?? 7:?? 2694921227? 2630505090 (-64416137)????? -----??? 389.300 383.160 ( -6.140)????? --- >> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ----???? 388.370 381.060 ( -7.310)????? --- >> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ----???? 389.560 381.100 ( -8.460)????? ---- >> ? 10:?? 2689162942? 2635658943 (-53503999)????? ----???? 389.690 379.110 (-10.580)????? ----- >> ============================================================ >> ??????? 2689866989? 2637396210 (-52470778)????? ----???? 389.941 382.017 ( -7.924)????? ---- >> >> instr delta =??? -52470778??? -1.9507% >> >> time? delta =?????? -7.924 ms -2.0321% >> >> ? It will show much more improvement if compare with the default archive: >> >> ?????????? ? ? ? no patch??????????????????????? patch >> >> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >> >> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >> >> >> ? Tests: >> >> 1)jtreg on jdk/tools and hotspot/runtime local. >> >> 2) mach5 ter1,2 >> >> >> Thanks >> >> Yumin >> >> ? ` From yumin.qi at oracle.com Wed Aug 19 05:16:51 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 18 Aug 2020 22:16:51 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <9f1452b2-796a-1bf2-ab61-f0019d3192d3@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <2fc105ce-adaa-be64-fddf-c2f4b6293c54@oracle.com> <9f1452b2-796a-1bf2-ab61-f0019d3192d3@oracle.com> Message-ID: <7a3028db-2797-c38d-c334-ddd7274058b4@oracle.com> HI, Sundar, David ? Thanks. ? I have update the webrev at the same link: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ according Sundar's suggestion. ? Thanks. ? Yumin On 8/17/20 7:39 PM, sundararajan.athijegannathan at oracle.com wrote: > Hi David. > > Thanks. > > -Sundar > > On 18/08/20 8:04 am, David Holmes wrote: >> Hi Sundar, >> >> On 18/08/2020 12:25 pm, sundararajan.athijegannathan at oracle.com wrote: >>> Not a full review of fresh changes. But couple of comments: >>> >>> * src/hotspot/share/memory/lambdaFormInvokers.cpp and src/hotspot/share/memory/lambdaFormInvokers.hpp miss "Classpath exception" clause in the copyright header >> >> Hotspot files do not use the Classpath exception. That is for .java sources that will be"linked against" by applications. >> >> Cheers, >> David >> >>> * I had suggested a change GenerateJLIClassesPlugin.java in last round of review. That's still applicable. >>> >>> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068160.html >>> >>> -Sundar >>> >>> On 18/08/20 1:07 am, Yumin Qi wrote: >>>> Hi, Ioi >>>> >>>> ? Thanks for review/suggestion. I have updated the webrev at the following link: >>>> >>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>> >>>> ? Following changes done in this patch: >>>> >>>> ? 1) move regenerating holder classes into new added file: lambdaFormInvokers.[ch]pp >>>> >>>> ? "@lambda-form-invoker" tag only observed at jvm side. >>>> >>>> ? 2) remove InvokerBytecodeGeneratorHelper.java, move related functions to existing class, GenerateJLIClassesHelper.java >>>> >>>> ? 3) add tracing for SPECIES_RESOLVE, so archive more classes. >>>> >>>> ? 4) remove InvokerGenerateBytesException.java, using RuntimeException instead. >>>> >>>> ? 5) Changed make file to exclude the new added lamdaFormInvokers.cpp from the non-cds building list. >>>> >>>> ? 6) There is a bug in previous patch, jobject (typeArrayOop) should not be used directly for loading class since GC will move the java object. Fixed by copying the bytes from handle to C heap. >>>> >>>> ? 7) Tested with tier1,2 >>>> >>>> >>>> Thanks >>>> >>>> Yumin >>>> >>>> >>>> >>>> On 8/15/20 6:27 PM, Ioi Lam wrote: >>>>> On 8/15/20 6:19 PM, Ioi Lam wrote: >>>>>> To better capture what we're trying to do in this RFE, I've changed the RFE title (and the subject of this email thread) to >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8247536 >>>>>> Support for pre-generated java.lang.invoke classes in CDS static archive >>>>>> >>>>>> I also update the RFE Description to give an overview of the problem and the solution. >>>>>> >>>>>> The design document is also updated to reflect Yumin's latest implementation >>>>>> >>>>> >>>>> Oops, sent too quickly. The design doc's title is also updated: >>>>> >>>>> https://wiki.openjdk.java.net/display/HotSpot/Support+for+pre-generated+java.lang.invoke+classes+in+CDS+archive >>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: >>>>>>> Hi Yumin, >>>>>>> >>>>>>> I reviewed mostly the native code. Below are my comments: >>>>>>> >>>>>>> 1) classListParser.hpp >>>>>>> >>>>>>> 71?? bool??????????????? _lambda_format; >>>>>>> >>>>>>> The above name is too generic. How about _lambda_form or _is_lambda_form? >>>>>>> If you rename the above, please also rename the function which returns the above bool. >>>>>>> >>>>>>> 2) jvm.cpp >>>>>>> >>>>>>> 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line)) >>>>>>> >>>>>>> ignore -> ignored >>>>>>> >>>>>>> 3) jvm.hpp >>>>>>> >>>>>>> 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); >>>>>>> >>>>>>> Same comment as for jvm.cpp >>>>>>> >>>>>>> 4) metaspaceShared.cpp >>>>>>> >>>>>>> 2017?? size_t i = 0; >>>>>>> 2018?? while (i < size) { >>>>>>> 2019???? full_name[i++] = *start++; >>>>>>> 2020?? } >>>>>>> >>>>>>> Could the above be simplified to the following? >>>>>>> >>>>>>> ??? strncpy(full_name, start, size - 1); >>>>>>> >>>>>>> 2029?? char* class_name = get_full_class_name(path_name); >>>>>>> >>>>>>> Should os::free(class_name) be called before the function returns? >>>>>>> >>>>>>> 1870 static GrowableArray* lambda_list = NULL; >>>>>>> >>>>>>> The name lambda_list is a bit generic, how about lambda_form_list? >>>>>>> >>>>>>> 2112 lambda_list->append(parser.current_line()); >>>>>>> >>>>>>> Since parser.current_line() does a strdup, do those buffer need to be freed after its use? (i.e. after the call to regenerate_holder_classes()?) >>>>>>> >>>>>>> In MetaspaceShared::regenerate_holder_classes, before calling up to java, I think it's better to strip the prefix "@lambda-form-invoker" from the strings. So that the java InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need to handle it: >>>>>>> >>>>>>> ?143???????????????????? case "[LF_RESOLVE]": >>>>>>> ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: >>>>>>> >>>>>>> 5) DumpSymbolAndStringTable.java >>>>>>> >>>>>>> 37???? private static final String my_string = "DumpSymbolAndStringTable"; >>>>>>> >>>>>>> Unused variable? >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Calvin >>>>>>> >>>>>>> On 8/11/20 10:36 AM, Yumin Qi wrote: >>>>>>>> Forget to send to @core-lib-dev, the patch changed jdk code. >>>>>>>> >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> Yumin >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -------- Forwarded Message -------- >>>>>>>> Subject:???? RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS >>>>>>>> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >>>>>>>> From:???? Yumin Qi >>>>>>>> To:???? hotspot-runtime-dev at openjdk.java.net >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hi, Please reivew >>>>>>>> >>>>>>>> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >>>>>>>> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >>>>>>>> >>>>>>>> ? Summary: CDS does not archive pre-generated lambda form classes for method handles: >>>>>>>> >>>>>>>> [0.142s][info][class,load] java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: __JVM_LookupDefineClass__ >>>>>>>> >>>>>>>> The method handle resolution if not found in the holder class, a class with the method will be generated and loaded as vm internal created class and not archived like above. Those class names are mangled as combination of the class name and the class instance address. >>>>>>>> >>>>>>>> In this patch, collect those method holder class names and their associated methods' signatures when user specify DumpLoadedClassList, and record them in the log file in a format similar to the output format from traced by -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use another name for CDS: -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The line prefix also changed from "[LF_RESOLVE]" to "@lambda-invoke-handle".? At dump time, regenerate the holder class with those methods and replace the existing holder class and archived it. At runtime, the resolution of the holder class which contains those methods are loaded from CDS archive so avoid regenerating them again. The patch reorganized the code for Jlink plugin, so the new added InvokerBytecodeGeneratorHelper can be shared both by JLink plugin and CDS code. Also change made to the mangled generated class name at static dump, the class name combining the class name and a sequentially >>>>>>>> ordered number instead of the class instance address, which looks like a random. >>>>>>>> >>>>>>>> >>>>>>>> To use this feature, one can do the dump/run just like done for a static dump/run, so no extra steps required. >>>>>>>> >>>>>>>> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >>>>>>>> >>>>>>>> ????? DumpLoadedClassList will turn on -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >>>>>>>> >>>>>>>> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist -XX:SharedArchiveFile=my.jsa JavaApp >>>>>>>> >>>>>>>> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >>>>>>>> >>>>>>>> ? The performance on javac HelloWorld.java (javac-benchmark), no patch vs patch: >>>>>>>> >>>>>>>> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ---- 391.720 382.990 ( -8.730)????? ---- >>>>>>>> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ---- 391.030 381.480 ( -9.550)????? ----- >>>>>>>> ?? 3:?? 2694664066? 2636523114 (-58140952) ----- 390.610 382.550 ( -8.060)????? ---- >>>>>>>> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ---- 390.700 383.390 ( -7.310)????? --- >>>>>>>> ?? 5:?? 2691072338? 2633016687 (-58055651) ----- 388.990 382.360 ( -6.630)????? --- >>>>>>>> ?? 6:?? 2684448174? 2644191854 (-40256320)????? --- 389.450 382.990 ( -6.460)????? --- >>>>>>>> ?? 7:?? 2694921227? 2630505090 (-64416137) ----- 389.300 383.160 ( -6.140)????? --- >>>>>>>> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ---- 388.370 381.060 ( -7.310)????? --- >>>>>>>> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ---- 389.560 381.100 ( -8.460)????? ---- >>>>>>>> ? 10:?? 2689162942? 2635658943 (-53503999)????? ---- 389.690 379.110 (-10.580)????? ----- >>>>>>>> ============================================================ >>>>>>>> ??????? 2689866989? 2637396210 (-52470778)????? ---- 389.941 382.017 ( -7.924)????? ---- >>>>>>>> >>>>>>>> instr delta =??? -52470778??? -1.9507% >>>>>>>> >>>>>>>> time? delta =?????? -7.924 ms -2.0321% >>>>>>>> >>>>>>>> ? It will show much more improvement if compare with the default archive: >>>>>>>> >>>>>>>> ?????????? ? ? ? no patch patch >>>>>>>> >>>>>>>> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >>>>>>>> >>>>>>>> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >>>>>>>> >>>>>>>> >>>>>>>> ? Tests: >>>>>>>> >>>>>>>> 1)jtreg on jdk/tools and hotspot/runtime local. >>>>>>>> >>>>>>>> 2) mach5 ter1,2 >>>>>>>> >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> Yumin >>>>>>>> >>>>>>>> ? ` >>>>>> >>>>> From thomas.stuefe at gmail.com Wed Aug 19 07:43:56 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 19 Aug 2020 09:43:56 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: <8bd366a3-a40b-c2d6-8e19-1abf7a4c7e04@oracle.com> References: <8bd366a3-a40b-c2d6-8e19-1abf7a4c7e04@oracle.com> Message-ID: Hi Coleen, On Wed, Aug 19, 2020 at 12:19 AM Coleen Phillimore < coleen.phillimore at oracle.com> wrote: > > Hi Thomas, > > I've read through the new implementation files. This looks really > good! These are my initial comments. There aren't many and nothing of > substance really. Some files need more in-depth review though. I > haven't looked at the tests, which are more difficult. I'll do that > tomorrow. > > General comment: Some files have a lot of vertical white space, like > two or more blank lines together: eg. metachunk.cpp at the end. More > about that below. > A bad habit of mine, idly pressing space white thinking. I'll fix those. > > === counter.hpp > > This seems like a general utility that should be in utilities so that it > can be generalized for use in other places. I didn't review it > carefully to suggest it go there right now, but we should file another > RFE to do that. Kim has some ideas about it. > I agree. I kept this class here to keep the patch confined to memory/metaspace. I would like cleanups like this as follow ups to the main patch, as you suggested. I created https://bugs.openjdk.java.net/browse/JDK-8252014 to keep track of this. > === freeBlocks.hpp > > I like how the small blocks and tree of freed blocks are together > (except see below). > This has a really good comment. > Thanks. > > #ifndef SHARE_MEMORY_METASPACE_LEFTOVERBINS_HPP > #define SHARE_MEMORY_METASPACE_LEFTOVERBINS_HPP > Wrong ifdef guard names. > #endif // SHARE_MEMORY_METASPACE_CHUNKMANAGER_HPP > > Ok > === binList.hpp > > block_t* _v[num_lists]; > > Some slightly longer name would be better than _v. Maybe _small_blocks > or something like that? > > Would _blocks be acceptable? Since the distinguisher 'small' makes no sense here, it is the only member in the class. > // These aren't used > typedef BinListImpl<2, 8> BinList8; > typedef BinListImpl<2, 16> BinList16; > typedef BinListImpl<2, 64> BinList64; > > This file should be binList.hpp rather than binlist.hpp. > > Ok > === blockTree.hpp/cpp > > I didn't review this code. Someone suggested using std::map instead but > we're not using std:: yet. > > I don't really like that idea. Using stl maps for structures as these in-place blocks seems, depending on how it is done, either risky or wasteful. In this tree payload and nodes are identical and the size of the payload is the key. Like the Linux rbtree, really. STL map keeps these entities separate: a KV pair holds K (block size) and V (the block). So we spent additional memory and allocation to keep the pair itself, and K is also kept redundant (since the block already knows its size). This is if we use stl map safely-but-naively. I guess one could get stl::map to behave exactly as the Linux rb tree - taking KV pair allocation into our own hand and somehow eliminating the K storage redundancy. But I would think we end up with a solution way less maintainable than what I have suggested. You'd need intimate knowledge of stl map to make sure we this works, and across all platforms and all STL implementations. I'd think maintaining this little tree thing is cheaper. blocktree.hpp/cpp should be blockTree.hpp/cpp > > Since for class metaspace, the reclaimed InstanceKlass are generally > bigger than the largest BinList size, maybe class type MetaspaceArenas > should only have a blockTree and not a SmallBlocks, so not the whole > freeList.hpp structure. I think we could do some footprint analysis > with NMT and see if this might help. It's fairly rare unless you have a > lot of redefinition. I don't think this should be changed in this > change though. > > Oh good point. I would not like to hard-code the assumption that class space only contains Klass though; I dimly remember an RFR with Ioi where he tried storing other things there as well. IIRC he found some other solution. But my point is that this assumption is fragile. But a "soft assumption" would be fine. E.g. creating the BinList on the fly, only when needed, and separately from the blocktree. Easy to implement too. We don't need NMT to analyze. BinList currently has 32 slots, + counter + mask. We pay 272 bytes per structure, or 34 words. Which we would save per CLD. === metaspaceContext.cpp > > #define LOGFMT "SpcMgr @" PTR_FORMAT " (%s)" > #define LOGFMT_ARGS p2i(this), this->_name > > These don't seem to be used and is SpcMgr a reference to the old > SpaceManager? Correction, I see how it's used but you should change it > to not refer to Space manager. > I'll fix that. > > // Destroys the context: deletes chunkmanager and virtualspacelist. > // If this is a non-expandable context over an existing space, that > space remains > // untouched, otherwise all memory is unmapped. > MetaspaceContext::~MetaspaceContext() { > delete _cm; > delete _vslist; > } > > Why would you ever do this? I thought the two contexts were global. If > it's for the tests, add a comment here. > Yes, it only happens when testing. The rest is future thoughts, in case we ever use a metaspace context for other purposes (I am currently experimenting with using them as Arena replacement). I will adapt the comment. > === metachunk.cpp > > class UnsortedMetachunkList : public AbstractMetachunkList { > > This class is declared in a strange place (in a function) and appears > unused. check_pattern() appears unused which is good because it's a lot > of code. > > My god how did this get there. How did this ever compile. This is an accidental paste from some code move I did before posting the review. And yes, seems both fill_with_pattern and check_pattern are unused. Remnants from some earlier version. I'll remove both. > > === metaspace_test.hpp/cpp > > I keep thinking this file is in the wrong place. Can you rename > metaspaceTestHelper? or some name like that? > Will do. These classes must live somewhere in metaspace since they are used by whitebox.cpp. I could move them there directly. But I wanted to keep them separate. > > === internStats.hpp > > Indentation is inconstent and too much vertical white space at the end. > Should be renamed internalStats.hpp/cpp to match the class name. This > looks like a nice improvement. One of the things that made metaspace > code hard to read was that there was a lot of code for statistics mixed > in everywhere. > I agree. > > === arenaGrowthPolicy.hpp/cpp > > I like this! > > :) > === chunkLevel.hpp/cpp > > I'm fine with the nested namespace chunklevel, rather than an AllStatic > class ChunkLevel. The names of the files should probably be changed to > chunklevel.hpp/cpp though. > > Ok. > > === metaspaceEnums.hpp/cpp > > This set of files seems like they are misnamed and should be > encorporated into metaspace.hpp, since they are used outside of the > memory/metaspace directory. > > I agree, this seems odd. Will move as suggested. > === settings.hpp > > Include guards are wrong names. > > #ifndef SHARE_MEMORY_METASPACE_CONSTANTS_HPP > #define SHARE_MEMORY_METASPACE_CONSTANTS_HPP > > #endif // SHARE_MEMORY_METASPACE_BLOCKFREELIST_HPP > > I actually took the allocation path when looking at this. Thank you for > keeping many of the function names because that helped. > > thanks, > Coleen > Thanks for the review, Coleen. I will fix and post an update. I'll have vacation the rest of the week and will post an updated webrev start of next week, with your changes and some other small fish I did. Cheers, Thomas From martin.doerr at sap.com Wed Aug 19 09:18:55 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 19 Aug 2020 09:18:55 +0000 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: <17f9e7d8-d5fa-d4c6-8846-7c57f026cacb@oss.nttdata.com> References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> <08584797-0841-945e-ec6d-7050a12326b4@oracle.com> <0facc9ee-eb47-a40c-4064-75d648b48eab@oss.nttdata.com> <145b716c-54ee-b32a-d008-a4ca3e7359db@oracle.com> <17f9e7d8-d5fa-d4c6-8846-7c57f026cacb@oss.nttdata.com> Message-ID: Hi Yasumasa, > > I still wonder what the reason was for using a stub. There was probably a > reason, but I don't remember. > > It might be called in many times, so I made change would generate stub in > first. I understand that generating a stub once is better than every time. That was not my point. What I wonder about is that we need to generate code in the code cache for this at all. The old implementation uses inline assembler which probably works everywhere except on Windows. And on Windows we were generating code instead of using the __cpuid intrinsic. I don't remember why we didn't use __cpuid which would have been my preferred implementation. (This intrinsic is already used for other purposes in orderAccess_windows_x86.hpp.) Maybe Matthias can remember if there was an issue with it. Maybe we were using a very old Visual Studio compiler at that point of time. Best regards, Martin > -----Original Message----- > From: Yasumasa Suenaga > Sent: Mittwoch, 19. August 2020 03:33 > To: Doerr, Martin ; David Holmes > ; Baesken, Matthias > ; hotspot-runtime-dev at openjdk.java.net > Subject: Re: PING: RFR: 8250598: Hyper-V is detected in spite of running on > host OS > > Hi Martin, > > On 2020/08/18 23:39, Doerr, Martin wrote: > > Hi Yasumasa, > > > > thanks a lot for measuring and improving the overhead. This version looks > much better. > > > > I still wonder what the reason was for using a stub. There was probably a > reason, but I don't remember. > > It might be called in many times, so I made change would generate stub in > first. > > VM_Version::check_virtualizations() would call CPUID 256 times (0x40000000 > to 0x40010000 step 0x100), > and also I changed it is used to call CPUID with EAX = 0x40000007 in Hyper-V. > > > > Hopefully, Matthias can find time for another review when he's back next > week. > > Yeah, I want to commit this change after his review. > > > Thanks, > > Yasumasa > > > > Thanks and best regards, > > Martin > > > > > >> -----Original Message----- > >> From: Yasumasa Suenaga > >> Sent: Dienstag, 18. August 2020 05:03 > >> To: David Holmes ; Baesken, Matthias > >> ; hotspot-runtime-dev at openjdk.java.net; > >> build-dev at openjdk.java.net; Doerr, Martin > >> Subject: Re: PING: RFR: 8250598: Hyper-V is detected in spite of running > on > >> host OS > >> > >> On 2020/08/18 10:42, David Holmes wrote: > >>> Hi Yasumasa, > >>> > >>> On 14/08/2020 12:36 pm, Yasumasa Suenaga wrote: > >>>> Hi David, Matthias, Martin, > >>>> > >>>> I uploaded new webrev. Could you review again? > >>>> > >>>> ?? http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.02/ > >>>> > >>>> It generates runtime stub for hypervisor detection, and it can > distinguish > >> between Hyper-V host (root partition) and guest. And also it works on > 32bit > >> platforms. > >>>> > >>>> I measured startup performance (java --version) with Measure- > Command > >> on PowerShell, this change is faster than current implementation. > >>>> > >>>> TotalMilliseconds : 57.8196 > >>>> TotalMilliseconds : 66.8379 > >>>> TotalMilliseconds : 64.7693 > >>>> TotalMilliseconds : 55.6546 > >>>> TotalMilliseconds : 63.848 > >>>> average : 61.78588 > >>> > >>> This seems good to me, but again I can't review the details. My main > >> concern is how you are testing this across a range of virtualized hosts? But > if it > >> fixes your Hyper-V issue and doesn't break anything else then I am fine > with > >> it. > >> > >> Thanks David! > >> > >> I don't have any other virtualized hosts excepting Hyper-V, so I cannot > test > >> any more. > >> However detection logic is equivalent from current implementation, so I > >> believe it does not break anything else if it works fine on Hyper-V. > >> > >> And also Matthias has queued this change to internal tests in SAP. I guess > he > >> will share the result next week. > >> > >> > >>> I will leave the technical review to Martin and Matthias. > >>> > >>> FYI I'm heading out on a weeks vacation after today so won't be able to > >> follow up further. > >> > >> Have a nice vacation! > >> > >> > >> Thanks, > >> > >> Yasumasa > >> > >> > >>> Thanks, > >>> David > >>> > >>>> This change has passed tests on submit repo (mach5-one-ysuenaga- > JDK- > >> 8250598-20200814-0119-13424118), and also it works fine on following > >> environments: > >>>> > >>>> ?? - Hyper-V host (Windows 10 x64) > >>>> ?? - Hyper-V guest (Windows 10 x64) > >>>> ?? - Hyper-V guest (Linux x64) > >>>> ?? - Hyper-V guest (32bit JDK on Linux x64) > >>>> > >>>> > >>>> Thanks, > >>>> > >>>> Yasumasa > >>>> > >>>> > >>>> On 2020/08/13 20:39, David Holmes wrote: > >>>>> On 13/08/2020 5:39 pm, Yasumasa Suenaga wrote: > >>>>>> Hi Matthias, David, > >>>>>> > >>>>>> On 2020/08/13 15:52, Baesken, Matthias wrote: > >>>>>>> > >>>>>>>> > >>>>>>>> Should we make the change to determine just before it is needed > >> (e.g. VM.info or hs_err log) at first? > >>>>>>>> It is out of scope of this change, so I want to work as another issue > if > >> it is needed. > >>>>>>>> > >>>>>>> > >>>>>>> Hi , > >>>>>>> I think we better do not call into WMI , when? writing an hs_err > file?? . > >>>>>> > >>>>>> I found out from Hypervisor Top Level Functional Specification [1] > >> section 2.4.8. > >>>>>> That says CPUID leaf 0x40000007 is available to the root partition > only. > >>>>>> > >>>>>> I changed VM_Version::is_in_VM() as below, and it works fine. > >>>>>> (I use __cpuid intrinsic in here because this code would be compiled > by > >> cl.exe only.) > >>>>>> > >>>>>> ``` > >>>>>> #include > >>>>>> > >>>>>> #define EAX 0 > >>>>>> #define EBX 1 > >>>>>> #define ECX 2 > >>>>>> #define EDX 3 > >>>>>> > >>>>>> bool VM_Version::is_in_VM() { > >>>>>> ?? int regs[4]; > >>>>>> ?? __cpuid(regs, 0x40000007); > >>>>>> > >>>>>> ?? // CPUID leaf 0x40000007 is available to the root partition only. > >>>>>> ?? return (regs[EAX] == 0x0) && (regs[EBX] == 0x0) && (regs[ECX] == > >> 0x0) && (regs[EDX] == 0x0); > >>>>>> } > >>>>>> ``` > >>>>>> > >>>>>> Also startup performance is better than WMI call. > >>>>>> > >>>>>> TotalMilliseconds : 66.7863 > >>>>>> TotalMilliseconds : 56.8123 > >>>>>> TotalMilliseconds : 57.0015 > >>>>>> TotalMilliseconds : 62.4755 > >>>>>> TotalMilliseconds : 67.7632 > >>>>>> average : 62.16776 > >>>>>> > >>>>>> If you are ok this change, I will update webrev, and will send review > >> request. > >>>>>> (new webrev will include both loop for CPUID leaf and renaming > >> is_in_VM) > >>>>> > >>>>> If it works and addresses the startup hit then I'm fine with it - but still > >> can't actually review the code. > >>>>> > >>>>> Thanks, > >>>>> David > >>>>> > >>>>>> > >>>>>> Thanks, > >>>>>> > >>>>>> Yasumasa > >>>>>> > >>>>>> > >>>>>> [1] https://github.com/MicrosoftDocs/Virtualization- > >> > Documentation/raw/master/tlfs/Hypervisor%20Top%20Level%20Functional > >> %20Specification%20v6.0b.pdf > >>>>>> > >>>>>> > >>>>>>> Best regards, Matthias > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: Yasumasa Suenaga > >>>>>>> Sent: Donnerstag, 13. August 2020 06:15 > >>>>>>> To: David Holmes ; Baesken, Matthias > >> ; hotspot-runtime-dev at openjdk.java.net; > >> build-dev at openjdk.java.net > >>>>>>> Cc: Doerr, Martin > >>>>>>> Subject: Re: PING: RFR: 8250598: Hyper-V is detected in spite of > >> running on host OS > >>>>>>> > >>>>>>> On 2020/08/13 11:54, David Holmes wrote: > >>>>>>>> On 13/08/2020 11:12 am, Yasumasa Suenaga wrote: > >>>>>>>>> Hi Matthias, David, > >>>>>>>>> > >>>>>>>>> I measured startup benchmarks with `Measure-Command > >> {.\jdk\build\windows-x86_64-server-release\images\jdk\bin\java.exe -- > >> version}` on PowerShell. > >>>>>>>>> > >>>>>>>>> * PC: Ryzen 3 3300X, 16GB memory > >>>>>>>>> * OS: Windows 10 x64 (May 2020 Update) > >>>>>>>>> * Java: jdk/jdk revision 60537 > >>>>>>>>> ????? (Compiled by VS 2019 (16.7.0)) > >>>>>>>>> > >>>>>>>>> * without patch > >>>>>>>>> TotalMilliseconds : 70.2124 > >>>>>>>>> TotalMilliseconds : 64.4465 > >>>>>>>>> TotalMilliseconds : 59.0854 > >>>>>>>>> TotalMilliseconds : 68.0255 > >>>>>>>>> TotalMilliseconds : 72.6467 > >>>>>>>>> average : 66.8833 > >>>>>>>>> > >>>>>>>>> * with webrev.01 > >>>>>>>>> TotalMilliseconds : 81.7185 > >>>>>>>>> TotalMilliseconds : 68.539 > >>>>>>>>> TotalMilliseconds : 85.7226 > >>>>>>>>> TotalMilliseconds : 72.6584 > >>>>>>>>> TotalMilliseconds : 75.6091 > >>>>>>>>> average : 76.84952 > >>>>>>>>> > >>>>>>>>> Overhead of WMI seems to be +10ms in this case. > >>>>>>>> > >>>>>>>> Which is nearly 15%! Sorry but I just know Claes will be very > unhappy > >> if this were to go in as-is. > >>>>>>> > >>>>>>> Should we make the change to determine just before it is needed > >> (e.g. VM.info or hs_err log) at first? > >>>>>>> It is out of scope of this change, so I want to work as another issue > if it > >> is needed. > >>>>>>> > >>>>>>> > >>>>>>> Yasumasa > >>>>>>> > >>>>>>> > >>>>>>>> David > >>>>>>>> ----- > >>>>>>>> > >>>>>>>>> > >>>>>>>>> Yasumasa > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On 2020/08/13 0:05, Baesken, Matthias wrote: > >>>>>>>>>>> I understand that if the process runs on Xen on other > hypervisor > >> (e.g. KVM), information for Xen would be set between 0x40000100 and > >> 0x40010000. > >>>>>>>>>>> Ok, I will not remove the loop in new webrev, and will add > >> comment about it. > >>>>>>>>>> > >>>>>>>>>> Hi Yasumasa? , thanks ! > >>>>>>>>>> > >>>>>>>>>> Regarding the WMI overhead , if you could? get some more info > >> on this it would be better to judge if it is perfectly okay or concerning . > >>>>>>>>>> > >>>>>>>>>> (I remember that I looked into it a couple of years ago , but > >> decided not to use it in early VM startup ;? but have to confess this was > just > >> based on a "gut feeling", > >>>>>>>>>> ?? No micro benchmarks? ) > >>>>>>>>>> > >>>>>>>>>> Best regards, Matthias > >>>>>>>>>> > >>>>>>>>>> From thomas.stuefe at gmail.com Wed Aug 19 10:18:19 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 19 Aug 2020 12:18:19 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: <8bd366a3-a40b-c2d6-8e19-1abf7a4c7e04@oracle.com> References: <8bd366a3-a40b-c2d6-8e19-1abf7a4c7e04@oracle.com> Message-ID: > I > haven't looked at the tests, which are more difficult. I'll do that > tomorrow. > I noticed while looking at the gtest files that I forgot to remove a dead file pair, remnants from a code reshuffle: test/hotspot/gtest/metaspace/metaspace_testhelper.hpp/cpp. Please ignore those. I will remove them. Cheers, Thomas From gerard.ziemski at oracle.com Wed Aug 19 16:53:45 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Wed, 19 Aug 2020 11:53:45 -0500 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: <2BAD9BBF-B8F9-45E7-8378-E69622856C7A@oracle.com> References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> <95b46d0f-8a81-10c6-78bb-a3abbd7cb955@oracle.com> <55ae5763-476b-562c-c7cc-a913992d304c@oracle.com> <4b9dfeda-c6f7-3ee5-fb23-85cf58f1130a@oracle.com> <2BAD9BBF-B8F9-45E7-8378-E69622856C7A@oracle.com> Message-ID: <44706c40-d3f2-6b63-1491-c4412a0c4347@oracle.com> hi Igor, Thank you for the review. Here is the webrev, but please see my comments inline for more: bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev4 testing: Mach5 hs-tier1,2,3,4,5 in progress... On 8/18/20 12:18 PM, Igor Ignatyev wrote: > Hi Gerard, > > looks be > 1. >> 26 * @bug 8222671 > @bug tag should refer to a product bug which can be reproduced w/ this test, not the bug id used to push a test. The bug 8222671 is the issue here, i.e. timeouts in the existing test, which is what we are trying to fix, so we have a new test that replaces the old and less robust one. I did try to find an issue that originated the old test and I couldn?t, but if you know how to find, I can add a reference to it in the new test. To me, the old test looks more like a general functionality test though. > 2. >> 28 * @run main/othervm ThreadCountLimit > what's the reason to run in othervm mode? Oh, I just copied that from the existing test, without giving it any thought - I will remove it. > 3. >> 34 public class ThreadCountLimit extends Object { > you don't need to explicitly extend Object. True, but I like the code explicit like that if that?s OK. > 4. in ThreadCountLimit::run() >> 67 } catch (InterruptedException ex) { >> 68 } > shouldn't this mark the test as failed? Good question. In a perfect world we would want to know if a thread got unexpectedly interrupted, but realistically a process might (will?) exhibit spurious interruptions from the OS sooner or later, which are not the focus of this test, so instead of marking it a failure and making someone look into it, only later to determine that it was just a spurious interrupt, we simply ignore it and focus on the core goal of this test instead. I do see, however, that in one place I report it as Error and ignore it in the other instance, so I fixed that inconsistency. I added comments as well. > Thanks, > -- Igor > >> On Aug 18, 2020, at 7:38 AM, gerard ziemski wrote: >> >> Thank you David for the review! >> >> I incorporated all your feedback. >> >> I'm surprised that "assert" is not available by default in our tests though - what is a reason for that? Of all the places where asserts should be available, a testing suite would seem to be it to me. >> >> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev3 >> testing: passes locally, Mach5 hs-tier1,2,3,4,5 in progress... >> >> >> cheers >> >> On 8/18/20 12:15 AM, David Holmes wrote: >>> Hi Gerard, >>> >>> On 18/08/2020 1:28 am, gerard ziemski wrote: >>>> Thank you David and Igor for the feedback! >>>> >>>> I rewrote the test again, this time basing it off test/hotspot/jtreg/runtime/Thread/* tests. >>>> >>>> For macOS and Linux we create threads until we hit a process limit thread count. Windows, however, allows unlimited (?) number of threads, though the process gradually slows down as the count grows, so I had to add time limit and I set it to 5 sec. I chose 5 sec because the value is well below jtreg time out (i.e. 120s), but still quite above how long my laptop needs to hit the process thread count limit (i.e. my Mac laptop needs about 1 sec to hit that limit). >>>> >>>> Otherwise there are no other hardwired values or assumptions (I hope). What do you think of the test now? >>> Not unreasonable :) It could still have tried for an upper limit of 10000 instead of using the time-limit, but it makes no real difference. >>> >>> A few nits/suggestions: >>> >>> Please add an @bug line to the test comment. >>> >>> 49 if ((this.index % 250) == 0) { >>> >>> Outside of the constructor it is poor style to use this.x to refer to fields methods (unless needed due to local variable shadowing). This applies throughout the test. >>> >>> 54 doWork(); >>> >>> Not sure why you factored things out the way you did - everything could go in run(). >>> >>> 61 setName(""+this.index); >>> >>> This could go in constructor, but I presume it adds something to the workload? Is the use of ""+index deliberate in that regard (generate string concatenation work), rather than using String.valueOf(index)? >>> >>> 65 assert(this.index == Integer.parseInt(getName())); >>> >>> We never run tests with Java asserts enabled so this is "dead" code. >>> >>> 87 // Windows path >>> >>> Windows path or system with very large ulimit? >>> >>> 92 } catch (OutOfMemoryError e) { >>> >>> I suggest checking this is actually the "unable to create native thread" OOME rather than a general Java OOME (if run with a small heap for example you might get OOME growing the ArrayList). >>> >>> 110 String cmd = System.getProperty("sun.java.command"); >>> 111 if (cmd != null && !cmd.startsWith("com.sun.javatest.regtest.agent.MainWrapper")) { >>> 112 // Exit with success in a non-JavaTest environment: >>> 113 System.exit(0); >>> 114 } >>> >>> I never noticed when this was used in the little set of runtime/Thread tests! I don't know what the rationale was but it is completely unnecessary as a normal return from main would be the equivalent of exit(0) - so the tests work fine without System.exit when run standalone. Please don't emulate this part of the existing tests. I'll file a RFE to remove it from the other tests. >>> >>> I won't be around for any further follow up for a week, so feel free to proceed once Igor is okay with things. >>> >>> Thanks, >>> David >>> ----- >>> >>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev2 >>>> testing: Mach5 hs-tier1,2,3,4,5 in progress... >>>> >>>> >>>> >>>> >>>> On 8/7/20 2:41 PM, Igor Ignatyev wrote: >>>>> >>>>>> On Aug 6, 2020, at 12:33 AM, David Holmes > wrote: >>>>>> >>>>>> Basing the new test around the existing thread stress tests was not unreasonable but I don't think we're actively adding to those tests. I would check with Igor whether it is better to just add this under runtime/thread rather than vmTestBase. >>>>> Hi Gerard, >>>>> >>>>> David is right, we are trying to get rid of vmTestbase directory and move all tests into corresponding component directories cleaning them up at the same time. so runtime/thread directory sound like a much better location for a new test. >>>>> >>>>> I don't think you need to cary all this baggage of 'VM testbase keywords', 'converted from ...' and the rest, I'd suggest you just use the text from 'DESCRIPTION' as @summary. I'd also remove System.exit call and would just make thread009::run to return bool. it's also unclear to me if someone is ever going to change THREADS_EXPECTED , TIMEOUT, and DEBUG_MODE values, I'd just hardcode them in the test and get rid of parsing. it seems that debug output won't clutter output, so you might want to always print them and remove DEBUG_MODE field. >>>>> >>>>> using `assert` in test code isn't a good idea as it will work only if someone runs test w/ `-ea` which not all people do. it's better to replace w/ `if (p) throw new Error(...)` or use jdk.test.lib.Asserts class. >>>>> >>>>> Cheers, >>>>> -- Igor From gerard.ziemski at oracle.com Wed Aug 19 17:00:09 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Wed, 19 Aug 2020 12:00:09 -0500 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: <585124BE-BDE7-496B-8A1F-B4E345DC5411@oracle.com> References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> <95b46d0f-8a81-10c6-78bb-a3abbd7cb955@oracle.com> <55ae5763-476b-562c-c7cc-a913992d304c@oracle.com> <585124BE-BDE7-496B-8A1F-B4E345DC5411@oracle.com> Message-ID: On 8/18/20 12:04 PM, Igor Ignatyev wrote: > Dan/Gerard, > > we use '-ea -esa' in some of our configurations, yet not in all configurations and not all people use these configurations when they run tests. also some people run tests directly thru jtreg (or even w/o jtreg), hence adding it into make won't really help. therefore the tests shouldn't depend on -ea/-esa. > > generally speaking, `assert` makes sense only for product code where you might want some checks to be disabled in production runs, for tests you really want all checks to be always done. Very true, but: assert(condition, "message") is so much nicer to write than: if (!condition) { ? throw new Exception("message"); } And when running locally one could turn them OFF/ON to help debug an issue. cheers > > -- Igor > > >> On Aug 18, 2020, at 6:31 AM, Daniel D. Daugherty wrote: >> >> This caught my eye: >> >>> We never run tests with Java asserts enabled so this is "dead" code. >> I'm pretty certain that I've seen test tasks in Mach5 with enable assertion >> options passed. I don't have an example at my fingertips though... >> >> Dan >> >> >> On 8/18/20 1:15 AM, David Holmes wrote: >>> Hi Gerard, >>> >>> On 18/08/2020 1:28 am, gerard ziemski wrote: >>>> Thank you David and Igor for the feedback! >>>> >>>> I rewrote the test again, this time basing it off test/hotspot/jtreg/runtime/Thread/* tests. >>>> >>>> For macOS and Linux we create threads until we hit a process limit thread count. Windows, however, allows unlimited (?) number of threads, though the process gradually slows down as the count grows, so I had to add time limit and I set it to 5 sec. I chose 5 sec because the value is well below jtreg time out (i.e. 120s), but still quite above how long my laptop needs to hit the process thread count limit (i.e. my Mac laptop needs about 1 sec to hit that limit). >>>> >>>> Otherwise there are no other hardwired values or assumptions (I hope). What do you think of the test now? >>> Not unreasonable :) It could still have tried for an upper limit of 10000 instead of using the time-limit, but it makes no real difference. >>> >>> A few nits/suggestions: >>> >>> Please add an @bug line to the test comment. >>> >>> 49 if ((this.index % 250) == 0) { >>> >>> Outside of the constructor it is poor style to use this.x to refer to fields methods (unless needed due to local variable shadowing). This applies throughout the test. >>> >>> 54 doWork(); >>> >>> Not sure why you factored things out the way you did - everything could go in run(). >>> >>> 61 setName(""+this.index); >>> >>> This could go in constructor, but I presume it adds something to the workload? Is the use of ""+index deliberate in that regard (generate string concatenation work), rather than using String.valueOf(index)? >>> >>> 65 assert(this.index == Integer.parseInt(getName())); >>> >>> We never run tests with Java asserts enabled so this is "dead" code. >>> >>> 87 // Windows path >>> >>> Windows path or system with very large ulimit? >>> >>> 92 } catch (OutOfMemoryError e) { >>> >>> I suggest checking this is actually the "unable to create native thread" OOME rather than a general Java OOME (if run with a small heap for example you might get OOME growing the ArrayList). >>> >>> 110 String cmd = System.getProperty("sun.java.command"); >>> 111 if (cmd != null && !cmd.startsWith("com.sun.javatest.regtest.agent.MainWrapper")) { >>> 112 // Exit with success in a non-JavaTest environment: >>> 113 System.exit(0); >>> 114 } >>> >>> I never noticed when this was used in the little set of runtime/Thread tests! I don't know what the rationale was but it is completely unnecessary as a normal return from main would be the equivalent of exit(0) - so the tests work fine without System.exit when run standalone. Please don't emulate this part of the existing tests. I'll file a RFE to remove it from the other tests. >>> >>> I won't be around for any further follow up for a week, so feel free to proceed once Igor is okay with things. >>> >>> Thanks, >>> David >>> ----- >>> >>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev2 >>>> testing: Mach5 hs-tier1,2,3,4,5 in progress... >>>> >>>> >>>> >>>> >>>> On 8/7/20 2:41 PM, Igor Ignatyev wrote: >>>>> >>>>>> On Aug 6, 2020, at 12:33 AM, David Holmes > wrote: >>>>>> >>>>>> Basing the new test around the existing thread stress tests was not unreasonable but I don't think we're actively adding to those tests. I would check with Igor whether it is better to just add this under runtime/thread rather than vmTestBase. >>>>> Hi Gerard, >>>>> >>>>> David is right, we are trying to get rid of vmTestbase directory and move all tests into corresponding component directories cleaning them up at the same time. so runtime/thread directory sound like a much better location for a new test. >>>>> >>>>> I don't think you need to cary all this baggage of 'VM testbase keywords', 'converted from ...' and the rest, I'd suggest you just use the text from 'DESCRIPTION' as @summary. I'd also remove System.exit call and would just make thread009::run to return bool. it's also unclear to me if someone is ever going to change THREADS_EXPECTED , TIMEOUT, and DEBUG_MODE values, I'd just hardcode them in the test and get rid of parsing. it seems that debug output won't clutter output, so you might want to always print them and remove DEBUG_MODE field. >>>>> >>>>> using `assert` in test code isn't a good idea as it will work only if someone runs test w/ `-ea` which not all people do. it's better to replace w/ `if (p) throw new Error(...)` or use jdk.test.lib.Asserts class. >>>>> >>>>> Cheers, >>>>> -- Igor From gerard.ziemski at oracle.com Wed Aug 19 17:17:20 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Wed, 19 Aug 2020 12:17:20 -0500 Subject: RFR 8251490: [TESTBUG] The Java thread stack size specified is too small for nsk/stress/stack. Specify at least 448k In-Reply-To: <1e0f8292-5929-2356-d1b8-638443e9a4fd@oracle.com> References: <9f12fc32-646c-1aff-86fa-003db2641c91@oracle.com> <11880712-97f7-efee-2cd3-fb22edf89af9@oracle.com> <1e0f8292-5929-2356-d1b8-638443e9a4fd@oracle.com> Message-ID: <9466f8ce-7045-664b-4c3b-777c73e05076@oracle.com> hi Harold, On 8/18/20 2:54 PM, Harold Seigel wrote: > > Hi Gerard, > > Thanks for looking at this change.? Please see inline comments. > > On 8/18/2020 2:06 PM, gerard ziemski wrote: >> hi Harold, >> >> Thank you for fixing this. >> >> >> On 8/18/20 12:19 PM, Harold Seigel wrote: >>> Hi, >>> >>> Please review this small change to fix five >>> vmTestbase/nsk/stress/stack tests that are failing in tier5 on Linux >>> AArch64.? The change increases the stack size for three of the >>> tests.? Increasing the stack size of the other two tests could >>> result in timeouts on other platforms.? So, those tests were changed >>> to no longer run on AArch64. >>> >>> Open Webrev: >>> http://cr.openjdk.java.net/~hseigel/bug_8251490/webrev/index.html >>> >>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251490 >>> >>> The change was tested by running the tests as part of mach5 tier5 on >>> Windows, Mac, Linux x64, and Linux AArch64. >>> >>> Thanks, Harold >>> >> It looks reasonable to me, but I have some questions: >> >> #1 Does "-eager" really make much difference here? Do you have any >> timing numbers you can provide with and without that flag, just so >> that we know how much it helps? > > This change does not affect the use of "-eager".? "-eager" has been > part of the tests at least as far back as JDK-11.? The only affect of > running with "-eager" is to prevent the creation of a Terminator > thread that waits a designated amount of time before calling > System.exit().? Its purpose is described here in Terminator.java: > > /** > ?* Terminator is used to terminate a stress test with PASS exit status > ?* before the test is terminated as timed out (and so failed). > ?* > ?* Terminator class holds a thread which sleeps for the given amount > ?* of time, and then wakes up and executes System.exit() > ?* with the given exit status. That thread is daemon, so it doesn't > ?* prevent application from exiting once all its threads finish > ?* before it's time for termination. Appointing terminator in zero > ?* delay implies immediate exit(). > ?* > ?* There is a limitation: you may appoint no more than one terminator > ?* per application. > ?*/ > Sorry Harold, I did not mean to make you investigate this option. I thought this was something else (i.e. compile java methods instantly). >> >> #2 How did you determine the the 900s timeout? How long does it take >> for a regular, middle of the road machine to execute each test? Do we >> use any slow/low end machines in our testing that could take longer >> than this time out? (I'm worried about that hardcoded time out value >> if you can tell) > > I did not determine the 900s timeout.? That also has been in the tests > for at least as far back as JDK-11.? When working on JDK-8139875, I > ran the tests many times with different stack sizes both locally and > with mach5 and looked at the run times for the tests, making sure they > were all significantly below the timeout.? So far, the tests have run > as part of tier5 for almost two weeks without timing out. > > The purpose of this proposed change is provide stack size values that > will work on AArch64. > That part looks fine. > Do you think a new bug is needed to address the use of -eager and the > 900s timeout? > 900s is 15 minutes. On one hand it needs to be long enough for tests not to timeout, but if any of these test need to run for such long, then it seems excessive. But no, I don't think this needs to be investigated further if your fix solves the issue at hand. Thank you for taking the time to answer my questions. cheers > > Thanks, Harold > >> >> >> cheers >> From igor.ignatyev at oracle.com Wed Aug 19 17:25:40 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 19 Aug 2020 10:25:40 -0700 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: <44706c40-d3f2-6b63-1491-c4412a0c4347@oracle.com> References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> <95b46d0f-8a81-10c6-78bb-a3abbd7cb955@oracle.com> <55ae5763-476b-562c-c7cc-a913992d304c@oracle.com> <4b9dfeda-c6f7-3ee5-fb23-85cf58f1130a@oracle.com> <2BAD9BBF-B8F9-45E7-8378-E69622856C7A@oracle.com> <44706c40-d3f2-6b63-1491-c4412a0c4347@oracle.com> Message-ID: <62F99ACE-282A-47B8-A213-626108712F46@oracle.com> Hi Gerard, please see my comments inline. -- Igor > On Aug 19, 2020, at 9:53 AM, gerard ziemski wrote: > > hi Igor, > > Thank you for the review. Here is the webrev, but please see my comments inline for more: > > bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 > open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev4 > testing: Mach5 hs-tier1,2,3,4,5 in progress... > > > > On 8/18/20 12:18 PM, Igor Ignatyev wrote: >> Hi Gerard, >> >> looks be >> 1. >>> 26 * @bug 8222671 >> @bug tag should refer to a product bug which can be reproduced w/ this test, not the bug id used to push a test. > The bug 8222671 is the issue here, i.e. timeouts in the existing test, which is what we are trying to fix, so we have a new test that replaces the old and less robust one. I did try to find an issue that originated the old test and I couldn?t, but if you know how to find, I can add a reference to it in the new test. To me, the old test looks more like a general functionality test though. I'd just remove @bug tag, b/c from my point of view listing all test bugs which caused modification of the test in @bug is just useless and pollute metadata. others have other opinion though, so I don't insist on removal. > > >> 2. >>> 28 * @run main/othervm ThreadCountLimit >> what's the reason to run in othervm mode? > Oh, I just copied that from the existing test, without giving it any thought - I will remove it. in hotspot tests, we tend not to relay on implicit @run (as it's error prone) and always have explicit `@run`, that's to say, please add `@run main ThreadCountLimit` > > >> 3. >>> 34 public class ThreadCountLimit extends Object { >> you don't need to explicitly extend Object. > True, but I like the code explicit like that if that?s OK. fine w/ me, but it's kinda against regular java code guidelines and doesn't bring any information. >> 4. in ThreadCountLimit::run() >>> 67 } catch (InterruptedException ex) { >>> 68 } >> shouldn't this mark the test as failed? > Good question. In a perfect world we would want to know if a thread got unexpectedly interrupted, but realistically a process might (will?) exhibit spurious interruptions from the OS sooner or later, which are not the focus of this test, so instead of marking it a failure and making someone look into it, only later to determine that it was just a spurious interrupt, we simply ignore it and focus on the core goal of this test instead. I'm not sure what you are refer here by 'spurious interruptions', java threads must be interrupted only if someone interrupted them, all other behaviors are JVM bugs and are to be analyzed. given the goal of this test is to check how JVM behaves when lots of threads have been created and started, I see catching unexpected InterruptedException as a valid failure mode of this test. > > > I do see, however, that in one place I report it as Error and ignore it in the other instance, so I fixed that inconsistency. I added comments as well. > >> Thanks, >> -- Igor >> >>> On Aug 18, 2020, at 7:38 AM, gerard ziemski wrote: >>> >>> Thank you David for the review! >>> >>> I incorporated all your feedback. >>> >>> I'm surprised that "assert" is not available by default in our tests though - what is a reason for that? Of all the places where asserts should be available, a testing suite would seem to be it to me. >>> >>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev3 >>> testing: passes locally, Mach5 hs-tier1,2,3,4,5 in progress... >>> >>> >>> cheers >>> >>> On 8/18/20 12:15 AM, David Holmes wrote: >>>> Hi Gerard, >>>> >>>> On 18/08/2020 1:28 am, gerard ziemski wrote: >>>>> Thank you David and Igor for the feedback! >>>>> >>>>> I rewrote the test again, this time basing it off test/hotspot/jtreg/runtime/Thread/* tests. >>>>> >>>>> For macOS and Linux we create threads until we hit a process limit thread count. Windows, however, allows unlimited (?) number of threads, though the process gradually slows down as the count grows, so I had to add time limit and I set it to 5 sec. I chose 5 sec because the value is well below jtreg time out (i.e. 120s), but still quite above how long my laptop needs to hit the process thread count limit (i.e. my Mac laptop needs about 1 sec to hit that limit). >>>>> >>>>> Otherwise there are no other hardwired values or assumptions (I hope). What do you think of the test now? >>>> Not unreasonable :) It could still have tried for an upper limit of 10000 instead of using the time-limit, but it makes no real difference. >>>> >>>> A few nits/suggestions: >>>> >>>> Please add an @bug line to the test comment. >>>> >>>> 49 if ((this.index % 250) == 0) { >>>> >>>> Outside of the constructor it is poor style to use this.x to refer to fields methods (unless needed due to local variable shadowing). This applies throughout the test. >>>> >>>> 54 doWork(); >>>> >>>> Not sure why you factored things out the way you did - everything could go in run(). >>>> >>>> 61 setName(""+this.index); >>>> >>>> This could go in constructor, but I presume it adds something to the workload? Is the use of ""+index deliberate in that regard (generate string concatenation work), rather than using String.valueOf(index)? >>>> >>>> 65 assert(this.index == Integer.parseInt(getName())); >>>> >>>> We never run tests with Java asserts enabled so this is "dead" code. >>>> >>>> 87 // Windows path >>>> >>>> Windows path or system with very large ulimit? >>>> >>>> 92 } catch (OutOfMemoryError e) { >>>> >>>> I suggest checking this is actually the "unable to create native thread" OOME rather than a general Java OOME (if run with a small heap for example you might get OOME growing the ArrayList). >>>> >>>> 110 String cmd = System.getProperty("sun.java.command"); >>>> 111 if (cmd != null && !cmd.startsWith("com.sun.javatest.regtest.agent.MainWrapper")) { >>>> 112 // Exit with success in a non-JavaTest environment: >>>> 113 System.exit(0); >>>> 114 } >>>> >>>> I never noticed when this was used in the little set of runtime/Thread tests! I don't know what the rationale was but it is completely unnecessary as a normal return from main would be the equivalent of exit(0) - so the tests work fine without System.exit when run standalone. Please don't emulate this part of the existing tests. I'll file a RFE to remove it from the other tests. >>>> >>>> I won't be around for any further follow up for a week, so feel free to proceed once Igor is okay with things. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>>>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev2 >>>>> testing: Mach5 hs-tier1,2,3,4,5 in progress... >>>>> >>>>> >>>>> >>>>> >>>>> On 8/7/20 2:41 PM, Igor Ignatyev wrote: >>>>>> >>>>>>> On Aug 6, 2020, at 12:33 AM, David Holmes > wrote: >>>>>>> >>>>>>> Basing the new test around the existing thread stress tests was not unreasonable but I don't think we're actively adding to those tests. I would check with Igor whether it is better to just add this under runtime/thread rather than vmTestBase. >>>>>> Hi Gerard, >>>>>> >>>>>> David is right, we are trying to get rid of vmTestbase directory and move all tests into corresponding component directories cleaning them up at the same time. so runtime/thread directory sound like a much better location for a new test. >>>>>> >>>>>> I don't think you need to cary all this baggage of 'VM testbase keywords', 'converted from ...' and the rest, I'd suggest you just use the text from 'DESCRIPTION' as @summary. I'd also remove System.exit call and would just make thread009::run to return bool. it's also unclear to me if someone is ever going to change THREADS_EXPECTED , TIMEOUT, and DEBUG_MODE values, I'd just hardcode them in the test and get rid of parsing. it seems that debug output won't clutter output, so you might want to always print them and remove DEBUG_MODE field. >>>>>> >>>>>> using `assert` in test code isn't a good idea as it will work only if someone runs test w/ `-ea` which not all people do. it's better to replace w/ `if (p) throw new Error(...)` or use jdk.test.lib.Asserts class. >>>>>> >>>>>> Cheers, >>>>>> -- Igor > From harold.seigel at oracle.com Wed Aug 19 17:31:51 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Wed, 19 Aug 2020 13:31:51 -0400 Subject: RFR 8251490: [TESTBUG] The Java thread stack size specified is too small for nsk/stress/stack. Specify at least 448k In-Reply-To: <9466f8ce-7045-664b-4c3b-777c73e05076@oracle.com> References: <9f12fc32-646c-1aff-86fa-003db2641c91@oracle.com> <11880712-97f7-efee-2cd3-fb22edf89af9@oracle.com> <1e0f8292-5929-2356-d1b8-638443e9a4fd@oracle.com> <9466f8ce-7045-664b-4c3b-777c73e05076@oracle.com> Message-ID: Hi Gerard, Thanks for approving these changes! I had already looked into what "-eager" meant so it wasn't difficult to look at it again.? I think it is poorly named and worth understanding. I agree that 15 minutes is too long to run these tests.? Normally they run in a few minutes or less. Thanks, Harold On 8/19/2020 1:17 PM, gerard ziemski wrote: > hi Harold, > > > On 8/18/20 2:54 PM, Harold Seigel wrote: >> >> Hi Gerard, >> >> Thanks for looking at this change.? Please see inline comments. >> >> On 8/18/2020 2:06 PM, gerard ziemski wrote: >>> hi Harold, >>> >>> Thank you for fixing this. >>> >>> >>> On 8/18/20 12:19 PM, Harold Seigel wrote: >>>> Hi, >>>> >>>> Please review this small change to fix five >>>> vmTestbase/nsk/stress/stack tests that are failing in tier5 on >>>> Linux AArch64.? The change increases the stack size for three of >>>> the tests.? Increasing the stack size of the other two tests could >>>> result in timeouts on other platforms.? So, those tests were >>>> changed to no longer run on AArch64. >>>> >>>> Open Webrev: >>>> http://cr.openjdk.java.net/~hseigel/bug_8251490/webrev/index.html >>>> >>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251490 >>>> >>>> The change was tested by running the tests as part of mach5 tier5 >>>> on Windows, Mac, Linux x64, and Linux AArch64. >>>> >>>> Thanks, Harold >>>> >>> It looks reasonable to me, but I have some questions: >>> >>> #1 Does "-eager" really make much difference here? Do you have any >>> timing numbers you can provide with and without that flag, just so >>> that we know how much it helps? >> >> This change does not affect the use of "-eager".? "-eager" has been >> part of the tests at least as far back as JDK-11. The only affect of >> running with "-eager" is to prevent the creation of a Terminator >> thread that waits a designated amount of time before calling >> System.exit().? Its purpose is described here in Terminator.java: >> >> /** >> ?* Terminator is used to terminate a stress test with PASS exit >> status >> ?* before the test is terminated as timed out (and so failed). >> ?* >> ?* Terminator class holds a thread which sleeps for the given amount >> ?* of time, and then wakes up and executes System.exit() >> ?* with the given exit status. That thread is daemon, so it doesn't >> ?* prevent application from exiting once all its threads finish >> ?* before it's time for termination. Appointing terminator in zero >> ?* delay implies immediate exit(). >> ?* >> ?* There is a limitation: you may appoint no more than one terminator >> ?* per application. >> ?*/ >> > Sorry Harold, I did not mean to make you investigate this option. I > thought this was something else (i.e. compile java methods instantly). > > >>> >>> #2 How did you determine the the 900s timeout? How long does it take >>> for a regular, middle of the road machine to execute each test? Do >>> we use any slow/low end machines in our testing that could take >>> longer than this time out? (I'm worried about that hardcoded time >>> out value if you can tell) >> >> I did not determine the 900s timeout.? That also has been in the >> tests for at least as far back as JDK-11.? When working on >> JDK-8139875, I ran the tests many times with different stack sizes >> both locally and with mach5 and looked at the run times for the >> tests, making sure they were all significantly below the timeout.? So >> far, the tests have run as part of tier5 for almost two weeks without >> timing out. >> >> The purpose of this proposed change is provide stack size values that >> will work on AArch64. >> > That part looks fine. > > >> Do you think a new bug is needed to address the use of -eager and the >> 900s timeout? >> > 900s is 15 minutes. On one hand it needs to be long enough for tests > not to timeout, but if any of these test need to run for such long, > then it seems excessive. But no, I don't think this needs to be > investigated further if your fix solves the issue at hand. > > > Thank you for taking the time to answer my questions. > > > cheers >> >> Thanks, Harold >> >>> >>> >>> cheers >>> > From igor.ignatyev at oracle.com Wed Aug 19 17:32:28 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 19 Aug 2020 10:32:28 -0700 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> <95b46d0f-8a81-10c6-78bb-a3abbd7cb955@oracle.com> <55ae5763-476b-562c-c7cc-a913992d304c@oracle.com> <585124BE-BDE7-496B-8A1F-B4E345DC5411@oracle.com> Message-ID: <2C08D47F-FB61-4E2E-8FF8-BFD6D8407C9E@oracle.com> > On Aug 19, 2020, at 10:00 AM, gerard ziemski wrote: > > > On 8/18/20 12:04 PM, Igor Ignatyev wrote: >> Dan/Gerard, >> >> we use '-ea -esa' in some of our configurations, yet not in all configurations and not all people use these configurations when they run tests. also some people run tests directly thru jtreg (or even w/o jtreg), hence adding it into make won't really help. therefore the tests shouldn't depend on -ea/-esa. >> >> generally speaking, `assert` makes sense only for product code where you might want some checks to be disabled in production runs, for tests you really want all checks to be always done. > > Very true, but: > > assert(condition, "message") > > is so much nicer to write than: > > if (!condition) { > throw new Exception("message"); > } > you can use jdk.test.lib.Asserts.assertEq(index, Integer.parseInt(getName(), "setName/getName failed!") > And when running locally one could turn them OFF/ON to help debug an issue. > true, but unless you ran it w/ -ea you ran the test w/ no checks, which is a bad test. there are still cases when it's ok to use `assert` in test code (e.g. early checking some internal test invariants, related not to the tested functionality, but to the test and only if braking these invariants will inevitably lead to harder to understand failure down the road), but it's very limited, so it's much easier to just never use `assert` in test code. -- Igor > > cheers > >> >> -- Igor >> >>> On Aug 18, 2020, at 6:31 AM, Daniel D. Daugherty wrote: >>> >>> This caught my eye: >>> >>>> We never run tests with Java asserts enabled so this is "dead" code. >>> I'm pretty certain that I've seen test tasks in Mach5 with enable assertion >>> options passed. I don't have an example at my fingertips though... >>> >>> Dan >>> >>> >>> On 8/18/20 1:15 AM, David Holmes wrote: >>>> Hi Gerard, >>>> >>>> On 18/08/2020 1:28 am, gerard ziemski wrote: >>>>> Thank you David and Igor for the feedback! >>>>> >>>>> I rewrote the test again, this time basing it off test/hotspot/jtreg/runtime/Thread/* tests. >>>>> >>>>> For macOS and Linux we create threads until we hit a process limit thread count. Windows, however, allows unlimited (?) number of threads, though the process gradually slows down as the count grows, so I had to add time limit and I set it to 5 sec. I chose 5 sec because the value is well below jtreg time out (i.e. 120s), but still quite above how long my laptop needs to hit the process thread count limit (i.e. my Mac laptop needs about 1 sec to hit that limit). >>>>> >>>>> Otherwise there are no other hardwired values or assumptions (I hope). What do you think of the test now? >>>> Not unreasonable :) It could still have tried for an upper limit of 10000 instead of using the time-limit, but it makes no real difference. >>>> >>>> A few nits/suggestions: >>>> >>>> Please add an @bug line to the test comment. >>>> >>>> 49 if ((this.index % 250) == 0) { >>>> >>>> Outside of the constructor it is poor style to use this.x to refer to fields methods (unless needed due to local variable shadowing). This applies throughout the test. >>>> >>>> 54 doWork(); >>>> >>>> Not sure why you factored things out the way you did - everything could go in run(). >>>> >>>> 61 setName(""+this.index); >>>> >>>> This could go in constructor, but I presume it adds something to the workload? Is the use of ""+index deliberate in that regard (generate string concatenation work), rather than using String.valueOf(index)? >>>> >>>> 65 assert(this.index == Integer.parseInt(getName())); >>>> >>>> We never run tests with Java asserts enabled so this is "dead" code. >>>> >>>> 87 // Windows path >>>> >>>> Windows path or system with very large ulimit? >>>> >>>> 92 } catch (OutOfMemoryError e) { >>>> >>>> I suggest checking this is actually the "unable to create native thread" OOME rather than a general Java OOME (if run with a small heap for example you might get OOME growing the ArrayList). >>>> >>>> 110 String cmd = System.getProperty("sun.java.command"); >>>> 111 if (cmd != null && !cmd.startsWith("com.sun.javatest.regtest.agent.MainWrapper")) { >>>> 112 // Exit with success in a non-JavaTest environment: >>>> 113 System.exit(0); >>>> 114 } >>>> >>>> I never noticed when this was used in the little set of runtime/Thread tests! I don't know what the rationale was but it is completely unnecessary as a normal return from main would be the equivalent of exit(0) - so the tests work fine without System.exit when run standalone. Please don't emulate this part of the existing tests. I'll file a RFE to remove it from the other tests. >>>> >>>> I won't be around for any further follow up for a week, so feel free to proceed once Igor is okay with things. >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>>>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev2 >>>>> testing: Mach5 hs-tier1,2,3,4,5 in progress... >>>>> >>>>> >>>>> >>>>> >>>>> On 8/7/20 2:41 PM, Igor Ignatyev wrote: >>>>>> >>>>>>> On Aug 6, 2020, at 12:33 AM, David Holmes > wrote: >>>>>>> >>>>>>> Basing the new test around the existing thread stress tests was not unreasonable but I don't think we're actively adding to those tests. I would check with Igor whether it is better to just add this under runtime/thread rather than vmTestBase. >>>>>> Hi Gerard, >>>>>> >>>>>> David is right, we are trying to get rid of vmTestbase directory and move all tests into corresponding component directories cleaning them up at the same time. so runtime/thread directory sound like a much better location for a new test. >>>>>> >>>>>> I don't think you need to cary all this baggage of 'VM testbase keywords', 'converted from ...' and the rest, I'd suggest you just use the text from 'DESCRIPTION' as @summary. I'd also remove System.exit call and would just make thread009::run to return bool. it's also unclear to me if someone is ever going to change THREADS_EXPECTED , TIMEOUT, and DEBUG_MODE values, I'd just hardcode them in the test and get rid of parsing. it seems that debug output won't clutter output, so you might want to always print them and remove DEBUG_MODE field. >>>>>> >>>>>> using `assert` in test code isn't a good idea as it will work only if someone runs test w/ `-ea` which not all people do. it's better to replace w/ `if (p) throw new Error(...)` or use jdk.test.lib.Asserts class. >>>>>> >>>>>> Cheers, >>>>>> -- Igor > From markus.gronlund at oracle.com Wed Aug 19 18:20:12 2020 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Wed, 19 Aug 2020 11:20:12 -0700 (PDT) Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <9e6a0427-8d56-3bab-33f6-b2aa64268099@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <29c3d48b-1875-8ea4-792a-157cdd07fe2c@oracle.com> <5f3a48b0-274f-c0f6-57a1-8fbb2bebf823@oracle.com> <9e6a0427-8d56-3bab-33f6-b2aa64268099@oracle.com> Message-ID: <6fb5da7a-28a6-4a2e-b93f-14409097abbb@default> Hi Patricio, Touching on something that Erik alluded to, and that is that the flag: 811 diagnostic(intx, DiagnoseSyncOnPrimitiveWrappers, 0, \ 812 "Detect and take action upon identifying synchronization on " \ 813 "primitive wrappers. Modes: " \ 814 "0: off; " \ 815 "1: exit with fatal error; " \ 816 "2: log message to stdout. Log options can be customized " \ 817 " with -Xlog:primitivewrappers; " \ 818 "3: emit JFR event") \ 819 range(0, 3) has become quite rich - also trying to auto-activate functionality if certain parts are enabled - for example it ergonomically enables UL. For JFR, this is not the best situation, because there all event control is already detailed in the .jfc configuration files. The flag as it is suggested would now add yet another place to know about to have the JFR events be generated. The flag mechanics can be simplified, at least for the "3. emit JFR event" option. This is because it is already implied when DiagnoseSyncOnPrimitiveWrappers is set (they are set to "enabled" in the .jfc's). It can therefore be removed as an option. In general, and to simplify matters, maybe just let the individual subsystems (UL, JFR) configure if, and if so what they will do if information comes routed their way by means of this flag? If so, you might only need two options: "1. Exit with fatal error", "2. Log information" (send to UL, send to JFR) - (perhaps you want to provide convenience by activating the UL tags in this case). Thanks Markus -----Original Message----- From: Patricio Chilano Sent: den 12 augusti 2020 00:14 To: David Holmes ; Erik Gahlin Cc: hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR 8242263: Diagnose synchronization on primitive wrappers On 8/10/20 9:40 PM, David Holmes wrote: > Hi Patricio, > > On 11/08/2020 6:03 am, Patricio Chilano wrote: >> Hi Erik, >> >> Thanks for looking into this and for the offline discussions. > > If there are still issues with this regarding JFR then I would suggest > we simply drop the use of JFR for reporting this. Ok, I'll wait to see about the JFR changes. > Meanwhile looking at v5 incremental that all seems fine to me. Great, thanks for looking into this David! Thanks, Patricio > Thanks, > David > ----- > >> On 8/10/20 10:19 AM, Erik Gahlin wrote: >>> >>> Hi Patricio, >>> >>> I have tried to review the JFR changes, but I need more information >>> on how the feature is going to be used. >>> >>> Is this something temporary that will be used gain insight for the >>> Valhalla project, or do you think this will be needed longer term? >>> If it is longer term, the bar is higher on what can be accepted. >>> >>> If this is a temporary feature, we could mark the event as >>> experimental in metadata.xml, similar to what we did with GC events >>> during the development of ZGC. Events that are experimental do not >>> show up by default in visualization tools such as JMC and can be >>> removed when they are not needed, or we have a better solution. >>> >> Right, it's temporary so we can mark it as experimental. I also >> changed the category name to be "Java Virtual Machine, Diagnostics". >> >>> For events to be enabled in default.jfc, they should not cause more >>> than 1% overhead, not even in pathological cases. To me, it seems >>> this could happen if you make a loop where it is triggered all the >>> time. >>> >>> For events in profile.jfc, the overhead should still be low (1-2%), >>> but the target is the typical application. For example, allocation >>> profiling is only enabled in profile.jfc. Some application that are >>> allocation intensive could cause a higher overhead than 1%, but that >>> is OK because that configuration is only to be used for a short >>> period of time. That said, there is still a budget on how much space >>> in a recording an event can take up, so it also depends on how >>> important the information is for the user. To me, it seems this >>> event will not be that important for the average user, which makes >>> me think it should be disabled by default. >>> >> After our off-list discussion I left it as enabled. I also removed >> the threshold setting and added the startTime=false in metadata.xml. >> >>> To complicate things, I noticed that aa command line flag is also >>> used to enable the event. This is something we have worked hard to >>> avoid when it comes to JFR events. All configuration should happen >>> using configuration files. We had problem with this in the past >>> where users gets confused why their events are not enabled. When we >>> ported JFR to Hotspot, we got rid of those of those flags/events and >>> it is a much better situation. Again, if this a temporary event to >>> discover usage pattern for a release or two, it might be OK, but if >>> we believe this event to stick around, I think we should look into >>> alternatives, or not use JFR at all for this. >>> >> Yes, I see your point. Since we want to diagnose synchronization on >> these classes but we don't want to affect performance the flag has to >> be specified at startup otherwise the diagnostic instrumentation is >> disabled, so I don't see a clear way to enable the events from JFR >> alone. In any case, this will be a temporary event. >> >> Here is v5. Let me know if you are okay with the changes. >> >> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ >> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/webrev/ >> >> Thanks Erik! >> >> Patricio >>> >>> Thanks >>> >>> Erik >>> >>>> On 28 Jul 2020, at 21:16, Patricio Chilano >>>> >>> > wrote: >>>> >>>> Hi all, >>>> >>>> Please review the following change that adds diagnostic >>>> capabilities when synchronizing on primitive wrapper classes. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8242263 >>>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8242263/v1/webrev/ >>>> >>>> The new flag allows to identify synchronization on these classes >>>> and to take one of the following actions: exit the VM with fatal >>>> error, log a warning message, or issue a JFR event. The >>>> implementation uses a simple approach where a check is added at the >>>> beginning of the monitorenter generated code when the flag is >>>> enabled to check whether the object is of a primitive wrapper >>>> class. If it is, we jump to the slow path, otherwise we just >>>> continue as always. The extra instructions will be: load the klass >>>> of the object, load the _access_flags field for that klass, AND >>>> with a constant, and branch based on the result. The code will only >>>> be generated whenever the new opt-in diagnostic flag is enabled so >>>> performance won't be affected when off. >>>> >>>> In addition to the purpose described in the description of the bug, >>>> this flag will also be useful when trying to diagnose possible >>>> synchronization issues if these classes ever become inline types as >>>> part of the Valhalla project. >>>> >>>> I added test SyncOnPrimitiveWrapperTest.java that tests for the >>>> exit and logging cases. I added test >>>> TestSyncOnPrimitiveWrapperEvent.java to test for the JFR event case. >>>> I tested the patch running tiers1-6 in mach5 with the flag set to >>>> DiagnoseSyncOnPrimitiveWrappers=2. >>>> I checked it builds with arm32 and ppc but can't run any tests on >>>> those platforms, so it would be good if somebody can run the new >>>> test included in the patch. >>>> >>>> Let me know if you think I should run or add any more tests. >>>> >>>> Thanks! >>>> >>>> Patricio >>> >> From lois.foltan at oracle.com Wed Aug 19 18:21:22 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 19 Aug 2020 14:21:22 -0400 Subject: RFR 8251490: [TESTBUG] The Java thread stack size specified is too small for nsk/stress/stack. Specify at least 448k In-Reply-To: <9f12fc32-646c-1aff-86fa-003db2641c91@oracle.com> References: <9f12fc32-646c-1aff-86fa-003db2641c91@oracle.com> Message-ID: <1c973f2c-e760-3640-8606-06e02993b70f@oracle.com> Looks good. Lois On 8/18/2020 1:19 PM, Harold Seigel wrote: > Hi, > > Please review this small change to fix five > vmTestbase/nsk/stress/stack tests that are failing in tier5 on Linux > AArch64.? The change increases the stack size for three of the tests.? > Increasing the stack size of the other two tests could result in > timeouts on other platforms.? So, those tests were changed to no > longer run on AArch64. > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8251490/webrev/index.html > > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251490 > > The change was tested by running the tests as part of mach5 tier5 on > Windows, Mac, Linux x64, and Linux AArch64. > > Thanks, Harold > From harold.seigel at oracle.com Wed Aug 19 18:22:50 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Wed, 19 Aug 2020 14:22:50 -0400 Subject: RFR 8251490: [TESTBUG] The Java thread stack size specified is too small for nsk/stress/stack. Specify at least 448k In-Reply-To: <1c973f2c-e760-3640-8606-06e02993b70f@oracle.com> References: <9f12fc32-646c-1aff-86fa-003db2641c91@oracle.com> <1c973f2c-e760-3640-8606-06e02993b70f@oracle.com> Message-ID: <1e7508b1-bc1e-06dd-9b43-aa03d64f1e75@oracle.com> Thanks Lois! Harold On 8/19/2020 2:21 PM, Lois Foltan wrote: > Looks good. > Lois > > On 8/18/2020 1:19 PM, Harold Seigel wrote: >> Hi, >> >> Please review this small change to fix five >> vmTestbase/nsk/stress/stack tests that are failing in tier5 on Linux >> AArch64.? The change increases the stack size for three of the >> tests.? Increasing the stack size of the other two tests could result >> in timeouts on other platforms.? So, those tests were changed to no >> longer run on AArch64. >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8251490/webrev/index.html >> >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251490 >> >> The change was tested by running the tests as part of mach5 tier5 on >> Windows, Mac, Linux x64, and Linux AArch64. >> >> Thanks, Harold >> > From gerard.ziemski at oracle.com Wed Aug 19 19:55:20 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Wed, 19 Aug 2020 14:55:20 -0500 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: <62F99ACE-282A-47B8-A213-626108712F46@oracle.com> References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> <95b46d0f-8a81-10c6-78bb-a3abbd7cb955@oracle.com> <55ae5763-476b-562c-c7cc-a913992d304c@oracle.com> <4b9dfeda-c6f7-3ee5-fb23-85cf58f1130a@oracle.com> <2BAD9BBF-B8F9-45E7-8378-E69622856C7A@oracle.com> <44706c40-d3f2-6b63-1491-c4412a0c4347@oracle.com> <62F99ACE-282A-47B8-A213-626108712F46@oracle.com> Message-ID: <226bee07-711f-9687-8318-b72826e9e32d@oracle.com> hi Igor, I have responses to your questions inline: bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev5 testing: Mach5 hs-tier1,2,3,4,5 in progress... On 8/19/20 12:25 PM, Igor Ignatyev wrote: > Hi Gerard, > > please see my comments inline. > > -- Igor > >> On Aug 19, 2020, at 9:53 AM, gerard ziemski wrote: >> >> hi Igor, >> >> Thank you for the review. Here is the webrev, but please see my comments inline for more: >> >> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev4 >> testing: Mach5 hs-tier1,2,3,4,5 in progress... >> >> >> >> On 8/18/20 12:18 PM, Igor Ignatyev wrote: >>> Hi Gerard, >>> >>> looks be >>> 1. >>>> 26 * @bug 8222671 >>> @bug tag should refer to a product bug which can be reproduced w/ this test, not the bug id used to push a test. >> The bug 8222671 is the issue here, i.e. timeouts in the existing test, which is what we are trying to fix, so we have a new test that replaces the old and less robust one. I did try to find an issue that originated the old test and I couldn?t, but if you know how to find, I can add a reference to it in the new test. To me, the old test looks more like a general functionality test though. > I'd just remove @bug tag, b/c from my point of view listing all test bugs which caused modification of the test in @bug is just useless and pollute metadata. others have other opinion though, so I don't insist on removal. You are the expert here, removed. >>> 2. >>>> 28 * @run main/othervm ThreadCountLimit >>> what's the reason to run in othervm mode? >> Oh, I just copied that from the existing test, without giving it any thought - I will remove it. > in hotspot tests, we tend not to relay on implicit @run (as it's error prone) and always have explicit `@run`, that's to say, please add `@run main ThreadCountLimit` Is there a template for a new test, or a wiki page that explains all this? I learned quite a bit writing this test, but it would be nice to have some documentation I could use as a reference in the future. >>> 3. >>>> 34 public class ThreadCountLimit extends Object { >>> you don't need to explicitly extend Object. >> True, but I like the code explicit like that if that?s OK. > fine w/ me, but it's kinda against regular java code guidelines and doesn't bring any information. Fixed. >>> 4. in ThreadCountLimit::run() >>>> 67 } catch (InterruptedException ex) { >>>> 68 } >>> shouldn't this mark the test as failed? >> Good question. In a perfect world we would want to know if a thread got unexpectedly interrupted, but realistically a process might (will?) exhibit spurious interruptions from the OS sooner or later, which are not the focus of this test, so instead of marking it a failure and making someone look into it, only later to determine that it was just a spurious interrupt, we simply ignore it and focus on the core goal of this test instead. > I'm not sure what you are refer here by 'spurious interruptions', java threads must be interrupted only if someone interrupted them, all other behaviors are JVM bugs and are to be analyzed. given the goal of this test is to check how JVM behaves when lots of threads have been created and started, I see catching unexpected InterruptedException as a valid failure mode of this test. I think I might be mixing up native wait/notify on mutexes here, where a wait on a native mutex can be spuriously awaken. According to https://www.ibm.com/developerworks/library/j-jtp05236/index.html Thread interruption is a cooperative mechanism, and since we are not interrupting any of the blocking methods we use in the test ourselves, it would be really curious if we ever did get InterruptedException, so it might be worth it to report it. I tightened up the blocking code and its try/catch code block, to make it to clear what is the subject to interrupts and we will now throw an Error if we ever catch one (never?) Have we ever seen one of those unexpected interrupts in our testing so far? Thank you for the feedback. cheers > >> >> I do see, however, that in one place I report it as Error and ignore it in the other instance, so I fixed that inconsistency. I added comments as well. >> >>> Thanks, >>> -- Igor >>> >>>> On Aug 18, 2020, at 7:38 AM, gerard ziemski wrote: >>>> >>>> Thank you David for the review! >>>> >>>> I incorporated all your feedback. >>>> >>>> I'm surprised that "assert" is not available by default in our tests though - what is a reason for that? Of all the places where asserts should be available, a testing suite would seem to be it to me. >>>> >>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev3 >>>> testing: passes locally, Mach5 hs-tier1,2,3,4,5 in progress... >>>> >>>> >>>> cheers >>>> >>>> On 8/18/20 12:15 AM, David Holmes wrote: >>>>> Hi Gerard, >>>>> >>>>> On 18/08/2020 1:28 am, gerard ziemski wrote: >>>>>> Thank you David and Igor for the feedback! >>>>>> >>>>>> I rewrote the test again, this time basing it off test/hotspot/jtreg/runtime/Thread/* tests. >>>>>> >>>>>> For macOS and Linux we create threads until we hit a process limit thread count. Windows, however, allows unlimited (?) number of threads, though the process gradually slows down as the count grows, so I had to add time limit and I set it to 5 sec. I chose 5 sec because the value is well below jtreg time out (i.e. 120s), but still quite above how long my laptop needs to hit the process thread count limit (i.e. my Mac laptop needs about 1 sec to hit that limit). >>>>>> >>>>>> Otherwise there are no other hardwired values or assumptions (I hope). What do you think of the test now? >>>>> Not unreasonable :) It could still have tried for an upper limit of 10000 instead of using the time-limit, but it makes no real difference. >>>>> >>>>> A few nits/suggestions: >>>>> >>>>> Please add an @bug line to the test comment. >>>>> >>>>> 49 if ((this.index % 250) == 0) { >>>>> >>>>> Outside of the constructor it is poor style to use this.x to refer to fields methods (unless needed due to local variable shadowing). This applies throughout the test. >>>>> >>>>> 54 doWork(); >>>>> >>>>> Not sure why you factored things out the way you did - everything could go in run(). >>>>> >>>>> 61 setName(""+this.index); >>>>> >>>>> This could go in constructor, but I presume it adds something to the workload? Is the use of ""+index deliberate in that regard (generate string concatenation work), rather than using String.valueOf(index)? >>>>> >>>>> 65 assert(this.index == Integer.parseInt(getName())); >>>>> >>>>> We never run tests with Java asserts enabled so this is "dead" code. >>>>> >>>>> 87 // Windows path >>>>> >>>>> Windows path or system with very large ulimit? >>>>> >>>>> 92 } catch (OutOfMemoryError e) { >>>>> >>>>> I suggest checking this is actually the "unable to create native thread" OOME rather than a general Java OOME (if run with a small heap for example you might get OOME growing the ArrayList). >>>>> >>>>> 110 String cmd = System.getProperty("sun.java.command"); >>>>> 111 if (cmd != null && !cmd.startsWith("com.sun.javatest.regtest.agent.MainWrapper")) { >>>>> 112 // Exit with success in a non-JavaTest environment: >>>>> 113 System.exit(0); >>>>> 114 } >>>>> >>>>> I never noticed when this was used in the little set of runtime/Thread tests! I don't know what the rationale was but it is completely unnecessary as a normal return from main would be the equivalent of exit(0) - so the tests work fine without System.exit when run standalone. Please don't emulate this part of the existing tests. I'll file a RFE to remove it from the other tests. >>>>> >>>>> I won't be around for any further follow up for a week, so feel free to proceed once Igor is okay with things. >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>>>>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev2 >>>>>> testing: Mach5 hs-tier1,2,3,4,5 in progress... >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On 8/7/20 2:41 PM, Igor Ignatyev wrote: >>>>>>>> On Aug 6, 2020, at 12:33 AM, David Holmes > wrote: >>>>>>>> >>>>>>>> Basing the new test around the existing thread stress tests was not unreasonable but I don't think we're actively adding to those tests. I would check with Igor whether it is better to just add this under runtime/thread rather than vmTestBase. >>>>>>> Hi Gerard, >>>>>>> >>>>>>> David is right, we are trying to get rid of vmTestbase directory and move all tests into corresponding component directories cleaning them up at the same time. so runtime/thread directory sound like a much better location for a new test. >>>>>>> >>>>>>> I don't think you need to cary all this baggage of 'VM testbase keywords', 'converted from ...' and the rest, I'd suggest you just use the text from 'DESCRIPTION' as @summary. I'd also remove System.exit call and would just make thread009::run to return bool. it's also unclear to me if someone is ever going to change THREADS_EXPECTED , TIMEOUT, and DEBUG_MODE values, I'd just hardcode them in the test and get rid of parsing. it seems that debug output won't clutter output, so you might want to always print them and remove DEBUG_MODE field. >>>>>>> >>>>>>> using `assert` in test code isn't a good idea as it will work only if someone runs test w/ `-ea` which not all people do. it's better to replace w/ `if (p) throw new Error(...)` or use jdk.test.lib.Asserts class. >>>>>>> >>>>>>> Cheers, >>>>>>> -- Igor From igor.ignatyev at oracle.com Wed Aug 19 20:09:21 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 19 Aug 2020 13:09:21 -0700 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: <226bee07-711f-9687-8318-b72826e9e32d@oracle.com> References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> <95b46d0f-8a81-10c6-78bb-a3abbd7cb955@oracle.com> <55ae5763-476b-562c-c7cc-a913992d304c@oracle.com> <4b9dfeda-c6f7-3ee5-fb23-85cf58f1130a@oracle.com> <2BAD9BBF-B8F9-45E7-8378-E69622856C7A@oracle.com> <44706c40-d3f2-6b63-1491-c4412a0c4347@oracle.com> <62F99ACE-282A-47B8-A213-626108712F46@oracle.com> <226bee07-711f-9687-8318-b72826e9e32d@oracle.com> Message-ID: Hi Gerard, thanks for addressing my comments. 8222671_rev5 looks good to me, reviewed. Cheers, -- Igor > On Aug 19, 2020, at 12:55 PM, gerard ziemski wrote: > > hi Igor, > > I have responses to your questions inline: > > bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 > open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev5 > testing: Mach5 hs-tier1,2,3,4,5 in progress... > > > On 8/19/20 12:25 PM, Igor Ignatyev wrote: >> Hi Gerard, >> >> please see my comments inline. >> >> -- Igor >> >>> On Aug 19, 2020, at 9:53 AM, gerard ziemski wrote: >>> >>> hi Igor, >>> >>> Thank you for the review. Here is the webrev, but please see my comments inline for more: >>> >>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev4 >>> testing: Mach5 hs-tier1,2,3,4,5 in progress... >>> >>> >>> >>> On 8/18/20 12:18 PM, Igor Ignatyev wrote: >>>> Hi Gerard, >>>> >>>> looks be >>>> 1. >>>>> 26 * @bug 8222671 >>>> @bug tag should refer to a product bug which can be reproduced w/ this test, not the bug id used to push a test. >>> The bug 8222671 is the issue here, i.e. timeouts in the existing test, which is what we are trying to fix, so we have a new test that replaces the old and less robust one. I did try to find an issue that originated the old test and I couldn?t, but if you know how to find, I can add a reference to it in the new test. To me, the old test looks more like a general functionality test though. >> I'd just remove @bug tag, b/c from my point of view listing all test bugs which caused modification of the test in @bug is just useless and pollute metadata. others have other opinion though, so I don't insist on removal. > You are the expert here, removed. > > >>>> 2. >>>>> 28 * @run main/othervm ThreadCountLimit >>>> what's the reason to run in othervm mode? >>> Oh, I just copied that from the existing test, without giving it any thought - I will remove it. >> in hotspot tests, we tend not to relay on implicit @run (as it's error prone) and always have explicit `@run`, that's to say, please add `@run main ThreadCountLimit` > Is there a template for a new test, or a wiki page that explains all this? I learned quite a bit writing this test, but it would be nice to have some documentation I could use as a reference in the future. there are, please take a look at materials linked from https://wiki.openjdk.java.net/display/CodeTools/jtreg and http://openjdk.java.net/jtreg/ . I recall there was an initiative to write hotspot specific doc/templates/examples, but I don't think we have published it. by some reasons, wiki.openjdk.java.net is very slow for me, so I can't double-check. > > >>>> 3. >>>>> 34 public class ThreadCountLimit extends Object { >>>> you don't need to explicitly extend Object. >>> True, but I like the code explicit like that if that?s OK. >> fine w/ me, but it's kinda against regular java code guidelines and doesn't bring any information. > Fixed. > > >>>> 4. in ThreadCountLimit::run() >>>>> 67 } catch (InterruptedException ex) { >>>>> 68 } >>>> shouldn't this mark the test as failed? >>> Good question. In a perfect world we would want to know if a thread got unexpectedly interrupted, but realistically a process might (will?) exhibit spurious interruptions from the OS sooner or later, which are not the focus of this test, so instead of marking it a failure and making someone look into it, only later to determine that it was just a spurious interrupt, we simply ignore it and focus on the core goal of this test instead. >> I'm not sure what you are refer here by 'spurious interruptions', java threads must be interrupted only if someone interrupted them, all other behaviors are JVM bugs and are to be analyzed. given the goal of this test is to check how JVM behaves when lots of threads have been created and started, I see catching unexpected InterruptedException as a valid failure mode of this test. > I think I might be mixing up native wait/notify on mutexes here, where a wait on a native mutex can be spuriously awaken. wait on java monitors can indeed be spuriously awaken, but that it's a different story ;) > > > According to https://www.ibm.com/developerworks/library/j-jtp05236/index.html Thread interruption is a cooperative mechanism, and since we are not interrupting any of the blocking methods we use in the test ourselves, it would be really curious if we ever did get InterruptedException, so it might be worth it to report it. > > I tightened up the blocking code and its try/catch code block, to make it to clear what is the subject to interrupts and we will now throw an Error if we ever catch one (never?) Have we ever seen one of those unexpected interrupts in our testing so far? I don't think we have, one of the reasons might be that some tests just silently ignore IE. > > > Thank you for the feedback. > > > cheers > >> >>> >>> I do see, however, that in one place I report it as Error and ignore it in the other instance, so I fixed that inconsistency. I added comments as well. >>> >>>> Thanks, >>>> -- Igor >>>> >>>>> On Aug 18, 2020, at 7:38 AM, gerard ziemski wrote: >>>>> >>>>> Thank you David for the review! >>>>> >>>>> I incorporated all your feedback. >>>>> >>>>> I'm surprised that "assert" is not available by default in our tests though - what is a reason for that? Of all the places where asserts should be available, a testing suite would seem to be it to me. >>>>> >>>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>>>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev3 >>>>> testing: passes locally, Mach5 hs-tier1,2,3,4,5 in progress... >>>>> >>>>> >>>>> cheers >>>>> >>>>> On 8/18/20 12:15 AM, David Holmes wrote: >>>>>> Hi Gerard, >>>>>> >>>>>> On 18/08/2020 1:28 am, gerard ziemski wrote: >>>>>>> Thank you David and Igor for the feedback! >>>>>>> >>>>>>> I rewrote the test again, this time basing it off test/hotspot/jtreg/runtime/Thread/* tests. >>>>>>> >>>>>>> For macOS and Linux we create threads until we hit a process limit thread count. Windows, however, allows unlimited (?) number of threads, though the process gradually slows down as the count grows, so I had to add time limit and I set it to 5 sec. I chose 5 sec because the value is well below jtreg time out (i.e. 120s), but still quite above how long my laptop needs to hit the process thread count limit (i.e. my Mac laptop needs about 1 sec to hit that limit). >>>>>>> >>>>>>> Otherwise there are no other hardwired values or assumptions (I hope). What do you think of the test now? >>>>>> Not unreasonable :) It could still have tried for an upper limit of 10000 instead of using the time-limit, but it makes no real difference. >>>>>> >>>>>> A few nits/suggestions: >>>>>> >>>>>> Please add an @bug line to the test comment. >>>>>> >>>>>> 49 if ((this.index % 250) == 0) { >>>>>> >>>>>> Outside of the constructor it is poor style to use this.x to refer to fields methods (unless needed due to local variable shadowing). This applies throughout the test. >>>>>> >>>>>> 54 doWork(); >>>>>> >>>>>> Not sure why you factored things out the way you did - everything could go in run(). >>>>>> >>>>>> 61 setName(""+this.index); >>>>>> >>>>>> This could go in constructor, but I presume it adds something to the workload? Is the use of ""+index deliberate in that regard (generate string concatenation work), rather than using String.valueOf(index)? >>>>>> >>>>>> 65 assert(this.index == Integer.parseInt(getName())); >>>>>> >>>>>> We never run tests with Java asserts enabled so this is "dead" code. >>>>>> >>>>>> 87 // Windows path >>>>>> >>>>>> Windows path or system with very large ulimit? >>>>>> >>>>>> 92 } catch (OutOfMemoryError e) { >>>>>> >>>>>> I suggest checking this is actually the "unable to create native thread" OOME rather than a general Java OOME (if run with a small heap for example you might get OOME growing the ArrayList). >>>>>> >>>>>> 110 String cmd = System.getProperty("sun.java.command"); >>>>>> 111 if (cmd != null && !cmd.startsWith("com.sun.javatest.regtest.agent.MainWrapper")) { >>>>>> 112 // Exit with success in a non-JavaTest environment: >>>>>> 113 System.exit(0); >>>>>> 114 } >>>>>> >>>>>> I never noticed when this was used in the little set of runtime/Thread tests! I don't know what the rationale was but it is completely unnecessary as a normal return from main would be the equivalent of exit(0) - so the tests work fine without System.exit when run standalone. Please don't emulate this part of the existing tests. I'll file a RFE to remove it from the other tests. >>>>>> >>>>>> I won't be around for any further follow up for a week, so feel free to proceed once Igor is okay with things. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>>>>>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev2 >>>>>>> testing: Mach5 hs-tier1,2,3,4,5 in progress... >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 8/7/20 2:41 PM, Igor Ignatyev wrote: >>>>>>>>> On Aug 6, 2020, at 12:33 AM, David Holmes > wrote: >>>>>>>>> >>>>>>>>> Basing the new test around the existing thread stress tests was not unreasonable but I don't think we're actively adding to those tests. I would check with Igor whether it is better to just add this under runtime/thread rather than vmTestBase. >>>>>>>> Hi Gerard, >>>>>>>> >>>>>>>> David is right, we are trying to get rid of vmTestbase directory and move all tests into corresponding component directories cleaning them up at the same time. so runtime/thread directory sound like a much better location for a new test. >>>>>>>> >>>>>>>> I don't think you need to cary all this baggage of 'VM testbase keywords', 'converted from ...' and the rest, I'd suggest you just use the text from 'DESCRIPTION' as @summary. I'd also remove System.exit call and would just make thread009::run to return bool. it's also unclear to me if someone is ever going to change THREADS_EXPECTED , TIMEOUT, and DEBUG_MODE values, I'd just hardcode them in the test and get rid of parsing. it seems that debug output won't clutter output, so you might want to always print them and remove DEBUG_MODE field. >>>>>>>> >>>>>>>> using `assert` in test code isn't a good idea as it will work only if someone runs test w/ `-ea` which not all people do. it's better to replace w/ `if (p) throw new Error(...)` or use jdk.test.lib.Asserts class. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> -- Igor From mandy.chung at oracle.com Wed Aug 19 20:30:57 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 19 Aug 2020 13:30:57 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> Message-ID: <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> On 8/17/20 12:37 PM, Yumin Qi wrote: > Hi, Ioi > > ? Thanks for review/suggestion. I have updated the webrev at the > following link: > > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ > This patch leverages the TRACE_RESOLVE output and passes the trace output to VM.? VM then calls GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing and generate Holder class per the resolved LFs.?? I think there are other cleaner alternatives implementing this.?? jlink --generate-jli-classes plugin depends the trace output whereas -Xshare:dump does not.?? It's cleaner to skip generating the trace output and parsing for dumping shared archive purpose.? In addition, the implementation needs some cleanup (I can send you feedback on the next revision) Instead of relying on a system property "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. One possible approach: - Add the trace methods in GenerateJLIClassesHelper class to trace these resolved members: ?? traceSpeciesType(String cn) ?? tracePregeneratedLambdaForm(String name, MethodType type, Class holder) ?? * I'm not sure if these method should also accept the case when an exception is thrown but it's okay we add it as well to match existing implementation. - these trace methods will either (1) send the trace output to System.out or (2) collect speciesTypes, invokerTypes, callSiteTypes, and dmhMethods in the runtime if -Xshare:dump is specified. - ClassSpecializer and InvokerBytecodeGenerator will call these trace methods instead. - TRACE_RESOLVE is set when the system property is set to true. If -Xshare:dump is enabled,? the trace methods will not write to System.out.? Does CDS want a log of these entry?? I assume not? - jlink plugin will continue to do the parsing.? It can construct a new? class that takes the speciesTypes, invokerTypes, callSiteTypes, and dmhMethods parameters parsed from the trace output. jlink plugin can invoke an instance method to return Map. I suggest the key for this return map is the class name rather than a jimage entry name and the caller can map it to an jimage entry name if needed. - CDS will invoke a different entry to return Object[] as you currently have. What do you think? Mandy From ioi.lam at oracle.com Wed Aug 19 20:47:19 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 19 Aug 2020 13:47:19 -0700 Subject: RFR(XS) 8252056 Move DumpRegion/ReadClosure/WriteClosure to archiveUtils.hpp Message-ID: <6b312a7a-df99-4399-10d2-1122ce689dd3@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8252056 http://cr.openjdk.java.net/~iklam/jdk16/8252056-move-dump-region-to-archive-utils.v01/ Part of CDS code clean up: The classes are currently in metaspaceShared.hpp/cpp, which are too big. It's better to move these utility classes to archiveUtils.cpp/hpp Thanks - Ioi From ioi.lam at oracle.com Wed Aug 19 21:39:06 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 19 Aug 2020 14:39:06 -0700 Subject: RFR 8251557 Avoid dumping unused symbols/strings into the CDS archive Message-ID: <6d64004b-4aeb-f95f-1551-d6d608191f3a@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8251557 http://cr.openjdk.java.net/~iklam/jdk16/8251557-cds-dont-dump-unused-symbols.v01/ Part of CDS code clean up: Currently all entries in the SymbolTable/StringTable are dumped into the CDS archive. The problems are (1) We end up writing many unused items, such as the mangled names ? ? of hidden classes. (2) We have to scan the symbol table inside a safepoint, which had ? ? caused bugs before (JDK-8245264). Since JDK-8250990, we already maintain all used Symbols in a growable array. We can dump the shared symbol table using this array, and avoid walking the SymbolTable. For StringTable, we scan all archived classes, and dump only the interned strings used by those classes (plus extra strings specified by SharedArchiveConfigFile) into the CDS interned string table. This reduces complexity in the CDS code, and reduce the size of the CDS archives. Testing - tiers 1-4 Archive file size for "java -Xshare:dump" ?? old: 39,368 symbols 7,028 strings 11,784,200 bytes ?? new: 37,387 symbols 7,020 strings 11,683,320 bytes (~0.9% reduction) Thanks - Ioi From coleen.phillimore at oracle.com Wed Aug 19 21:47:51 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 19 Aug 2020 17:47:51 -0400 Subject: RFR (S) 8252043: Move inner class metaspace cleaning out of safepoint cleanup tasks Message-ID: <50e42ffa-a210-8787-56c5-e19aebb58443@oracle.com> Summary: Clean up inner metaspaces from ServiceThread if cleanup is needed for concurrent GCs. Moved inner metaspace cleaning back to class unloading, unless using concurrent GC.? There we initiate a safepoint if inner metaspaces require cleaning. Tested with tier1-6. open webrev at http://cr.openjdk.java.net/~coleenp/2020/8252043.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8252043 Thanks, Coleen From yumin.qi at oracle.com Wed Aug 19 23:08:26 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Wed, 19 Aug 2020 16:08:26 -0700 Subject: RFR(XS) 8252056 Move DumpRegion/ReadClosure/WriteClosure to archiveUtils.hpp In-Reply-To: <6b312a7a-df99-4399-10d2-1122ce689dd3@oracle.com> References: <6b312a7a-df99-4399-10d2-1122ce689dd3@oracle.com> Message-ID: Hi, Ioi ? Looks good to me. ? One question: with removing of #if INCLUDE_CDS from file archiveUtils.cpp, will this file be added to non-CDS build? Thanks Yumin On 8/19/20 1:47 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8252056 > http://cr.openjdk.java.net/~iklam/jdk16/8252056-move-dump-region-to-archive-utils.v01/ > > Part of CDS code clean up: > > The classes are currently in metaspaceShared.hpp/cpp, which are too big. > It's better to move these utility classes to archiveUtils.cpp/hpp > > Thanks > - Ioi From serguei.spitsyn at oracle.com Wed Aug 19 23:22:08 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 19 Aug 2020 16:22:08 -0700 Subject: RFR(T) : 8252005 : narrow disabling of allowSmartActionArgs in vmTestbase In-Reply-To: <4E6FECE6-9103-46ED-84B2-79DBA0123ED9@oracle.com> References: <4E6FECE6-9103-46ED-84B2-79DBA0123ED9@oracle.com> Message-ID: <17a8369e-5f38-ebab-974b-28e083378aa2@oracle.com> Hi Igor, This looks reasonable. Thanks, Serguei On 8/18/20 16:42, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8252005/webrev.00/ >> 0 lines changed: 0 ins; 0 del; 0 mod; > Hi all, > > could you please review this trivial (and apparently empty) patch which sets allowSmartActionArgs to false only in subdirectories of vmTestbase which currently use PropertyResolvingWrapper? > > (it's hard to tell from webrev or patch, but test/hotspot/jtreg/vmTestbase/TEST.properties is effectively removed) > > webrev: http://cr.openjdk.java.net/~iignatyev//8252005/webrev.00/ > JBS: https://bugs.openjdk.java.net/browse/JDK-8252005 > > Thanks, > -- Igor > > From coleen.phillimore at oracle.com Thu Aug 20 00:31:39 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 19 Aug 2020 20:31:39 -0400 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: <8bd366a3-a40b-c2d6-8e19-1abf7a4c7e04@oracle.com> Message-ID: Sorry I dropped off hotspot-gc in my first review. A few replies below. On 8/19/20 3:43 AM, Thomas St?fe wrote: > Hi Coleen, > > On Wed, Aug 19, 2020 at 12:19 AM Coleen Phillimore > > > wrote: > > > Hi Thomas, > > I've read through the new implementation files.? This looks really > good!? These are my initial comments.? There aren't many and > nothing of > substance really.? Some files need more in-depth review though. I > haven't looked at the tests, which are more difficult.? I'll do that > tomorrow. > > General comment:? Some files have a lot of vertical white space, like > two or more blank lines together: eg. metachunk.cpp at the end.? More > about that below. > > > A bad habit of mine, idly pressing space white?thinking. I'll fix those. > > > === counter.hpp > > This seems like a general utility that should be in utilities so > that it > can be generalized for use in other places.? I didn't review it > carefully to suggest it go there right now, but we should file > another > RFE to do that.? Kim has some ideas about it. > > > I agree. I kept this class here to keep the patch confined to > memory/metaspace. I would like cleanups like this as follow ups to the > main patch, as you suggested. I created > https://bugs.openjdk.java.net/browse/JDK-8252014?to keep track of this. > > > === freeBlocks.hpp > > I like how the small blocks and tree of freed blocks are together > (except see below). > This has a really good comment. > > > Thanks. > > > #ifndef SHARE_MEMORY_METASPACE_LEFTOVERBINS_HPP > #define SHARE_MEMORY_METASPACE_LEFTOVERBINS_HPP > Wrong ifdef guard names. > #endif // SHARE_MEMORY_METASPACE_CHUNKMANAGER_HPP > > > Ok > > === binList.hpp > > ?? block_t* _v[num_lists]; > > Some slightly longer name would be better than _v.? Maybe > _small_blocks > or something like that? > > > Would _blocks be acceptable? Since the distinguisher 'small' makes no > sense here, it is the only member in the class. Yes, _blocks is good. > // These aren't used > typedef BinListImpl<2, 8>? BinList8; > typedef BinListImpl<2, 16> BinList16; > typedef BinListImpl<2, 64> BinList64; > > This file should be binList.hpp rather than binlist.hpp. > > > Ok > > === blockTree.hpp/cpp > > I didn't review this code.? Someone suggested using std::map > instead but > we're not using std:: yet. > > > I don't really like that idea.?Using stl maps for structures as these > in-place blocks seems, depending on how it is done, either risky or > wasteful. > > In this tree payload and nodes are identical and the size of the > payload is the key. Like the Linux rbtree, really. STL map > keeps?these?entities separate: a KV pair holds K (block size) and V > (the block). So we spent additional memory and allocation to keep the > pair itself, and K is also kept redundant (since the block already > knows its size). This is if we use stl map safely-but-naively. I guess > one could get stl::map to behave exactly as the Linux rb tree - taking > KV pair allocation into our own hand and somehow eliminating the K > storage redundancy. But I would think we end up with a solution way > less maintainable?than what I have suggested. You'd need intimate > knowledge of stl map to make sure we this?works, and across all > platforms and all STL implementations. I'd think maintaining this > little tree thing is cheaper. I agree with this. > > blocktree.hpp/cpp should be blockTree.hpp/cpp > > Since for class metaspace, the reclaimed InstanceKlass are generally > bigger than the largest BinList size, maybe class type > MetaspaceArenas > should only have a blockTree and not a SmallBlocks, so not the whole > freeList.hpp structure.? I think we could do some footprint analysis > with NMT and see if this might help.? It's fairly rare unless you > have a > lot of redefinition.? I don't think this should be changed in this > change though. > > > Oh good point. > > I would not like to hard-code the assumption that class space only > contains Klass though; I dimly remember an RFR with Ioi where he tried > storing other things there as well. IIRC he found some other solution. > But my point is that this assumption is fragile. > > But a "soft assumption" would be fine. E.g. creating the BinList on > the fly, only when needed, and separately from the blocktree. Easy to > implement too. > > We don't need NMT to analyze. BinList currently has 32 slots, + > counter + mask. We pay 272 bytes per structure, or 34 words. Which we > would save per CLD. This seems like something useful to explore later. > > === metaspaceContext.cpp > > #define LOGFMT???????? "SpcMgr @" PTR_FORMAT " (%s)" > #define LOGFMT_ARGS??? p2i(this), this->_name > > These don't seem to be used and is SpcMgr a reference to the old > SpaceManager?? Correction, I see how it's used but you should > change it > to not refer to Space manager. > > > I'll fix that. > > > // Destroys the context: deletes chunkmanager and virtualspacelist. > // If this is a non-expandable context over an existing space, that > space remains > // untouched, otherwise all memory is unmapped. > MetaspaceContext::~MetaspaceContext() { > ?? delete _cm; > ?? delete _vslist; > } > > Why would you ever do this?? I thought the two contexts were > global.? If > it's for the tests, add a comment here. > > > Yes, it only happens when testing. The rest is future thoughts, in > case we ever use a metaspace context for other purposes (I am > currently experimenting with using them as Arena replacement). > > I will adapt the comment. > > > === metachunk.cpp > > ???? class UnsortedMetachunkList : public AbstractMetachunkList { > > This class is declared in a strange place (in a function) and appears > unused. check_pattern() appears unused which is good because it's > a lot > of code. > > > My god how did this get there. How did this ever compile. This is an > accidental paste from some code move I did before posting the review. > And yes, seems both fill_with_pattern and check_pattern are unused. > Remnants from some earlier version. I'll remove both. > > > === metaspace_test.hpp/cpp > > I keep thinking this file is in the wrong place.? Can you rename > metaspaceTestHelper? or some name like that? > > > Will do. These classes must live somewhere in metaspace since they are > used by whitebox.cpp. I could move them there directly. But I wanted > to keep them separate. > > > === internStats.hpp > > Indentation is inconstent and too much vertical white space at the > end. > Should be renamed internalStats.hpp/cpp to match the class name.? > This > looks like a nice improvement. One of the things that made metaspace > code hard to read was that there was a lot of code for statistics > mixed > in everywhere. > > > I agree. > > > === arenaGrowthPolicy.hpp/cpp > > I like this! > > > :) > > === chunkLevel.hpp/cpp > > I'm fine with the nested namespace chunklevel, rather than an > AllStatic > class ChunkLevel.? The names of the files should probably be > changed to > chunklevel.hpp/cpp though. > > > Ok. > > > === metaspaceEnums.hpp/cpp > > This set of files seems like they are misnamed and should be > encorporated into metaspace.hpp, since they are used outside of the > memory/metaspace directory. > > > I agree, this seems odd. Will move as suggested. > > === settings.hpp > > Include guards are wrong names. > > #ifndef SHARE_MEMORY_METASPACE_CONSTANTS_HPP > #define SHARE_MEMORY_METASPACE_CONSTANTS_HPP > > #endif // SHARE_MEMORY_METASPACE_BLOCKFREELIST_HPP > > I actually took the allocation path when looking at this. Thank > you for > keeping many of the function names because that helped. > > thanks, > Coleen > > > Thanks for the review, Coleen. I will fix and post an update. I'll > have vacation the rest of the week and will post an updated webrev > start of next week, with your changes and some other small fish I did. Hopefully, there will be more comments from others waiting for you when you get back. Thanks, Coleen > > Cheers, Thomas From coleen.phillimore at oracle.com Thu Aug 20 00:35:37 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 19 Aug 2020 20:35:37 -0400 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: Message-ID: <5554e3f6-ba8e-c183-13f0-6e43abc34858@oracle.com> Hi Thomas, Here are some comments from reading through the gtests.? One you already answered. === Global test comment: ?* Copyright (c) 2018, 2020, Oracle and/or its affiliates. All rights reserved. These should probably only have 2020 for the Oracle copyright since these are new files. === metaspace_testhelper.hpp/cpp and metaspaceTestsContexts.hpp metaspace_testhelper.hpp/cpp is unused.? Looks like it was replaced with metaspaceTestContexts.hpp but left in the changeset. Ok, you said that already. The include guards weren't updated with the name change: #ifndef GTEST_METASPACE_METASPACE_TESTHELPER_HPP #define GTEST_METASPACE_METASPACE_TESTHELPER_HPP === metaspaceTestsCommon.hpp Why does it #include and use everything? The functions (get_random_size .. check_marked_range) appear to be in the global scope, which might affect linking libjvm.so for gtest in the future.? Could they be added to metaspaceTestContext? Or put in a namespace metaspace ? === test_chunkManager.cpp/test_chunkManager_stress.cpp Can these be combined? They are mostly the same. === test_is_metaspace_obj.cpp // Test the cheerful multitude of metaspace-contains-functions. Thank you for this! === test_metachunk.cpp You even have ascii art in the test.? This is great. >?? // start each split test with a fresh MetaspaceTestHelper. Should be ChunkTestsContext. === test_metaspacearena.cpp > // TODO: this class is very similar to MetaspaceArenaTestBed in test_metaspacearena_stress.cpp. > // should be unified. These tests have a lot of diffs with each other, and don't look the same to me. Maybe this is an out-of-date comment to be removed? === test_metaspace_counters.cpp This seems kind of obvious.? It looks like you wanted it to do more, because of the extra includes. I think you should remove it and add a new test with https://bugs.openjdk.java.net/browse/JDK-8252014. === test_report.cpp Why not test this with a jtreg test so you can search the output for useful strings. This test doesn't seem useful. That's all for gtests.? They seem quite thorough, so it's unlikely for anyone to accidentally break metaspace. Thanks, Coleen On 8/12/20 1:59 PM, Thomas St?fe wrote: > Dear all, > > I would like to start the review for the implementation of JEP 387 "Elastic > Metaspace". > > The relevant JBS items are: > > JEP: https://openjdk.java.net/jeps/387 > > Implementation Issue (pretty much only a placeholder currently): > https://bugs.openjdk.java.net/browse/JDK-8251158 > > -- > > Reminder of why we do this: > > 1. The new metaspace saves memory. How much depends on context: if it is > not a problem today it will continue to be none. But in cases where today > we face large metaspace consumption we may get reductions, sometimes > drastic ones. These reductions are caused by two facts: > - after mass class unloading we release memory more promptly to the OS > - even without class unloading chunk allocation is just plain smarter, > which reduces the overhead per class loader. This is especially true for > scenarios involving masses of small loaders which only load very few > classes. > > As an example, see [1] where two VMs - one stock, one patched - run the > same test program which creates tons of small loaders. The second run shows > a much reduced memory footprint and increased elasticity. > > 2. The rewritten metaspace code base got cleaner and less complex and thus > should be much easier to maintain and to extend. It also would be possible > to easily reuse the allocator for different parts of the VM, since it is > less tightly tied to the notion of just storing class metadata. > > -- > > In preparation of this review I prepared a guide [2], [3], which gives an > architectural overview and should be the starting point for an interested > Reviewer. > > The prototype has been tested extensively for quite some time in SAP's CI > system. We regularly run JCK test, JTReg tests and a whole battery of SAP > internal tests on 8 platforms. Tests are also currently ongoing at Oracle > and Red Hat. > > -- > > The full webrev [4] is somewhat large. In order to make this more bearable > I broke the patch up into three parts: > > 1) The core parts [5] > > This is the heart of the metaspace, mostly rewritten from scratch. I > propose any Reviewer should not look at the diff but rather just examine > the new implementation. One possible exception is metaspace.hpp, which is > the outside interface to metaspace. > > There are several ways to get a feeling for this code (after reading at > least the concept part of the provided guide [2]). The central, most > "beefy" classes are: > > - VirtualSpaceNode - does all the work of reserving, committing, > uncommitting memory > - RootChunkArea - does the grunt work of chunk splitting and merging > - ChunkManager - which takes care of the chunk freelists, of directing > chunk splits and merges, of enlarging chunks in place > - MetaspaceArena - which takes care of fine granular allocation on behalf > of a CLD, and of managing deallocated blocks. > > One way to review could be bottom up: starting at > VirtualSpaceNode/CommitMask/RootChunkArea(LUT), working your way up to > ChunkManager and Metachunk; finally up to to MetaspaceArena while taking a > side stroll to FreeBlocks/BinList/BlockTree. > > Another way would be to follow typical paths: > > Allocation path: Starting at MetaspaceArena::allocate() -> > Metachunk::allocate() (note the committing-on-demand code path starting > here) -> ChunkManager::get_chunk() -> > VirtualSpaceList/Node->allocate_root_chunk(). > > Destruction: ~MetaspaceArena() -> ChunkManager::return_chunk() -> (merging > chunks) -> (uncommitting chunks) > > Premature deallocation: -> MetaspaceArena::deallocate() -> see freeblock > list handling > > 2) The tests [6] > > This part of the patch contains all the new tests. There are a lot; the > test coverage of the new metaspace is very thorough. > > New gtests have been added under 'test/hotspot/gtest/metaspace'. > New jtreg have been added under > 'test/hotspot/jtreg/runtime/Metaspace/elastic' and under > 'test/hotspot/jtreg/gtest/MetaspaceGtests.java'. > > 4) Miscellaneous diffs [7] > > This is the part of the patch which is neither considered core nor test. > These changes are small, often uninteresting, and can be reviewed via diff. > > --- > > Out of scope for this patch is revamping outside metaspace statistics: > monitoring of metaspace statistics is mostly left untouched, beyond the > work needed to keep existing tests green. I wanted to keep those changes > separate from the core changes for JEP387. They are tracked in JDK-8251392 > [8] and JDK-8251342 [9], respectively. > > --- > > Code complexity: > > Numerically, lines of code went ever so slightly down with this patch: > > Before: (cloc hotspot/share/memory) > ------------------------------------------------------------------------------- > C++ 36 2533 3097 12735 > C/C++ Header 54 1728 2867 6278 > ------------------------------------------------------------------------------- > SUM: 90 4261 5964 19013 > > After: > ------------------------------------------------------------------------------- > C++ 50 3048 3774 13127 > C/C++ Header 63 2006 3575 5738 > ------------------------------------------------------------------------------- > SUM: 113 5054 7349 18865 > > But since the new implementation is more powerful than the old one - doing > things like committing/uncommitting on demand, guarding allocated blocks > etc - one could argue that the new solution does quite a lot more with > slightly less code, which I think is a good result. > > --- > > Idle musing: it would simplify matters quite a bit if we were to unify > class space and non-class metaspace. If we keep the current narrow Klass > pointer encoding scheme, we would still need to keep the space they are > stored in contiguous. But we could use the class space for everything, in > effect dropping the "class" moniker from the name. It would have to be > larger than it is currently, of course. > > Cons: > - we would have to abandon the notion of "limitless metaspace" - but if we > run with class space this has never been true anyway since the number of > classes is limited by the size of the compressed class space. > - virtual process size would go up since it now needs to be as large as the > total projected metaspace size. > - we may need to expand narrow Klass pointer encoding to include shifting, > if 4G are not enough to hold all metaspace data. > > Pros: > - this would simplify a lot of code. > - this would save real (committed) memory, since we save quite a bit of > overhead. > - we could narrow-pointer-encode other metadata too, not only Klass > pointers, should that ever be interesting > > ... but that is not part of this JEP. > > --- > > With this patch, we have a much stronger separation of concerns, and it > should be easy to reuse the metaspace allocator in other hotspot areas as > well. For instance, ResourceAreas and friends could be replaced by > MetaspaceArenas. They do almost the same thing. And as we have seen in the > past at SAP, the C-heap backed hotspot Arenas can be a pain since spikes in > Arena usage lingers forever as process footprint (we even once rewrote > parts of the arena code to use mmap, which is just on a more primitive > level what Metaspace does). > > --- > > I know this is short notice, but there will be a call for interested people > tomorrow at 11AM ET. In case any potential Reviewers are interested just > drop me a short note. > > --- > > Thank you, Thomas > > > [1] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/reduction-small-loaders.pdf > [2] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.pdf > [3] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.html > [4] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-all/webrev/ > [5] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-core/webrev/ > [6] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-test/webrev/ > [7] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-misc/webrev/ > [8] https://bugs.openjdk.java.net/browse/JDK-8251342 > [9] https://bugs.openjdk.java.net/browse/JDK-8251392 From nick.gasson at arm.com Thu Aug 20 03:08:27 2020 From: nick.gasson at arm.com (Nick Gasson) Date: Thu, 20 Aug 2020 11:08:27 +0800 Subject: RFR 8251490: [TESTBUG] The Java thread stack size specified is too small for nsk/stress/stack. Specify at least 448k In-Reply-To: <9f12fc32-646c-1aff-86fa-003db2641c91@oracle.com> References: <9f12fc32-646c-1aff-86fa-003db2641c91@oracle.com> Message-ID: <85imdexbdg.fsf@nicgas01-pc.shanghai.arm.com> Hi Harold, On 08/19/20 01:19 am, Harold Seigel wrote: > > Please review this small change to fix five vmTestbase/nsk/stress/stack > tests that are failing in tier5 on Linux AArch64. The change increases > the stack size for three of the tests. Increasing the stack size of the > other two tests could result in timeouts on other platforms. So, those > tests were changed to no longer run on AArch64. > > Open Webrev: > http://cr.openjdk.java.net/~hseigel/bug_8251490/webrev/index.html > Sorry this comment is a bit late, but stack008 and stack0019 are passing for me on an AArch64 system with 4k page size (the default for Ubuntu) so it seems a shame to disable them unconditionally. I think PPC64 can also be configured with 64k pages which would presumably have the same problem. Could we add a new jtreg property vm.pageSize and then make the @requires line vm.pageSize == 4096 instead of os.arch != "aarch64"? I'm happy to do that myself as this was pushed already. -- Thanks, Nick From yumin.qi at oracle.com Thu Aug 20 05:14:59 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Wed, 19 Aug 2020 22:14:59 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> Message-ID: <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> HI, Mandy ? Thanks for the review, I took one day off yesterday so just got a detail look of your reply. On 8/19/20 1:30 PM, Mandy Chung wrote: > On 8/17/20 12:37 PM, Yumin Qi wrote: >> Hi, Ioi >> >> ? Thanks for review/suggestion. I have updated the webrev at the following link: >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >> > > This patch leverages the TRACE_RESOLVE output and passes the trace output to VM.? VM then calls GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing and generate Holder class per the resolved LFs.?? I think there are other cleaner alternatives implementing this.?? jlink --generate-jli-classes plugin depends the trace output whereas -Xshare:dump does not.?? It's cleaner to skip generating the trace output and parsing for dumping shared archive purpose.? In addition, the implementation needs some cleanup (I can send you feedback on the next revision) > Current patch did not change the existing code for JLinkPlugin part. I just moved the parsing code from GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java since the former is an internal class to which we shouldn't call to generate holder classes. > Instead of relying on a system property "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. I remember we have such API to query if flag -Xshare:dump or -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This flag is the one used to log class name to list file. In GenerateLinkOptData.gmk: $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR) ??????? $(call MakeDir, $(LINK_OPT_DIR)) ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $(JLI_TRACE_FILE))) ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw \ ??????????? -Duser.language=en -Duser.country=US \ ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ ??????????? -Xmx128M -Xms128M $(LOG_INFO) ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw.2 \ ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ ??????????? -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ ??????????? -Duser.language=en -Duser.country=US \ ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ ??????????? build.tools.classlist.HelloClasslist \ ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ ??????????? || ( \ ??????????????? exitcode=$$? ; \ ??????????????? $(ECHO) "ERROR: Failed to generate link optimization data." \ ??????????????????? "This is likely a problem with the newly built JVM/JDK." ; \ ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ ??????????????? exit $$exitcode \ ??????????? ) ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current implementation, DumpLoadedClassList will turn on property java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same output sent to stdout and log file DumpLoadedClassList specified. Now instead of this property, using a vm interface API to query if this flag is set, I think it is better choice. But here I am NOT sure I understand your suggestion, I think there are two choices: 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set 2) Do not use DumpLoadedClassList, when -Xshare:dump? collecting those name, type and holder name to regenerate holder classes? At dump time, CDS generation relies on the classlist file, it may not trigger the generation of LF classes as at pre-run. So choice 2) may not work as expected. > > One possible approach: > > - Add the trace methods in GenerateJLIClassesHelper class to trace these resolved members: > ?? traceSpeciesType(String cn) > ?? tracePregeneratedLambdaForm(String name, MethodType type, Class holder) > > ?? * I'm not sure if these method should also accept the case when an exception is thrown but it's okay we add it as well to match existing implementation. > I will keep consistent with the current logic. > - these trace methods will either (1) send the trace output to System.out or (2) collect speciesTypes, invokerTypes, callSiteTypes, and dmhMethods in the runtime if -Xshare:dump is specified. > The trace method will send the log to DumpLoadedClassList file. At dump time, we reply on the trace log lines to regenerate holder classes. But if using choice 2), the trace will be store in a list and used to generate holder classes. > - ClassSpecializer and InvokerBytecodeGenerator will call these trace methods instead. > > - TRACE_RESOLVE is set when the system property is set to true. ? If -Xshare:dump is enabled,? the trace methods will not write to System.out.? Does CDS want a log of these entry?? I assume not? If DumpLoadedClassList set, we wish to get output like from TRACE_RESOLVE added to the log file. At pre-run with -XX:DumpLoadedClassList= we wish can collect as most loaded classes as we can for next dump run. > > - jlink plugin will continue to do the parsing.? It can construct a new? class that takes the > speciesTypes, invokerTypes, callSiteTypes, and dmhMethods parameters parsed from the trace output. jlink plugin can invoke an instance method to return Map. I suggest the key for this return map is the class name rather than a jimage entry name and the caller can map it to an jimage entry name if needed. I just move the parsing part to GenerateJLIClassesHelper, if keep the code in jlink plugin, we need a call interface from GenerateJLIClassesHelper and GenerateJLIPlugin to get the parse result, is that right? So the chain of calls will be cds(vm) -> GenerateJLIClassesHelper.generateMHHolderClasses -> JlinkPlugin parse the lines to get name, types -> GenerateJLIClassesHelper (via JLI) generate classes. My question is for a package java.lang.invoke to access jdk.internal.jlink looks not right. > > - CDS will invoke a different entry to return Object[] as you currently have. > Please let me know your comments next so I could go for a new patch. Thanks Yumin > What do you think? > > Mandy From ioi.lam at oracle.com Thu Aug 20 05:37:01 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 19 Aug 2020 22:37:01 -0700 Subject: RFR(XS) 8252056 Move DumpRegion/ReadClosure/WriteClosure to archiveUtils.hpp In-Reply-To: References: <6b312a7a-df99-4399-10d2-1122ce689dd3@oracle.com> Message-ID: <7856ff03-2654-bf0b-8bb6-95ec5aa94ccd@oracle.com> On 8/19/20 4:08 PM, Yumin Qi wrote: > Hi, Ioi > > ? Looks good to me. > > ? One question: with removing of #if INCLUDE_CDS from file > archiveUtils.cpp, will this file be added to non-CDS build? > Hi Yumin, Thanks for the review. archiveUtils.cpp is excluded from non-CDS build by the makefiles: http://hg.openjdk.java.net/jdk/jdk/file/fc8e62b399bd/make/hotspot/lib/JvmFeatures.gmk#l120 Our coding convention has been that "#if INCLUDE_CDS" should be avoided in such excluded files, to reduce clutter in the code. Thanks - Ioi > > Thanks > > Yumin > > > On 8/19/20 1:47 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8252056 >> http://cr.openjdk.java.net/~iklam/jdk16/8252056-move-dump-region-to-archive-utils.v01/ >> >> >> Part of CDS code clean up: >> >> The classes are currently in metaspaceShared.hpp/cpp, which are too big. >> It's better to move these utility classes to archiveUtils.cpp/hpp >> >> Thanks >> - Ioi From patricio.chilano.mateo at oracle.com Thu Aug 20 06:23:43 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Thu, 20 Aug 2020 03:23:43 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <6fb5da7a-28a6-4a2e-b93f-14409097abbb@default> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <29c3d48b-1875-8ea4-792a-157cdd07fe2c@oracle.com> <5f3a48b0-274f-c0f6-57a1-8fbb2bebf823@oracle.com> <9e6a0427-8d56-3bab-33f6-b2aa64268099@oracle.com> <6fb5da7a-28a6-4a2e-b93f-14409097abbb@default> Message-ID: <9e0e4233-afbf-eb3b-2aa3-e9a58c2de693@oracle.com> Hi Markus, Thanks for looking at this. On 8/19/20 3:20 PM, Markus Gronlund wrote: > Hi Patricio, > > Touching on something that Erik alluded to, and that is that the flag: > > 811 diagnostic(intx, DiagnoseSyncOnPrimitiveWrappers, 0, \ > 812 "Detect and take action upon identifying synchronization on " \ > 813 "primitive wrappers. Modes: " \ > 814 "0: off; " \ > 815 "1: exit with fatal error; " \ > 816 "2: log message to stdout. Log options can be customized " \ > 817 " with -Xlog:primitivewrappers; " \ > 818 "3: emit JFR event") \ > 819 range(0, 3) > > has become quite rich - also trying to auto-activate functionality if certain parts are enabled - for example it ergonomically enables UL. > > For JFR, this is not the best situation, because there all event control is already detailed in the .jfc configuration files. The flag as it is suggested would now add yet another place to know about to have the JFR events be generated. The flag mechanics can be simplified, at least for the "3. emit JFR event" option. This is because it is already implied when DiagnoseSyncOnPrimitiveWrappers is set (they are set to "enabled" in the .jfc's). It can therefore be removed as an option. Ok, makes sense. I removed it as an extra option. The JFR event will just be posted now if JFR is running for the non fatal case. > In general, and to simplify matters, maybe just let the individual subsystems (UL, JFR) configure if, and if so what they will do if information comes routed their way by means of this flag? If so, you might only need two options: "1. Exit with fatal error", "2. Log information" (send to UL, send to JFR) - (perhaps you want to provide convenience by activating the UL tags in this case). Right, in this case I need to activate the UL tag. Since the only purpose of this flag is to alert when an event occurs, just enabling the flag means I should output something without having to force the user to specify yet additional flags. That's why by default I activate the UL tag primitivewrappers to print to stdout. I could also just print to stdout without using UL, but then the user will not be able to change the output file unless I either add another flag, which I think would be too much for this, or I actually allow the user to specify that through UL and then when having to print I check if the tag is enabled to decide how to print. But for that last scenario since I will have to print anyways I might as well always use UL and configure the tag by default to print to stdout. I was also considering maybe to remove the posting of the JFR event altogether, but looking at this last version I think there is no harm in keeping it. If JFR is enabled we will just post an event when this flag is set to logging. Anyways, below is v6, let me know if you are okay with the changes: Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v6/inc/webrev/ Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v6/webrev/ Thanks! Patricio > Thanks > Markus > > > -----Original Message----- > From: Patricio Chilano > Sent: den 12 augusti 2020 00:14 > To: David Holmes ; Erik Gahlin > Cc: hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR 8242263: Diagnose synchronization on primitive wrappers > > > On 8/10/20 9:40 PM, David Holmes wrote: >> Hi Patricio, >> >> On 11/08/2020 6:03 am, Patricio Chilano wrote: >>> Hi Erik, >>> >>> Thanks for looking into this and for the offline discussions. >> If there are still issues with this regarding JFR then I would suggest >> we simply drop the use of JFR for reporting this. > Ok, I'll wait to see about the JFR changes. > >> Meanwhile looking at v5 incremental that all seems fine to me. > Great, thanks for looking into this David! > > > Thanks, > Patricio >> Thanks, >> David >> ----- >> >>> On 8/10/20 10:19 AM, Erik Gahlin wrote: >>>> Hi Patricio, >>>> >>>> I have tried to review the JFR changes, but I need more information >>>> on how the feature is going to be used. >>>> >>>> Is this something temporary that will be used gain insight for the >>>> Valhalla project, or do you think this will be needed longer term? >>>> If it is longer term, the bar is higher on what can be accepted. >>>> >>>> If this is a temporary feature, we could mark the event as >>>> experimental in metadata.xml, similar to what we did with GC events >>>> during the development of ZGC. Events that are experimental do not >>>> show up by default in visualization tools such as JMC and can be >>>> removed when they are not needed, or we have a better solution. >>>> >>> Right, it's temporary so we can mark it as experimental. I also >>> changed the category name to be "Java Virtual Machine, Diagnostics". >>> >>>> For events to be enabled in default.jfc, they should not cause more >>>> than 1% overhead, not even in pathological cases. To me, it seems >>>> this could happen if you make a loop where it is triggered all the >>>> time. >>>> >>>> For events in profile.jfc, the overhead should still be low (1-2%), >>>> but the target is the typical application. For example, allocation >>>> profiling is only enabled in profile.jfc. Some application that are >>>> allocation intensive could cause a higher overhead than 1%, but that >>>> is OK because that configuration is only to be used for a short >>>> period of time. That said, there is still a budget on how much space >>>> in a recording an event can take up, so it also depends on how >>>> important the information is for the user. To me, it seems this >>>> event will not be that important for the average user, which makes >>>> me think it should be disabled by default. >>>> >>> After our off-list discussion I left it as enabled. I also removed >>> the threshold setting and added the startTime=false in metadata.xml. >>> >>>> To complicate things, I noticed that aa command line flag is also >>>> used to enable the event. This is something we have worked hard to >>>> avoid when it comes to JFR events. All configuration should happen >>>> using configuration files. We had problem with this in the past >>>> where users gets confused why their events are not enabled. When we >>>> ported JFR to Hotspot, we got rid of those of those flags/events and >>>> it is a much better situation. Again, if this a temporary event to >>>> discover usage pattern for a release or two, it might be OK, but if >>>> we believe this event to stick around, I think we should look into >>>> alternatives, or not use JFR at all for this. >>>> >>> Yes, I see your point. Since we want to diagnose synchronization on >>> these classes but we don't want to affect performance the flag has to >>> be specified at startup otherwise the diagnostic instrumentation is >>> disabled, so I don't see a clear way to enable the events from JFR >>> alone. In any case, this will be a temporary event. >>> >>> Here is v5. Let me know if you are okay with the changes. >>> >>> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ >>> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/webrev/ >>> >>> Thanks Erik! >>> >>> Patricio >>>> Thanks >>>> >>>> Erik >>>> >>>>> On 28 Jul 2020, at 21:16, Patricio Chilano >>>>> >>>> > wrote: >>>>> >>>>> Hi all, >>>>> >>>>> Please review the following change that adds diagnostic >>>>> capabilities when synchronizing on primitive wrapper classes. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8242263 >>>>> Webrev: http://cr.openjdk.java.net/~pchilanomate/8242263/v1/webrev/ >>>>> >>>>> The new flag allows to identify synchronization on these classes >>>>> and to take one of the following actions: exit the VM with fatal >>>>> error, log a warning message, or issue a JFR event. The >>>>> implementation uses a simple approach where a check is added at the >>>>> beginning of the monitorenter generated code when the flag is >>>>> enabled to check whether the object is of a primitive wrapper >>>>> class. If it is, we jump to the slow path, otherwise we just >>>>> continue as always. The extra instructions will be: load the klass >>>>> of the object, load the _access_flags field for that klass, AND >>>>> with a constant, and branch based on the result. The code will only >>>>> be generated whenever the new opt-in diagnostic flag is enabled so >>>>> performance won't be affected when off. >>>>> >>>>> In addition to the purpose described in the description of the bug, >>>>> this flag will also be useful when trying to diagnose possible >>>>> synchronization issues if these classes ever become inline types as >>>>> part of the Valhalla project. >>>>> >>>>> I added test SyncOnPrimitiveWrapperTest.java that tests for the >>>>> exit and logging cases. I added test >>>>> TestSyncOnPrimitiveWrapperEvent.java to test for the JFR event case. >>>>> I tested the patch running tiers1-6 in mach5 with the flag set to >>>>> DiagnoseSyncOnPrimitiveWrappers=2. >>>>> I checked it builds with arm32 and ppc but can't run any tests on >>>>> those platforms, so it would be good if somebody can run the new >>>>> test included in the patch. >>>>> >>>>> Let me know if you think I should run or add any more tests. >>>>> >>>>> Thanks! >>>>> >>>>> Patricio From john.r.rose at oracle.com Thu Aug 20 07:44:38 2020 From: john.r.rose at oracle.com (John Rose) Date: Thu, 20 Aug 2020 00:44:38 -0700 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> Message-ID: <25F61E2F-327C-4924-8CB7-845C517BEBAA@oracle.com> On Aug 18, 2020, at 12:49 AM, Kim Barrett wrote: > > Poorly named and poorly commented variable from before the beginning of the mercurial age, > so hard to track down the changeset that introduced it to see if that shed some light. Good > thing we have some institutional memory :) You rang? The SCCS record looks like this: 1.3 97/08/11 11:39:04 steffen 4 3 +? +// ResourceArea ////////////////// + +#ifdef ASSERT +int ResourceArea::_warned = 0; +#endif + +ResourceArea::ResourceArea() { +? That delta (1.3/#4) replaced this code (along with lots of related code, in a large refactoring of ResourceArea). It is the first occurrence of the name ?warned?: - char* allocate_bytes(int size) { - assert(size >= 0, "negative size in allocate_bytes"); -#ifdef ASSERT - // NB: don't make it a fatal error -- otherwise, if you call certain functions - // from the debugger, it might report a leak since there might not be a - // ResourceMark. - // However, in all other situations, calling allocate_bytes with nesting == 0 - // is a definitive memory leak. -Urs 10/95 - static int warned = 0; // to suppress multiple warnings (e.g. when allocating from the debugger) - if (nesting < 1 && !warned++) error("memory leak: allocating w/o ResourceMark!"); -#endif Several months later we find today?s line: 1.41 99/01/28 14:01:01 renes 53 52 +debug_only(int ResourceArea::_warned;) // to suppress multiple warnings Yes, those little lines are all the repo metadata present (in per-file SCCS records). There are no checkin comments. Those were the days? ? John From markus.gronlund at oracle.com Thu Aug 20 09:29:21 2020 From: markus.gronlund at oracle.com (Markus Gronlund) Date: Thu, 20 Aug 2020 02:29:21 -0700 (PDT) Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: <9e0e4233-afbf-eb3b-2aa3-e9a58c2de693@oracle.com> References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <29c3d48b-1875-8ea4-792a-157cdd07fe2c@oracle.com> <5f3a48b0-274f-c0f6-57a1-8fbb2bebf823@oracle.com> <9e6a0427-8d56-3bab-33f6-b2aa64268099@oracle.com> <6fb5da7a-28a6-4a2e-b93f-14409097abbb@default> <9e0e4233-afbf-eb3b-2aa3-e9a58c2de693@oracle.com> Message-ID: Hi Patricio, The incremental update looks ok. Thank you Markus -----Original Message----- From: Patricio Chilano Sent: den 20 augusti 2020 08:24 To: Markus Gronlund ; David Holmes ; Erik Gahlin Cc: hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR 8242263: Diagnose synchronization on primitive wrappers Hi Markus, Thanks for looking at this. On 8/19/20 3:20 PM, Markus Gronlund wrote: > Hi Patricio, > > Touching on something that Erik alluded to, and that is that the flag: > > 811 diagnostic(intx, DiagnoseSyncOnPrimitiveWrappers, 0, \ > 812 "Detect and take action upon identifying synchronization on " \ > 813 "primitive wrappers. Modes: " \ > 814 "0: off; " \ > 815 "1: exit with fatal error; " \ > 816 "2: log message to stdout. Log options can be customized " \ > 817 " with -Xlog:primitivewrappers; " \ > 818 "3: emit JFR event") \ > 819 range(0, 3) > > has become quite rich - also trying to auto-activate functionality if certain parts are enabled - for example it ergonomically enables UL. > > For JFR, this is not the best situation, because there all event control is already detailed in the .jfc configuration files. The flag as it is suggested would now add yet another place to know about to have the JFR events be generated. The flag mechanics can be simplified, at least for the "3. emit JFR event" option. This is because it is already implied when DiagnoseSyncOnPrimitiveWrappers is set (they are set to "enabled" in the .jfc's). It can therefore be removed as an option. Ok, makes sense. I removed it as an extra option. The JFR event will just be posted now if JFR is running for the non fatal case. > In general, and to simplify matters, maybe just let the individual subsystems (UL, JFR) configure if, and if so what they will do if information comes routed their way by means of this flag? If so, you might only need two options: "1. Exit with fatal error", "2. Log information" (send to UL, send to JFR) - (perhaps you want to provide convenience by activating the UL tags in this case). Right, in this case I need to activate the UL tag. Since the only purpose of this flag is to alert when an event occurs, just enabling the flag means I should output something without having to force the user to specify yet additional flags. That's why by default I activate the UL tag primitivewrappers to print to stdout. I could also just print to stdout without using UL, but then the user will not be able to change the output file unless I either add another flag, which I think would be too much for this, or I actually allow the user to specify that through UL and then when having to print I check if the tag is enabled to decide how to print. But for that last scenario since I will have to print anyways I might as well always use UL and configure the tag by default to print to stdout. I was also considering maybe to remove the posting of the JFR event altogether, but looking at this last version I think there is no harm in keeping it. If JFR is enabled we will just post an event when this flag is set to logging. Anyways, below is v6, let me know if you are okay with the changes: Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v6/inc/webrev/ Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v6/webrev/ Thanks! Patricio > Thanks > Markus > > > -----Original Message----- > From: Patricio Chilano > Sent: den 12 augusti 2020 00:14 > To: David Holmes ; Erik Gahlin > > Cc: hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR 8242263: Diagnose synchronization on primitive > wrappers > > > On 8/10/20 9:40 PM, David Holmes wrote: >> Hi Patricio, >> >> On 11/08/2020 6:03 am, Patricio Chilano wrote: >>> Hi Erik, >>> >>> Thanks for looking into this and for the offline discussions. >> If there are still issues with this regarding JFR then I would >> suggest we simply drop the use of JFR for reporting this. > Ok, I'll wait to see about the JFR changes. > >> Meanwhile looking at v5 incremental that all seems fine to me. > Great, thanks for looking into this David! > > > Thanks, > Patricio >> Thanks, >> David >> ----- >> >>> On 8/10/20 10:19 AM, Erik Gahlin wrote: >>>> Hi Patricio, >>>> >>>> I have tried to review the JFR changes, but I need more information >>>> on how the feature is going to be used. >>>> >>>> Is this something temporary that will be used gain insight for the >>>> Valhalla project, or do you think this will be needed longer term? >>>> If it is longer term, the bar is higher on what can be accepted. >>>> >>>> If this is a temporary feature, we could mark the event as >>>> experimental in metadata.xml, similar to what we did with GC events >>>> during the development of ZGC. Events that are experimental do not >>>> show up by default in visualization tools such as JMC and can be >>>> removed when they are not needed, or we have a better solution. >>>> >>> Right, it's temporary so we can mark it as experimental. I also >>> changed the category name to be "Java Virtual Machine, Diagnostics". >>> >>>> For events to be enabled in default.jfc, they should not cause more >>>> than 1% overhead, not even in pathological cases. To me, it seems >>>> this could happen if you make a loop where it is triggered all the >>>> time. >>>> >>>> For events in profile.jfc, the overhead should still be low (1-2%), >>>> but the target is the typical application. For example, allocation >>>> profiling is only enabled in profile.jfc. Some application that are >>>> allocation intensive could cause a higher overhead than 1%, but >>>> that is OK because that configuration is only to be used for a >>>> short period of time. That said, there is still a budget on how >>>> much space in a recording an event can take up, so it also depends >>>> on how important the information is for the user. To me, it seems >>>> this event will not be that important for the average user, which >>>> makes me think it should be disabled by default. >>>> >>> After our off-list discussion I left it as enabled. I also removed >>> the threshold setting and added the startTime=false in metadata.xml. >>> >>>> To complicate things, I noticed that aa command line flag is also >>>> used to enable the event. This is something we have worked hard to >>>> avoid when it comes to JFR events. All configuration should happen >>>> using configuration files. We had problem with this in the past >>>> where users gets confused why their events are not enabled. When we >>>> ported JFR to Hotspot, we got rid of those of those flags/events >>>> and it is a much better situation. Again, if this a temporary event >>>> to discover usage pattern for a release or two, it might be OK, but >>>> if we believe this event to stick around, I think we should look >>>> into alternatives, or not use JFR at all for this. >>>> >>> Yes, I see your point. Since we want to diagnose synchronization on >>> these classes but we don't want to affect performance the flag has >>> to be specified at startup otherwise the diagnostic instrumentation >>> is disabled, so I don't see a clear way to enable the events from >>> JFR alone. In any case, this will be a temporary event. >>> >>> Here is v5. Let me know if you are okay with the changes. >>> >>> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ >>> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/webrev/ >>> >>> Thanks Erik! >>> >>> Patricio >>>> Thanks >>>> >>>> Erik >>>> >>>>> On 28 Jul 2020, at 21:16, Patricio Chilano >>>>> >>>> > wrote: >>>>> >>>>> Hi all, >>>>> >>>>> Please review the following change that adds diagnostic >>>>> capabilities when synchronizing on primitive wrapper classes. >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8242263 >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~pchilanomate/8242263/v1/webrev/ >>>>> >>>>> The new flag allows to identify synchronization on these classes >>>>> and to take one of the following actions: exit the VM with fatal >>>>> error, log a warning message, or issue a JFR event. The >>>>> implementation uses a simple approach where a check is added at >>>>> the beginning of the monitorenter generated code when the flag is >>>>> enabled to check whether the object is of a primitive wrapper >>>>> class. If it is, we jump to the slow path, otherwise we just >>>>> continue as always. The extra instructions will be: load the klass >>>>> of the object, load the _access_flags field for that klass, AND >>>>> with a constant, and branch based on the result. The code will >>>>> only be generated whenever the new opt-in diagnostic flag is >>>>> enabled so performance won't be affected when off. >>>>> >>>>> In addition to the purpose described in the description of the >>>>> bug, this flag will also be useful when trying to diagnose >>>>> possible synchronization issues if these classes ever become >>>>> inline types as part of the Valhalla project. >>>>> >>>>> I added test SyncOnPrimitiveWrapperTest.java that tests for the >>>>> exit and logging cases. I added test >>>>> TestSyncOnPrimitiveWrapperEvent.java to test for the JFR event case. >>>>> I tested the patch running tiers1-6 in mach5 with the flag set to >>>>> DiagnoseSyncOnPrimitiveWrappers=2. >>>>> I checked it builds with arm32 and ppc but can't run any tests on >>>>> those platforms, so it would be good if somebody can run the new >>>>> test included in the patch. >>>>> >>>>> Let me know if you think I should run or add any more tests. >>>>> >>>>> Thanks! >>>>> >>>>> Patricio From leo.korinth at oracle.com Thu Aug 20 10:24:24 2020 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 20 Aug 2020 12:24:24 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: Message-ID: Hi! Thanks for doing this, it does indeed look like a great improvement to me, also many thanks for the /great/ documentation of your change. Until I start my review on each part, I have a few general comments. 1) I believe structs should be named the same way as classes (UpperCamelCase) and not end with a "_t" (a naming convention that at least to me signals a typedef) 2) struct fields (and public class fields should to my knowledge be prefixed with underscore), there should be no naming difference between public and private fields. 3) I can see that you make most struct classes without constructors. Maybe there is some issue with trivial/standard-layout of classes, but otherwise I think constructors are really helpful. They both force you to initialize data and also gives your IDE help to warn you about uninitialized fields in the constructor. 4) As our build system (unfortunately) discard the path and thus requires all source files to be uniquely named, it might be good to use some kind of prefix for all metaspace files. If you do not like that idea (we use it in the different GCs for example), I think at least a few of the files could be given more specific names (counter.hpp and settings.?pp for example). Maybe prefix all files with "ms"? 5) No space between variable and increment operator (i ++ -> i++) 6) Enums should be named UpperCamelCase, as should their values (ALL_UPPER_CASE values are allowed according to the style guide, but I think most new code uses UpperCamelCase). 7) With c++ 14, I would like all enums to instead be enum classes, they are safer and better scoped, their underlying type can be specified and they have (to my knowledge) no downside except for eventual backports. 9) metaspace_test.?pp, I have to look more into this, but it does not feel right to mix in test this way, I guess it is hard to move though? 10) Constructor initialization lists use at least three different styles (placement of "," and ":") and while I personally like the Haskell style with comma first on the line, I think we should keep the style consistent with the rest of HotSpot. Thanks, Leo From harold.seigel at oracle.com Thu Aug 20 12:43:22 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Thu, 20 Aug 2020 08:43:22 -0400 Subject: RFR 8251490: [TESTBUG] The Java thread stack size specified is too small for nsk/stress/stack. Specify at least 448k In-Reply-To: <85imdexbdg.fsf@nicgas01-pc.shanghai.arm.com> References: <9f12fc32-646c-1aff-86fa-003db2641c91@oracle.com> <85imdexbdg.fsf@nicgas01-pc.shanghai.arm.com> Message-ID: <347361eb-a6cc-5373-cd8b-cce9c2056a7e@oracle.com> Hi Nick, I think that's a very good idea! I created JBS bug https://bugs.openjdk.java.net/browse/JDK-8252108. Please assign it to yourself if you want to do the fix. Thanks, Harold On 8/19/2020 11:08 PM, Nick Gasson wrote: > Hi Harold, > > On 08/19/20 01:19 am, Harold Seigel wrote: >> Please review this small change to fix five vmTestbase/nsk/stress/stack >> tests that are failing in tier5 on Linux AArch64. The change increases >> the stack size for three of the tests. Increasing the stack size of the >> other two tests could result in timeouts on other platforms. So, those >> tests were changed to no longer run on AArch64. >> >> Open Webrev: >> http://cr.openjdk.java.net/~hseigel/bug_8251490/webrev/index.html >> > Sorry this comment is a bit late, but stack008 and stack0019 are passing > for me on an AArch64 system with 4k page size (the default for Ubuntu) > so it seems a shame to disable them unconditionally. I think PPC64 can > also be configured with 64k pages which would presumably have the same > problem. > > Could we add a new jtreg property vm.pageSize and then make the > @requires line vm.pageSize == 4096 instead of os.arch != "aarch64"? I'm > happy to do that myself as this was pushed already. > > -- > Thanks, > Nick From coleen.phillimore at oracle.com Thu Aug 20 13:46:39 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 20 Aug 2020 09:46:39 -0400 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: Message-ID: <30cdde4f-3b2e-b8bb-e972-4f8ad131e890@oracle.com> Hi Thomas, Here are my comments from reading the jtreg test files: === MetaspaceGTests.java /* ?* Note: This runs the metaspace-related parts of gtest in configurations which ?*? are not tested explicitely in the standard gtests. ?* ?*/ explicitly typo. This is an interesting change.? I don't think there's another way to do this.? Others might have other suggestions. === MetaspaceTestContext.java Missing copyright. There are two classes called this (or close to this). I think the gtest version should be called (gtest) Helper, just to avoid confusion. === TestMetaspaceAllocation* ?* @test id=debug Is this something in the version of jtreg we test? These tests need @bug 8251158 even though it's not a bug. The tests look really good.? They cover and stress metaspace way more than all of our indirect testing. Coleen On 8/12/20 1:59 PM, Thomas St?fe wrote: > Dear all, > > I would like to start the review for the implementation of JEP 387 "Elastic > Metaspace". > > The relevant JBS items are: > > JEP: https://openjdk.java.net/jeps/387 > > Implementation Issue (pretty much only a placeholder currently): > https://bugs.openjdk.java.net/browse/JDK-8251158 > > -- > > Reminder of why we do this: > > 1. The new metaspace saves memory. How much depends on context: if it is > not a problem today it will continue to be none. But in cases where today > we face large metaspace consumption we may get reductions, sometimes > drastic ones. These reductions are caused by two facts: > - after mass class unloading we release memory more promptly to the OS > - even without class unloading chunk allocation is just plain smarter, > which reduces the overhead per class loader. This is especially true for > scenarios involving masses of small loaders which only load very few > classes. > > As an example, see [1] where two VMs - one stock, one patched - run the > same test program which creates tons of small loaders. The second run shows > a much reduced memory footprint and increased elasticity. > > 2. The rewritten metaspace code base got cleaner and less complex and thus > should be much easier to maintain and to extend. It also would be possible > to easily reuse the allocator for different parts of the VM, since it is > less tightly tied to the notion of just storing class metadata. > > -- > > In preparation of this review I prepared a guide [2], [3], which gives an > architectural overview and should be the starting point for an interested > Reviewer. > > The prototype has been tested extensively for quite some time in SAP's CI > system. We regularly run JCK test, JTReg tests and a whole battery of SAP > internal tests on 8 platforms. Tests are also currently ongoing at Oracle > and Red Hat. > > -- > > The full webrev [4] is somewhat large. In order to make this more bearable > I broke the patch up into three parts: > > 1) The core parts [5] > > This is the heart of the metaspace, mostly rewritten from scratch. I > propose any Reviewer should not look at the diff but rather just examine > the new implementation. One possible exception is metaspace.hpp, which is > the outside interface to metaspace. > > There are several ways to get a feeling for this code (after reading at > least the concept part of the provided guide [2]). The central, most > "beefy" classes are: > > - VirtualSpaceNode - does all the work of reserving, committing, > uncommitting memory > - RootChunkArea - does the grunt work of chunk splitting and merging > - ChunkManager - which takes care of the chunk freelists, of directing > chunk splits and merges, of enlarging chunks in place > - MetaspaceArena - which takes care of fine granular allocation on behalf > of a CLD, and of managing deallocated blocks. > > One way to review could be bottom up: starting at > VirtualSpaceNode/CommitMask/RootChunkArea(LUT), working your way up to > ChunkManager and Metachunk; finally up to to MetaspaceArena while taking a > side stroll to FreeBlocks/BinList/BlockTree. > > Another way would be to follow typical paths: > > Allocation path: Starting at MetaspaceArena::allocate() -> > Metachunk::allocate() (note the committing-on-demand code path starting > here) -> ChunkManager::get_chunk() -> > VirtualSpaceList/Node->allocate_root_chunk(). > > Destruction: ~MetaspaceArena() -> ChunkManager::return_chunk() -> (merging > chunks) -> (uncommitting chunks) > > Premature deallocation: -> MetaspaceArena::deallocate() -> see freeblock > list handling > > 2) The tests [6] > > This part of the patch contains all the new tests. There are a lot; the > test coverage of the new metaspace is very thorough. > > New gtests have been added under 'test/hotspot/gtest/metaspace'. > New jtreg have been added under > 'test/hotspot/jtreg/runtime/Metaspace/elastic' and under > 'test/hotspot/jtreg/gtest/MetaspaceGtests.java'. > > 4) Miscellaneous diffs [7] > > This is the part of the patch which is neither considered core nor test. > These changes are small, often uninteresting, and can be reviewed via diff. > > --- > > Out of scope for this patch is revamping outside metaspace statistics: > monitoring of metaspace statistics is mostly left untouched, beyond the > work needed to keep existing tests green. I wanted to keep those changes > separate from the core changes for JEP387. They are tracked in JDK-8251392 > [8] and JDK-8251342 [9], respectively. > > --- > > Code complexity: > > Numerically, lines of code went ever so slightly down with this patch: > > Before: (cloc hotspot/share/memory) > ------------------------------------------------------------------------------- > C++ 36 2533 3097 12735 > C/C++ Header 54 1728 2867 6278 > ------------------------------------------------------------------------------- > SUM: 90 4261 5964 19013 > > After: > ------------------------------------------------------------------------------- > C++ 50 3048 3774 13127 > C/C++ Header 63 2006 3575 5738 > ------------------------------------------------------------------------------- > SUM: 113 5054 7349 18865 > > But since the new implementation is more powerful than the old one - doing > things like committing/uncommitting on demand, guarding allocated blocks > etc - one could argue that the new solution does quite a lot more with > slightly less code, which I think is a good result. > > --- > > Idle musing: it would simplify matters quite a bit if we were to unify > class space and non-class metaspace. If we keep the current narrow Klass > pointer encoding scheme, we would still need to keep the space they are > stored in contiguous. But we could use the class space for everything, in > effect dropping the "class" moniker from the name. It would have to be > larger than it is currently, of course. > > Cons: > - we would have to abandon the notion of "limitless metaspace" - but if we > run with class space this has never been true anyway since the number of > classes is limited by the size of the compressed class space. > - virtual process size would go up since it now needs to be as large as the > total projected metaspace size. > - we may need to expand narrow Klass pointer encoding to include shifting, > if 4G are not enough to hold all metaspace data. > > Pros: > - this would simplify a lot of code. > - this would save real (committed) memory, since we save quite a bit of > overhead. > - we could narrow-pointer-encode other metadata too, not only Klass > pointers, should that ever be interesting > > ... but that is not part of this JEP. > > --- > > With this patch, we have a much stronger separation of concerns, and it > should be easy to reuse the metaspace allocator in other hotspot areas as > well. For instance, ResourceAreas and friends could be replaced by > MetaspaceArenas. They do almost the same thing. And as we have seen in the > past at SAP, the C-heap backed hotspot Arenas can be a pain since spikes in > Arena usage lingers forever as process footprint (we even once rewrote > parts of the arena code to use mmap, which is just on a more primitive > level what Metaspace does). > > --- > > I know this is short notice, but there will be a call for interested people > tomorrow at 11AM ET. In case any potential Reviewers are interested just > drop me a short note. > > --- > > Thank you, Thomas > > > [1] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/reduction-small-loaders.pdf > [2] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.pdf > [3] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.html > [4] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-all/webrev/ > [5] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-core/webrev/ > [6] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-test/webrev/ > [7] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-misc/webrev/ > [8] https://bugs.openjdk.java.net/browse/JDK-8251342 > [9] https://bugs.openjdk.java.net/browse/JDK-8251392 From erik.osterlund at oracle.com Thu Aug 20 13:58:47 2020 From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=) Date: Thu, 20 Aug 2020 15:58:47 +0200 Subject: RFR (S) 8252043: Move inner class metaspace cleaning out of safepoint cleanup tasks In-Reply-To: <50e42ffa-a210-8787-56c5-e19aebb58443@oracle.com> References: <50e42ffa-a210-8787-56c5-e19aebb58443@oracle.com> Message-ID: Hi Coleen, This is definitely a step in the right direction. Looks good! Thanks, /Erik On 2020-08-19 23:47, Coleen Phillimore wrote: > Summary: Clean up inner metaspaces from ServiceThread if cleanup is > needed for concurrent GCs. > > Moved inner metaspace cleaning back to class unloading, unless using > concurrent GC.? There we initiate a safepoint if inner metaspaces > require cleaning. > > Tested with tier1-6. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8252043.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8252043 > > Thanks, > Coleen From leo.korinth at oracle.com Thu Aug 20 14:36:00 2020 From: leo.korinth at oracle.com (Leo Korinth) Date: Thu, 20 Aug 2020 16:36:00 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: Message-ID: <4831ad6d-aaa6-24f9-d06d-a0d17179edda@oracle.com> Hi, some more general comments. I can see many tabs in the sources, please grep and replace them with spaces. Naming of constants (static const for example) seems to be a mix of lower_case or UPPER_CASE; the style guide says UPPER_CASE or UpperCamelCase (as far as I understand). This seems to be a mix of usage in HotSpot, what is preferred? We use UpperCamelCase in GC land. Static fields should not start with underscore as is done in the AllStatic Settings for example. They should not be lower_case either. Also, some of the includes could change order, I prefer the directory metaspace being ordered before header names starting with metaspace. In file /src/hotspot/share/gc/shared/gcTrace.cpp: [not sorted]: #include "gc/shared/referenceProcessorStats.hpp" (Context) #include "memory/metaspace.hpp" (Addition) #include "memory/heapInspection.hpp" (Context) In file /src/hotspot/share/gc/shared/genCollectedHeap.cpp: [not sorted]: #include "memory/metaspaceCounters.hpp" (Context) #include "memory/metaspace/metaspaceSizesSnapshot.hpp" (Addition) #include "memory/resourceArea.hpp" (Context) In file /src/hotspot/share/jfr/recorder/checkpoint/types/jfrType.cpp: [not sorted]: #include "memory/metaspace.hpp" (Addition) #include "memory/metaspace/metaspaceEnums.hpp" (Addition) #include "jfr/utilities/jfrThreadIterator.hpp" (Context) In file /src/hotspot/share/memory/metaspace.cpp: [not sorted]: #include "memory/metaspaceShared.hpp" (Addition) #include "memory/metaspaceTracer.hpp" (Addition) #include "memory/metaspace/chunkHeaderPool.hpp" (Addition) In file /src/hotspot/share/memory/metaspace.cpp: [not sorted]: #include "memory/metaspaceTracer.hpp" (Addition) #include "memory/metaspace/chunkHeaderPool.hpp" (Addition) #include "memory/metaspace/chunkManager.hpp" (Context) In file /src/hotspot/share/memory/metaspace/binlist.hpp: [not sorted]: #include "utilities/debug.hpp" (Addition) #include "utilities/globalDefinitions.hpp" (Addition) #include "memory/metaspace/counter.hpp" (Addition) In file /src/hotspot/share/memory/metaspace/binlist.hpp: [not sorted]: #include "utilities/globalDefinitions.hpp" (Addition) #include "memory/metaspace/counter.hpp" (Addition) #include "memory/metaspace/metaspaceCommon.hpp" (Addition) Thanks, Leo From coleen.phillimore at oracle.com Thu Aug 20 15:00:19 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 20 Aug 2020 11:00:19 -0400 Subject: RFR (S) 8252043: Move inner class metaspace cleaning out of safepoint cleanup tasks In-Reply-To: References: <50e42ffa-a210-8787-56c5-e19aebb58443@oracle.com> Message-ID: <7192652e-ddfd-a50a-2fbc-3d3ad6c85b2a@oracle.com> Thanks Erik! Coleen On 8/20/20 9:58 AM, Erik ?sterlund wrote: > Hi Coleen, > > This is definitely a step in the right direction. Looks good! > > Thanks, > /Erik > > On 2020-08-19 23:47, Coleen Phillimore wrote: >> Summary: Clean up inner metaspaces from ServiceThread if cleanup is >> needed for concurrent GCs. >> >> Moved inner metaspace cleaning back to class unloading, unless using >> concurrent GC.? There we initiate a safepoint if inner metaspaces >> require cleaning. >> >> Tested with tier1-6. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8252043.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8252043 >> >> Thanks, >> Coleen > From igor.ignatyev at oracle.com Thu Aug 20 17:16:31 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 20 Aug 2020 10:16:31 -0700 Subject: RFR(T) : 8252005 : narrow disabling of allowSmartActionArgs in vmTestbase In-Reply-To: <17a8369e-5f38-ebab-974b-28e083378aa2@oracle.com> References: <4E6FECE6-9103-46ED-84B2-79DBA0123ED9@oracle.com> <17a8369e-5f38-ebab-974b-28e083378aa2@oracle.com> Message-ID: Hi Serguei, thanks for your review. I've decided to slightly modify the patch and use the ids of subtasks in TEST.properties files (instead of main bug id) in order to avoid possible confusion in the future: - incremental: http://cr.openjdk.java.net/~iignatyev//8252005/webrev.0-1/index.html - whole: http://cr.openjdk.java.net/~iignatyev//8252005/webrev.01/index.html could you please re-review it? Thanks, -- Igor > On Aug 19, 2020, at 4:22 PM, serguei.spitsyn at oracle.com wrote: > > Hi Igor, > > This looks reasonable. > > Thanks, > Serguei > > > On 8/18/20 16:42, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8252005/webrev.00/ >>> 0 lines changed: 0 ins; 0 del; 0 mod; >> Hi all, >> >> could you please review this trivial (and apparently empty) patch which sets allowSmartActionArgs to false only in subdirectories of vmTestbase which currently use PropertyResolvingWrapper? >> >> (it's hard to tell from webrev or patch, but test/hotspot/jtreg/vmTestbase/TEST.properties is effectively removed) >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8252005/webrev.00/ >> JBS: https://bugs.openjdk.java.net/browse/JDK-8252005 >> >> Thanks, >> -- Igor >> >> > From yumin.qi at oracle.com Thu Aug 20 17:46:51 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Thu, 20 Aug 2020 10:46:51 -0700 Subject: RFR(XS) 8249096: Clean up code for DumpLoadedClassList Message-ID: Hi, Please review the very small change for cleaning up DumpLoadedClassList code. bug: https://bugs.openjdk.java.net/browse/JDK-8249096 Webrev: http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-01/ There are two places where when DumpLoadedClassList is set we log loaded class to log file, one is in ClassFileParser::parse_stream, and the other is in SystemDictionaryShared::load_shared_class_misc which is after shared class loaded from CDS. The former is positioned after parsing constants, and before parsing interface, fields and methods etc. There is possibility an exception or error happens before the InstanceKlass is created, so the fix move the logging to after the InstanceKlass is successfully created, also move the logging code to InstanceKlass which seems more reasonable. Test: local jtreg on cds. Mach5 tier1-4 Thanks Yumin From igor.ignatyev at oracle.com Thu Aug 20 18:18:19 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 20 Aug 2020 11:18:19 -0700 Subject: RFR(T) : 8252005 : narrow disabling of allowSmartActionArgs in vmTestbase In-Reply-To: <8eb1187f-8030-2adf-b20d-d289bfa35198@oracle.com> References: <4E6FECE6-9103-46ED-84B2-79DBA0123ED9@oracle.com> <17a8369e-5f38-ebab-974b-28e083378aa2@oracle.com> <8eb1187f-8030-2adf-b20d-d289bfa35198@oracle.com> Message-ID: <3CB6B3FF-458B-4B76-872B-46A6D30B7A33@oracle.com> thanks Serguei, pushed. -- Igor > On Aug 20, 2020, at 10:55 AM, serguei.spitsyn at oracle.com wrote: > > Hi Igor, > > Still looks good to me. > The webrev is veeeeery slow. > > Thanks, > Serguei > > > On 8/20/20 10:16, Igor Ignatyev wrote: >> Hi Serguei, >> >> thanks for your review. I've decided to slightly modify the patch and use the ids of subtasks in TEST.properties files (instead of main bug id) in order to avoid possible confusion in the future: >> - incremental: http://cr.openjdk.java.net/~iignatyev//8252005/webrev.0-1/index.html >> - whole: http://cr.openjdk.java.net/~iignatyev//8252005/webrev.01/index.html >> >> could you please re-review it? >> >> Thanks, >> -- Igor >> >>> On Aug 19, 2020, at 4:22 PM, serguei.spitsyn at oracle.com wrote: >>> >>> Hi Igor, >>> >>> This looks reasonable. >>> >>> Thanks, >>> Serguei >>> >>> >>> On 8/18/20 16:42, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev//8252005/webrev.00/ >>>>> 0 lines changed: 0 ins; 0 del; 0 mod; >>>> Hi all, >>>> >>>> could you please review this trivial (and apparently empty) patch which sets allowSmartActionArgs to false only in subdirectories of vmTestbase which currently use PropertyResolvingWrapper? >>>> >>>> (it's hard to tell from webrev or patch, but test/hotspot/jtreg/vmTestbase/TEST.properties is effectively removed) >>>> >>>> webrev: http://cr.openjdk.java.net/~iignatyev//8252005/webrev.00/ >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8252005 >>>> >>>> Thanks, >>>> -- Igor >>>> >>>> >>> >> > From ioi.lam at oracle.com Thu Aug 20 18:37:43 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 20 Aug 2020 11:37:43 -0700 Subject: RFR(XS) 8249096: Clean up code for DumpLoadedClassList In-Reply-To: References: Message-ID: <2343e9d9-ea24-0558-9867-d11418bbba61@oracle.com> Hi Yumin, This looks like a good clean up. I think it can be further simplified by moving the call to InstanceKlass::log_to_classlist to beginning of InstanceKlass::print_class_load_logging(). Thanks - Ioi On 8/20/20 10:46 AM, Yumin Qi wrote: > Hi, Please review the very small change for cleaning up > DumpLoadedClassList code. > > > bug: https://bugs.openjdk.java.net/browse/JDK-8249096 > > Webrev: http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-01/ > > > There are two places where when DumpLoadedClassList is set we log > loaded class to log file, one is in ClassFileParser::parse_stream, and > the other is in SystemDictionaryShared::load_shared_class_misc which > is after shared class loaded from CDS. The former is positioned after > parsing constants, and before parsing interface, fields and methods > etc. There is possibility an exception or error happens before the > InstanceKlass is created, so the fix move the logging to after the > InstanceKlass is successfully created, also move the logging code to > InstanceKlass which seems more reasonable. > > > Test: local jtreg on cds. > > Mach5 tier1-4 > > > Thanks > > Yumin > From patricio.chilano.mateo at oracle.com Thu Aug 20 18:46:26 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Thu, 20 Aug 2020 15:46:26 -0300 Subject: RFR 8242263: Diagnose synchronization on primitive wrappers In-Reply-To: References: <55fd6e9c-4007-2e67-ddff-f13c278cfefc@oracle.com> <29c3d48b-1875-8ea4-792a-157cdd07fe2c@oracle.com> <5f3a48b0-274f-c0f6-57a1-8fbb2bebf823@oracle.com> <9e6a0427-8d56-3bab-33f6-b2aa64268099@oracle.com> <6fb5da7a-28a6-4a2e-b93f-14409097abbb@default> <9e0e4233-afbf-eb3b-2aa3-e9a58c2de693@oracle.com> Message-ID: On 8/20/20 6:29 AM, Markus Gronlund wrote: > Hi Patricio, > > The incremental update looks ok. Great, thanks for looking at this Markus! Patricio > Thank you > Markus > > -----Original Message----- > From: Patricio Chilano > Sent: den 20 augusti 2020 08:24 > To: Markus Gronlund ; David Holmes ; Erik Gahlin > Cc: hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR 8242263: Diagnose synchronization on primitive wrappers > > Hi Markus, > > Thanks for looking at this. > > On 8/19/20 3:20 PM, Markus Gronlund wrote: >> Hi Patricio, >> >> Touching on something that Erik alluded to, and that is that the flag: >> >> 811 diagnostic(intx, DiagnoseSyncOnPrimitiveWrappers, 0, \ >> 812 "Detect and take action upon identifying synchronization on " \ >> 813 "primitive wrappers. Modes: " \ >> 814 "0: off; " \ >> 815 "1: exit with fatal error; " \ >> 816 "2: log message to stdout. Log options can be customized " \ >> 817 " with -Xlog:primitivewrappers; " \ >> 818 "3: emit JFR event") \ >> 819 range(0, 3) >> >> has become quite rich - also trying to auto-activate functionality if certain parts are enabled - for example it ergonomically enables UL. >> >> For JFR, this is not the best situation, because there all event control is already detailed in the .jfc configuration files. The flag as it is suggested would now add yet another place to know about to have the JFR events be generated. The flag mechanics can be simplified, at least for the "3. emit JFR event" option. This is because it is already implied when DiagnoseSyncOnPrimitiveWrappers is set (they are set to "enabled" in the .jfc's). It can therefore be removed as an option. > Ok, makes sense. I removed it as an extra option. The JFR event will just be posted now if JFR is running for the non fatal case. > >> In general, and to simplify matters, maybe just let the individual subsystems (UL, JFR) configure if, and if so what they will do if information comes routed their way by means of this flag? If so, you might only need two options: "1. Exit with fatal error", "2. Log information" (send to UL, send to JFR) - (perhaps you want to provide convenience by activating the UL tags in this case). > Right, in this case I need to activate the UL tag. Since the only purpose of this flag is to alert when an event occurs, just enabling the flag means I should output something without having to force the user to specify yet additional flags. That's why by default I activate the UL tag primitivewrappers to print to stdout. I could also just print to stdout without using UL, but then the user will not be able to change the output file unless I either add another flag, which I think would be too much for this, or I actually allow the user to specify that through UL and then when having to print I check if the tag is enabled to decide how to print. But for that last scenario since I will have to print anyways I might as well always use UL and configure the tag by default to print to stdout. > > I was also considering maybe to remove the posting of the JFR event altogether, but looking at this last version I think there is no harm in keeping it. If JFR is enabled we will just post an event when this flag is set to logging. > > Anyways, below is v6, let me know if you are okay with the changes: > > Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v6/inc/webrev/ > Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v6/webrev/ > > Thanks! > > Patricio >> Thanks >> Markus >> >> >> -----Original Message----- >> From: Patricio Chilano >> Sent: den 12 augusti 2020 00:14 >> To: David Holmes ; Erik Gahlin >> >> Cc: hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR 8242263: Diagnose synchronization on primitive >> wrappers >> >> >> On 8/10/20 9:40 PM, David Holmes wrote: >>> Hi Patricio, >>> >>> On 11/08/2020 6:03 am, Patricio Chilano wrote: >>>> Hi Erik, >>>> >>>> Thanks for looking into this and for the offline discussions. >>> If there are still issues with this regarding JFR then I would >>> suggest we simply drop the use of JFR for reporting this. >> Ok, I'll wait to see about the JFR changes. >> >>> Meanwhile looking at v5 incremental that all seems fine to me. >> Great, thanks for looking into this David! >> >> >> Thanks, >> Patricio >>> Thanks, >>> David >>> ----- >>> >>>> On 8/10/20 10:19 AM, Erik Gahlin wrote: >>>>> Hi Patricio, >>>>> >>>>> I have tried to review the JFR changes, but I need more information >>>>> on how the feature is going to be used. >>>>> >>>>> Is this something temporary that will be used gain insight for the >>>>> Valhalla project, or do you think this will be needed longer term? >>>>> If it is longer term, the bar is higher on what can be accepted. >>>>> >>>>> If this is a temporary feature, we could mark the event as >>>>> experimental in metadata.xml, similar to what we did with GC events >>>>> during the development of ZGC. Events that are experimental do not >>>>> show up by default in visualization tools such as JMC and can be >>>>> removed when they are not needed, or we have a better solution. >>>>> >>>> Right, it's temporary so we can mark it as experimental. I also >>>> changed the category name to be "Java Virtual Machine, Diagnostics". >>>> >>>>> For events to be enabled in default.jfc, they should not cause more >>>>> than 1% overhead, not even in pathological cases. To me, it seems >>>>> this could happen if you make a loop where it is triggered all the >>>>> time. >>>>> >>>>> For events in profile.jfc, the overhead should still be low (1-2%), >>>>> but the target is the typical application. For example, allocation >>>>> profiling is only enabled in profile.jfc. Some application that are >>>>> allocation intensive could cause a higher overhead than 1%, but >>>>> that is OK because that configuration is only to be used for a >>>>> short period of time. That said, there is still a budget on how >>>>> much space in a recording an event can take up, so it also depends >>>>> on how important the information is for the user. To me, it seems >>>>> this event will not be that important for the average user, which >>>>> makes me think it should be disabled by default. >>>>> >>>> After our off-list discussion I left it as enabled. I also removed >>>> the threshold setting and added the startTime=false in metadata.xml. >>>> >>>>> To complicate things, I noticed that aa command line flag is also >>>>> used to enable the event. This is something we have worked hard to >>>>> avoid when it comes to JFR events. All configuration should happen >>>>> using configuration files. We had problem with this in the past >>>>> where users gets confused why their events are not enabled. When we >>>>> ported JFR to Hotspot, we got rid of those of those flags/events >>>>> and it is a much better situation. Again, if this a temporary event >>>>> to discover usage pattern for a release or two, it might be OK, but >>>>> if we believe this event to stick around, I think we should look >>>>> into alternatives, or not use JFR at all for this. >>>>> >>>> Yes, I see your point. Since we want to diagnose synchronization on >>>> these classes but we don't want to affect performance the flag has >>>> to be specified at startup otherwise the diagnostic instrumentation >>>> is disabled, so I don't see a clear way to enable the events from >>>> JFR alone. In any case, this will be a temporary event. >>>> >>>> Here is v5. Let me know if you are okay with the changes. >>>> >>>> Inc: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/inc/webrev/ >>>> Full: http://cr.openjdk.java.net/~pchilanomate/8242263/v5/webrev/ >>>> >>>> Thanks Erik! >>>> >>>> Patricio >>>>> Thanks >>>>> >>>>> Erik >>>>> >>>>>> On 28 Jul 2020, at 21:16, Patricio Chilano >>>>>> >>>>> > wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> Please review the following change that adds diagnostic >>>>>> capabilities when synchronizing on primitive wrapper classes. >>>>>> >>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8242263 >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~pchilanomate/8242263/v1/webrev/ >>>>>> >>>>>> The new flag allows to identify synchronization on these classes >>>>>> and to take one of the following actions: exit the VM with fatal >>>>>> error, log a warning message, or issue a JFR event. The >>>>>> implementation uses a simple approach where a check is added at >>>>>> the beginning of the monitorenter generated code when the flag is >>>>>> enabled to check whether the object is of a primitive wrapper >>>>>> class. If it is, we jump to the slow path, otherwise we just >>>>>> continue as always. The extra instructions will be: load the klass >>>>>> of the object, load the _access_flags field for that klass, AND >>>>>> with a constant, and branch based on the result. The code will >>>>>> only be generated whenever the new opt-in diagnostic flag is >>>>>> enabled so performance won't be affected when off. >>>>>> >>>>>> In addition to the purpose described in the description of the >>>>>> bug, this flag will also be useful when trying to diagnose >>>>>> possible synchronization issues if these classes ever become >>>>>> inline types as part of the Valhalla project. >>>>>> >>>>>> I added test SyncOnPrimitiveWrapperTest.java that tests for the >>>>>> exit and logging cases. I added test >>>>>> TestSyncOnPrimitiveWrapperEvent.java to test for the JFR event case. >>>>>> I tested the patch running tiers1-6 in mach5 with the flag set to >>>>>> DiagnoseSyncOnPrimitiveWrappers=2. >>>>>> I checked it builds with arm32 and ppc but can't run any tests on >>>>>> those platforms, so it would be good if somebody can run the new >>>>>> test included in the patch. >>>>>> >>>>>> Let me know if you think I should run or add any more tests. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> Patricio From daniel.daugherty at oracle.com Thu Aug 20 20:49:23 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 20 Aug 2020 16:49:23 -0400 Subject: RFR(T): 8252126: 'GVars.stw_random = os::random()' lost by JDK-8246476 Message-ID: <113d695e-fb19-90ad-d271-8fe7da6e265b@oracle.com> Greetings, Thanks to Erik O. for spotting this lost line of code! I have a trivial fix for a piece of code that was accidentally lost with the push of another ObjectMonitor fix (8246476). Here's the bug ID for restoring the line of code: ??? JDK-8252126 'GVars.stw_random = os::random()' lost by JDK-8246476 ??? https://bugs.openjdk.java.net/browse/JDK-8252126 And here's the context diff for the trivial change itself: $ hg diff -r qparent diff -r 8f73aeccb27c src/hotspot/share/runtime/synchronizer.cpp --- a/src/hotspot/share/runtime/synchronizer.cpp??? Thu Aug 20 11:12:00 2020 -0700 +++ b/src/hotspot/share/runtime/synchronizer.cpp??? Thu Aug 20 16:40:10 2020 -0400 @@ -2322,6 +2322,8 @@ Atomic::load(&om_list_globals._free_count), Atomic::load(&om_list_globals._wait_count)); +? GVars.stw_random = os::random(); + ?? // The ServiceThread's async deflation request has been processed. ?? _last_async_deflation_time_ns = os::javaTimeNanos(); ?? set_is_async_deflation_requested(false); I've added a couple of analysis comments to the bug report for anyone that wants to read the gory details. This fix has been tested as part of a larger ObjectMonitor fix that will take some time to get through the code review, stress test and performance testing process. Thanks, in advance, for any comments, questions or suggestions. Dan From erik.osterlund at oracle.com Thu Aug 20 20:53:14 2020 From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=) Date: Thu, 20 Aug 2020 22:53:14 +0200 Subject: RFR(T): 8252126: 'GVars.stw_random = os::random()' lost by JDK-8246476 In-Reply-To: <113d695e-fb19-90ad-d271-8fe7da6e265b@oracle.com> References: <113d695e-fb19-90ad-d271-8fe7da6e265b@oracle.com> Message-ID: Hi Dan, Looks good. Thanks, /Erik > On 20 Aug 2020, at 22:49, Daniel D. Daugherty wrote: > > ?Greetings, > > Thanks to Erik O. for spotting this lost line of code! > > I have a trivial fix for a piece of code that was accidentally lost with > the push of another ObjectMonitor fix (8246476). Here's the bug ID for > restoring the line of code: > > JDK-8252126 'GVars.stw_random = os::random()' lost by JDK-8246476 > https://bugs.openjdk.java.net/browse/JDK-8252126 > > And here's the context diff for the trivial change itself: > > $ hg diff -r qparent > diff -r 8f73aeccb27c src/hotspot/share/runtime/synchronizer.cpp > --- a/src/hotspot/share/runtime/synchronizer.cpp Thu Aug 20 11:12:00 2020 -0700 > +++ b/src/hotspot/share/runtime/synchronizer.cpp Thu Aug 20 16:40:10 2020 -0400 > @@ -2322,6 +2322,8 @@ > Atomic::load(&om_list_globals._free_count), > Atomic::load(&om_list_globals._wait_count)); > > + GVars.stw_random = os::random(); > + > // The ServiceThread's async deflation request has been processed. > _last_async_deflation_time_ns = os::javaTimeNanos(); > set_is_async_deflation_requested(false); > > > I've added a couple of analysis comments to the bug report for anyone > that wants to read the gory details. > > This fix has been tested as part of a larger ObjectMonitor fix that > will take some time to get through the code review, stress test and > performance testing process. > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > From daniel.daugherty at oracle.com Thu Aug 20 20:54:05 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 20 Aug 2020 16:54:05 -0400 Subject: RFR(T): 8252126: 'GVars.stw_random = os::random()' lost by JDK-8246476 In-Reply-To: References: <113d695e-fb19-90ad-d271-8fe7da6e265b@oracle.com> Message-ID: Thanks for the fast review Erik! Dan On 8/20/20 4:53 PM, Erik ?sterlund wrote: > Hi Dan, > > Looks good. > > Thanks, > /Erik > >> On 20 Aug 2020, at 22:49, Daniel D. Daugherty wrote: >> >> ?Greetings, >> >> Thanks to Erik O. for spotting this lost line of code! >> >> I have a trivial fix for a piece of code that was accidentally lost with >> the push of another ObjectMonitor fix (8246476). Here's the bug ID for >> restoring the line of code: >> >> JDK-8252126 'GVars.stw_random = os::random()' lost by JDK-8246476 >> https://bugs.openjdk.java.net/browse/JDK-8252126 >> >> And here's the context diff for the trivial change itself: >> >> $ hg diff -r qparent >> diff -r 8f73aeccb27c src/hotspot/share/runtime/synchronizer.cpp >> --- a/src/hotspot/share/runtime/synchronizer.cpp Thu Aug 20 11:12:00 2020 -0700 >> +++ b/src/hotspot/share/runtime/synchronizer.cpp Thu Aug 20 16:40:10 2020 -0400 >> @@ -2322,6 +2322,8 @@ >> Atomic::load(&om_list_globals._free_count), >> Atomic::load(&om_list_globals._wait_count)); >> >> + GVars.stw_random = os::random(); >> + >> // The ServiceThread's async deflation request has been processed. >> _last_async_deflation_time_ns = os::javaTimeNanos(); >> set_is_async_deflation_requested(false); >> >> >> I've added a couple of analysis comments to the bug report for anyone >> that wants to read the gory details. >> >> This fix has been tested as part of a larger ObjectMonitor fix that >> will take some time to get through the code review, stress test and >> performance testing process. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> From daniel.daugherty at oracle.com Thu Aug 20 20:56:41 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 20 Aug 2020 16:56:41 -0400 Subject: RFR(T): 8252125: add an "inflating" entry to the "table" of bit patterns in share/oops/markWord.hpp Message-ID: Greetings, I have a trivial fix to add more clarification to markWord bit patterns. Here's the bug report: ??? JDK-8252125 add an "inflating" entry to the "table" of bit patterns in ??????????????? share/oops/markWord.hpp ??? https://bugs.openjdk.java.net/browse/JDK-8252125 And here's the context diff for the trivial change itself: $ hg diff -r qparent diff -r 8f73aeccb27c src/hotspot/share/oops/markWord.hpp --- a/src/hotspot/share/oops/markWord.hpp??? Thu Aug 20 11:12:00 2020 -0700 +++ b/src/hotspot/share/oops/markWord.hpp??? Thu Aug 20 16:39:11 2020 -0400 @@ -84,12 +84,13 @@ ?//??? [header????? | 0 | 01]? unlocked?????????? regular object header ?//??? [ptr???????????? | 10]? monitor??????????? inflated lock (header is wapped out) ?//??? [ptr???????????? | 11]? marked???????????? used to mark an object +//??? [0 ............ 0| 00]? inflating????????? inflation in progress ?// ?//??? We assume that stack/thread pointers have the lowest two bits cleared. ?// -//? - INFLATING() is a distinguished markword value that is used when -//??? inflating an existing stack-lock into an ObjectMonitor. See below -//??? for is_being_inflated() and INFLATING(). +//? - INFLATING() is a distinguished markword value of all zeros that is +//??? used when inflating an existing stack-lock into an ObjectMonitor. +//??? See below for is_being_inflated() and INFLATING(). ?class BasicLock; ?class ObjectMonitor; Thanks, in advance, for any comments, questions or suggestions. Dan From yumin.qi at oracle.com Thu Aug 20 20:57:02 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Thu, 20 Aug 2020 13:57:02 -0700 Subject: RFR(XS) 8249096: Clean up code for DumpLoadedClassList In-Reply-To: <2343e9d9-ea24-0558-9867-d11418bbba61@oracle.com> References: <2343e9d9-ea24-0558-9867-d11418bbba61@oracle.com> Message-ID: <9157b9e6-2ba5-e79e-d487-e98e44aad9dd@oracle.com> Ioi, ? Thanks for the review, I will update with new webrev --- yes, print_class_load_logging will work the same way. Thanks YUmin On 8/20/20 11:37 AM, Ioi Lam wrote: > Hi Yumin, > > This looks like a good clean up. I think it can be further simplified by moving the call to InstanceKlass::log_to_classlist to beginning of InstanceKlass::print_class_load_logging(). > > Thanks > - Ioi > > On 8/20/20 10:46 AM, Yumin Qi wrote: >> Hi, Please review the very small change for cleaning up DumpLoadedClassList code. >> >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8249096 >> >> Webrev: http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-01/ >> >> >> There are two places where when DumpLoadedClassList is set we log loaded class to log file, one is in ClassFileParser::parse_stream, and the other is in SystemDictionaryShared::load_shared_class_misc which is after shared class loaded from CDS. The former is positioned after parsing constants, and before parsing interface, fields and methods etc. There is possibility an exception or error happens before the InstanceKlass is created, so the fix move the logging to after the InstanceKlass is successfully created, also move the logging code to InstanceKlass which seems more reasonable. >> >> >> Test: local jtreg on cds. >> >> Mach5 tier1-4 >> >> >> Thanks >> >> Yumin >> > From patricio.chilano.mateo at oracle.com Thu Aug 20 21:11:40 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Thu, 20 Aug 2020 18:11:40 -0300 Subject: RFR (S) 8252043: Move inner class metaspace cleaning out of safepoint cleanup tasks In-Reply-To: <50e42ffa-a210-8787-56c5-e19aebb58443@oracle.com> References: <50e42ffa-a210-8787-56c5-e19aebb58443@oracle.com> Message-ID: <8fd8dcc2-276c-2db2-e300-d0b89eb170ab@oracle.com> Hi Coleen, Changes look good to me. I think you could have also just checked for SafepointSynchronize::is_at_safepoint() in ClassLoaderDataGraph::purge() instead of having the caller figure it out. I see that for all the current callers the value is always constant though so you save that check I guess. Thanks, Patricio On 8/19/20 6:47 PM, Coleen Phillimore wrote: > Summary: Clean up inner metaspaces from ServiceThread if cleanup is > needed for concurrent GCs. > > Moved inner metaspace cleaning back to class unloading, unless using > concurrent GC.? There we initiate a safepoint if inner metaspaces > require cleaning. > > Tested with tier1-6. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8252043.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8252043 > > Thanks, > Coleen From coleen.phillimore at oracle.com Thu Aug 20 21:43:35 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 20 Aug 2020 17:43:35 -0400 Subject: RFR (S) 8252043: Move inner class metaspace cleaning out of safepoint cleanup tasks In-Reply-To: <8fd8dcc2-276c-2db2-e300-d0b89eb170ab@oracle.com> References: <50e42ffa-a210-8787-56c5-e19aebb58443@oracle.com> <8fd8dcc2-276c-2db2-e300-d0b89eb170ab@oracle.com> Message-ID: <3622ef3b-e3c6-fcc1-d8cf-32885f1016e1@oracle.com> On 8/20/20 5:11 PM, Patricio Chilano wrote: > Hi Coleen, > > Changes look good to me. > I think you could have also just checked for > SafepointSynchronize::is_at_safepoint() in > ClassLoaderDataGraph::purge() instead of having the caller figure it > out. I see that for all the current callers the value is always > constant though so you save that check I guess. Actually it turns out that CLDG::purge() is called concurrently by ZGC in a non-Java thread, so calling SafepointSynchronize::is_at_safepoint() may be true but the safepoint can end while the purging thread is running.? I had the code that way and it failed with ZGC. Thanks for reviewing. Coleen > > Thanks, > Patricio > On 8/19/20 6:47 PM, Coleen Phillimore wrote: >> Summary: Clean up inner metaspaces from ServiceThread if cleanup is >> needed for concurrent GCs. >> >> Moved inner metaspace cleaning back to class unloading, unless using >> concurrent GC.? There we initiate a safepoint if inner metaspaces >> require cleaning. >> >> Tested with tier1-6. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8252043.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8252043 >> >> Thanks, >> Coleen > From calvin.cheung at oracle.com Thu Aug 20 23:13:03 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Thu, 20 Aug 2020 16:13:03 -0700 Subject: RFR(XS) 8252056 Move DumpRegion/ReadClosure/WriteClosure to archiveUtils.hpp In-Reply-To: <6b312a7a-df99-4399-10d2-1122ce689dd3@oracle.com> References: <6b312a7a-df99-4399-10d2-1122ce689dd3@oracle.com> Message-ID: Hi Ioi, The code cleanup looks good to me. thanks, Calvin On 8/19/20 1:47 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8252056 > http://cr.openjdk.java.net/~iklam/jdk16/8252056-move-dump-region-to-archive-utils.v01/ > > > Part of CDS code clean up: > > The classes are currently in metaspaceShared.hpp/cpp, which are too big. > It's better to move these utility classes to archiveUtils.cpp/hpp > > Thanks > - Ioi From yumin.qi at oracle.com Thu Aug 20 23:28:19 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Thu, 20 Aug 2020 16:28:19 -0700 Subject: RFR(XS) 8249096: Clean up code for DumpLoadedClassList In-Reply-To: <9157b9e6-2ba5-e79e-d487-e98e44aad9dd@oracle.com> References: <2343e9d9-ea24-0558-9867-d11418bbba61@oracle.com> <9157b9e6-2ba5-e79e-d487-e98e44aad9dd@oracle.com> Message-ID: Hi, Ioi ? updated at new link: http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-02/ ? Moved the call to 'log_to_classlist' to beginning of 'print_class_load_logging'. Removed the check for if class+load logging is enabled, since it will by pass the output for DumpLoadedClassList if not enabled. ? In 'print_class_load_logging', there exists a check for if the log is enabled. Thanks Yumin On 8/20/20 1:57 PM, Yumin Qi wrote: > Ioi, > > ? Thanks for the review, I will update with new webrev --- yes, print_class_load_logging will work the same way. > > > Thanks > > YUmin > > On 8/20/20 11:37 AM, Ioi Lam wrote: >> Hi Yumin, >> >> This looks like a good clean up. I think it can be further simplified by moving the call to InstanceKlass::log_to_classlist to beginning of InstanceKlass::print_class_load_logging(). >> >> Thanks >> - Ioi >> >> On 8/20/20 10:46 AM, Yumin Qi wrote: >>> Hi, Please review the very small change for cleaning up DumpLoadedClassList code. >>> >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8249096 >>> >>> Webrev: http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-01/ >>> >>> >>> There are two places where when DumpLoadedClassList is set we log loaded class to log file, one is in ClassFileParser::parse_stream, and the other is in SystemDictionaryShared::load_shared_class_misc which is after shared class loaded from CDS. The former is positioned after parsing constants, and before parsing interface, fields and methods etc. There is possibility an exception or error happens before the InstanceKlass is created, so the fix move the logging to after the InstanceKlass is successfully created, also move the logging code to InstanceKlass which seems more reasonable. >>> >>> >>> Test: local jtreg on cds. >>> >>> Mach5 tier1-4 >>> >>> >>> Thanks >>> >>> Yumin >>> >> From mandy.chung at oracle.com Fri Aug 21 00:10:46 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 20 Aug 2020 17:10:46 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> Message-ID: <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> On 8/19/20 10:14 PM, Yumin Qi wrote: > > HI, Mandy > > ? Thanks for the review, I took one day off yesterday so just got a > detail look of your reply. > > On 8/19/20 1:30 PM, Mandy Chung wrote: >> On 8/17/20 12:37 PM, Yumin Qi wrote: >>> Hi, Ioi >>> >>> ? Thanks for review/suggestion. I have updated the webrev at the >>> following link: >>> >>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>> >> >> This patch leverages the TRACE_RESOLVE output and passes the trace >> output to VM.? VM then calls >> GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing >> and generate Holder class per the resolved LFs.?? I think there are >> other cleaner alternatives implementing this. jlink >> --generate-jli-classes plugin depends the trace output whereas >> -Xshare:dump does not.?? It's cleaner to skip generating the trace >> output and parsing for dumping shared archive purpose.? In addition, >> the implementation needs some cleanup (I can send you feedback on the >> next revision) >> > Current patch did not change the existing code for JLinkPlugin part. I > just moved the parsing code from GenerateJLIClassesPlugin.java to > GenerateJLIClassesHelper.java since the former is an internal class to > which we shouldn't call to generate holder classes. >> Instead of relying on a system property >> "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use >> jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. > > I remember we have such API to query if flag -Xshare:dump or > -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This > flag is the one used to log class name to list file. In > GenerateLinkOptData.gmk: > > $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) > $(CLASSLIST_JAR) > ??????? $(call MakeDir, $(LINK_OPT_DIR)) > ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) > ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, > $(JLI_TRACE_FILE))) > ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java > -XX:DumpLoadedClassList=$@.raw \ > ??????????? -Duser.language=en -Duser.country=US \ > ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ > ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) > ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim > ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ > ??????????? -XX:SharedClassListFile=$@.interim > -XX:SharedArchiveFile=$@.jsa \ > ??????????? -Xmx128M -Xms128M $(LOG_INFO) > ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java > -XX:DumpLoadedClassList=$@.raw.2 \ > ??????????? -XX:SharedClassListFile=$@.interim > -XX:SharedArchiveFile=$@.jsa \ > ??????????? -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ > ??????????? -Duser.language=en -Duser.country=US \ > ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ > ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ > ??????????? build.tools.classlist.HelloClasslist \ > ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ > ??????????? || ( \ > ??????????????? exitcode=$$? ; \ > ??????????????? $(ECHO) "ERROR: Failed to generate link optimization > data." \ > ??????????????????? "This is likely a problem with the newly built > JVM/JDK." ; \ > ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ > ??????????????? exit $$exitcode \ > ??????????? ) > ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ > > The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList > and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current > implementation, DumpLoadedClassList will turn on property > java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same > output sent to stdout and log file DumpLoadedClassList specified. > These entries are duplicated in two different files: one for jlink --generate-jli-classes plugin and another for CDS use.? CDS -Xshare:dump attempts to do what jlink plugin does but writes those generated classes in to shared archive. Like the above make logic to build JDK image, the same entries are written in both default-jli-trace.txt via System.out and to classlist via JNI call to the VM.? I guess VM also implements the logic to do some kind of diffing and write to CDS archive. > Now instead of this property, using a vm interface API to query if > this flag is set, I think it is better choice. But here I am NOT sure > I understand your suggestion, I think there are two choices: > > 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via > CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set > > 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting those > name, type and holder name to regenerate holder classes? > I misunderstood that this CDS_TRACE_RESOLVE flag is set during -Xshare:dump time. Ioi has clarified to me offline that this step is actually part of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the given class list file, i.e. you repurpose the class list file to include the log output that was initially designed for jlink plugin. To me, I'd prefer to see this support depending on `jlink --generate-jli-classes` which is an existing functionality and much cleaner.? This way this does not require any VM change.?? It will generate the holder classes in the custom image per the application-specific config file. What it means is that: a customer would need to create a custom image with their application-specific config file.? It might need a new CDS? option to specify a separate TRACE_RESOLVE file.?? It would turn on this feature by default by defining a default path of the log file if it helps. I understand that this is not the existing CDS work flow and CDS archive does not require to run on a custom image.?? I see the value of this approach which can prepare customers to start building and using its own custom image. Of course the implementation would be much simpler (adding a flag to write these traces to a given file path, that is). Mandy From nick.gasson at arm.com Fri Aug 21 02:37:31 2020 From: nick.gasson at arm.com (Nick Gasson) Date: Fri, 21 Aug 2020 10:37:31 +0800 Subject: RFR 8251490: [TESTBUG] The Java thread stack size specified is too small for nsk/stress/stack. Specify at least 448k In-Reply-To: <347361eb-a6cc-5373-cd8b-cce9c2056a7e@oracle.com> References: <9f12fc32-646c-1aff-86fa-003db2641c91@oracle.com> <85imdexbdg.fsf@nicgas01-pc.shanghai.arm.com> <347361eb-a6cc-5373-cd8b-cce9c2056a7e@oracle.com> Message-ID: <85zh6owwpg.fsf@nicgas01-pc.shanghai.arm.com> On 08/20/20 20:43 pm, Harold Seigel wrote: > > I think that's a very good idea! > > I created JBS bug https://bugs.openjdk.java.net/browse/JDK-8252108. > Please assign it to yourself if you want to do the fix. > Will do, thanks! -- Nick From yumin.qi at oracle.com Fri Aug 21 03:05:11 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Thu, 20 Aug 2020 20:05:11 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> Message-ID: <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> Hi, Mandy On 8/20/20 5:10 PM, Mandy Chung wrote: > > > On 8/19/20 10:14 PM, Yumin Qi wrote: >> >> HI, Mandy >> >> ? Thanks for the review, I took one day off yesterday so just got a detail look of your reply. >> >> On 8/19/20 1:30 PM, Mandy Chung wrote: >>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>> Hi, Ioi >>>> >>>> ? Thanks for review/suggestion. I have updated the webrev at the following link: >>>> >>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>> >>> >>> This patch leverages the TRACE_RESOLVE output and passes the trace output to VM.? VM then calls GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing and generate Holder class per the resolved LFs.?? I think there are other cleaner alternatives implementing this.?? jlink --generate-jli-classes plugin depends the trace output whereas -Xshare:dump does not.?? It's cleaner to skip generating the trace output and parsing for dumping shared archive purpose.? In addition, the implementation needs some cleanup (I can send you feedback on the next revision) >>> >> Current patch did not change the existing code for JLinkPlugin part. I just moved the parsing code from GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java since the former is an internal class to which we shouldn't call to generate holder classes. >>> Instead of relying on a system property "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. >> >> I remember we have such API to query if flag -Xshare:dump or -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This flag is the one used to log class name to list file. In GenerateLinkOptData.gmk: >> >> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR) >> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) >> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $(JLI_TRACE_FILE))) >> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw \ >> ??????????? -Duser.language=en -Duser.country=US \ >> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw.2 \ >> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >> ??????????? -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >> ??????????? -Duser.language=en -Duser.country=US \ >> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >> ??????????? build.tools.classlist.HelloClasslist \ >> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >> ??????????? || ( \ >> ??????????????? exitcode=$$? ; \ >> ??????????????? $(ECHO) "ERROR: Failed to generate link optimization data." \ >> ??????????????????? "This is likely a problem with the newly built JVM/JDK." ; \ >> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >> ??????????????? exit $$exitcode \ >> ??????????? ) >> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >> >> The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current implementation, DumpLoadedClassList will turn on property java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same output sent to stdout and log file DumpLoadedClassList specified. >> > > These entries are duplicated in two different files: one for jlink --generate-jli-classes plugin and another for CDS use.? CDS -Xshare:dump attempts to do what jlink plugin does but writes those generated classes in to shared archive. > > Like the above make logic to build JDK image, the same entries are written in both default-jli-trace.txt via System.out and to classlist via JNI call to the VM.? I guess VM also implements the logic to do some kind of diffing and write to CDS archive. > In current implementation, vm side only records the line as from TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at dump time, call back to java for parsing those recordings and generating the holder classes, this uses the existing JLI code. >> >> Now instead of this property, using a vm interface API to query if this flag is set, I think it is better choice. But here I am NOT sure I understand your suggestion, I think there are two choices: >> >> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set >> >> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting those name, type and holder name to regenerate holder classes? >> > > I misunderstood that this CDS_TRACE_RESOLVE flag is set during -Xshare:dump time. > > Ioi has clarified to me offline that this step is actually part of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the given class list file, i.e. you repurpose the class list file to include the log output that was initially designed for jlink plugin. > > To me, I'd prefer to see this support depending on `jlink --generate-jli-classes` which is an existing functionality and much cleaner.? This way this does not require any VM change.?? It will generate the holder classes in the custom image per the application-specific config file. > > What it means is that: a customer would need to create a custom image with their application-specific config file.? It might need a new CDS? option to specify a separate TRACE_RESOLVE file.?? It would turn on this feature by default by defining a default path of the log file if it helps. > So for now, I would implement an API to query if flag DumpLoadedClassList set in cmd line, remove new added property of java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address custom image with CDS in future in a separate issue. Thanks Yumin > I understand that this is not the existing CDS work flow and CDS archive does not require to run on a custom image.?? I see the value of this approach which can prepare customers to start building and using its own custom image. > > Of course the implementation would be much simpler (adding a flag to write these traces to a given file path, that is). > > Mandy From thomas.stuefe at gmail.com Fri Aug 21 04:42:03 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 21 Aug 2020 06:42:03 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: <30cdde4f-3b2e-b8bb-e972-4f8ad131e890@oracle.com> References: <30cdde4f-3b2e-b8bb-e972-4f8ad131e890@oracle.com> Message-ID: Hi Coleen, great, thanks for your three reviews! Nothing much to say really, makes all sense. I'll prepare a new webrev with yours and Leo's feedback incorporated, and whoever else chimes in until then. Cheers, Thomas On Thu, Aug 20, 2020 at 3:48 PM Coleen Phillimore < coleen.phillimore at oracle.com> wrote: > > Hi Thomas, > > Here are my comments from reading the jtreg test files: > > === MetaspaceGTests.java > > /* > * Note: This runs the metaspace-related parts of gtest in > configurations which > * are not tested explicitely in the standard gtests. > * > */ > > explicitly typo. > > This is an interesting change. I don't think there's another way to do > this. Others might have other suggestions. > > === MetaspaceTestContext.java > > Missing copyright. > There are two classes called this (or close to this). I think the gtest > version should be called (gtest) Helper, just to > avoid confusion. > > === TestMetaspaceAllocation* > > * @test id=debug > > Is this something in the version of jtreg we test? > > These tests need @bug 8251158 > even though it's not a bug. > > The tests look really good. They cover and stress metaspace way more > than all of our indirect testing. > > Coleen > > On 8/12/20 1:59 PM, Thomas St?fe wrote: > > Dear all, > > > > I would like to start the review for the implementation of JEP 387 > "Elastic > > Metaspace". > > > > The relevant JBS items are: > > > > JEP: https://openjdk.java.net/jeps/387 > > > > Implementation Issue (pretty much only a placeholder currently): > > https://bugs.openjdk.java.net/browse/JDK-8251158 > > > > -- > > > > Reminder of why we do this: > > > > 1. The new metaspace saves memory. How much depends on context: if it is > > not a problem today it will continue to be none. But in cases where today > > we face large metaspace consumption we may get reductions, sometimes > > drastic ones. These reductions are caused by two facts: > > - after mass class unloading we release memory more promptly to the OS > > - even without class unloading chunk allocation is just plain smarter, > > which reduces the overhead per class loader. This is especially true for > > scenarios involving masses of small loaders which only load very few > > classes. > > > > As an example, see [1] where two VMs - one stock, one patched - run the > > same test program which creates tons of small loaders. The second run > shows > > a much reduced memory footprint and increased elasticity. > > > > 2. The rewritten metaspace code base got cleaner and less complex and > thus > > should be much easier to maintain and to extend. It also would be > possible > > to easily reuse the allocator for different parts of the VM, since it is > > less tightly tied to the notion of just storing class metadata. > > > > -- > > > > In preparation of this review I prepared a guide [2], [3], which gives an > > architectural overview and should be the starting point for an interested > > Reviewer. > > > > The prototype has been tested extensively for quite some time in SAP's CI > > system. We regularly run JCK test, JTReg tests and a whole battery of SAP > > internal tests on 8 platforms. Tests are also currently ongoing at Oracle > > and Red Hat. > > > > -- > > > > The full webrev [4] is somewhat large. In order to make this more > bearable > > I broke the patch up into three parts: > > > > 1) The core parts [5] > > > > This is the heart of the metaspace, mostly rewritten from scratch. I > > propose any Reviewer should not look at the diff but rather just examine > > the new implementation. One possible exception is metaspace.hpp, which is > > the outside interface to metaspace. > > > > There are several ways to get a feeling for this code (after reading at > > least the concept part of the provided guide [2]). The central, most > > "beefy" classes are: > > > > - VirtualSpaceNode - does all the work of reserving, committing, > > uncommitting memory > > - RootChunkArea - does the grunt work of chunk splitting and merging > > - ChunkManager - which takes care of the chunk freelists, of directing > > chunk splits and merges, of enlarging chunks in place > > - MetaspaceArena - which takes care of fine granular allocation on behalf > > of a CLD, and of managing deallocated blocks. > > > > One way to review could be bottom up: starting at > > VirtualSpaceNode/CommitMask/RootChunkArea(LUT), working your way up to > > ChunkManager and Metachunk; finally up to to MetaspaceArena while taking > a > > side stroll to FreeBlocks/BinList/BlockTree. > > > > Another way would be to follow typical paths: > > > > Allocation path: Starting at MetaspaceArena::allocate() -> > > Metachunk::allocate() (note the committing-on-demand code path starting > > here) -> ChunkManager::get_chunk() -> > > VirtualSpaceList/Node->allocate_root_chunk(). > > > > Destruction: ~MetaspaceArena() -> ChunkManager::return_chunk() -> > (merging > > chunks) -> (uncommitting chunks) > > > > Premature deallocation: -> MetaspaceArena::deallocate() -> see freeblock > > list handling > > > > 2) The tests [6] > > > > This part of the patch contains all the new tests. There are a lot; the > > test coverage of the new metaspace is very thorough. > > > > New gtests have been added under 'test/hotspot/gtest/metaspace'. > > New jtreg have been added under > > 'test/hotspot/jtreg/runtime/Metaspace/elastic' and under > > 'test/hotspot/jtreg/gtest/MetaspaceGtests.java'. > > > > 4) Miscellaneous diffs [7] > > > > This is the part of the patch which is neither considered core nor test. > > These changes are small, often uninteresting, and can be reviewed via > diff. > > > > --- > > > > Out of scope for this patch is revamping outside metaspace statistics: > > monitoring of metaspace statistics is mostly left untouched, beyond the > > work needed to keep existing tests green. I wanted to keep those changes > > separate from the core changes for JEP387. They are tracked in > JDK-8251392 > > [8] and JDK-8251342 [9], respectively. > > > > --- > > > > Code complexity: > > > > Numerically, lines of code went ever so slightly down with this patch: > > > > Before: (cloc hotspot/share/memory) > > > ------------------------------------------------------------------------------- > > C++ 36 2533 3097 12735 > > C/C++ Header 54 1728 2867 6278 > > > ------------------------------------------------------------------------------- > > SUM: 90 4261 5964 19013 > > > > After: > > > ------------------------------------------------------------------------------- > > C++ 50 3048 3774 13127 > > C/C++ Header 63 2006 3575 5738 > > > ------------------------------------------------------------------------------- > > SUM: 113 5054 7349 18865 > > > > But since the new implementation is more powerful than the old one - > doing > > things like committing/uncommitting on demand, guarding allocated blocks > > etc - one could argue that the new solution does quite a lot more with > > slightly less code, which I think is a good result. > > > > --- > > > > Idle musing: it would simplify matters quite a bit if we were to unify > > class space and non-class metaspace. If we keep the current narrow Klass > > pointer encoding scheme, we would still need to keep the space they are > > stored in contiguous. But we could use the class space for everything, in > > effect dropping the "class" moniker from the name. It would have to be > > larger than it is currently, of course. > > > > Cons: > > - we would have to abandon the notion of "limitless metaspace" - but if > we > > run with class space this has never been true anyway since the number of > > classes is limited by the size of the compressed class space. > > - virtual process size would go up since it now needs to be as large as > the > > total projected metaspace size. > > - we may need to expand narrow Klass pointer encoding to include > shifting, > > if 4G are not enough to hold all metaspace data. > > > > Pros: > > - this would simplify a lot of code. > > - this would save real (committed) memory, since we save quite a bit of > > overhead. > > - we could narrow-pointer-encode other metadata too, not only Klass > > pointers, should that ever be interesting > > > > ... but that is not part of this JEP. > > > > --- > > > > With this patch, we have a much stronger separation of concerns, and it > > should be easy to reuse the metaspace allocator in other hotspot areas as > > well. For instance, ResourceAreas and friends could be replaced by > > MetaspaceArenas. They do almost the same thing. And as we have seen in > the > > past at SAP, the C-heap backed hotspot Arenas can be a pain since spikes > in > > Arena usage lingers forever as process footprint (we even once rewrote > > parts of the arena code to use mmap, which is just on a more primitive > > level what Metaspace does). > > > > --- > > > > I know this is short notice, but there will be a call for interested > people > > tomorrow at 11AM ET. In case any potential Reviewers are interested just > > drop me a short note. > > > > --- > > > > Thank you, Thomas > > > > > > [1] > > > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/reduction-small-loaders.pdf > > [2] > > > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.pdf > > [3] > > > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.html > > [4] > > > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-all/webrev/ > > [5] > > > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-core/webrev/ > > [6] > > > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-test/webrev/ > > [7] > > > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-misc/webrev/ > > [8] https://bugs.openjdk.java.net/browse/JDK-8251342 > > [9] https://bugs.openjdk.java.net/browse/JDK-8251392 > > From thomas.stuefe at gmail.com Fri Aug 21 05:04:36 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 21 Aug 2020 07:04:36 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: Message-ID: Hi Leo, thanks a lot for your review work! Remarks inline. On Thu, Aug 20, 2020 at 12:24 PM Leo Korinth wrote: > Hi! > > Thanks for doing this, it does indeed look like a great improvement to > me, also many thanks for the /great/ documentation of your change. > > Until I start my review on each part, I have a few general comments. > > 1) I believe structs should be named the same way as classes > (UpperCamelCase) and not end with a "_t" (a naming convention that > at least to me signals a typedef) > > 2) struct fields (and public class fields should to my knowledge be > prefixed with underscore), there should be no naming difference > between public and private fields. > > 3) I can see that you make most struct classes without > constructors. Maybe there is some issue with > trivial/standard-layout of classes, but otherwise I think > constructors are really helpful. They both force you to initialize > data and also gives your IDE help to warn you about uninitialized > fields in the constructor. > (1-3) Yes, I may be a secret C programmer inside. I find simple structs with public members and without ctors to be easier to read and less boilerplate for structured output data. But no problem, I will fix this. When in Rome. > > 4) As our build system (unfortunately) discard the path and thus > requires all source files to be uniquely named, it might be good to > use some kind of prefix for all metaspace files. If you do not > like that idea (we use it in the different GCs for example), I > think at least a few of the files could be given more specific > names (counter.hpp and settings.?pp for example). Maybe prefix all > files with "ms"? > > I thought this myself. Especially since currently it's really inconsistent. I refrained from doing it in order to keep the patches smaller but that ship may have long sailed :) I think something up. > 5) No space between variable and increment operator (i ++ -> i++) > > 6) Enums should be named UpperCamelCase, as should their values > (ALL_UPPER_CASE values are allowed according to the style guide, > but I think most new code uses UpperCamelCase). > > 5, 6: Ok > 7) With c++ 14, I would like all enums to instead be enum classes, > they are safer and better scoped, their underlying type can be > specified and they have (to my knowledge) no downside except for > eventual backports. > > I would love to use them, especially for MetaspaceType/MetadataType, but cannot use them for all cases. Enum classes miss the ability to iterate over values. Which is really a pity, how could they miss that. But I will use them for Metachunk::state, I thought so myself already. I love the fact that one can specify enum size explicitly with enum classes. > 9) metaspace_test.?pp, I have to look more into this, but it does not > feel right to mix in test this way, I guess it is hard to move though? > > Yes, unfortunately. I could hide those two classes away in some other file, but that seems not like a great improvement. They are used by whitebox APIs as well as gtests, and because of the former they are hard to move. I have some smaller improvements in mind after we get this patch out of the door though, among others this: https://bugs.openjdk.java.net/browse/JDK-8252132, which would do away at least with MetaspaceTestArena. > 10) Constructor initialization lists use at least three different > styles (placement of "," and ":") and while I personally like the > Haskell style with comma first on the line, I think we should keep > the style consistent with the rest of HotSpot. > Okay. > > Thanks, > Leo > Thanks, Thomas From thomas.schatzl at oracle.com Fri Aug 21 08:34:13 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Fri, 21 Aug 2020 10:34:13 +0200 Subject: RFR(T): 8252125: add an "inflating" entry to the "table" of bit patterns in share/oops/markWord.hpp In-Reply-To: References: Message-ID: <426e93af-101c-2522-a2f4-4413173ea2f9@oracle.com> Hi, On 20.08.20 22:56, Daniel D. Daugherty wrote: > Greetings, > > I have a trivial fix to add more clarification to markWord bit patterns. > > Here's the bug report: > > ??? JDK-8252125 add an "inflating" entry to the "table" of bit patterns in > ??????????????? share/oops/markWord.hpp > ??? https://bugs.openjdk.java.net/browse/JDK-8252125 > > And here's the context diff for the trivial change itself: > > $ hg diff -r qparent > diff -r 8f73aeccb27c src/hotspot/share/oops/markWord.hpp > --- a/src/hotspot/share/oops/markWord.hpp??? Thu Aug 20 11:12:00 2020 -0700 > +++ b/src/hotspot/share/oops/markWord.hpp??? Thu Aug 20 16:39:11 2020 -0400 > @@ -84,12 +84,13 @@ > ?//??? [header????? | 0 | 01]? unlocked?????????? regular object header > ?//??? [ptr???????????? | 10]? monitor??????????? inflated lock (header > is wapped out) > ?//??? [ptr???????????? | 11]? marked???????????? used to mark an object > +//??? [0 ............ 0| 00]? inflating????????? inflation in progress > ?// > ?//??? We assume that stack/thread pointers have the lowest two bits > cleared. > ?// > -//? - INFLATING() is a distinguished markword value that is used when > -//??? inflating an existing stack-lock into an ObjectMonitor. See below > -//??? for is_being_inflated() and INFLATING(). > +//? - INFLATING() is a distinguished markword value of all zeros that is > +//??? used when inflating an existing stack-lock into an ObjectMonitor. > +//??? See below for is_being_inflated() and INFLATING(). > > ?class BasicLock; > ?class ObjectMonitor; > > Thanks, in advance, for any comments, questions or suggestions. > lgtm and trivial. Thomas From coleen.phillimore at oracle.com Fri Aug 21 11:37:47 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 21 Aug 2020 07:37:47 -0400 Subject: RFR(T): 8252125: add an "inflating" entry to the "table" of bit patterns in share/oops/markWord.hpp In-Reply-To: References: Message-ID: Looks good to me too (and trivial). Coleen On 8/20/20 4:56 PM, Daniel D. Daugherty wrote: > Greetings, > > I have a trivial fix to add more clarification to markWord bit patterns. > > Here's the bug report: > > ??? JDK-8252125 add an "inflating" entry to the "table" of bit > patterns in > ??????????????? share/oops/markWord.hpp > ??? https://bugs.openjdk.java.net/browse/JDK-8252125 > > And here's the context diff for the trivial change itself: > > $ hg diff -r qparent > diff -r 8f73aeccb27c src/hotspot/share/oops/markWord.hpp > --- a/src/hotspot/share/oops/markWord.hpp??? Thu Aug 20 11:12:00 2020 > -0700 > +++ b/src/hotspot/share/oops/markWord.hpp??? Thu Aug 20 16:39:11 2020 > -0400 > @@ -84,12 +84,13 @@ > ?//??? [header????? | 0 | 01]? unlocked?????????? regular object header > ?//??? [ptr???????????? | 10]? monitor??????????? inflated lock > (header is wapped out) > ?//??? [ptr???????????? | 11]? marked???????????? used to mark an object > +//??? [0 ............ 0| 00]? inflating????????? inflation in progress > ?// > ?//??? We assume that stack/thread pointers have the lowest two bits > cleared. > ?// > -//? - INFLATING() is a distinguished markword value that is used when > -//??? inflating an existing stack-lock into an ObjectMonitor. See below > -//??? for is_being_inflated() and INFLATING(). > +//? - INFLATING() is a distinguished markword value of all zeros that is > +//??? used when inflating an existing stack-lock into an ObjectMonitor. > +//??? See below for is_being_inflated() and INFLATING(). > > ?class BasicLock; > ?class ObjectMonitor; > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > > From daniel.daugherty at oracle.com Fri Aug 21 13:13:55 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 21 Aug 2020 09:13:55 -0400 Subject: RFR(T): 8252125: add an "inflating" entry to the "table" of bit patterns in share/oops/markWord.hpp In-Reply-To: <426e93af-101c-2522-a2f4-4413173ea2f9@oracle.com> References: <426e93af-101c-2522-a2f4-4413173ea2f9@oracle.com> Message-ID: Hi Thomas, Thanks for the review. Dan On 8/21/20 4:34 AM, Thomas Schatzl wrote: > Hi, > > On 20.08.20 22:56, Daniel D. Daugherty wrote: >> Greetings, >> >> I have a trivial fix to add more clarification to markWord bit patterns. >> >> Here's the bug report: >> >> ???? JDK-8252125 add an "inflating" entry to the "table" of bit >> patterns in >> ???????????????? share/oops/markWord.hpp >> ???? https://bugs.openjdk.java.net/browse/JDK-8252125 >> >> And here's the context diff for the trivial change itself: >> >> $ hg diff -r qparent >> diff -r 8f73aeccb27c src/hotspot/share/oops/markWord.hpp >> --- a/src/hotspot/share/oops/markWord.hpp??? Thu Aug 20 11:12:00 2020 >> -0700 >> +++ b/src/hotspot/share/oops/markWord.hpp??? Thu Aug 20 16:39:11 2020 >> -0400 >> @@ -84,12 +84,13 @@ >> ??//??? [header????? | 0 | 01]? unlocked?????????? regular object header >> ??//??? [ptr???????????? | 10]? monitor??????????? inflated lock >> (header is wapped out) >> ??//??? [ptr???????????? | 11]? marked???????????? used to mark an >> object >> +//??? [0 ............ 0| 00]? inflating????????? inflation in progress >> ??// >> ??//??? We assume that stack/thread pointers have the lowest two bits >> cleared. >> ??// >> -//? - INFLATING() is a distinguished markword value that is used when >> -//??? inflating an existing stack-lock into an ObjectMonitor. See below >> -//??? for is_being_inflated() and INFLATING(). >> +//? - INFLATING() is a distinguished markword value of all zeros >> that is >> +//??? used when inflating an existing stack-lock into an ObjectMonitor. >> +//??? See below for is_being_inflated() and INFLATING(). >> >> ??class BasicLock; >> ??class ObjectMonitor; >> >> Thanks, in advance, for any comments, questions or suggestions. >> > > ? lgtm and trivial. > > Thomas From daniel.daugherty at oracle.com Fri Aug 21 13:14:13 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 21 Aug 2020 09:14:13 -0400 Subject: RFR(T): 8252125: add an "inflating" entry to the "table" of bit patterns in share/oops/markWord.hpp In-Reply-To: References: Message-ID: <3efc2dee-2a6f-dd65-e96a-e0e9befcee3a@oracle.com> Hi Coleen, Thanks for the review. Dan On 8/21/20 7:37 AM, Coleen Phillimore wrote: > Looks good to me too (and trivial). > Coleen > > On 8/20/20 4:56 PM, Daniel D. Daugherty wrote: >> Greetings, >> >> I have a trivial fix to add more clarification to markWord bit patterns. >> >> Here's the bug report: >> >> ??? JDK-8252125 add an "inflating" entry to the "table" of bit >> patterns in >> ??????????????? share/oops/markWord.hpp >> ??? https://bugs.openjdk.java.net/browse/JDK-8252125 >> >> And here's the context diff for the trivial change itself: >> >> $ hg diff -r qparent >> diff -r 8f73aeccb27c src/hotspot/share/oops/markWord.hpp >> --- a/src/hotspot/share/oops/markWord.hpp??? Thu Aug 20 11:12:00 2020 >> -0700 >> +++ b/src/hotspot/share/oops/markWord.hpp??? Thu Aug 20 16:39:11 2020 >> -0400 >> @@ -84,12 +84,13 @@ >> ?//??? [header????? | 0 | 01]? unlocked?????????? regular object header >> ?//??? [ptr???????????? | 10]? monitor??????????? inflated lock >> (header is wapped out) >> ?//??? [ptr???????????? | 11]? marked???????????? used to mark an object >> +//??? [0 ............ 0| 00]? inflating????????? inflation in progress >> ?// >> ?//??? We assume that stack/thread pointers have the lowest two bits >> cleared. >> ?// >> -//? - INFLATING() is a distinguished markword value that is used when >> -//??? inflating an existing stack-lock into an ObjectMonitor. See below >> -//??? for is_being_inflated() and INFLATING(). >> +//? - INFLATING() is a distinguished markword value of all zeros >> that is >> +//??? used when inflating an existing stack-lock into an ObjectMonitor. >> +//??? See below for is_being_inflated() and INFLATING(). >> >> ?class BasicLock; >> ?class ObjectMonitor; >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> > From coleen.phillimore at oracle.com Fri Aug 21 15:10:24 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 21 Aug 2020 11:10:24 -0400 Subject: RFR (urgent, trivial) 8252149: Compilation error after JDK-8252043 Message-ID: <0a641e2c-5ca5-aac8-3532-b870334b130a@oracle.com> open webrev at http://cr.openjdk.java.net/~coleenp/2020/8252149.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8252149 See bug for details.? This is the version I had tested but I thought I'd change it.? The first version was better and cross compiles on linux-aarch64. should_clean_metaspace_and_reset has the #if INCLUDE_JVMTI code we need for the cross compilation on linux-aarch64 which uses the minimal configuration.? See: bool ClassLoaderDataGraph::should_clean_metaspaces_and_reset() { ? // Only clean metaspaces after full GC. ? bool do_cleaning = _safepoint_cleanup_needed; #if INCLUDE_JVMTI ? do_cleaning = do_cleaning && (_should_clean_deallocate_lists || InstanceKlass::has_previous_versions()); #else ? do_cleaning = do_cleaning && _should_clean_deallocate_lists; #endif ? _safepoint_cleanup_needed = false;? // reset ? return do_cleaning; } thanks, Coleen From harold.seigel at oracle.com Fri Aug 21 15:11:43 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Fri, 21 Aug 2020 11:11:43 -0400 Subject: RFR (urgent, trivial) 8252149: Compilation error after JDK-8252043 In-Reply-To: <0a641e2c-5ca5-aac8-3532-b870334b130a@oracle.com> References: <0a641e2c-5ca5-aac8-3532-b870334b130a@oracle.com> Message-ID: <01ac4da4-dd13-d977-f8d8-6391d7584604@oracle.com> looks good and trivial. Harold On 8/21/2020 11:10 AM, Coleen Phillimore wrote: > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8252149.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8252149 > > See bug for details.? This is the version I had tested but I thought > I'd change it.? The first version was better and cross compiles on > linux-aarch64. > should_clean_metaspace_and_reset has the #if INCLUDE_JVMTI code we > need for the cross compilation on linux-aarch64 which uses the minimal > configuration.? See: > > bool ClassLoaderDataGraph::should_clean_metaspaces_and_reset() { > ? // Only clean metaspaces after full GC. > ? bool do_cleaning = _safepoint_cleanup_needed; > #if INCLUDE_JVMTI > ? do_cleaning = do_cleaning && (_should_clean_deallocate_lists || > InstanceKlass::has_previous_versions()); > #else > ? do_cleaning = do_cleaning && _should_clean_deallocate_lists; > #endif > ? _safepoint_cleanup_needed = false;? // reset > ? return do_cleaning; > } > > > thanks, > Coleen From coleen.phillimore at oracle.com Fri Aug 21 15:23:10 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 21 Aug 2020 11:23:10 -0400 Subject: RFR (urgent, trivial) 8252149: Compilation error after JDK-8252043 In-Reply-To: <01ac4da4-dd13-d977-f8d8-6391d7584604@oracle.com> References: <0a641e2c-5ca5-aac8-3532-b870334b130a@oracle.com> <01ac4da4-dd13-d977-f8d8-6391d7584604@oracle.com> Message-ID: Thanks Harold! Coleen On 8/21/20 11:11 AM, Harold Seigel wrote: > looks good and trivial. > > Harold > > On 8/21/2020 11:10 AM, Coleen Phillimore wrote: >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8252149.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8252149 >> >> See bug for details.? This is the version I had tested but I thought >> I'd change it.? The first version was better and cross compiles on >> linux-aarch64. >> should_clean_metaspace_and_reset has the #if INCLUDE_JVMTI code we >> need for the cross compilation on linux-aarch64 which uses the >> minimal configuration.? See: >> >> bool ClassLoaderDataGraph::should_clean_metaspaces_and_reset() { >> ? // Only clean metaspaces after full GC. >> ? bool do_cleaning = _safepoint_cleanup_needed; >> #if INCLUDE_JVMTI >> ? do_cleaning = do_cleaning && (_should_clean_deallocate_lists || >> InstanceKlass::has_previous_versions()); >> #else >> ? do_cleaning = do_cleaning && _should_clean_deallocate_lists; >> #endif >> ? _safepoint_cleanup_needed = false;? // reset >> ? return do_cleaning; >> } >> >> >> thanks, >> Coleen From kim.barrett at oracle.com Fri Aug 21 16:25:27 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 21 Aug 2020 12:25:27 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <25F61E2F-327C-4924-8CB7-845C517BEBAA@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> <25F61E2F-327C-4924-8CB7-845C517BEBAA@oracle.com> Message-ID: <550245E6-A4B0-4BD9-B717-C3D0B11A0CD9@oracle.com> > On Aug 20, 2020, at 3:44 AM, John Rose wrote: > > On Aug 18, 2020, at 12:49 AM, Kim Barrett wrote: >> >> Poorly named and poorly commented variable from before the beginning of the mercurial age, >> so hard to track down the changeset that introduced it to see if that shed some light. Good >> thing we have some institutional memory :) > > You rang? The SCCS record looks like this: Thanks for tracking this down. That confirms what David suggested. From kim.barrett at oracle.com Fri Aug 21 16:26:12 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 21 Aug 2020 12:26:12 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> Message-ID: <77F6BD2A-DFE1-4676-AD31-55D33C658825@oracle.com> > On Aug 18, 2020, at 5:40 AM, Thomas St?fe wrote: > > > > On Tue, Aug 18, 2020 at 9:52 AM Kim Barrett wrote: > > On Aug 17, 2020, at 10:44 PM, David Holmes wrote: > > > > Hi Kim, > > > > Not a full review - sorry. Have you tested what happens if a resource leak is introduced? The _warned variable was used, IIUC, to avoid hitting recursive errors during error reporting. > > OH! The recursive case hadn?t occurred to me! > > Poorly named and poorly commented variable from before the beginning of the mercurial age, > so hard to track down the changeset that introduced it to see if that shed some light. Good > thing we have some institutional memory :) > > I will do something about that, and then try to figure out how to trigger it. > > You could rig up something using -XX:ErrorHandlerTest (VMError::controlled_crash) and allocate from the same ResourceArea again inside the error handler. There are tests for similar things, e.g. SafeFetch, see hotspot/jtreg/runtime/ErrorReporting. Thanks for the testing hints. I've reinstated the check for resource allocation without a ResourceMark, though with more descriptive naming and comments. Also dealt with the flag being potentially concurrently accessed, because I'm paranoid. I added a test for reporting of allocation without a ResourceMark. I didn't add a test for the recursion case, though I hand-tested it. New webrevs: full: https://cr.openjdk.java.net/~kbarrett/8251850/open.02/ incr: https://cr.openjdk.java.net/~kbarrett/8251850/open.02.inc/ Ran tier1 testing. From ioi.lam at oracle.com Fri Aug 21 16:46:13 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 21 Aug 2020 09:46:13 -0700 Subject: RFR(S) Remove excessive inclusion of arguments.hpp Message-ID: <09c2f053-2c61-5b7d-ea83-18c1db3c2b8e@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8252151 http://cr.openjdk.java.net/~iklam/jdk16/8252151-dont-include-arguments-hpp.v01/ The large header file arguments.hpp is included by popular header files such as constMethod.hpp and methodCounters.hpp (which are included by method.hpp, which is included by almost every cpp file). The dependencies on arguments.hpp by constMethod.hpp are very small, and can be moved to constMethod.cpp. methodCounters.hpp does not depend on arguments.hpp === by removing these dependencies, we can reduce the total number of include files in the hotspot build from 530,921 to 523,010, or a reduction of about 1.5%. From ioi.lam at oracle.com Fri Aug 21 16:48:02 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 21 Aug 2020 09:48:02 -0700 Subject: RFR(XS) 8249096: Clean up code for DumpLoadedClassList In-Reply-To: References: <2343e9d9-ea24-0558-9867-d11418bbba61@oracle.com> <9157b9e6-2ba5-e79e-d487-e98e44aad9dd@oracle.com> Message-ID: <5131e051-7fd8-3bf9-d46e-b304b8859375@oracle.com> Hi Yumin, This new version looks good to me. Thanks! - Ioi On 8/20/20 4:28 PM, Yumin Qi wrote: > Hi, Ioi > > ? updated at new link: > http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-02/ > > ? Moved the call to 'log_to_classlist' to beginning of > 'print_class_load_logging'. Removed the check for if class+load > logging is enabled, since it will by pass the output for > DumpLoadedClassList if not enabled. > > ? In 'print_class_load_logging', there exists a check for if the log > is enabled. > > Thanks > > Yumin > > > On 8/20/20 1:57 PM, Yumin Qi wrote: >> Ioi, >> >> ? Thanks for the review, I will update with new webrev --- yes, >> print_class_load_logging will work the same way. >> >> >> Thanks >> >> YUmin >> >> On 8/20/20 11:37 AM, Ioi Lam wrote: >>> Hi Yumin, >>> >>> This looks like a good clean up. I think it can be further >>> simplified by moving the call to InstanceKlass::log_to_classlist to >>> beginning of InstanceKlass::print_class_load_logging(). >>> >>> Thanks >>> - Ioi >>> >>> On 8/20/20 10:46 AM, Yumin Qi wrote: >>>> Hi, Please review the very small change for cleaning up >>>> DumpLoadedClassList code. >>>> >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8249096 >>>> >>>> Webrev: http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-01/ >>>> >>>> >>>> There are two places where when DumpLoadedClassList is set we log >>>> loaded class to log file, one is in ClassFileParser::parse_stream, >>>> and the other is in SystemDictionaryShared::load_shared_class_misc >>>> which is after shared class loaded from CDS. The former is >>>> positioned after parsing constants, and before parsing interface, >>>> fields and methods etc. There is possibility an exception or error >>>> happens before the InstanceKlass is created, so the fix move the >>>> logging to after the InstanceKlass is successfully created, also >>>> move the logging code to InstanceKlass which seems more reasonable. >>>> >>>> >>>> Test: local jtreg on cds. >>>> >>>> Mach5 tier1-4 >>>> >>>> >>>> Thanks >>>> >>>> Yumin >>>> >>> From igor.ignatyev at oracle.com Fri Aug 21 17:46:41 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 21 Aug 2020 10:46:41 -0700 Subject: RFR(S) : 8252004 : remove usage of PropertyResolvingWrapper in vmTestbase/nsk/sysdict Message-ID: http://cr.openjdk.java.net/~iignatyev/8252004/webrev.00/ > 22 lines changed: 0 ins; 22 del; 0 mod; Hi all, could you please review this small patch which removes usage of PropertyResolvingWrapper from nsk/sysdict tests and reenables allowSmartActionArgs? background from the main bug: > CODETOOLS-7902352 added support of using ${property} in action directive, so PropertyResolvingWrapper isn't needed anymore and can be removed. JBS: https://bugs.openjdk.java.net/browse/JDK-8252004 webrev: http://cr.openjdk.java.net/~iignatyev/8252004/webrev.00 testing: :vmTestbase_nsk_sysdict Thanks, -- Igor From calvin.cheung at oracle.com Fri Aug 21 18:22:55 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Fri, 21 Aug 2020 11:22:55 -0700 Subject: RFR(S): 8251918: [Graal] Crash in DumpTimeSharedClassInfo::add_verification_constraint Message-ID: <69b55ee0-0841-a7b8-23d5-25f644b57646@oracle.com> bug: https://bugs.openjdk.java.net/browse/JDK-8251918 webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8251918/webrev.00/ After the fix for JDK-8247529, the SystemDictionaryShared::find_or_allocate_info_for(InstanceKlass* k) could return NULL if dynamic CDS dump is in progress. The proposed fix is to do a NULL check on the return value of SystemDictionaryShared::find_or_allocate_info_for(). Testing: Tier1 - 4, hs-tier4-graal, hs-tier8-graal thanks, Calvin From yumin.qi at oracle.com Fri Aug 21 18:51:11 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Fri, 21 Aug 2020 11:51:11 -0700 Subject: RFR(S): 8251918: [Graal] Crash in DumpTimeSharedClassInfo::add_verification_constraint In-Reply-To: <69b55ee0-0841-a7b8-23d5-25f644b57646@oracle.com> References: <69b55ee0-0841-a7b8-23d5-25f644b57646@oracle.com> Message-ID: Hi, Calvin ? Looks good. Thanks Yumin On 8/21/20 11:22 AM, calvin.cheung at oracle.com wrote: > bug: https://bugs.openjdk.java.net/browse/JDK-8251918 > > webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8251918/webrev.00/ > > After the fix for JDK-8247529, the SystemDictionaryShared::find_or_allocate_info_for(InstanceKlass* k) could return NULL if dynamic CDS dump is in progress. > > The proposed fix is to do a NULL check on the return value of SystemDictionaryShared::find_or_allocate_info_for(). > > Testing: Tier1 - 4, hs-tier4-graal, hs-tier8-graal > > thanks, > > Calvin > From yumin.qi at oracle.com Fri Aug 21 18:52:21 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Fri, 21 Aug 2020 11:52:21 -0700 Subject: RFR(XS) 8249096: Clean up code for DumpLoadedClassList In-Reply-To: <5131e051-7fd8-3bf9-d46e-b304b8859375@oracle.com> References: <2343e9d9-ea24-0558-9867-d11418bbba61@oracle.com> <9157b9e6-2ba5-e79e-d487-e98e44aad9dd@oracle.com> <5131e051-7fd8-3bf9-d46e-b304b8859375@oracle.com> Message-ID: <083affc4-7df4-fbdf-53a4-2abd1658d6ac@oracle.com> Hi, Ioi ? Thanks! Do I need second review? I think I could push since it is XS. Thanks Yumin On 8/21/20 9:48 AM, Ioi Lam wrote: > Hi Yumin, > > This new version looks good to me. Thanks! > > - Ioi > > On 8/20/20 4:28 PM, Yumin Qi wrote: >> Hi, Ioi >> >> ? updated at new link: http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-02/ >> >> ? Moved the call to 'log_to_classlist' to beginning of 'print_class_load_logging'. Removed the check for if class+load logging is enabled, since it will by pass the output for DumpLoadedClassList if not enabled. >> >> ? In 'print_class_load_logging', there exists a check for if the log is enabled. >> >> Thanks >> >> Yumin >> >> >> On 8/20/20 1:57 PM, Yumin Qi wrote: >>> Ioi, >>> >>> ? Thanks for the review, I will update with new webrev --- yes, print_class_load_logging will work the same way. >>> >>> >>> Thanks >>> >>> YUmin >>> >>> On 8/20/20 11:37 AM, Ioi Lam wrote: >>>> Hi Yumin, >>>> >>>> This looks like a good clean up. I think it can be further simplified by moving the call to InstanceKlass::log_to_classlist to beginning of InstanceKlass::print_class_load_logging(). >>>> >>>> Thanks >>>> - Ioi >>>> >>>> On 8/20/20 10:46 AM, Yumin Qi wrote: >>>>> Hi, Please review the very small change for cleaning up DumpLoadedClassList code. >>>>> >>>>> >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8249096 >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-01/ >>>>> >>>>> >>>>> There are two places where when DumpLoadedClassList is set we log loaded class to log file, one is in ClassFileParser::parse_stream, and the other is in SystemDictionaryShared::load_shared_class_misc which is after shared class loaded from CDS. The former is positioned after parsing constants, and before parsing interface, fields and methods etc. There is possibility an exception or error happens before the InstanceKlass is created, so the fix move the logging to after the InstanceKlass is successfully created, also move the logging code to InstanceKlass which seems more reasonable. >>>>> >>>>> >>>>> Test: local jtreg on cds. >>>>> >>>>> Mach5 tier1-4 >>>>> >>>>> >>>>> Thanks >>>>> >>>>> Yumin >>>>> >>>> > From daniel.daugherty at oracle.com Fri Aug 21 19:02:31 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 21 Aug 2020 15:02:31 -0400 Subject: RFR(XS) 8249096: Clean up code for DumpLoadedClassList In-Reply-To: <083affc4-7df4-fbdf-53a4-2abd1658d6ac@oracle.com> References: <2343e9d9-ea24-0558-9867-d11418bbba61@oracle.com> <9157b9e6-2ba5-e79e-d487-e98e44aad9dd@oracle.com> <5131e051-7fd8-3bf9-d46e-b304b8859375@oracle.com> <083affc4-7df4-fbdf-53a4-2abd1658d6ac@oracle.com> Message-ID: On 8/21/20 2:52 PM, Yumin Qi wrote: > Hi, Ioi > > ? Thanks! Do I need second review? I think I could push since it is XS. It is Trivial reviews that only require a single (R)eviewer and don't have to wait for 24 hours. And your single reviewer has to agree that the review is Trivial. Hold on and I'll be your second reviewer. Dan > > > Thanks > > Yumin > > On 8/21/20 9:48 AM, Ioi Lam wrote: >> Hi Yumin, >> >> This new version looks good to me. Thanks! >> >> - Ioi >> >> On 8/20/20 4:28 PM, Yumin Qi wrote: >>> Hi, Ioi >>> >>> ? updated at new link: >>> http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-02/ >>> >>> ? Moved the call to 'log_to_classlist' to beginning of >>> 'print_class_load_logging'. Removed the check for if class+load >>> logging is enabled, since it will by pass the output for >>> DumpLoadedClassList if not enabled. >>> >>> ? In 'print_class_load_logging', there exists a check for if the log >>> is enabled. >>> >>> Thanks >>> >>> Yumin >>> >>> >>> On 8/20/20 1:57 PM, Yumin Qi wrote: >>>> Ioi, >>>> >>>> ? Thanks for the review, I will update with new webrev --- yes, >>>> print_class_load_logging will work the same way. >>>> >>>> >>>> Thanks >>>> >>>> YUmin >>>> >>>> On 8/20/20 11:37 AM, Ioi Lam wrote: >>>>> Hi Yumin, >>>>> >>>>> This looks like a good clean up. I think it can be further >>>>> simplified by moving the call to InstanceKlass::log_to_classlist >>>>> to beginning of InstanceKlass::print_class_load_logging(). >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> On 8/20/20 10:46 AM, Yumin Qi wrote: >>>>>> Hi, Please review the very small change for cleaning up >>>>>> DumpLoadedClassList code. >>>>>> >>>>>> >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8249096 >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-01/ >>>>>> >>>>>> >>>>>> There are two places where when DumpLoadedClassList is set we log >>>>>> loaded class to log file, one is in >>>>>> ClassFileParser::parse_stream, and the other is in >>>>>> SystemDictionaryShared::load_shared_class_misc which is after >>>>>> shared class loaded from CDS. The former is positioned after >>>>>> parsing constants, and before parsing interface, fields and >>>>>> methods etc. There is possibility an exception or error happens >>>>>> before the InstanceKlass is created, so the fix move the logging >>>>>> to after the InstanceKlass is successfully created, also move the >>>>>> logging code to InstanceKlass which seems more reasonable. >>>>>> >>>>>> >>>>>> Test: local jtreg on cds. >>>>>> >>>>>> Mach5 tier1-4 >>>>>> >>>>>> >>>>>> Thanks >>>>>> >>>>>> Yumin >>>>>> >>>>> >> From yumin.qi at oracle.com Fri Aug 21 19:05:03 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Fri, 21 Aug 2020 12:05:03 -0700 Subject: RFR(XS) 8249096: Clean up code for DumpLoadedClassList In-Reply-To: References: <2343e9d9-ea24-0558-9867-d11418bbba61@oracle.com> <9157b9e6-2ba5-e79e-d487-e98e44aad9dd@oracle.com> <5131e051-7fd8-3bf9-d46e-b304b8859375@oracle.com> <083affc4-7df4-fbdf-53a4-2abd1658d6ac@oracle.com> Message-ID: Hi, Dan ?Thanks Yumin On 8/21/20 12:02 PM, Daniel D. Daugherty wrote: > On 8/21/20 2:52 PM, Yumin Qi wrote: >> Hi, Ioi >> >> ? Thanks! Do I need second review? I think I could push since it is XS. > > It is Trivial reviews that only require a single (R)eviewer and don't > have to wait for 24 hours. And your single reviewer has to agree that > the review is Trivial. > > Hold on and I'll be your second reviewer. > > Dan > >> >> >> Thanks >> >> Yumin >> >> On 8/21/20 9:48 AM, Ioi Lam wrote: >>> Hi Yumin, >>> >>> This new version looks good to me. Thanks! >>> >>> - Ioi >>> >>> On 8/20/20 4:28 PM, Yumin Qi wrote: >>>> Hi, Ioi >>>> >>>> ? updated at new link: http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-02/ >>>> >>>> ? Moved the call to 'log_to_classlist' to beginning of 'print_class_load_logging'. Removed the check for if class+load logging is enabled, since it will by pass the output for DumpLoadedClassList if not enabled. >>>> >>>> ? In 'print_class_load_logging', there exists a check for if the log is enabled. >>>> >>>> Thanks >>>> >>>> Yumin >>>> >>>> >>>> On 8/20/20 1:57 PM, Yumin Qi wrote: >>>>> Ioi, >>>>> >>>>> ? Thanks for the review, I will update with new webrev --- yes, print_class_load_logging will work the same way. >>>>> >>>>> >>>>> Thanks >>>>> >>>>> YUmin >>>>> >>>>> On 8/20/20 11:37 AM, Ioi Lam wrote: >>>>>> Hi Yumin, >>>>>> >>>>>> This looks like a good clean up. I think it can be further simplified by moving the call to InstanceKlass::log_to_classlist to beginning of InstanceKlass::print_class_load_logging(). >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> On 8/20/20 10:46 AM, Yumin Qi wrote: >>>>>>> Hi, Please review the very small change for cleaning up DumpLoadedClassList code. >>>>>>> >>>>>>> >>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8249096 >>>>>>> >>>>>>> Webrev: http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-01/ >>>>>>> >>>>>>> >>>>>>> There are two places where when DumpLoadedClassList is set we log loaded class to log file, one is in ClassFileParser::parse_stream, and the other is in SystemDictionaryShared::load_shared_class_misc which is after shared class loaded from CDS. The former is positioned after parsing constants, and before parsing interface, fields and methods etc. There is possibility an exception or error happens before the InstanceKlass is created, so the fix move the logging to after the InstanceKlass is successfully created, also move the logging code to InstanceKlass which seems more reasonable. >>>>>>> >>>>>>> >>>>>>> Test: local jtreg on cds. >>>>>>> >>>>>>> Mach5 tier1-4 >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Yumin >>>>>>> >>>>>> >>> > From daniel.daugherty at oracle.com Fri Aug 21 19:18:08 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 21 Aug 2020 15:18:08 -0400 Subject: RFR(XS) 8249096: Clean up code for DumpLoadedClassList In-Reply-To: References: <2343e9d9-ea24-0558-9867-d11418bbba61@oracle.com> <9157b9e6-2ba5-e79e-d487-e98e44aad9dd@oracle.com> Message-ID: On 8/20/20 7:28 PM, Yumin Qi wrote: > Hi, Ioi > > ? updated at new link: > http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-02/ > Summary of changes:??? 108 lines changed: 54 ins; 51 del; 3 mod; 14941 unchg At >100 lines changed I wouldn't call this an XS review and it definitely does not qualify as Trivial. src/hotspot/share/oops/instanceKlass.hpp ??? No comments. src/hotspot/share/classfile/classFileParser.cpp ??? No comments. src/hotspot/share/classfile/systemDictionary.cpp ??? No comments. src/hotspot/share/oops/instanceKlass.cpp ??? InstanceKlass::log_to_classlist() appears to be an equivalent ??? rewrite of the big logic block removed from classFileParser.cpp, ??? but that's really the call of someone that knows CDS better than ??? I do. Ioi has already said it's good so... Thumbs up. What testing was done on the webrev-02 version? Dan > > ? Moved the call to 'log_to_classlist' to beginning of > 'print_class_load_logging'. Removed the check for if class+load > logging is enabled, since it will by pass the output for > DumpLoadedClassList if not enabled. > > ? In 'print_class_load_logging', there exists a check for if the log > is enabled. > > Thanks > > Yumin > > > On 8/20/20 1:57 PM, Yumin Qi wrote: >> Ioi, >> >> ? Thanks for the review, I will update with new webrev --- yes, >> print_class_load_logging will work the same way. >> >> >> Thanks >> >> YUmin >> >> On 8/20/20 11:37 AM, Ioi Lam wrote: >>> Hi Yumin, >>> >>> This looks like a good clean up. I think it can be further >>> simplified by moving the call to InstanceKlass::log_to_classlist to >>> beginning of InstanceKlass::print_class_load_logging(). >>> >>> Thanks >>> - Ioi >>> >>> On 8/20/20 10:46 AM, Yumin Qi wrote: >>>> Hi, Please review the very small change for cleaning up >>>> DumpLoadedClassList code. >>>> >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8249096 >>>> >>>> Webrev: http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-01/ >>>> >>>> >>>> There are two places where when DumpLoadedClassList is set we log >>>> loaded class to log file, one is in ClassFileParser::parse_stream, >>>> and the other is in SystemDictionaryShared::load_shared_class_misc >>>> which is after shared class loaded from CDS. The former is >>>> positioned after parsing constants, and before parsing interface, >>>> fields and methods etc. There is possibility an exception or error >>>> happens before the InstanceKlass is created, so the fix move the >>>> logging to after the InstanceKlass is successfully created, also >>>> move the logging code to InstanceKlass which seems more reasonable. >>>> >>>> >>>> Test: local jtreg on cds. >>>> >>>> Mach5 tier1-4 >>>> >>>> >>>> Thanks >>>> >>>> Yumin >>>> >>> From yumin.qi at oracle.com Fri Aug 21 19:25:54 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Fri, 21 Aug 2020 12:25:54 -0700 Subject: RFR(XS) 8249096: Clean up code for DumpLoadedClassList In-Reply-To: References: <2343e9d9-ea24-0558-9867-d11418bbba61@oracle.com> <9157b9e6-2ba5-e79e-d487-e98e44aad9dd@oracle.com> Message-ID: <8746a3be-4d43-a3ab-5b84-09a39eef6acf@oracle.com> Hi, Dan ? Thanks for the review. For local tests on jtreg/runtime/cds. I will submit a mach5 test on tier1-4 to make sure there is no regression. Thanks Yumin On 8/21/20 12:18 PM, Daniel D. Daugherty wrote: > On 8/20/20 7:28 PM, Yumin Qi wrote: >> Hi, Ioi >> >> ? updated at new link: http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-02/ > > > Summary of changes:??? 108 lines changed: 54 ins; 51 del; 3 mod; 14941 unchg > > At >100 lines changed I wouldn't call this an XS review and it definitely > does not qualify as Trivial. > > > src/hotspot/share/oops/instanceKlass.hpp > ??? No comments. > > src/hotspot/share/classfile/classFileParser.cpp > ??? No comments. > > src/hotspot/share/classfile/systemDictionary.cpp > ??? No comments. > > src/hotspot/share/oops/instanceKlass.cpp > ??? InstanceKlass::log_to_classlist() appears to be an equivalent > ??? rewrite of the big logic block removed from classFileParser.cpp, > ??? but that's really the call of someone that knows CDS better than > ??? I do. Ioi has already said it's good so... > > > Thumbs up. > > What testing was done on the webrev-02 version? > > Dan > >> >> ? Moved the call to 'log_to_classlist' to beginning of 'print_class_load_logging'. Removed the check for if class+load logging is enabled, since it will by pass the output for DumpLoadedClassList if not enabled. >> >> ? In 'print_class_load_logging', there exists a check for if the log is enabled. >> >> Thanks >> >> Yumin >> >> >> On 8/20/20 1:57 PM, Yumin Qi wrote: >>> Ioi, >>> >>> ? Thanks for the review, I will update with new webrev --- yes, print_class_load_logging will work the same way. >>> >>> >>> Thanks >>> >>> YUmin >>> >>> On 8/20/20 11:37 AM, Ioi Lam wrote: >>>> Hi Yumin, >>>> >>>> This looks like a good clean up. I think it can be further simplified by moving the call to InstanceKlass::log_to_classlist to beginning of InstanceKlass::print_class_load_logging(). >>>> >>>> Thanks >>>> - Ioi >>>> >>>> On 8/20/20 10:46 AM, Yumin Qi wrote: >>>>> Hi, Please review the very small change for cleaning up DumpLoadedClassList code. >>>>> >>>>> >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8249096 >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~minqi/2020/8249096/webrev-01/ >>>>> >>>>> >>>>> There are two places where when DumpLoadedClassList is set we log loaded class to log file, one is in ClassFileParser::parse_stream, and the other is in SystemDictionaryShared::load_shared_class_misc which is after shared class loaded from CDS. The former is positioned after parsing constants, and before parsing interface, fields and methods etc. There is possibility an exception or error happens before the InstanceKlass is created, so the fix move the logging to after the InstanceKlass is successfully created, also move the logging code to InstanceKlass which seems more reasonable. >>>>> >>>>> >>>>> Test: local jtreg on cds. >>>>> >>>>> Mach5 tier1-4 >>>>> >>>>> >>>>> Thanks >>>>> >>>>> Yumin >>>>> >>>> > From calvin.cheung at oracle.com Fri Aug 21 20:29:40 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Fri, 21 Aug 2020 13:29:40 -0700 Subject: RFR(S): 8251918: [Graal] Crash in DumpTimeSharedClassInfo::add_verification_constraint In-Reply-To: References: <69b55ee0-0841-a7b8-23d5-25f644b57646@oracle.com> Message-ID: <17770c6b-7ab5-cf52-2d15-9707bd1cbb08@oracle.com> Thanks, Yumin! On 8/21/20 11:51 AM, Yumin Qi wrote: > Hi, Calvin > > ? Looks good. > > > Thanks > > Yumin > > On 8/21/20 11:22 AM, calvin.cheung at oracle.com wrote: >> bug: https://bugs.openjdk.java.net/browse/JDK-8251918 >> >> webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8251918/webrev.00/ >> >> After the fix for JDK-8247529, the >> SystemDictionaryShared::find_or_allocate_info_for(InstanceKlass* k) >> could return NULL if dynamic CDS dump is in progress. >> >> The proposed fix is to do a NULL check on the return value of >> SystemDictionaryShared::find_or_allocate_info_for(). >> >> Testing: Tier1 - 4, hs-tier4-graal, hs-tier8-graal >> >> thanks, >> >> Calvin >> From IOI.LAM at ORACLE.COM Fri Aug 21 21:22:16 2020 From: IOI.LAM at ORACLE.COM (Ioi Lam) Date: Fri, 21 Aug 2020 14:22:16 -0700 Subject: RFR(S): 8251918: [Graal] Crash in DumpTimeSharedClassInfo::add_verification_constraint In-Reply-To: <17770c6b-7ab5-cf52-2d15-9707bd1cbb08@oracle.com> References: <17770c6b-7ab5-cf52-2d15-9707bd1cbb08@oracle.com> Message-ID: <5CF01503-3EBD-47BF-931E-54A4ADA5577B@ORACLE.COM> Looks good to me too. Thanks Ioi > On Aug 21, 2020, at 1:30 PM, calvin.cheung at oracle.com wrote: > > ?Thanks, Yumin! > >> On 8/21/20 11:51 AM, Yumin Qi wrote: >> Hi, Calvin >> >> Looks good. >> >> >> Thanks >> >> Yumin >> >>> On 8/21/20 11:22 AM, calvin.cheung at oracle.com wrote: >>> bug: https://bugs.openjdk.java.net/browse/JDK-8251918 >>> >>> webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8251918/webrev.00/ >>> >>> After the fix for JDK-8247529, the SystemDictionaryShared::find_or_allocate_info_for(InstanceKlass* k) could return NULL if dynamic CDS dump is in progress. >>> >>> The proposed fix is to do a NULL check on the return value of SystemDictionaryShared::find_or_allocate_info_for(). >>> >>> Testing: Tier1 - 4, hs-tier4-graal, hs-tier8-graal >>> >>> thanks, >>> >>> Calvin >>> From calvin.cheung at oracle.com Fri Aug 21 21:57:30 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Fri, 21 Aug 2020 14:57:30 -0700 Subject: RFR(S): 8251918: [Graal] Crash in DumpTimeSharedClassInfo::add_verification_constraint In-Reply-To: <5CF01503-3EBD-47BF-931E-54A4ADA5577B@ORACLE.COM> References: <17770c6b-7ab5-cf52-2d15-9707bd1cbb08@oracle.com> <5CF01503-3EBD-47BF-931E-54A4ADA5577B@ORACLE.COM> Message-ID: <6667a29f-409f-8e39-279a-b363c08ad374@oracle.com> Thanks, Ioi! On 8/21/20 2:22 PM, Ioi Lam wrote: > Looks good to me too. > > Thanks > Ioi > >> On Aug 21, 2020, at 1:30 PM, calvin.cheung at oracle.com wrote: >> >> ?Thanks, Yumin! >> >>> On 8/21/20 11:51 AM, Yumin Qi wrote: >>> Hi, Calvin >>> >>> Looks good. >>> >>> >>> Thanks >>> >>> Yumin >>> >>>> On 8/21/20 11:22 AM, calvin.cheung at oracle.com wrote: >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8251918 >>>> >>>> webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8251918/webrev.00/ >>>> >>>> After the fix for JDK-8247529, the SystemDictionaryShared::find_or_allocate_info_for(InstanceKlass* k) could return NULL if dynamic CDS dump is in progress. >>>> >>>> The proposed fix is to do a NULL check on the return value of SystemDictionaryShared::find_or_allocate_info_for(). >>>> >>>> Testing: Tier1 - 4, hs-tier4-graal, hs-tier8-graal >>>> >>>> thanks, >>>> >>>> Calvin >>>> From goetz.lindenmaier at sap.com Sat Aug 22 05:45:40 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Sat, 22 Aug 2020 05:45:40 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Richard, I read through your change again. It looks good to me now. The new naming and additional comments make it easier to read I think, thank you. One small thing: deoptimization.cpp, l. 1503 You don't really need the brackets. Two lines below you don't use them either. (No webrev needed) Best regards, Goetz. -----Original Message----- From: Reingruber, Richard Sent: Dienstag, 18. August 2020 10:44 To: Lindenmaier, Goetz ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi Goetz, I have collected the changes based on your feedback in a new webrev: Webrev.7: http://cr.openjdk.java.net/~rrich/webrevs/8227745/webrev.7/ Delta: http://cr.openjdk.java.net/~rrich/webrevs/8227745/webrev.7.inc/ Most of the changes are renamings, commenting, and reformatting. Besides that ... - I converted the native agent of the test IterateHeapWithEscapeAnalysisEnabled from C to C++, because this seems to be preferred by serviceability developers. I also re-indented the file, but excluded this from the delta webrev. - I had to adapt test/jdk/com/sun/jdi/EATests.java to the fact that background compilation (-Xbatch) cannot be reliably disabled for JVMCI compilers. E.g. the compile broker will compile in the background if JVMCI is not yet fully initialized. Therefore it is possible that test cases are executed before the main test method is compiled on the highest level and then the test case fails. The higher the system load the higher the probability for this to happen. In webrev.7 I skip the compilation level check if the vm is configured to use the JVMCI compiler. I also answered you inline below. Thanks, Richard. -----Original Message----- From: Lindenmaier, Goetz Sent: Donnerstag, 23. Juli 2020 16:20 To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi Richard, Thanks for your two further explanations in the other thread. That made the points clear to me. > > I was not that happy with the names saying not_global_escape > > and similar. I now agreed you have to use the terms of the escape > > analysis (NoEscape ArgEscape= throughout the runtime code. I'm still not happy with > > the 'not' in the term, I always try to expand the name to some > > sentence with a negated verb, but it makes no sense. > > For example, "has_not_global_escape_in_scope" expands to > > "Hasn't a global escape in its scope." in my thinking, which makes > > no sense. You probably mean > > "Has not-global escape in its scope." or "Has {ArgEscape|NoEscape} > > in its scope." > > > C2 is using the word "non" in this context, e.g., here > > alloc->is_non_escaping. > > There is also ConnectionGraph::not_global_escape() That talks about a single node that represents a single Object. An object has a single state wrt. ea. You use the term for safepoint which tracks a set of objects. Here, has_not_global_excape can mean 1. None of the several objects does escape globaly. 2. There is at least one object that escapes globaly. > > non obviously negates the adjective 'global', > > non-global or nonglobal even is a English term I find in the > > net. > > So what about "has_non_global_escape_in_scope?" > > And what about has_ea_local_in_scope? That's good. Please document somewhere that Ea_local == ArgEscape | NoEscape. That's what it is, right? > > Does jvmti specify that the same limits are used ...? > > ok on your side. > > I don't know and didn't find anything in a quick search. Ok, not your business. > > > jvmtiEnvBase.cpp ok > > jvmtiImpl.h|cpp ok > > jvmtiTagMap.cpp ok > > whitebox.cpp ok > > > deoptimization.cpp > > > line 177: Please break line > > line 246, 281: Please break line > > 1578, 1583, 1589, 1632, 1649, 1651 Break line > > > 1651: You use 'non'-terms, too: non-escaping :) > > I know :) At least here it is wrong I'd say. "...has to be a not escaping obj..." > sounds better > (hopefully not only to my german ears). I thought the term non-escpaing makes it quite clear. I just wanted to point out that using non above would be similar to the wording here. > > IterateHeapWithEscapeAnalysisEnabled.java > > > line 415: > > msg("wait until target thread has set testMethod_result"); > > while (testMethod_result == 0) { > > Thread.sleep(50); > > } > > Might the test run into timeouts at this place? > > The field is volatile, i.e. it will be reloaded > > in each iteration. But will dontinline_testMethod > > write it back to main memory in time? > > You mean, the test could hang in that loop for a couple of minutes? I don't > think so. There are cache coherence protocols in place which will invalidate > stale data very timely. Ok, anyways, it would only be a hanging test. > > Ok. I've removed quite a lot of the occurrances. > > > Also, I like full sentences in comments. > > Especially for me as foreign speaker, this makes > > things much more clear. I.e., I try to make it > > a real sentence with articles, capitalized and a > > dot at the end if there is a subject and a verb > > in first place. > > E.g., jvmtiEnvBase.cpp:1327 > > Are you referring to the following? > (from > http://cr.openjdk.java.net/~rrich/webrevs/2019/8227745/webrev.6/src/hots > pot/share/prims/jvmtiEnvBase.cpp.frames.html) > > 1326 > 1327 // If the frame is a compiled one, need to deoptimize it. > 1328 if (vf->is_compiled_frame()) { > > This line 1327 is preexisting. Sorry, wrong line number again. I think I meant 1333 // eagerly reallocate scalar replaced objects. But I must admit, the subject is missing. It's one of these imperative sentences where the subject is left out, which are used throughout documentation. Bad example, but still a correct sentence, so qualifies for punctuation? Best regards, Goetz. From thomas.stuefe at gmail.com Sat Aug 22 06:13:00 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 22 Aug 2020 08:13:00 +0200 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <77F6BD2A-DFE1-4676-AD31-55D33C658825@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> <77F6BD2A-DFE1-4676-AD31-55D33C658825@oracle.com> Message-ID: On Fri, Aug 21, 2020 at 6:26 PM Kim Barrett wrote: > > On Aug 18, 2020, at 5:40 AM, Thomas St?fe > wrote: > > > > > > > > On Tue, Aug 18, 2020 at 9:52 AM Kim Barrett > wrote: > > > On Aug 17, 2020, at 10:44 PM, David Holmes > wrote: > > > > > > Hi Kim, > > > > > > Not a full review - sorry. Have you tested what happens if a resource > leak is introduced? The _warned variable was used, IIUC, to avoid hitting > recursive errors during error reporting. > > > > OH! The recursive case hadn?t occurred to me! > > > > Poorly named and poorly commented variable from before the beginning of > the mercurial age, > > so hard to track down the changeset that introduced it to see if that > shed some light. Good > > thing we have some institutional memory :) > > > > I will do something about that, and then try to figure out how to > trigger it. > > > > You could rig up something using -XX:ErrorHandlerTest > (VMError::controlled_crash) and allocate from the same ResourceArea again > inside the error handler. There are tests for similar things, e.g. > SafeFetch, see hotspot/jtreg/runtime/ErrorReporting. > > Thanks for the testing hints. > > I've reinstated the check for resource allocation without a ResourceMark, > though with more descriptive naming and comments. Also dealt with the flag > being potentially concurrently accessed, because I'm paranoid. > > I added a test for reporting of allocation without a ResourceMark. I > didn't > add a test for the recursion case, though I hand-tested it. > > New webrevs: > full: https://cr.openjdk.java.net/~kbarrett/8251850/open.02/ > incr: https://cr.openjdk.java.net/~kbarrett/8251850/open.02.inc/ > > Ran tier1 testing. > > Good. + // Only report the first occurrence of an allocating thread that + // is missing a ResourceMark, to avoid possible recursive errors + // in error handling. The comment is clear. Should be still clear 40 years from now when we all moved on from git and there is only one guy left who can read git history :) Very minor nit: comment is somewhat imprecise here: + case 18: { + // Allocation from resource area without a ResourceMark. + Thread::current()->resource_area()->allocate_bytes(100); + break; + } because this is only true if this function is called from argument handling where no RM has been established yet. I am fine with this change. I leave if up to you if you change the comment or not. Cheers, Thomas From thomas.stuefe at gmail.com Sat Aug 22 07:03:50 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 22 Aug 2020 09:03:50 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: <8bd366a3-a40b-c2d6-8e19-1abf7a4c7e04@oracle.com> Message-ID: > > >> blocktree.hpp/cpp should be blockTree.hpp/cpp >> >> Since for class metaspace, the reclaimed InstanceKlass are generally >> bigger than the largest BinList size, maybe class type MetaspaceArenas >> should only have a blockTree and not a SmallBlocks, so not the whole >> freeList.hpp structure. I think we could do some footprint analysis >> with NMT and see if this might help. It's fairly rare unless you have a >> lot of redefinition. I don't think this should be changed in this >> change though. >> >> > Oh good point. > > I would not like to hard-code the assumption that class space only > contains Klass though; I dimly remember an RFR with Ioi where he tried > storing other things there as well. IIRC he found some other solution. But > my point is that this assumption is fragile. > > But a "soft assumption" would be fine. E.g. creating the BinList on the > fly, only when needed, and separately from the blocktree. Easy to implement > too. > > We don't need NMT to analyze. BinList currently has 32 slots, + counter + > mask. We pay 272 bytes per structure, or 34 words. Which we would save per > CLD. > > > This seems like something useful to explore later. > > I created follow up https://bugs.openjdk.java.net/browse/JDK-8252187 to track this. From rkennke at redhat.com Sat Aug 22 12:16:52 2020 From: rkennke at redhat.com (Roman Kennke) Date: Sat, 22 Aug 2020 14:16:52 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: Message-ID: Hi Thomas, this looks like very useful work! Thank you! Not a review, just a note: I put the original patch through a bunch of my regular (Shenandoah) testing on x86 and aarch64, and haven't found any issues. I will try to give it a deeper look next week (no promises). Cheers, Roman > (Am Mittwoch, den 12.08.2020, 19:59 +0200 schrieb Thomas St?fe: > Dear all, > > I would like to start the review for the implementation of JEP 387 > "Elastic > Metaspace". > > The relevant JBS items are: > > JEP: https://openjdk.java.net/jeps/387 > > Implementation Issue (pretty much only a placeholder currently): > https://bugs.openjdk.java.net/browse/JDK-8251158 > > -- > > Reminder of why we do this: > > 1. The new metaspace saves memory. How much depends on context: if it > is > not a problem today it will continue to be none. But in cases where > today > we face large metaspace consumption we may get reductions, sometimes > drastic ones. These reductions are caused by two facts: > - after mass class unloading we release memory more promptly to the > OS > - even without class unloading chunk allocation is just plain > smarter, > which reduces the overhead per class loader. This is especially true > for > scenarios involving masses of small loaders which only load very few > classes. > > As an example, see [1] where two VMs - one stock, one patched - run > the > same test program which creates tons of small loaders. The second run > shows > a much reduced memory footprint and increased elasticity. > > 2. The rewritten metaspace code base got cleaner and less complex and > thus > should be much easier to maintain and to extend. It also would be > possible > to easily reuse the allocator for different parts of the VM, since it > is > less tightly tied to the notion of just storing class metadata. > > -- > > In preparation of this review I prepared a guide [2], [3], which > gives an > architectural overview and should be the starting point for an > interested > Reviewer. > > The prototype has been tested extensively for quite some time in > SAP's CI > system. We regularly run JCK test, JTReg tests and a whole battery of > SAP > internal tests on 8 platforms. Tests are also currently ongoing at > Oracle > and Red Hat. > > -- > > The full webrev [4] is somewhat large. In order to make this more > bearable > I broke the patch up into three parts: > > 1) The core parts [5] > > This is the heart of the metaspace, mostly rewritten from scratch. I > propose any Reviewer should not look at the diff but rather just > examine > the new implementation. One possible exception is metaspace.hpp, > which is > the outside interface to metaspace. > > There are several ways to get a feeling for this code (after reading > at > least the concept part of the provided guide [2]). The central, most > "beefy" classes are: > > - VirtualSpaceNode - does all the work of reserving, committing, > uncommitting memory > - RootChunkArea - does the grunt work of chunk splitting and merging > - ChunkManager - which takes care of the chunk freelists, of > directing > chunk splits and merges, of enlarging chunks in place > - MetaspaceArena - which takes care of fine granular allocation on > behalf > of a CLD, and of managing deallocated blocks. > > One way to review could be bottom up: starting at > VirtualSpaceNode/CommitMask/RootChunkArea(LUT), working your way up > to > ChunkManager and Metachunk; finally up to to MetaspaceArena while > taking a > side stroll to FreeBlocks/BinList/BlockTree. > > Another way would be to follow typical paths: > > Allocation path: Starting at MetaspaceArena::allocate() -> > Metachunk::allocate() (note the committing-on-demand code path > starting > here) -> ChunkManager::get_chunk() -> > VirtualSpaceList/Node->allocate_root_chunk(). > > Destruction: ~MetaspaceArena() -> ChunkManager::return_chunk() -> > (merging > chunks) -> (uncommitting chunks) > > Premature deallocation: -> MetaspaceArena::deallocate() -> see > freeblock > list handling > > 2) The tests [6] > > This part of the patch contains all the new tests. There are a lot; > the > test coverage of the new metaspace is very thorough. > > New gtests have been added under 'test/hotspot/gtest/metaspace'. > New jtreg have been added under > 'test/hotspot/jtreg/runtime/Metaspace/elastic' and under > 'test/hotspot/jtreg/gtest/MetaspaceGtests.java'. > > 4) Miscellaneous diffs [7] > > This is the part of the patch which is neither considered core nor > test. > These changes are small, often uninteresting, and can be reviewed via > diff. > > --- > > Out of scope for this patch is revamping outside metaspace > statistics: > monitoring of metaspace statistics is mostly left untouched, beyond > the > work needed to keep existing tests green. I wanted to keep those > changes > separate from the core changes for JEP387. They are tracked in JDK- > 8251392 > [8] and JDK-8251342 [9], respectively. > > --- > > Code complexity: > > Numerically, lines of code went ever so slightly down with this > patch: > > Before: (cloc hotspot/share/memory) > ------------------------------------------------------------------- > ------------ > C++ 36 2533 3097 12735 > C/C++ Header 54 1728 2867 6278 > ------------------------------------------------------------------- > ------------ > SUM: 90 4261 5964 19013 > > After: > ------------------------------------------------------------------- > ------------ > C++ 50 3048 3774 13127 > C/C++ Header 63 2006 3575 5738 > ------------------------------------------------------------------- > ------------ > SUM: 113 5054 7349 18865 > > But since the new implementation is more powerful than the old one - > doing > things like committing/uncommitting on demand, guarding allocated > blocks > etc - one could argue that the new solution does quite a lot more > with > slightly less code, which I think is a good result. > > --- > > Idle musing: it would simplify matters quite a bit if we were to > unify > class space and non-class metaspace. If we keep the current narrow > Klass > pointer encoding scheme, we would still need to keep the space they > are > stored in contiguous. But we could use the class space for > everything, in > effect dropping the "class" moniker from the name. It would have to > be > larger than it is currently, of course. > > Cons: > - we would have to abandon the notion of "limitless metaspace" - but > if we > run with class space this has never been true anyway since the number > of > classes is limited by the size of the compressed class space. > - virtual process size would go up since it now needs to be as large > as the > total projected metaspace size. > - we may need to expand narrow Klass pointer encoding to include > shifting, > if 4G are not enough to hold all metaspace data. > > Pros: > - this would simplify a lot of code. > - this would save real (committed) memory, since we save quite a bit > of > overhead. > - we could narrow-pointer-encode other metadata too, not only Klass > pointers, should that ever be interesting > > ... but that is not part of this JEP. > > --- > > With this patch, we have a much stronger separation of concerns, and > it > should be easy to reuse the metaspace allocator in other hotspot > areas as > well. For instance, ResourceAreas and friends could be replaced by > MetaspaceArenas. They do almost the same thing. And as we have seen > in the > past at SAP, the C-heap backed hotspot Arenas can be a pain since > spikes in > Arena usage lingers forever as process footprint (we even once > rewrote > parts of the arena code to use mmap, which is just on a more > primitive > level what Metaspace does). > > --- > > I know this is short notice, but there will be a call for interested > people > tomorrow at 11AM ET. In case any potential Reviewers are interested > just > drop me a short note. > > --- > > Thank you, Thomas > > > [1] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/reduction-small-loaders.pdf > [2] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.pdf > [3] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.html > [4] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-all/webrev/ > [5] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-core/webrev/ > [6] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-test/webrev/ > [7] > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-misc/webrev/ > [8] https://bugs.openjdk.java.net/browse/JDK-8251342 > [9] https://bugs.openjdk.java.net/browse/JDK-8251392 > From thomas.stuefe at gmail.com Sat Aug 22 12:17:34 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 22 Aug 2020 14:17:34 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: Message-ID: Excellent, Roman, thanks a bunch! ..Thomas On Sat, Aug 22, 2020 at 2:17 PM Roman Kennke wrote: > Hi Thomas, > > this looks like very useful work! Thank you! > > Not a review, just a note: I put the original patch through a bunch of > my regular (Shenandoah) testing on x86 and aarch64, and haven't found > any issues. > > I will try to give it a deeper look next week (no promises). > > > Cheers, > Roman > > > (Am Mittwoch, den 12.08.2020, 19:59 +0200 schrieb Thomas St?fe: > > Dear all, > > > > I would like to start the review for the implementation of JEP 387 > > "Elastic > > Metaspace". > > > > The relevant JBS items are: > > > > JEP: https://openjdk.java.net/jeps/387 > > > > Implementation Issue (pretty much only a placeholder currently): > > https://bugs.openjdk.java.net/browse/JDK-8251158 > > > > -- > > > > Reminder of why we do this: > > > > 1. The new metaspace saves memory. How much depends on context: if it > > is > > not a problem today it will continue to be none. But in cases where > > today > > we face large metaspace consumption we may get reductions, sometimes > > drastic ones. These reductions are caused by two facts: > > - after mass class unloading we release memory more promptly to the > > OS > > - even without class unloading chunk allocation is just plain > > smarter, > > which reduces the overhead per class loader. This is especially true > > for > > scenarios involving masses of small loaders which only load very few > > classes. > > > > As an example, see [1] where two VMs - one stock, one patched - run > > the > > same test program which creates tons of small loaders. The second run > > shows > > a much reduced memory footprint and increased elasticity. > > > > 2. The rewritten metaspace code base got cleaner and less complex and > > thus > > should be much easier to maintain and to extend. It also would be > > possible > > to easily reuse the allocator for different parts of the VM, since it > > is > > less tightly tied to the notion of just storing class metadata. > > > > -- > > > > In preparation of this review I prepared a guide [2], [3], which > > gives an > > architectural overview and should be the starting point for an > > interested > > Reviewer. > > > > The prototype has been tested extensively for quite some time in > > SAP's CI > > system. We regularly run JCK test, JTReg tests and a whole battery of > > SAP > > internal tests on 8 platforms. Tests are also currently ongoing at > > Oracle > > and Red Hat. > > > > -- > > > > The full webrev [4] is somewhat large. In order to make this more > > bearable > > I broke the patch up into three parts: > > > > 1) The core parts [5] > > > > This is the heart of the metaspace, mostly rewritten from scratch. I > > propose any Reviewer should not look at the diff but rather just > > examine > > the new implementation. One possible exception is metaspace.hpp, > > which is > > the outside interface to metaspace. > > > > There are several ways to get a feeling for this code (after reading > > at > > least the concept part of the provided guide [2]). The central, most > > "beefy" classes are: > > > > - VirtualSpaceNode - does all the work of reserving, committing, > > uncommitting memory > > - RootChunkArea - does the grunt work of chunk splitting and merging > > - ChunkManager - which takes care of the chunk freelists, of > > directing > > chunk splits and merges, of enlarging chunks in place > > - MetaspaceArena - which takes care of fine granular allocation on > > behalf > > of a CLD, and of managing deallocated blocks. > > > > One way to review could be bottom up: starting at > > VirtualSpaceNode/CommitMask/RootChunkArea(LUT), working your way up > > to > > ChunkManager and Metachunk; finally up to to MetaspaceArena while > > taking a > > side stroll to FreeBlocks/BinList/BlockTree. > > > > Another way would be to follow typical paths: > > > > Allocation path: Starting at MetaspaceArena::allocate() -> > > Metachunk::allocate() (note the committing-on-demand code path > > starting > > here) -> ChunkManager::get_chunk() -> > > VirtualSpaceList/Node->allocate_root_chunk(). > > > > Destruction: ~MetaspaceArena() -> ChunkManager::return_chunk() -> > > (merging > > chunks) -> (uncommitting chunks) > > > > Premature deallocation: -> MetaspaceArena::deallocate() -> see > > freeblock > > list handling > > > > 2) The tests [6] > > > > This part of the patch contains all the new tests. There are a lot; > > the > > test coverage of the new metaspace is very thorough. > > > > New gtests have been added under 'test/hotspot/gtest/metaspace'. > > New jtreg have been added under > > 'test/hotspot/jtreg/runtime/Metaspace/elastic' and under > > 'test/hotspot/jtreg/gtest/MetaspaceGtests.java'. > > > > 4) Miscellaneous diffs [7] > > > > This is the part of the patch which is neither considered core nor > > test. > > These changes are small, often uninteresting, and can be reviewed via > > diff. > > > > --- > > > > Out of scope for this patch is revamping outside metaspace > > statistics: > > monitoring of metaspace statistics is mostly left untouched, beyond > > the > > work needed to keep existing tests green. I wanted to keep those > > changes > > separate from the core changes for JEP387. They are tracked in JDK- > > 8251392 > > [8] and JDK-8251342 [9], respectively. > > > > --- > > > > Code complexity: > > > > Numerically, lines of code went ever so slightly down with this > > patch: > > > > Before: (cloc hotspot/share/memory) > > ------------------------------------------------------------------- > > ------------ > > C++ 36 2533 3097 12735 > > C/C++ Header 54 1728 2867 6278 > > ------------------------------------------------------------------- > > ------------ > > SUM: 90 4261 5964 19013 > > > > After: > > ------------------------------------------------------------------- > > ------------ > > C++ 50 3048 3774 13127 > > C/C++ Header 63 2006 3575 5738 > > ------------------------------------------------------------------- > > ------------ > > SUM: 113 5054 7349 18865 > > > > But since the new implementation is more powerful than the old one - > > doing > > things like committing/uncommitting on demand, guarding allocated > > blocks > > etc - one could argue that the new solution does quite a lot more > > with > > slightly less code, which I think is a good result. > > > > --- > > > > Idle musing: it would simplify matters quite a bit if we were to > > unify > > class space and non-class metaspace. If we keep the current narrow > > Klass > > pointer encoding scheme, we would still need to keep the space they > > are > > stored in contiguous. But we could use the class space for > > everything, in > > effect dropping the "class" moniker from the name. It would have to > > be > > larger than it is currently, of course. > > > > Cons: > > - we would have to abandon the notion of "limitless metaspace" - but > > if we > > run with class space this has never been true anyway since the number > > of > > classes is limited by the size of the compressed class space. > > - virtual process size would go up since it now needs to be as large > > as the > > total projected metaspace size. > > - we may need to expand narrow Klass pointer encoding to include > > shifting, > > if 4G are not enough to hold all metaspace data. > > > > Pros: > > - this would simplify a lot of code. > > - this would save real (committed) memory, since we save quite a bit > > of > > overhead. > > - we could narrow-pointer-encode other metadata too, not only Klass > > pointers, should that ever be interesting > > > > ... but that is not part of this JEP. > > > > --- > > > > With this patch, we have a much stronger separation of concerns, and > > it > > should be easy to reuse the metaspace allocator in other hotspot > > areas as > > well. For instance, ResourceAreas and friends could be replaced by > > MetaspaceArenas. They do almost the same thing. And as we have seen > > in the > > past at SAP, the C-heap backed hotspot Arenas can be a pain since > > spikes in > > Arena usage lingers forever as process footprint (we even once > > rewrote > > parts of the arena code to use mmap, which is just on a more > > primitive > > level what Metaspace does). > > > > --- > > > > I know this is short notice, but there will be a call for interested > > people > > tomorrow at 11AM ET. In case any potential Reviewers are interested > > just > > drop me a short note. > > > > --- > > > > Thank you, Thomas > > > > > > [1] > > > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/reduction-small-loaders.pdf > > [2] > > > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.pdf > > [3] > > > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.html > > [4] > > > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-all/webrev/ > > [5] > > > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-core/webrev/ > > [6] > > > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-test/webrev/ > > [7] > > > http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-misc/webrev/ > > [8] https://bugs.openjdk.java.net/browse/JDK-8251342 > > [9] https://bugs.openjdk.java.net/browse/JDK-8251392 > > > > From thomas.stuefe at gmail.com Sat Aug 22 13:40:19 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 22 Aug 2020 15:40:19 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: <858c2f0b-c6a8-0726-cd97-9641396d8d14@oracle.com> References: <858c2f0b-c6a8-0726-cd97-9641396d8d14@oracle.com> Message-ID: Hi Albert, On Sat, Aug 22, 2020 at 10:47 AM Albert Yang wrote: > Hi Thomas, > > Thank you for preparing the review guide, which has been quite helpful > to see the big picture, and in providing a starting point to embrace > this large patch/code base. > > ## Metachunk exposing committed memory status > > One thing I find odd is that chunk doesn't encapsulate committed memory > status. In other words, a component interacting with a chuck needs to be aware of how much memory is committed. I will provide two examples in > code, one is the "consumer" of `Metachunk`, the other the "producer" of > `Metachunk`. > > For the "consumer" case, `MetaspaceArena::allocate`, allocation is > broken into two steps, 1. for committed memory, 2. for virtual memory > (+committed memory from step 1). > > ``` > if (!current_chunk_too_small) { > if (!current_chunk()->ensure_committed_additional(raw_word_size)) { > UL2(info, "commit failure (requested size: " SIZE_FORMAT ")", > raw_word_size); > commit_failure = true; > } > } > > // Allocate from the current chunk. This should work now. > if (!current_chunk_too_small && !commit_failure) { > p = current_chunk()->allocate(raw_word_size); > assert(p != NULL, "Allocation from chunk failed."); > } > ``` > > If the "committed memory status" is properly encapsulated in > `Metachunk::allocate`, the above code could be like: > > ``` > if (!current_chunk_too_small) { > p = current_chunk()->allocate(raw_word_size); > } > ``` > > For the "producer" case, `ChunkManager::get_chunk` mostly deals with > identifying a chunk which satisfied some constraints, but the following > code focuses on committed bytes, which, in my opinion, is on another > abstraction level. I believe that moving such logic to > `Metachunk::allocate` could make code more readable. > > ``` > // Attempt to commit the chunk (depending on settings, we either fully > commit it or just > // commit enough to get the caller going). That may fail if we hit a > commit limit. In > // that case put the chunk back to the freelist (re-merging it with its > neighbors if we > // did split it) and return NULL. > const size_t to_commit = Settings::new_chunks_are_fully_committed() ? > c->word_size() : min_committed_words; > if (c->committed_words() < to_commit) { > if (c->ensure_committed_locked(to_commit) == false) { > UL2(info, "failed to commit " SIZE_FORMAT " words on chunk " > METACHUNK_FORMAT ".", > to_commit, METACHUNK_FORMAT_ARGS(c)); > c->set_in_use(); // gets asserted in return_chunk(). > return_chunk_locked(c); > c = NULL; > } > } > ``` > > These are two questions (both good ones): 1) why does Metachunk::allocate not commit its own memory 2) why does ChunkManager::get_chunk() concern itself with committing the returned chunk (1) this is a matter of taste, rather unimportant. I played around with more "intelligent" Metachunk but found it easier to understand if the chunk was left dumb - not to take any action on its own. Note that there are cases where we allocate from a chunk but do not want to auto-commit further memory, see MetaspaceArena::salvage_chunk(). Not a strong reason, just a matter of taste. (2): Earlier prototypes of ChunkManager::get_chunk() did not concern itself with committing and did not make any promises toward the commit state of the returned chunk. So, MetaspaceArena::allocate had to take care to commit the chunk they got before allocating. Two problems though. One is a small performance issue. To commit memory we will need the global metaspace expand lock. In ChunkManager::get_chunk() we already have that lock. Since the caller, MetaspaceArena::allocate(), would invariably commit the start of the chunk anyway we might just as well commit it right now, in ChunkManager::get_chunk(). Since we are here, and have the lock.. The second problem is more subtle but also more important. Error handling is intricate. Committing memory may fail, we may hit a limit (gc threshold or MaxMetaspaceSize). In that case MetaspaceArena would be left with a fully uncommitted useless current chunk. What to do with this chunk? Keeping it as current chunk in the Arena is no option. MetaspaceArena::allocate() will now return NULL to signal allocation failure. Let's say we hit MayMetaspaceSize. Caller will attempt a GC. Maybe succeed. Maybe free some loaders. Maybe that causes the ChunkManager freelists to be repopulated with free committed chunks. But maybe not enough free chunks to actually uncommit something (none of the returned chuks being >= commit granule size). So we still have no commit-quota left to commit more memory. What we now do have is free committed chunks in the freelists, which we could use. So, caller calls MetaspaceArena::allocate() again. It still has its current uncommitted chunk. It would, again, try to commit that chunk. Fail again. Return NULL again. Wrongly so, since the ChunkManager free lists are populated with free committed chunks now which we could have used had we ignored the current chunk. So we get an OOM even though the GC beforehand did clean some loaders successfully. So, Arena needs to let go of the uncommitted chunk after a commit failure. Okay, easy enough. We could salvage that thing - add it to the list of retired chunks in MetaspaceArena. But since this "retry allocation" business is done repeatedly (see MetaspaceGC) we would, in this corner case, needlessly accumulate a number of completely uncommitted retired chunks. Last option, failing to commit this new chunk we could return it to the ChunkManager. And attempt to get a new one, one of those which had just been added after the GC. This is basically what happens now, only that this is completely handled inside of ChunkManager::get_chunk(). Bottom line: error handling is just simpler and cleaner if ChunkManager::get_chunk() never hands out a fully uncommitted chunk in the first place, and if we make it ChunkManager's business to ensure the returned chunk is at least usable to satisfy the allocation which caused its creation. > ## The motivation for using a buddy-allocator > > It seems that `MetaspaceArena::deallocate()` never releases the memory > to OS (memory is still committed); instead, it maintains those memory in > a free list for future allocation. In other words, memory is never > uncommitted until `~MetaspaceArena()`. Then, it's not obvious to me why > a buddy-allocator is used. I was think a lazily committed would be > enough, something like the following. Maybe I overlooked some > constraints in the original problem spec. Any clarification/elaboration > is appreciated, and they should probably be documented in the > corresponding file(s). > > ``` > struct root_chuck { > word* _end; > word* _committed_end; > word* _top; > > word* alloc(size_t s_in_words) { > if (_top + s_in_words > _end) { > // no space for this chuck > return nullptr; > } > if (_top + s_in_words > _committed_end) { > if (!commit(_committed_end, round_up(s_in_words, 64K))) { > // can't commit > return nullptr; > } > // update commit end > _committed_end += ... > } > // success path > auto ret = _top; > _top += s_in_words; > return ret; > } > }; > ``` > I am not sure I understand your question. The buddy allocator has nothing to do with the deallocation free list, and both have not much to do with uncommitting. Deallocation is a rare path taken when we prematurely deallocate metadata, e.g. on class redefinition or if we have class loading errors. The standard way to release metaspace is by a loader dying. The story behind deallocate() is basically: In 99.9% of all cases we have an allocation pattern very well suited for arena allocators - bulk free and so on - but in rare cases this story breaks apart. E.g. when facing an insane level of class redefinitions. For those pathological non-bulk-delete cases the allocator shall be good enough to not make the VM explode. Deallocation happens, like allocation, on block level (arbitrarily sized allocation units of 8...~512K bytes), with 99% of those allocations being in the single-to-dozen-word-range. The buddy allocator works on chunk level, typically much coarser than blocks, 1K...4M range. Or do you, perhaps, ask why we do not just allocate uniformly-sized chunks, give those to the Arenas and let them be committed on demand? That way we would not need a buddy allocator since all chunks would have the same size. This is what I think Leo asked in the call. The question is valid. The answer: To have a truly uniform chunk size it would have to be large enough for every conceivable allocation. So, a root chunk. If we give each loader a root chunk and let those chunks lazily commit themselves, we have the following problems: - We'd run up a ridiculous process vsize. Let's say you have 10000 class loaders (not unreasonable, and far from the worst I have ever seen). You'd use up about 80GB of address space for metaspace alone (with compressed class space active). Plain impossible on 32bit (okay, we would have no ccs but we would still need 40G). On 64bit, on most platforms vsize may not matter that much but still cause comments. - You would of course very quickly run against CompressedClassSpaceSize - one place where reserved size actually matters. Lets say we run with the default 1G of ccs. That gives us space enough for 256 root chunks. Assuming no arena ever wants a second chunk for class space allocation, that would mean we cannot have more than 256 loaders/CLDs. - Most importantly each arena would use up a complete commit granule. By default 64K. Way too much. Even if we shrink granule size to the lowest possible size, one 4K page (lets ignore 64K paged platforms): we would still use up 4K per loader. Still too much. As can be seen by the valiant attempt of RedHat when they tuned Metaspace usage for wildfly swarms: we have small loaders which only load a handful of classes or sometimes only one class; these were the reason the original Metaspace has those "specialized chunks" of 1K. In other words, chunks can be smaller than a commit granule and that makes sense since for small loaders we want to share a granule among multiple loaders. Finally, if you then say "okay, then let's introduce multiple chunks, maybe really small ones for small loaders" you are down the rabbit hole of the original Metaspace. The moment you have multiple chunk sizes you need a way to merge / split them etc. Witness the awkward way we do this now (Occupancy-Bitmap) in current Metaspace. The moment you have multiple chunk sizes the proposed buddy allocator is just the simplest implementation. --- > > ## minor comments > > `MetaspaceArena::retire_current_chunk()` is referenced in the some > places, but there's no such method. Maybe stale comments. > > Good catch. > > Better to use more informative names than `_chunks`, e.g. > `_allocated_chunks` vs `_free_chunks`, since they are quite different. > > Matter of taste. I disagree, since the "allocated" is implicit for an arena. I would only adorn the member names with adjectives if there were other members in the same class I have to differentiate from. > In `MetaspaceArena`, > > ``` > // List of chunks. Head of the list is the current chunk. > MetachunkList _chunks; > ``` > and `ChunkManager` > ``` > // Freelists > FreeChunkListVector _chunks; > ``` > -- > /Albert > Thanks Albert. Hope I could answer your questions. Cheers, Thomas From t-maana at microsoft.com Fri Aug 14 20:56:11 2020 From: t-maana at microsoft.com (Ana Marsh) Date: Fri, 14 Aug 2020 20:56:11 +0000 Subject: Timing Differences Between JFR and GC Logs Message-ID: Hi everyone, My name is Ana Marsh and I am interning at Microsoft on the Java Engineering Team (under Martijn Verburg) and this summer I have been diving into JFR in the JVM and analyzing how it gathers information on garbage collection events. I have noticed over the past two months of digging that the start time of a garbage collection event returned by JFR varies widely from the start time of the same event given with Unified Logging. I decided to explore this further by looking at the specifics of OpenJDK code and found some interesting points to highlight. 1. Logging decides to log the time of events as it prints and not necessarily as the events happen. That being said, it appears that JFR collects the time directly at the beginning of an event making it appear as a better marker of the start time of an event. 1. Garbage collection timing is measured in Ticks which holds values from two clocks when using an x86 machine. These clocks are called "os" (aka ElapsedCounterSource) and "rdtsc" (aka FastUnorderedElapsedCounterSource) and one value of Ticks will hold a relevant time for each clock. When a GC event begins (specifically Parallel GC because that is what I have been looking at) it registers the start in Ticks by calling now(). This is what now() looks like: jdk/src/hotspot/share/utilities/ticks.cpp CompositeElapsedCounterSource::Type CompositeElapsedCounterSource::now() { CompositeTime ct; ct.val1 = ElapsedCounterSource::now(); #if defined(X86) && !defined(ZERO) static bool initialized = false; static bool valid_rdtsc = false; if (!initialized) { valid_rdtsc = Rdtsc::initialize(); initialized = true; } if (valid_rdtsc) { ct.val2 = Rdtsc::elapsed_counter(); } #endif return ct; } ElapsedCounterSource::Type ElapsedCounterSource::now() { return os::elapsed_counter(); } For JFR files, the start time is set to the RDTSC value while in gc logs it uses the os clock as it prints out the log. To find the differences between the two I added this to the code: jdk/src/hotspot/share/utilities/ticks.cpp CompositeElapsedCounterSource::Type CompositeElapsedCounterSource::now() { CompositeTime ct; ct.val1 = ElapsedCounterSource::now(); #if defined(X86) && !defined(ZERO) static bool initialized = false; static bool valid_rdtsc = false; if (!initialized) { valid_rdtsc = Rdtsc::initialize(); initialized = true; } if (valid_rdtsc) { ct.val2 = Rdtsc::elapsed_counter(); } #endif std::cout << "I got ticks!!\n"; std::cout << "OS time: "; std::cout << ElapsedCounterSource::seconds(ct.val1) << " seconds\n"; std::cout<< "RDTSC time: "; std::cout << FastUnorderedElapsedCounterSource::seconds(ct.val2) << " seconds\n\n"; return ct; } Both ElapsedCounterSource (os) and FastUnorderedElapsedCounterSource (RDTSC) have their own way of determining seconds so there could be some issues in there as well. Additionally in this example the clocks are not being set instantaneously causing some numbers to be off, however at the beginning of my test the os clock ends up reporting a later time even though it is called first. Over time though RDTSC seems to begin reporting later times. Overall these clocks get far very off, in degrees of seconds which seems big in the clock world. I ran this with a test application that forces garbage collection events and I also had a JFR and GC log being recorded during the run as well. For reference I am running on a Windows 10 machine but using WSL with Ubuntu. I attached the output of my run to this email. Any insight or thoughts on this matter would be helpful. Thank you! Best, Ana Marsh -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: seconds.txt URL: From albert.m.yang at oracle.com Sat Aug 22 08:47:36 2020 From: albert.m.yang at oracle.com (Albert Yang) Date: Sat, 22 Aug 2020 10:47:36 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: Message-ID: <858c2f0b-c6a8-0726-cd97-9641396d8d14@oracle.com> Hi Thomas, Thank you for preparing the review guide, which has been quite helpful to see the big picture, and in providing a starting point to embrace this large patch/code base. ## Metachunk exposing committed memory status One thing I find odd is that chunk doesn't encapsulate committed memory status. In other words, a component interacting with a chuck needs to be aware of how much memory is committed. I will provide two examples in code, one is the "consumer" of `Metachunk`, the other the "producer" of `Metachunk`. For the "consumer" case, `MetaspaceArena::allocate`, allocation is broken into two steps, 1. for committed memory, 2. for virtual memory (+committed memory from step 1). ``` if (!current_chunk_too_small) { if (!current_chunk()->ensure_committed_additional(raw_word_size)) { UL2(info, "commit failure (requested size: " SIZE_FORMAT ")", raw_word_size); commit_failure = true; } } // Allocate from the current chunk. This should work now. if (!current_chunk_too_small && !commit_failure) { p = current_chunk()->allocate(raw_word_size); assert(p != NULL, "Allocation from chunk failed."); } ``` If the "committed memory status" is properly encapsulated in `Metachunk::allocate`, the above code could be like: ``` if (!current_chunk_too_small) { p = current_chunk()->allocate(raw_word_size); } ``` For the "producer" case, `ChunkManager::get_chunk` mostly deals with identifying a chunk which satisfied some constraints, but the following code focuses on committed bytes, which, in my opinion, is on another abstraction level. I believe that moving such logic to `Metachunk::allocate` could make code more readable. ``` // Attempt to commit the chunk (depending on settings, we either fully commit it or just // commit enough to get the caller going). That may fail if we hit a commit limit. In // that case put the chunk back to the freelist (re-merging it with its neighbors if we // did split it) and return NULL. const size_t to_commit = Settings::new_chunks_are_fully_committed() ? c->word_size() : min_committed_words; if (c->committed_words() < to_commit) { if (c->ensure_committed_locked(to_commit) == false) { UL2(info, "failed to commit " SIZE_FORMAT " words on chunk " METACHUNK_FORMAT ".", to_commit, METACHUNK_FORMAT_ARGS(c)); c->set_in_use(); // gets asserted in return_chunk(). return_chunk_locked(c); c = NULL; } } ``` ## The motivation for using a buddy-allocator It seems that `MetaspaceArena::deallocate()` never releases the memory to OS (memory is still committed); instead, it maintains those memory in a free list for future allocation. In other words, memory is never uncommitted until `~MetaspaceArena()`. Then, it's not obvious to me why a buddy-allocator is used. I was think a lazily committed would be enough, something like the following. Maybe I overlooked some constraints in the original problem spec. Any clarification/elaboration is appreciated, and they should probably be documented in the corresponding file(s). ``` struct root_chuck { word* _end; word* _committed_end; word* _top; word* alloc(size_t s_in_words) { if (_top + s_in_words > _end) { // no space for this chuck return nullptr; } if (_top + s_in_words > _committed_end) { if (!commit(_committed_end, round_up(s_in_words, 64K))) { // can't commit return nullptr; } // update commit end _committed_end += ... } // success path auto ret = _top; _top += s_in_words; return ret; } }; ``` ## minor comments `MetaspaceArena::retire_current_chunk()` is referenced in the some places, but there's no such method. Maybe stale comments. Better to use more informative names than `_chunks`, e.g. `_allocated_chunks` vs `_free_chunks`, since they are quite different. In `MetaspaceArena`, ``` // List of chunks. Head of the list is the current chunk. MetachunkList _chunks; ``` and `ChunkManager` ``` // Freelists FreeChunkListVector _chunks; ``` -- /Albert From thomas.stuefe at gmail.com Sun Aug 23 08:31:18 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sun, 23 Aug 2020 10:31:18 +0200 Subject: RFR(S) Remove excessive inclusion of arguments.hpp In-Reply-To: <09c2f053-2c61-5b7d-ea83-18c1db3c2b8e@oracle.com> References: <09c2f053-2c61-5b7d-ea83-18c1db3c2b8e@oracle.com> Message-ID: Hi Ioi, looks okay, but why the added java.hpp includes? At first glance does not seem related to your bug description. Thanks Thomas On Fri, Aug 21, 2020 at 6:47 PM Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8252151 > > http://cr.openjdk.java.net/~iklam/jdk16/8252151-dont-include-arguments-hpp.v01/ > > The large header file arguments.hpp is included by popular header files > such as constMethod.hpp and methodCounters.hpp (which are included by > method.hpp, which is included by almost every cpp file). > > The dependencies on arguments.hpp by constMethod.hpp are very small, and > can be moved to constMethod.cpp. > > methodCounters.hpp does not depend on arguments.hpp > > === > by removing these dependencies, we can reduce the total number of > include files in the hotspot build from 530,921 to 523,010, or a > reduction of about 1.5%. > > > From albert.m.yang at oracle.com Sun Aug 23 22:03:43 2020 From: albert.m.yang at oracle.com (Albert Yang) Date: Mon, 24 Aug 2020 00:03:43 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: <858c2f0b-c6a8-0726-cd97-9641396d8d14@oracle.com> Message-ID: <6dd80f85-325d-f88a-2799-666981efdd00@oracle.com> Hi Thomas, > error handling is just simpler and cleaner if ChunkManager::get_chunk() never hands out a fully uncommitted chunk in the first place I see. Then, when the current chunk is salvaged, it must have a positive used words, as the first request will always be satisfied, right? I added `assert(false)` in `MetaspaceArena::salvage_chunk()`, running `make test TEST='gtest:metaspace test/hotspot/jtreg/gtest/MetaspaceGtests.java test/hotspot/jtreg/runtime/Metaspace/elastic'` doesn't trigger the assertion. Is it truly unreachable or am I missing some tests? ``` if (c->used_words() == 0) { assert(false, ""); // newly added UL2(trace, "salvage: returning empty chunk " METACHUNK_FORMAT ".", METACHUNK_FORMAT_ARGS(c)); _chunk_manager->return_chunk(c); return; } ``` Memory allocation in metaspace involves both virtual (potentially enlarging a chunk) and physical (potentially committing memory) address space, and either one could fail. The second step, if failed, needs to undo the first step as well. Such tight-coupling is the main reason (to me) to not expose this as two steps to the caller. As I read it, `MetaspaceArena::allocate()` actually doesn't release the virtual memory used in the first if the second step fails. Not sure if there are any consequences or the severity. I wonder if the problem could be simpler if chunks smaller/larger than the commit granule are dealt with in two different ways. This way when we deal with chunks smaller than the commit granule, we know it's already committed. > In other words, chunks can be smaller than a commit granule and that makes sense since for small loaders we want to share a granule among multiple loaders. It makes sense. Somehow, I got the impression that MetaspaceArena has 1:1 mapping to ChunkManager, but actually all arenas share the singleton class/non-class chunk manager. The diagram in https://cr.openjdk.java.net/~stuefe/JEP-Improve-Metaspace-Allocator/review-guide/review-guide-1.0.html#2-subsystems was quite clear, but I think I ignored it after failing to find "SpaceManager" in the code. (Maybe the diagram is out of sync with the code?) Thank you for the explanation, which, I believe, should be included in the comments for documentation purpose. (I didn't find the rational for using a buddy allocator in the code comments.) Thank you once again for the clarification; I have got a better understanding of the patch. -- /Albert From yumin.qi at oracle.com Sun Aug 23 22:56:44 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Sun, 23 Aug 2020 15:56:44 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> Message-ID: Hi, Mandy, Ioi and Calvin ? I have updated the new changed at: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-03/ ? In this version: ??? 1) Added a new API to check if flag DumpLoadedClassList set and the file is open. If true, call into vm to print out the trace line to the log file. ??????? Just thinking if we just call the cdsTraceResolve without checking if the flag DumpLoadedClassList set and file open, this way, the check logic is in the vm side like before, so save code by not adding the new API. ??? 2) The returned holder class names now are just 'package/className', removed head and tail. ??? 3)? Remove add_extra_classes from CollectClassesClosure since after bug 8250990: https://bugs.openjdk.java.net/browse/JDK-8250990 pushed, the CollectClassesClosure no longer exist. ??? 4)? Still keep the parsing for TRACE_RESOLVE in java/lang/invoke/GenerateJLIClassesHelper.java, so VM call its function to regenerate holder classes. ? Re-tested Mach5 tier1-4 Thanks Yumin ** On 8/20/20 8:05 PM, Yumin Qi wrote: > Hi, Mandy > > > On 8/20/20 5:10 PM, Mandy Chung wrote: >> >> >> On 8/19/20 10:14 PM, Yumin Qi wrote: >>> >>> HI, Mandy >>> >>> ? Thanks for the review, I took one day off yesterday so just got a detail look of your reply. >>> >>> On 8/19/20 1:30 PM, Mandy Chung wrote: >>>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>>> Hi, Ioi >>>>> >>>>> ? Thanks for review/suggestion. I have updated the webrev at the following link: >>>>> >>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>>> >>>> >>>> This patch leverages the TRACE_RESOLVE output and passes the trace output to VM.? VM then calls GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing and generate Holder class per the resolved LFs.?? I think there are other cleaner alternatives implementing this.?? jlink --generate-jli-classes plugin depends the trace output whereas -Xshare:dump does not.?? It's cleaner to skip generating the trace output and parsing for dumping shared archive purpose.? In addition, the implementation needs some cleanup (I can send you feedback on the next revision) >>>> >>> Current patch did not change the existing code for JLinkPlugin part. I just moved the parsing code from GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java since the former is an internal class to which we shouldn't call to generate holder classes. >>>> Instead of relying on a system property "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. >>> >>> I remember we have such API to query if flag -Xshare:dump or -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This flag is the one used to log class name to list file. In GenerateLinkOptData.gmk: >>> >>> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR) >>> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) >>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $(JLI_TRACE_FILE))) >>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw \ >>> ??????????? -Duser.language=en -Duser.country=US \ >>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >>> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw.2 \ >>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>> ??????????? -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >>> ??????????? -Duser.language=en -Duser.country=US \ >>> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>> ??????????? build.tools.classlist.HelloClasslist \ >>> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >>> ??????????? || ( \ >>> ??????????????? exitcode=$$? ; \ >>> ??????????????? $(ECHO) "ERROR: Failed to generate link optimization data." \ >>> ??????????????????? "This is likely a problem with the newly built JVM/JDK." ; \ >>> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >>> ??????????????? exit $$exitcode \ >>> ??????????? ) >>> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >>> >>> The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current implementation, DumpLoadedClassList will turn on property java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same output sent to stdout and log file DumpLoadedClassList specified. >>> >> >> These entries are duplicated in two different files: one for jlink --generate-jli-classes plugin and another for CDS use. CDS -Xshare:dump attempts to do what jlink plugin does but writes those generated classes in to shared archive. >> >> Like the above make logic to build JDK image, the same entries are written in both default-jli-trace.txt via System.out and to classlist via JNI call to the VM.? I guess VM also implements the logic to do some kind of diffing and write to CDS archive. >> > In current implementation, vm side only records the line as from TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at dump time, call back to java for parsing those recordings and generating the holder classes, this uses the existing JLI code. >>> >>> Now instead of this property, using a vm interface API to query if this flag is set, I think it is better choice. But here I am NOT sure I understand your suggestion, I think there are two choices: >>> >>> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set >>> >>> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting those name, type and holder name to regenerate holder classes? >>> >> >> I misunderstood that this CDS_TRACE_RESOLVE flag is set during -Xshare:dump time. >> >> Ioi has clarified to me offline that this step is actually part of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the given class list file, i.e. you repurpose the class list file to include the log output that was initially designed for jlink plugin. >> >> To me, I'd prefer to see this support depending on `jlink --generate-jli-classes` which is an existing functionality and much cleaner.? This way this does not require any VM change. It will generate the holder classes in the custom image per the application-specific config file. >> >> What it means is that: a customer would need to create a custom image with their application-specific config file.? It might need a new CDS? option to specify a separate TRACE_RESOLVE file.?? It would turn on this feature by default by defining a default path of the log file if it helps. >> > So for now, I would implement an API to query if flag DumpLoadedClassList set in cmd line, remove new added property of java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address custom image with CDS in future in a separate issue. > > > Thanks > > Yumin > >> I understand that this is not the existing CDS work flow and CDS archive does not require to run on a custom image.?? I see the value of this approach which can prepare customers to start building and using its own custom image. >> >> Of course the implementation would be much simpler (adding a flag to write these traces to a given file path, that is). >> >> Mandy From jamsheed.c.m at oracle.com Mon Aug 24 05:36:51 2020 From: jamsheed.c.m at oracle.com (Jamsheed C M) Date: Mon, 24 Aug 2020 11:06:51 +0530 Subject: RFR: 8249451: Unconditional exceptions clearing logic in compiler code should honor Async Exceptions In-Reply-To: <442caa21-ca0a-f6eb-60a5-1e74bf994894@oracle.com> References: <442caa21-ca0a-f6eb-60a5-1e74bf994894@oracle.com> Message-ID: <03df9364-817d-04d6-6434-80be93a66526@oracle.com> Hi David, Thank you for the review and feedback. Agree on all of them. I will rework and get back. On 10/08/2020 07:33, David Holmes wrote: > Hi Jamsheed, > > On 6/08/2020 10:07 pm, Jamsheed C M wrote: >> Hi all, >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8249451 >> >> webrev: http://cr.openjdk.java.net/~jcm/8249451/webrev.00/ > > Thanks for tackling this messy issue. Overall I like the use of TRAPS > to more clearly document which methods can return with an exception > pending. I think there are some problems with the proposed changes. > I'll start with those comments and then move on to more general comments. > > src/hotspot/share/utilities/exceptions.cpp > src/hotspot/share/utilities/exceptions.hpp > > I don't think the changes here are correct or safe in general. > > First, adding the new macro and function to only clear non-async > exceptions is fine itself. But naming wise the fact only non-async > exceptions are cleared should be evident, and there is no "check" > involved (in the sense of the existing CHECK_ macros) so I suggest: > > s/CHECK_CLEAR_PENDING_EXCEPTION/CLEAR_PENDING_NONASYNC_EXCEPTIONS/ > s/check_clear_pending_exception/clear_pending_nonasync_exceptions/ > Ok > But changing the existing CHECK_AND_CLEAR macros to now leave async > exceptions pending seems potentially dangerous as calling code may not > be prepared for there to now be a pending exception. For example the > use in thread.cpp: > > ?JDK_Version::set_runtime_name(get_java_runtime_name(THREAD)); > ?JDK_Version::set_runtime_version(get_java_runtime_version(THREAD)); > > get_java_runtime_name() is currently guaranteed to clear all > exceptions, so all the other code is known to be safe to call. But > that would no longer be true. That said, this is VM initialization > code and an async exception is impossible at this stage. > > I think I would rather see CHECK_AND_CLEAR left as-is, and an actual > CHECK_AND_CLEAR_NONASYNC introduced for those users of CHECK_AND_CLEAR > that can encounter async exceptions and which should not clear them. > > +?? if > (!_pending_exception->is_a(SystemDictionary::ThreadDeath_klass()) && > +?????? _pending_exception->klass() != > SystemDictionary::InternalError_klass()) { > Ok > Flagging all InternalErrors as async exceptions is probably also not > correct. I don't see a good solution to this at the moment. I think we > would need to introduce a new subclass of InternalError for the unsafe > access error case**. Now it may be that all the other InternalError > usages are "impossible" in the context of where the new macros are to > be used, but that is very difficult to establish or assert. > > ** Or perhaps we could inject a field that allows the VM to identify > instances related to unsafe access errors ... Ideally of course these > unsafe access errors would be distinct from the async exception > mechanism - something I would still like to pursue. > Ok > --- > > General comments ... > > There is a general change from "JavaThread* thread" to "Thread* > THREAD" (or TRAPS) to allow the use of the CHECK macros. This is > unfortunate because the fact the thread is restricted to being a > JavaThread is no longer evident in the method signatures. That is a > flaw with the TRAPS/CHECK mechanism unfortunately :( . But as the > methods no longer take a JavaThread* arg, they should assert that > THREAD->is_Java_thread(). I will also look at an RFE to have > as_JavaThread() to avoid the need for separate assertion checks before > casting from "Thread*" to "JavaThread*". > Ok > Note there's no need to use CHECK when the enclosing method is going > to return immediately after the call that contains the CHECK. It just > adds unnecessary checking of the exception state. The use of TRAPS > shows that the methods may return with an exception pending. I've > flagged all such occurrences I spotted below. > Ok > --- > > +?? // Only metaspace OOM is expected. no Java code executed. > > Nit: s/no/No > > > src/hotspot/share/compiler/compilationPolicy.cpp > > > ?410?????? method_invocation_event(method, CHECK_NULL); > ?489?????? CompileBroker::compile_method(m, InvocationEntryBci, > comp_level, m, hot_count, CompileTask::Reason_InvocationCount, CHECK); > > Nit: there's no need to use CHECK here. > > --- > > src/hotspot/share/compiler/tieredThresholdPolicy.cpp > > ?504???? method_invocation_event(method, inlinee, comp_level, nm, > CHECK_NULL); > ?570???????? compile(mh, bci, CompLevel_simple, CHECK); > ?581???????? compile(mh, bci, CompLevel_simple, CHECK); > ?595???? CompileBroker::compile_method(mh, bci, level, mh, hot_count, > CompileTask::Reason_Tiered, CHECK); > 1062?????? compile(mh, InvocationEntryBci, next_level, CHECK); > > Nit: there's no need to use CHECK here. > > 814 void TieredThresholdPolicy::create_mdo(const methodHandle& mh, > Thread* THREAD) { > > Thank you for correcting this misuse of the THREAD name on a > JavaThread* type. > > --- > > src/hotspot/share/interpreter/linkResolver.cpp > > ?128?? CompilationPolicy::compile_if_required(selected_method, CHECK); > > Nit: there's no need to use CHECK here. > > --- > > src/hotspot/share/jvmci/compilerRuntime.cpp > > ?260???? CompilationPolicy::policy()->event(emh, mh, > InvocationEntryBci, InvocationEntryBci, CompLevel_aot, cm, CHECK); > ?280???? nmethod* osr_nm = CompilationPolicy::policy()->event(emh, mh, > branch_bci, target_bci, CompLevel_aot, cm, CHECK); > > Nit: there's no need to use CHECK here. > > --- > > src/hotspot/share/jvmci/jvmciRuntime.cpp > > ?102???????? // Donot clear probable async exceptions. > > typo: s/Donot/Do not/ > > --- > > src/hotspot/share/runtime/deoptimization.cpp > > 1686 void Deoptimization::load_class_by_index(const > constantPoolHandle& constant_pool, int index) { > > This method should be declared with TRAPS now. > > 1693???? // Donot clear probable Async Exceptions. > > typo: s/Donot/Do not/ > > Ok >> testing : mach1-5(links in jbs) > > There is very little existing testing that will actually test the key > changes you have made here. You will need to do direct fault-injection > testing anywhere you now allow async exceptions to remain, to see if > the calling code can tolerate that. It will be difficult to test > thoroughly. > Ok > Thanks again for tackling this difficult problem! Best regards, Jamsheed > > David > ----- > >> >> While working on JDK-8246381 it was noticed that compilation request >> path clears all exceptions(including async) and doesn't propagate[1]. >> >> Fix: patch restores the propagation behavior for the probable async >> exceptions. >> >> Compilation request path propagate exception as in [2]. MDO and >> MethodCounter doesn't expect any exception other than metaspace >> OOM(added comments). >> >> Deoptimization path doesn't clear probable async exceptions and take >> unpack_exception path for non uncommontraps. >> >> Added java_lang_InternalError to well known classes. >> >> Request for review. >> >> Best Regards, >> >> Jamsheed >> >> [1] w.r.t changes done for JDK-7131259 >> >> [2] >> >> ???? (a) >> ???? -----> c1_Runtime1.cpp/interpreterRuntime.cpp/compilerRuntime.cpp >> ?????? | >> ??????? ----- compilationPolicy.cpp/tieredThresholdPolicy.cpp >> ????????? | >> ?????????? ------ compileBroker.cpp >> >> ???? (b) >> ???? Xcomp versions >> ???? ------> compilationPolicy.cpp >> ??????? | >> ???????? ------> compileBroker.cpp >> >> ???? (c) >> >> ???? Direct call to? compile_method in compileBroker.cpp >> >> ???? JVMCI bootstrap, whitebox, replayCompile. >> >> From thomas.stuefe at gmail.com Mon Aug 24 06:11:48 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 24 Aug 2020 08:11:48 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: <6dd80f85-325d-f88a-2799-666981efdd00@oracle.com> References: <858c2f0b-c6a8-0726-cd97-9641396d8d14@oracle.com> <6dd80f85-325d-f88a-2799-666981efdd00@oracle.com> Message-ID: Hi Albert, On Mon, Aug 24, 2020 at 12:06 AM Albert Yang wrote: > Hi Thomas, > > > error handling is just simpler and cleaner if ChunkManager::get_chunk() > never hands out > a fully uncommitted chunk in the first place > > I see. Then, when the current chunk is salvaged, it must have a positive > used words, as > the first request will always be satisfied, right? I added `assert(false)` > in > `MetaspaceArena::salvage_chunk()`, running `make test > TEST='gtest:metaspace > test/hotspot/jtreg/gtest/MetaspaceGtests.java > test/hotspot/jtreg/runtime/Metaspace/elastic'` doesn't trigger the > assertion. Is it truly > unreachable or am I missing some tests? > > ``` > if (c->used_words() == 0) { > assert(false, ""); // newly added > UL2(trace, "salvage: returning empty chunk " METACHUNK_FORMAT ".", > METACHUNK_FORMAT_ARGS(c)); > _chunk_manager->return_chunk(c); > return; > } > ``` > Good spotting. I removed this piece of code last week, but that change did not find its way yet into a new webrev. summary: Remove a dead portion of code http://hg.openjdk.java.net/jdk/sandbox/rev/074a374e9c5c It was a leftover from the time where ChunkManager::get_chunk() could return uncommitted chunks, the arene then committed the chunk, failed and retired that thing. As I said the way it's done now is just simpler. > > Memory allocation in metaspace involves both virtual (potentially > enlarging a chunk) Enlarging the chunk has nothing to do with reservation. Reservation happens in a layer below that, in VirtualSpaceList/VirtualSpaceNode. When we enlarge the chunk, or carve out a new chunk, even a root chunk, space is already reserved. MetaspareArena::allocate() may indirectly cause additional reservation - rarely - if the underlying VSList is expandable and the allocation causes creation of a new VSNode. > and > physical (potentially committing memory) address space, and either one > could fail. The > second step, if failed, needs to undo the first step as well. Such > tight-coupling is the > main reason (to me) to not expose this as two steps to the caller. As I > read it, > `MetaspaceArena::allocate()` actually doesn't release the virtual memory > used in the first > if the second step fails. Not sure if there are any consequences or the > severity. > Yes, but quite irrelevant. This can happen in two cases: - the commit quota hits us when the underlying VSNode has not been fully committed yet. For the non-class metaspace this means delta between committed and reserved memory for a single MetaspaceContext is smaller than Settings::virtual_space_node_default_word_size() which is atm 8MB. For the compressed class space, whose VSList only has one node which is as large as CompressedClassSpaceSize, this delta can be as large as CompressedClassSpaceSize-. However, the latter case means someone specified a smaller MaxMetaspaceSize than CompressedClassSpaceSize, and we cannot use up the compressed class space since the commit quota is too small. Which is expected behaviour. - the commit quota hits us when allocating from non-class metaspace and the allocation just caused us to create a new VSNode, which then remains completely uncommitted. In that rare case we are left with a delta of Settings::virtual_space_node_default_word_size(), so 8MB. Bottomline, the delta will be, for correctly sized Metaspace, at most ever 8MB ever. Which in the big scheme of things does not matter at all. Any attempt to get rid of that would complicate the coding for a miniscule gain in process vsize. Not sure I follow your "tight coupling" remark. The coupling between the sub systems is quite loose. > I wonder if the problem could be simpler if chunks smaller/larger than the > commit granule > are dealt with in two different ways. This way when we deal with chunks > smaller than the > commit granule, we know it's already committed. > I do not understand what the problem is you want to solve. > > > In other words, chunks can be smaller than a commit granule and that > makes sense since > for small loaders we want to share a granule among multiple loaders. > > It makes sense. Somehow, I got the impression that MetaspaceArena has 1:1 > mapping to > ChunkManager, but actually all arenas share the singleton class/non-class > chunk manager. > The diagram in > > https://cr.openjdk.java.net/~stuefe/JEP-Improve-Metaspace-Allocator/review-guide/review-guide-1.0.html#2-subsystems > was quite clear, but I think I ignored it after failing to find > "SpaceManager" in the > code. (Maybe the diagram is out of sync with the code?) Yes, you have an old link. The current review guide version was mentioned in my original RFR mail: http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.html When putting out a new webrev, it will come with an updated version of the review guide. The master data for both are kept at github: https://github.com/tstuefe/jep387/tree/master/review in case openjdk.java.net is not reachable. Thank you for the explanation, > which, I believe, should be included in the comments for documentation > purpose. (I didn't > find the rational for using a buddy allocator in the code comments.) > > I can try and further beef up the comments. > Thank you once again for the clarification; I have got a better > understanding of the patch. > > Sure. Thanks for the review work, Thomas > -- > /Albert > From albert.m.yang at oracle.com Mon Aug 24 10:45:34 2020 From: albert.m.yang at oracle.com (Albert Yang) Date: Mon, 24 Aug 2020 12:45:34 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: <858c2f0b-c6a8-0726-cd97-9641396d8d14@oracle.com> <6dd80f85-325d-f88a-2799-666981efdd00@oracle.com> Message-ID: Hi Thomas, > Enlarging the chunk has nothing to do with reservation. I didn't mean reserving the virtual space; instead, the chunk is "reserved" and can't be used for later allocation. Using `MetaspaceArena::allocate` to illustrate my point. ``` if (current_chunk()->free_words() < raw_word_size) { if (!attempt_enlarge_current_chunk(raw_word_size)) { // step 1: reserve the chunk; assume the current chunk is 64K, and after enlarging, it becomes (64+64) K. current_chunk_too_small = true; } else { DEBUG_ONLY(InternalStats::inc_num_chunks_enlarged();) UL(debug, "enlarged chunk."); } } // Commit the chunk far enough to hold the requested word size. If that fails, we // hit a limit (either GC threshold or MaxMetaspaceSize). In that case retire the // chunk. if (!current_chunk_too_small) { if (!current_chunk()->ensure_committed_additional(raw_word_size)) { // step 2: commit to physical memory, if fails, the newly "reserved" 64K is leaked, right? UL2(info, "commit failure (requested size: " SIZE_FORMAT ")", raw_word_size); commit_failure = true; } } ``` The step 2, if failed, needs to undo step 1. That's what I meant that step 1 and 2 are tightly-coupled. Maybe leaking 64K in virtual address is not that significant. Better explain it in the comments so that future readers know this problem is known. > When putting out a new webrev, it will come with an updated version of the review guide. The master data for both are kept at github: https://github.com/tstuefe/jep387/tree/master/review in case openjdk.java.net is not reachable. Thank you very much; the new guide is even better than the version I originally followed. -- /Albert From thomas.stuefe at gmail.com Mon Aug 24 12:26:02 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Mon, 24 Aug 2020 14:26:02 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: <858c2f0b-c6a8-0726-cd97-9641396d8d14@oracle.com> <6dd80f85-325d-f88a-2799-666981efdd00@oracle.com> Message-ID: On Mon, Aug 24, 2020 at 12:47 PM Albert Yang wrote: > Hi Thomas, > > > Enlarging the chunk has nothing to do with reservation. > > I didn't mean reserving the virtual space; instead, the chunk is > "reserved" and can't be > used for later allocation. Using `MetaspaceArena::allocate` to illustrate > my point. > > ``` > if (current_chunk()->free_words() < raw_word_size) { > if (!attempt_enlarge_current_chunk(raw_word_size)) { // step 1: > reserve the chunk; > assume the current chunk is 64K, and after enlarging, it becomes (64+64) K. > current_chunk_too_small = true; > } else { > DEBUG_ONLY(InternalStats::inc_num_chunks_enlarged();) > UL(debug, "enlarged chunk."); > } > } > > // Commit the chunk far enough to hold the requested word size. If > that fails, we > // hit a limit (either GC threshold or MaxMetaspaceSize). In that > case retire the > // chunk. > if (!current_chunk_too_small) { > if (!current_chunk()->ensure_committed_additional(raw_word_size)) { > // step 2: > commit to physical memory, if fails, the newly "reserved" 64K is leaked, > right? > UL2(info, "commit failure (requested size: " SIZE_FORMAT ")", > raw_word_size); > commit_failure = true; > } > } > ``` > > The step 2, if failed, needs to undo step 1. That's what I meant that step > 1 and 2 are > tightly-coupled. > > OK I see now what you mean. There are two kinds of "reserved": "reserved and publicly available"; and "reserved but earmarked for one loader, denied to other loaders". The latter comes in the form of uncommitted memory in the current or retired chunks of a live arena. The problem is more generic than the enlarge-chunk path. If an arena is unable to further commit the current chunk, it asks the ChunkManager for a new, suitably committed chunk. If (and only if) it got one, it will switch to that new chunk from now on. Before changing the current chunk to the new one, the arena will salvage the old chunk. Salvaging means sucking the last words of remaining committed space out of it and storing those words in the freeblocklist. The remaining uncommitted space is ignored right now, "wasted" in the sense you perceive. While this is true, I do not think this is a real problem. It only can happen if we hit MaxMetaspaceSize (so not: GC threshold - there, caller will increase commit quota, and follow up commits on the current chunk will be successful). Hitting MaxMetaspaceSize is very rare; the option, by default, is infinite. We may conceivably run into a pathological situation where we "hover" at that limit, again and again hitting it, the subsequent GCs releasing some loaders, enough to avoid OOM but never quite enough to get away from the limit for good... I don't know. Seems far fetched. The simplest and most elegant solution to that would be to split off unused portions of a salvaged chunk into separate chunks (e.g. when you salvage a 64K chunk of which you used 13K, you could split it into (16K+16K+32K), retain the first 16K chunk as retired chunk, return the second and third chunk back to the ChunkManager. I thought about a similar solution at other points, e.g. periodically to "shave off" remaining space from in-use chunks from live arenas. But there is no pressing need and I want to keep the patch simple. Lets keep this in mind for a future enhancement. > Maybe leaking 64K in virtual address is not that significant. Better > explain it in the > comments so that future readers know this problem is known. > > > When putting out a new webrev, it will come with an updated version of > the review > guide. The master data for both are kept at github: > https://github.com/tstuefe/jep387/tree/master/review in case > openjdk.java.net is not > reachable. > > Thank you very much; the new guide is even better than the version I > originally followed. > > -- > /Albert > Thanks, Thomas From ioi.lam at oracle.com Mon Aug 24 17:14:38 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 24 Aug 2020 10:14:38 -0700 Subject: RFR(S) Remove excessive inclusion of arguments.hpp In-Reply-To: References: <09c2f053-2c61-5b7d-ea83-18c1db3c2b8e@oracle.com> Message-ID: Hi Thomas, thanks for the review. Some cpp files use vm_exit_during_initialization(), which is defined in java.hpp, which was included by arguments.hpp. Now that I have removed arguments.hpp (indirectly) from these cpp files, I have to add java.hpp to fix the missing dependency. Thanks - Ioi On 8/23/20 1:31 AM, Thomas St?fe wrote: > Hi Ioi, > > looks okay, but why the added java.hpp includes? At first glance does > not seem related to your bug description. > > Thanks Thomas > > On Fri, Aug 21, 2020 at 6:47 PM Ioi Lam > wrote: > > https://bugs.openjdk.java.net/browse/JDK-8252151 > http://cr.openjdk.java.net/~iklam/jdk16/8252151-dont-include-arguments-hpp.v01/ > > The large header file arguments.hpp is included by popular header > files > such as constMethod.hpp and methodCounters.hpp (which are included by > method.hpp, which is included by almost every cpp file). > > The dependencies on arguments.hpp by constMethod.hpp are very > small, and > can be moved to constMethod.cpp. > > methodCounters.hpp does not depend on arguments.hpp > > === > by removing these dependencies, we can reduce the total number of > include files in the hotspot build from 530,921 to 523,010, or a > reduction of about 1.5%. > > From coleen.phillimore at oracle.com Mon Aug 24 17:18:42 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 24 Aug 2020 10:18:42 -0700 (PDT) Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: <8bd366a3-a40b-c2d6-8e19-1abf7a4c7e04@oracle.com> Message-ID: Looking though more code: === metaspaceArena.cpp bool MetaspaceArena::attempt_enlarge_current_chunk(size_t requested_word_size) { Since this function has most lines starting with c->, it seems like this function should be in metachunk current_chunk()->attempt_enlarge_current_chunk(requested_size, next_chunk_level()); Coleen On 8/19/20 8:31 PM, Coleen Phillimore wrote: > > Sorry I dropped off hotspot-gc in my first review. > > A few replies below. > > On 8/19/20 3:43 AM, Thomas St?fe wrote: >> Hi Coleen, >> >> On Wed, Aug 19, 2020 at 12:19 AM Coleen Phillimore >> > >> wrote: >> >> >> ??? Hi Thomas, >> >> ??? I've read through the new implementation files.? This looks really >> ??? good!? These are my initial comments.? There aren't many and >> ??? nothing of >> ??? substance really.? Some files need more in-depth review though. I >> ??? haven't looked at the tests, which are more difficult.? I'll do that >> ??? tomorrow. >> >> ??? General comment:? Some files have a lot of vertical white space, >> like >> ??? two or more blank lines together: eg. metachunk.cpp at the end.? >> More >> ??? about that below. >> >> >> A bad habit of mine, idly pressing space white?thinking. I'll fix those. >> >> >> ??? === counter.hpp >> >> ??? This seems like a general utility that should be in utilities so >> ??? that it >> ??? can be generalized for use in other places.? I didn't review it >> ??? carefully to suggest it go there right now, but we should file >> ??? another >> ??? RFE to do that.? Kim has some ideas about it. >> >> >> I agree. I kept this class here to keep the patch confined to >> memory/metaspace. I would like cleanups like this as follow ups to >> the main patch, as you suggested. I created >> https://bugs.openjdk.java.net/browse/JDK-8252014?to keep track of this. >> >> >> ??? === freeBlocks.hpp >> >> ??? I like how the small blocks and tree of freed blocks are together >> ??? (except see below). >> ??? This has a really good comment. >> >> >> Thanks. >> >> >> ??? #ifndef SHARE_MEMORY_METASPACE_LEFTOVERBINS_HPP >> ??? #define SHARE_MEMORY_METASPACE_LEFTOVERBINS_HPP >> ??? Wrong ifdef guard names. >> ??? #endif // SHARE_MEMORY_METASPACE_CHUNKMANAGER_HPP >> >> >> Ok >> >> ??? === binList.hpp >> >> ??? ?? block_t* _v[num_lists]; >> >> ??? Some slightly longer name would be better than _v.? Maybe >> ??? _small_blocks >> ??? or something like that? >> >> >> Would _blocks be acceptable? Since the distinguisher 'small' makes no >> sense here, it is the only member in the class. > > Yes, _blocks is good. > >> ??? // These aren't used >> ??? typedef BinListImpl<2, 8>? BinList8; >> ??? typedef BinListImpl<2, 16> BinList16; >> ??? typedef BinListImpl<2, 64> BinList64; >> >> ??? This file should be binList.hpp rather than binlist.hpp. >> >> >> Ok >> >> ??? === blockTree.hpp/cpp >> >> ??? I didn't review this code.? Someone suggested using std::map >> ??? instead but >> ??? we're not using std:: yet. >> >> >> I don't really like that idea.?Using stl maps for structures as these >> in-place blocks seems, depending on how it is done, either risky or >> wasteful. >> >> In this tree payload and nodes are identical and the size of the >> payload is the key. Like the Linux rbtree, really. STL map >> keeps?these?entities separate: a KV pair holds K (block size) and V >> (the block). So we spent additional memory and allocation to keep the >> pair itself, and K is also kept redundant (since the block already >> knows its size). This is if we use stl map safely-but-naively. I >> guess one could get stl::map to behave exactly as the Linux rb tree - >> taking KV pair allocation into our own hand and somehow eliminating >> the K storage redundancy. But I would think we end up with a solution >> way less maintainable?than what I have suggested. You'd need intimate >> knowledge of stl map to make sure we this?works, and across all >> platforms and all STL implementations. I'd think maintaining this >> little tree thing is cheaper. > > I agree with this. >> >> ??? blocktree.hpp/cpp should be blockTree.hpp/cpp >> >> ??? Since for class metaspace, the reclaimed InstanceKlass are generally >> ??? bigger than the largest BinList size, maybe class type >> ??? MetaspaceArenas >> ??? should only have a blockTree and not a SmallBlocks, so not the whole >> ??? freeList.hpp structure.? I think we could do some footprint analysis >> ??? with NMT and see if this might help.? It's fairly rare unless you >> ??? have a >> ??? lot of redefinition.? I don't think this should be changed in this >> ??? change though. >> >> >> Oh good point. >> >> I would not like to hard-code the assumption that class space only >> contains Klass though; I dimly remember an RFR with Ioi where he >> tried storing other things there as well. IIRC he found some other >> solution. But my point is that this assumption is fragile. >> >> But a "soft assumption" would be fine. E.g. creating the BinList on >> the fly, only when needed, and separately from the blocktree. Easy to >> implement too. >> >> We don't need NMT to analyze. BinList currently has 32 slots, + >> counter + mask. We pay 272 bytes per structure, or 34 words. Which we >> would save per CLD. > > This seems like something useful to explore later. > >> >> ??? === metaspaceContext.cpp >> >> ??? #define LOGFMT???????? "SpcMgr @" PTR_FORMAT " (%s)" >> ??? #define LOGFMT_ARGS??? p2i(this), this->_name >> >> ??? These don't seem to be used and is SpcMgr a reference to the old >> ??? SpaceManager?? Correction, I see how it's used but you should >> ??? change it >> ??? to not refer to Space manager. >> >> >> I'll fix that. >> >> >> ??? // Destroys the context: deletes chunkmanager and virtualspacelist. >> ??? // If this is a non-expandable context over an existing space, that >> ??? space remains >> ??? // untouched, otherwise all memory is unmapped. >> ??? MetaspaceContext::~MetaspaceContext() { >> ??? ?? delete _cm; >> ??? ?? delete _vslist; >> ??? } >> >> ??? Why would you ever do this?? I thought the two contexts were >> ??? global.? If >> ??? it's for the tests, add a comment here. >> >> >> Yes, it only happens when testing. The rest is future thoughts, in >> case we ever use a metaspace context for other purposes (I am >> currently experimenting with using them as Arena replacement). >> >> I will adapt the comment. >> >> >> ??? === metachunk.cpp >> >> ??? ???? class UnsortedMetachunkList : public AbstractMetachunkList { >> >> ??? This class is declared in a strange place (in a function) and >> appears >> ??? unused. check_pattern() appears unused which is good because it's >> ??? a lot >> ??? of code. >> >> >> My god how did this get there. How did this ever compile. This is an >> accidental paste from some code move I did before posting the review. >> And yes, seems both fill_with_pattern and check_pattern are unused. >> Remnants from some earlier version. I'll remove both. >> >> >> ??? === metaspace_test.hpp/cpp >> >> ??? I keep thinking this file is in the wrong place.? Can you rename >> ??? metaspaceTestHelper? or some name like that? >> >> >> Will do. These classes must live somewhere in metaspace since they >> are used by whitebox.cpp. I could move them there directly. But I >> wanted to keep them separate. >> >> >> ??? === internStats.hpp >> >> ??? Indentation is inconstent and too much vertical white space at the >> ??? end. >> ??? Should be renamed internalStats.hpp/cpp to match the class name. >> ??? This >> ??? looks like a nice improvement. One of the things that made metaspace >> ??? code hard to read was that there was a lot of code for statistics >> ??? mixed >> ??? in everywhere. >> >> >> I agree. >> >> >> ??? === arenaGrowthPolicy.hpp/cpp >> >> ??? I like this! >> >> >> :) >> >> ??? === chunkLevel.hpp/cpp >> >> ??? I'm fine with the nested namespace chunklevel, rather than an >> ??? AllStatic >> ??? class ChunkLevel.? The names of the files should probably be >> ??? changed to >> ??? chunklevel.hpp/cpp though. >> >> >> Ok. >> >> >> ??? === metaspaceEnums.hpp/cpp >> >> ??? This set of files seems like they are misnamed and should be >> ??? encorporated into metaspace.hpp, since they are used outside of the >> ??? memory/metaspace directory. >> >> >> I agree, this seems odd. Will move as suggested. >> >> ??? === settings.hpp >> >> ??? Include guards are wrong names. >> >> ??? #ifndef SHARE_MEMORY_METASPACE_CONSTANTS_HPP >> ??? #define SHARE_MEMORY_METASPACE_CONSTANTS_HPP >> >> ??? #endif // SHARE_MEMORY_METASPACE_BLOCKFREELIST_HPP >> >> ??? I actually took the allocation path when looking at this. Thank >> ??? you for >> ??? keeping many of the function names because that helped. >> >> ??? thanks, >> ??? Coleen >> >> >> Thanks for the review, Coleen. I will fix and post an update. I'll >> have vacation the rest of the week and will post an updated webrev >> start of next week, with your changes and some other small fish I did. > > Hopefully, there will be more comments from others waiting for you > when you get back. > Thanks, > Coleen >> >> Cheers, Thomas > From albert.m.yang at oracle.com Mon Aug 24 17:29:40 2020 From: albert.m.yang at oracle.com (Albert Yang) Date: Mon, 24 Aug 2020 19:29:40 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: <858c2f0b-c6a8-0726-cd97-9641396d8d14@oracle.com> <6dd80f85-325d-f88a-2799-666981efdd00@oracle.com> Message-ID: <37d797ad-8fd3-28ab-ef12-ded31a5c46a7@oracle.com> Hi Thomas, > While this is true, I do not think this is a real problem. It only can happen if we hit MaxMetaspaceSize (so not: GC threshold - there, caller will increase commit quota, and follow up commits on the current chunk will be successful). Hitting MaxMetaspaceSize is very rare; the option, by default, is infinite. We may conceivably run into a pathological situation where we "hover" at that limit, again and again hitting it, the subsequent GCs releasing some loaders, enough to avoid OOM but never quite enough to get away from the limit for good... I don't know. Seems far fetched. I see; thank you for the detailed example. I believe that it makes sense to include something like "some virtual space might be leaked, but we don't think it's a real problem, because ..." in `MetaspaceArena::allocate` to avoid confusing future readers. -- /Albert From ioi.lam at oracle.com Mon Aug 24 22:53:52 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 24 Aug 2020 15:53:52 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> Message-ID: <98b8b0c5-b9b5-8c68-15fc-629164203e3d@oracle.com> On 8/20/20 5:10 PM, Mandy Chung wrote: > > > On 8/19/20 10:14 PM, Yumin Qi wrote: >> >> HI, Mandy >> >> ? Thanks for the review, I took one day off yesterday so just got a >> detail look of your reply. >> >> On 8/19/20 1:30 PM, Mandy Chung wrote: >>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>> Hi, Ioi >>>> >>>> ? Thanks for review/suggestion. I have updated the webrev at the >>>> following link: >>>> >>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>> >>> >>> This patch leverages the TRACE_RESOLVE output and passes the trace >>> output to VM.? VM then calls >>> GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing >>> and generate Holder class per the resolved LFs.?? I think there are >>> other cleaner alternatives implementing this.?? jlink >>> --generate-jli-classes plugin depends the trace output whereas >>> -Xshare:dump does not.?? It's cleaner to skip generating the trace >>> output and parsing for dumping shared archive purpose.? In addition, >>> the implementation needs some cleanup (I can send you feedback on >>> the next revision) >>> >> Current patch did not change the existing code for JLinkPlugin part. >> I just moved the parsing code from GenerateJLIClassesPlugin.java to >> GenerateJLIClassesHelper.java since the former is an internal class >> to which we shouldn't call to generate holder classes. >>> Instead of relying on a system property >>> "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to >>> use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS >>> dump time. >> >> I remember we have such API to query if flag -Xshare:dump or >> -Xshare:on used. Do you mean if DumpLoadedClassList flag set?? This >> flag is the one used to log class name to list file. In ? >> GenerateLinkOptData.gmk: >> >> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) >> $(CLASSLIST_JAR) >> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) >> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, >> $(JLI_TRACE_FILE))) >> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >> -XX:DumpLoadedClassList=$@.raw \ >> ??????????? -Duser.language=en -Duser.country=US \ >> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >> ??????????? -XX:SharedClassListFile=$@.interim >> -XX:SharedArchiveFile=$@.jsa \ >> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >> -XX:DumpLoadedClassList=$@.raw.2 \ >> ??????????? -XX:SharedClassListFile=$@.interim >> -XX:SharedArchiveFile=$@.jsa \ >> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >> ??????????? -Duser.language=en -Duser.country=US \ >> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >> ??????????? build.tools.classlist.HelloClasslist \ >> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >> ??????????? || ( \ >> ??????????????? exitcode=$$? ; \ >> ??????????????? $(ECHO) "ERROR: Failed to generate link optimization >> data." \ >> ??????????????????? "This is likely a problem with the newly built >> JVM/JDK." ; \ >> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >> ??????????????? exit $$exitcode \ >> ??????????? ) >> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >> >> The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList >> and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current >> implementation, DumpLoadedClassList will turn on property >> java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same >> output sent to stdout and log file DumpLoadedClassList specified. >> > > These entries are duplicated in two different files: one for jlink > --generate-jli-classes plugin and another for CDS use.? CDS > -Xshare:dump attempts to do what jlink plugin does but writes those > generated classes in to shared archive. > > Like the above make logic to build JDK image, the same entries are > written in both default-jli-trace.txt via System.out and to classlist > via JNI call to the VM.? I guess VM also implements the logic to do > some kind of diffing and write to CDS archive. > >> Now instead of this property, using a vm interface API to query if >> this flag is set, I think it is better choice. But here I am NOT sure >> I understand your suggestion, I think there are two choices: >> >> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via >> CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set >> >> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting those >> name, type and holder name to regenerate holder classes? >> > > I misunderstood that this CDS_TRACE_RESOLVE flag is set during > -Xshare:dump time. > > Ioi has clarified to me offline that this step is actually part of > -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the > given class list file, i.e. you repurpose the class list file to > include the log output that was initially designed for jlink plugin. > > To me, I'd prefer to see this support depending on `jlink > --generate-jli-classes` which is an existing functionality and much > cleaner.? This way this does not require any VM change.?? It will > generate the holder classes in the custom image per the > application-specific config file. > > What it means is that: a customer would need to create a custom image > with their application-specific config file.? It might need a new CDS? > option to specify a separate TRACE_RESOLVE file.?? It would turn on > this feature by default by defining a default path of the log file if > it helps. > > I understand that this is not the existing CDS work flow and CDS > archive does not require to run on a custom image.?? I see the value > of this approach which can prepare customers to start building and > using its own custom image. > > Of course the implementation would be much simpler (adding a flag to > write these traces to a given file path, that is). > Hi Mandy, jlink already supports everything that's provided by this patch (so why are we doing it?): $ java -cp . -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true TestJLI | grep RESOLVE > jli-trace.txt $ grep fail jli-trace.txt [LF_RESOLVE] java.lang.invoke.DirectMethodHandle$Holder invokeSpecial LLD3_L (fail) [LF_RESOLVE] java.lang.invoke.Invokers$Holder linkToTargetMethod D3L_L (fail) $ jlink --add-modules java.base --generate-jli-classes=@jli-trace.txt --output myjdk $ du -sm myjdk 48??? myjdk $ ./myjdk/bin/java -cp . -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true TestJLI > after.txt $ grep D3 after.txt [LF_RESOLVE] java.lang.invoke.DirectMethodHandle$Holder invokeSpecial LLD3_L (success) [LF_RESOLVE] java.lang.invoke.Invokers$Holder linkToTargetMethod D3L_L (success) However, I suspect most people won't do this, because the benefit is relatively small. Also, jlink doesn't support classpath apps, so you would need to figure out what to use for "--add-modules". In the worst case, the custom JDK would be over 100MB in size. So what we want to accomplish with this patch is to make the "--generate-jli-classes" feature available to people who are already using CDS, so that they don't need to create a custom JDK just to get a few optimized XXX$Holder classes. In fact, CDS users don't need to do anything extra, as the LF_RESOLVE traces are automatically included in the output of -XX:DumpLoadedClassList. Once they upgrade to JDK 16, they will automatically benefit from this patch. I understand that this is a little messy. We need to move some code from the jdk.jlink module into java.base, such as GenerateJLIClassesHelper::generateMethodHandleHolderClasses. These are simple text parsing functions. They are called by hotspot, and hotspot is conceptually part of java.base. If we leave them inside jdk.jlink, then we would have part of java.base depending on jdk.link, which is not good. ======== In the long term, in project Leyden, we will have better integration between jlink, AOT, CDS, etc. Today we use "jaotc" to create AOT-compiled code, and "java -Xshare:dump" to create the CDS archive. I think eventually we will just use "jlink", which will internally drive AOT and CDS. Today jlink create a whole JDK image, but we may want to generate just an executable, something like ??? $ jlink --add-modules myapp ..... --trace-file myapp.trace --output myapp.exe ??? $ ./myapp.exe options ... For space saving, we may not copy all the contents of the JDK into myapp.exe. Instead, some contents may be loaded from $JAVA_HOME. Of course, there will also be a way to create a "stand-alone" myapp.exe that doesn't depend on $JAVA_HOME. (In this new world, we will probably move the GenerateJLIClassesHelper::generateMethodHandleHolderClasses code back to jdk.jlink, as they won't be needed by CDS anymore. Similarly, today we have CDS hooks in the Lambda proxy generation code in java.lang.invoke. I suspect those will also be unnecessary, as the pre-resolution of dynamic bytecodes can probably be done with less intervention from CDS.) ========= As you can probably see, the changes in jlink in Leyden will be substantial, and will take multiple JDK releases to roll out. There are a lot of design that need to happen, and it will be much more complicated that just invoking "jlink" with "--cds". While Leyden is making progress, in the short term, we want to keep making improvements in the existing usage model of CDS. Patches like this one is a little messy, but I think it's still under control, and hopefully we won't have many patches of this kind. Thanks - Ioi > Mandy From ioi.lam at oracle.com Mon Aug 24 23:23:10 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 24 Aug 2020 16:23:10 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> Message-ID: Hi Yumin, This looks good overall. Here are my comments: ===================== 6065???? size_t new_id = Atomic::add(&counter, (size_t)1); 6066???? jio_snprintf(addr_buf, 20, INTPTR_FORMAT, new_id); I think this should be SIZE_FORMAT ===================== ? 65 class KlassFactory : AllStatic { ? 66 ? 67?? // approved clients ? 68?? friend class ClassLoader; ? 69?? friend class ClassLoaderExt; ? 70?? friend class SystemDictionary; ? 71?? friend class LambdaFormInvokers; ? 72 ? 73? private: ? 74?? static InstanceKlass* create_from_stream(ClassFileStream* stream, I think instead of adding everyone who uses create_from_stream as a friend class, we should just change create_from_stream into a public function and remove the friend declarations. ===================== ?146?? // add to hierarchy and set state to loaded. ?147?? { ?148???? MutexLocker mu_r(THREAD, Compile_lock); // add_shared_to_hierarchy asserts this. ?149???? SystemDictionaryShared::add_shared_to_hierarchy(result, THREAD); ?150?? } I think the function name can be changed to SystemDictionaryShared::add_to_hierarchy as the "_shared" seems redundant. The "set state to loaded" comment seems wrong, as we have the assert on line 1155. I think the comment can be removed. 1153 void SystemDictionaryShared::add_shared_to_hierarchy(InstanceKlass* k, TRAPS) { 1154?? Arguments::assert_is_dumping_archive(); 1155?? assert(!k->is_loaded(), "Invariant"); 1156?? assert_locked_or_safepoint(Compile_lock); // add_to_hierarchy assert on it. 1157?? SystemDictionary::add_to_hierarchy(k, CHECK); 1158 } Also, I think it's better to move the MutexLocker call into SystemDictionaryShared::add_shared_to_hierarchy. ======================== before: ?478???????????????? if (TRACE_RESOLVE && salvage != null) { ?479???????????????????? // Used by jlink species pregeneration plugin, see ?480???????????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin ?481???????????????????? System.out.println("[SPECIES_RESOLVE] " + className + " (salvaged)"); ?482???????????????? } after: ?488???????????????? // Used by jlink species pregeneration plugin, see ?489???????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin ?490???????????????? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); When tracing is disabled, this will make extra allocations and cause a small slowdown. I think it's better to ??? if ((TRACE_RESOLVE|TRACE_RESOLVE_CDS) && salvage != null) { ? ? ?? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); ??? } Because TRACE_RESOLVE is a static final boolean, the JIT compiler will completely optimize this block out. For the same reason, instead of calling VM.isDumpLoadedClassListSetAndOpen() every time, it's better to use a static final variable. ======================= ?698???????? if (TRACE_RESOLVE) { ?699???????????? System.out.println("[LF_RESOLVE] " + holder.getName() + " " + name + " " + ?700 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); ?701???????? } ?702???????? if (VM.isDumpLoadedClassListSetAndOpen()) { ?703 GenerateJLIClassesHelper.cdsTraceResolve("[LF_RESOLVE] " + holder.getName() + " " + name + " " + ?704 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); ?705???????? } ?706???????? return resolvedMember; I think the two "if" blocks should be combined similarly to ClassSpecializer::traceResolve(). ========================= ? 34 Java_java_lang_invoke_GenerateJLIClassesHelper_cdsTraceResolve(JNIEnv *env, jclass ignore, jstring line) { Maybe this should be moved to the "VM" class as well? ========================= lambdaFormInvokers.hpp: Need these declarations: ??? #include "memory/allocation.hpp"??? <-- for AllStatic ??? #include "runtime/handles.hpp"????? <-- for typeArrayHandle and Handle ??? #include "utilities/exceptions.hpp" <-- for TRAPS ??? template class GrowableArray; ========================= Thanks - Ioi On 8/23/20 3:56 PM, Yumin Qi wrote: > > Hi, Mandy, Ioi and Calvin > > > ? I have updated the new changed at: > > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-03/ > > ? In this version: > > ??? 1) Added a new API to check if flag DumpLoadedClassList set and > the file is open. If true, call into vm to print out the trace line to > the log file. > > ??????? Just thinking if we just call the cdsTraceResolve without > checking if the flag DumpLoadedClassList set and file open, this way, > the check logic is in the vm side like before, so save code by not > adding the new API. > > ??? 2) The returned holder class names now are just > 'package/className', removed head and tail. > > ??? 3)? Remove add_extra_classes from CollectClassesClosure since > after bug 8250990: https://bugs.openjdk.java.net/browse/JDK-8250990 > pushed, the CollectClassesClosure no longer exist. > > ??? 4)? Still keep the parsing for TRACE_RESOLVE in > java/lang/invoke/GenerateJLIClassesHelper.java, so VM call its > function to regenerate holder classes. > > > ? Re-tested Mach5 tier1-4 > > Thanks > > Yumin > > > On 8/20/20 8:05 PM, Yumin Qi wrote: >> Hi, Mandy >> >> >> On 8/20/20 5:10 PM, Mandy Chung wrote: >>> >>> >>> On 8/19/20 10:14 PM, Yumin Qi wrote: >>>> >>>> HI, Mandy >>>> >>>> ? Thanks for the review, I took one day off yesterday so just got a >>>> detail look of your reply. >>>> >>>> On 8/19/20 1:30 PM, Mandy Chung wrote: >>>>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>>>> Hi, Ioi >>>>>> >>>>>> ? Thanks for review/suggestion. I have updated the webrev at the >>>>>> following link: >>>>>> >>>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>>>> >>>>> >>>>> This patch leverages the TRACE_RESOLVE output and passes the trace >>>>> output to VM.? VM then calls >>>>> GenerateJLIClassesHelper::generateMHHolderClasses to do the >>>>> parsing and generate Holder class per the resolved LFs.?? I think >>>>> there are other cleaner alternatives implementing this.?? jlink >>>>> --generate-jli-classes plugin depends the trace output whereas >>>>> -Xshare:dump does not. It's cleaner to skip generating the trace >>>>> output and parsing for dumping shared archive purpose.? In >>>>> addition, the implementation needs some cleanup (I can send you >>>>> feedback on the next revision) >>>>> >>>> Current patch did not change the existing code for JLinkPlugin >>>> part. I just moved the parsing code from >>>> GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java >>>> since the former is an internal class to which we shouldn't call to >>>> generate holder classes. >>>>> Instead of relying on a system property >>>>> "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to >>>>> use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS >>>>> dump time. >>>> >>>> I remember we have such API to query if flag -Xshare:dump or >>>> -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This >>>> flag is the one used to log class name to list file. In >>>> GenerateLinkOptData.gmk: >>>> >>>> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) >>>> $(CLASSLIST_JAR) >>>> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) >>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, >>>> $(JLI_TRACE_FILE))) >>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >>>> -XX:DumpLoadedClassList=$@.raw \ >>>> ??????????? -Duser.language=en -Duser.country=US \ >>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >>>> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >>>> ??????????? -XX:SharedClassListFile=$@.interim >>>> -XX:SharedArchiveFile=$@.jsa \ >>>> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >>>> -XX:DumpLoadedClassList=$@.raw.2 \ >>>> ??????????? -XX:SharedClassListFile=$@.interim >>>> -XX:SharedArchiveFile=$@.jsa \ >>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >>>> ??????????? -Duser.language=en -Duser.country=US \ >>>> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>> ??????????? build.tools.classlist.HelloClasslist \ >>>> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >>>> ??????????? || ( \ >>>> ??????????????? exitcode=$$? ; \ >>>> ??????????????? $(ECHO) "ERROR: Failed to generate link >>>> optimization data." \ >>>> ??????????????????? "This is likely a problem with the newly built >>>> JVM/JDK." ; \ >>>> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >>>> ??????????????? exit $$exitcode \ >>>> ??????????? ) >>>> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >>>> >>>> The $(JLI_TRACE_FILE) is generated with both >>>> -XX:DumpLoadedClassList and >>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current >>>> implementation, DumpLoadedClassList will turn on property >>>> java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same >>>> output sent to stdout and log file DumpLoadedClassList specified. >>>> >>> >>> These entries are duplicated in two different files: one for jlink >>> --generate-jli-classes plugin and another for CDS use. CDS >>> -Xshare:dump attempts to do what jlink plugin does but writes those >>> generated classes in to shared archive. >>> >>> Like the above make logic to build JDK image, the same entries are >>> written in both default-jli-trace.txt via System.out and to >>> classlist via JNI call to the VM.? I guess VM also implements the >>> logic to do some kind of diffing and write to CDS archive. >>> >> In current implementation, vm side only records the line as from >> TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at dump >> time, call back to java for parsing those recordings and generating >> the holder classes, this uses the existing JLI code. >>>> >>>> Now instead of this property, using a vm interface API to query if >>>> this flag is set, I think it is better choice. But here I am NOT >>>> sure I understand your suggestion, I think there are two choices: >>>> >>>> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via >>>> CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is >>>> set >>>> >>>> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting >>>> those name, type and holder name to regenerate holder classes? >>>> >>> >>> I misunderstood that this CDS_TRACE_RESOLVE flag is set during >>> -Xshare:dump time. >>> >>> Ioi has clarified to me offline that this step is actually part of >>> -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the >>> given class list file, i.e. you repurpose the class list file to >>> include the log output that was initially designed for jlink plugin. >>> >>> To me, I'd prefer to see this support depending on `jlink >>> --generate-jli-classes` which is an existing functionality and much >>> cleaner.? This way this does not require any VM change. It will >>> generate the holder classes in the custom image per the >>> application-specific config file. >>> >>> What it means is that: a customer would need to create a custom >>> image with their application-specific config file.? It might need a >>> new CDS? option to specify a separate TRACE_RESOLVE file.?? It would >>> turn on this feature by default by defining a default path of the >>> log file if it helps. >>> >> So for now, I would implement an API to query if flag >> DumpLoadedClassList set in cmd line, remove new added property of >> java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address >> custom image with CDS in future in a separate issue. >> >> >> Thanks >> >> Yumin >> >>> I understand that this is not the existing CDS work flow and CDS >>> archive does not require to run on a custom image.?? I see the value >>> of this approach which can prepare customers to start building and >>> using its own custom image. >>> >>> Of course the implementation would be much simpler (adding a flag to >>> write these traces to a given file path, that is). >>> >>> Mandy From mandy.chung at oracle.com Tue Aug 25 02:19:38 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 24 Aug 2020 19:19:38 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <98b8b0c5-b9b5-8c68-15fc-629164203e3d@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <98b8b0c5-b9b5-8c68-15fc-629164203e3d@oracle.com> Message-ID: <6e224101-b75d-f825-c3fb-80637d183d07@oracle.com> On 8/24/20 3:53 PM, Ioi Lam wrote: > > : > > However, I suspect most people won't do this, because the benefit is > relatively small. Also, jlink doesn't support classpath apps, so you > would need to figure out what to use for "--add-modules". In the worst > case, the custom JDK would be over 100MB in size. > `jdeps --print-module-deps` can find the static dependencies.? It will need to know its reflective dependencies and any service providers it depends on.? It's no different for creating a custom image for modular apps. Mandy From kim.barrett at oracle.com Tue Aug 25 02:30:30 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 24 Aug 2020 22:30:30 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> <77F6BD2A-DFE1-4676-AD31-55D33C658825@oracle.com> Message-ID: <47D913FE-00BE-41D6-B2F1-0AC79151CBB2@oracle.com> > On Aug 22, 2020, at 2:13 AM, Thomas St?fe wrote: > Good. > > + // Only report the first occurrence of an allocating thread that > + // is missing a ResourceMark, to avoid possible recursive errors > + // in error handling. > > The comment is clear. Should be still clear 40 years from now when we all moved on from git and there is only one guy left who can read git history :) Assuming nobody deletes most of the comment (as happened to the original comment) and that git history is still readable. I?ve lost *so* much digital data due to formats becoming unreadable. > Very minor nit: comment is somewhat imprecise here: > > + case 18: { > + // Allocation from resource area without a ResourceMark. > + Thread::current()->resource_area()->allocate_bytes(100); > + break; > + } > > because this is only true if this function is called from argument handling where no RM has been established yet. Good point. I've tweaked the code and the comment. New webrevs: full: https://cr.openjdk.java.net/~kbarrett/8251850/open.03/ incr: https://cr.openjdk.java.net/~kbarrett/8251850/open.03.inc/ Reran the ErrorHandler.java test locally. > I am fine with this change. I leave if up to you if you change the comment or not. > > Cheers, Thomas Thanks, but since I (lightly) touched the code again? From nick.gasson at arm.com Tue Aug 25 03:21:01 2020 From: nick.gasson at arm.com (Nick Gasson) Date: Tue, 25 Aug 2020 11:21:01 +0800 Subject: RFR: 8252108: Modify nsk/stress/stack tests to check page size Message-ID: <85wo1ne7he.fsf@nicgas01-pc.shanghai.arm.com> Hi, Bug: https://bugs.openjdk.java.net/browse/JDK-8252108 Webrev: http://cr.openjdk.java.net/~ngasson/8252108/webrev.1/ As discussed on the review thread for JDK-8251490 [1], it's better to have these tests enabled conditionally on 4K page size, rather than always disabled on AArch64, since the original issue was only present on AArch64 systems configured with 64K pages. Added a new "vm.pageSize" jtreg property that can be used in @requires. stack018.java intermittently times out with the increased -Xss448K so I've lowered this to -Xss220k and added @requires vm.pageSize == 4096 (the original value -Xss200k fails with "Maximal recursion depth: 0" on AArch64 fastdebug build). [1] https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/041299.html -- Thanks, Nick From thomas.stuefe at gmail.com Tue Aug 25 04:02:30 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 25 Aug 2020 06:02:30 +0200 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <47D913FE-00BE-41D6-B2F1-0AC79151CBB2@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> <77F6BD2A-DFE1-4676-AD31-55D33C658825@oracle.com> <47D913FE-00BE-41D6-B2F1-0AC79151CBB2@oracle.com> Message-ID: On Tue, Aug 25, 2020 at 4:30 AM Kim Barrett wrote: > > On Aug 22, 2020, at 2:13 AM, Thomas St?fe > wrote: > > Good. > > > > + // Only report the first occurrence of an allocating thread that > > + // is missing a ResourceMark, to avoid possible recursive errors > > + // in error handling. > > > > The comment is clear. Should be still clear 40 years from now when we > all moved on from git and there is only one guy left who can read git > history :) > > Assuming nobody deletes most of the comment (as happened to the original > comment) and that > git history is still readable. I?ve lost *so* much digital data due to > formats becoming unreadable. > > > > Very minor nit: comment is somewhat imprecise here: > > > > + case 18: { > > + // Allocation from resource area without a ResourceMark. > > + Thread::current()->resource_area()->allocate_bytes(100); > > + break; > > + } > > > > because this is only true if this function is called from argument > handling where no RM has been established yet. > > Good point. I've tweaked the code and the comment. > > New webrevs: > full: https://cr.openjdk.java.net/~kbarrett/8251850/open.03/ > incr: https://cr.openjdk.java.net/~kbarrett/8251850/open.03.inc/ > > Reran the ErrorHandler.java test locally. > > > I am fine with this change. I leave if up to you if you change the > comment or not. > > > > Cheers, Thomas > > Thanks, but since I (lightly) touched the code again? > All (still) good. Thank you for the clear comments, I wish the original code had been this clear, would have saved us a bit of head scratching years ago. ..Thomas From thomas.stuefe at gmail.com Tue Aug 25 04:03:40 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 25 Aug 2020 06:03:40 +0200 Subject: RFR(S) Remove excessive inclusion of arguments.hpp In-Reply-To: References: <09c2f053-2c61-5b7d-ea83-18c1db3c2b8e@oracle.com> Message-ID: Hi Ioi, thanks for the explanation. Looks all good to me. ..Thomas On Mon, Aug 24, 2020 at 7:16 PM Ioi Lam wrote: > Hi Thomas, thanks for the review. > > Some cpp files use vm_exit_during_initialization(), which is defined in > java.hpp, which was included by arguments.hpp. > > Now that I have removed arguments.hpp (indirectly) from these cpp files, I > have to add java.hpp to fix the missing dependency. > > Thanks > - Ioi > > On 8/23/20 1:31 AM, Thomas St?fe wrote: > > Hi Ioi, > > looks okay, but why the added java.hpp includes? At first glance does not > seem related to your bug description. > > Thanks Thomas > > On Fri, Aug 21, 2020 at 6:47 PM Ioi Lam wrote: > >> https://bugs.openjdk.java.net/browse/JDK-8252151 >> >> http://cr.openjdk.java.net/~iklam/jdk16/8252151-dont-include-arguments-hpp.v01/ >> >> The large header file arguments.hpp is included by popular header files >> such as constMethod.hpp and methodCounters.hpp (which are included by >> method.hpp, which is included by almost every cpp file). >> >> The dependencies on arguments.hpp by constMethod.hpp are very small, and >> can be moved to constMethod.cpp. >> >> methodCounters.hpp does not depend on arguments.hpp >> >> === >> by removing these dependencies, we can reduce the total number of >> include files in the hotspot build from 530,921 to 523,010, or a >> reduction of about 1.5%. >> >> >> > From aleksei.voitylov at bell-sw.com Tue Aug 25 08:21:38 2020 From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov) Date: Tue, 25 Aug 2020 11:21:38 +0300 Subject: [16] RFR 8252250: isnanf is obsolete Message-ID: Hi, in preparation for a review request for Alpine Linux JEP [1] it was suggested [2] to send RFRs which are not specific to musl or busybox for review upstream as separate patches. Bug: https://bugs.openjdk.java.net/browse/JDK-8252250 Fix: http://cr.openjdk.java.net/~avoitylov/webrev.8252250/ ifnanf is obsolete [3], and musl does not define it. Proof of correctness of replacement of isnanf with isnan for all practical versions of gcc and llvm is provided in a comment to [4]. A backport of this change to 14 passes JCK and shows no jtreg regressions on Linux. Thanks, -Aleksei [1] https://openjdk.java.net/jeps/386 [2] https://mail.openjdk.java.net/pipermail/portola-dev/2020-August/000471.html [3] https://man7.org/linux/man-pages/man3/isnanf.3.html [4] https://bugs.openjdk.java.net/browse/JDK-8178689 From matthias.baesken at sap.com Tue Aug 25 11:22:19 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Tue, 25 Aug 2020 11:22:19 +0000 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: <17f9e7d8-d5fa-d4c6-8846-7c57f026cacb@oss.nttdata.com> References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <3ff5ac95-7664-cf03-9c25-05f47a2509a8@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> <08584797-0841-945e-ec6d-7050a12326b4@oracle.com> <0facc9ee-eb47-a40c-4064-75d648b48eab@oss.nttdata.com> <145b716c-54ee-b32a-d008-a4ca3e7359db@oracle.com> <17f9e7d8-d5fa-d4c6-8846-7c57f026cacb@oss.nttdata.com> Message-ID: Hello, the latest webrev looks good to me . I tested additionally on VMWare / Xen / KVM and the detection worked nicely . You might want to rephrase this comment (it sounds a little strange to me) http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.02/src/hotspot/cpu/x86/vm_version_x86.cpp.frames.html 1821 // In case of Xen, it might be appear until 0x40010000. Don't need to see another webrev for this . Best regards, Matthias >On 2020/08/18 23:39, Doerr, Martin wrote: >> Hi Yasumasa, >> >> thanks a lot for measuring and improving the overhead. This version looks much better. >> >> I still wonder what the reason was for using a stub. There was probably a reason, but I don't remember. > >It might be called in many times, so I made change would generate stub in first. > >VM_Version::check_virtualizations() would call CPUID 256 times (0x40000000 to 0x40010000 step 0x100), >and also I changed it is used to call CPUID with EAX = 0x40000007 in Hyper-V. > > >> Hopefully, Matthias can find time for another review when he's back next week. > >Yeah, I want to commit this change after his review. From coleen.phillimore at oracle.com Tue Aug 25 12:11:13 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 25 Aug 2020 08:11:13 -0400 Subject: RFR (S) 8244386: convert runtime/Safepoint/AssertSafepointCheckConsistency tests to gtest Message-ID: <28416485-be67-ea89-02bc-b12b7ba7ad4a@oracle.com> This is a cool feature of gtest.? Tested with tier1 and with product (outside of tier1), since that's a typical failure mode. open webrev at http://cr.openjdk.java.net/~coleenp/2020/8244386.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8244386 Thanks, Coleen From thomas.stuefe at gmail.com Tue Aug 25 12:27:59 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 25 Aug 2020 14:27:59 +0200 Subject: RFR (S) 8244386: convert runtime/Safepoint/AssertSafepointCheckConsistency tests to gtest In-Reply-To: <28416485-be67-ea89-02bc-b12b7ba7ad4a@oracle.com> References: <28416485-be67-ea89-02bc-b12b7ba7ad4a@oracle.com> Message-ID: Looks good to me. Nice to see this used more often. Cheers, Thomas On Tue, Aug 25, 2020 at 2:11 PM Coleen Phillimore < coleen.phillimore at oracle.com> wrote: > This is a cool feature of gtest. Tested with tier1 and with product > (outside of tier1), since that's a typical failure mode. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8244386.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8244386 > > Thanks, > Coleen > From lois.foltan at oracle.com Tue Aug 25 12:30:21 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 25 Aug 2020 08:30:21 -0400 Subject: RFR (S) 8244386: convert runtime/Safepoint/AssertSafepointCheckConsistency tests to gtest In-Reply-To: <28416485-be67-ea89-02bc-b12b7ba7ad4a@oracle.com> References: <28416485-be67-ea89-02bc-b12b7ba7ad4a@oracle.com> Message-ID: <7582caf9-b7fa-a139-7596-82f7782947d8@oracle.com> On 8/25/2020 8:11 AM, Coleen Phillimore wrote: > This is a cool feature of gtest.? Tested with tier1 and with product > (outside of tier1), since that's a typical failure mode. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8244386.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8244386 > > Thanks, > Coleen Looks good.? Where are the AssertSafepointCheckConsistency2.java (false, false) and AssertSafepointCheckConsistency3.java (false, true) covered in the new test? Thanks, Lois From harold.seigel at oracle.com Tue Aug 25 12:31:42 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Tue, 25 Aug 2020 08:31:42 -0400 Subject: RFR: 8252108: Modify nsk/stress/stack tests to check page size In-Reply-To: <85wo1ne7he.fsf@nicgas01-pc.shanghai.arm.com> References: <85wo1ne7he.fsf@nicgas01-pc.shanghai.arm.com> Message-ID: <6ae728e4-74e8-9b68-8e28-3125af83253c@oracle.com> Hi Nick, The changes look good. Thanks for doing this. Harold On 8/24/2020 11:21 PM, Nick Gasson wrote: > Hi, > > Bug: https://bugs.openjdk.java.net/browse/JDK-8252108 > Webrev: http://cr.openjdk.java.net/~ngasson/8252108/webrev.1/ > > As discussed on the review thread for JDK-8251490 [1], it's better to > have these tests enabled conditionally on 4K page size, rather than > always disabled on AArch64, since the original issue was only present on > AArch64 systems configured with 64K pages. > > Added a new "vm.pageSize" jtreg property that can be used in @requires. > > stack018.java intermittently times out with the increased -Xss448K so > I've lowered this to -Xss220k and added @requires vm.pageSize == > 4096 (the original value -Xss200k fails with "Maximal recursion depth: > 0" on AArch64 fastdebug build). > > [1] https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/041299.html > > -- > Thanks, > Nick From kim.barrett at oracle.com Tue Aug 25 12:33:42 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 25 Aug 2020 08:33:42 -0400 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> <77F6BD2A-DFE1-4676-AD31-55D33C658825@oracle.com> <47D913FE-00BE-41D6-B2F1-0AC79151CBB2@oracle.com> Message-ID: > On Aug 25, 2020, at 12:02 AM, Thomas St?fe wrote: > > > > On Tue, Aug 25, 2020 at 4:30 AM Kim Barrett wrote: > > New webrevs: > full: https://cr.openjdk.java.net/~kbarrett/8251850/open.03/ > incr: https://cr.openjdk.java.net/~kbarrett/8251850/open.03.inc/ > > Reran the ErrorHandler.java test locally. > > > I am fine with this change. I leave if up to you if you change the comment or not. > > > > Cheers, Thomas > > Thanks, but since I (lightly) touched the code again? > > All (still) good. Thank you for the clear comments, I wish the original code had been this clear, would have saved us a bit of head scratching years ago. > > ..Thomas > Thanks. From thomas.stuefe at gmail.com Tue Aug 25 12:36:54 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Tue, 25 Aug 2020 14:36:54 +0200 Subject: RFR: 8252108: Modify nsk/stress/stack tests to check page size In-Reply-To: <85wo1ne7he.fsf@nicgas01-pc.shanghai.arm.com> References: <85wo1ne7he.fsf@nicgas01-pc.shanghai.arm.com> Message-ID: Hi Nick, looks good, useful for ppc64 too. Thanks, Thomas On Tue, Aug 25, 2020 at 5:21 AM Nick Gasson wrote: > Hi, > > Bug: https://bugs.openjdk.java.net/browse/JDK-8252108 > Webrev: http://cr.openjdk.java.net/~ngasson/8252108/webrev.1/ > > As discussed on the review thread for JDK-8251490 [1], it's better to > have these tests enabled conditionally on 4K page size, rather than > always disabled on AArch64, since the original issue was only present on > AArch64 systems configured with 64K pages. > > Added a new "vm.pageSize" jtreg property that can be used in @requires. > > stack018.java intermittently times out with the increased -Xss448K so > I've lowered this to -Xss220k and added @requires vm.pageSize == > 4096 (the original value -Xss200k fails with "Maximal recursion depth: > 0" on AArch64 fastdebug build). > > [1] > https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/041299.html > > -- > Thanks, > Nick > From coleen.phillimore at oracle.com Tue Aug 25 12:51:08 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 25 Aug 2020 08:51:08 -0400 Subject: RFR (S) 8244386: convert runtime/Safepoint/AssertSafepointCheckConsistency tests to gtest In-Reply-To: References: <28416485-be67-ea89-02bc-b12b7ba7ad4a@oracle.com> Message-ID: Thanks Thomas! Coleen On 8/25/20 8:27 AM, Thomas St?fe wrote: > Looks good to me. Nice to see this used more often. > > Cheers, Thomas > > On Tue, Aug 25, 2020 at 2:11 PM Coleen Phillimore > > > wrote: > > This is a cool feature of gtest.? Tested with tier1 and with product > (outside of tier1), since that's a typical failure mode. > > open webrev at > http://cr.openjdk.java.net/~coleenp/2020/8244386.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8244386 > > Thanks, > Coleen > From coleen.phillimore at oracle.com Tue Aug 25 12:52:39 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 25 Aug 2020 08:52:39 -0400 Subject: RFR (S) 8244386: convert runtime/Safepoint/AssertSafepointCheckConsistency tests to gtest In-Reply-To: <7582caf9-b7fa-a139-7596-82f7782947d8@oracle.com> References: <28416485-be67-ea89-02bc-b12b7ba7ad4a@oracle.com> <7582caf9-b7fa-a139-7596-82f7782947d8@oracle.com> Message-ID: <6a5342a8-3308-d267-0194-081bcfbcf298@oracle.com> On 8/25/20 8:30 AM, Lois Foltan wrote: > On 8/25/2020 8:11 AM, Coleen Phillimore wrote: >> This is a cool feature of gtest.? Tested with tier1 and with product >> (outside of tier1), since that's a typical failure mode. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8244386.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8244386 >> >> Thanks, >> Coleen > > Looks good.? Where are the AssertSafepointCheckConsistency2.java > (false, false) and AssertSafepointCheckConsistency3.java (false, true) > covered in the new test? Thanks Lois.? I decided that the test only needs to verify that we check for the erroneous case.? The cases that pass are in the code. Coleen > > Thanks, > Lois From lois.foltan at oracle.com Tue Aug 25 13:17:12 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 25 Aug 2020 09:17:12 -0400 Subject: RFR (S) 8244386: convert runtime/Safepoint/AssertSafepointCheckConsistency tests to gtest In-Reply-To: <6a5342a8-3308-d267-0194-081bcfbcf298@oracle.com> References: <28416485-be67-ea89-02bc-b12b7ba7ad4a@oracle.com> <7582caf9-b7fa-a139-7596-82f7782947d8@oracle.com> <6a5342a8-3308-d267-0194-081bcfbcf298@oracle.com> Message-ID: On 8/25/2020 8:52 AM, Coleen Phillimore wrote: > > > On 8/25/20 8:30 AM, Lois Foltan wrote: >> On 8/25/2020 8:11 AM, Coleen Phillimore wrote: >>> This is a cool feature of gtest.? Tested with tier1 and with product >>> (outside of tier1), since that's a typical failure mode. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2020/8244386.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8244386 >>> >>> Thanks, >>> Coleen >> >> Looks good.? Where are the AssertSafepointCheckConsistency2.java >> (false, false) and AssertSafepointCheckConsistency3.java (false, >> true) covered in the new test? > > Thanks Lois.? I decided that the test only needs to verify that we > check for the erroneous case.? The cases that pass are in the code. Okay, I'm good with your change! Lois > > Coleen > >> >> Thanks, >> Lois > From coleen.phillimore at oracle.com Tue Aug 25 13:19:15 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 25 Aug 2020 09:19:15 -0400 Subject: RFR (S) 8244386: convert runtime/Safepoint/AssertSafepointCheckConsistency tests to gtest In-Reply-To: References: <28416485-be67-ea89-02bc-b12b7ba7ad4a@oracle.com> <7582caf9-b7fa-a139-7596-82f7782947d8@oracle.com> <6a5342a8-3308-d267-0194-081bcfbcf298@oracle.com> Message-ID: Thanks Lois! Coleen On 8/25/20 9:17 AM, Lois Foltan wrote: > On 8/25/2020 8:52 AM, Coleen Phillimore wrote: >> >> >> On 8/25/20 8:30 AM, Lois Foltan wrote: >>> On 8/25/2020 8:11 AM, Coleen Phillimore wrote: >>>> This is a cool feature of gtest. Tested with tier1 and with product >>>> (outside of tier1), since that's a typical failure mode. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/2020/8244386.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8244386 >>>> >>>> Thanks, >>>> Coleen >>> >>> Looks good.? Where are the AssertSafepointCheckConsistency2.java >>> (false, false) and AssertSafepointCheckConsistency3.java (false, >>> true) covered in the new test? >> >> Thanks Lois.? I decided that the test only needs to verify that we >> check for the erroneous case.? The cases that pass are in the code. > > Okay, I'm good with your change! > Lois > >> >> Coleen >> >>> >>> Thanks, >>> Lois >> > From daniel.daugherty at oracle.com Tue Aug 25 13:24:31 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Tue, 25 Aug 2020 09:24:31 -0400 Subject: [16] RFR 8252250: isnanf is obsolete In-Reply-To: References: Message-ID: On 8/25/20 4:21 AM, Aleksei Voitylov wrote: > Hi, > > in preparation for a review request for Alpine Linux JEP [1] it was > suggested [2] to send RFRs which are not specific to musl or busybox for > review upstream as separate patches. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8252250 > Fix: http://cr.openjdk.java.net/~avoitylov/webrev.8252250/ src/hotspot/share/utilities/globalDefinitions_gcc.hpp ??? No comments. Thanks for including the details in the bug report. Thumbs up. This is a trivial change and does not need to wait for 24 hours. Dan > > ifnanf is obsolete [3], and musl does not define it. Proof of > correctness of replacement of isnanf with isnan for all practical > versions of gcc and llvm is provided in a comment to [4]. A backport of > this change to 14 passes JCK and shows no jtreg regressions on Linux. > > Thanks, > -Aleksei > > > [1] https://openjdk.java.net/jeps/386 > [2] > https://mail.openjdk.java.net/pipermail/portola-dev/2020-August/000471.html > [3] https://man7.org/linux/man-pages/man3/isnanf.3.html > [4] https://bugs.openjdk.java.net/browse/JDK-8178689 > > From leo.korinth at oracle.com Tue Aug 25 14:28:49 2020 From: leo.korinth at oracle.com (Leo Korinth) Date: Tue, 25 Aug 2020 16:28:49 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: Message-ID: <4f7a8afb-4207-f92a-14ea-8e212ea239db@oracle.com> Hi, review of: freeBlocks.?pp, blockTree.?pp and binList.hpp below I have some comments in addition to those from Coleen: ### freeBlocks.hpp: 75 76 static const size_t splinter_threshold = 0;// 0x100; 77 splinter_threshold with the strange comment // 0X100, what is it supposed to be used for? (always 0, but with a comment 0x100;). Maybe related to this, is that I have problems to understand how allocation guards are handled when memory is put back to the freelist, I can not see the pointer being decremented (the inverse operation of payload_from_prefix, prefix_from_payload seems never to be used, and I fail to find the operation done manually either. 68 typedef BinList32 SmallBlocksType; 69 70 // _small_blocks takes care of small to very small blocks. 71 SmallBlocksType _small_blocks; I think all of BinListXX typedefs can be removed and also the SmallBlocksType typedef. The SmallBlocksType is only used once if we rewrite const static size_t minimal_word_size = SmallBlocksType::minimal_word_size; to const static size_t minimal_word_size = _small_blocks.minimal_word_size; (and also change the name of minimal_word_size so that we can see it is a constant) 84 85 // Retrieve a block of at least requested_word_size. 86 MetaWord* get_block(size_t requested_word_size); 87 The function named get_block, this is not a getter, this mutates the object. I would prefer remove_block (goes well with add_block), and I would like that rename to propagate to both blockTree and binList. ### binList.hpp: Maybe another name for binList, binLookup for example? When I hear list I am thinking /linked/ list and that might just be my problem, sorry if that is that case, and then ignore this point. 76 77 template 78 class BinListImpl { 79 Template argument smallest_size would be nice if the unit of the size (words) is mentioned in the name. It is so easy to mix sizes up. 82 // a mask to speed up searching for populated lists. 83 // 0 marks an empty list, 1 for a non-empty one. 84 typedef uint32_t mask_t; Is a mask really speeding up anything? Seems the code would be a bit simpler removing the mask. 158 // Given a word_size, searches and returns a block of at least that size. 159 // Block may be larger. Real block size is returned in *p_real_word_size. 160 MetaWord* get_block(size_t word_size, size_t* p_real_word_size) { get_block -> remove_block 217 typedef BinListImpl<2, 8> BinList8; 218 typedef BinListImpl<2, 16> BinList16; 219 typedef BinListImpl<2, 32> BinList32; 220 typedef BinListImpl<2, 64> BinList64; BinListXX typedefs could be removed I think. (see comment above) ### blocktree.hpp 79 struct node_t { 80 81 // Normal tree node stuff... 82 node_t* parent; 83 node_t* left; 84 node_t* right; 85 I think in the future we could make the node_t without a parent pointer, that would make the node size somewhat smaller and we could then be able to fit smaller memory areas into the tree. The code should be somewhat simpler as well I think. Maybe file a future improvement for this? 98 // Largest node size, (bit arbitrarily) capped at 4M since we know this to 99 // be the max possible metaspace allocation size. TODO. Do this better. 100 const static size_t maximal_word_size = 4 * M; maximal_word_size should take its value from MAX_CHUNK_BYTE_SIZE in some way if it is not too hard. 102 // We need nodes to be at least large enough to hold a node_t 103 const static size_t minimal_word_size = 104 (sizeof(node_t) + sizeof(MetaWord) - 1) / sizeof(MetaWord); maybe add a division-ceil comment? 172 // Given a node n, return its predecessor in the tree 173 // (node with the next-smaller size). 174 static node_t* successor(node_t* n) { The comment is wrong on successor (I guess it is copied from predecessor). 199 if (parent != NULL) { 200 if (parent->left == child) { // I am a left child I think "// child is a left child" is better 215 // Given a node n and a node forebear, insert n under forebear 216 void insert(node_t* forebear, node_t* n) { The name forebear confused me, it is just a root for insertion, and has no existing ancestor relation to "n". 230 if (_largest_size_added < n->size) { 231 _largest_size_added = n->size; 232 } This is unnecessary as it is already done by add_block(MetaWord* p, size_t word_size). I also think that add_block is the better place to have the update. The method maybe could be static after this change? I also think the opposite decrease update should be done in remove_block to keep the symmetry. There are at least two recursive calls of insert that could be tail-called instead (it would be somewhat harder to read, so I am not proposing it). However, with an unbalanced tree I guess we could get deep call stacks if we are unlucky. I guess it is best to leave this as is, just wanted to comment on it. 278 // Given a node n, remove it from the tree and repair tree. 279 void remove_node_from_tree(node_t* n) { 280 281 assert(n->next == NULL, "do not delete a node which has a non-empty list"); Is there any reason for not calling the remove_from_list logic here instead of in get_block? We would get rid of this assert, and I think it would be better placed logically. 283 // Maintain largest size node to speed up lookup 284 if (n->size == _largest_size_added) { 285 node_t* pred = predecessor(n); 286 if (pred != NULL) { 287 _largest_size_added = pred->size; 288 } else { 289 _largest_size_added = 0; 290 } 291 } As I said above, the _largest_size_added logic seems (to me) to fit better in remove_block(). It is not really a tree operation but a cached result. I also wonder if the caching is really needed. If we have a huge tree, /and/ we fail to find a free area /repeatedly/, the caching will perform better. But If the tree is empty, it will be as fast to check anyway, and I guess that is the common case). Every time we remove an element from the tree, the predecessor() function needs to be called at the end (just) to update the cached value, so each remove (that successfully fetch free memory) is a bit more expensive with caching. Just a thought. 379 node_t* n = (node_t*)p; 380 n->size = word_size; 381 n->next = n->left = n->right = n->parent = NULL; Placement new would look better in my opinion. Thanks, Leo From suenaga at oss.nttdata.com Tue Aug 25 14:50:25 2020 From: suenaga at oss.nttdata.com (Yasumasa Suenaga) Date: Tue, 25 Aug 2020 23:50:25 +0900 Subject: PING: RFR: 8250598: Hyper-V is detected in spite of running on host OS In-Reply-To: References: <3a1a5ff7-308d-c003-fc34-14b276ecaf43@oss.nttdata.com> <416db93b-6d12-266f-9210-5f92a4840e6c@oracle.com> <0f535934-731b-0989-3bfa-51c332902399@oss.nttdata.com> <97d20ede-53ba-86f8-674b-4ec7e244b12c@oracle.com> <08584797-0841-945e-ec6d-7050a12326b4@oracle.com> <0facc9ee-eb47-a40c-4064-75d648b48eab@oss.nttdata.com> <145b716c-54ee-b32a-d008-a4ca3e7359db@oracle.com> <17f9e7d8-d5fa-d4c6-8846-7c57f026cacb@oss.nttdata.com> Message-ID: <88245e61-c9b4-d9c8-ed74-a6b8a5daabc4@oss.nttdata.com> Hi Matthias, Thanks for reviewing and testing this change! On 2020/08/25 20:22, Baesken, Matthias wrote: > Hello, > the latest webrev looks good to me . > I tested additionally on VMWare / Xen / KVM and the detection worked nicely . > > You might want to rephrase this comment (it sounds a little strange to me) > > http://cr.openjdk.java.net/~ysuenaga/JDK-8250598/webrev.02/src/hotspot/cpu/x86/vm_version_x86.cpp.frames.html > > 1821 // In case of Xen, it might be appear until 0x40010000. Ok, I will set following comment here: "Xen cpuid leaves can be found 0x100 aligned boundary starting from 0x40000000 until 0x40010000." > Don't need to see another webrev for this . I will push the change tomorrow. Thanks, Yasumasa > Best regards, Matthias > > > >> On 2020/08/18 23:39, Doerr, Martin wrote: >>> Hi Yasumasa, >>> >>> thanks a lot for measuring and improving the overhead. This version looks much better. >>> >>> I still wonder what the reason was for using a stub. There was probably a reason, but I don't remember. >> >> It might be called in many times, so I made change would generate stub in first. >> >> VM_Version::check_virtualizations() would call CPUID 256 times (0x40000000 to 0x40010000 step 0x100), >> and also I changed it is used to call CPUID with EAX = 0x40000007 in Hyper-V. >> >> >>> Hopefully, Matthias can find time for another review when he's back next week. >> >> Yeah, I want to commit this change after his review. > > > From lois.foltan at oracle.com Tue Aug 25 14:58:50 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 25 Aug 2020 10:58:50 -0400 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: References: <9e6b0043-65a1-dd97-a3d2-33679c8048d4@oracle.com> Message-ID: <234856fa-1eb1-8514-6786-bce6689afd16@oracle.com> Hi Ioi, Changes looks really good.? Comments interspersed below. Thanks, Lois On 8/12/2020 6:06 PM, Ioi Lam wrote: > Hi Lois, > > Thanks for the comments. I have an updated webrev > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/ > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02.delta/ > > > Here are the general notes on the changes. Replies to your questions > are in-line below. > > (1) Integrated updates in the Java code from Alan Bateman. Here are > Alan's > ? ? notes: > > ??? The archive of the boot layer is as before, except that archiving is > ??? skipped if there are split packages or incubator modules. Incubating > ??? modules aren't resolved by default so we shouldn't have them in the > ??? boot layer by default anyway. > > ??? I've dropped the module finders from the boot layer archive. I've > ??? left the IllegalAccessLogger.Builder in the acrhive for now (even > ??? though it is not the boot layer). We should be able to remove that > ??? once the JEP to disallow illegal access by default is in. > > ??? Related is that I don't like the archive for the module graph > ??? (ArchivedModuleGraph, pre-dates this RFE) including the set of > ??? packages to export/open for illegal access as they aren't part > ??? of the module graph. I've left it for now but we can also remove > ??? that once we disallow illegal access by default (as those sets > ??? will be empty). > > ??? The archive of built-in class loaders is now in one object > ??? jdk.internal.loader.ArchivedClassLoaders which I think will be a > ??? bit more maintainable. I've also drop the ucp field from the > ??? AppClassLoader as the changes to BuiltinClassLoader means is no > ??? longer needs to duplicated. > > ??? There's one remaining issue in ModuleBootstrap.boot where it has fix > ??? an app class loader value (ModuleLayer.CLV). Ideally the > initialization > ??? of the built-in class loaders would do this but we are kinda forced > ??? to separate the archiving of the built-in class loaders from the boot > ??? layer. I might look at this again some time. > > > (2) Moved code from classLoaderData.cpp -> classLoaderSharedData.cpp > > (3) Reverted unnecessary changes in JavaClasses::compute_offset > > (4) Minor clean up to use QuickSort::sort() instead of qsort() > > (5) Moved the C-side of module initialization for platform/system > ? ? loaders to inside java.lang.System::initPhase2(), so this happens > ??? at the same time as without full module archiving. > > (6) Moved the use of Module_lock to so all modules in a class loader > ??? are restored atomically. See ArchivedClassLoaderData::restore() > > ??? This fixed a bug where test/jdk/com/sun/jdi/ModulesTest.java > ? ? would fail as it sees a partially restored set of modules. > > > > On 8/7/20 12:06 PM, Lois Foltan wrote: >> Hi Ioi, >> >> Overall looks promising.? I have some review comments below, but not >> a complete review.? I concentrated on the JVM side primarily how the >> archived module graph is read in, how the ModuleEntry and >> PackageEntry tables are created from the archive for the 3 builtin >> loaders and potential timing issues during start up. >> >> On 7/22/2020 3:36 PM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8244778 >>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/ >>> >>> >>> Please review this patch that stores the full module graph in the CDS >>> archive heap. This reduces the initialization time of the basic JVM by >>> about 22%: >>> >>> $ perf stat -r 100 bin/java -version >>> before: 98,219,329 instructions 0.03971 secs elapsed (+- 0.44%) >>> after:? 55,835,815 instructions 0.03109 secs elapsed (+- 0.65%) >>> >>> [1] Start with ModuleBootstrap.java. The current implementation is >>> ??? quite restrictive: the archived module graph is used only when no >>> ??? module options are specified. >>> >>> ??? See ModuleBootstrap.mayUseArchivedBootLayer(). >>> >>> ??? We can probably support options such as main module and module path >>> ??? in a future RFE. >> >> Yes, I noticed the restrictions.? The JBS issue discusses the >> possibility of using the dumped module graph in the event that the >> value of the options are the same.? I think for this implementation >> to be viable and have a positive impact you should consider comparing >> at least the options --add-modules, --add-exports and --add-reads. > > I agree. I want to keep the changes minimal in this RFE, and will add > more support for other use cases in follow-on RFEs. Instead of > requiring these options to be unspecified, we can relax the > restriction such that these options must be the same between archive > dump time and run time. Sounds good! > >> >> >>> >>> [2] In the current JDK implementation, there is no single object >>> ??? that represents "the module graph". Most of the information >>> ??? is stored in the archive bootLayer object, but a few additional >>> ??? restoration operations need to be performed: >>> >>> ??? + See ModuleBootstrap.getArchivedBootLayer() >>> ??? + Some static fields need to be archived/restored in >>> ????? Module.java, BuiltinClassLoader.java, ClassLoaders.java >>> ????? and BootLoader.java >>> >>> [3] I ran into a complication with two loader instances of >>> ??? PlatformClassLoader and AppClassLoader. They are stored in >>> ??? multiple tables inside the module graph (e.g., >>> ??? BuiltinClassLoader$LoadedModule) so I cannot easily recreate >>> ??? them at runtime. >>> >>> ??? However, these two loaders contain information specific to the >>> ??? dump time VM lifecycle (such as the classes that were loaded >>> ??? during CDS dumping) that need to be scrubbed. I couldn't find an >>> ??? elegant way of doing this, so I added a private >>> "resetArchivedStates" >>> ??? method to a few classes. They are called inside >>> ??? HeapShared::reset_archived_object_states(). >>> >>> [4] Related native data structures (PackageEntry and ModuleEntry) >>> ??? are also archived. Start with classLoaderData.cpp >>> >>> Passes mach5 tiers 1-4. I will test with additional tiers. >>> >>> Thanks >>> - Ioi >> >> classfile/classLoader.cpp >> - line #1644 pulling the javabase_moduleEntry() check outside of the >> Module_lock maybe problematic if after you check it another >> ? thread initializes in the time between the check and the obtaining >> of the Module_lock on line #1645. > > Fixed. That looks good.? I think it is fine that you are checking if java.base is defined via a call to javabase_moduleEntry() because you are just trying to determine if a ModuleEntry should be created or not.? In most cases though using ModuleEntryTable::javabase_defined() is the correct way to ensure that both the ModuleEntry for java.base has been created and that the ModuleEntry has been injected in the corresponding java.lang.Module oop. > >> >> classfile/classLoader.hpp >> - somewhere in ArchivedClassLoaderData there should be an assert to >> make sure that the CLD, whose package entries and module entries are >> being archived is not a "_has_class_mirror_holder" CLD for a weakly >> defined hidden class.? Those dedicated CLDs should never have package >> entries or module entries. >> > > I added ArchivedClassLoaderData::assert_valid() > >> classfile/moduleEntry.cpp >> - line #400, typo "conver" --> "convert" >> - line #500, maybe sort if n is greater than 1 instead of 0 (same >> comment for packageEntry.cpp line #270) >> > Fixed > >> classfile/systemDictionary.cpp >> - It looks like the PackageEntry and ModuleEntry tables for the >> system and platform class loaders are? added within >> SystemDictionary::compute_java_loaders() which is called from >> Thread::create_vm() but after the call in Thread::create_vm() to >> call_initPhase2 which is when Universe::_module_initialized is set to >> true.? Did I read this correctly?? If yes, then I think you have to >> be sure that Universe::_module_initialized is not set until the >> module graph for the 3 builtin loaders is loaded from the archive. >> > > I moved the code to be called by ModuleBootstrap::boot() so it should > now happen inside call_initPhase2. Yes, that looks good. > >> memory/filemap.cpp >> - line #237 typo "modue" --> "module" >> > > Fixed > >> - Since the CDS support for capturing the module graph does not >> archive unnamed modules, I assume >> Modules::set_bootloader_unnamed_module() is still called.? Is this >> true or does the unnamed module for the boot loader get archived? > The unnamed module for the boot loader is not archived. > > Modules::set_bootloader_unnamed_module()? wasn't called in my last > webrev. Thanks for catching it. > > I added a call to BootLoader.getUnnamedModule() in > ModuleBootstrap::boot()? to trigger of BootLoader, which will > call into the VM for Modules::set_bootloader_unnamed_module(). Looks good. > > > >> Clarification/timing questions: >> > > Here's an overall problem I am facing: > > The module graph is archived after the module system has fully started > up. This means that for the boot loader, we only know the full set of > modules/packages, but we don't know which part is the subset that was > initialized during early VM bootstraping (e.g., when > ModuleEntryTable::javabase_defined() == false). > > So the behavior is a bit different during the early bootstrap phase > (all the way before we reach ModuleBootstrap::boot()): > ClassLoaderData::the_null_class_loader_data()->modules() and > ClassLoaderData::the_null_class_loader_data()->packages() are already > populated before a single class is loaded. If this is the case then, at the point when a ModuleEntry is created for java.base using the archived module graph, there should be a assertion that java_lang_Class::_fixup_module_field_list is NULL. To confirm no class has been loaded before java.base is defined. Maybe in ClassLoaderDataShared::serialize where you restore the boot loader's archived modules? > This difference doesn't seem to make a practical difference. E.g., > none of our code seems to assume that "before any classes in the > java.util package is loaded, > ClassLoaderData::the_null_class_loader_data()->packages() must not > contain an entry for java.util". > > I think we have two choices when the archived module graph is used: > > [1] We require that the state of the module system must, at every step > of VM initialization, be identical to that of a VM that doesn't use an > archived module graph. > > [2] We make sure that the VM/JDK bootstrap code can tolerate the > existence of module/packages that are added earlier than a VM that > doesn't use an archived module graph. > > I tried doing a version of [1] and found that to be too difficult. [2] > seems much simpler and is the approach I am using now. I think [2] is reasonable. > >> oops/instanceKlass.cpp >> - line #2545, comment "TO DO? -- point it to the archived >> PackageEntry (JDK-8249262)" >> are you thinking that since the module graph is read in ahead of time >> that it can be set when an InstanceKlass is created?? There is a >> point before java.base is defined that InstanceKlass::set_package >> takes into account that could be a timing issue. >> >> > > I think it will work as if another class in the same package has > already been defined. > >> - There are some checks in modules.cpp that are valuable to have >> during bootstrapping.? For example, packages that are encountered >> during bootstrapping before java.base is defined are put into the >> java.base module but then verified after java.base is defined via >> verify_javabase_packages.? So at what point in the bootstrapping >> process does the boot loader's archived module's become known? Could >> classes have been defined to the VM before this point? Then their >> packages will have to be verified to be sure that they are indeed >> packages within java.base.? Consider looking at other checks that >> occur within classfile/modules.cpp as well. >> > As mentioned above, calling verify_javabase_packages() at run time > will fail, as we have loaded all packages for the boot loader, not > just those for java.base. I think not calling verify_javabase_packages works because as you stated above no classes have been loaded before java.base is defined which is not the same situation as running without the archived module graph. A couple of additional comments: - ModuleEntry's reads list and PackageEntry's exports list.? We had hoped eventually to change these lists from being a c heap allocated GrowableArray over to a ResourceHashTable for faster lookup.? Doing that as a separate RFE might help CDS archiving not to have to archive those lists as an Array? - moduleEntry.cpp and packageEntry.cpp: Both methods "compare_module_by_name" and "compare_package_by_name" should have an assert if the names are equal.? No ModuleEntryTable or PackageEntryTable should ever have 2 same named modules or packages. - Another timing clarification question for me.? I assume that the module graph is dumped post module initialization when Java has completely defined the module graph to the JVM, is this correct?? My concern is that there could be a window post module initialization and pre archiving the module graph where another thread could define a new module or add Module readability or add Package exportability.? So that the module graph you are dumping is not the same module graph at the end of module initialization.? Is this a concern? Thanks, Lois > > Well, verify_javabase_packages() was called once and it succeeded, but > that was during CDS dump time. So we know at least we have verified > this once :-) > > Thanks > - Ioi > >> I may have more review comments as I continue to look at this! >> >> Thanks, >> Lois >> >> >> >> >> > From yumin.qi at oracle.com Tue Aug 25 18:59:47 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 25 Aug 2020 11:59:47 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> Message-ID: <1ca66b6a-8a23-c4b1-c95d-10fc23b668f8@oracle.com> HI, Ioi ? Thanks for the re-review, updated webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/ ? Changed according to your suggestion, the only a little difference is move traceResolve(String line) to InvokerBytecodeGenerator as a static package public function so other package classes can call it. ? re-tested local on runtime/cds Thanks Yumin On 8/24/20 4:23 PM, Ioi Lam wrote: > Hi Yumin, > > This looks good overall. Here are my comments: > > ===================== > 6065???? size_t new_id = Atomic::add(&counter, (size_t)1); > 6066???? jio_snprintf(addr_buf, 20, INTPTR_FORMAT, new_id); > > I think this should be SIZE_FORMAT > > ===================== > ? 65 class KlassFactory : AllStatic { > ? 66 > ? 67?? // approved clients > ? 68?? friend class ClassLoader; > ? 69?? friend class ClassLoaderExt; > ? 70?? friend class SystemDictionary; > ? 71?? friend class LambdaFormInvokers; > ? 72 > ? 73? private: > ? 74?? static InstanceKlass* create_from_stream(ClassFileStream* stream, > > I think instead of adding everyone who uses create_from_stream as a friend class, we should just change create_from_stream into a public function and remove the friend declarations. > > ===================== > ?146?? // add to hierarchy and set state to loaded. > ?147?? { > ?148???? MutexLocker mu_r(THREAD, Compile_lock); // add_shared_to_hierarchy asserts this. > ?149???? SystemDictionaryShared::add_shared_to_hierarchy(result, THREAD); > ?150?? } > > I think the function name can be changed to SystemDictionaryShared::add_to_hierarchy as the "_shared" seems redundant. The "set state to loaded" comment seems wrong, as we have the assert on line 1155. I think the comment can be removed. > > 1153 void SystemDictionaryShared::add_shared_to_hierarchy(InstanceKlass* k, TRAPS) { > 1154?? Arguments::assert_is_dumping_archive(); > 1155?? assert(!k->is_loaded(), "Invariant"); > 1156?? assert_locked_or_safepoint(Compile_lock); // add_to_hierarchy assert on it. > 1157?? SystemDictionary::add_to_hierarchy(k, CHECK); > 1158 } > > Also, I think it's better to move the MutexLocker call into SystemDictionaryShared::add_shared_to_hierarchy. > > ======================== > > before: > > ?478???????????????? if (TRACE_RESOLVE && salvage != null) { > ?479???????????????????? // Used by jlink species pregeneration plugin, see > ?480???????????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin > ?481???????????????????? System.out.println("[SPECIES_RESOLVE] " + className + " (salvaged)"); > ?482???????????????? } > > after: > > ?488???????????????? // Used by jlink species pregeneration plugin, see > ?489???????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin > ?490???????????????? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); > > > When tracing is disabled, this will make extra allocations and cause a small slowdown. I think it's better to > > ??? if ((TRACE_RESOLVE|TRACE_RESOLVE_CDS) && salvage != null) { > ? ? ?? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); > ??? } > > Because TRACE_RESOLVE is a static final boolean, the JIT compiler will completely optimize this block out. > > For the same reason, instead of calling VM.isDumpLoadedClassListSetAndOpen() every time, it's better to use a static final variable. > > > ======================= > > ?698???????? if (TRACE_RESOLVE) { > ?699???????????? System.out.println("[LF_RESOLVE] " + holder.getName() + " " + name + " " + > ?700 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); > ?701???????? } > ?702???????? if (VM.isDumpLoadedClassListSetAndOpen()) { > ?703 GenerateJLIClassesHelper.cdsTraceResolve("[LF_RESOLVE] " + holder.getName() + " " + name + " " + > ?704 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); > ?705???????? } > ?706???????? return resolvedMember; > > > I think the two "if" blocks should be combined similarly to ClassSpecializer::traceResolve(). > > ========================= > ? 34 Java_java_lang_invoke_GenerateJLIClassesHelper_cdsTraceResolve(JNIEnv *env, jclass ignore, jstring line) { > > Maybe this should be moved to the "VM" class as well? > > ========================= > lambdaFormInvokers.hpp: > > Need these declarations: > > ??? #include "memory/allocation.hpp"??? <-- for AllStatic > ??? #include "runtime/handles.hpp"????? <-- for typeArrayHandle and Handle > ??? #include "utilities/exceptions.hpp" <-- for TRAPS > > ??? template class GrowableArray; > ========================= > > > Thanks > - Ioi > > > > On 8/23/20 3:56 PM, Yumin Qi wrote: >> >> Hi, Mandy, Ioi and Calvin >> >> >> ? I have updated the new changed at: >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-03/ >> >> ? In this version: >> >> ??? 1) Added a new API to check if flag DumpLoadedClassList set and the file is open. If true, call into vm to print out the trace line to the log file. >> >> ??????? Just thinking if we just call the cdsTraceResolve without checking if the flag DumpLoadedClassList set and file open, this way, the check logic is in the vm side like before, so save code by not adding the new API. >> >> ??? 2) The returned holder class names now are just 'package/className', removed head and tail. >> >> ??? 3)? Remove add_extra_classes from CollectClassesClosure since after bug 8250990: https://bugs.openjdk.java.net/browse/JDK-8250990 pushed, the CollectClassesClosure no longer exist. >> >> ??? 4)? Still keep the parsing for TRACE_RESOLVE in java/lang/invoke/GenerateJLIClassesHelper.java, so VM call its function to regenerate holder classes. >> >> >> ? Re-tested Mach5 tier1-4 >> >> Thanks >> >> Yumin >> >> >> On 8/20/20 8:05 PM, Yumin Qi wrote: >>> Hi, Mandy >>> >>> >>> On 8/20/20 5:10 PM, Mandy Chung wrote: >>>> >>>> >>>> On 8/19/20 10:14 PM, Yumin Qi wrote: >>>>> >>>>> HI, Mandy >>>>> >>>>> ? Thanks for the review, I took one day off yesterday so just got a detail look of your reply. >>>>> >>>>> On 8/19/20 1:30 PM, Mandy Chung wrote: >>>>>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>>>>> Hi, Ioi >>>>>>> >>>>>>> ? Thanks for review/suggestion. I have updated the webrev at the following link: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>>>>> >>>>>> >>>>>> This patch leverages the TRACE_RESOLVE output and passes the trace output to VM.? VM then calls GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing and generate Holder class per the resolved LFs.?? I think there are other cleaner alternatives implementing this.?? jlink --generate-jli-classes plugin depends the trace output whereas -Xshare:dump does not.?? It's cleaner to skip generating the trace output and parsing for dumping shared archive purpose.? In addition, the implementation needs some cleanup (I can send you feedback on the next revision) >>>>>> >>>>> Current patch did not change the existing code for JLinkPlugin part. I just moved the parsing code from GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java since the former is an internal class to which we shouldn't call to generate holder classes. >>>>>> Instead of relying on a system property "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. >>>>> >>>>> I remember we have such API to query if flag -Xshare:dump or -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This flag is the one used to log class name to list file. In GenerateLinkOptData.gmk: >>>>> >>>>> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR) >>>>> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) >>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $(JLI_TRACE_FILE))) >>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw \ >>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >>>>> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >>>>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>>>> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw.2 \ >>>>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>> ??????????? build.tools.classlist.HelloClasslist \ >>>>> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >>>>> ??????????? || ( \ >>>>> ??????????????? exitcode=$$? ; \ >>>>> ??????????????? $(ECHO) "ERROR: Failed to generate link optimization data." \ >>>>> ??????????????????? "This is likely a problem with the newly built JVM/JDK." ; \ >>>>> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >>>>> ??????????????? exit $$exitcode \ >>>>> ??????????? ) >>>>> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >>>>> >>>>> The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current implementation, DumpLoadedClassList will turn on property java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same output sent to stdout and log file DumpLoadedClassList specified. >>>>> >>>> >>>> These entries are duplicated in two different files: one for jlink --generate-jli-classes plugin and another for CDS use.? CDS -Xshare:dump attempts to do what jlink plugin does but writes those generated classes in to shared archive. >>>> >>>> Like the above make logic to build JDK image, the same entries are written in both default-jli-trace.txt via System.out and to classlist via JNI call to the VM.? I guess VM also implements the logic to do some kind of diffing and write to CDS archive. >>>> >>> In current implementation, vm side only records the line as from TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at dump time, call back to java for parsing those recordings and generating the holder classes, this uses the existing JLI code. >>>>> >>>>> Now instead of this property, using a vm interface API to query if this flag is set, I think it is better choice. But here I am NOT sure I understand your suggestion, I think there are two choices: >>>>> >>>>> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set >>>>> >>>>> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting those name, type and holder name to regenerate holder classes? >>>>> >>>> >>>> I misunderstood that this CDS_TRACE_RESOLVE flag is set during -Xshare:dump time. >>>> >>>> Ioi has clarified to me offline that this step is actually part of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the given class list file, i.e. you repurpose the class list file to include the log output that was initially designed for jlink plugin. >>>> >>>> To me, I'd prefer to see this support depending on `jlink --generate-jli-classes` which is an existing functionality and much cleaner.? This way this does not require any VM change.?? It will generate the holder classes in the custom image per the application-specific config file. >>>> >>>> What it means is that: a customer would need to create a custom image with their application-specific config file. It might need a new CDS? option to specify a separate TRACE_RESOLVE file.?? It would turn on this feature by default by defining a default path of the log file if it helps. >>>> >>> So for now, I would implement an API to query if flag DumpLoadedClassList set in cmd line, remove new added property of java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address custom image with CDS in future in a separate issue. >>> >>> >>> Thanks >>> >>> Yumin >>> >>>> I understand that this is not the existing CDS work flow and CDS archive does not require to run on a custom image.?? I see the value of this approach which can prepare customers to start building and using its own custom image. >>>> >>>> Of course the implementation would be much simpler (adding a flag to write these traces to a given file path, that is). >>>> >>>> Mandy > From coleen.phillimore at oracle.com Tue Aug 25 19:41:14 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 25 Aug 2020 15:41:14 -0400 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <1ca66b6a-8a23-c4b1-c95d-10fc23b668f8@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> <1ca66b6a-8a23-c4b1-c95d-10fc23b668f8@oracle.com> Message-ID: <6344748b-f484-e743-8bef-5750d0b49155@oracle.com> Hi, I'm sorry I haven't been watching this change. http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html Can you move find_class() into systemDictionary.hpp/cpp with the rest of the find_class().? This is strange here.? You don't want to expose that the dictionary is here to the rest of the code.? The caller should call one of the SystemDictionary versions, or add this to SystemDictionary with the other functions with the same name. http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/memory/lambdaFormInvokers.cpp.html These new files belongs in classfile.? It's closer to systemDictionaryShared code than memory code. 2 * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All rights reserved. Also copyright shouldn't start with 2012. http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html This add_to_hierarchy is a strange delegation and a surprise that it's also defined here.? Please call this inline from lambdaFormInvokers, ie take the Compile_lock and call SystemDictionary::add_to_hierarchy(). I don't have any deeper comments. Thanks, Coleen On 8/25/20 2:59 PM, Yumin Qi wrote: > HI, Ioi > > ? Thanks for the re-review, updated webrev: > > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/ > > ? Changed according to your suggestion, the only a little difference > is move traceResolve(String line) to InvokerBytecodeGenerator as a > static package public function so other package classes can call it. > > ? re-tested local on runtime/cds > > > Thanks > > Yumin > > > On 8/24/20 4:23 PM, Ioi Lam wrote: >> Hi Yumin, >> >> This looks good overall. Here are my comments: >> >> ===================== >> 6065???? size_t new_id = Atomic::add(&counter, (size_t)1); >> 6066???? jio_snprintf(addr_buf, 20, INTPTR_FORMAT, new_id); >> >> I think this should be SIZE_FORMAT >> >> ===================== >> ? 65 class KlassFactory : AllStatic { >> ? 66 >> ? 67?? // approved clients >> ? 68?? friend class ClassLoader; >> ? 69?? friend class ClassLoaderExt; >> ? 70?? friend class SystemDictionary; >> ? 71?? friend class LambdaFormInvokers; >> ? 72 >> ? 73? private: >> ? 74?? static InstanceKlass* create_from_stream(ClassFileStream* stream, >> >> I think instead of adding everyone who uses create_from_stream as a >> friend class, we should just change create_from_stream into a public >> function and remove the friend declarations. >> >> ===================== >> ?146?? // add to hierarchy and set state to loaded. >> ?147?? { >> ?148???? MutexLocker mu_r(THREAD, Compile_lock); // >> add_shared_to_hierarchy asserts this. >> ?149???? SystemDictionaryShared::add_shared_to_hierarchy(result, >> THREAD); >> ?150?? } >> >> I think the function name can be changed to >> SystemDictionaryShared::add_to_hierarchy as the "_shared" seems >> redundant. The "set state to loaded" comment seems wrong, as we have >> the assert on line 1155. I think the comment can be removed. >> >> 1153 void >> SystemDictionaryShared::add_shared_to_hierarchy(InstanceKlass* k, >> TRAPS) { >> 1154?? Arguments::assert_is_dumping_archive(); >> 1155?? assert(!k->is_loaded(), "Invariant"); >> 1156?? assert_locked_or_safepoint(Compile_lock); // add_to_hierarchy >> assert on it. >> 1157?? SystemDictionary::add_to_hierarchy(k, CHECK); >> 1158 } >> >> Also, I think it's better to move the MutexLocker call into >> SystemDictionaryShared::add_shared_to_hierarchy. >> >> ======================== >> >> before: >> >> ?478???????????????? if (TRACE_RESOLVE && salvage != null) { >> ?479???????????????????? // Used by jlink species pregeneration >> plugin, see >> ?480???????????????????? // >> jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >> ?481???????????????????? System.out.println("[SPECIES_RESOLVE] " + >> className + " (salvaged)"); >> ?482???????????????? } >> >> after: >> >> ?488???????????????? // Used by jlink species pregeneration plugin, see >> ?489???????????????? // >> jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >> ?490???????????????? traceResolve("[SPECIES_RESOLVE] " + className + >> " (salvaged)"); >> >> >> When tracing is disabled, this will make extra allocations and cause >> a small slowdown. I think it's better to >> >> ??? if ((TRACE_RESOLVE|TRACE_RESOLVE_CDS) && salvage != null) { >> ? ? ?? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); >> ??? } >> >> Because TRACE_RESOLVE is a static final boolean, the JIT compiler >> will completely optimize this block out. >> >> For the same reason, instead of calling >> VM.isDumpLoadedClassListSetAndOpen() every time, it's better to use a >> static final variable. >> >> >> ======================= >> >> ?698???????? if (TRACE_RESOLVE) { >> ?699???????????? System.out.println("[LF_RESOLVE] " + >> holder.getName() + " " + name + " " + >> ?700 shortenSignature(basicTypeSignature(type)) + (resolvedMember != >> null ? " (success)" : " (fail)") ); >> ?701???????? } >> ?702???????? if (VM.isDumpLoadedClassListSetAndOpen()) { >> ?703 GenerateJLIClassesHelper.cdsTraceResolve("[LF_RESOLVE] " + >> holder.getName() + " " + name + " " + >> ?704 shortenSignature(basicTypeSignature(type)) + (resolvedMember != >> null ? " (success)" : " (fail)") ); >> ?705???????? } >> ?706???????? return resolvedMember; >> >> >> I think the two "if" blocks should be combined similarly to >> ClassSpecializer::traceResolve(). >> >> ========================= >> ? 34 >> Java_java_lang_invoke_GenerateJLIClassesHelper_cdsTraceResolve(JNIEnv >> *env, jclass ignore, jstring line) { >> >> Maybe this should be moved to the "VM" class as well? >> >> ========================= >> lambdaFormInvokers.hpp: >> >> Need these declarations: >> >> ??? #include "memory/allocation.hpp"??? <-- for AllStatic >> ??? #include "runtime/handles.hpp"????? <-- for typeArrayHandle and >> Handle >> ??? #include "utilities/exceptions.hpp" <-- for TRAPS >> >> ??? template class GrowableArray; >> ========================= >> >> >> Thanks >> - Ioi >> >> >> >> On 8/23/20 3:56 PM, Yumin Qi wrote: >>> >>> Hi, Mandy, Ioi and Calvin >>> >>> >>> ? I have updated the new changed at: >>> >>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-03/ >>> >>> ? In this version: >>> >>> ??? 1) Added a new API to check if flag DumpLoadedClassList set and >>> the file is open. If true, call into vm to print out the trace line >>> to the log file. >>> >>> ??????? Just thinking if we just call the cdsTraceResolve without >>> checking if the flag DumpLoadedClassList set and file open, this >>> way, the check logic is in the vm side like before, so save code by >>> not adding the new API. >>> >>> ??? 2) The returned holder class names now are just >>> 'package/className', removed head and tail. >>> >>> ??? 3)? Remove add_extra_classes from CollectClassesClosure since >>> after bug 8250990: https://bugs.openjdk.java.net/browse/JDK-8250990 >>> pushed, the CollectClassesClosure no longer exist. >>> >>> ??? 4)? Still keep the parsing for TRACE_RESOLVE in >>> java/lang/invoke/GenerateJLIClassesHelper.java, so VM call its >>> function to regenerate holder classes. >>> >>> >>> ? Re-tested Mach5 tier1-4 >>> >>> Thanks >>> >>> Yumin >>> >>> >>> On 8/20/20 8:05 PM, Yumin Qi wrote: >>>> Hi, Mandy >>>> >>>> >>>> On 8/20/20 5:10 PM, Mandy Chung wrote: >>>>> >>>>> >>>>> On 8/19/20 10:14 PM, Yumin Qi wrote: >>>>>> >>>>>> HI, Mandy >>>>>> >>>>>> ? Thanks for the review, I took one day off yesterday so just got >>>>>> a detail look of your reply. >>>>>> >>>>>> On 8/19/20 1:30 PM, Mandy Chung wrote: >>>>>>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>>>>>> Hi, Ioi >>>>>>>> >>>>>>>> ? Thanks for review/suggestion. I have updated the webrev at >>>>>>>> the following link: >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>>>>>> >>>>>>> >>>>>>> This patch leverages the TRACE_RESOLVE output and passes the >>>>>>> trace output to VM.? VM then calls >>>>>>> GenerateJLIClassesHelper::generateMHHolderClasses to do the >>>>>>> parsing and generate Holder class per the resolved LFs.?? I >>>>>>> think there are other cleaner alternatives implementing this.?? >>>>>>> jlink --generate-jli-classes plugin depends the trace output >>>>>>> whereas -Xshare:dump does not.?? It's cleaner to skip generating >>>>>>> the trace output and parsing for dumping shared archive >>>>>>> purpose.? In addition, the implementation needs some cleanup (I >>>>>>> can send you feedback on the next revision) >>>>>>> >>>>>> Current patch did not change the existing code for JLinkPlugin >>>>>> part. I just moved the parsing code from >>>>>> GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java >>>>>> since the former is an internal class to which we shouldn't call >>>>>> to generate holder classes. >>>>>>> Instead of relying on a system property >>>>>>> "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better >>>>>>> to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this >>>>>>> is CDS dump time. >>>>>> >>>>>> I remember we have such API to query if flag -Xshare:dump or >>>>>> -Xshare:on used. Do you mean if DumpLoadedClassList flag set? >>>>>> This flag is the one used to log class name to list file. In >>>>>> GenerateLinkOptData.gmk: >>>>>> >>>>>> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) >>>>>> $(CLASSLIST_JAR) >>>>>> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, >>>>>> $@)) >>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, >>>>>> $(JLI_TRACE_FILE))) >>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >>>>>> -XX:DumpLoadedClassList=$@.raw \ >>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >>>>>> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >>>>>> ??????????? -XX:SharedClassListFile=$@.interim >>>>>> -XX:SharedArchiveFile=$@.jsa \ >>>>>> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >>>>>> -XX:DumpLoadedClassList=$@.raw.2 \ >>>>>> ??????????? -XX:SharedClassListFile=$@.interim >>>>>> -XX:SharedArchiveFile=$@.jsa \ >>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>> ??????????? build.tools.classlist.HelloClasslist \ >>>>>> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >>>>>> ??????????? || ( \ >>>>>> ??????????????? exitcode=$$? ; \ >>>>>> ??????????????? $(ECHO) "ERROR: Failed to generate link >>>>>> optimization data." \ >>>>>> ??????????????????? "This is likely a problem with the newly >>>>>> built JVM/JDK." ; \ >>>>>> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >>>>>> ??????????????? exit $$exitcode \ >>>>>> ??????????? ) >>>>>> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >>>>>> >>>>>> The $(JLI_TRACE_FILE) is generated with both >>>>>> -XX:DumpLoadedClassList and >>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current >>>>>> implementation, DumpLoadedClassList will turn on property >>>>>> java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same >>>>>> output sent to stdout and log file DumpLoadedClassList specified. >>>>>> >>>>> >>>>> These entries are duplicated in two different files: one for jlink >>>>> --generate-jli-classes plugin and another for CDS use.? CDS >>>>> -Xshare:dump attempts to do what jlink plugin does but writes >>>>> those generated classes in to shared archive. >>>>> >>>>> Like the above make logic to build JDK image, the same entries are >>>>> written in both default-jli-trace.txt via System.out and to >>>>> classlist via JNI call to the VM.? I guess VM also implements the >>>>> logic to do some kind of diffing and write to CDS archive. >>>>> >>>> In current implementation, vm side only records the line as from >>>> TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at dump >>>> time, call back to java for parsing those recordings and generating >>>> the holder classes, this uses the existing JLI code. >>>>>> >>>>>> Now instead of this property, using a vm interface API to query >>>>>> if this flag is set, I think it is better choice. But here I am >>>>>> NOT sure I understand your suggestion, I think there are two >>>>>> choices: >>>>>> >>>>>> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via >>>>>> CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList >>>>>> is set >>>>>> >>>>>> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting >>>>>> those name, type and holder name to regenerate holder classes? >>>>>> >>>>> >>>>> I misunderstood that this CDS_TRACE_RESOLVE flag is set during >>>>> -Xshare:dump time. >>>>> >>>>> Ioi has clarified to me offline that this step is actually part of >>>>> -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to >>>>> the given class list file, i.e. you repurpose the class list file >>>>> to include the log output that was initially designed for jlink >>>>> plugin. >>>>> >>>>> To me, I'd prefer to see this support depending on `jlink >>>>> --generate-jli-classes` which is an existing functionality and >>>>> much cleaner.? This way this does not require any VM change.?? It >>>>> will generate the holder classes in the custom image per the >>>>> application-specific config file. >>>>> >>>>> What it means is that: a customer would need to create a custom >>>>> image with their application-specific config file. It might need a >>>>> new CDS? option to specify a separate TRACE_RESOLVE file.?? It >>>>> would turn on this feature by default by defining a default path >>>>> of the log file if it helps. >>>>> >>>> So for now, I would implement an API to query if flag >>>> DumpLoadedClassList set in cmd line, remove new added property of >>>> java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address >>>> custom image with CDS in future in a separate issue. >>>> >>>> >>>> Thanks >>>> >>>> Yumin >>>> >>>>> I understand that this is not the existing CDS work flow and CDS >>>>> archive does not require to run on a custom image.?? I see the >>>>> value of this approach which can prepare customers to start >>>>> building and using its own custom image. >>>>> >>>>> Of course the implementation would be much simpler (adding a flag >>>>> to write these traces to a given file path, that is). >>>>> >>>>> Mandy >> From coleen.phillimore at oracle.com Tue Aug 25 20:24:43 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 25 Aug 2020 16:24:43 -0400 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: References: Message-ID: <3234fa66-5199-32dd-1f13-e15b61c7dc63@oracle.com> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html Are these #includes still needed? http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/classLoaderDataShared.hpp.html Looks like the include guards are wrong.? Should have pushed my #pragma once change then nobody would have to worry about these. http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/classLoaderDataShared.cpp.html 136 void ClassLoaderDataShared::init_archived_tables() { 137 assert(DumpSharedSpaces, "must be"); 138 if (MetaspaceShared::use_full_module_graph()) { Shouldn't these functions be called only if MetaspaceShared::use_full_module_graph() instead of having them guarded entirely by these if statements. ie. assert(DumpSharedSpaces && MetaspaceShared::use_full_module_graph(), "must be"); Thank you for moving the new code to classLoaderDataShared.cpp http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/packageEntry.cpp.frames.html Can you organize this so that the archiving functions for PackageEntry are after the PackageEntry:: functions, and the ones for PackageEntryTable are after the PackageEntryTable:: functions? They'll still be sort of mixed up, but the CDS special code in the middle isn't good.? And if we ever want to change the entries or the tables themselves it'll be easier. http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/moduleEntry.cpp.frames.html Same comment.? The code in both looks fine, even though it looks the same. I don't know how to generalize it without being artificial and not resembling either.? Separating the functions will help. On 8/12/20 6:11 PM, Ioi Lam wrote: > Hi Coleen, > > Thanks for the comments. I have an updated webrev > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/ > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02.delta/ > > > > On 8/4/20 7:49 AM, Coleen Phillimore wrote: >> This is sort of massive, so I'll only make a couple initial comments. >> >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html >> >> >> All this new code in ClassLoaderData should go somewhere else, like >> maybe classLoaderDataShared.hpp/cpp ?? I don't see how it needs to be >> owned by CLD, except maybe the serialized function. >> >> > Fixed > >> + if (loader_data) { >> >> >> There are many checks that should be != NULL here. >> > > Fixed and added comments > > >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/javaClasses.cpp.udiff.html >> >> >> I don't see why compute_offset/s() has been exported outside >> javaClasses.cpp. >> > > It was left-over code that shouldn't be there. I removed it. > > >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/moduleEntry.cpp.udiff.html >> >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/packageEntry.cpp.udiff.html >> >> >> These look like the same code to me.? Maybe they can be generalized >> and put together in a moduleEntryShared.cpp/hpp ? >> > > They look similar but also do different things. Any suggestion for how > to generalize them? > > >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/systemDictionary.cpp.udiff.html >> >> >> Why do you register the loaders here? >> > > It's to force the creation of the ClassLoaderData if a loader oop > doesn't have one yet. Also, in the new version, I moved the code to > Modules::define_archived_modules(). Yes, this is a better place. Thanks, Coleen > > > Thanks > - Ioi > >> I don't know enough about the Java code to review that. >> >> I assume that this going to be rebased on other changes, so this is >> only an incomplete review. >> >> Coleen >> >> >> On 7/22/20 3:36 PM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8244778 >>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/ >>> >>> >>> Please review this patch that stores the full module graph in the CDS >>> archive heap. This reduces the initialization time of the basic JVM by >>> about 22%: >>> >>> $ perf stat -r 100 bin/java -version >>> before: 98,219,329 instructions 0.03971 secs elapsed (+- 0.44%) >>> after:? 55,835,815 instructions 0.03109 secs elapsed (+- 0.65%) >>> >>> [1] Start with ModuleBootstrap.java. The current implementation is >>> ??? quite restrictive: the archived module graph is used only when no >>> ??? module options are specified. >>> >>> ??? See ModuleBootstrap.mayUseArchivedBootLayer(). >>> >>> ??? We can probably support options such as main module and module path >>> ??? in a future RFE. >>> >>> [2] In the current JDK implementation, there is no single object >>> ??? that represents "the module graph". Most of the information >>> ??? is stored in the archive bootLayer object, but a few additional >>> ??? restoration operations need to be performed: >>> >>> ??? + See ModuleBootstrap.getArchivedBootLayer() >>> ??? + Some static fields need to be archived/restored in >>> ????? Module.java, BuiltinClassLoader.java, ClassLoaders.java >>> ????? and BootLoader.java >>> >>> [3] I ran into a complication with two loader instances of >>> ??? PlatformClassLoader and AppClassLoader. They are stored in >>> ??? multiple tables inside the module graph (e.g., >>> ??? BuiltinClassLoader$LoadedModule) so I cannot easily recreate >>> ??? them at runtime. >>> >>> ??? However, these two loaders contain information specific to the >>> ??? dump time VM lifecycle (such as the classes that were loaded >>> ??? during CDS dumping) that need to be scrubbed. I couldn't find an >>> ??? elegant way of doing this, so I added a private >>> "resetArchivedStates" >>> ??? method to a few classes. They are called inside >>> ??? HeapShared::reset_archived_object_states(). >>> >>> [4] Related native data structures (PackageEntry and ModuleEntry) >>> ??? are also archived. Start with classLoaderData.cpp >>> >>> Passes mach5 tiers 1-4. I will test with additional tiers. >>> >>> Thanks >>> - Ioi >> > From coleen.phillimore at oracle.com Tue Aug 25 20:32:50 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 25 Aug 2020 16:32:50 -0400 Subject: RFR(XS) 8252056 Move DumpRegion/ReadClosure/WriteClosure to archiveUtils.hpp In-Reply-To: <6b312a7a-df99-4399-10d2-1122ce689dd3@oracle.com> References: <6b312a7a-df99-4399-10d2-1122ce689dd3@oracle.com> Message-ID: <41210581-f242-4456-b5d3-d745248e5f6e@oracle.com> Looks good to me! Coleen On 8/19/20 4:47 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8252056 > http://cr.openjdk.java.net/~iklam/jdk16/8252056-move-dump-region-to-archive-utils.v01/ > > > Part of CDS code clean up: > > The classes are currently in metaspaceShared.hpp/cpp, which are too big. > It's better to move these utility classes to archiveUtils.cpp/hpp > > Thanks > - Ioi From coleen.phillimore at oracle.com Tue Aug 25 20:33:36 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 25 Aug 2020 16:33:36 -0400 Subject: RFR(XS) 8252056 Move DumpRegion/ReadClosure/WriteClosure to archiveUtils.hpp In-Reply-To: <41210581-f242-4456-b5d3-d745248e5f6e@oracle.com> References: <6b312a7a-df99-4399-10d2-1122ce689dd3@oracle.com> <41210581-f242-4456-b5d3-d745248e5f6e@oracle.com> Message-ID: <3bdd4835-8873-4e7e-8d08-cac0607d1a32@oracle.com> oops too late... On 8/25/20 4:32 PM, Coleen Phillimore wrote: > > Looks good to me! > Coleen > > On 8/19/20 4:47 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8252056 >> http://cr.openjdk.java.net/~iklam/jdk16/8252056-move-dump-region-to-archive-utils.v01/ >> >> >> Part of CDS code clean up: >> >> The classes are currently in metaspaceShared.hpp/cpp, which are too big. >> It's better to move these utility classes to archiveUtils.cpp/hpp >> >> Thanks >> - Ioi > From mikael.vidstedt at oracle.com Tue Aug 25 20:44:35 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 25 Aug 2020 13:44:35 -0700 Subject: [16] RFR 8252250: isnanf is obsolete In-Reply-To: References: Message-ID: Looks good, thanks for doing the necessary research! Cheers, Mikael > On Aug 25, 2020, at 1:21 AM, Aleksei Voitylov wrote: > > Hi, > > in preparation for a review request for Alpine Linux JEP [1] it was > suggested [2] to send RFRs which are not specific to musl or busybox for > review upstream as separate patches. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8252250 > Fix: http://cr.openjdk.java.net/~avoitylov/webrev.8252250/ > > ifnanf is obsolete [3], and musl does not define it. Proof of > correctness of replacement of isnanf with isnan for all practical > versions of gcc and llvm is provided in a comment to [4]. A backport of > this change to 14 passes JCK and shows no jtreg regressions on Linux. > > Thanks, > -Aleksei > > > [1] https://openjdk.java.net/jeps/386 > [2] > https://mail.openjdk.java.net/pipermail/portola-dev/2020-August/000471.html > [3] https://man7.org/linux/man-pages/man3/isnanf.3.html > [4] https://bugs.openjdk.java.net/browse/JDK-8178689 > > From coleen.phillimore at oracle.com Tue Aug 25 21:00:06 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 25 Aug 2020 17:00:06 -0400 Subject: RFR(S) Remove excessive inclusion of arguments.hpp In-Reply-To: References: <09c2f053-2c61-5b7d-ea83-18c1db3c2b8e@oracle.com> Message-ID: <26141468-6793-40c2-dd48-57caf2bcd279@oracle.com> +1 Coleen On 8/25/20 12:03 AM, Thomas St?fe wrote: > Hi Ioi, > > thanks for the explanation. Looks all good to me. > > ..Thomas > > On Mon, Aug 24, 2020 at 7:16 PM Ioi Lam wrote: > >> Hi Thomas, thanks for the review. >> >> Some cpp files use vm_exit_during_initialization(), which is defined in >> java.hpp, which was included by arguments.hpp. >> >> Now that I have removed arguments.hpp (indirectly) from these cpp files, I >> have to add java.hpp to fix the missing dependency. >> >> Thanks >> - Ioi >> >> On 8/23/20 1:31 AM, Thomas St?fe wrote: >> >> Hi Ioi, >> >> looks okay, but why the added java.hpp includes? At first glance does not >> seem related to your bug description. >> >> Thanks Thomas >> >> On Fri, Aug 21, 2020 at 6:47 PM Ioi Lam wrote: >> >>> https://bugs.openjdk.java.net/browse/JDK-8252151 >>> >>> http://cr.openjdk.java.net/~iklam/jdk16/8252151-dont-include-arguments-hpp.v01/ >>> >>> The large header file arguments.hpp is included by popular header files >>> such as constMethod.hpp and methodCounters.hpp (which are included by >>> method.hpp, which is included by almost every cpp file). >>> >>> The dependencies on arguments.hpp by constMethod.hpp are very small, and >>> can be moved to constMethod.cpp. >>> >>> methodCounters.hpp does not depend on arguments.hpp >>> >>> === >>> by removing these dependencies, we can reduce the total number of >>> include files in the hotspot build from 530,921 to 523,010, or a >>> reduction of about 1.5%. >>> >>> >>> From yumin.qi at oracle.com Tue Aug 25 21:09:34 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 25 Aug 2020 14:09:34 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <6344748b-f484-e743-8bef-5750d0b49155@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> <1ca66b6a-8a23-c4b1-c95d-10fc23b668f8@oracle.com> <6344748b-f484-e743-8bef-5750d0b49155@oracle.com> Message-ID: HI, Coleen ? Thanks for review. On 8/25/20 12:41 PM, Coleen Phillimore wrote: > > Hi, I'm sorry I haven't been watching this change. > > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html > > Can you move find_class() into systemDictionary.hpp/cpp with the rest of the find_class().? This is strange here.? You don't want to expose that the dictionary is here to the rest of the code. The caller should call one of the SystemDictionary versions, or add this to SystemDictionary with the other functions with the same name. > There exist two find_class functions: ? // Basic find on loaded classes ? static InstanceKlass* find_class(unsigned int hash, ?????????????????????????????????? Symbol* name, Dictionary* dictionary); ? static InstanceKlass* find_class(Symbol* class_name, ClassLoaderData* loader_data); ? They are all protected. I think? I can change the second version to public access? Then using it should be more convenient. I added find_class is because those functions are all protected in SystemDictionary, they are not available to public. > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/memory/lambdaFormInvokers.cpp.html > > These new files belongs in classfile.? It's closer to systemDictionaryShared code than memory code. > > ?? 2? * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All rights reserved. > Oops, copy error. > > Also copyright shouldn't start with 2012. > > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html > > This add_to_hierarchy is a strange delegation and a surprise that it's also defined here.? Please call this inline from lambdaFormInvokers, ie take the Compile_lock and call SystemDictionary::add_to_hierarchy(). > Yes, this extra layout seems not necessary --- it just indicates that this is from shared code. I will use SystemDictionary::add_to_hierarchy directly. Thanks Yumin > I don't have any deeper comments. > > Thanks, > Coleen > > On 8/25/20 2:59 PM, Yumin Qi wrote: >> HI, Ioi >> >> ? Thanks for the re-review, updated webrev: >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/ >> >> ? Changed according to your suggestion, the only a little difference is move traceResolve(String line) to InvokerBytecodeGenerator as a static package public function so other package classes can call it. >> >> ? re-tested local on runtime/cds >> >> >> Thanks >> >> Yumin >> >> >> On 8/24/20 4:23 PM, Ioi Lam wrote: >>> Hi Yumin, >>> >>> This looks good overall. Here are my comments: >>> >>> ===================== >>> 6065???? size_t new_id = Atomic::add(&counter, (size_t)1); >>> 6066???? jio_snprintf(addr_buf, 20, INTPTR_FORMAT, new_id); >>> >>> I think this should be SIZE_FORMAT >>> >>> ===================== >>> ? 65 class KlassFactory : AllStatic { >>> ? 66 >>> ? 67?? // approved clients >>> ? 68?? friend class ClassLoader; >>> ? 69?? friend class ClassLoaderExt; >>> ? 70?? friend class SystemDictionary; >>> ? 71?? friend class LambdaFormInvokers; >>> ? 72 >>> ? 73? private: >>> ? 74?? static InstanceKlass* create_from_stream(ClassFileStream* stream, >>> >>> I think instead of adding everyone who uses create_from_stream as a friend class, we should just change create_from_stream into a public function and remove the friend declarations. >>> >>> ===================== >>> ?146?? // add to hierarchy and set state to loaded. >>> ?147?? { >>> ?148???? MutexLocker mu_r(THREAD, Compile_lock); // add_shared_to_hierarchy asserts this. >>> ?149 SystemDictionaryShared::add_shared_to_hierarchy(result, THREAD); >>> ?150?? } >>> >>> I think the function name can be changed to SystemDictionaryShared::add_to_hierarchy as the "_shared" seems redundant. The "set state to loaded" comment seems wrong, as we have the assert on line 1155. I think the comment can be removed. >>> >>> 1153 void SystemDictionaryShared::add_shared_to_hierarchy(InstanceKlass* k, TRAPS) { >>> 1154?? Arguments::assert_is_dumping_archive(); >>> 1155?? assert(!k->is_loaded(), "Invariant"); >>> 1156?? assert_locked_or_safepoint(Compile_lock); // add_to_hierarchy assert on it. >>> 1157?? SystemDictionary::add_to_hierarchy(k, CHECK); >>> 1158 } >>> >>> Also, I think it's better to move the MutexLocker call into SystemDictionaryShared::add_shared_to_hierarchy. >>> >>> ======================== >>> >>> before: >>> >>> ?478???????????????? if (TRACE_RESOLVE && salvage != null) { >>> ?479???????????????????? // Used by jlink species pregeneration plugin, see >>> ?480???????????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >>> ?481???????????????????? System.out.println("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>> ?482???????????????? } >>> >>> after: >>> >>> ?488???????????????? // Used by jlink species pregeneration plugin, see >>> ?489???????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >>> ?490???????????????? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>> >>> >>> When tracing is disabled, this will make extra allocations and cause a small slowdown. I think it's better to >>> >>> ??? if ((TRACE_RESOLVE|TRACE_RESOLVE_CDS) && salvage != null) { >>> ? ? ?? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>> ??? } >>> >>> Because TRACE_RESOLVE is a static final boolean, the JIT compiler will completely optimize this block out. >>> >>> For the same reason, instead of calling VM.isDumpLoadedClassListSetAndOpen() every time, it's better to use a static final variable. >>> >>> >>> ======================= >>> >>> ?698???????? if (TRACE_RESOLVE) { >>> ?699???????????? System.out.println("[LF_RESOLVE] " + holder.getName() + " " + name + " " + >>> ?700 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); >>> ?701???????? } >>> ?702???????? if (VM.isDumpLoadedClassListSetAndOpen()) { >>> ?703 GenerateJLIClassesHelper.cdsTraceResolve("[LF_RESOLVE] " + holder.getName() + " " + name + " " + >>> ?704 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); >>> ?705???????? } >>> ?706???????? return resolvedMember; >>> >>> >>> I think the two "if" blocks should be combined similarly to ClassSpecializer::traceResolve(). >>> >>> ========================= >>> ? 34 Java_java_lang_invoke_GenerateJLIClassesHelper_cdsTraceResolve(JNIEnv *env, jclass ignore, jstring line) { >>> >>> Maybe this should be moved to the "VM" class as well? >>> >>> ========================= >>> lambdaFormInvokers.hpp: >>> >>> Need these declarations: >>> >>> ??? #include "memory/allocation.hpp"??? <-- for AllStatic >>> ??? #include "runtime/handles.hpp"????? <-- for typeArrayHandle and Handle >>> ??? #include "utilities/exceptions.hpp" <-- for TRAPS >>> >>> ??? template class GrowableArray; >>> ========================= >>> >>> >>> Thanks >>> - Ioi >>> >>> >>> >>> On 8/23/20 3:56 PM, Yumin Qi wrote: >>>> >>>> Hi, Mandy, Ioi and Calvin >>>> >>>> >>>> ? I have updated the new changed at: >>>> >>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-03/ >>>> >>>> ? In this version: >>>> >>>> ??? 1) Added a new API to check if flag DumpLoadedClassList set and the file is open. If true, call into vm to print out the trace line to the log file. >>>> >>>> ??????? Just thinking if we just call the cdsTraceResolve without checking if the flag DumpLoadedClassList set and file open, this way, the check logic is in the vm side like before, so save code by not adding the new API. >>>> >>>> ??? 2) The returned holder class names now are just 'package/className', removed head and tail. >>>> >>>> ??? 3)? Remove add_extra_classes from CollectClassesClosure since after bug 8250990: https://bugs.openjdk.java.net/browse/JDK-8250990 pushed, the CollectClassesClosure no longer exist. >>>> >>>> ??? 4)? Still keep the parsing for TRACE_RESOLVE in java/lang/invoke/GenerateJLIClassesHelper.java, so VM call its function to regenerate holder classes. >>>> >>>> >>>> ? Re-tested Mach5 tier1-4 >>>> >>>> Thanks >>>> >>>> Yumin >>>> >>>> >>>> On 8/20/20 8:05 PM, Yumin Qi wrote: >>>>> Hi, Mandy >>>>> >>>>> >>>>> On 8/20/20 5:10 PM, Mandy Chung wrote: >>>>>> >>>>>> >>>>>> On 8/19/20 10:14 PM, Yumin Qi wrote: >>>>>>> >>>>>>> HI, Mandy >>>>>>> >>>>>>> ? Thanks for the review, I took one day off yesterday so just got a detail look of your reply. >>>>>>> >>>>>>> On 8/19/20 1:30 PM, Mandy Chung wrote: >>>>>>>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>>>>>>> Hi, Ioi >>>>>>>>> >>>>>>>>> ? Thanks for review/suggestion. I have updated the webrev at the following link: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>>>>>>> >>>>>>>> >>>>>>>> This patch leverages the TRACE_RESOLVE output and passes the trace output to VM.? VM then calls GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing and generate Holder class per the resolved LFs.?? I think there are other cleaner alternatives implementing this.?? jlink --generate-jli-classes plugin depends the trace output whereas -Xshare:dump does not.?? It's cleaner to skip generating the trace output and parsing for dumping shared archive purpose.? In addition, the implementation needs some cleanup (I can send you feedback on the next revision) >>>>>>>> >>>>>>> Current patch did not change the existing code for JLinkPlugin part. I just moved the parsing code from GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java since the former is an internal class to which we shouldn't call to generate holder classes. >>>>>>>> Instead of relying on a system property "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. >>>>>>> >>>>>>> I remember we have such API to query if flag -Xshare:dump or -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This flag is the one used to log class name to list file. In GenerateLinkOptData.gmk: >>>>>>> >>>>>>> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR) >>>>>>> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >>>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) >>>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $(JLI_TRACE_FILE))) >>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw \ >>>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >>>>>>> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >>>>>>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>>>>>> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw.2 \ >>>>>>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >>>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>>> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>> ??????????? build.tools.classlist.HelloClasslist \ >>>>>>> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >>>>>>> ??????????? || ( \ >>>>>>> ??????????????? exitcode=$$? ; \ >>>>>>> ??????????????? $(ECHO) "ERROR: Failed to generate link optimization data." \ >>>>>>> ??????????????????? "This is likely a problem with the newly built JVM/JDK." ; \ >>>>>>> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >>>>>>> ??????????????? exit $$exitcode \ >>>>>>> ??????????? ) >>>>>>> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >>>>>>> >>>>>>> The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current implementation, DumpLoadedClassList will turn on property java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same output sent to stdout and log file DumpLoadedClassList specified. >>>>>>> >>>>>> >>>>>> These entries are duplicated in two different files: one for jlink --generate-jli-classes plugin and another for CDS use.? CDS -Xshare:dump attempts to do what jlink plugin does but writes those generated classes in to shared archive. >>>>>> >>>>>> Like the above make logic to build JDK image, the same entries are written in both default-jli-trace.txt via System.out and to classlist via JNI call to the VM.? I guess VM also implements the logic to do some kind of diffing and write to CDS archive. >>>>>> >>>>> In current implementation, vm side only records the line as from TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at dump time, call back to java for parsing those recordings and generating the holder classes, this uses the existing JLI code. >>>>>>> >>>>>>> Now instead of this property, using a vm interface API to query if this flag is set, I think it is better choice. But here I am NOT sure I understand your suggestion, I think there are two choices: >>>>>>> >>>>>>> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set >>>>>>> >>>>>>> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting those name, type and holder name to regenerate holder classes? >>>>>>> >>>>>> >>>>>> I misunderstood that this CDS_TRACE_RESOLVE flag is set during -Xshare:dump time. >>>>>> >>>>>> Ioi has clarified to me offline that this step is actually part of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the given class list file, i.e. you repurpose the class list file to include the log output that was initially designed for jlink plugin. >>>>>> >>>>>> To me, I'd prefer to see this support depending on `jlink --generate-jli-classes` which is an existing functionality and much cleaner.? This way this does not require any VM change.?? It will generate the holder classes in the custom image per the application-specific config file. >>>>>> >>>>>> What it means is that: a customer would need to create a custom image with their application-specific config file. It might need a new CDS? option to specify a separate TRACE_RESOLVE file.?? It would turn on this feature by default by defining a default path of the log file if it helps. >>>>>> >>>>> So for now, I would implement an API to query if flag DumpLoadedClassList set in cmd line, remove new added property of java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address custom image with CDS in future in a separate issue. >>>>> >>>>> >>>>> Thanks >>>>> >>>>> Yumin >>>>> >>>>>> I understand that this is not the existing CDS work flow and CDS archive does not require to run on a custom image.?? I see the value of this approach which can prepare customers to start building and using its own custom image. >>>>>> >>>>>> Of course the implementation would be much simpler (adding a flag to write these traces to a given file path, that is). >>>>>> >>>>>> Mandy >>> > From coleen.phillimore at oracle.com Tue Aug 25 21:15:26 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Tue, 25 Aug 2020 17:15:26 -0400 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> <1ca66b6a-8a23-c4b1-c95d-10fc23b668f8@oracle.com> <6344748b-f484-e743-8bef-5750d0b49155@oracle.com> Message-ID: <39ac8c9f-9efb-dfd1-4df9-e03c0744f7b7@oracle.com> On 8/25/20 5:09 PM, Yumin Qi wrote: > HI, Coleen > > ? Thanks for review. > > On 8/25/20 12:41 PM, Coleen Phillimore wrote: >> >> Hi, I'm sorry I haven't been watching this change. >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html >> >> >> Can you move find_class() into systemDictionary.hpp/cpp with the rest >> of the find_class().? This is strange here.? You don't want to expose >> that the dictionary is here to the rest of the code. The caller >> should call one of the SystemDictionary versions, or add this to >> SystemDictionary with the other functions with the same name. >> > There exist two find_class functions: > > ? // Basic find on loaded classes > ? static InstanceKlass* find_class(unsigned int hash, > ?????????????????????????????????? Symbol* name, Dictionary* dictionary); > ? static InstanceKlass* find_class(Symbol* class_name, > ClassLoaderData* loader_data); > > ? They are all protected. I think? I can change the second version to > public access? Then using it should be more convenient. I added > find_class is because those functions are all protected in > SystemDictionary, they are not available to public. Yes, it would be better to make the second one available to your caller. > >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/memory/lambdaFormInvokers.cpp.html >> >> >> These new files belongs in classfile.? It's closer to >> systemDictionaryShared code than memory code. >> >> ?? 2? * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All >> rights reserved. >> > Oops, copy error. >> >> Also copyright shouldn't start with 2012. >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >> >> >> This add_to_hierarchy is a strange delegation and a surprise that >> it's also defined here.? Please call this inline from >> lambdaFormInvokers, ie take the Compile_lock and call >> SystemDictionary::add_to_hierarchy(). >> > Yes, this extra layout seems not necessary --- it just indicates that > this is from shared code. I will use > SystemDictionary::add_to_hierarchy directly. > Thanks! Coleen > > Thanks > > Yumin > >> I don't have any deeper comments. >> >> Thanks, >> Coleen >> >> On 8/25/20 2:59 PM, Yumin Qi wrote: >>> HI, Ioi >>> >>> ? Thanks for the re-review, updated webrev: >>> >>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/ >>> >>> ? Changed according to your suggestion, the only a little difference >>> is move traceResolve(String line) to InvokerBytecodeGenerator as a >>> static package public function so other package classes can call it. >>> >>> ? re-tested local on runtime/cds >>> >>> >>> Thanks >>> >>> Yumin >>> >>> >>> On 8/24/20 4:23 PM, Ioi Lam wrote: >>>> Hi Yumin, >>>> >>>> This looks good overall. Here are my comments: >>>> >>>> ===================== >>>> 6065???? size_t new_id = Atomic::add(&counter, (size_t)1); >>>> 6066???? jio_snprintf(addr_buf, 20, INTPTR_FORMAT, new_id); >>>> >>>> I think this should be SIZE_FORMAT >>>> >>>> ===================== >>>> ? 65 class KlassFactory : AllStatic { >>>> ? 66 >>>> ? 67?? // approved clients >>>> ? 68?? friend class ClassLoader; >>>> ? 69?? friend class ClassLoaderExt; >>>> ? 70?? friend class SystemDictionary; >>>> ? 71?? friend class LambdaFormInvokers; >>>> ? 72 >>>> ? 73? private: >>>> ? 74?? static InstanceKlass* create_from_stream(ClassFileStream* >>>> stream, >>>> >>>> I think instead of adding everyone who uses create_from_stream as a >>>> friend class, we should just change create_from_stream into a >>>> public function and remove the friend declarations. >>>> >>>> ===================== >>>> ?146?? // add to hierarchy and set state to loaded. >>>> ?147?? { >>>> ?148???? MutexLocker mu_r(THREAD, Compile_lock); // >>>> add_shared_to_hierarchy asserts this. >>>> ?149 SystemDictionaryShared::add_shared_to_hierarchy(result, THREAD); >>>> ?150?? } >>>> >>>> I think the function name can be changed to >>>> SystemDictionaryShared::add_to_hierarchy as the "_shared" seems >>>> redundant. The "set state to loaded" comment seems wrong, as we >>>> have the assert on line 1155. I think the comment can be removed. >>>> >>>> 1153 void >>>> SystemDictionaryShared::add_shared_to_hierarchy(InstanceKlass* k, >>>> TRAPS) { >>>> 1154?? Arguments::assert_is_dumping_archive(); >>>> 1155?? assert(!k->is_loaded(), "Invariant"); >>>> 1156?? assert_locked_or_safepoint(Compile_lock); // >>>> add_to_hierarchy assert on it. >>>> 1157?? SystemDictionary::add_to_hierarchy(k, CHECK); >>>> 1158 } >>>> >>>> Also, I think it's better to move the MutexLocker call into >>>> SystemDictionaryShared::add_shared_to_hierarchy. >>>> >>>> ======================== >>>> >>>> before: >>>> >>>> ?478???????????????? if (TRACE_RESOLVE && salvage != null) { >>>> ?479???????????????????? // Used by jlink species pregeneration >>>> plugin, see >>>> ?480???????????????????? // >>>> jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >>>> ?481 System.out.println("[SPECIES_RESOLVE] " + className + " >>>> (salvaged)"); >>>> ?482???????????????? } >>>> >>>> after: >>>> >>>> ?488???????????????? // Used by jlink species pregeneration plugin, >>>> see >>>> ?489???????????????? // >>>> jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >>>> ?490???????????????? traceResolve("[SPECIES_RESOLVE] " + className >>>> + " (salvaged)"); >>>> >>>> >>>> When tracing is disabled, this will make extra allocations and >>>> cause a small slowdown. I think it's better to >>>> >>>> ??? if ((TRACE_RESOLVE|TRACE_RESOLVE_CDS) && salvage != null) { >>>> ? ? ?? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>>> ??? } >>>> >>>> Because TRACE_RESOLVE is a static final boolean, the JIT compiler >>>> will completely optimize this block out. >>>> >>>> For the same reason, instead of calling >>>> VM.isDumpLoadedClassListSetAndOpen() every time, it's better to use >>>> a static final variable. >>>> >>>> >>>> ======================= >>>> >>>> ?698???????? if (TRACE_RESOLVE) { >>>> ?699???????????? System.out.println("[LF_RESOLVE] " + >>>> holder.getName() + " " + name + " " + >>>> ?700 shortenSignature(basicTypeSignature(type)) + (resolvedMember >>>> != null ? " (success)" : " (fail)") ); >>>> ?701???????? } >>>> ?702???????? if (VM.isDumpLoadedClassListSetAndOpen()) { >>>> ?703 GenerateJLIClassesHelper.cdsTraceResolve("[LF_RESOLVE] " + >>>> holder.getName() + " " + name + " " + >>>> ?704 shortenSignature(basicTypeSignature(type)) + (resolvedMember >>>> != null ? " (success)" : " (fail)") ); >>>> ?705???????? } >>>> ?706???????? return resolvedMember; >>>> >>>> >>>> I think the two "if" blocks should be combined similarly to >>>> ClassSpecializer::traceResolve(). >>>> >>>> ========================= >>>> ? 34 >>>> Java_java_lang_invoke_GenerateJLIClassesHelper_cdsTraceResolve(JNIEnv >>>> *env, jclass ignore, jstring line) { >>>> >>>> Maybe this should be moved to the "VM" class as well? >>>> >>>> ========================= >>>> lambdaFormInvokers.hpp: >>>> >>>> Need these declarations: >>>> >>>> ??? #include "memory/allocation.hpp"??? <-- for AllStatic >>>> ??? #include "runtime/handles.hpp"????? <-- for typeArrayHandle and >>>> Handle >>>> ??? #include "utilities/exceptions.hpp" <-- for TRAPS >>>> >>>> ??? template class GrowableArray; >>>> ========================= >>>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> >>>> On 8/23/20 3:56 PM, Yumin Qi wrote: >>>>> >>>>> Hi, Mandy, Ioi and Calvin >>>>> >>>>> >>>>> ? I have updated the new changed at: >>>>> >>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-03/ >>>>> >>>>> ? In this version: >>>>> >>>>> ??? 1) Added a new API to check if flag DumpLoadedClassList set >>>>> and the file is open. If true, call into vm to print out the trace >>>>> line to the log file. >>>>> >>>>> ??????? Just thinking if we just call the cdsTraceResolve without >>>>> checking if the flag DumpLoadedClassList set and file open, this >>>>> way, the check logic is in the vm side like before, so save code >>>>> by not adding the new API. >>>>> >>>>> ??? 2) The returned holder class names now are just >>>>> 'package/className', removed head and tail. >>>>> >>>>> ??? 3)? Remove add_extra_classes from CollectClassesClosure since >>>>> after bug 8250990: >>>>> https://bugs.openjdk.java.net/browse/JDK-8250990 pushed, the >>>>> CollectClassesClosure no longer exist. >>>>> >>>>> ??? 4)? Still keep the parsing for TRACE_RESOLVE in >>>>> java/lang/invoke/GenerateJLIClassesHelper.java, so VM call its >>>>> function to regenerate holder classes. >>>>> >>>>> >>>>> ? Re-tested Mach5 tier1-4 >>>>> >>>>> Thanks >>>>> >>>>> Yumin >>>>> >>>>> >>>>> On 8/20/20 8:05 PM, Yumin Qi wrote: >>>>>> Hi, Mandy >>>>>> >>>>>> >>>>>> On 8/20/20 5:10 PM, Mandy Chung wrote: >>>>>>> >>>>>>> >>>>>>> On 8/19/20 10:14 PM, Yumin Qi wrote: >>>>>>>> >>>>>>>> HI, Mandy >>>>>>>> >>>>>>>> ? Thanks for the review, I took one day off yesterday so just >>>>>>>> got a detail look of your reply. >>>>>>>> >>>>>>>> On 8/19/20 1:30 PM, Mandy Chung wrote: >>>>>>>>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>>>>>>>> Hi, Ioi >>>>>>>>>> >>>>>>>>>> ? Thanks for review/suggestion. I have updated the webrev at >>>>>>>>>> the following link: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>>>>>>>> >>>>>>>>> >>>>>>>>> This patch leverages the TRACE_RESOLVE output and passes the >>>>>>>>> trace output to VM.? VM then calls >>>>>>>>> GenerateJLIClassesHelper::generateMHHolderClasses to do the >>>>>>>>> parsing and generate Holder class per the resolved LFs.?? I >>>>>>>>> think there are other cleaner alternatives implementing >>>>>>>>> this.?? jlink --generate-jli-classes plugin depends the trace >>>>>>>>> output whereas -Xshare:dump does not.?? It's cleaner to skip >>>>>>>>> generating the trace output and parsing for dumping shared >>>>>>>>> archive purpose.? In addition, the implementation needs some >>>>>>>>> cleanup (I can send you feedback on the next revision) >>>>>>>>> >>>>>>>> Current patch did not change the existing code for JLinkPlugin >>>>>>>> part. I just moved the parsing code from >>>>>>>> GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java >>>>>>>> since the former is an internal class to which we shouldn't >>>>>>>> call to generate holder classes. >>>>>>>>> Instead of relying on a system property >>>>>>>>> "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better >>>>>>>>> to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this >>>>>>>>> is CDS dump time. >>>>>>>> >>>>>>>> I remember we have such API to query if flag -Xshare:dump or >>>>>>>> -Xshare:on used. Do you mean if DumpLoadedClassList flag set? >>>>>>>> This flag is the one used to log class name to list file. In >>>>>>>> GenerateLinkOptData.gmk: >>>>>>>> >>>>>>>> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) >>>>>>>> $(CLASSLIST_JAR) >>>>>>>> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >>>>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, >>>>>>>> %, $@)) >>>>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, >>>>>>>> %, $(JLI_TRACE_FILE))) >>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >>>>>>>> -XX:DumpLoadedClassList=$@.raw \ >>>>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >>>>>>>> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >>>>>>>> ??????????? -XX:SharedClassListFile=$@.interim >>>>>>>> -XX:SharedArchiveFile=$@.jsa \ >>>>>>>> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >>>>>>>> -XX:DumpLoadedClassList=$@.raw.2 \ >>>>>>>> ??????????? -XX:SharedClassListFile=$@.interim >>>>>>>> -XX:SharedArchiveFile=$@.jsa \ >>>>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >>>>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>>>> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>> ??????????? build.tools.classlist.HelloClasslist \ >>>>>>>> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >>>>>>>> ??????????? || ( \ >>>>>>>> ??????????????? exitcode=$$? ; \ >>>>>>>> ??????????????? $(ECHO) "ERROR: Failed to generate link >>>>>>>> optimization data." \ >>>>>>>> ??????????????????? "This is likely a problem with the newly >>>>>>>> built JVM/JDK." ; \ >>>>>>>> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) >>>>>>>> ; \ >>>>>>>> ??????????????? exit $$exitcode \ >>>>>>>> ??????????? ) >>>>>>>> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >>>>>>>> >>>>>>>> The $(JLI_TRACE_FILE) is generated with both >>>>>>>> -XX:DumpLoadedClassList and >>>>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current >>>>>>>> implementation, DumpLoadedClassList will turn on property >>>>>>>> java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the >>>>>>>> same output sent to stdout and log file DumpLoadedClassList >>>>>>>> specified. >>>>>>>> >>>>>>> >>>>>>> These entries are duplicated in two different files: one for >>>>>>> jlink --generate-jli-classes plugin and another for CDS use.? >>>>>>> CDS -Xshare:dump attempts to do what jlink plugin does but >>>>>>> writes those generated classes in to shared archive. >>>>>>> >>>>>>> Like the above make logic to build JDK image, the same entries >>>>>>> are written in both default-jli-trace.txt via System.out and to >>>>>>> classlist via JNI call to the VM.? I guess VM also implements >>>>>>> the logic to do some kind of diffing and write to CDS archive. >>>>>>> >>>>>> In current implementation, vm side only records the line as from >>>>>> TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at >>>>>> dump time, call back to java for parsing those recordings and >>>>>> generating the holder classes, this uses the existing JLI code. >>>>>>>> >>>>>>>> Now instead of this property, using a vm interface API to query >>>>>>>> if this flag is set, I think it is better choice. But here I am >>>>>>>> NOT sure I understand your suggestion, I think there are two >>>>>>>> choices: >>>>>>>> >>>>>>>> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not >>>>>>>> via CDS_TRACE_RESOLVE, using new API to query if >>>>>>>> DumpLoadedClassList is set >>>>>>>> >>>>>>>> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting >>>>>>>> those name, type and holder name to regenerate holder classes? >>>>>>>> >>>>>>> >>>>>>> I misunderstood that this CDS_TRACE_RESOLVE flag is set during >>>>>>> -Xshare:dump time. >>>>>>> >>>>>>> Ioi has clarified to me offline that this step is actually part >>>>>>> of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs >>>>>>> to the given class list file, i.e. you repurpose the class list >>>>>>> file to include the log output that was initially designed for >>>>>>> jlink plugin. >>>>>>> >>>>>>> To me, I'd prefer to see this support depending on `jlink >>>>>>> --generate-jli-classes` which is an existing functionality and >>>>>>> much cleaner.? This way this does not require any VM change.?? >>>>>>> It will generate the holder classes in the custom image per the >>>>>>> application-specific config file. >>>>>>> >>>>>>> What it means is that: a customer would need to create a custom >>>>>>> image with their application-specific config file. It might need >>>>>>> a new CDS? option to specify a separate TRACE_RESOLVE file.?? It >>>>>>> would turn on this feature by default by defining a default path >>>>>>> of the log file if it helps. >>>>>>> >>>>>> So for now, I would implement an API to query if flag >>>>>> DumpLoadedClassList set in cmd line, remove new added property of >>>>>> java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address >>>>>> custom image with CDS in future in a separate issue. >>>>>> >>>>>> >>>>>> Thanks >>>>>> >>>>>> Yumin >>>>>> >>>>>>> I understand that this is not the existing CDS work flow and CDS >>>>>>> archive does not require to run on a custom image.?? I see the >>>>>>> value of this approach which can prepare customers to start >>>>>>> building and using its own custom image. >>>>>>> >>>>>>> Of course the implementation would be much simpler (adding a >>>>>>> flag to write these traces to a given file path, that is). >>>>>>> >>>>>>> Mandy >>>> >> From yumin.qi at oracle.com Wed Aug 26 00:17:51 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 25 Aug 2020 17:17:51 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <6344748b-f484-e743-8bef-5750d0b49155@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> <1ca66b6a-8a23-c4b1-c95d-10fc23b668f8@oracle.com> <6344748b-f484-e743-8bef-5750d0b49155@oracle.com> Message-ID: <68edf582-841b-890e-0951-8f8ecd5d132a@oracle.com> Hi, Coleen and all ? I have updated at new link: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-05/ ? I changed find_class and add_to_hierarchy to public access in SystemDictionary for access from outside. ? new added find_class removed from ClassLoaderData. ? new added add_shared_to_hierarchy removed from SystemDictionaryShared. ? re-tested local on runtime/cds ? Thanks ? Yumin On 8/25/20 12:41 PM, Coleen Phillimore wrote: > > Hi, I'm sorry I haven't been watching this change. > > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html > > Can you move find_class() into systemDictionary.hpp/cpp with the rest of the find_class().? This is strange here.? You don't want to expose that the dictionary is here to the rest of the code. The caller should call one of the SystemDictionary versions, or add this to SystemDictionary with the other functions with the same name. > > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/memory/lambdaFormInvokers.cpp.html > > These new files belongs in classfile.? It's closer to systemDictionaryShared code than memory code. > > ?? 2? * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All rights reserved. > > > Also copyright shouldn't start with 2012. > > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html > > This add_to_hierarchy is a strange delegation and a surprise that it's also defined here.? Please call this inline from lambdaFormInvokers, ie take the Compile_lock and call SystemDictionary::add_to_hierarchy(). > > I don't have any deeper comments. > > Thanks, > Coleen > > On 8/25/20 2:59 PM, Yumin Qi wrote: >> HI, Ioi >> >> ? Thanks for the re-review, updated webrev: >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/ >> >> ? Changed according to your suggestion, the only a little difference is move traceResolve(String line) to InvokerBytecodeGenerator as a static package public function so other package classes can call it. >> >> ? re-tested local on runtime/cds >> >> >> Thanks >> >> Yumin >> >> >> On 8/24/20 4:23 PM, Ioi Lam wrote: >>> Hi Yumin, >>> >>> This looks good overall. Here are my comments: >>> >>> ===================== >>> 6065???? size_t new_id = Atomic::add(&counter, (size_t)1); >>> 6066???? jio_snprintf(addr_buf, 20, INTPTR_FORMAT, new_id); >>> >>> I think this should be SIZE_FORMAT >>> >>> ===================== >>> ? 65 class KlassFactory : AllStatic { >>> ? 66 >>> ? 67?? // approved clients >>> ? 68?? friend class ClassLoader; >>> ? 69?? friend class ClassLoaderExt; >>> ? 70?? friend class SystemDictionary; >>> ? 71?? friend class LambdaFormInvokers; >>> ? 72 >>> ? 73? private: >>> ? 74?? static InstanceKlass* create_from_stream(ClassFileStream* stream, >>> >>> I think instead of adding everyone who uses create_from_stream as a friend class, we should just change create_from_stream into a public function and remove the friend declarations. >>> >>> ===================== >>> ?146?? // add to hierarchy and set state to loaded. >>> ?147?? { >>> ?148???? MutexLocker mu_r(THREAD, Compile_lock); // add_shared_to_hierarchy asserts this. >>> ?149 SystemDictionaryShared::add_shared_to_hierarchy(result, THREAD); >>> ?150?? } >>> >>> I think the function name can be changed to SystemDictionaryShared::add_to_hierarchy as the "_shared" seems redundant. The "set state to loaded" comment seems wrong, as we have the assert on line 1155. I think the comment can be removed. >>> >>> 1153 void SystemDictionaryShared::add_shared_to_hierarchy(InstanceKlass* k, TRAPS) { >>> 1154?? Arguments::assert_is_dumping_archive(); >>> 1155?? assert(!k->is_loaded(), "Invariant"); >>> 1156?? assert_locked_or_safepoint(Compile_lock); // add_to_hierarchy assert on it. >>> 1157?? SystemDictionary::add_to_hierarchy(k, CHECK); >>> 1158 } >>> >>> Also, I think it's better to move the MutexLocker call into SystemDictionaryShared::add_shared_to_hierarchy. >>> >>> ======================== >>> >>> before: >>> >>> ?478???????????????? if (TRACE_RESOLVE && salvage != null) { >>> ?479???????????????????? // Used by jlink species pregeneration plugin, see >>> ?480???????????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >>> ?481???????????????????? System.out.println("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>> ?482???????????????? } >>> >>> after: >>> >>> ?488???????????????? // Used by jlink species pregeneration plugin, see >>> ?489???????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >>> ?490???????????????? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>> >>> >>> When tracing is disabled, this will make extra allocations and cause a small slowdown. I think it's better to >>> >>> ??? if ((TRACE_RESOLVE|TRACE_RESOLVE_CDS) && salvage != null) { >>> ? ? ?? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>> ??? } >>> >>> Because TRACE_RESOLVE is a static final boolean, the JIT compiler will completely optimize this block out. >>> >>> For the same reason, instead of calling VM.isDumpLoadedClassListSetAndOpen() every time, it's better to use a static final variable. >>> >>> >>> ======================= >>> >>> ?698???????? if (TRACE_RESOLVE) { >>> ?699???????????? System.out.println("[LF_RESOLVE] " + holder.getName() + " " + name + " " + >>> ?700 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); >>> ?701???????? } >>> ?702???????? if (VM.isDumpLoadedClassListSetAndOpen()) { >>> ?703 GenerateJLIClassesHelper.cdsTraceResolve("[LF_RESOLVE] " + holder.getName() + " " + name + " " + >>> ?704 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); >>> ?705???????? } >>> ?706???????? return resolvedMember; >>> >>> >>> I think the two "if" blocks should be combined similarly to ClassSpecializer::traceResolve(). >>> >>> ========================= >>> ? 34 Java_java_lang_invoke_GenerateJLIClassesHelper_cdsTraceResolve(JNIEnv *env, jclass ignore, jstring line) { >>> >>> Maybe this should be moved to the "VM" class as well? >>> >>> ========================= >>> lambdaFormInvokers.hpp: >>> >>> Need these declarations: >>> >>> ??? #include "memory/allocation.hpp"??? <-- for AllStatic >>> ??? #include "runtime/handles.hpp"????? <-- for typeArrayHandle and Handle >>> ??? #include "utilities/exceptions.hpp" <-- for TRAPS >>> >>> ??? template class GrowableArray; >>> ========================= >>> >>> >>> Thanks >>> - Ioi >>> >>> >>> >>> On 8/23/20 3:56 PM, Yumin Qi wrote: >>>> >>>> Hi, Mandy, Ioi and Calvin >>>> >>>> >>>> ? I have updated the new changed at: >>>> >>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-03/ >>>> >>>> ? In this version: >>>> >>>> ??? 1) Added a new API to check if flag DumpLoadedClassList set and the file is open. If true, call into vm to print out the trace line to the log file. >>>> >>>> ??????? Just thinking if we just call the cdsTraceResolve without checking if the flag DumpLoadedClassList set and file open, this way, the check logic is in the vm side like before, so save code by not adding the new API. >>>> >>>> ??? 2) The returned holder class names now are just 'package/className', removed head and tail. >>>> >>>> ??? 3)? Remove add_extra_classes from CollectClassesClosure since after bug 8250990: https://bugs.openjdk.java.net/browse/JDK-8250990 pushed, the CollectClassesClosure no longer exist. >>>> >>>> ??? 4)? Still keep the parsing for TRACE_RESOLVE in java/lang/invoke/GenerateJLIClassesHelper.java, so VM call its function to regenerate holder classes. >>>> >>>> >>>> ? Re-tested Mach5 tier1-4 >>>> >>>> Thanks >>>> >>>> Yumin >>>> >>>> >>>> On 8/20/20 8:05 PM, Yumin Qi wrote: >>>>> Hi, Mandy >>>>> >>>>> >>>>> On 8/20/20 5:10 PM, Mandy Chung wrote: >>>>>> >>>>>> >>>>>> On 8/19/20 10:14 PM, Yumin Qi wrote: >>>>>>> >>>>>>> HI, Mandy >>>>>>> >>>>>>> ? Thanks for the review, I took one day off yesterday so just got a detail look of your reply. >>>>>>> >>>>>>> On 8/19/20 1:30 PM, Mandy Chung wrote: >>>>>>>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>>>>>>> Hi, Ioi >>>>>>>>> >>>>>>>>> ? Thanks for review/suggestion. I have updated the webrev at the following link: >>>>>>>>> >>>>>>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>>>>>>> >>>>>>>> >>>>>>>> This patch leverages the TRACE_RESOLVE output and passes the trace output to VM.? VM then calls GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing and generate Holder class per the resolved LFs.?? I think there are other cleaner alternatives implementing this.?? jlink --generate-jli-classes plugin depends the trace output whereas -Xshare:dump does not.?? It's cleaner to skip generating the trace output and parsing for dumping shared archive purpose.? In addition, the implementation needs some cleanup (I can send you feedback on the next revision) >>>>>>>> >>>>>>> Current patch did not change the existing code for JLinkPlugin part. I just moved the parsing code from GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java since the former is an internal class to which we shouldn't call to generate holder classes. >>>>>>>> Instead of relying on a system property "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. >>>>>>> >>>>>>> I remember we have such API to query if flag -Xshare:dump or -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This flag is the one used to log class name to list file. In GenerateLinkOptData.gmk: >>>>>>> >>>>>>> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR) >>>>>>> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >>>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) >>>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $(JLI_TRACE_FILE))) >>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw \ >>>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >>>>>>> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >>>>>>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>>>>>> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw.2 \ >>>>>>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >>>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>>> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>> ??????????? build.tools.classlist.HelloClasslist \ >>>>>>> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >>>>>>> ??????????? || ( \ >>>>>>> ??????????????? exitcode=$$? ; \ >>>>>>> ??????????????? $(ECHO) "ERROR: Failed to generate link optimization data." \ >>>>>>> ??????????????????? "This is likely a problem with the newly built JVM/JDK." ; \ >>>>>>> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >>>>>>> ??????????????? exit $$exitcode \ >>>>>>> ??????????? ) >>>>>>> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >>>>>>> >>>>>>> The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current implementation, DumpLoadedClassList will turn on property java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same output sent to stdout and log file DumpLoadedClassList specified. >>>>>>> >>>>>> >>>>>> These entries are duplicated in two different files: one for jlink --generate-jli-classes plugin and another for CDS use.? CDS -Xshare:dump attempts to do what jlink plugin does but writes those generated classes in to shared archive. >>>>>> >>>>>> Like the above make logic to build JDK image, the same entries are written in both default-jli-trace.txt via System.out and to classlist via JNI call to the VM.? I guess VM also implements the logic to do some kind of diffing and write to CDS archive. >>>>>> >>>>> In current implementation, vm side only records the line as from TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at dump time, call back to java for parsing those recordings and generating the holder classes, this uses the existing JLI code. >>>>>>> >>>>>>> Now instead of this property, using a vm interface API to query if this flag is set, I think it is better choice. But here I am NOT sure I understand your suggestion, I think there are two choices: >>>>>>> >>>>>>> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set >>>>>>> >>>>>>> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting those name, type and holder name to regenerate holder classes? >>>>>>> >>>>>> >>>>>> I misunderstood that this CDS_TRACE_RESOLVE flag is set during -Xshare:dump time. >>>>>> >>>>>> Ioi has clarified to me offline that this step is actually part of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the given class list file, i.e. you repurpose the class list file to include the log output that was initially designed for jlink plugin. >>>>>> >>>>>> To me, I'd prefer to see this support depending on `jlink --generate-jli-classes` which is an existing functionality and much cleaner.? This way this does not require any VM change.?? It will generate the holder classes in the custom image per the application-specific config file. >>>>>> >>>>>> What it means is that: a customer would need to create a custom image with their application-specific config file. It might need a new CDS? option to specify a separate TRACE_RESOLVE file.?? It would turn on this feature by default by defining a default path of the log file if it helps. >>>>>> >>>>> So for now, I would implement an API to query if flag DumpLoadedClassList set in cmd line, remove new added property of java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address custom image with CDS in future in a separate issue. >>>>> >>>>> >>>>> Thanks >>>>> >>>>> Yumin >>>>> >>>>>> I understand that this is not the existing CDS work flow and CDS archive does not require to run on a custom image.?? I see the value of this approach which can prepare customers to start building and using its own custom image. >>>>>> >>>>>> Of course the implementation would be much simpler (adding a flag to write these traces to a given file path, that is). >>>>>> >>>>>> Mandy >>> > From ioi.lam at oracle.com Wed Aug 26 02:04:30 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 25 Aug 2020 19:04:30 -0700 Subject: RFR(S) Remove excessive inclusion of arguments.hpp In-Reply-To: <26141468-6793-40c2-dd48-57caf2bcd279@oracle.com> References: <09c2f053-2c61-5b7d-ea83-18c1db3c2b8e@oracle.com> <26141468-6793-40c2-dd48-57caf2bcd279@oracle.com> Message-ID: Thanks Coleen and Thomas for the review. - Ioi On 8/25/20 2:00 PM, Coleen Phillimore wrote: > +1 > Coleen > > On 8/25/20 12:03 AM, Thomas St?fe wrote: >> Hi Ioi, >> >> thanks for the explanation. Looks all good to me. >> >> ..Thomas >> >> On Mon, Aug 24, 2020 at 7:16 PM Ioi Lam wrote: >> >>> Hi Thomas, thanks for the review. >>> >>> Some cpp files use vm_exit_during_initialization(), which is defined in >>> java.hpp, which was included by arguments.hpp. >>> >>> Now that I have removed arguments.hpp (indirectly) from these cpp >>> files, I >>> have to add java.hpp to fix the missing dependency. >>> >>> Thanks >>> - Ioi >>> >>> On 8/23/20 1:31 AM, Thomas St?fe wrote: >>> >>> Hi Ioi, >>> >>> looks okay, but why the added java.hpp includes? At first glance >>> does not >>> seem related to your bug description. >>> >>> Thanks Thomas >>> >>> On Fri, Aug 21, 2020 at 6:47 PM Ioi Lam wrote: >>> >>>> https://bugs.openjdk.java.net/browse/JDK-8252151 >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk16/8252151-dont-include-arguments-hpp.v01/ >>>> >>>> >>>> The large header file arguments.hpp is included by popular header >>>> files >>>> such as constMethod.hpp and methodCounters.hpp (which are included by >>>> method.hpp, which is included by almost every cpp file). >>>> >>>> The dependencies on arguments.hpp by constMethod.hpp are very >>>> small, and >>>> can be moved to constMethod.cpp. >>>> >>>> methodCounters.hpp does not depend on arguments.hpp >>>> >>>> === >>>> by removing these dependencies, we can reduce the total number of >>>> include files in the hotspot build from 530,921 to 523,010, or a >>>> reduction of about 1.5%. >>>> >>>> >>>> > From thomas.stuefe at gmail.com Wed Aug 26 06:31:19 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 26 Aug 2020 08:31:19 +0200 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: <4f7a8afb-4207-f92a-14ea-8e212ea239db@oracle.com> References: <4f7a8afb-4207-f92a-14ea-8e212ea239db@oracle.com> Message-ID: Hi Leo, On Tue, Aug 25, 2020 at 4:29 PM Leo Korinth wrote: > Hi, review of: freeBlocks.?pp, blockTree.?pp and binList.hpp below > > I have some comments in addition to those from Coleen: > > ### freeBlocks.hpp: > > 75 > 76 static const size_t splinter_threshold = 0;// 0x100; > 77 > > splinter_threshold with the strange comment // 0X100, what is it > supposed to be used for? (always 0, but with a comment 0x100;). The splinter threshold was supposed to aid in the decision of whether it is worth the trouble preserving the remainder of a block. Lets say someone needs 100 words, I only find a 120 word block in the tree. I hand this to the caller, but do I chop off the remaining 20 blocks and re-store them or do I waste 20 blocks? This is handled in FreeBlocks::get_block(). After many tests, it turned out to be just better to always chop off the remaining space instead of wasting it. Especially since if it is small enough it falls under the responsibility of the BinList, which has O(1) for insert. That is also the reason why a single threshold value for both tree and binlist is not a good fit anyway. Therefore last week I removed the threshold value. Makes the coding also easier to read. > Maybe > related to this, is that I have problems to understand how allocation > guards are handled when memory is put back to the freelist, I can not > see the pointer being decremented (the inverse operation of > payload_from_prefix, prefix_from_payload seems never to be used, and I > fail to find the operation done manually either. > Not related to that, but you are correct. Switching on include guards switches off deallocation, see settings.cpp:109ff. So, FreeBlocks does not get used with guards. Which means we lose deallocated blocks, but the guards are a debug feature only and in most scenarios deallocation plays a small role. I could make guards work with deallocation too; I did not do so since which would have made the freeblock handling more complicated. Since I was not sure how the guards would be received by Reviewers, and since they are a non-essential feature, I wanted to wait for feedback before putting in further work. (Note that despite this shortcoming the guards work just fine). About prefix_from_payload - you are right, that can be removed. > > 68 typedef BinList32 SmallBlocksType; > 69 > 70 // _small_blocks takes care of small to very small blocks. > 71 SmallBlocksType _small_blocks; > > > I think all of BinListXX typedefs can be removed and also the > SmallBlocksType typedef. The SmallBlocksType is only used once if we > rewrite > > const static size_t minimal_word_size = SmallBlocksType::minimal_word_size; > to > const static size_t minimal_word_size = _small_blocks.minimal_word_size; > > This would not compile: msFreeBlocks.hpp:79:43: error: invalid use of non-static data member 'metaspace::FreeBlocks::_small_blocks' > (and also change the name of minimal_word_size so that we can see it > is a constant) > > How? > 84 > 85 // Retrieve a block of at least requested_word_size. > 86 MetaWord* get_block(size_t requested_word_size); > 87 > > The function named get_block, this is not a getter, this mutates the > object. I would prefer remove_block (goes well with add_block), and I > would like that rename to propagate to both blockTree and binList. > > My only misgiving would be that remove_block evokes a function which takes a block reference and removes it, not one that finds a block and removes it. But any alternative I can come up with ("find_and_remove_block" e.g) is too much of a mouthful. Okay, remove_block it is. > ### binList.hpp: > > Maybe another name for binList, binLookup for example? When I hear > list I am thinking /linked/ list and that might just be my problem, > sorry if that is that case, and then ignore this point. > > I'd rather keep that name. "BinList" is very roughly inspired by the fast bins in glibc heap. It is also a list, since these are threaded blocks. Strictly speaking it would be a BinListVector - a vector of list heads. If your heart is set on a name change, I'd use that one. > 76 > 77 template > 78 class BinListImpl { > 79 > > Template argument smallest_size would be nice if the unit of the size > (words) is mentioned in the name. It is so easy to mix sizes up. > Agreed. > > 82 // a mask to speed up searching for populated lists. > 83 // 0 marks an empty list, 1 for a non-empty one. > 84 typedef uint32_t mask_t; > > Is a mask really speeding up anything? Seems the code would be a bit > simpler removing the mask. > > I'll check. > 158 // Given a word_size, searches and returns a block of at least that > size. > 159 // Block may be larger. Real block size is returned in > *p_real_word_size. > 160 MetaWord* get_block(size_t word_size, size_t* p_real_word_size) { > get_block -> remove_block > > 217 typedef BinListImpl<2, 8> BinList8; > 218 typedef BinListImpl<2, 16> BinList16; > 219 typedef BinListImpl<2, 32> BinList32; > 220 typedef BinListImpl<2, 64> BinList64; > > BinListXX typedefs could be removed I think. (see comment above) > > Okay. > > ### blocktree.hpp > > 79 struct node_t { > 80 > 81 // Normal tree node stuff... > 82 node_t* parent; > 83 node_t* left; > 84 node_t* right; > 85 > > > I think in the future we could make the node_t without a parent > pointer, that would make the node size somewhat smaller and we could > then be able to fit smaller memory areas into the tree. The way the tree is used node size does not really matter since into the tree go only larger blocks, smaller blocks are handled by the BinList. It would be a factor if we wanted to generalize the tree, but otherwise I think this is unnecessary. You need the parent pointer for searching predecessor/successor nodes in a BST, as well as for modifying the parent when removing a node. I do not see offhand how this could be done without a parent pointer (maybe, when walking, build up a stack of visited nodes - assuming all operations always start at root - but I think that coding would be more complicated and non-standard). > The code > should be somewhat simpler as well I think. Not sure how; I tried to dumb it down as much as possible. In the end it's pretty standard stuff. I'm open for ideas of course, but would prefer to do it as a follow up. > Maybe file a future > improvement for this? > > 98 // Largest node size, (bit arbitrarily) capped at 4M since we know > this to > 99 // be the max possible metaspace allocation size. TODO. Do this > better. > 100 const static size_t maximal_word_size = 4 * M; > > maximal_word_size should take its value from MAX_CHUNK_BYTE_SIZE in > some way if it is not too hard. > > Agreed. > 102 // We need nodes to be at least large enough to hold a node_t > 103 const static size_t minimal_word_size = > 104 (sizeof(node_t) + sizeof(MetaWord) - 1) / sizeof(MetaWord); > > maybe add a division-ceil comment? > > Okay. I could also use align_up, that may be clearer. No wait, I think I cannot use align_up in constant expressions :/ I'll add a comment. > 172 // Given a node n, return its predecessor in the tree > 173 // (node with the next-smaller size). > 174 static node_t* successor(node_t* n) { > > The comment is wrong on successor (I guess it is copied from predecessor). > > Good catch. > 199 if (parent != NULL) { > 200 if (parent->left == child) { // I am a left child > > I think "// child is a left child" is better > > Ok > 215 // Given a node n and a node forebear, insert n under forebear > 216 void insert(node_t* forebear, node_t* n) { > > The name forebear confused me, it is just a root for > insertion, and has no existing ancestor relation to "n". > > Well, we talk about parent and child nodes, in that terminology a removed-by-n parent of a child would be a distant forebear. But if it confuses you it will confuse others too. I'll change the name to "insertion_root". > 230 if (_largest_size_added < n->size) { > 231 _largest_size_added = n->size; > 232 } > > This is unnecessary as it is already done by add_block(MetaWord* p, > size_t word_size). I also think that add_block is the better place to > have the update. The method maybe could be static after this change? I > also think the opposite decrease update should be done in remove_block > to keep the symmetry. > Good catch, but I think otherwise: the better place would be consistently in the lower regions, in insert(), since this bookkeeping is only necessary when adding a new unique block size. Then, for symmetry, it should happen in remove_block_from_tree. Also, I rather keep bookkeeping functionality as close as possible to where the mutation happens. Among other reasons because the code may change at some point, someone calls remove_node_from_tree() from somewhere else, bookkeeping has to be still correct. > There are at least two recursive calls of insert that could be > tail-called instead (it would be somewhat harder to read, so I am not > proposing it). However, with an unbalanced tree I guess we > could get deep call stacks if we are unlucky. I guess it is best to > leave this as is, just wanted to comment on it. > > Ok > 278 // Given a node n, remove it from the tree and repair tree. > 279 void remove_node_from_tree(node_t* n) { > 280 > 281 assert(n->next == NULL, "do not delete a node which has a > non-empty list"); > > Is there any reason for not calling the remove_from_list logic > here instead of in get_block? We would get rid of this assert, and I > think it would be better placed logically. > > Hmm I rather like methods which have only one job. We have remove_from_tree and remove_from_list. I can add a multiplexer function, e.g. remove() (which would be symmetric with insert()) in between get_block and remove_from_tree/list. I'll play around a bit. > 283 // Maintain largest size node to speed up lookup > 284 if (n->size == _largest_size_added) { > 285 node_t* pred = predecessor(n); > 286 if (pred != NULL) { > 287 _largest_size_added = pred->size; > 288 } else { > 289 _largest_size_added = 0; > 290 } > 291 } > > As I said above, the _largest_size_added logic seems (to me) to fit > better in remove_block(). It is not really a tree operation but a cached > result. > > See remarks above. > I also wonder if the caching is really needed. > > If we have a huge tree, /and/ we fail to find a free area > /repeatedly/, the caching will perform better. > > But If the tree is empty, it will be as fast to check anyway, and I > guess that is the common case). > > Every time we remove an element from the tree, the predecessor() > function needs to be called at the end (just) to update the cached > value, so each remove (that successfully fetch free memory) is a bit > more expensive with caching. > > Just a thought. > Maybe, I'll think about this. It was thought to be a safety valve. However, I think now that a different form of valve could be a "good enough" logic. When removing a block from a tree, it is not highly important to have an ideal match. We will retrieve a block and chop off the remainder to be reused later. Especially if the remainder space is small enough to fit into the binlist (O(1) insertion) it is already a very good fit. So I may just add a stop at an arbitrary depth. If we are e.g. 30 levels deep in this tree something has gone deeply wrong and the tree has degenerated. May just as well take that block then and use it. Even if there is too much remaining space, and if it will be re-added to the tree, it will end up in a different tree branch and thus reform the tree. I still think a better fit would be a rbtree, like the linux kernel has. I have a prototype somewhere but did not get it waterproof, then ran out of time. It also seemed too big a gun for just this one use case. > 379 node_t* n = (node_t*)p; > 380 n->size = word_size; > 381 n->next = n->left = n->right = n->parent = NULL; > > Placement new would look better in my opinion. > Okay, I'll change that. > > Thanks, > Leo > Thanks for the review! Thomas From nick.gasson at arm.com Wed Aug 26 09:35:09 2020 From: nick.gasson at arm.com (Nick Gasson) Date: Wed, 26 Aug 2020 17:35:09 +0800 Subject: RFR: 8252108: Modify nsk/stress/stack tests to check page size In-Reply-To: <6ae728e4-74e8-9b68-8e28-3125af83253c@oracle.com> References: <85wo1ne7he.fsf@nicgas01-pc.shanghai.arm.com> <6ae728e4-74e8-9b68-8e28-3125af83253c@oracle.com> Message-ID: <85r1rteomq.fsf@nicgas01-pc.shanghai.arm.com> Thank you for the reviews, pushed. -- Nick On 08/25/20 20:31 pm, Harold Seigel wrote: > Hi Nick, > > The changes look good. > > Thanks for doing this. > > Harold > > On 8/24/2020 11:21 PM, Nick Gasson wrote: >> Hi, >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8252108 >> Webrev: http://cr.openjdk.java.net/~ngasson/8252108/webrev.1/ >> >> As discussed on the review thread for JDK-8251490 [1], it's better to >> have these tests enabled conditionally on 4K page size, rather than >> always disabled on AArch64, since the original issue was only present on >> AArch64 systems configured with 64K pages. >> >> Added a new "vm.pageSize" jtreg property that can be used in @requires. >> >> stack018.java intermittently times out with the increased -Xss448K so >> I've lowered this to -Xss220k and added @requires vm.pageSize == >> 4096 (the original value -Xss200k fails with "Maximal recursion depth: >> 0" on AArch64 fastdebug build). >> >> [1] https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-August/041299.html >> >> -- >> Thanks, >> Nick From Alan.Bateman at oracle.com Wed Aug 26 10:46:57 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 26 Aug 2020 11:46:57 +0100 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: References: <9e6b0043-65a1-dd97-a3d2-33679c8048d4@oracle.com> Message-ID: <1280b9d5-265b-f0d4-b606-75dc93758d41@oracle.com> On 12/08/2020 23:06, Ioi Lam wrote: > Hi Lois, > > Thanks for the comments. I have an updated webrev > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/ > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02.delta/ > > > Here are the general notes on the changes. Replies to your questions > are in-line below. > > (1) Integrated updates in the Java code from Alan Bateman. Thanks for incorporating these changes, I don't have any other comments on the module changes. I see Mandy has reviewed that part too. -Alan From albert.m.yang at oracle.com Wed Aug 26 13:59:14 2020 From: albert.m.yang at oracle.com (Albert Yang) Date: Wed, 26 Aug 2020 15:59:14 +0200 Subject: RFR: 8252093: formula used to calculate decaying variance in numberSeq Message-ID: <54b1cc85-9f2d-cdf8-c513-1cbee4fa7f3d@oracle.com> Hi, I am sending this to both runtime-dev and gc-dev, since the changed code lives in runtime, but it's used by GC. https://bugs.openjdk.java.net/browse/JDK-8252093 http://cr.openjdk.java.net/~ayang/webrevs/8252093/webrev.1/ Both the code and the literature use `alpha`, but its meaning is actually inverted. I could include this refactoring in later revision if people feel that it's desirable. Tested: tier1 (locally) and tier1-3 (mach5) -- /Albert From calvin.cheung at oracle.com Wed Aug 26 16:40:34 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Wed, 26 Aug 2020 09:40:34 -0700 Subject: RFR 8251557 Avoid dumping unused symbols/strings into the CDS archive In-Reply-To: <6d64004b-4aeb-f95f-1551-d6d608191f3a@oracle.com> References: <6d64004b-4aeb-f95f-1551-d6d608191f3a@oracle.com> Message-ID: <699f0beb-5272-8ca8-a644-5383c4c4f8e2@oracle.com> Hi Ioi, The proposed change does make the code easier to read than before. I've comments on 2 files: heapShared.hpp: 42 class DumpedInternedStrings; Instead of forward declaration, is it possible to move the class definition there since it is defined at the end of the file? archiveBuilder.hpp: 37 class DumpRegion; The above forward declaration was removed but I think it's required for the following: ?132?? DumpRegion* _rw_region; ?133?? DumpRegion* _ro_region; Also, the following constructor needs to be moved to archiveBuilder.cpp or I got compile error about invalid use of incomplete type 'class DumpRegion'. ?158???? OtherROAllocMark() { ?159?????? _oldtop = _singleton->_ro_region->top(); ?160???? } thanks, Calvin On 8/19/20 2:39 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8251557 > http://cr.openjdk.java.net/~iklam/jdk16/8251557-cds-dont-dump-unused-symbols.v01/ > > > Part of CDS code clean up: > > Currently all entries in the SymbolTable/StringTable are dumped into > the CDS archive. The problems are > > (1) We end up writing many unused items, such as the mangled names > ? ? of hidden classes. > (2) We have to scan the symbol table inside a safepoint, which had > ? ? caused bugs before (JDK-8245264). > > Since JDK-8250990, we already maintain all used Symbols in a growable > array. We can dump the shared symbol table using this array, and > avoid walking the SymbolTable. > > For StringTable, we scan all archived classes, and dump only the > interned strings used by those classes (plus extra strings > specified by SharedArchiveConfigFile) into the CDS interned string > table. > > This reduces complexity in the CDS code, and reduce the size of > the CDS archives. > > Testing - tiers 1-4 > > Archive file size for "java -Xshare:dump" > > ?? old: 39,368 symbols 7,028 strings 11,784,200 bytes > ?? new: 37,387 symbols 7,020 strings 11,683,320 bytes (~0.9% reduction) > > Thanks > - Ioi > > From ioi.lam at oracle.com Wed Aug 26 17:08:05 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 26 Aug 2020 10:08:05 -0700 Subject: RFR 8251557 Avoid dumping unused symbols/strings into the CDS archive In-Reply-To: <699f0beb-5272-8ca8-a644-5383c4c4f8e2@oracle.com> References: <6d64004b-4aeb-f95f-1551-d6d608191f3a@oracle.com> <699f0beb-5272-8ca8-a644-5383c4c4f8e2@oracle.com> Message-ID: Hi Calvin, Thanks for the review. On 8/26/20 9:40 AM, calvin.cheung at oracle.com wrote: > Hi Ioi, > > The proposed change does make the code easier to read than before. > > I've comments on 2 files: > > heapShared.hpp: > > 42 class DumpedInternedStrings; > > Instead of forward declaration, is it possible to move the class > definition there since it is defined at the end of the file? > I moved it as you suggested. > archiveBuilder.hpp: > > 37 class DumpRegion; > > The above forward declaration was removed but I think it's required > for the following: > I removed the forward declaration because DumpRegion is declared in archiveUtils.hpp. Since JDK-8252056, archiveUtils.hpp is already included by archiveBuilder.hpp: http://hg.openjdk.java.net/jdk/jdk/rev/ba51bc0772ac > ?132 DumpRegion* _rw_region; > ?133?? DumpRegion* _ro_region; > > Also, the following constructor needs to be moved to > archiveBuilder.cpp or I got compile error about invalid use of > incomplete type 'class DumpRegion'. > > ?158???? OtherROAllocMark() { > ?159?????? _oldtop = _singleton->_ro_region->top(); > ?160???? } > Maybe you? applied the patch on a repo without the JDK-8252056 fix? I did find a problem with archiveUtils.hpp. It was using the Klass type without including klass.hpp, so I added diff -r 3920d1a29976 src/hotspot/share/memory/archiveBuilder.hpp --- a/src/hotspot/share/memory/archiveBuilder.hpp??? Wed Aug 19 14:14:20 2020 -0700 +++ b/src/hotspot/share/memory/archiveBuilder.hpp??? Wed Aug 26 10:06:59 2020 -0700 @@ -27,6 +27,7 @@ ?#include "memory/archiveUtils.hpp" ?#include "memory/metaspaceClosure.hpp" +#include "oops/klass.hpp" ?#include "utilities/bitMap.hpp" ?#include "utilities/growableArray.hpp" ?#include "utilities/hashtable.hpp" Thanks - Ioi > thanks, > Calvin > > > On 8/19/20 2:39 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8251557 >> http://cr.openjdk.java.net/~iklam/jdk16/8251557-cds-dont-dump-unused-symbols.v01/ >> >> >> Part of CDS code clean up: >> >> Currently all entries in the SymbolTable/StringTable are dumped into >> the CDS archive. The problems are >> >> (1) We end up writing many unused items, such as the mangled names >> ? ? of hidden classes. >> (2) We have to scan the symbol table inside a safepoint, which had >> ? ? caused bugs before (JDK-8245264). >> >> Since JDK-8250990, we already maintain all used Symbols in a growable >> array. We can dump the shared symbol table using this array, and >> avoid walking the SymbolTable. >> >> For StringTable, we scan all archived classes, and dump only the >> interned strings used by those classes (plus extra strings >> specified by SharedArchiveConfigFile) into the CDS interned string >> table. >> >> This reduces complexity in the CDS code, and reduce the size of >> the CDS archives. >> >> Testing - tiers 1-4 >> >> Archive file size for "java -Xshare:dump" >> >> ?? old: 39,368 symbols 7,028 strings 11,784,200 bytes >> ?? new: 37,387 symbols 7,020 strings 11,683,320 bytes (~0.9% reduction) >> >> Thanks >> - Ioi >> >> From ioi.lam at oracle.com Wed Aug 26 17:18:31 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 26 Aug 2020 10:18:31 -0700 (PDT) Subject: RFR 8251557 Avoid dumping unused symbols/strings into the CDS archive In-Reply-To: References: <6d64004b-4aeb-f95f-1551-d6d608191f3a@oracle.com> <699f0beb-5272-8ca8-a644-5383c4c4f8e2@oracle.com> Message-ID: <69beaf48-8c3e-bbd8-d964-99d5becd7e1b@oracle.com> On 8/26/20 10:08 AM, Ioi Lam wrote: > Hi Calvin, > > Thanks for the review. > > On 8/26/20 9:40 AM, calvin.cheung at oracle.com wrote: >> Hi Ioi, >> >> The proposed change does make the code easier to read than before. >> >> I've comments on 2 files: >> >> heapShared.hpp: >> >> 42 class DumpedInternedStrings; >> >> Instead of forward declaration, is it possible to move the class >> definition there since it is defined at the end of the file? >> > > I moved it as you suggested. Oops, sorry I hit "Send" too fast. DumpedInternedStrings depends on function declared by HeapShared, so it must be declared after HeapShared Thanks - Ioi > >> archiveBuilder.hpp: >> >> 37 class DumpRegion; >> >> The above forward declaration was removed but I think it's required >> for the following: >> > > I removed the forward declaration because DumpRegion is declared in > archiveUtils.hpp. Since JDK-8252056, archiveUtils.hpp is already > included by archiveBuilder.hpp: > > http://hg.openjdk.java.net/jdk/jdk/rev/ba51bc0772ac > >> ?132 DumpRegion* _rw_region; >> ?133?? DumpRegion* _ro_region; >> >> Also, the following constructor needs to be moved to >> archiveBuilder.cpp or I got compile error about invalid use of >> incomplete type 'class DumpRegion'. >> >> ?158???? OtherROAllocMark() { >> ?159?????? _oldtop = _singleton->_ro_region->top(); >> ?160???? } >> > Maybe you? applied the patch on a repo without the JDK-8252056 fix? > > I did find a problem with archiveUtils.hpp. It was using the Klass > type without including klass.hpp, so I added > > diff -r 3920d1a29976 src/hotspot/share/memory/archiveBuilder.hpp > --- a/src/hotspot/share/memory/archiveBuilder.hpp??? Wed Aug 19 > 14:14:20 2020 -0700 > +++ b/src/hotspot/share/memory/archiveBuilder.hpp??? Wed Aug 26 > 10:06:59 2020 -0700 > @@ -27,6 +27,7 @@ > > ?#include "memory/archiveUtils.hpp" > ?#include "memory/metaspaceClosure.hpp" > +#include "oops/klass.hpp" > ?#include "utilities/bitMap.hpp" > ?#include "utilities/growableArray.hpp" > ?#include "utilities/hashtable.hpp" > > > Thanks > - Ioi > > >> thanks, >> Calvin >> >> >> On 8/19/20 2:39 PM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8251557 >>> http://cr.openjdk.java.net/~iklam/jdk16/8251557-cds-dont-dump-unused-symbols.v01/ >>> >>> >>> Part of CDS code clean up: >>> >>> Currently all entries in the SymbolTable/StringTable are dumped into >>> the CDS archive. The problems are >>> >>> (1) We end up writing many unused items, such as the mangled names >>> ? ? of hidden classes. >>> (2) We have to scan the symbol table inside a safepoint, which had >>> ? ? caused bugs before (JDK-8245264). >>> >>> Since JDK-8250990, we already maintain all used Symbols in a growable >>> array. We can dump the shared symbol table using this array, and >>> avoid walking the SymbolTable. >>> >>> For StringTable, we scan all archived classes, and dump only the >>> interned strings used by those classes (plus extra strings >>> specified by SharedArchiveConfigFile) into the CDS interned string >>> table. >>> >>> This reduces complexity in the CDS code, and reduce the size of >>> the CDS archives. >>> >>> Testing - tiers 1-4 >>> >>> Archive file size for "java -Xshare:dump" >>> >>> ?? old: 39,368 symbols 7,028 strings 11,784,200 bytes >>> ?? new: 37,387 symbols 7,020 strings 11,683,320 bytes (~0.9% reduction) >>> >>> Thanks >>> - Ioi >>> >>> > From calvin.cheung at oracle.com Wed Aug 26 18:13:54 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Wed, 26 Aug 2020 11:13:54 -0700 Subject: RFR 8251557 Avoid dumping unused symbols/strings into the CDS archive In-Reply-To: <69beaf48-8c3e-bbd8-d964-99d5becd7e1b@oracle.com> References: <6d64004b-4aeb-f95f-1551-d6d608191f3a@oracle.com> <699f0beb-5272-8ca8-a644-5383c4c4f8e2@oracle.com> <69beaf48-8c3e-bbd8-d964-99d5becd7e1b@oracle.com> Message-ID: <38a3d6da-c5ea-b2cd-50c7-6910d06a3b7f@oracle.com> On 8/26/20 10:18 AM, Ioi Lam wrote: > > > On 8/26/20 10:08 AM, Ioi Lam wrote: >> Hi Calvin, >> >> Thanks for the review. >> >> On 8/26/20 9:40 AM, calvin.cheung at oracle.com wrote: >>> Hi Ioi, >>> >>> The proposed change does make the code easier to read than before. >>> >>> I've comments on 2 files: >>> >>> heapShared.hpp: >>> >>> 42 class DumpedInternedStrings; >>> >>> Instead of forward declaration, is it possible to move the class >>> definition there since it is defined at the end of the file? >>> >> >> I moved it as you suggested. > > Oops, sorry I hit "Send" too fast. DumpedInternedStrings depends on > function declared by HeapShared, so it must be declared after HeapShared I see. So the change looks good. > > Thanks > - Ioi > > >> >>> archiveBuilder.hpp: >>> >>> 37 class DumpRegion; >>> >>> The above forward declaration was removed but I think it's required >>> for the following: >>> >> >> I removed the forward declaration because DumpRegion is declared in >> archiveUtils.hpp. Since JDK-8252056, archiveUtils.hpp is already >> included by archiveBuilder.hpp: >> >> http://hg.openjdk.java.net/jdk/jdk/rev/ba51bc0772ac >> >>> ?132 DumpRegion* _rw_region; >>> ?133?? DumpRegion* _ro_region; >>> >>> Also, the following constructor needs to be moved to >>> archiveBuilder.cpp or I got compile error about invalid use of >>> incomplete type 'class DumpRegion'. >>> >>> ?158???? OtherROAllocMark() { >>> ?159?????? _oldtop = _singleton->_ro_region->top(); >>> ?160???? } >>> >> Maybe you? applied the patch on a repo without the JDK-8252056 fix? Right, I haven't updated my repo with the JDK-8252056 fix yet. >> >> I did find a problem with archiveUtils.hpp. It was using the Klass >> type without including klass.hpp, so I added >> >> diff -r 3920d1a29976 src/hotspot/share/memory/archiveBuilder.hpp >> --- a/src/hotspot/share/memory/archiveBuilder.hpp??? Wed Aug 19 >> 14:14:20 2020 -0700 >> +++ b/src/hotspot/share/memory/archiveBuilder.hpp??? Wed Aug 26 >> 10:06:59 2020 -0700 >> @@ -27,6 +27,7 @@ >> >> ?#include "memory/archiveUtils.hpp" >> ?#include "memory/metaspaceClosure.hpp" >> +#include "oops/klass.hpp" >> ?#include "utilities/bitMap.hpp" >> ?#include "utilities/growableArray.hpp" >> ?#include "utilities/hashtable.hpp" >> Since you're including klass.hpp, maybe the following forward declaration can be removed? 36 class Klass; thanks, Calvin >> >> Thanks >> - Ioi >> >> >>> thanks, >>> Calvin >>> >>> >>> On 8/19/20 2:39 PM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8251557 >>>> http://cr.openjdk.java.net/~iklam/jdk16/8251557-cds-dont-dump-unused-symbols.v01/ >>>> >>>> >>>> Part of CDS code clean up: >>>> >>>> Currently all entries in the SymbolTable/StringTable are dumped into >>>> the CDS archive. The problems are >>>> >>>> (1) We end up writing many unused items, such as the mangled names >>>> ? ? of hidden classes. >>>> (2) We have to scan the symbol table inside a safepoint, which had >>>> ? ? caused bugs before (JDK-8245264). >>>> >>>> Since JDK-8250990, we already maintain all used Symbols in a growable >>>> array. We can dump the shared symbol table using this array, and >>>> avoid walking the SymbolTable. >>>> >>>> For StringTable, we scan all archived classes, and dump only the >>>> interned strings used by those classes (plus extra strings >>>> specified by SharedArchiveConfigFile) into the CDS interned string >>>> table. >>>> >>>> This reduces complexity in the CDS code, and reduce the size of >>>> the CDS archives. >>>> >>>> Testing - tiers 1-4 >>>> >>>> Archive file size for "java -Xshare:dump" >>>> >>>> ?? old: 39,368 symbols 7,028 strings 11,784,200 bytes >>>> ?? new: 37,387 symbols 7,020 strings 11,683,320 bytes (~0.9% >>>> reduction) >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >> > From ioi.lam at oracle.com Wed Aug 26 18:30:07 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 26 Aug 2020 11:30:07 -0700 Subject: RFR 8251557 Avoid dumping unused symbols/strings into the CDS archive In-Reply-To: <38a3d6da-c5ea-b2cd-50c7-6910d06a3b7f@oracle.com> References: <6d64004b-4aeb-f95f-1551-d6d608191f3a@oracle.com> <699f0beb-5272-8ca8-a644-5383c4c4f8e2@oracle.com> <69beaf48-8c3e-bbd8-d964-99d5becd7e1b@oracle.com> <38a3d6da-c5ea-b2cd-50c7-6910d06a3b7f@oracle.com> Message-ID: <6db5f819-8dba-84d6-961a-d03841858a9a@oracle.com> On 8/26/20 11:13 AM, calvin.cheung at oracle.com wrote: > > On 8/26/20 10:18 AM, Ioi Lam wrote: >> >> >> On 8/26/20 10:08 AM, Ioi Lam wrote: >>> Hi Calvin, >>> >>> Thanks for the review. >>> >>> On 8/26/20 9:40 AM, calvin.cheung at oracle.com wrote: >>>> Hi Ioi, >>>> >>>> The proposed change does make the code easier to read than before. >>>> >>>> I've comments on 2 files: >>>> >>>> heapShared.hpp: >>>> >>>> 42 class DumpedInternedStrings; >>>> >>>> Instead of forward declaration, is it possible to move the class >>>> definition there since it is defined at the end of the file? >>>> >>> >>> I moved it as you suggested. >> >> Oops, sorry I hit "Send" too fast. DumpedInternedStrings depends on >> function declared by HeapShared, so it must be declared after HeapShared > I see. So the change looks good. >> >> Thanks >> - Ioi >> >> >>> >>>> archiveBuilder.hpp: >>>> >>>> 37 class DumpRegion; >>>> >>>> The above forward declaration was removed but I think it's required >>>> for the following: >>>> >>> >>> I removed the forward declaration because DumpRegion is declared in >>> archiveUtils.hpp. Since JDK-8252056, archiveUtils.hpp is already >>> included by archiveBuilder.hpp: >>> >>> http://hg.openjdk.java.net/jdk/jdk/rev/ba51bc0772ac >>> >>>> ?132 DumpRegion* _rw_region; >>>> ?133?? DumpRegion* _ro_region; >>>> >>>> Also, the following constructor needs to be moved to >>>> archiveBuilder.cpp or I got compile error about invalid use of >>>> incomplete type 'class DumpRegion'. >>>> >>>> ?158???? OtherROAllocMark() { >>>> ?159?????? _oldtop = _singleton->_ro_region->top(); >>>> ?160???? } >>>> >>> Maybe you? applied the patch on a repo without the JDK-8252056 fix? > Right, I haven't updated my repo with the JDK-8252056 fix yet. >>> >>> I did find a problem with archiveUtils.hpp. It was using the Klass >>> type without including klass.hpp, so I added >>> >>> diff -r 3920d1a29976 src/hotspot/share/memory/archiveBuilder.hpp >>> --- a/src/hotspot/share/memory/archiveBuilder.hpp??? Wed Aug 19 >>> 14:14:20 2020 -0700 >>> +++ b/src/hotspot/share/memory/archiveBuilder.hpp??? Wed Aug 26 >>> 10:06:59 2020 -0700 >>> @@ -27,6 +27,7 @@ >>> >>> ?#include "memory/archiveUtils.hpp" >>> ?#include "memory/metaspaceClosure.hpp" >>> +#include "oops/klass.hpp" >>> ?#include "utilities/bitMap.hpp" >>> ?#include "utilities/growableArray.hpp" >>> ?#include "utilities/hashtable.hpp" >>> > Since you're including klass.hpp, maybe the following forward > declaration can be removed? > > 36 class Klass; > Right. I will remove that. Thanks - Ioi > thanks, > > Calvin > >>> >>> Thanks >>> - Ioi >>> >>> >>>> thanks, >>>> Calvin >>>> >>>> >>>> On 8/19/20 2:39 PM, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8251557 >>>>> http://cr.openjdk.java.net/~iklam/jdk16/8251557-cds-dont-dump-unused-symbols.v01/ >>>>> >>>>> >>>>> Part of CDS code clean up: >>>>> >>>>> Currently all entries in the SymbolTable/StringTable are dumped into >>>>> the CDS archive. The problems are >>>>> >>>>> (1) We end up writing many unused items, such as the mangled names >>>>> ? ? of hidden classes. >>>>> (2) We have to scan the symbol table inside a safepoint, which had >>>>> ? ? caused bugs before (JDK-8245264). >>>>> >>>>> Since JDK-8250990, we already maintain all used Symbols in a growable >>>>> array. We can dump the shared symbol table using this array, and >>>>> avoid walking the SymbolTable. >>>>> >>>>> For StringTable, we scan all archived classes, and dump only the >>>>> interned strings used by those classes (plus extra strings >>>>> specified by SharedArchiveConfigFile) into the CDS interned string >>>>> table. >>>>> >>>>> This reduces complexity in the CDS code, and reduce the size of >>>>> the CDS archives. >>>>> >>>>> Testing - tiers 1-4 >>>>> >>>>> Archive file size for "java -Xshare:dump" >>>>> >>>>> ?? old: 39,368 symbols 7,028 strings 11,784,200 bytes >>>>> ?? new: 37,387 symbols 7,020 strings 11,683,320 bytes (~0.9% >>>>> reduction) >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>> >> From ioi.lam at oracle.com Wed Aug 26 20:32:36 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 26 Aug 2020 13:32:36 -0700 Subject: RFR(T) 8252398 minimal debug build broken - CURRENT_PC undefined in resourceArea.inline.hpp Message-ID: https://bugs.openjdk.java.net/browse/JDK-8252398 This seems to have been a long standing issue but recent changes caused it to surface and break the minimal VM debug build. The fix is simple. CURRENT_PC is #define'ed in memTracker.hpp so we should add the include. Thanks - Ioi ======================== $ hg diff diff -r 34a78b672b7c src/hotspot/share/memory/resourceArea.inline.hpp --- a/src/hotspot/share/memory/resourceArea.inline.hpp Fri Aug 21 02:03:53 2020 -0700 +++ b/src/hotspot/share/memory/resourceArea.inline.hpp Wed Aug 26 13:27:32 2020 -0700 @@ -26,6 +26,7 @@ ?#define SHARE_MEMORY_RESOURCEAREA_INLINE_HPP ?#include "memory/resourceArea.hpp" +#include "services/memTracker.hpp" ?inline char* ResourceArea::allocate_bytes(size_t size, AllocFailType alloc_failmode) { From kim.barrett at oracle.com Wed Aug 26 20:34:39 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 26 Aug 2020 16:34:39 -0400 Subject: RFR(T) 8252398 minimal debug build broken - CURRENT_PC undefined in resourceArea.inline.hpp In-Reply-To: References: Message-ID: <65BD6389-A539-4909-9CE5-F8D8DDABAED0@oracle.com> > On Aug 26, 2020, at 4:32 PM, Ioi Lam wrote: > > https://bugs.openjdk.java.net/browse/JDK-8252398 > > This seems to have been a long standing issue but recent changes caused it to surface and break the minimal VM debug build. > > The fix is simple. CURRENT_PC is #define'ed in memTracker.hpp so we should add the include. > > Thanks > - Ioi > > ======================== > $ hg diff > diff -r 34a78b672b7c src/hotspot/share/memory/resourceArea.inline.hpp > --- a/src/hotspot/share/memory/resourceArea.inline.hpp Fri Aug 21 02:03:53 2020 -0700 > +++ b/src/hotspot/share/memory/resourceArea.inline.hpp Wed Aug 26 13:27:32 2020 -0700 > @@ -26,6 +26,7 @@ > #define SHARE_MEMORY_RESOURCEAREA_INLINE_HPP > > #include "memory/resourceArea.hpp" > +#include "services/memTracker.hpp" > > inline char* ResourceArea::allocate_bytes(size_t size, AllocFailType alloc_failmode) { Looks good, and trivial. From yumin.qi at oracle.com Wed Aug 26 20:44:52 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Wed, 26 Aug 2020 13:44:52 -0700 Subject: RFR 8251557 Avoid dumping unused symbols/strings into the CDS archive In-Reply-To: <6db5f819-8dba-84d6-961a-d03841858a9a@oracle.com> References: <6d64004b-4aeb-f95f-1551-d6d608191f3a@oracle.com> <699f0beb-5272-8ca8-a644-5383c4c4f8e2@oracle.com> <69beaf48-8c3e-bbd8-d964-99d5becd7e1b@oracle.com> <38a3d6da-c5ea-b2cd-50c7-6910d06a3b7f@oracle.com> <6db5f819-8dba-84d6-961a-d03841858a9a@oracle.com> Message-ID: <5eb5f983-bebd-8f06-bc2d-5c6d83ea5613@oracle.com> Hi, Ioi ?? Looks good to me. One minor comment: ?? stringTable.cpp: 746 unsigned int hash = java_lang_String::hash_code(s); hash is not used so this can be removed. No new webrev needed. Thanks Yumin On 8/26/20 11:30 AM, Ioi Lam wrote: > > > On 8/26/20 11:13 AM, calvin.cheung at oracle.com wrote: >> >> On 8/26/20 10:18 AM, Ioi Lam wrote: >>> >>> >>> On 8/26/20 10:08 AM, Ioi Lam wrote: >>>> Hi Calvin, >>>> >>>> Thanks for the review. >>>> >>>> On 8/26/20 9:40 AM, calvin.cheung at oracle.com wrote: >>>>> Hi Ioi, >>>>> >>>>> The proposed change does make the code easier to read than before. >>>>> >>>>> I've comments on 2 files: >>>>> >>>>> heapShared.hpp: >>>>> >>>>> 42 class DumpedInternedStrings; >>>>> >>>>> Instead of forward declaration, is it possible to move the class definition there since it is defined at the end of the file? >>>>> >>>> >>>> I moved it as you suggested. >>> >>> Oops, sorry I hit "Send" too fast. DumpedInternedStrings depends on function declared by HeapShared, so it must be declared after HeapShared >> I see. So the change looks good. >>> >>> Thanks >>> - Ioi >>> >>> >>>> >>>>> archiveBuilder.hpp: >>>>> >>>>> 37 class DumpRegion; >>>>> >>>>> The above forward declaration was removed but I think it's required for the following: >>>>> >>>> >>>> I removed the forward declaration because DumpRegion is declared in archiveUtils.hpp. Since JDK-8252056, archiveUtils.hpp is already included by archiveBuilder.hpp: >>>> >>>> http://hg.openjdk.java.net/jdk/jdk/rev/ba51bc0772ac >>>> >>>>> ?132 DumpRegion* _rw_region; >>>>> ?133?? DumpRegion* _ro_region; >>>>> >>>>> Also, the following constructor needs to be moved to archiveBuilder.cpp or I got compile error about invalid use of incomplete type 'class DumpRegion'. >>>>> >>>>> ?158???? OtherROAllocMark() { >>>>> ?159?????? _oldtop = _singleton->_ro_region->top(); >>>>> ?160???? } >>>>> >>>> Maybe you? applied the patch on a repo without the JDK-8252056 fix? >> Right, I haven't updated my repo with the JDK-8252056 fix yet. >>>> >>>> I did find a problem with archiveUtils.hpp. It was using the Klass type without including klass.hpp, so I added >>>> >>>> diff -r 3920d1a29976 src/hotspot/share/memory/archiveBuilder.hpp >>>> --- a/src/hotspot/share/memory/archiveBuilder.hpp??? Wed Aug 19 14:14:20 2020 -0700 >>>> +++ b/src/hotspot/share/memory/archiveBuilder.hpp??? Wed Aug 26 10:06:59 2020 -0700 >>>> @@ -27,6 +27,7 @@ >>>> >>>> ?#include "memory/archiveUtils.hpp" >>>> ?#include "memory/metaspaceClosure.hpp" >>>> +#include "oops/klass.hpp" >>>> ?#include "utilities/bitMap.hpp" >>>> ?#include "utilities/growableArray.hpp" >>>> ?#include "utilities/hashtable.hpp" >>>> >> Since you're including klass.hpp, maybe the following forward declaration can be removed? >> >> 36 class Klass; >> > > Right. I will remove that. > > Thanks > - Ioi > >> thanks, >> >> Calvin >> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>>> thanks, >>>>> Calvin >>>>> >>>>> >>>>> On 8/19/20 2:39 PM, Ioi Lam wrote: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8251557 >>>>>> http://cr.openjdk.java.net/~iklam/jdk16/8251557-cds-dont-dump-unused-symbols.v01/ >>>>>> >>>>>> Part of CDS code clean up: >>>>>> >>>>>> Currently all entries in the SymbolTable/StringTable are dumped into >>>>>> the CDS archive. The problems are >>>>>> >>>>>> (1) We end up writing many unused items, such as the mangled names >>>>>> ? ? of hidden classes. >>>>>> (2) We have to scan the symbol table inside a safepoint, which had >>>>>> ? ? caused bugs before (JDK-8245264). >>>>>> >>>>>> Since JDK-8250990, we already maintain all used Symbols in a growable >>>>>> array. We can dump the shared symbol table using this array, and >>>>>> avoid walking the SymbolTable. >>>>>> >>>>>> For StringTable, we scan all archived classes, and dump only the >>>>>> interned strings used by those classes (plus extra strings >>>>>> specified by SharedArchiveConfigFile) into the CDS interned string >>>>>> table. >>>>>> >>>>>> This reduces complexity in the CDS code, and reduce the size of >>>>>> the CDS archives. >>>>>> >>>>>> Testing - tiers 1-4 >>>>>> >>>>>> Archive file size for "java -Xshare:dump" >>>>>> >>>>>> ?? old: 39,368 symbols 7,028 strings 11,784,200 bytes >>>>>> ?? new: 37,387 symbols 7,020 strings 11,683,320 bytes (~0.9% reduction) >>>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> >>>> >>> > From calvin.cheung at oracle.com Wed Aug 26 21:35:37 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Wed, 26 Aug 2020 14:35:37 -0700 Subject: RFR(S): 8251860: ClassLoaderData::loaded_classes_do fails with "assert(ZAddress::is_marked(addr)) failed: Should be marked" Message-ID: <9217b9d3-4be7-b4f4-7be3-e4473fe18cf9@oracle.com> bug: https://bugs.openjdk.java.net/browse/JDK-8251860 webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.00/ Please refer to the bug report for an analysis of the crash. The proposed fix is to acquire the ClassLoaderDataGraph_lock before calling ClassLoaderDataGraph::loaded_classes_do(&link_closure). The link_closure will not do the actual linking but just to store the InstanceKlass'es in an array and increment the keep_alive counter of the class_loader_data of each InstanceKlass. After the call to ClassLoaderDataGraph::loaded_classes_do, each InstanceKlass in the array will be linked and the keep_alive counter of the class_loader_data will be decremented. Testing: ??? - tested about 40 times on the MacOS host where the crash was seen ?????? (before the fix, the crash was seen about once in 20 runs) ??? - Tier1 - 4. thanks, Calvin From coleen.phillimore at oracle.com Wed Aug 26 22:24:21 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 26 Aug 2020 18:24:21 -0400 Subject: RFR(S): 8251860: ClassLoaderData::loaded_classes_do fails with "assert(ZAddress::is_marked(addr)) failed: Should be marked" In-Reply-To: <9217b9d3-4be7-b4f4-7be3-e4473fe18cf9@oracle.com> References: <9217b9d3-4be7-b4f4-7be3-e4473fe18cf9@oracle.com> Message-ID: Hi Calvin, I think you should change this to call ClassLoaderDataGraph::loaded_cld_do(&collect_cld) and collect ClassLoaderData in the GrowableArray, since you then walk through them again to link the classes in the class loader data. Can linking one klass cause a new class to be added to a _klasses list of a CLD that you've already walked?? It seems like you need the do() loop in the current code wrapped around walking the clds, then link the classes until no progress.? You can fast exit when you find a klass linked, I believe, since the classes are added to the front of the _klasses list. Also with this change, you can remove locked_loaded_classes_do() because it was added for CDS and apparently the comment is wrong. // This case can block but cannot do unloading (called from CDS) void ClassLoaderDataGraph::unlocked_loaded_classes_do(KlassClosure* klass_closure) { ? for (ClassLoaderData* cld = _head; cld != NULL; cld = cld->next()) { ??? cld->loaded_classes_do(klass_closure); ? } } Thanks, Coleen On 8/26/20 5:35 PM, calvin.cheung at oracle.com wrote: > bug: https://bugs.openjdk.java.net/browse/JDK-8251860 > > webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.00/ > > Please refer to the bug report for an analysis of the crash. > > The proposed fix is to acquire the ClassLoaderDataGraph_lock before > calling ClassLoaderDataGraph::loaded_classes_do(&link_closure). The > link_closure will not do the actual linking but just to store the > InstanceKlass'es in an array and increment the keep_alive counter of > the class_loader_data of each InstanceKlass. After the call to > ClassLoaderDataGraph::loaded_classes_do, each InstanceKlass in the > array will be linked and the keep_alive counter of the > class_loader_data will be decremented. > > Testing: > ??? - tested about 40 times on the MacOS host where the crash was seen > ?????? (before the fix, the crash was seen about once in 20 runs) > ??? - Tier1 - 4. > thanks, > Calvin From coleen.phillimore at oracle.com Wed Aug 26 22:36:30 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Wed, 26 Aug 2020 18:36:30 -0400 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <68edf582-841b-890e-0951-8f8ecd5d132a@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> <1ca66b6a-8a23-c4b1-c95d-10fc23b668f8@oracle.com> <6344748b-f484-e743-8bef-5750d0b49155@oracle.com> <68edf582-841b-890e-0951-8f8ecd5d132a@oracle.com> Message-ID: On 8/25/20 8:17 PM, Yumin Qi wrote: > Hi, Coleen and all > > ? I have updated at new link: > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-05/ > > ? I changed find_class and add_to_hierarchy to public access in > SystemDictionary for access from outside. > > ? new added find_class removed from ClassLoaderData. > > ? new added add_shared_to_hierarchy removed from SystemDictionaryShared. http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-05/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-05/src/hotspot/share/classfile/systemDictionaryShared.hpp.udiff.html Thanks for making the change to call add_to_hierarchy directly in SystemDictionary, but it looks like you left it here in this patch. Also, can lambdaFormInvokers.hpp/cpp be put in the classfile directory rather than the memory directory? Otherwise looks good.? I didn't review the Java code. Coleen > > ? re-tested local on runtime/cds > > > ? Thanks > > ? Yumin > > > On 8/25/20 12:41 PM, Coleen Phillimore wrote: >> >> Hi, I'm sorry I haven't been watching this change. >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html >> >> >> Can you move find_class() into systemDictionary.hpp/cpp with the rest >> of the find_class().? This is strange here.? You don't want to expose >> that the dictionary is here to the rest of the code. The caller >> should call one of the SystemDictionary versions, or add this to >> SystemDictionary with the other functions with the same name. >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/memory/lambdaFormInvokers.cpp.html >> >> >> These new files belongs in classfile.? It's closer to >> systemDictionaryShared code than memory code. >> >> ?? 2? * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All >> rights reserved. >> >> >> Also copyright shouldn't start with 2012. >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >> >> >> This add_to_hierarchy is a strange delegation and a surprise that >> it's also defined here.? Please call this inline from >> lambdaFormInvokers, ie take the Compile_lock and call >> SystemDictionary::add_to_hierarchy(). >> >> I don't have any deeper comments. >> >> Thanks, >> Coleen >> >> On 8/25/20 2:59 PM, Yumin Qi wrote: >>> HI, Ioi >>> >>> ? Thanks for the re-review, updated webrev: >>> >>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/ >>> >>> ? Changed according to your suggestion, the only a little difference >>> is move traceResolve(String line) to InvokerBytecodeGenerator as a >>> static package public function so other package classes can call it. >>> >>> ? re-tested local on runtime/cds >>> >>> >>> Thanks >>> >>> Yumin >>> >>> >>> On 8/24/20 4:23 PM, Ioi Lam wrote: >>>> Hi Yumin, >>>> >>>> This looks good overall. Here are my comments: >>>> >>>> ===================== >>>> 6065???? size_t new_id = Atomic::add(&counter, (size_t)1); >>>> 6066???? jio_snprintf(addr_buf, 20, INTPTR_FORMAT, new_id); >>>> >>>> I think this should be SIZE_FORMAT >>>> >>>> ===================== >>>> ? 65 class KlassFactory : AllStatic { >>>> ? 66 >>>> ? 67?? // approved clients >>>> ? 68?? friend class ClassLoader; >>>> ? 69?? friend class ClassLoaderExt; >>>> ? 70?? friend class SystemDictionary; >>>> ? 71?? friend class LambdaFormInvokers; >>>> ? 72 >>>> ? 73? private: >>>> ? 74?? static InstanceKlass* create_from_stream(ClassFileStream* >>>> stream, >>>> >>>> I think instead of adding everyone who uses create_from_stream as a >>>> friend class, we should just change create_from_stream into a >>>> public function and remove the friend declarations. >>>> >>>> ===================== >>>> ?146?? // add to hierarchy and set state to loaded. >>>> ?147?? { >>>> ?148???? MutexLocker mu_r(THREAD, Compile_lock); // >>>> add_shared_to_hierarchy asserts this. >>>> ?149 SystemDictionaryShared::add_shared_to_hierarchy(result, THREAD); >>>> ?150?? } >>>> >>>> I think the function name can be changed to >>>> SystemDictionaryShared::add_to_hierarchy as the "_shared" seems >>>> redundant. The "set state to loaded" comment seems wrong, as we >>>> have the assert on line 1155. I think the comment can be removed. >>>> >>>> 1153 void >>>> SystemDictionaryShared::add_shared_to_hierarchy(InstanceKlass* k, >>>> TRAPS) { >>>> 1154?? Arguments::assert_is_dumping_archive(); >>>> 1155?? assert(!k->is_loaded(), "Invariant"); >>>> 1156?? assert_locked_or_safepoint(Compile_lock); // >>>> add_to_hierarchy assert on it. >>>> 1157?? SystemDictionary::add_to_hierarchy(k, CHECK); >>>> 1158 } >>>> >>>> Also, I think it's better to move the MutexLocker call into >>>> SystemDictionaryShared::add_shared_to_hierarchy. >>>> >>>> ======================== >>>> >>>> before: >>>> >>>> ?478???????????????? if (TRACE_RESOLVE && salvage != null) { >>>> ?479???????????????????? // Used by jlink species pregeneration >>>> plugin, see >>>> ?480???????????????????? // >>>> jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >>>> ?481 System.out.println("[SPECIES_RESOLVE] " + className + " >>>> (salvaged)"); >>>> ?482???????????????? } >>>> >>>> after: >>>> >>>> ?488???????????????? // Used by jlink species pregeneration plugin, >>>> see >>>> ?489???????????????? // >>>> jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >>>> ?490???????????????? traceResolve("[SPECIES_RESOLVE] " + className >>>> + " (salvaged)"); >>>> >>>> >>>> When tracing is disabled, this will make extra allocations and >>>> cause a small slowdown. I think it's better to >>>> >>>> ??? if ((TRACE_RESOLVE|TRACE_RESOLVE_CDS) && salvage != null) { >>>> ? ? ?? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>>> ??? } >>>> >>>> Because TRACE_RESOLVE is a static final boolean, the JIT compiler >>>> will completely optimize this block out. >>>> >>>> For the same reason, instead of calling >>>> VM.isDumpLoadedClassListSetAndOpen() every time, it's better to use >>>> a static final variable. >>>> >>>> >>>> ======================= >>>> >>>> ?698???????? if (TRACE_RESOLVE) { >>>> ?699???????????? System.out.println("[LF_RESOLVE] " + >>>> holder.getName() + " " + name + " " + >>>> ?700 shortenSignature(basicTypeSignature(type)) + (resolvedMember >>>> != null ? " (success)" : " (fail)") ); >>>> ?701???????? } >>>> ?702???????? if (VM.isDumpLoadedClassListSetAndOpen()) { >>>> ?703 GenerateJLIClassesHelper.cdsTraceResolve("[LF_RESOLVE] " + >>>> holder.getName() + " " + name + " " + >>>> ?704 shortenSignature(basicTypeSignature(type)) + (resolvedMember >>>> != null ? " (success)" : " (fail)") ); >>>> ?705???????? } >>>> ?706???????? return resolvedMember; >>>> >>>> >>>> I think the two "if" blocks should be combined similarly to >>>> ClassSpecializer::traceResolve(). >>>> >>>> ========================= >>>> ? 34 >>>> Java_java_lang_invoke_GenerateJLIClassesHelper_cdsTraceResolve(JNIEnv >>>> *env, jclass ignore, jstring line) { >>>> >>>> Maybe this should be moved to the "VM" class as well? >>>> >>>> ========================= >>>> lambdaFormInvokers.hpp: >>>> >>>> Need these declarations: >>>> >>>> ??? #include "memory/allocation.hpp"??? <-- for AllStatic >>>> ??? #include "runtime/handles.hpp"????? <-- for typeArrayHandle and >>>> Handle >>>> ??? #include "utilities/exceptions.hpp" <-- for TRAPS >>>> >>>> ??? template class GrowableArray; >>>> ========================= >>>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> >>>> On 8/23/20 3:56 PM, Yumin Qi wrote: >>>>> >>>>> Hi, Mandy, Ioi and Calvin >>>>> >>>>> >>>>> ? I have updated the new changed at: >>>>> >>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-03/ >>>>> >>>>> ? In this version: >>>>> >>>>> ??? 1) Added a new API to check if flag DumpLoadedClassList set >>>>> and the file is open. If true, call into vm to print out the trace >>>>> line to the log file. >>>>> >>>>> ??????? Just thinking if we just call the cdsTraceResolve without >>>>> checking if the flag DumpLoadedClassList set and file open, this >>>>> way, the check logic is in the vm side like before, so save code >>>>> by not adding the new API. >>>>> >>>>> ??? 2) The returned holder class names now are just >>>>> 'package/className', removed head and tail. >>>>> >>>>> ??? 3)? Remove add_extra_classes from CollectClassesClosure since >>>>> after bug 8250990: >>>>> https://bugs.openjdk.java.net/browse/JDK-8250990 pushed, the >>>>> CollectClassesClosure no longer exist. >>>>> >>>>> ??? 4)? Still keep the parsing for TRACE_RESOLVE in >>>>> java/lang/invoke/GenerateJLIClassesHelper.java, so VM call its >>>>> function to regenerate holder classes. >>>>> >>>>> >>>>> ? Re-tested Mach5 tier1-4 >>>>> >>>>> Thanks >>>>> >>>>> Yumin >>>>> >>>>> >>>>> On 8/20/20 8:05 PM, Yumin Qi wrote: >>>>>> Hi, Mandy >>>>>> >>>>>> >>>>>> On 8/20/20 5:10 PM, Mandy Chung wrote: >>>>>>> >>>>>>> >>>>>>> On 8/19/20 10:14 PM, Yumin Qi wrote: >>>>>>>> >>>>>>>> HI, Mandy >>>>>>>> >>>>>>>> ? Thanks for the review, I took one day off yesterday so just >>>>>>>> got a detail look of your reply. >>>>>>>> >>>>>>>> On 8/19/20 1:30 PM, Mandy Chung wrote: >>>>>>>>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>>>>>>>> Hi, Ioi >>>>>>>>>> >>>>>>>>>> ? Thanks for review/suggestion. I have updated the webrev at >>>>>>>>>> the following link: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>>>>>>>> >>>>>>>>> >>>>>>>>> This patch leverages the TRACE_RESOLVE output and passes the >>>>>>>>> trace output to VM.? VM then calls >>>>>>>>> GenerateJLIClassesHelper::generateMHHolderClasses to do the >>>>>>>>> parsing and generate Holder class per the resolved LFs.?? I >>>>>>>>> think there are other cleaner alternatives implementing >>>>>>>>> this.?? jlink --generate-jli-classes plugin depends the trace >>>>>>>>> output whereas -Xshare:dump does not.?? It's cleaner to skip >>>>>>>>> generating the trace output and parsing for dumping shared >>>>>>>>> archive purpose.? In addition, the implementation needs some >>>>>>>>> cleanup (I can send you feedback on the next revision) >>>>>>>>> >>>>>>>> Current patch did not change the existing code for JLinkPlugin >>>>>>>> part. I just moved the parsing code from >>>>>>>> GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java >>>>>>>> since the former is an internal class to which we shouldn't >>>>>>>> call to generate holder classes. >>>>>>>>> Instead of relying on a system property >>>>>>>>> "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better >>>>>>>>> to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this >>>>>>>>> is CDS dump time. >>>>>>>> >>>>>>>> I remember we have such API to query if flag -Xshare:dump or >>>>>>>> -Xshare:on used. Do you mean if DumpLoadedClassList flag set? >>>>>>>> This flag is the one used to log class name to list file. In >>>>>>>> GenerateLinkOptData.gmk: >>>>>>>> >>>>>>>> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) >>>>>>>> $(CLASSLIST_JAR) >>>>>>>> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >>>>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, >>>>>>>> %, $@)) >>>>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, >>>>>>>> %, $(JLI_TRACE_FILE))) >>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >>>>>>>> -XX:DumpLoadedClassList=$@.raw \ >>>>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >>>>>>>> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >>>>>>>> ??????????? -XX:SharedClassListFile=$@.interim >>>>>>>> -XX:SharedArchiveFile=$@.jsa \ >>>>>>>> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >>>>>>>> -XX:DumpLoadedClassList=$@.raw.2 \ >>>>>>>> ??????????? -XX:SharedClassListFile=$@.interim >>>>>>>> -XX:SharedArchiveFile=$@.jsa \ >>>>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >>>>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>>>> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>> ??????????? build.tools.classlist.HelloClasslist \ >>>>>>>> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >>>>>>>> ??????????? || ( \ >>>>>>>> ??????????????? exitcode=$$? ; \ >>>>>>>> ??????????????? $(ECHO) "ERROR: Failed to generate link >>>>>>>> optimization data." \ >>>>>>>> ??????????????????? "This is likely a problem with the newly >>>>>>>> built JVM/JDK." ; \ >>>>>>>> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) >>>>>>>> ; \ >>>>>>>> ??????????????? exit $$exitcode \ >>>>>>>> ??????????? ) >>>>>>>> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >>>>>>>> >>>>>>>> The $(JLI_TRACE_FILE) is generated with both >>>>>>>> -XX:DumpLoadedClassList and >>>>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current >>>>>>>> implementation, DumpLoadedClassList will turn on property >>>>>>>> java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the >>>>>>>> same output sent to stdout and log file DumpLoadedClassList >>>>>>>> specified. >>>>>>>> >>>>>>> >>>>>>> These entries are duplicated in two different files: one for >>>>>>> jlink --generate-jli-classes plugin and another for CDS use.? >>>>>>> CDS -Xshare:dump attempts to do what jlink plugin does but >>>>>>> writes those generated classes in to shared archive. >>>>>>> >>>>>>> Like the above make logic to build JDK image, the same entries >>>>>>> are written in both default-jli-trace.txt via System.out and to >>>>>>> classlist via JNI call to the VM.? I guess VM also implements >>>>>>> the logic to do some kind of diffing and write to CDS archive. >>>>>>> >>>>>> In current implementation, vm side only records the line as from >>>>>> TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at >>>>>> dump time, call back to java for parsing those recordings and >>>>>> generating the holder classes, this uses the existing JLI code. >>>>>>>> >>>>>>>> Now instead of this property, using a vm interface API to query >>>>>>>> if this flag is set, I think it is better choice. But here I am >>>>>>>> NOT sure I understand your suggestion, I think there are two >>>>>>>> choices: >>>>>>>> >>>>>>>> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not >>>>>>>> via CDS_TRACE_RESOLVE, using new API to query if >>>>>>>> DumpLoadedClassList is set >>>>>>>> >>>>>>>> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting >>>>>>>> those name, type and holder name to regenerate holder classes? >>>>>>>> >>>>>>> >>>>>>> I misunderstood that this CDS_TRACE_RESOLVE flag is set during >>>>>>> -Xshare:dump time. >>>>>>> >>>>>>> Ioi has clarified to me offline that this step is actually part >>>>>>> of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs >>>>>>> to the given class list file, i.e. you repurpose the class list >>>>>>> file to include the log output that was initially designed for >>>>>>> jlink plugin. >>>>>>> >>>>>>> To me, I'd prefer to see this support depending on `jlink >>>>>>> --generate-jli-classes` which is an existing functionality and >>>>>>> much cleaner.? This way this does not require any VM change.?? >>>>>>> It will generate the holder classes in the custom image per the >>>>>>> application-specific config file. >>>>>>> >>>>>>> What it means is that: a customer would need to create a custom >>>>>>> image with their application-specific config file. It might need >>>>>>> a new CDS? option to specify a separate TRACE_RESOLVE file.?? It >>>>>>> would turn on this feature by default by defining a default path >>>>>>> of the log file if it helps. >>>>>>> >>>>>> So for now, I would implement an API to query if flag >>>>>> DumpLoadedClassList set in cmd line, remove new added property of >>>>>> java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address >>>>>> custom image with CDS in future in a separate issue. >>>>>> >>>>>> >>>>>> Thanks >>>>>> >>>>>> Yumin >>>>>> >>>>>>> I understand that this is not the existing CDS work flow and CDS >>>>>>> archive does not require to run on a custom image.?? I see the >>>>>>> value of this approach which can prepare customers to start >>>>>>> building and using its own custom image. >>>>>>> >>>>>>> Of course the implementation would be much simpler (adding a >>>>>>> flag to write these traces to a given file path, that is). >>>>>>> >>>>>>> Mandy >>>> >> From yumin.qi at oracle.com Wed Aug 26 22:59:40 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Wed, 26 Aug 2020 15:59:40 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> <1ca66b6a-8a23-c4b1-c95d-10fc23b668f8@oracle.com> <6344748b-f484-e743-8bef-5750d0b49155@oracle.com> <68edf582-841b-890e-0951-8f8ecd5d132a@oracle.com> Message-ID: <94df328f-21c1-88ea-14ca-2948061b4c11@oracle.com> Hi, Coleen On 8/26/20 3:36 PM, Coleen Phillimore wrote: > > > On 8/25/20 8:17 PM, Yumin Qi wrote: >> Hi, Coleen and all >> >> ? I have updated at new link: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-05/ >> >> ? I changed find_class and add_to_hierarchy to public access in SystemDictionary for access from outside. >> >> ? new added find_class removed from ClassLoaderData. >> >> ? new added add_shared_to_hierarchy removed from SystemDictionaryShared. > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-05/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-05/src/hotspot/share/classfile/systemDictionaryShared.hpp.udiff.html > > Thanks for making the change to call add_to_hierarchy directly in SystemDictionary, but it looks like you left it here in this patch. > Thanks! will remove them. > Also, can lambdaFormInvokers.hpp/cpp be put in the classfile directory rather than the memory directory? > Sorry forget to do so. Will move them to classfile directory. > Otherwise looks good.? I didn't review the Java code. > About this part, I will wait for Mandy to come back from her vacation, she had a proposal and patch for consolidating JLI code. Thanks Yumin > Coleen >> >> ? re-tested local on runtime/cds >> >> >> ? Thanks >> >> ? Yumin >> >> >> On 8/25/20 12:41 PM, Coleen Phillimore wrote: >>> >>> Hi, I'm sorry I haven't been watching this change. >>> >>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html >>> >>> Can you move find_class() into systemDictionary.hpp/cpp with the rest of the find_class().? This is strange here.? You don't want to expose that the dictionary is here to the rest of the code. The caller should call one of the SystemDictionary versions, or add this to SystemDictionary with the other functions with the same name. >>> >>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/memory/lambdaFormInvokers.cpp.html >>> >>> These new files belongs in classfile.? It's closer to systemDictionaryShared code than memory code. >>> >>> ?? 2? * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All rights reserved. >>> >>> >>> Also copyright shouldn't start with 2012. >>> >>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >>> >>> This add_to_hierarchy is a strange delegation and a surprise that it's also defined here.? Please call this inline from lambdaFormInvokers, ie take the Compile_lock and call SystemDictionary::add_to_hierarchy(). >>> >>> I don't have any deeper comments. >>> >>> Thanks, >>> Coleen >>> >>> On 8/25/20 2:59 PM, Yumin Qi wrote: >>>> HI, Ioi >>>> >>>> ? Thanks for the re-review, updated webrev: >>>> >>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/ >>>> >>>> ? Changed according to your suggestion, the only a little difference is move traceResolve(String line) to InvokerBytecodeGenerator as a static package public function so other package classes can call it. >>>> >>>> ? re-tested local on runtime/cds >>>> >>>> >>>> Thanks >>>> >>>> Yumin >>>> >>>> >>>> On 8/24/20 4:23 PM, Ioi Lam wrote: >>>>> Hi Yumin, >>>>> >>>>> This looks good overall. Here are my comments: >>>>> >>>>> ===================== >>>>> 6065???? size_t new_id = Atomic::add(&counter, (size_t)1); >>>>> 6066???? jio_snprintf(addr_buf, 20, INTPTR_FORMAT, new_id); >>>>> >>>>> I think this should be SIZE_FORMAT >>>>> >>>>> ===================== >>>>> ? 65 class KlassFactory : AllStatic { >>>>> ? 66 >>>>> ? 67?? // approved clients >>>>> ? 68?? friend class ClassLoader; >>>>> ? 69?? friend class ClassLoaderExt; >>>>> ? 70?? friend class SystemDictionary; >>>>> ? 71?? friend class LambdaFormInvokers; >>>>> ? 72 >>>>> ? 73? private: >>>>> ? 74?? static InstanceKlass* create_from_stream(ClassFileStream* stream, >>>>> >>>>> I think instead of adding everyone who uses create_from_stream as a friend class, we should just change create_from_stream into a public function and remove the friend declarations. >>>>> >>>>> ===================== >>>>> ?146?? // add to hierarchy and set state to loaded. >>>>> ?147?? { >>>>> ?148???? MutexLocker mu_r(THREAD, Compile_lock); // add_shared_to_hierarchy asserts this. >>>>> ?149 SystemDictionaryShared::add_shared_to_hierarchy(result, THREAD); >>>>> ?150?? } >>>>> >>>>> I think the function name can be changed to SystemDictionaryShared::add_to_hierarchy as the "_shared" seems redundant. The "set state to loaded" comment seems wrong, as we have the assert on line 1155. I think the comment can be removed. >>>>> >>>>> 1153 void SystemDictionaryShared::add_shared_to_hierarchy(InstanceKlass* k, TRAPS) { >>>>> 1154?? Arguments::assert_is_dumping_archive(); >>>>> 1155?? assert(!k->is_loaded(), "Invariant"); >>>>> 1156?? assert_locked_or_safepoint(Compile_lock); // add_to_hierarchy assert on it. >>>>> 1157?? SystemDictionary::add_to_hierarchy(k, CHECK); >>>>> 1158 } >>>>> >>>>> Also, I think it's better to move the MutexLocker call into SystemDictionaryShared::add_shared_to_hierarchy. >>>>> >>>>> ======================== >>>>> >>>>> before: >>>>> >>>>> ?478???????????????? if (TRACE_RESOLVE && salvage != null) { >>>>> ?479???????????????????? // Used by jlink species pregeneration plugin, see >>>>> ?480???????????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >>>>> ?481 System.out.println("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>>>> ?482???????????????? } >>>>> >>>>> after: >>>>> >>>>> ?488???????????????? // Used by jlink species pregeneration plugin, see >>>>> ?489???????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >>>>> ?490???????????????? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>>>> >>>>> >>>>> When tracing is disabled, this will make extra allocations and cause a small slowdown. I think it's better to >>>>> >>>>> ??? if ((TRACE_RESOLVE|TRACE_RESOLVE_CDS) && salvage != null) { >>>>> ? ? ?? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>>>> ??? } >>>>> >>>>> Because TRACE_RESOLVE is a static final boolean, the JIT compiler will completely optimize this block out. >>>>> >>>>> For the same reason, instead of calling VM.isDumpLoadedClassListSetAndOpen() every time, it's better to use a static final variable. >>>>> >>>>> >>>>> ======================= >>>>> >>>>> ?698???????? if (TRACE_RESOLVE) { >>>>> ?699???????????? System.out.println("[LF_RESOLVE] " + holder.getName() + " " + name + " " + >>>>> ?700 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); >>>>> ?701???????? } >>>>> ?702???????? if (VM.isDumpLoadedClassListSetAndOpen()) { >>>>> ?703 GenerateJLIClassesHelper.cdsTraceResolve("[LF_RESOLVE] " + holder.getName() + " " + name + " " + >>>>> ?704 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); >>>>> ?705???????? } >>>>> ?706???????? return resolvedMember; >>>>> >>>>> >>>>> I think the two "if" blocks should be combined similarly to ClassSpecializer::traceResolve(). >>>>> >>>>> ========================= >>>>> ? 34 Java_java_lang_invoke_GenerateJLIClassesHelper_cdsTraceResolve(JNIEnv *env, jclass ignore, jstring line) { >>>>> >>>>> Maybe this should be moved to the "VM" class as well? >>>>> >>>>> ========================= >>>>> lambdaFormInvokers.hpp: >>>>> >>>>> Need these declarations: >>>>> >>>>> ??? #include "memory/allocation.hpp"??? <-- for AllStatic >>>>> ??? #include "runtime/handles.hpp"????? <-- for typeArrayHandle and Handle >>>>> ??? #include "utilities/exceptions.hpp" <-- for TRAPS >>>>> >>>>> ??? template class GrowableArray; >>>>> ========================= >>>>> >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> >>>>> On 8/23/20 3:56 PM, Yumin Qi wrote: >>>>>> >>>>>> Hi, Mandy, Ioi and Calvin >>>>>> >>>>>> >>>>>> ? I have updated the new changed at: >>>>>> >>>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-03/ >>>>>> >>>>>> ? In this version: >>>>>> >>>>>> ??? 1) Added a new API to check if flag DumpLoadedClassList set and the file is open. If true, call into vm to print out the trace line to the log file. >>>>>> >>>>>> ??????? Just thinking if we just call the cdsTraceResolve without checking if the flag DumpLoadedClassList set and file open, this way, the check logic is in the vm side like before, so save code by not adding the new API. >>>>>> >>>>>> ??? 2) The returned holder class names now are just 'package/className', removed head and tail. >>>>>> >>>>>> ??? 3)? Remove add_extra_classes from CollectClassesClosure since after bug 8250990: https://bugs.openjdk.java.net/browse/JDK-8250990 pushed, the CollectClassesClosure no longer exist. >>>>>> >>>>>> ??? 4)? Still keep the parsing for TRACE_RESOLVE in java/lang/invoke/GenerateJLIClassesHelper.java, so VM call its function to regenerate holder classes. >>>>>> >>>>>> >>>>>> ? Re-tested Mach5 tier1-4 >>>>>> >>>>>> Thanks >>>>>> >>>>>> Yumin >>>>>> >>>>>> >>>>>> On 8/20/20 8:05 PM, Yumin Qi wrote: >>>>>>> Hi, Mandy >>>>>>> >>>>>>> >>>>>>> On 8/20/20 5:10 PM, Mandy Chung wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 8/19/20 10:14 PM, Yumin Qi wrote: >>>>>>>>> >>>>>>>>> HI, Mandy >>>>>>>>> >>>>>>>>> ? Thanks for the review, I took one day off yesterday so just got a detail look of your reply. >>>>>>>>> >>>>>>>>> On 8/19/20 1:30 PM, Mandy Chung wrote: >>>>>>>>>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>>>>>>>>> Hi, Ioi >>>>>>>>>>> >>>>>>>>>>> ? Thanks for review/suggestion. I have updated the webrev at the following link: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This patch leverages the TRACE_RESOLVE output and passes the trace output to VM.? VM then calls GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing and generate Holder class per the resolved LFs.?? I think there are other cleaner alternatives implementing this.?? jlink --generate-jli-classes plugin depends the trace output whereas -Xshare:dump does not.?? It's cleaner to skip generating the trace output and parsing for dumping shared archive purpose.? In addition, the implementation needs some cleanup (I can send you feedback on the next revision) >>>>>>>>>> >>>>>>>>> Current patch did not change the existing code for JLinkPlugin part. I just moved the parsing code from GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java since the former is an internal class to which we shouldn't call to generate holder classes. >>>>>>>>>> Instead of relying on a system property "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. >>>>>>>>> >>>>>>>>> I remember we have such API to query if flag -Xshare:dump or -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This flag is the one used to log class name to list file. In GenerateLinkOptData.gmk: >>>>>>>>> >>>>>>>>> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR) >>>>>>>>> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >>>>>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) >>>>>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $(JLI_TRACE_FILE))) >>>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw \ >>>>>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>>> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >>>>>>>>> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >>>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >>>>>>>>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>>>>>>>> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >>>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw.2 \ >>>>>>>>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>>>>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >>>>>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>>>>> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>>> ??????????? build.tools.classlist.HelloClasslist \ >>>>>>>>> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >>>>>>>>> ??????????? || ( \ >>>>>>>>> ??????????????? exitcode=$$? ; \ >>>>>>>>> ??????????????? $(ECHO) "ERROR: Failed to generate link optimization data." \ >>>>>>>>> ??????????????????? "This is likely a problem with the newly built JVM/JDK." ; \ >>>>>>>>> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >>>>>>>>> ??????????????? exit $$exitcode \ >>>>>>>>> ??????????? ) >>>>>>>>> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >>>>>>>>> >>>>>>>>> The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current implementation, DumpLoadedClassList will turn on property java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same output sent to stdout and log file DumpLoadedClassList specified. >>>>>>>>> >>>>>>>> >>>>>>>> These entries are duplicated in two different files: one for jlink --generate-jli-classes plugin and another for CDS use.? CDS -Xshare:dump attempts to do what jlink plugin does but writes those generated classes in to shared archive. >>>>>>>> >>>>>>>> Like the above make logic to build JDK image, the same entries are written in both default-jli-trace.txt via System.out and to classlist via JNI call to the VM.? I guess VM also implements the logic to do some kind of diffing and write to CDS archive. >>>>>>>> >>>>>>> In current implementation, vm side only records the line as from TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at dump time, call back to java for parsing those recordings and generating the holder classes, this uses the existing JLI code. >>>>>>>>> >>>>>>>>> Now instead of this property, using a vm interface API to query if this flag is set, I think it is better choice. But here I am NOT sure I understand your suggestion, I think there are two choices: >>>>>>>>> >>>>>>>>> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set >>>>>>>>> >>>>>>>>> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting those name, type and holder name to regenerate holder classes? >>>>>>>>> >>>>>>>> >>>>>>>> I misunderstood that this CDS_TRACE_RESOLVE flag is set during -Xshare:dump time. >>>>>>>> >>>>>>>> Ioi has clarified to me offline that this step is actually part of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the given class list file, i.e. you repurpose the class list file to include the log output that was initially designed for jlink plugin. >>>>>>>> >>>>>>>> To me, I'd prefer to see this support depending on `jlink --generate-jli-classes` which is an existing functionality and much cleaner.? This way this does not require any VM change.?? It will generate the holder classes in the custom image per the application-specific config file. >>>>>>>> >>>>>>>> What it means is that: a customer would need to create a custom image with their application-specific config file. It might need a new CDS? option to specify a separate TRACE_RESOLVE file.?? It would turn on this feature by default by defining a default path of the log file if it helps. >>>>>>>> >>>>>>> So for now, I would implement an API to query if flag DumpLoadedClassList set in cmd line, remove new added property of java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address custom image with CDS in future in a separate issue. >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Yumin >>>>>>> >>>>>>>> I understand that this is not the existing CDS work flow and CDS archive does not require to run on a custom image.?? I see the value of this approach which can prepare customers to start building and using its own custom image. >>>>>>>> >>>>>>>> Of course the implementation would be much simpler (adding a flag to write these traces to a given file path, that is). >>>>>>>> >>>>>>>> Mandy >>>>> >>> > From calvin.cheung at oracle.com Wed Aug 26 23:05:29 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Wed, 26 Aug 2020 16:05:29 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <68edf582-841b-890e-0951-8f8ecd5d132a@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> <1ca66b6a-8a23-c4b1-c95d-10fc23b668f8@oracle.com> <6344748b-f484-e743-8bef-5750d0b49155@oracle.com> <68edf582-841b-890e-0951-8f8ecd5d132a@oracle.com> Message-ID: <8fca4eb0-119f-83cb-3c2c-9400b66f027b@oracle.com> Hi Yumin, Some minor comments: metaspaceShared.cpp ? 45 #include "memory/lambdaFormInvokers.hpp" ? 46 #include "memory/heapShared.inline.hpp" Please swap the order of the above 2 lines. ------ lambdaFormInvokers.cpp 64?? guarantee(helper_klass != NULL, "java/lang/invoke/GenerateJLIClassesHelper exist!"); I think it is clearer to say: "java/lang/invoke/GenerateJLIClassesHelper must exist!". 70???? record += strlen(lambda_form_invoker_tag()) + 1; // @lambda_form_invoker [...] How about ?skip the @lambda_form_invoker prefix? for the comment? ------ lambdaFormInvokers.hpp 34?? static void reload_class(Handle name, typeArrayHandle bytes, TRAPS); reload_class could be private since it?s called only from lambdaFormInvokers.cpp. ------ DumpSymbolAndStringTable.java 37???? private static final String my_string = "DumpSymbolAndStringTable"; The above declaration is unnecessary. I've tried your patch and the test passed without it. ------ No need to spin another webrev for the above changes. thanks, Calvin On 8/25/20 5:17 PM, Yumin Qi wrote: > Hi, Coleen and all > > ? I have updated at new link: > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-05/ > > ? I changed find_class and add_to_hierarchy to public access in > SystemDictionary for access from outside. > > ? new added find_class removed from ClassLoaderData. > > ? new added add_shared_to_hierarchy removed from SystemDictionaryShared. > > ? re-tested local on runtime/cds > > > ? Thanks > > ? Yumin > > > On 8/25/20 12:41 PM, Coleen Phillimore wrote: >> >> Hi, I'm sorry I haven't been watching this change. >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html >> >> >> Can you move find_class() into systemDictionary.hpp/cpp with the rest >> of the find_class().? This is strange here.? You don't want to expose >> that the dictionary is here to the rest of the code. The caller >> should call one of the SystemDictionary versions, or add this to >> SystemDictionary with the other functions with the same name. >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/memory/lambdaFormInvokers.cpp.html >> >> >> These new files belongs in classfile.? It's closer to >> systemDictionaryShared code than memory code. >> >> ?? 2? * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All >> rights reserved. >> >> >> Also copyright shouldn't start with 2012. >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >> >> >> This add_to_hierarchy is a strange delegation and a surprise that >> it's also defined here.? Please call this inline from >> lambdaFormInvokers, ie take the Compile_lock and call >> SystemDictionary::add_to_hierarchy(). >> >> I don't have any deeper comments. >> >> Thanks, >> Coleen >> >> On 8/25/20 2:59 PM, Yumin Qi wrote: >>> HI, Ioi >>> >>> ? Thanks for the re-review, updated webrev: >>> >>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/ >>> >>> ? Changed according to your suggestion, the only a little difference >>> is move traceResolve(String line) to InvokerBytecodeGenerator as a >>> static package public function so other package classes can call it. >>> >>> ? re-tested local on runtime/cds >>> >>> >>> Thanks >>> >>> Yumin >>> >>> >>> On 8/24/20 4:23 PM, Ioi Lam wrote: >>>> Hi Yumin, >>>> >>>> This looks good overall. Here are my comments: >>>> >>>> ===================== >>>> 6065???? size_t new_id = Atomic::add(&counter, (size_t)1); >>>> 6066???? jio_snprintf(addr_buf, 20, INTPTR_FORMAT, new_id); >>>> >>>> I think this should be SIZE_FORMAT >>>> >>>> ===================== >>>> ? 65 class KlassFactory : AllStatic { >>>> ? 66 >>>> ? 67?? // approved clients >>>> ? 68?? friend class ClassLoader; >>>> ? 69?? friend class ClassLoaderExt; >>>> ? 70?? friend class SystemDictionary; >>>> ? 71?? friend class LambdaFormInvokers; >>>> ? 72 >>>> ? 73? private: >>>> ? 74?? static InstanceKlass* create_from_stream(ClassFileStream* >>>> stream, >>>> >>>> I think instead of adding everyone who uses create_from_stream as a >>>> friend class, we should just change create_from_stream into a >>>> public function and remove the friend declarations. >>>> >>>> ===================== >>>> ?146?? // add to hierarchy and set state to loaded. >>>> ?147?? { >>>> ?148???? MutexLocker mu_r(THREAD, Compile_lock); // >>>> add_shared_to_hierarchy asserts this. >>>> ?149 SystemDictionaryShared::add_shared_to_hierarchy(result, THREAD); >>>> ?150?? } >>>> >>>> I think the function name can be changed to >>>> SystemDictionaryShared::add_to_hierarchy as the "_shared" seems >>>> redundant. The "set state to loaded" comment seems wrong, as we >>>> have the assert on line 1155. I think the comment can be removed. >>>> >>>> 1153 void >>>> SystemDictionaryShared::add_shared_to_hierarchy(InstanceKlass* k, >>>> TRAPS) { >>>> 1154?? Arguments::assert_is_dumping_archive(); >>>> 1155?? assert(!k->is_loaded(), "Invariant"); >>>> 1156?? assert_locked_or_safepoint(Compile_lock); // >>>> add_to_hierarchy assert on it. >>>> 1157?? SystemDictionary::add_to_hierarchy(k, CHECK); >>>> 1158 } >>>> >>>> Also, I think it's better to move the MutexLocker call into >>>> SystemDictionaryShared::add_shared_to_hierarchy. >>>> >>>> ======================== >>>> >>>> before: >>>> >>>> ?478???????????????? if (TRACE_RESOLVE && salvage != null) { >>>> ?479???????????????????? // Used by jlink species pregeneration >>>> plugin, see >>>> ?480???????????????????? // >>>> jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >>>> ?481 System.out.println("[SPECIES_RESOLVE] " + className + " >>>> (salvaged)"); >>>> ?482???????????????? } >>>> >>>> after: >>>> >>>> ?488???????????????? // Used by jlink species pregeneration plugin, >>>> see >>>> ?489???????????????? // >>>> jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >>>> ?490???????????????? traceResolve("[SPECIES_RESOLVE] " + className >>>> + " (salvaged)"); >>>> >>>> >>>> When tracing is disabled, this will make extra allocations and >>>> cause a small slowdown. I think it's better to >>>> >>>> ??? if ((TRACE_RESOLVE|TRACE_RESOLVE_CDS) && salvage != null) { >>>> ? ? ?? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>>> ??? } >>>> >>>> Because TRACE_RESOLVE is a static final boolean, the JIT compiler >>>> will completely optimize this block out. >>>> >>>> For the same reason, instead of calling >>>> VM.isDumpLoadedClassListSetAndOpen() every time, it's better to use >>>> a static final variable. >>>> >>>> >>>> ======================= >>>> >>>> ?698???????? if (TRACE_RESOLVE) { >>>> ?699???????????? System.out.println("[LF_RESOLVE] " + >>>> holder.getName() + " " + name + " " + >>>> ?700 shortenSignature(basicTypeSignature(type)) + (resolvedMember >>>> != null ? " (success)" : " (fail)") ); >>>> ?701???????? } >>>> ?702???????? if (VM.isDumpLoadedClassListSetAndOpen()) { >>>> ?703 GenerateJLIClassesHelper.cdsTraceResolve("[LF_RESOLVE] " + >>>> holder.getName() + " " + name + " " + >>>> ?704 shortenSignature(basicTypeSignature(type)) + (resolvedMember >>>> != null ? " (success)" : " (fail)") ); >>>> ?705???????? } >>>> ?706???????? return resolvedMember; >>>> >>>> >>>> I think the two "if" blocks should be combined similarly to >>>> ClassSpecializer::traceResolve(). >>>> >>>> ========================= >>>> ? 34 >>>> Java_java_lang_invoke_GenerateJLIClassesHelper_cdsTraceResolve(JNIEnv >>>> *env, jclass ignore, jstring line) { >>>> >>>> Maybe this should be moved to the "VM" class as well? >>>> >>>> ========================= >>>> lambdaFormInvokers.hpp: >>>> >>>> Need these declarations: >>>> >>>> ??? #include "memory/allocation.hpp"??? <-- for AllStatic >>>> ??? #include "runtime/handles.hpp"????? <-- for typeArrayHandle and >>>> Handle >>>> ??? #include "utilities/exceptions.hpp" <-- for TRAPS >>>> >>>> ??? template class GrowableArray; >>>> ========================= >>>> >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>> >>>> On 8/23/20 3:56 PM, Yumin Qi wrote: >>>>> >>>>> Hi, Mandy, Ioi and Calvin >>>>> >>>>> >>>>> ? I have updated the new changed at: >>>>> >>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-03/ >>>>> >>>>> ? In this version: >>>>> >>>>> ??? 1) Added a new API to check if flag DumpLoadedClassList set >>>>> and the file is open. If true, call into vm to print out the trace >>>>> line to the log file. >>>>> >>>>> ??????? Just thinking if we just call the cdsTraceResolve without >>>>> checking if the flag DumpLoadedClassList set and file open, this >>>>> way, the check logic is in the vm side like before, so save code >>>>> by not adding the new API. >>>>> >>>>> ??? 2) The returned holder class names now are just >>>>> 'package/className', removed head and tail. >>>>> >>>>> ??? 3)? Remove add_extra_classes from CollectClassesClosure since >>>>> after bug 8250990: >>>>> https://bugs.openjdk.java.net/browse/JDK-8250990 pushed, the >>>>> CollectClassesClosure no longer exist. >>>>> >>>>> ??? 4)? Still keep the parsing for TRACE_RESOLVE in >>>>> java/lang/invoke/GenerateJLIClassesHelper.java, so VM call its >>>>> function to regenerate holder classes. >>>>> >>>>> >>>>> ? Re-tested Mach5 tier1-4 >>>>> >>>>> Thanks >>>>> >>>>> Yumin >>>>> >>>>> >>>>> On 8/20/20 8:05 PM, Yumin Qi wrote: >>>>>> Hi, Mandy >>>>>> >>>>>> >>>>>> On 8/20/20 5:10 PM, Mandy Chung wrote: >>>>>>> >>>>>>> >>>>>>> On 8/19/20 10:14 PM, Yumin Qi wrote: >>>>>>>> >>>>>>>> HI, Mandy >>>>>>>> >>>>>>>> ? Thanks for the review, I took one day off yesterday so just >>>>>>>> got a detail look of your reply. >>>>>>>> >>>>>>>> On 8/19/20 1:30 PM, Mandy Chung wrote: >>>>>>>>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>>>>>>>> Hi, Ioi >>>>>>>>>> >>>>>>>>>> ? Thanks for review/suggestion. I have updated the webrev at >>>>>>>>>> the following link: >>>>>>>>>> >>>>>>>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>>>>>>>> >>>>>>>>> >>>>>>>>> This patch leverages the TRACE_RESOLVE output and passes the >>>>>>>>> trace output to VM.? VM then calls >>>>>>>>> GenerateJLIClassesHelper::generateMHHolderClasses to do the >>>>>>>>> parsing and generate Holder class per the resolved LFs.?? I >>>>>>>>> think there are other cleaner alternatives implementing >>>>>>>>> this.?? jlink --generate-jli-classes plugin depends the trace >>>>>>>>> output whereas -Xshare:dump does not.?? It's cleaner to skip >>>>>>>>> generating the trace output and parsing for dumping shared >>>>>>>>> archive purpose.? In addition, the implementation needs some >>>>>>>>> cleanup (I can send you feedback on the next revision) >>>>>>>>> >>>>>>>> Current patch did not change the existing code for JLinkPlugin >>>>>>>> part. I just moved the parsing code from >>>>>>>> GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java >>>>>>>> since the former is an internal class to which we shouldn't >>>>>>>> call to generate holder classes. >>>>>>>>> Instead of relying on a system property >>>>>>>>> "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better >>>>>>>>> to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this >>>>>>>>> is CDS dump time. >>>>>>>> >>>>>>>> I remember we have such API to query if flag -Xshare:dump or >>>>>>>> -Xshare:on used. Do you mean if DumpLoadedClassList flag set? >>>>>>>> This flag is the one used to log class name to list file. In >>>>>>>> GenerateLinkOptData.gmk: >>>>>>>> >>>>>>>> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) >>>>>>>> $(CLASSLIST_JAR) >>>>>>>> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >>>>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, >>>>>>>> %, $@)) >>>>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, >>>>>>>> %, $(JLI_TRACE_FILE))) >>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >>>>>>>> -XX:DumpLoadedClassList=$@.raw \ >>>>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >>>>>>>> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >>>>>>>> ??????????? -XX:SharedClassListFile=$@.interim >>>>>>>> -XX:SharedArchiveFile=$@.jsa \ >>>>>>>> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >>>>>>>> -XX:DumpLoadedClassList=$@.raw.2 \ >>>>>>>> ??????????? -XX:SharedClassListFile=$@.interim >>>>>>>> -XX:SharedArchiveFile=$@.jsa \ >>>>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >>>>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>>>> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>> ??????????? build.tools.classlist.HelloClasslist \ >>>>>>>> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >>>>>>>> ??????????? || ( \ >>>>>>>> ??????????????? exitcode=$$? ; \ >>>>>>>> ??????????????? $(ECHO) "ERROR: Failed to generate link >>>>>>>> optimization data." \ >>>>>>>> ??????????????????? "This is likely a problem with the newly >>>>>>>> built JVM/JDK." ; \ >>>>>>>> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) >>>>>>>> ; \ >>>>>>>> ??????????????? exit $$exitcode \ >>>>>>>> ??????????? ) >>>>>>>> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >>>>>>>> >>>>>>>> The $(JLI_TRACE_FILE) is generated with both >>>>>>>> -XX:DumpLoadedClassList and >>>>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current >>>>>>>> implementation, DumpLoadedClassList will turn on property >>>>>>>> java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the >>>>>>>> same output sent to stdout and log file DumpLoadedClassList >>>>>>>> specified. >>>>>>>> >>>>>>> >>>>>>> These entries are duplicated in two different files: one for >>>>>>> jlink --generate-jli-classes plugin and another for CDS use.? >>>>>>> CDS -Xshare:dump attempts to do what jlink plugin does but >>>>>>> writes those generated classes in to shared archive. >>>>>>> >>>>>>> Like the above make logic to build JDK image, the same entries >>>>>>> are written in both default-jli-trace.txt via System.out and to >>>>>>> classlist via JNI call to the VM.? I guess VM also implements >>>>>>> the logic to do some kind of diffing and write to CDS archive. >>>>>>> >>>>>> In current implementation, vm side only records the line as from >>>>>> TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at >>>>>> dump time, call back to java for parsing those recordings and >>>>>> generating the holder classes, this uses the existing JLI code. >>>>>>>> >>>>>>>> Now instead of this property, using a vm interface API to query >>>>>>>> if this flag is set, I think it is better choice. But here I am >>>>>>>> NOT sure I understand your suggestion, I think there are two >>>>>>>> choices: >>>>>>>> >>>>>>>> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not >>>>>>>> via CDS_TRACE_RESOLVE, using new API to query if >>>>>>>> DumpLoadedClassList is set >>>>>>>> >>>>>>>> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting >>>>>>>> those name, type and holder name to regenerate holder classes? >>>>>>>> >>>>>>> >>>>>>> I misunderstood that this CDS_TRACE_RESOLVE flag is set during >>>>>>> -Xshare:dump time. >>>>>>> >>>>>>> Ioi has clarified to me offline that this step is actually part >>>>>>> of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs >>>>>>> to the given class list file, i.e. you repurpose the class list >>>>>>> file to include the log output that was initially designed for >>>>>>> jlink plugin. >>>>>>> >>>>>>> To me, I'd prefer to see this support depending on `jlink >>>>>>> --generate-jli-classes` which is an existing functionality and >>>>>>> much cleaner.? This way this does not require any VM change.?? >>>>>>> It will generate the holder classes in the custom image per the >>>>>>> application-specific config file. >>>>>>> >>>>>>> What it means is that: a customer would need to create a custom >>>>>>> image with their application-specific config file. It might need >>>>>>> a new CDS? option to specify a separate TRACE_RESOLVE file.?? It >>>>>>> would turn on this feature by default by defining a default path >>>>>>> of the log file if it helps. >>>>>>> >>>>>> So for now, I would implement an API to query if flag >>>>>> DumpLoadedClassList set in cmd line, remove new added property of >>>>>> java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address >>>>>> custom image with CDS in future in a separate issue. >>>>>> >>>>>> >>>>>> Thanks >>>>>> >>>>>> Yumin >>>>>> >>>>>>> I understand that this is not the existing CDS work flow and CDS >>>>>>> archive does not require to run on a custom image.?? I see the >>>>>>> value of this approach which can prepare customers to start >>>>>>> building and using its own custom image. >>>>>>> >>>>>>> Of course the implementation would be much simpler (adding a >>>>>>> flag to write these traces to a given file path, that is). >>>>>>> >>>>>>> Mandy >>>> >> From ioi.lam at oracle.com Wed Aug 26 23:21:05 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 26 Aug 2020 16:21:05 -0700 Subject: RFR 8251557 Avoid dumping unused symbols/strings into the CDS archive In-Reply-To: <5eb5f983-bebd-8f06-bc2d-5c6d83ea5613@oracle.com> References: <6d64004b-4aeb-f95f-1551-d6d608191f3a@oracle.com> <699f0beb-5272-8ca8-a644-5383c4c4f8e2@oracle.com> <69beaf48-8c3e-bbd8-d964-99d5becd7e1b@oracle.com> <38a3d6da-c5ea-b2cd-50c7-6910d06a3b7f@oracle.com> <6db5f819-8dba-84d6-961a-d03841858a9a@oracle.com> <5eb5f983-bebd-8f06-bc2d-5c6d83ea5613@oracle.com> Message-ID: <2b44e5b4-c7d9-7396-5978-b8592cf71c34@oracle.com> Hi Yumin, Thanks for the review. On 8/26/20 1:44 PM, Yumin Qi wrote: > > Hi, Ioi > > ?? Looks good to me. One minor comment: > > ?? stringTable.cpp: > > 746 unsigned int hash = java_lang_String::hash_code(s); > > hash is not used so this can be removed. > No new webrev needed. Actually hash is used a few lines down, so it can't be removed: 753???? _writer->add(hash, CompressedOops::encode(new_s)); Thanks - Ioi > Thanks > Yumin > > On 8/26/20 11:30 AM, Ioi Lam wrote: >> >> >> On 8/26/20 11:13 AM, calvin.cheung at oracle.com wrote: >>> >>> On 8/26/20 10:18 AM, Ioi Lam wrote: >>>> >>>> >>>> On 8/26/20 10:08 AM, Ioi Lam wrote: >>>>> Hi Calvin, >>>>> >>>>> Thanks for the review. >>>>> >>>>> On 8/26/20 9:40 AM, calvin.cheung at oracle.com wrote: >>>>>> Hi Ioi, >>>>>> >>>>>> The proposed change does make the code easier to read than before. >>>>>> >>>>>> I've comments on 2 files: >>>>>> >>>>>> heapShared.hpp: >>>>>> >>>>>> 42 class DumpedInternedStrings; >>>>>> >>>>>> Instead of forward declaration, is it possible to move the class >>>>>> definition there since it is defined at the end of the file? >>>>>> >>>>> >>>>> I moved it as you suggested. >>>> >>>> Oops, sorry I hit "Send" too fast. DumpedInternedStrings depends on >>>> function declared by HeapShared, so it must be declared after >>>> HeapShared >>> I see. So the change looks good. >>>> >>>> Thanks >>>> - Ioi >>>> >>>> >>>>> >>>>>> archiveBuilder.hpp: >>>>>> >>>>>> 37 class DumpRegion; >>>>>> >>>>>> The above forward declaration was removed but I think it's >>>>>> required for the following: >>>>>> >>>>> >>>>> I removed the forward declaration because DumpRegion is declared >>>>> in archiveUtils.hpp. Since JDK-8252056, archiveUtils.hpp is >>>>> already included by archiveBuilder.hpp: >>>>> >>>>> http://hg.openjdk.java.net/jdk/jdk/rev/ba51bc0772ac >>>>> >>>>>> ?132 DumpRegion* _rw_region; >>>>>> ?133?? DumpRegion* _ro_region; >>>>>> >>>>>> Also, the following constructor needs to be moved to >>>>>> archiveBuilder.cpp or I got compile error about invalid use of >>>>>> incomplete type 'class DumpRegion'. >>>>>> >>>>>> ?158???? OtherROAllocMark() { >>>>>> ?159?????? _oldtop = _singleton->_ro_region->top(); >>>>>> ?160???? } >>>>>> >>>>> Maybe you? applied the patch on a repo without the JDK-8252056 fix? >>> Right, I haven't updated my repo with the JDK-8252056 fix yet. >>>>> >>>>> I did find a problem with archiveUtils.hpp. It was using the Klass >>>>> type without including klass.hpp, so I added >>>>> >>>>> diff -r 3920d1a29976 src/hotspot/share/memory/archiveBuilder.hpp >>>>> --- a/src/hotspot/share/memory/archiveBuilder.hpp??? Wed Aug 19 >>>>> 14:14:20 2020 -0700 >>>>> +++ b/src/hotspot/share/memory/archiveBuilder.hpp??? Wed Aug 26 >>>>> 10:06:59 2020 -0700 >>>>> @@ -27,6 +27,7 @@ >>>>> >>>>> ?#include "memory/archiveUtils.hpp" >>>>> ?#include "memory/metaspaceClosure.hpp" >>>>> +#include "oops/klass.hpp" >>>>> ?#include "utilities/bitMap.hpp" >>>>> ?#include "utilities/growableArray.hpp" >>>>> ?#include "utilities/hashtable.hpp" >>>>> >>> Since you're including klass.hpp, maybe the following forward >>> declaration can be removed? >>> >>> 36 class Klass; >>> >> >> Right. I will remove that. >> >> Thanks >> - Ioi >> >>> thanks, >>> >>> Calvin >>> >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>>> thanks, >>>>>> Calvin >>>>>> >>>>>> >>>>>> On 8/19/20 2:39 PM, Ioi Lam wrote: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8251557 >>>>>>> http://cr.openjdk.java.net/~iklam/jdk16/8251557-cds-dont-dump-unused-symbols.v01/ >>>>>>> >>>>>>> >>>>>>> Part of CDS code clean up: >>>>>>> >>>>>>> Currently all entries in the SymbolTable/StringTable are dumped >>>>>>> into >>>>>>> the CDS archive. The problems are >>>>>>> >>>>>>> (1) We end up writing many unused items, such as the mangled names >>>>>>> ? ? of hidden classes. >>>>>>> (2) We have to scan the symbol table inside a safepoint, which had >>>>>>> ? ? caused bugs before (JDK-8245264). >>>>>>> >>>>>>> Since JDK-8250990, we already maintain all used Symbols in a >>>>>>> growable >>>>>>> array. We can dump the shared symbol table using this array, and >>>>>>> avoid walking the SymbolTable. >>>>>>> >>>>>>> For StringTable, we scan all archived classes, and dump only the >>>>>>> interned strings used by those classes (plus extra strings >>>>>>> specified by SharedArchiveConfigFile) into the CDS interned string >>>>>>> table. >>>>>>> >>>>>>> This reduces complexity in the CDS code, and reduce the size of >>>>>>> the CDS archives. >>>>>>> >>>>>>> Testing - tiers 1-4 >>>>>>> >>>>>>> Archive file size for "java -Xshare:dump" >>>>>>> >>>>>>> ?? old: 39,368 symbols 7,028 strings 11,784,200 bytes >>>>>>> ?? new: 37,387 symbols 7,020 strings 11,683,320 bytes (~0.9% >>>>>>> reduction) >>>>>>> >>>>>>> Thanks >>>>>>> - Ioi >>>>>>> >>>>>>> >>>>> >>>> >> From yumin.qi at oracle.com Wed Aug 26 23:39:44 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Wed, 26 Aug 2020 16:39:44 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <8fca4eb0-119f-83cb-3c2c-9400b66f027b@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> <1ca66b6a-8a23-c4b1-c95d-10fc23b668f8@oracle.com> <6344748b-f484-e743-8bef-5750d0b49155@oracle.com> <68edf582-841b-890e-0951-8f8ecd5d132a@oracle.com> <8fca4eb0-119f-83cb-3c2c-9400b66f027b@oracle.com> Message-ID: <92c9d3a3-118e-d08d-3519-642050a41b09@oracle.com> HI, Calvin ? Thanks for re-review. On 8/26/20 4:05 PM, calvin.cheung at oracle.com wrote: > Hi Yumin, > > Some minor comments: > > metaspaceShared.cpp > > ? 45 #include "memory/lambdaFormInvokers.hpp" > ? 46 #include "memory/heapShared.inline.hpp" > > Please swap the order of the above 2 lines. This will be moved to classfile directory and ordered. > ------ > lambdaFormInvokers.cpp > > 64?? guarantee(helper_klass != NULL, "java/lang/invoke/GenerateJLIClassesHelper exist!"); > > I think it is clearer to say: "java/lang/invoke/GenerateJLIClassesHelper must exist!". Will change to your suggestion. > > 70???? record += strlen(lambda_form_invoker_tag()) + 1; // @lambda_form_invoker [...] > > How about ?skip the @lambda_form_invoker prefix? for the comment? Good suggestion, will change. > ------ > lambdaFormInvokers.hpp > > 34?? static void reload_class(Handle name, typeArrayHandle bytes, TRAPS); > > reload_class could be private since it?s called only from lambdaFormInvokers.cpp. > Yes, it could be private. > ------ > > DumpSymbolAndStringTable.java > > 37???? private static final String my_string = "DumpSymbolAndStringTable"; > > The above declaration is unnecessary. I've tried your patch and the test passed without it. > I will retest without it, if no failure will revert the change. Thanks Yumin > ------ > > No need to spin another webrev for the above changes. > > thanks, > Calvin > On 8/25/20 5:17 PM, Yumin Qi wrote: >> Hi, Coleen and all >> >> ? I have updated at new link: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-05/ >> >> ? I changed find_class and add_to_hierarchy to public access in SystemDictionary for access from outside. >> >> ? new added find_class removed from ClassLoaderData. >> >> ? new added add_shared_to_hierarchy removed from SystemDictionaryShared. >> >> ? re-tested local on runtime/cds >> >> >> ? Thanks >> >> ? Yumin >> >> >> On 8/25/20 12:41 PM, Coleen Phillimore wrote: >>> >>> Hi, I'm sorry I haven't been watching this change. >>> >>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html >>> >>> Can you move find_class() into systemDictionary.hpp/cpp with the rest of the find_class().? This is strange here.? You don't want to expose that the dictionary is here to the rest of the code. The caller should call one of the SystemDictionary versions, or add this to SystemDictionary with the other functions with the same name. >>> >>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/memory/lambdaFormInvokers.cpp.html >>> >>> These new files belongs in classfile.? It's closer to systemDictionaryShared code than memory code. >>> >>> ?? 2? * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All rights reserved. >>> >>> >>> Also copyright shouldn't start with 2012. >>> >>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/src/hotspot/share/classfile/systemDictionaryShared.cpp.udiff.html >>> >>> This add_to_hierarchy is a strange delegation and a surprise that it's also defined here.? Please call this inline from lambdaFormInvokers, ie take the Compile_lock and call SystemDictionary::add_to_hierarchy(). >>> >>> I don't have any deeper comments. >>> >>> Thanks, >>> Coleen >>> >>> On 8/25/20 2:59 PM, Yumin Qi wrote: >>>> HI, Ioi >>>> >>>> ? Thanks for the re-review, updated webrev: >>>> >>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/ >>>> >>>> ? Changed according to your suggestion, the only a little difference is move traceResolve(String line) to InvokerBytecodeGenerator as a static package public function so other package classes can call it. >>>> >>>> ? re-tested local on runtime/cds >>>> >>>> >>>> Thanks >>>> >>>> Yumin >>>> >>>> >>>> On 8/24/20 4:23 PM, Ioi Lam wrote: >>>>> Hi Yumin, >>>>> >>>>> This looks good overall. Here are my comments: >>>>> >>>>> ===================== >>>>> 6065???? size_t new_id = Atomic::add(&counter, (size_t)1); >>>>> 6066???? jio_snprintf(addr_buf, 20, INTPTR_FORMAT, new_id); >>>>> >>>>> I think this should be SIZE_FORMAT >>>>> >>>>> ===================== >>>>> ? 65 class KlassFactory : AllStatic { >>>>> ? 66 >>>>> ? 67?? // approved clients >>>>> ? 68?? friend class ClassLoader; >>>>> ? 69?? friend class ClassLoaderExt; >>>>> ? 70?? friend class SystemDictionary; >>>>> ? 71?? friend class LambdaFormInvokers; >>>>> ? 72 >>>>> ? 73? private: >>>>> ? 74?? static InstanceKlass* create_from_stream(ClassFileStream* stream, >>>>> >>>>> I think instead of adding everyone who uses create_from_stream as a friend class, we should just change create_from_stream into a public function and remove the friend declarations. >>>>> >>>>> ===================== >>>>> ?146?? // add to hierarchy and set state to loaded. >>>>> ?147?? { >>>>> ?148???? MutexLocker mu_r(THREAD, Compile_lock); // add_shared_to_hierarchy asserts this. >>>>> ?149 SystemDictionaryShared::add_shared_to_hierarchy(result, THREAD); >>>>> ?150?? } >>>>> >>>>> I think the function name can be changed to SystemDictionaryShared::add_to_hierarchy as the "_shared" seems redundant. The "set state to loaded" comment seems wrong, as we have the assert on line 1155. I think the comment can be removed. >>>>> >>>>> 1153 void SystemDictionaryShared::add_shared_to_hierarchy(InstanceKlass* k, TRAPS) { >>>>> 1154?? Arguments::assert_is_dumping_archive(); >>>>> 1155?? assert(!k->is_loaded(), "Invariant"); >>>>> 1156?? assert_locked_or_safepoint(Compile_lock); // add_to_hierarchy assert on it. >>>>> 1157?? SystemDictionary::add_to_hierarchy(k, CHECK); >>>>> 1158 } >>>>> >>>>> Also, I think it's better to move the MutexLocker call into SystemDictionaryShared::add_shared_to_hierarchy. >>>>> >>>>> ======================== >>>>> >>>>> before: >>>>> >>>>> ?478???????????????? if (TRACE_RESOLVE && salvage != null) { >>>>> ?479???????????????????? // Used by jlink species pregeneration plugin, see >>>>> ?480???????????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >>>>> ?481 System.out.println("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>>>> ?482???????????????? } >>>>> >>>>> after: >>>>> >>>>> ?488???????????????? // Used by jlink species pregeneration plugin, see >>>>> ?489???????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin >>>>> ?490???????????????? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>>>> >>>>> >>>>> When tracing is disabled, this will make extra allocations and cause a small slowdown. I think it's better to >>>>> >>>>> ??? if ((TRACE_RESOLVE|TRACE_RESOLVE_CDS) && salvage != null) { >>>>> ? ? ?? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); >>>>> ??? } >>>>> >>>>> Because TRACE_RESOLVE is a static final boolean, the JIT compiler will completely optimize this block out. >>>>> >>>>> For the same reason, instead of calling VM.isDumpLoadedClassListSetAndOpen() every time, it's better to use a static final variable. >>>>> >>>>> >>>>> ======================= >>>>> >>>>> ?698???????? if (TRACE_RESOLVE) { >>>>> ?699???????????? System.out.println("[LF_RESOLVE] " + holder.getName() + " " + name + " " + >>>>> ?700 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); >>>>> ?701???????? } >>>>> ?702???????? if (VM.isDumpLoadedClassListSetAndOpen()) { >>>>> ?703 GenerateJLIClassesHelper.cdsTraceResolve("[LF_RESOLVE] " + holder.getName() + " " + name + " " + >>>>> ?704 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); >>>>> ?705???????? } >>>>> ?706???????? return resolvedMember; >>>>> >>>>> >>>>> I think the two "if" blocks should be combined similarly to ClassSpecializer::traceResolve(). >>>>> >>>>> ========================= >>>>> ? 34 Java_java_lang_invoke_GenerateJLIClassesHelper_cdsTraceResolve(JNIEnv *env, jclass ignore, jstring line) { >>>>> >>>>> Maybe this should be moved to the "VM" class as well? >>>>> >>>>> ========================= >>>>> lambdaFormInvokers.hpp: >>>>> >>>>> Need these declarations: >>>>> >>>>> ??? #include "memory/allocation.hpp"??? <-- for AllStatic >>>>> ??? #include "runtime/handles.hpp"????? <-- for typeArrayHandle and Handle >>>>> ??? #include "utilities/exceptions.hpp" <-- for TRAPS >>>>> >>>>> ??? template class GrowableArray; >>>>> ========================= >>>>> >>>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> >>>>> >>>>> On 8/23/20 3:56 PM, Yumin Qi wrote: >>>>>> >>>>>> Hi, Mandy, Ioi and Calvin >>>>>> >>>>>> >>>>>> ? I have updated the new changed at: >>>>>> >>>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-03/ >>>>>> >>>>>> ? In this version: >>>>>> >>>>>> ??? 1) Added a new API to check if flag DumpLoadedClassList set and the file is open. If true, call into vm to print out the trace line to the log file. >>>>>> >>>>>> ??????? Just thinking if we just call the cdsTraceResolve without checking if the flag DumpLoadedClassList set and file open, this way, the check logic is in the vm side like before, so save code by not adding the new API. >>>>>> >>>>>> ??? 2) The returned holder class names now are just 'package/className', removed head and tail. >>>>>> >>>>>> ??? 3)? Remove add_extra_classes from CollectClassesClosure since after bug 8250990: https://bugs.openjdk.java.net/browse/JDK-8250990 pushed, the CollectClassesClosure no longer exist. >>>>>> >>>>>> ??? 4)? Still keep the parsing for TRACE_RESOLVE in java/lang/invoke/GenerateJLIClassesHelper.java, so VM call its function to regenerate holder classes. >>>>>> >>>>>> >>>>>> ? Re-tested Mach5 tier1-4 >>>>>> >>>>>> Thanks >>>>>> >>>>>> Yumin >>>>>> >>>>>> >>>>>> On 8/20/20 8:05 PM, Yumin Qi wrote: >>>>>>> Hi, Mandy >>>>>>> >>>>>>> >>>>>>> On 8/20/20 5:10 PM, Mandy Chung wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 8/19/20 10:14 PM, Yumin Qi wrote: >>>>>>>>> >>>>>>>>> HI, Mandy >>>>>>>>> >>>>>>>>> ? Thanks for the review, I took one day off yesterday so just got a detail look of your reply. >>>>>>>>> >>>>>>>>> On 8/19/20 1:30 PM, Mandy Chung wrote: >>>>>>>>>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>>>>>>>>> Hi, Ioi >>>>>>>>>>> >>>>>>>>>>> ? Thanks for review/suggestion. I have updated the webrev at the following link: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This patch leverages the TRACE_RESOLVE output and passes the trace output to VM.? VM then calls GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing and generate Holder class per the resolved LFs.?? I think there are other cleaner alternatives implementing this.?? jlink --generate-jli-classes plugin depends the trace output whereas -Xshare:dump does not.?? It's cleaner to skip generating the trace output and parsing for dumping shared archive purpose.? In addition, the implementation needs some cleanup (I can send you feedback on the next revision) >>>>>>>>>> >>>>>>>>> Current patch did not change the existing code for JLinkPlugin part. I just moved the parsing code from GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java since the former is an internal class to which we shouldn't call to generate holder classes. >>>>>>>>>> Instead of relying on a system property "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. >>>>>>>>> >>>>>>>>> I remember we have such API to query if flag -Xshare:dump or -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This flag is the one used to log class name to list file. In GenerateLinkOptData.gmk: >>>>>>>>> >>>>>>>>> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR) >>>>>>>>> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >>>>>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) >>>>>>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $(JLI_TRACE_FILE))) >>>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw \ >>>>>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>>> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >>>>>>>>> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >>>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >>>>>>>>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>>>>>>>> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >>>>>>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw.2 \ >>>>>>>>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>>>>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >>>>>>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>>>>>> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>>>>>> ??????????? build.tools.classlist.HelloClasslist \ >>>>>>>>> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >>>>>>>>> ??????????? || ( \ >>>>>>>>> ??????????????? exitcode=$$? ; \ >>>>>>>>> ??????????????? $(ECHO) "ERROR: Failed to generate link optimization data." \ >>>>>>>>> ??????????????????? "This is likely a problem with the newly built JVM/JDK." ; \ >>>>>>>>> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >>>>>>>>> ??????????????? exit $$exitcode \ >>>>>>>>> ??????????? ) >>>>>>>>> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >>>>>>>>> >>>>>>>>> The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current implementation, DumpLoadedClassList will turn on property java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same output sent to stdout and log file DumpLoadedClassList specified. >>>>>>>>> >>>>>>>> >>>>>>>> These entries are duplicated in two different files: one for jlink --generate-jli-classes plugin and another for CDS use.? CDS -Xshare:dump attempts to do what jlink plugin does but writes those generated classes in to shared archive. >>>>>>>> >>>>>>>> Like the above make logic to build JDK image, the same entries are written in both default-jli-trace.txt via System.out and to classlist via JNI call to the VM.? I guess VM also implements the logic to do some kind of diffing and write to CDS archive. >>>>>>>> >>>>>>> In current implementation, vm side only records the line as from TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at dump time, call back to java for parsing those recordings and generating the holder classes, this uses the existing JLI code. >>>>>>>>> >>>>>>>>> Now instead of this property, using a vm interface API to query if this flag is set, I think it is better choice. But here I am NOT sure I understand your suggestion, I think there are two choices: >>>>>>>>> >>>>>>>>> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set >>>>>>>>> >>>>>>>>> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting those name, type and holder name to regenerate holder classes? >>>>>>>>> >>>>>>>> >>>>>>>> I misunderstood that this CDS_TRACE_RESOLVE flag is set during -Xshare:dump time. >>>>>>>> >>>>>>>> Ioi has clarified to me offline that this step is actually part of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the given class list file, i.e. you repurpose the class list file to include the log output that was initially designed for jlink plugin. >>>>>>>> >>>>>>>> To me, I'd prefer to see this support depending on `jlink --generate-jli-classes` which is an existing functionality and much cleaner.? This way this does not require any VM change.?? It will generate the holder classes in the custom image per the application-specific config file. >>>>>>>> >>>>>>>> What it means is that: a customer would need to create a custom image with their application-specific config file. It might need a new CDS? option to specify a separate TRACE_RESOLVE file.?? It would turn on this feature by default by defining a default path of the log file if it helps. >>>>>>>> >>>>>>> So for now, I would implement an API to query if flag DumpLoadedClassList set in cmd line, remove new added property of java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address custom image with CDS in future in a separate issue. >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Yumin >>>>>>> >>>>>>>> I understand that this is not the existing CDS work flow and CDS archive does not require to run on a custom image.?? I see the value of this approach which can prepare customers to start building and using its own custom image. >>>>>>>> >>>>>>>> Of course the implementation would be much simpler (adding a flag to write these traces to a given file path, that is). >>>>>>>> >>>>>>>> Mandy >>>>> >>> From kim.barrett at oracle.com Wed Aug 26 23:39:50 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 26 Aug 2020 19:39:50 -0400 Subject: RFR: 8252093: formula used to calculate decaying variance in numberSeq In-Reply-To: <54b1cc85-9f2d-cdf8-c513-1cbee4fa7f3d@oracle.com> References: <54b1cc85-9f2d-cdf8-c513-1cbee4fa7f3d@oracle.com> Message-ID: > On Aug 26, 2020, at 9:59 AM, Albert Yang wrote: > > Hi, > > I am sending this to both runtime-dev and gc-dev, since the changed code lives in runtime, but it's used by GC. > > https://bugs.openjdk.java.net/browse/JDK-8252093 > http://cr.openjdk.java.net/~ayang/webrevs/8252093/webrev.1/ > > Both the code and the literature use `alpha`, but its meaning is actually inverted. I could include this refactoring in later revision if people feel that it's desirable. > > Tested: tier1 (locally) and tier1-3 (mach5) > > -- > /Albert Nice find. Looks good. I wonder if this has any noticeable performance effect. For G1 it affects many/all policy-based predictions, since it affects the G1Predictions::predict function. From david.holmes at oracle.com Thu Aug 27 02:08:48 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 27 Aug 2020 12:08:48 +1000 Subject: RFR 8222671 : thread_large/thread_large.java times out on MacOSX In-Reply-To: <226bee07-711f-9687-8318-b72826e9e32d@oracle.com> References: <7465F494-9793-42D1-9A46-910FED5D2D48@me.com> <256bdc79-2c29-512c-9d98-f9d447dcbbb0@oracle.com> <84F93270-12BB-41FE-82C3-3225561A906F@oracle.com> <95b46d0f-8a81-10c6-78bb-a3abbd7cb955@oracle.com> <55ae5763-476b-562c-c7cc-a913992d304c@oracle.com> <4b9dfeda-c6f7-3ee5-fb23-85cf58f1130a@oracle.com> <2BAD9BBF-B8F9-45E7-8378-E69622856C7A@oracle.com> <44706c40-d3f2-6b63-1491-c4412a0c4347@oracle.com> <62F99ACE-282A-47B8-A213-626108712F46@oracle.com> <226bee07-711f-9687-8318-b72826e9e32d@oracle.com> Message-ID: Hi Gerard, I would have preferred the @bug to remain for traceability purposes (without having to check repo history). I would also have kept the main/othervm as this is something of a stress test and I would be concerned that the hosting VM might be de-stabilised in some way when a huge number of threads are created. But I don't insist on any further changes and this test has already consumed far too many resources IMO so feel free to push as-is. Thanks, David On 20/08/2020 5:55 am, gerard ziemski wrote: > hi Igor, > > I have responses to your questions inline: > > bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 > open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev5 > testing: Mach5 hs-tier1,2,3,4,5 in progress... > > > On 8/19/20 12:25 PM, Igor Ignatyev wrote: >> Hi Gerard, >> >> please see my comments inline. >> >> -- Igor >> >>> On Aug 19, 2020, at 9:53 AM, gerard ziemski >>> wrote: >>> >>> hi Igor, >>> >>> Thank you for the review. Here is the webrev, but please see my >>> comments inline for more: >>> >>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev4 >>> testing: Mach5 hs-tier1,2,3,4,5 in progress... >>> >>> >>> >>> On 8/18/20 12:18 PM, Igor Ignatyev wrote: >>>> Hi Gerard, >>>> >>>> looks be >>>> 1. >>>>> ?? 26 * @bug 8222671 >>>> @bug tag should refer to a product bug which can be reproduced w/ >>>> this test, not the bug id used to push a test. >>> The bug 8222671 is the issue here, i.e. timeouts in the existing >>> test, which is what we are trying to fix, so we have a new test that >>> replaces the old and less robust one. I did try to find an issue that >>> originated the old test and I couldn?t, but if you know how to find, >>> I can add a reference to it in the new test. To me, the old test >>> looks more like a general functionality test though. >> I'd just remove @bug tag, b/c from my point of view listing all test >> bugs which caused modification of the test in @bug is just useless and >> pollute metadata. others have other opinion though, so I don't insist >> on removal. > You are the expert here, removed. > > >>>> 2. >>>>> ?? 28 * @run main/othervm ThreadCountLimit >>>> what's the reason to run in othervm mode? >>> Oh, I just copied that from the existing test, without giving it any >>> thought - I will remove it. >> in hotspot tests, we tend not to relay on implicit @run (as it's error >> prone) and always have explicit `@run`, that's to say, please add >> `@run main ThreadCountLimit` > Is there a template for a new test, or a wiki page that explains all > this? I learned quite a bit writing this test, but it would be nice to > have some documentation I could use as a reference in the future. > > >>>> 3. >>>>> ?? 34 public class ThreadCountLimit extends Object { >>>> you don't need to explicitly extend Object. >>> True, but I like the code explicit like that if that?s OK. >> fine w/ me, but it's kinda against regular java code guidelines and >> doesn't bring any information. > Fixed. > > >>>> 4. in ThreadCountLimit::run() >>>>> ?? 67?????? } catch (InterruptedException ex) { >>>>> ?? 68?????? } >>>> shouldn't this mark the test as failed? >>> Good question. In a perfect world we would want to know if a thread >>> got unexpectedly interrupted, but realistically a process might >>> (will?) exhibit spurious interruptions from the OS sooner or later, >>> which are not the focus of this test, so instead of marking it a >>> failure and making someone look into it, only later to determine that >>> it was just a spurious interrupt, we simply ignore it and focus on >>> the core goal of this test instead. >> I'm not sure what you are refer here by 'spurious interruptions', java >> threads must be interrupted only if someone interrupted them, all >> other behaviors are JVM bugs and are to be analyzed. given the goal of >> this test is to check how JVM behaves when lots of threads have been >> created and started, I see catching unexpected InterruptedException as >> a valid failure mode of this test. > I think I might be mixing up native wait/notify on mutexes here, where a > wait on a native mutex can be spuriously awaken. > > According to > https://www.ibm.com/developerworks/library/j-jtp05236/index.html Thread > interruption is a cooperative mechanism, and since we are not > interrupting any of the blocking methods we use in the test ourselves, > it would be really curious if we ever did get InterruptedException, so > it might be worth it to report it. > > I tightened up the blocking code and its try/catch code block, to make > it to clear what is the subject to interrupts and we will now throw an > Error if we ever catch one (never?) Have we ever seen one of those > unexpected interrupts in our testing so far? > > > Thank you for the feedback. > > > cheers > >>> >>> I do see, however, that in one place I report it as Error and ignore >>> it in the other instance, so I fixed that inconsistency. I added >>> comments as well. >>> >>>> Thanks, >>>> -- Igor >>>> >>>>> On Aug 18, 2020, at 7:38 AM, gerard ziemski >>>>> wrote: >>>>> >>>>> Thank you David for the review! >>>>> >>>>> I incorporated all your feedback. >>>>> >>>>> I'm surprised that "assert" is not available by default in our >>>>> tests though - what is a reason for that? Of all the places where >>>>> asserts should be available, a testing suite would seem to be it to >>>>> me. >>>>> >>>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>>>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev3 >>>>> testing: passes locally, Mach5 hs-tier1,2,3,4,5 in progress... >>>>> >>>>> >>>>> cheers >>>>> >>>>> On 8/18/20 12:15 AM, David Holmes wrote: >>>>>> Hi Gerard, >>>>>> >>>>>> On 18/08/2020 1:28 am, gerard ziemski wrote: >>>>>>> Thank you David and Igor for the feedback! >>>>>>> >>>>>>> I rewrote the test again, this time basing it off >>>>>>> test/hotspot/jtreg/runtime/Thread/* tests. >>>>>>> >>>>>>> For macOS and Linux we create threads until we hit a process >>>>>>> limit thread count. Windows, however, allows unlimited (?) number >>>>>>> of threads, though the process gradually slows down as the count >>>>>>> grows, so I had to add time limit and I set it to 5 sec. I chose >>>>>>> 5 sec because the value is well below jtreg time out (i.e. 120s), >>>>>>> but still quite above how long my laptop needs to hit the process >>>>>>> thread count limit (i.e. my Mac laptop needs about 1 sec to hit >>>>>>> that limit). >>>>>>> >>>>>>> Otherwise there are no other hardwired values or assumptions (I >>>>>>> hope). What do you think of the test now? >>>>>> Not unreasonable :) It could still have tried for an upper limit >>>>>> of 10000 instead of using the time-limit, but it makes no real >>>>>> difference. >>>>>> >>>>>> A few nits/suggestions: >>>>>> >>>>>> Please add an @bug line to the test comment. >>>>>> >>>>>> ? 49???????? if ((this.index % 250) == 0) { >>>>>> >>>>>> Outside of the constructor it is poor style to use this.x to refer >>>>>> to fields methods (unless needed due to local variable shadowing). >>>>>> This applies throughout the test. >>>>>> >>>>>> ? 54???????? doWork(); >>>>>> >>>>>> Not sure why you factored things out the way you did - everything >>>>>> could go in run(). >>>>>> >>>>>> ? 61?????? setName(""+this.index); >>>>>> >>>>>> This could go in constructor, but I presume it adds something to >>>>>> the workload? Is the use of ""+index deliberate in that regard >>>>>> (generate string concatenation work), rather than using >>>>>> String.valueOf(index)? >>>>>> >>>>>> ? 65?????? assert(this.index == Integer.parseInt(getName())); >>>>>> >>>>>> We never run tests with Java asserts enabled so this is "dead" code. >>>>>> >>>>>> ? 87?????????? // Windows path >>>>>> >>>>>> Windows path or system with very large ulimit? >>>>>> >>>>>> ?? 92???? } catch (OutOfMemoryError e) { >>>>>> >>>>>> I suggest checking this is actually the "unable to create native >>>>>> thread" OOME rather than a general Java OOME (if run with a small >>>>>> heap for example you might get OOME growing the ArrayList). >>>>>> >>>>>> ? 110???? String cmd = System.getProperty("sun.java.command"); >>>>>> ? 111???? if (cmd != null && >>>>>> !cmd.startsWith("com.sun.javatest.regtest.agent.MainWrapper")) { >>>>>> ? 112?????? // Exit with success in a non-JavaTest environment: >>>>>> ? 113?????? System.exit(0); >>>>>> ? 114???? } >>>>>> >>>>>> I never noticed when this was used in the little set of >>>>>> runtime/Thread tests! I don't know what the rationale was but it >>>>>> is completely unnecessary as a normal return from main would be >>>>>> the equivalent of exit(0) - so the tests work fine without >>>>>> System.exit when run standalone. Please don't emulate this part of >>>>>> the existing tests. I'll file a RFE to remove it from the other >>>>>> tests. >>>>>> >>>>>> I won't be around for any further follow up for a week, so feel >>>>>> free to proceed once Igor is okay with things. >>>>>> >>>>>> Thanks, >>>>>> David >>>>>> ----- >>>>>> >>>>>>> bug link at https://bugs.openjdk.java.net/browse/JDK-8222671 >>>>>>> open webrev at http://cr.openjdk.java.net/~gziemski/8222671_rev2 >>>>>>> testing: Mach5 hs-tier1,2,3,4,5 in progress... >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 8/7/20 2:41 PM, Igor Ignatyev wrote: >>>>>>>>> On Aug 6, 2020, at 12:33 AM, David Holmes >>>>>>>>> > wrote: >>>>>>>>> >>>>>>>>> Basing the new test around the existing thread stress tests was >>>>>>>>> not unreasonable but I don't think we're actively adding to >>>>>>>>> those tests. I would check with Igor whether it is better to >>>>>>>>> just add this under runtime/thread rather than vmTestBase. >>>>>>>> Hi Gerard, >>>>>>>> >>>>>>>> David is right, we are trying to get rid of vmTestbase directory >>>>>>>> and move all tests into corresponding component directories >>>>>>>> cleaning them up at the same time. so runtime/thread directory >>>>>>>> sound like a much better location for a new test. >>>>>>>> >>>>>>>> I don't think you need to cary all this baggage of 'VM testbase >>>>>>>> keywords', 'converted from ...' and the rest, I'd suggest you >>>>>>>> just use the text from 'DESCRIPTION' as @summary. I'd also >>>>>>>> remove System.exit call and would just make thread009::run to >>>>>>>> return bool. it's also unclear to me if someone is ever going to >>>>>>>> change THREADS_EXPECTED , TIMEOUT, and DEBUG_MODE values, I'd >>>>>>>> just hardcode them in the test and get rid of parsing. it seems >>>>>>>> that debug output won't clutter output, so you might want to >>>>>>>> always print them and remove DEBUG_MODE field. >>>>>>>> >>>>>>>> using `assert` in test code isn't a good idea as it will work >>>>>>>> only if someone runs test w/ `-ea` which not all people do. it's >>>>>>>> better to replace w/ `if (p) throw new Error(...)`? or use >>>>>>>> jdk.test.lib.Asserts class. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> -- Igor > From david.holmes at oracle.com Thu Aug 27 02:22:12 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 27 Aug 2020 12:22:12 +1000 Subject: RFR 8251414: Add test that invokeinterface of a protected method in java.lang.Object throws NoSuchMethodError In-Reply-To: <64ae2e82-c08c-66ac-2a8d-13b0afbf42ab@oracle.com> References: <97328605-aa99-51db-25eb-e1465919b8db@oracle.com> <6173e896-dd66-a8ba-f524-02e9e350b654@oracle.com> <64ae2e82-c08c-66ac-2a8d-13b0afbf42ab@oracle.com> Message-ID: Hi Harold, On 19/08/2020 3:36 am, Harold Seigel wrote: > Hi David, > > The test was adapted from an existing test that used reflection. It > probably doesn't need to use it. Okay ... > If test file InterfaceObj.jasm is written in java, as described in the > file's comments, then its compilation fails with the following errors: > > javac I.java InterfaceObj.java InterfaceObjectTest.java > InterfaceObj.java:3: error: cannot find symbol > ??????? I.finalize(); > ???????? ^ > ? symbol:?? method finalize() > ? location: interface I > InterfaceObj.java:9: error: cannot find symbol > ??????? I.clone(); > ???????? ^ > ? symbol:?? method clone() > ? location: interface I Yes I expect it would as those are not static methods. :) The comment block itself is incorrect: 29 static void f(I intf) throws Throwable { 30 I.finalize(); 31 } Should be intf.finalize() 35 static void c(I intf) throws Throwable { 36 I.clone(); 37 } Should be intf.clone() Of course after that change it still won't compile as finalize/clone are not defined in I and are non-public in Object - hence the need for jasm. Cheers, David > Thanks, Harold > > > On 8/17/2020 5:53 PM, David Holmes wrote: >> Hi Harold, >> >> On 18/08/2020 12:26 am, Harold Seigel wrote: >>> Hi David, >>> >>> This change got pushed before I saw your comments.? Let me know if I >>> should enter a new RFE to add the comments that you request below. >> >> No that's okay. But can you tell me why it needed to use jasm and >> reflection? :) >> >> Thanks, >> David >> >>> Thanks, Harold >>> >>> On 8/16/2020 10:06 PM, David Holmes wrote: >>>> Hi Harold, >>>> >>>> On 13/08/2020 4:14 am, Harold Seigel wrote: >>>>> Hi, >>>>> >>>>> Please review this change to add test cases to test >>>>> hotspot/jtreg/runtime/8026394/InterfaceObjectTest.java to check for >>>>> NoSuchMethodError exceptions when calling Object protected methods >>>>> clone() and finalize() using invokeinterface. >>>>> >>>>> Open Webrev: >>>>> http://cr.openjdk.java.net/~hseigel/bug_8251414/webrev/index.html >>>> >>>> Can you add a comment explaining why we need to use jasm and >>>> reflection in this new case. >>>> >>>> Thanks, >>>> David >>>> >>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8251414 >>>>> >>>>> The fix was tested by running the modified test on multiple >>>>> platforms, including Mac OS, Windows, and Linux. >>>>> >>>>> Thanks, Harold From robbin.ehn at oracle.com Thu Aug 27 07:21:19 2020 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 27 Aug 2020 09:21:19 +0200 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe Message-ID: Hi all, please review. In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a thread is always blocked when suspended. And added this nice assert. int JavaThread::java_suspend_self() { assert(thread_state() == _thread_blocked, "wrong state for java_suspend_self()"); When checking if a thread is safepoint/handshake safe there no need to look at ext suspend flag anymore, since the thread is blocked. Bug: https://bugs.openjdk.java.net/browse/JDK-8252414 Code: http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html Passes t1-5 Thanks, Robbin From david.holmes at oracle.com Thu Aug 27 08:07:40 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 27 Aug 2020 18:07:40 +1000 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: References: Message-ID: <0da72274-9673-a0ab-fa1e-88463039a856@oracle.com> Hi Robbin, On 27/08/2020 5:21 pm, Robbin Ehn wrote: > Hi all, please review. > > In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a > thread is always blocked when suspended. > > And added this nice assert. > int JavaThread::java_suspend_self() { > ? assert(thread_state() == _thread_blocked, "wrong state for > java_suspend_self()"); > > When checking if a thread is safepoint/handshake safe there no need to > look at ext suspend flag anymore, since the thread is blocked. That seems sound to me. :) Thanks, David ----- > Bug: > https://bugs.openjdk.java.net/browse/JDK-8252414 > Code: > http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html > > Passes t1-5 > > Thanks, Robbin From robbin.ehn at oracle.com Thu Aug 27 09:07:00 2020 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Thu, 27 Aug 2020 11:07:00 +0200 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: <0da72274-9673-a0ab-fa1e-88463039a856@oracle.com> References: <0da72274-9673-a0ab-fa1e-88463039a856@oracle.com> Message-ID: <6c026c74-45c6-5093-1644-358146663936@oracle.com> Thanks David! /Robbin On 2020-08-27 10:07, David Holmes wrote: > Hi Robbin, > > On 27/08/2020 5:21 pm, Robbin Ehn wrote: >> Hi all, please review. >> >> In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a >> thread is always blocked when suspended. >> >> And added this nice assert. >> int JavaThread::java_suspend_self() { >> ?? assert(thread_state() == _thread_blocked, "wrong state for >> java_suspend_self()"); >> >> When checking if a thread is safepoint/handshake safe there no need to >> look at ext suspend flag anymore, since the thread is blocked. > > That seems sound to me. :) > > Thanks, > David > ----- > > >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8252414 >> Code: >> http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html >> >> Passes t1-5 >> >> Thanks, Robbin From goetz.lindenmaier at sap.com Thu Aug 27 09:21:18 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 27 Aug 2020 09:21:18 +0000 Subject: [11u] RFR(S): 8241234: Unify monitor enter/exit runtime entries. In-Reply-To: References: Message-ID: Hi Martin, I had a look at your downport. I don't really understand which part you took from jdk15. I think you lost line - assert(thread->last_Java_sp(), "last_Java_sp must be set"); In jvmciRuntme.cpp, monitorexit(). Besides that it looks good. I'll also post this to jdk-updates-dev. Best regards, Goetz. -----Original Message----- From: hotspot-runtime-dev On Behalf Of Doerr, Martin Sent: Montag, 17. August 2020 23:57 To: hotspot-runtime-dev at openjdk.java.net Subject: [CAUTION] FW: [11u] RFR(S): 8241234: Unify monitor enter/exit runtime entries. Forwarding to hotspot-runtime. Hi, I'd like to backport https://bugs.openjdk.java.net/browse/JDK-8241234 to JDK11u. Original JDK15 patch (https://hg.openjdk.java.net/jdk/jdk/rev/87c506c8be63) doesn't fit to JDK11u because the locking code has been reworked by https://bugs.openjdk.java.net/browse/JDK-8229844 As mentioned by Vladimir, there's already a GraalVM version available which consists of 2 patches (original + addon) and which can be applied: https://github.com/graalvm/labs-openjdk-11/commit/6c162cb15262e6aa77e36eb3a268320ef0a206a4 https://github.com/graalvm/labs-openjdk-11/commit/6a28a618cdbe595f9a3993e0eb63c01ccae1a528 Only JVMCI part from GraalVM doesn't apply automatically. The version of this file from JDK15 is very simple and fits perfectly. Please review the JDK11u backport webrev: http://cr.openjdk.java.net/~mdoerr/8241234_monitorenterexit_11u/webrev.00/ Thanks and best regards, Martin From martin.doerr at sap.com Thu Aug 27 09:38:35 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 27 Aug 2020 09:38:35 +0000 Subject: [11u] RFR(S): 8241234: Unify monitor enter/exit runtime entries. In-Reply-To: References: Message-ID: Hi G?tz, thanks for looking at the backport. > I don't really understand which part you took from jdk15. I took the jvmciRuntime.cpp parts from the original change: https://hg.openjdk.java.net/jdk/jdk/rev/87c506c8be63 Right, the assertion "assert(thread->last_Java_sp(), "last_Java_sp must be set");" should not get touched by this change. I've added it back. Are you ok with it? Best regards, Martin > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Donnerstag, 27. August 2020 11:21 > To: Doerr, Martin ; hotspot-runtime- > dev at openjdk.java.net > Cc: jdk-updates-dev at openjdk.java.net > Subject: RE: [11u] RFR(S): 8241234: Unify monitor enter/exit runtime entries. > > Hi Martin, > > I had a look at your downport. > > I don't really understand which part you took from jdk15. > > I think you lost line > - assert(thread->last_Java_sp(), "last_Java_sp must be set"); > In jvmciRuntme.cpp, monitorexit(). > > Besides that it looks good. > > I'll also post this to jdk-updates-dev. > > Best regards, > Goetz. > > -----Original Message----- > From: hotspot-runtime-dev > On Behalf Of Doerr, Martin > Sent: Montag, 17. August 2020 23:57 > To: hotspot-runtime-dev at openjdk.java.net > Subject: [CAUTION] FW: [11u] RFR(S): 8241234: Unify monitor enter/exit > runtime entries. > > Forwarding to hotspot-runtime. > > > Hi, > > I'd like to backport https://bugs.openjdk.java.net/browse/JDK-8241234 to > JDK11u. > > Original JDK15 patch > (https://hg.openjdk.java.net/jdk/jdk/rev/87c506c8be63) doesn't fit to > JDK11u because the locking code has been reworked by > https://bugs.openjdk.java.net/browse/JDK-8229844 > As mentioned by Vladimir, there's already a GraalVM version available which > consists of 2 patches (original + addon) and which can be applied: > https://github.com/graalvm/labs-openjdk- > 11/commit/6c162cb15262e6aa77e36eb3a268320ef0a206a4 > https://github.com/graalvm/labs-openjdk- > 11/commit/6a28a618cdbe595f9a3993e0eb63c01ccae1a528 > Only JVMCI part from GraalVM doesn't apply automatically. The version of > this file from JDK15 is very simple and fits perfectly. > > Please review the JDK11u backport webrev: > http://cr.openjdk.java.net/~mdoerr/8241234_monitorenterexit_11u/webre > v.00/ > > Thanks and best regards, > Martin From goetz.lindenmaier at sap.com Thu Aug 27 10:45:24 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 27 Aug 2020 10:45:24 +0000 Subject: [11u] RFR(S): 8241234: Unify monitor enter/exit runtime entries. In-Reply-To: References: Message-ID: Hi Martin, Yes, it's good then. Ready to go. Best regards, Goetz. > -----Original Message----- > From: Doerr, Martin > Sent: Thursday, August 27, 2020 11:39 AM > To: Lindenmaier, Goetz ; hotspot-runtime- > dev at openjdk.java.net > Cc: jdk-updates-dev at openjdk.java.net > Subject: RE: [11u] RFR(S): 8241234: Unify monitor enter/exit runtime entries. > > Hi G?tz, > > thanks for looking at the backport. > > > I don't really understand which part you took from jdk15. > I took the jvmciRuntime.cpp parts from the original change: > https://hg.openjdk.java.net/jdk/jdk/rev/87c506c8be63 > > Right, the assertion "assert(thread->last_Java_sp(), "last_Java_sp must be > set");" should not get touched by this change. > I've added it back. > > Are you ok with it? > > Best regards, > Martin > > > > -----Original Message----- > > From: Lindenmaier, Goetz > > Sent: Donnerstag, 27. August 2020 11:21 > > To: Doerr, Martin ; hotspot-runtime- > > dev at openjdk.java.net > > Cc: jdk-updates-dev at openjdk.java.net > > Subject: RE: [11u] RFR(S): 8241234: Unify monitor enter/exit runtime > entries. > > > > Hi Martin, > > > > I had a look at your downport. > > > > I don't really understand which part you took from jdk15. > > > > I think you lost line > > - assert(thread->last_Java_sp(), "last_Java_sp must be set"); > > In jvmciRuntme.cpp, monitorexit(). > > > > Besides that it looks good. > > > > I'll also post this to jdk-updates-dev. > > > > Best regards, > > Goetz. > > > > -----Original Message----- > > From: hotspot-runtime-dev > > On Behalf Of Doerr, Martin > > Sent: Montag, 17. August 2020 23:57 > > To: hotspot-runtime-dev at openjdk.java.net > > Subject: [CAUTION] FW: [11u] RFR(S): 8241234: Unify monitor enter/exit > > runtime entries. > > > > Forwarding to hotspot-runtime. > > > > > > Hi, > > > > I'd like to backport https://bugs.openjdk.java.net/browse/JDK-8241234 to > > JDK11u. > > > > Original JDK15 patch > > (https://hg.openjdk.java.net/jdk/jdk/rev/87c506c8be63) doesn't fit to > > JDK11u because the locking code has been reworked by > > https://bugs.openjdk.java.net/browse/JDK-8229844 > > As mentioned by Vladimir, there's already a GraalVM version available > which > > consists of 2 patches (original + addon) and which can be applied: > > https://github.com/graalvm/labs-openjdk- > > 11/commit/6c162cb15262e6aa77e36eb3a268320ef0a206a4 > > https://github.com/graalvm/labs-openjdk- > > 11/commit/6a28a618cdbe595f9a3993e0eb63c01ccae1a528 > > Only JVMCI part from GraalVM doesn't apply automatically. The version of > > this file from JDK15 is very simple and fits perfectly. > > > > Please review the JDK11u backport webrev: > > > http://cr.openjdk.java.net/~mdoerr/8241234_monitorenterexit_11u/webre > > v.00/ > > > > Thanks and best regards, > > Martin From stefan.johansson at oracle.com Thu Aug 27 15:23:46 2020 From: stefan.johansson at oracle.com (stefan.johansson at oracle.com) Date: Thu, 27 Aug 2020 17:23:46 +0200 Subject: RFR: 8252093: formula used to calculate decaying variance in numberSeq In-Reply-To: <54b1cc85-9f2d-cdf8-c513-1cbee4fa7f3d@oracle.com> References: <54b1cc85-9f2d-cdf8-c513-1cbee4fa7f3d@oracle.com> Message-ID: <3577e2ab-f633-71ca-0b94-c7a39d6ca631@oracle.com> Hi Albert, On 2020-08-26 15:59, Albert Yang wrote: > Hi, > > I am sending this to both runtime-dev and gc-dev, since the changed code > lives in runtime, but it's used by GC. > > https://bugs.openjdk.java.net/browse/JDK-8252093 > http://cr.openjdk.java.net/~ayang/webrevs/8252093/webrev.1/ Looks good, nice to have the comment as reference. As Kim, I wonder if this will cause any visible change because of changed predictions. Have you run any benchmarks and compared scores as well as GC-behavior? > > Both the code and the literature use `alpha`, but its meaning is > actually inverted. I could include this refactoring in later revision if > people feel that it's desirable. Is there a good reason for keeping it inverted, I see that we have a few places where we specify an alpha when creating the seq, so I guess there are a few places to change. Otherwise maybe adding a comment to the defined default might be helpful. Thanks, Stefan > > Tested: tier1 (locally) and tier1-3 (mach5) > From gerard.ziemski at oracle.com Thu Aug 27 17:19:23 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Thu, 27 Aug 2020 12:19:23 -0500 Subject: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) In-Reply-To: References: Message-ID: <022e6096-a388-dc79-f959-00a2d933ae89@oracle.com> hi all, I?m hoping I have included the right people in this email, but everyone is welcome to provide feedback. I have a code for?JDK-8252324, which factors out the common POSIX signal related code out into common file, so that all platforms can benefit from fixes that apply to all POSIX platforms (like JDK-82514380) The preliminary webrev can be found here: http://cr.openjdk.java.net/~gziemski/8252324_pre1 I tried to port the changes to AIX myself, but got bogged down by more "#ifdef AIX? than I was hoping for (possibly too many differences to factor out the code for AIX, but frankly I don't think so), so I stopped (though I probably was on the right track) - to me it looks like AIX has some extra fixes and differences in the signal code than BSD/Linux. My question is: given my inability to test on AIX, what can I do to make it easier for AIX developers to adopt this fix? I will try porting my changes again, but in the meantime if anyone has any pointers or offer of other help, I?d appreciate it. I don?t want to straight break AIX port unexpectedly. Reference: https://bugs.openjdk.java.net/browse/JDK-8252324 Some notes about the affected APIs: -------------------------------------------------------------------------------------------------- The APIs which moved from os/bsd/os_bsd.cpp (similarly for linux) to to os/posix/PosixSignals.cpp: //////////////////////////////////////////////////////////////////////////////// // signal support void os::Bsd::signal_sets_init() sigset_t* os::Bsd::unblocked_signals() sigset_t* os::Bsd::vm_signals() void os::Bsd::hotspot_sigmask(Thread* thread) //////////////////////////////////////////////////////////////////////////////// // sun.misc.Signal support static void UserHandler(int sig, void *siginfo, void *context) void* os::user_handler() void* os::signal(int signal_number, void* handler) void os::signal_raise(int signal_number) int os::sigexitnum_pd() static void jdk_misc_signal_init() void os::signal_notify(int sig) static int check_pending_signals() int os::signal_wait() //////////////////////////////////////////////////////////////////////////////// // suspend/resume support static void resume_clear_context(OSThread *osthread) static void suspend_save_context(OSThread *osthread, siginfo_t* siginfo, ucontext_t* context) static void SR_handler(int sig, siginfo_t* siginfo, ucontext_t* context) static int SR_initialize() static int sr_notify(OSThread* osthread) static bool do_suspend(OSThread* osthread) static void do_resume(OSThread* osthread) /////////////////////////////////////////////////////////////////////////////////// // signal handling (except suspend/resume) static void signalHandler(int sig, siginfo_t* info, void* uc) struct sigaction* os::Bsd::get_chained_signal_action(int sig) static bool call_chained_handler(struct sigaction *actp, int sig, ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?siginfo_t *siginfo, void *context) bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* context) int os::Bsd::get_our_sigflags(int sig) void os::Bsd::set_our_sigflags(int sig, int flags) void os::Bsd::set_signal_handler(int sig, bool set_installed) void os::Bsd::install_signal_handlers() static const char* get_signal_handler_name(address handler, ?char* buf, int buflen) static void print_signal_handler(outputStream* st, int sig, ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?char* buf, size_t buflen) void os::run_periodic_checks() void os::Bsd::check_signal_handler(int sig) ----------------------------------------------------------------------------- The APIs which moved from os/posix/os_posix.cpp to os/posix/PosixSignals.cpp: const char* os::Posix::get_signal_name(int sig, char* out, size_t outlen) int os::Posix::get_signal_number(const char* signal_name) int os::get_signal_number(const char* signal_name) bool os::Posix::is_valid_signal(int sig) bool os::Posix::is_sig_ignored(int sig) const char* os::exception_name(int sig, char* buf, size_t size) const char* os::Posix::describe_signal_set_short(const sigset_t* set, char* buffer, size_t buf_size) void os::Posix::print_signal_set_short(outputStream* st, const sigset_t* set) const char* os::Posix::describe_sa_flags(int flags, char* buffer, size_t size) oid os::Posix::print_sa_flags(outputStream* st, int flags) static bool get_signal_code_description(const siginfo_t* si, enum_sigcode_desc_t* out) void os::print_siginfo(outputStream* os, const void* si0) bool os::signal_thread(Thread* thread, int sig, const char* reason) int os::Posix::unblock_thread_signal_mask(const sigset_t *set) address os::Posix::ucontext_get_pc(const ucontext_t* ctx) void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) struct sigaction* os::Posix::get_preinstalled_handler(int sig) void os::Posix::save_preinstalled_handler(int sig, struct sigaction& oldAct) DETAILS: -------------------------------------------------------- Public APIs which are now internal static PosixSignals:: sigset_t* os::Bsd::vm_signals() struct sigaction* os::Bsd::get_chained_signal_action(int sig) int os::Bsd::get_our_sigflags(int sig) void os::Bsd::set_our_sigflags(int sig, int flags) void os::Bsd::set_signal_handler(int sig, bool set_installed) void os::Bsd::check_signal_handler(int sig) const char* os::Posix::get_signal_name(int sig, char* out, size_t outlen) bool os::Posix::is_valid_signal(int sig) const char* os::Posix::describe_signal_set_short(const sigset_t* set, char* buffer, size_t buf_size) void os::Posix::print_signal_set_short(outputStream* st, const sigset_t* set) const char* os::Posix::describe_sa_flags(int flags, char* buffer, size_t size) oid os::Posix::print_sa_flags(outputStream* st, int flags) static bool get_signal_code_description(const siginfo_t* si, enum_sigcode_desc_t* out) void os::Posix::save_preinstalled_handler(int sig, struct sigaction& oldAct) ------------------------------------------------ Public APIs which moved to public PosixSignals:: void os::Bsd::signal_sets_init() void os::Bsd::hotspot_sigmask(Thread* thread) bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* context) void os::Bsd::install_signal_handlers() bool os::Posix::is_sig_ignored(int sig) int os::Posix::unblock_thread_signal_mask(const sigset_t *set) address os::Posix::ucontext_get_pc(const ucontext_t* ctx) void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) ---------------------------------------------------- Internal APIs which are now public in PosixSignals:: static void jdk_misc_signal_init() static int SR_initialize() static bool do_suspend(OSThread* osthread) static void do_resume(OSThread* osthread) static void print_signal_handler(outputStream* st, int sig, ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?char* buf, size_t buflen) -------------------------- New APIs in PosixSignals:: static bool are_signal_handlers_installed(); cheers From albert.m.yang at oracle.com Thu Aug 27 18:37:20 2020 From: albert.m.yang at oracle.com (Albert Yang) Date: Thu, 27 Aug 2020 20:37:20 +0200 Subject: RFR: 8252093: formula used to calculate decaying variance in numberSeq In-Reply-To: References: <54b1cc85-9f2d-cdf8-c513-1cbee4fa7f3d@oracle.com> Message-ID: Thank you for the review. Running SPECjbb2015 with G1, Shenandoah and ZGC now. Excluding ParallelGC, since it seems that it doesn't use numberSeq. -- /Albert From albert.m.yang at oracle.com Thu Aug 27 18:41:04 2020 From: albert.m.yang at oracle.com (Albert Yang) Date: Thu, 27 Aug 2020 20:41:04 +0200 Subject: RFR: 8252093: formula used to calculate decaying variance in numberSeq In-Reply-To: <3577e2ab-f633-71ca-0b94-c7a39d6ca631@oracle.com> References: <54b1cc85-9f2d-cdf8-c513-1cbee4fa7f3d@oracle.com> <3577e2ab-f633-71ca-0b94-c7a39d6ca631@oracle.com> Message-ID: <9d8182b5-94f7-d0ef-c3aa-6edb5b11a399@oracle.com> > Have you run any benchmarks and compared scores as well as GC-behavior? Checking SPECjbb2015 now. What kind of GC behavior do you have in mind? > Is there a good reason for keeping it inverted The only thing I can think of is to avoid invasive change. I am more leaned towards changing it to be consistent with the literature. -- /Albert From martin.doerr at sap.com Thu Aug 27 20:30:40 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 27 Aug 2020 20:30:40 +0000 Subject: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) In-Reply-To: <022e6096-a388-dc79-f959-00a2d933ae89@oracle.com> References: <022e6096-a388-dc79-f959-00a2d933ae89@oracle.com> Message-ID: Hi Gerard, > I don?t want to straight break AIX port unexpectedly. Thanks for that! We highly appreciate that you take care of it. It?s not surprising that adapting the AIX code is difficult. I?ve added Thomas to Cc. He?s surely the most experienced person for these AIX specific things. Unfortunately, he?s currently very busy because he?s contributing his new metaspace implementation. We have to discuss how we can help. Best regards, Martin From: gerard ziemski Sent: Donnerstag, 27. August 2020 19:19 To: hotspot-runtime-dev at openjdk.java.net Cc: Schmidt, Lutz ; Lindenmaier, Goetz ; Doerr, Martin Subject: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) hi all, I?m hoping I have included the right people in this email, but everyone is welcome to provide feedback. I have a code for JDK-8252324, which factors out the common POSIX signal related code out into common file, so that all platforms can benefit from fixes that apply to all POSIX platforms (like JDK-82514380) The preliminary webrev can be found here: http://cr.openjdk.java.net/~gziemski/8252324_pre1 I tried to port the changes to AIX myself, but got bogged down by more "#ifdef AIX? than I was hoping for (possibly too many differences to factor out the code for AIX, but frankly I don't think so), so I stopped (though I probably was on the right track) - to me it looks like AIX has some extra fixes and differences in the signal code than BSD/Linux. My question is: given my inability to test on AIX, what can I do to make it easier for AIX developers to adopt this fix? I will try porting my changes again, but in the meantime if anyone has any pointers or offer of other help, I?d appreciate it. I don?t want to straight break AIX port unexpectedly. Reference: https://bugs.openjdk.java.net/browse/JDK-8252324 Some notes about the affected APIs: -------------------------------------------------------------------------------------------------- The APIs which moved from os/bsd/os_bsd.cpp (similarly for linux) to to os/posix/PosixSignals.cpp: //////////////////////////////////////////////////////////////////////////////// // signal support void os::Bsd::signal_sets_init() sigset_t* os::Bsd::unblocked_signals() sigset_t* os::Bsd::vm_signals() void os::Bsd::hotspot_sigmask(Thread* thread) //////////////////////////////////////////////////////////////////////////////// // sun.misc.Signal support static void UserHandler(int sig, void *siginfo, void *context) void* os::user_handler() void* os::signal(int signal_number, void* handler) void os::signal_raise(int signal_number) int os::sigexitnum_pd() static void jdk_misc_signal_init() void os::signal_notify(int sig) static int check_pending_signals() int os::signal_wait() //////////////////////////////////////////////////////////////////////////////// // suspend/resume support static void resume_clear_context(OSThread *osthread) static void suspend_save_context(OSThread *osthread, siginfo_t* siginfo, ucontext_t* context) static void SR_handler(int sig, siginfo_t* siginfo, ucontext_t* context) static int SR_initialize() static int sr_notify(OSThread* osthread) static bool do_suspend(OSThread* osthread) static void do_resume(OSThread* osthread) /////////////////////////////////////////////////////////////////////////////////// // signal handling (except suspend/resume) static void signalHandler(int sig, siginfo_t* info, void* uc) struct sigaction* os::Bsd::get_chained_signal_action(int sig) static bool call_chained_handler(struct sigaction *actp, int sig, siginfo_t *siginfo, void *context) bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* context) int os::Bsd::get_our_sigflags(int sig) void os::Bsd::set_our_sigflags(int sig, int flags) void os::Bsd::set_signal_handler(int sig, bool set_installed) void os::Bsd::install_signal_handlers() static const char* get_signal_handler_name(address handler, char* buf, int buflen) static void print_signal_handler(outputStream* st, int sig, char* buf, size_t buflen) void os::run_periodic_checks() void os::Bsd::check_signal_handler(int sig) ----------------------------------------------------------------------------- The APIs which moved from os/posix/os_posix.cpp to os/posix/PosixSignals.cpp: const char* os::Posix::get_signal_name(int sig, char* out, size_t outlen) int os::Posix::get_signal_number(const char* signal_name) int os::get_signal_number(const char* signal_name) bool os::Posix::is_valid_signal(int sig) bool os::Posix::is_sig_ignored(int sig) const char* os::exception_name(int sig, char* buf, size_t size) const char* os::Posix::describe_signal_set_short(const sigset_t* set, char* buffer, size_t buf_size) void os::Posix::print_signal_set_short(outputStream* st, const sigset_t* set) const char* os::Posix::describe_sa_flags(int flags, char* buffer, size_t size) oid os::Posix::print_sa_flags(outputStream* st, int flags) static bool get_signal_code_description(const siginfo_t* si, enum_sigcode_desc_t* out) void os::print_siginfo(outputStream* os, const void* si0) bool os::signal_thread(Thread* thread, int sig, const char* reason) int os::Posix::unblock_thread_signal_mask(const sigset_t *set) address os::Posix::ucontext_get_pc(const ucontext_t* ctx) void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) struct sigaction* os::Posix::get_preinstalled_handler(int sig) void os::Posix::save_preinstalled_handler(int sig, struct sigaction& oldAct) DETAILS: -------------------------------------------------------- Public APIs which are now internal static PosixSignals:: sigset_t* os::Bsd::vm_signals() struct sigaction* os::Bsd::get_chained_signal_action(int sig) int os::Bsd::get_our_sigflags(int sig) void os::Bsd::set_our_sigflags(int sig, int flags) void os::Bsd::set_signal_handler(int sig, bool set_installed) void os::Bsd::check_signal_handler(int sig) const char* os::Posix::get_signal_name(int sig, char* out, size_t outlen) bool os::Posix::is_valid_signal(int sig) const char* os::Posix::describe_signal_set_short(const sigset_t* set, char* buffer, size_t buf_size) void os::Posix::print_signal_set_short(outputStream* st, const sigset_t* set) const char* os::Posix::describe_sa_flags(int flags, char* buffer, size_t size) oid os::Posix::print_sa_flags(outputStream* st, int flags) static bool get_signal_code_description(const siginfo_t* si, enum_sigcode_desc_t* out) void os::Posix::save_preinstalled_handler(int sig, struct sigaction& oldAct) ------------------------------------------------ Public APIs which moved to public PosixSignals:: void os::Bsd::signal_sets_init() void os::Bsd::hotspot_sigmask(Thread* thread) bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* context) void os::Bsd::install_signal_handlers() bool os::Posix::is_sig_ignored(int sig) int os::Posix::unblock_thread_signal_mask(const sigset_t *set) address os::Posix::ucontext_get_pc(const ucontext_t* ctx) void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) ---------------------------------------------------- Internal APIs which are now public in PosixSignals:: static void jdk_misc_signal_init() static int SR_initialize() static bool do_suspend(OSThread* osthread) static void do_resume(OSThread* osthread) static void print_signal_handler(outputStream* st, int sig, char* buf, size_t buflen) -------------------------- New APIs in PosixSignals:: static bool are_signal_handlers_installed(); cheers From richard.reingruber at sap.com Thu Aug 27 20:32:36 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Thu, 27 Aug 2020 20:32:36 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Goetz, > I read through your change again. It looks good to me now. > The new naming and additional comments make it > easier to read I think, thank you. Thanks for all your input! > One small thing: > deoptimization.cpp, l. 1503 > You don't really need the brackets. Two lines below you don't use them either. > (No webrev needed) Thanks for providing the correct line off list. Fixed! I prepared a new webrev, because I had to rebase after JDK-8249293 [1] and because I wanted to make use of JDK-8251384 [2] Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8227745/webrev.8/ Delta: http://cr.openjdk.java.net/~rrich/webrevs/8227745/webrev.8.inc/ The delta looks bigger than it is. Most of it is re-indentation of VM_GetOrSetLocal::deoptimize_objects(). You can see this if you look at http://cr.openjdk.java.net/~rrich/webrevs/8227745/webrev.8.inc/src/hotspot/share/prims/jvmtiImpl.cpp.udiff.html which does not include the whitespace change. Hope you are still ok with webrev.8. The changes are marginal. I've commented each below. Thanks, Richard. --- Details below --- src/hotspot/share/prims/jvmtiImpl.cpp @@ -425,11 +425,11 @@ , _depth(depth) , _index(index) , _type(type) , _jvf(NULL) , _set(false) - , _eb(NULL, NULL, false) // no references escape + , _eb(NULL, NULL, type == T_OBJECT) , _result(JVMTI_ERROR_NONE) Currently 'type' is never equal to T_OBJECT at this location, still I think it is better to check. The compiler will replace the compare with false. @@ -630,11 +630,11 @@ } // Revert optimizations based on escape analysis if this is an access to a local object bool VM_GetOrSetLocal::deoptimize_objects(javaVFrame* jvf) { #if COMPILER2_OR_JVMCI - if (NOT_JVMCI(DoEscapeAnalysis &&) _type == T_OBJECT) { + assert(_type == T_OBJECT, "EscapeBarrier should not be active if _type != T_OBJECT"); I removed the if from VM_GetOrSetLocal::deoptimize_objects(), because now it only gets called if the VM_GetOrSetLocal instance has an active EscapeBarrier which will be the case iff the local type is T_OBJECT and if either C2 escape analysis is enabled or Graal is used. src/hotspot/share/runtime/deoptimization.cpp You suggested to remove the braces. Done. src/hotspot/share/runtime/deoptimization.hpp Must provide definition of EscapeBarrier::barrier_active() for new call site in VM_GetOrSetLocal::doit_prologue() if building with COMPILER2_OR_JVMCI not defined. test/hotspot/jtreg/serviceability/jvmti/Heap/IterateHeapWithEscapeAnalysisEnabled.java Make use of [2] and pass test with minimal vm. [1] https://bugs.openjdk.java.net/browse/JDK-8249293 [2] https://bugs.openjdk.java.net/browse/JDK-8251384 -----Original Message----- From: Lindenmaier, Goetz Sent: Samstag, 22. August 2020 07:46 To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi Richard, I read through your change again. It looks good to me now. The new naming and additional comments make it easier to read I think, thank you. One small thing: deoptimization.cpp, l. 1503 You don't really need the brackets. Two lines below you don't use them either. (No webrev needed) Best regards, Goetz. From thomas.stuefe at gmail.com Thu Aug 27 21:48:59 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 27 Aug 2020 23:48:59 +0200 Subject: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) In-Reply-To: References: <022e6096-a388-dc79-f959-00a2d933ae89@oracle.com> Message-ID: Hi Gerard, Martin, I like the direction this patch is going, this is definitely a good cleanup! Also thanks from me for not wanting to break AIX :) Wrt to signals AIX should not be that different from other Posix platforms. Any differences probably stem from the fact that the AIX port had a long time brewing in our proprietary code base before we contributed it, with not much sync points with other ports after the initial fork. (Memory management in AIX is different for a number of reasons, but signal stuff should be pretty standard). There is this small issue that we use SIGTRAP for implicit null checks (@Martin: is this still a thing?) but strictly speaking this is a ppc issue, not an AIX issue, so it hits Linux too (@Martin: there seem to be small diffs between how sigtrap is handled in AIX and Linux, see e.g. the use of UseSigTrap, is that by design?). But all that should do is to add another signal to handle/unblock. I would say, give it your best shot and dry-code the changes. When in doubt, just ask. We will test your patch and correct if necessary. --- Some more general remarks to your preliminary changes: I would really like it if we could start documenting APIs. I am always astonished that this is so seldom done. Ironically, almost all existing API comments in os_posix.hpp were contributed by SAP :-) Since you now move all APIs to a central location, and it's all fresh in your head, maybe this would be a good point to add comments to them, describing behavior and input/output params. --- I quite liked the os::Posix namespace and think the signal functions are well placed there. But I see what you want, you want to separate the signal functions from the os namespace since that one requires including os.hpp? Unfortunately this also makes the diff more complicated since a lot of code just moved. Side note, I wish we would start using real C++ namespaces, we could make os a namespace and hence split it over as many header files as we want. Maybe something for future cleanups. --- ucontext_(s|g)et_pc() have not much to do with signals, even though they are only used for signal continuation currently. They seem misplaced in PosixSignals. --- Thanks, Thomas On Thu, Aug 27, 2020 at 10:33 PM Doerr, Martin wrote: > Hi Gerard, > > > I don?t want to straight break AIX port unexpectedly. > Thanks for that! We highly appreciate that you take care of it. > > It?s not surprising that adapting the AIX code is difficult. > > I?ve added Thomas to Cc. He?s surely the most experienced person for these > AIX specific things. > Unfortunately, he?s currently very busy because he?s contributing his new > metaspace implementation. > > We have to discuss how we can help. > > Best regards, > Martin > > > > From: gerard ziemski > Sent: Donnerstag, 27. August 2020 19:19 > To: hotspot-runtime-dev at openjdk.java.net > Cc: Schmidt, Lutz ; Lindenmaier, Goetz < > goetz.lindenmaier at sap.com>; Doerr, Martin > Subject: Attention AIX developers - factoring out POSIX signal code > (JDK-8252324) > > > hi all, > > I?m hoping I have included the right people in this email, but everyone is > welcome to provide feedback. > > I have a code for JDK-8252324, which factors out the common POSIX signal > related code out into common file, so that all platforms can benefit from > fixes that apply to all POSIX platforms (like JDK-82514380) > > The preliminary webrev can be found here: > http://cr.openjdk.java.net/~gziemski/8252324_pre1 > > > I tried to port the changes to AIX myself, but got bogged down by more > "#ifdef AIX? than I was hoping for (possibly too many differences to factor > out the code for AIX, but frankly I don't think so), so I stopped (though I > probably was on the right track) - to me it looks like AIX has some extra > fixes and differences in the signal code than BSD/Linux. > > My question is: given my inability to test on AIX, what can I do to make > it easier for AIX developers to adopt this fix? I will try porting my > changes again, but in the meantime if anyone has any pointers or offer of > other help, I?d appreciate it. I don?t want to straight break AIX port > unexpectedly. > > Reference: https://bugs.openjdk.java.net/browse/JDK-8252324 > > Some notes about the affected APIs: > > > -------------------------------------------------------------------------------------------------- > The APIs which moved from os/bsd/os_bsd.cpp (similarly for linux) to to > os/posix/PosixSignals.cpp: > > > //////////////////////////////////////////////////////////////////////////////// > // signal support > void os::Bsd::signal_sets_init() > sigset_t* os::Bsd::unblocked_signals() > sigset_t* os::Bsd::vm_signals() > void os::Bsd::hotspot_sigmask(Thread* thread) > > //////////////////////////////////////////////////////////////////////////////// > // sun.misc.Signal support > static void UserHandler(int sig, void *siginfo, void *context) > void* os::user_handler() > void* os::signal(int signal_number, void* handler) > void os::signal_raise(int signal_number) > int os::sigexitnum_pd() > static void jdk_misc_signal_init() > void os::signal_notify(int sig) > static int check_pending_signals() > int os::signal_wait() > > //////////////////////////////////////////////////////////////////////////////// > // suspend/resume support > static void resume_clear_context(OSThread *osthread) > static void suspend_save_context(OSThread *osthread, siginfo_t* siginfo, > ucontext_t* context) > static void SR_handler(int sig, siginfo_t* siginfo, ucontext_t* context) > static int SR_initialize() > static int sr_notify(OSThread* osthread) > static bool do_suspend(OSThread* osthread) > static void do_resume(OSThread* osthread) > > /////////////////////////////////////////////////////////////////////////////////// > // signal handling (except suspend/resume) > static void signalHandler(int sig, siginfo_t* info, void* uc) > struct sigaction* os::Bsd::get_chained_signal_action(int sig) > static bool call_chained_handler(struct sigaction *actp, int sig, > siginfo_t *siginfo, void *context) > bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* context) > int os::Bsd::get_our_sigflags(int sig) > void os::Bsd::set_our_sigflags(int sig, int flags) > void os::Bsd::set_signal_handler(int sig, bool set_installed) > void os::Bsd::install_signal_handlers() > static const char* get_signal_handler_name(address handler, > char* buf, int buflen) > static void print_signal_handler(outputStream* st, int sig, > char* buf, size_t buflen) > void os::run_periodic_checks() > void os::Bsd::check_signal_handler(int sig) > > > ----------------------------------------------------------------------------- > The APIs which moved from os/posix/os_posix.cpp to > os/posix/PosixSignals.cpp: > > const char* os::Posix::get_signal_name(int sig, char* out, size_t outlen) > int os::Posix::get_signal_number(const char* signal_name) > int os::get_signal_number(const char* signal_name) > bool os::Posix::is_valid_signal(int sig) > bool os::Posix::is_sig_ignored(int sig) > const char* os::exception_name(int sig, char* buf, size_t size) > const char* os::Posix::describe_signal_set_short(const sigset_t* set, > char* buffer, size_t buf_size) > void os::Posix::print_signal_set_short(outputStream* st, const sigset_t* > set) > const char* os::Posix::describe_sa_flags(int flags, char* buffer, size_t > size) > oid os::Posix::print_sa_flags(outputStream* st, int flags) > static bool get_signal_code_description(const siginfo_t* si, > enum_sigcode_desc_t* out) > void os::print_siginfo(outputStream* os, const void* si0) > bool os::signal_thread(Thread* thread, int sig, const char* reason) > int os::Posix::unblock_thread_signal_mask(const sigset_t *set) > address os::Posix::ucontext_get_pc(const ucontext_t* ctx) > void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) > struct sigaction* os::Posix::get_preinstalled_handler(int sig) > void os::Posix::save_preinstalled_handler(int sig, struct sigaction& > oldAct) > > > > DETAILS: > > -------------------------------------------------------- > Public APIs which are now internal static PosixSignals:: > > sigset_t* os::Bsd::vm_signals() > struct sigaction* os::Bsd::get_chained_signal_action(int sig) > int os::Bsd::get_our_sigflags(int sig) > void os::Bsd::set_our_sigflags(int sig, int flags) > void os::Bsd::set_signal_handler(int sig, bool set_installed) > void os::Bsd::check_signal_handler(int sig) > const char* os::Posix::get_signal_name(int sig, char* out, size_t outlen) > bool os::Posix::is_valid_signal(int sig) > const char* os::Posix::describe_signal_set_short(const sigset_t* set, > char* buffer, size_t buf_size) > void os::Posix::print_signal_set_short(outputStream* st, const sigset_t* > set) > const char* os::Posix::describe_sa_flags(int flags, char* buffer, size_t > size) > oid os::Posix::print_sa_flags(outputStream* st, int flags) > static bool get_signal_code_description(const siginfo_t* si, > enum_sigcode_desc_t* out) > void os::Posix::save_preinstalled_handler(int sig, struct sigaction& > oldAct) > > ------------------------------------------------ > Public APIs which moved to public PosixSignals:: > > void os::Bsd::signal_sets_init() > void os::Bsd::hotspot_sigmask(Thread* thread) > bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* context) > void os::Bsd::install_signal_handlers() > bool os::Posix::is_sig_ignored(int sig) > int os::Posix::unblock_thread_signal_mask(const sigset_t *set) > address os::Posix::ucontext_get_pc(const ucontext_t* ctx) > void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) > > ---------------------------------------------------- > Internal APIs which are now public in PosixSignals:: > > static void jdk_misc_signal_init() > static int SR_initialize() > static bool do_suspend(OSThread* osthread) > static void do_resume(OSThread* osthread) > static void print_signal_handler(outputStream* st, int sig, > char* buf, size_t buflen) > > -------------------------- > New APIs in PosixSignals:: > > static bool are_signal_handlers_installed(); > > > cheers > From coleen.phillimore at oracle.com Thu Aug 27 22:53:36 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 27 Aug 2020 18:53:36 -0400 Subject: RFR (S) 8230052: MLVM findDeadlock test timed out Message-ID: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> Summary: Add a System.gc() and reduce iteration count for more stability. open webrev at http://cr.openjdk.java.net/~coleenp/2020/8230052.01/webrev bug link https://bugs.openjdk.java.net/browse/JDK-8230052 Ran test 100x with < 1minute run time.? Should I add a timeout of 10 minutes also? Thanks, Coleen From daniel.daugherty at oracle.com Thu Aug 27 23:14:31 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 27 Aug 2020 19:14:31 -0400 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: References: Message-ID: <26c4f50c-13f2-3282-2007-35fa74231729@oracle.com> On 8/27/20 3:21 AM, Robbin Ehn wrote: > Hi all, please review. > > In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a > thread is always blocked when suspended. > > And added this nice assert. > int JavaThread::java_suspend_self() { > ? assert(thread_state() == _thread_blocked, "wrong state for > java_suspend_self()"); > > When checking if a thread is safepoint/handshake safe there no need to > look at ext suspend flag anymore, since the thread is blocked. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8252414 > Code: > http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html Sorry for the delay. I had to go back and reread the "8221207: Redo JDK-8218446 - SuspendAtExit hangs" review thread. src/hotspot/share/runtime/handshake.cpp ??? No comments. src/hotspot/share/runtime/safepoint.cpp ??? No comments. > When checking if a thread is safepoint/handshake safe there no need to > look at ext suspend flag anymore, since the thread is blocked. I wouldn't quite phrase it that way. We consider a thread to be is_ext_suspended() when it's running native code also. In that case, it is _thread_in_native, but if it should happen to return from native then we'll catch it in JavaThread::java_suspend_self_with_safepoint_check() so it will become properly blocked. Thumbs up! Dan > > Passes t1-5 > > Thanks, Robbin From coleen.phillimore at oracle.com Thu Aug 27 23:23:59 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Thu, 27 Aug 2020 19:23:59 -0400 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: References: Message-ID: In this change, the little function account_safe_thread() is only called in one place now.? Should it be inlined? The change looks good to me. Thanks, Coleen On 8/27/20 3:21 AM, Robbin Ehn wrote: > Hi all, please review. > > In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a > thread is always blocked when suspended. > > And added this nice assert. > int JavaThread::java_suspend_self() { > ? assert(thread_state() == _thread_blocked, "wrong state for > java_suspend_self()"); > > When checking if a thread is safepoint/handshake safe there no need to > look at ext suspend flag anymore, since the thread is blocked. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8252414 > Code: > http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html > > Passes t1-5 > > Thanks, Robbin From ioi.lam at oracle.com Thu Aug 27 23:59:09 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 27 Aug 2020 16:59:09 -0700 Subject: RFR (S) 8230052: MLVM findDeadlock test timed out In-Reply-To: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> References: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> Message-ID: Hi Coleen, From the comments of the bug, it looks like we are still not sure whether this happened because we're just running on a slow machine, or whether there's is an real underlying issue. Since the test fails very rarely (a few times a year??), if we are modifying it, maybe print out time stamps after every iteration? So if this fails again in six months we will have more info to look at. E.g., if the time stamps are well spaced out, we know we're running on a slow machine. If the time stamps get progressively longer, we probably have a bug in the test or in the VM. Thanks - Ioi On 8/27/20 3:53 PM, Coleen Phillimore wrote: > Summary: Add a System.gc() and reduce iteration count for more stability. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8230052.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8230052 > > Ran test 100x with < 1minute run time.? Should I add a timeout of 10 > minutes also? > > Thanks, > Coleen From david.holmes at oracle.com Fri Aug 28 01:14:44 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Aug 2020 11:14:44 +1000 Subject: RFR (S) 8230052: MLVM findDeadlock test timed out In-Reply-To: References: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> Message-ID: <8f14e525-5563-4224-f839-a300c9f48462@oracle.com> Hi Coleen, Fix looks good to me. I second what Ioi suggested - I think simply adding a timestamp here: Env.traceNormal("Iteration " + _iteration + " Starting test..."); would suffice. No need to see updated webrev if you choose that. Thanks, David On 28/08/2020 9:59 am, Ioi Lam wrote: > Hi Coleen, > > From the comments of the bug, it looks like we are still not sure > whether this happened because we're just running on a slow machine, or > whether there's is an real underlying issue. > > Since the test fails very rarely (a few times a year??), if we are > modifying it, maybe print out time stamps after every iteration? So if > this fails again in six months we will have more info to look at. E.g., > if the time stamps are well spaced out, we know we're running on a slow > machine. If the time stamps get progressively longer, we probably have a > bug in the test or in the VM. > > Thanks > - Ioi > > On 8/27/20 3:53 PM, Coleen Phillimore wrote: >> Summary: Add a System.gc() and reduce iteration count for more stability. >> >> open webrev at http://cr.openjdk.java.net/~coleenp/2020/8230052.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8230052 >> >> Ran test 100x with < 1minute run time.? Should I add a timeout of 10 >> minutes also? >> >> Thanks, >> Coleen > From david.holmes at oracle.com Fri Aug 28 01:38:47 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 28 Aug 2020 11:38:47 +1000 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: <26c4f50c-13f2-3282-2007-35fa74231729@oracle.com> References: <26c4f50c-13f2-3282-2007-35fa74231729@oracle.com> Message-ID: <71394efe-53a1-9599-ab80-65919f59af75@oracle.com> Hi Dan, Sorry need to be a bit pedantic here ... On 28/08/2020 9:14 am, Daniel D. Daugherty wrote: > On 8/27/20 3:21 AM, Robbin Ehn wrote: >> Hi all, please review. >> >> In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a >> thread is always blocked when suspended. >> >> And added this nice assert. >> int JavaThread::java_suspend_self() { >> ? assert(thread_state() == _thread_blocked, "wrong state for >> java_suspend_self()"); >> >> When checking if a thread is safepoint/handshake safe there no need to >> look at ext suspend flag anymore, since the thread is blocked. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8252414 >> Code: >> http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html > > Sorry for the delay. I had to go back and reread the > "8221207: Redo JDK-8218446 - SuspendAtExit hangs" review thread. > > > src/hotspot/share/runtime/handshake.cpp > ??? No comments. > > src/hotspot/share/runtime/safepoint.cpp > ??? No comments. > >> When checking if a thread is safepoint/handshake safe there no need to >> look at ext suspend flag anymore, since the thread is blocked. > > I wouldn't quite phrase it that way. We consider a thread to > be is_ext_suspended() when it's running native code also. In > that case, it is _thread_in_native, but if it should happen > to return from native then we'll catch it in > JavaThread::java_suspend_self_with_safepoint_check() so it > will become properly blocked. If a suspended thread is in native then we consider it is_external_suspend() but not is_ext_suspended(). The latter only occurs when the thread is back in the VM and self-suspends via JavaThread::java_suspend_self() (at which point it is already _thread_blocked). So Robbin's phrasing was quite accurate. Full disclosure: yes I had the same initial thought "or else it is in native", but then went and checked the details. :) Cheers, David > Thumbs up! > > Dan > > >> >> Passes t1-5 >> >> Thanks, Robbin > From ioi.lam at oracle.com Fri Aug 28 05:21:33 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Thu, 27 Aug 2020 22:21:33 -0700 Subject: RFR(S) 8252481 Remove excessive include of memTracker.hpp Message-ID: <1eda87bd-c398-ed0e-d6e4-63f2a1f88baf@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8252481 http://cr.openjdk.java.net/~iklam/jdk16/8252481-excessive-memTracker-hpp.v01/ memTracker.hpp is included by 847 out of 971 .o files in the HotSpot build. Most of these are caused by these header files: jfrAllocation.hpp -- completely unnecessary allocation.hpp, allocation.inline.hpp -- the use of CURRENT_PC can be avoid by calling this function instead: char* AllocateHeap(size_t size, ?????????????????? MEMFLAGS flags, ?????????????????? AllocFailType alloc_failmode /* = AllocFailStrategy::EXIT_OOM*/) { ? return AllocateHeap(size, flags, CALLER_PC); } ====================== Before: $ cd build/linux-x64 $ find hotspot -name \*.o | wc -l 971 $ find hotspot -name \*.d | xargs grep -l memTracker.hpp | wc -l 847 $ find hotspot -name \*.d | xargs cat | wc -l 528505 After: $ find hotspot -name \*.d | xargs grep -l memTracker.hpp | wc -l 95 $ find hotspot -name \*.d | xargs cat | wc -l 512731 The total number of included header files has reduced for about 3% 512731 / 528505 = 0.970 ====================== Testing -- I am running mach5 tiers 1-4. From goetz.lindenmaier at sap.com Fri Aug 28 06:37:39 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 28 Aug 2020 06:37:39 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Hi Richard, Thanks for the new webrev. The small improvements are fine, too. Reviewed from my side. Best regards, Goetz. > -----Original Message----- > From: Reingruber, Richard > Sent: Thursday, August 27, 2020 10:33 PM > To: Lindenmaier, Goetz ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > runtime-dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance > in the Presence of JVMTI Agents > > Hi Goetz, > > > I read through your change again. It looks good to me now. > > The new naming and additional comments make it > > easier to read I think, thank you. > > Thanks for all your input! > > > One small thing: > > deoptimization.cpp, l. 1503 > > You don't really need the brackets. Two lines below you don't use them > either. > > (No webrev needed) > > Thanks for providing the correct line off list. Fixed! > > I prepared a new webrev, because I had to rebase after JDK-8249293 [1] and > because I wanted to make use of JDK-8251384 [2] > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8227745/webrev.8/ > Delta: http://cr.openjdk.java.net/~rrich/webrevs/8227745/webrev.8.inc/ > > The delta looks bigger than it is. Most of it is re-indentation of > VM_GetOrSetLocal::deoptimize_objects(). You can see this if you look at > > http://cr.openjdk.java.net/~rrich/webrevs/8227745/webrev.8.inc/src/hotsp > ot/share/prims/jvmtiImpl.cpp.udiff.html > > which does not include the whitespace change. > > Hope you are still ok with webrev.8. The changes are marginal. I've > commented > each below. > > Thanks, Richard. > > --- Details below --- > > src/hotspot/share/prims/jvmtiImpl.cpp > > @@ -425,11 +425,11 @@ > , _depth(depth) > , _index(index) > , _type(type) > , _jvf(NULL) > , _set(false) > - , _eb(NULL, NULL, false) // no references escape > + , _eb(NULL, NULL, type == T_OBJECT) > , _result(JVMTI_ERROR_NONE) > > Currently 'type' is never equal to T_OBJECT at this location, still I think it > is better to check. The compiler will replace the compare with false. > > @@ -630,11 +630,11 @@ > } > > // Revert optimizations based on escape analysis if this is an access to a > local object > bool VM_GetOrSetLocal::deoptimize_objects(javaVFrame* jvf) { > #if COMPILER2_OR_JVMCI > - if (NOT_JVMCI(DoEscapeAnalysis &&) _type == T_OBJECT) { > + assert(_type == T_OBJECT, "EscapeBarrier should not be active if _type != > T_OBJECT"); > > I removed the if from VM_GetOrSetLocal::deoptimize_objects(), because > now it > only gets called if the VM_GetOrSetLocal instance has an active > EscapeBarrier > which will be the case iff the local type is T_OBJECT and if either C2 escape > analysis is enabled or Graal is used. > > src/hotspot/share/runtime/deoptimization.cpp > > You suggested to remove the braces. Done. > > src/hotspot/share/runtime/deoptimization.hpp > > Must provide definition of EscapeBarrier::barrier_active() for new call site in > VM_GetOrSetLocal::doit_prologue() if building with COMPILER2_OR_JVMCI > not > defined. > > test/hotspot/jtreg/serviceability/jvmti/Heap/IterateHeapWithEscapeAnalysis > Enabled.java > > Make use of [2] and pass test with minimal vm. > > [1] https://bugs.openjdk.java.net/browse/JDK-8249293 > [2] https://bugs.openjdk.java.net/browse/JDK-8251384 > > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Samstag, 22. August 2020 07:46 > To: Reingruber, Richard ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > runtime-dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance > in the Presence of JVMTI Agents > > Hi Richard, > > I read through your change again. It looks good to me now. > The new naming and additional comments make it > easier to read I think, thank you. > > One small thing: > deoptimization.cpp, l. 1503 > You don't really need the brackets. Two lines below you don't use them > either. > (No webrev needed) > > Best regards, > Goetz. From stefan.johansson at oracle.com Fri Aug 28 07:32:46 2020 From: stefan.johansson at oracle.com (stefan.johansson at oracle.com) Date: Fri, 28 Aug 2020 09:32:46 +0200 Subject: RFR: 8252093: formula used to calculate decaying variance in numberSeq In-Reply-To: <9d8182b5-94f7-d0ef-c3aa-6edb5b11a399@oracle.com> References: <54b1cc85-9f2d-cdf8-c513-1cbee4fa7f3d@oracle.com> <3577e2ab-f633-71ca-0b94-c7a39d6ca631@oracle.com> <9d8182b5-94f7-d0ef-c3aa-6edb5b11a399@oracle.com> Message-ID: <89a6cfee-093b-3381-730b-bab94a13a55b@oracle.com> On 2020-08-27 20:41, Albert Yang wrote: > > Have you run any benchmarks and compared scores as well as GC-behavior? > > Checking SPECjbb2015 now. What kind of GC behavior do you have in mind? > Something like running jbb2015 and other benchmarks without any options (maybe just max heap), and looking at if there is any change in for example heap/generation sizing or when we start concurrent cycles. It's probably also good to look at the logs for a tuned run and see that things looks the same there, but when tuning the heuristics don't get as much room. > > Is there a good reason for keeping it inverted > > The only thing I can think of is to avoid invasive change. I am more > leaned towards changing it to be consistent with the literature. > I agree with you, I guess it can be done as a follow up. Thanks, Stefan From robbin.ehn at oracle.com Fri Aug 28 07:38:23 2020 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 28 Aug 2020 09:38:23 +0200 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: <26c4f50c-13f2-3282-2007-35fa74231729@oracle.com> References: <26c4f50c-13f2-3282-2007-35fa74231729@oracle.com> Message-ID: Thanks Dan! /Robbin On 2020-08-28 01:14, Daniel D. Daugherty wrote: > On 8/27/20 3:21 AM, Robbin Ehn wrote: >> Hi all, please review. >> >> In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a >> thread is always blocked when suspended. >> >> And added this nice assert. >> int JavaThread::java_suspend_self() { >> ? assert(thread_state() == _thread_blocked, "wrong state for >> java_suspend_self()"); >> >> When checking if a thread is safepoint/handshake safe there no need to >> look at ext suspend flag anymore, since the thread is blocked. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8252414 >> Code: >> http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html > > Sorry for the delay. I had to go back and reread the > "8221207: Redo JDK-8218446 - SuspendAtExit hangs" review thread. > > > src/hotspot/share/runtime/handshake.cpp > ??? No comments. > > src/hotspot/share/runtime/safepoint.cpp > ??? No comments. > >> When checking if a thread is safepoint/handshake safe there no need to >> look at ext suspend flag anymore, since the thread is blocked. > > I wouldn't quite phrase it that way. We consider a thread to > be is_ext_suspended() when it's running native code also. In > that case, it is _thread_in_native, but if it should happen > to return from native then we'll catch it in > JavaThread::java_suspend_self_with_safepoint_check() so it > will become properly blocked. > > Thumbs up! > > Dan > > >> >> Passes t1-5 >> >> Thanks, Robbin > From richard.reingruber at sap.com Fri Aug 28 07:41:02 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 28 Aug 2020 07:41:02 +0000 Subject: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents In-Reply-To: References: <1f8a3c7a-fa0f-b5b2-4a8a-7d3d8dbbe1b5@oracle.com> <4b56a45c-a14c-6f74-2bfd-25deaabe8201@oracle.com> <5271429a-481d-ddb9-99dc-b3f6670fcc0b@oracle.com> Message-ID: Thanks a lot! Richard. -----Original Message----- From: Lindenmaier, Goetz Sent: Freitag, 28. August 2020 08:38 To: Reingruber, Richard ; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance in the Presence of JVMTI Agents Hi Richard, Thanks for the new webrev. The small improvements are fine, too. Reviewed from my side. Best regards, Goetz. > -----Original Message----- > From: Reingruber, Richard > Sent: Thursday, August 27, 2020 10:33 PM > To: Lindenmaier, Goetz ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > runtime-dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance > in the Presence of JVMTI Agents > > Hi Goetz, > > > I read through your change again. It looks good to me now. > > The new naming and additional comments make it > > easier to read I think, thank you. > > Thanks for all your input! > > > One small thing: > > deoptimization.cpp, l. 1503 > > You don't really need the brackets. Two lines below you don't use them > either. > > (No webrev needed) > > Thanks for providing the correct line off list. Fixed! > > I prepared a new webrev, because I had to rebase after JDK-8249293 [1] and > because I wanted to make use of JDK-8251384 [2] > > Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8227745/webrev.8/ > Delta: http://cr.openjdk.java.net/~rrich/webrevs/8227745/webrev.8.inc/ > > The delta looks bigger than it is. Most of it is re-indentation of > VM_GetOrSetLocal::deoptimize_objects(). You can see this if you look at > > http://cr.openjdk.java.net/~rrich/webrevs/8227745/webrev.8.inc/src/hotsp > ot/share/prims/jvmtiImpl.cpp.udiff.html > > which does not include the whitespace change. > > Hope you are still ok with webrev.8. The changes are marginal. I've > commented > each below. > > Thanks, Richard. > > --- Details below --- > > src/hotspot/share/prims/jvmtiImpl.cpp > > @@ -425,11 +425,11 @@ > , _depth(depth) > , _index(index) > , _type(type) > , _jvf(NULL) > , _set(false) > - , _eb(NULL, NULL, false) // no references escape > + , _eb(NULL, NULL, type == T_OBJECT) > , _result(JVMTI_ERROR_NONE) > > Currently 'type' is never equal to T_OBJECT at this location, still I think it > is better to check. The compiler will replace the compare with false. > > @@ -630,11 +630,11 @@ > } > > // Revert optimizations based on escape analysis if this is an access to a > local object > bool VM_GetOrSetLocal::deoptimize_objects(javaVFrame* jvf) { > #if COMPILER2_OR_JVMCI > - if (NOT_JVMCI(DoEscapeAnalysis &&) _type == T_OBJECT) { > + assert(_type == T_OBJECT, "EscapeBarrier should not be active if _type != > T_OBJECT"); > > I removed the if from VM_GetOrSetLocal::deoptimize_objects(), because > now it > only gets called if the VM_GetOrSetLocal instance has an active > EscapeBarrier > which will be the case iff the local type is T_OBJECT and if either C2 escape > analysis is enabled or Graal is used. > > src/hotspot/share/runtime/deoptimization.cpp > > You suggested to remove the braces. Done. > > src/hotspot/share/runtime/deoptimization.hpp > > Must provide definition of EscapeBarrier::barrier_active() for new call site in > VM_GetOrSetLocal::doit_prologue() if building with COMPILER2_OR_JVMCI > not > defined. > > test/hotspot/jtreg/serviceability/jvmti/Heap/IterateHeapWithEscapeAnalysis > Enabled.java > > Make use of [2] and pass test with minimal vm. > > [1] https://bugs.openjdk.java.net/browse/JDK-8249293 > [2] https://bugs.openjdk.java.net/browse/JDK-8251384 > > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Samstag, 22. August 2020 07:46 > To: Reingruber, Richard ; serviceability- > dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot- > runtime-dev at openjdk.java.net > Subject: RE: RFR(L) 8227745: Enable Escape Analysis for Better Performance > in the Presence of JVMTI Agents > > Hi Richard, > > I read through your change again. It looks good to me now. > The new naming and additional comments make it > easier to read I think, thank you. > > One small thing: > deoptimization.cpp, l. 1503 > You don't really need the brackets. Two lines below you don't use them > either. > (No webrev needed) > > Best regards, > Goetz. From robbin.ehn at oracle.com Fri Aug 28 07:50:30 2020 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 28 Aug 2020 09:50:30 +0200 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: References: Message-ID: <4fbceb15-36da-ca43-eff8-02bdb2790caf@oracle.com> Hi Coleen, On 2020-08-28 01:23, Coleen Phillimore wrote: > > In this change, the little function account_safe_thread() is only called > in one place now.? Should it be inlined? No, the method contains the actions needed to tick a thread off for safepoint, which is nice to have confined to a method. > > The change looks good to me. Thanks, Robbin > > Thanks, > Coleen > > On 8/27/20 3:21 AM, Robbin Ehn wrote: >> Hi all, please review. >> >> In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a >> thread is always blocked when suspended. >> >> And added this nice assert. >> int JavaThread::java_suspend_self() { >> ? assert(thread_state() == _thread_blocked, "wrong state for >> java_suspend_self()"); >> >> When checking if a thread is safepoint/handshake safe there no need to >> look at ext suspend flag anymore, since the thread is blocked. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8252414 >> Code: >> http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html >> >> Passes t1-5 >> >> Thanks, Robbin > From martin.doerr at sap.com Fri Aug 28 10:16:38 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 28 Aug 2020 10:16:38 +0000 Subject: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) In-Reply-To: References: <022e6096-a388-dc79-f959-00a2d933ae89@oracle.com> Message-ID: Hi Thomas and Gerard, > There is this small issue that we use SIGTRAP for implicit null checks (@Martin: is this still a thing?) Right, PPC64 is using SIGTRAP for implicit null checks and for quite a few other things. > but strictly speaking this is a ppc issue, not an AIX issue, so it hits Linux too Right. > (@Martin: there seem to be small diffs between how sigtrap is handled in AIX and Linux, see e.g. the use of UseSigTrap, is that by design?). The linux signal handler contains a workaround for an old linux on Power kernel bug. (I guess we don?t need it anymore.) Other than that, signal related code should work the same way on AIX as on linux AFAIK. (The logic which chooses which signal to use for what is different, but that shouldn?t be relevant here.) > I would say, give it your best shot and dry-code the changes. When in doubt, just ask. We will test your patch and correct if necessary. Yeah, we have machines to run build + tests. We can also put it into our internal nightly testing. Best regards, Martin From: Thomas St?fe Sent: Donnerstag, 27. August 2020 23:49 To: gerard ziemski ; Doerr, Martin Cc: hotspot-runtime-dev at openjdk.java.net; Schmidt, Lutz ; Lindenmaier, Goetz ; Stuefe, Thomas Subject: Re: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) Hi Gerard, Martin, I like the direction this patch is going, this is definitely a good cleanup! Also thanks from me for not wanting to break AIX :) Wrt to signals AIX should not be that different from other Posix platforms. Any differences probably stem from the fact that the AIX port had a long time brewing in our proprietary code base before we contributed it, with not much sync points with other ports after the initial fork. (Memory management in AIX is different for a number of reasons, but signal stuff should be pretty standard). There is this small issue that we use SIGTRAP for implicit null checks (@Martin: is this still a thing?) but strictly speaking this is a ppc issue, not an AIX issue, so it hits Linux too (@Martin: there seem to be small diffs between how sigtrap is handled in AIX and Linux, see e.g. the use of UseSigTrap, is that by design?). But all that should do is to add another signal to handle/unblock. I would say, give it your best shot and dry-code the changes. When in doubt, just ask. We will test your patch and correct if necessary. --- Some more general remarks to your preliminary changes: I would really like it if we could start documenting APIs. I am always astonished that this is so seldom done. Ironically, almost all existing API comments in os_posix.hpp were contributed by SAP :-) Since you now move all APIs to a central location, and it's all fresh in your head, maybe this would be a good point to add comments to them, describing behavior and input/output params. --- I quite liked the os::Posix namespace and think the signal functions are well placed there. But I see what you want, you want to separate the signal functions from the os namespace since that one requires including os.hpp? Unfortunately this also makes the diff more complicated since a lot of code just moved. Side note, I wish we would start using real C++ namespaces, we could make os a namespace and hence split it over as many header files as we want. Maybe something for future cleanups. --- ucontext_(s|g)et_pc() have not much to do with signals, even though they are only used for signal continuation currently. They seem misplaced in PosixSignals. --- Thanks, Thomas On Thu, Aug 27, 2020 at 10:33 PM Doerr, Martin > wrote: Hi Gerard, > I don?t want to straight break AIX port unexpectedly. Thanks for that! We highly appreciate that you take care of it. It?s not surprising that adapting the AIX code is difficult. I?ve added Thomas to Cc. He?s surely the most experienced person for these AIX specific things. Unfortunately, he?s currently very busy because he?s contributing his new metaspace implementation. We have to discuss how we can help. Best regards, Martin From: gerard ziemski > Sent: Donnerstag, 27. August 2020 19:19 To: hotspot-runtime-dev at openjdk.java.net Cc: Schmidt, Lutz >; Lindenmaier, Goetz >; Doerr, Martin > Subject: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) hi all, I?m hoping I have included the right people in this email, but everyone is welcome to provide feedback. I have a code for JDK-8252324, which factors out the common POSIX signal related code out into common file, so that all platforms can benefit from fixes that apply to all POSIX platforms (like JDK-82514380) The preliminary webrev can be found here: http://cr.openjdk.java.net/~gziemski/8252324_pre1 I tried to port the changes to AIX myself, but got bogged down by more "#ifdef AIX? than I was hoping for (possibly too many differences to factor out the code for AIX, but frankly I don't think so), so I stopped (though I probably was on the right track) - to me it looks like AIX has some extra fixes and differences in the signal code than BSD/Linux. My question is: given my inability to test on AIX, what can I do to make it easier for AIX developers to adopt this fix? I will try porting my changes again, but in the meantime if anyone has any pointers or offer of other help, I?d appreciate it. I don?t want to straight break AIX port unexpectedly. Reference: https://bugs.openjdk.java.net/browse/JDK-8252324 Some notes about the affected APIs: -------------------------------------------------------------------------------------------------- The APIs which moved from os/bsd/os_bsd.cpp (similarly for linux) to to os/posix/PosixSignals.cpp: //////////////////////////////////////////////////////////////////////////////// // signal support void os::Bsd::signal_sets_init() sigset_t* os::Bsd::unblocked_signals() sigset_t* os::Bsd::vm_signals() void os::Bsd::hotspot_sigmask(Thread* thread) //////////////////////////////////////////////////////////////////////////////// // sun.misc.Signal support static void UserHandler(int sig, void *siginfo, void *context) void* os::user_handler() void* os::signal(int signal_number, void* handler) void os::signal_raise(int signal_number) int os::sigexitnum_pd() static void jdk_misc_signal_init() void os::signal_notify(int sig) static int check_pending_signals() int os::signal_wait() //////////////////////////////////////////////////////////////////////////////// // suspend/resume support static void resume_clear_context(OSThread *osthread) static void suspend_save_context(OSThread *osthread, siginfo_t* siginfo, ucontext_t* context) static void SR_handler(int sig, siginfo_t* siginfo, ucontext_t* context) static int SR_initialize() static int sr_notify(OSThread* osthread) static bool do_suspend(OSThread* osthread) static void do_resume(OSThread* osthread) /////////////////////////////////////////////////////////////////////////////////// // signal handling (except suspend/resume) static void signalHandler(int sig, siginfo_t* info, void* uc) struct sigaction* os::Bsd::get_chained_signal_action(int sig) static bool call_chained_handler(struct sigaction *actp, int sig, siginfo_t *siginfo, void *context) bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* context) int os::Bsd::get_our_sigflags(int sig) void os::Bsd::set_our_sigflags(int sig, int flags) void os::Bsd::set_signal_handler(int sig, bool set_installed) void os::Bsd::install_signal_handlers() static const char* get_signal_handler_name(address handler, char* buf, int buflen) static void print_signal_handler(outputStream* st, int sig, char* buf, size_t buflen) void os::run_periodic_checks() void os::Bsd::check_signal_handler(int sig) ----------------------------------------------------------------------------- The APIs which moved from os/posix/os_posix.cpp to os/posix/PosixSignals.cpp: const char* os::Posix::get_signal_name(int sig, char* out, size_t outlen) int os::Posix::get_signal_number(const char* signal_name) int os::get_signal_number(const char* signal_name) bool os::Posix::is_valid_signal(int sig) bool os::Posix::is_sig_ignored(int sig) const char* os::exception_name(int sig, char* buf, size_t size) const char* os::Posix::describe_signal_set_short(const sigset_t* set, char* buffer, size_t buf_size) void os::Posix::print_signal_set_short(outputStream* st, const sigset_t* set) const char* os::Posix::describe_sa_flags(int flags, char* buffer, size_t size) oid os::Posix::print_sa_flags(outputStream* st, int flags) static bool get_signal_code_description(const siginfo_t* si, enum_sigcode_desc_t* out) void os::print_siginfo(outputStream* os, const void* si0) bool os::signal_thread(Thread* thread, int sig, const char* reason) int os::Posix::unblock_thread_signal_mask(const sigset_t *set) address os::Posix::ucontext_get_pc(const ucontext_t* ctx) void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) struct sigaction* os::Posix::get_preinstalled_handler(int sig) void os::Posix::save_preinstalled_handler(int sig, struct sigaction& oldAct) DETAILS: -------------------------------------------------------- Public APIs which are now internal static PosixSignals:: sigset_t* os::Bsd::vm_signals() struct sigaction* os::Bsd::get_chained_signal_action(int sig) int os::Bsd::get_our_sigflags(int sig) void os::Bsd::set_our_sigflags(int sig, int flags) void os::Bsd::set_signal_handler(int sig, bool set_installed) void os::Bsd::check_signal_handler(int sig) const char* os::Posix::get_signal_name(int sig, char* out, size_t outlen) bool os::Posix::is_valid_signal(int sig) const char* os::Posix::describe_signal_set_short(const sigset_t* set, char* buffer, size_t buf_size) void os::Posix::print_signal_set_short(outputStream* st, const sigset_t* set) const char* os::Posix::describe_sa_flags(int flags, char* buffer, size_t size) oid os::Posix::print_sa_flags(outputStream* st, int flags) static bool get_signal_code_description(const siginfo_t* si, enum_sigcode_desc_t* out) void os::Posix::save_preinstalled_handler(int sig, struct sigaction& oldAct) ------------------------------------------------ Public APIs which moved to public PosixSignals:: void os::Bsd::signal_sets_init() void os::Bsd::hotspot_sigmask(Thread* thread) bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* context) void os::Bsd::install_signal_handlers() bool os::Posix::is_sig_ignored(int sig) int os::Posix::unblock_thread_signal_mask(const sigset_t *set) address os::Posix::ucontext_get_pc(const ucontext_t* ctx) void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) ---------------------------------------------------- Internal APIs which are now public in PosixSignals:: static void jdk_misc_signal_init() static int SR_initialize() static bool do_suspend(OSThread* osthread) static void do_resume(OSThread* osthread) static void print_signal_handler(outputStream* st, int sig, char* buf, size_t buflen) -------------------------- New APIs in PosixSignals:: static bool are_signal_handlers_installed(); cheers From coleen.phillimore at oracle.com Fri Aug 28 11:59:03 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 28 Aug 2020 07:59:03 -0400 Subject: RFR (S) 8230052: MLVM findDeadlock test timed out In-Reply-To: References: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> Message-ID: <0324bc05-6082-fac0-d1ca-a22dc0bd0ce3@oracle.com> On 8/27/20 7:59 PM, Ioi Lam wrote: > Hi Coleen, > > From the comments of the bug, it looks like we are still not sure > whether this happened because we're just running on a slow machine, or > whether there's is an real underlying issue. Thanks for looking at this.? We don't know whether there's an issue unless we download all the artifacts and debug the core file.? In the one that failed, we did that and there was nothing wrong.? It was busy doing GC.? Honestly don't know what we're looking for if there's an underlying issue. > > Since the test fails very rarely (a few times a year??), if we are > modifying it, maybe print out time stamps after every iteration? So if > this fails again in six months we will have more info to look at. > E.g., if the time stamps are well spaced out, we know we're running on > a slow machine. If the time stamps get progressively longer, we > probably have a bug in the test or in the VM. That's a good idea.? I see David below has a suggestion, which is good! Coleen > > Thanks > - Ioi > > On 8/27/20 3:53 PM, Coleen Phillimore wrote: >> Summary: Add a System.gc() and reduce iteration count for more >> stability. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8230052.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8230052 >> >> Ran test 100x with < 1minute run time.? Should I add a timeout of 10 >> minutes also? >> >> Thanks, >> Coleen > From patricio.chilano.mateo at oracle.com Fri Aug 28 12:22:09 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Fri, 28 Aug 2020 09:22:09 -0300 Subject: RFR (S) 8230052: MLVM findDeadlock test timed out In-Reply-To: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> References: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> Message-ID: <13b41959-00fb-a647-dc15-d424246e13f4@oracle.com> Hi Coleen, On 8/27/20 7:53 PM, Coleen Phillimore wrote: > Summary: Add a System.gc() and reduce iteration count for more stability. > > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8230052.01/webrev > bug link https://bugs.openjdk.java.net/browse/JDK-8230052 Looks good to me. If we see any new failure I think we can try to run the test with extra VM logging enabled. Thanks, Patricio > Ran test 100x with < 1minute run time.? Should I add a timeout of 10 > minutes also? > > Thanks, > Coleen From albert.m.yang at oracle.com Fri Aug 28 12:25:41 2020 From: albert.m.yang at oracle.com (Albert Yang) Date: Fri, 28 Aug 2020 14:25:41 +0200 Subject: RFR: 8252093: formula used to calculate decaying variance in numberSeq In-Reply-To: <89a6cfee-093b-3381-730b-bab94a13a55b@oracle.com> References: <54b1cc85-9f2d-cdf8-c513-1cbee4fa7f3d@oracle.com> <3577e2ab-f633-71ca-0b94-c7a39d6ca631@oracle.com> <9d8182b5-94f7-d0ef-c3aa-6edb5b11a399@oracle.com> <89a6cfee-093b-3381-730b-bab94a13a55b@oracle.com> Message-ID: <2863d0f6-5cd1-f5c9-1fd2-5f4cf5f1ae23@oracle.com> > Something like running jbb2015 and other benchmarks without any options... I see; thank you for the elaboration. > I agree with you, I guess it can be done as a follow up. OK, will do so after the performance check. -- /Albert From richard.reingruber at sap.com Fri Aug 28 13:33:35 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 28 Aug 2020 13:33:35 +0000 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: References: Message-ID: Hi Robbin, your change looks correct to me. Thanks! I see a related issue that allows a suspended thread to continue executing java. Hope it's ok to ask about it here. Assume a thread T is resumed and then taken off CPU in JavaThread::java_suspend_self_with_safepoint_check() just before restoring its prior state [1]. In this situation T gets suspended again (by executing VM_ThreadSuspend) Now T is put back on the CPU. There are code paths where T won't check its suspend flags and return to executing java even though it is suspended. Example: JavaCallWrapper Stack: JavaThread::java_suspend_self_with_safepoint_check() : void JavaThread::handle_special_runtime_exit_condition(bool) : void JavaCallWrapper::JavaCallWrapper(const methodHandle &, Handle, JavaValue *, Thread *) Example: Polling page exception Stack: JavaThread::handle_special_runtime_exit_condition(bool) : void SafepointSynchronize::block(JavaThread *) : void SafepointMechanism::block_or_handshake(JavaThread *) : void SafepointMechanism::block_if_requested_slow(JavaThread *) : void SafepointMechanism::block_if_requested(JavaThread *) : void ThreadSafepointState::handle_polling_page_exception() : void Would you agree or have I missed something? Note that there is no problem when T is put back on CPU while a handshake is executed on its behalf because after restoring its prior state T calls SafepointMechanism::block_if_requested() in JavaThread::java_suspend_self_with_safepoint_check(). Thanks, Richard. [1] T is taken off CPU before restoring its prior state. https://github.com/openjdk/jdk/blob/30c2dbea95f817c93f84f0d8f7dc3b5c159b5e25/src/hotspot/share/runtime/thread.cpp#L2619 -----Original Message----- From: hotspot-runtime-dev On Behalf Of Robbin Ehn Sent: Donnerstag, 27. August 2020 09:21 To: hotspot-runtime-dev at openjdk.java.net Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe Hi all, please review. In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a thread is always blocked when suspended. And added this nice assert. int JavaThread::java_suspend_self() { assert(thread_state() == _thread_blocked, "wrong state for java_suspend_self()"); When checking if a thread is safepoint/handshake safe there no need to look at ext suspend flag anymore, since the thread is blocked. Bug: https://bugs.openjdk.java.net/browse/JDK-8252414 Code: http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html Passes t1-5 Thanks, Robbin From akozlov at azul.com Fri Aug 28 13:57:52 2020 From: akozlov at azul.com (Anton Kozlov) Date: Fri, 28 Aug 2020 16:57:52 +0300 Subject: RFR: 8234930: Use MAP_JIT when allocating pages for code cache on macOS Message-ID: <13480ffc-6afe-c59d-e671-88abf54d8179@azul.com> Hi, Could you please review a change for JDK-8234930: Use MAP_JIT when allocating pages for code cache on macOS Bug: https://bugs.openjdk.java.net/browse/JDK-8234930 Webrev: http://cr.openjdk.java.net/~akozlov/8234930/webrev.00/ On macOS, MAP_JIT cannot be used with MAP_FIXED[1]. So pd_reserve_memory have to provide MAP_JIT for mmap(NULL, PROT_NONE), the function was made aware of exec permissions. For executable and data regions, pd_commit_memory only unlocks the memory with mprotect, this should make no difference compared with old code. For data regions, pd_uncommit_memory still uses a new overlapping anonymous mmap which returns pages to the OS and immediately reflects this in diagnostic tools like ps. For executable regions it would require MAP_FIXED|MAP_JIT, so instead madvise(MADV_FREE)+mprotect(PROT_NONE) are used. They should also allow OS to reclaim pages, but apparently this does not happen immediately. In practice, it should not be a problem for executable regions, as codecache does not shrink (if I haven't missed anything, by the implementation and in principle). Tested: * local tier1 * jdk-submit * codesign[2] with hardened runtime and allow-jit but without allow-unsigned-executable-memory entitlements[3] produce a working bundle. Thanks, Anton [1] https://github.com/apple/darwin-xnu/blob/master/bsd/kern/kern_mman.c#L227 [2] codesign \ --sign - \ --options runtime \ --entitlements ents.plist \ --timestamp \ $J/bin/* $J/lib/server/*.dylib $J/lib/*.dylib [3] com.apple.security.cs.allow-jit com.apple.security.cs.disable-library-validation com.apple.security.cs.allow-dyld-environment-variables From daniel.daugherty at oracle.com Fri Aug 28 15:34:25 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 28 Aug 2020 11:34:25 -0400 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: <71394efe-53a1-9599-ab80-65919f59af75@oracle.com> References: <26c4f50c-13f2-3282-2007-35fa74231729@oracle.com> <71394efe-53a1-9599-ab80-65919f59af75@oracle.com> Message-ID: <59ee914d-47ba-0103-da7a-3d5e0d213e19@oracle.com> On 8/27/20 9:38 PM, David Holmes wrote: > Hi Dan, > > Sorry need to be a bit pedantic here ... With suspend/resume you must take things down to the pedantic level so no apology needed. > > On 28/08/2020 9:14 am, Daniel D. Daugherty wrote: >> On 8/27/20 3:21 AM, Robbin Ehn wrote: >>> Hi all, please review. >>> >>> In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a >>> thread is always blocked when suspended. >>> >>> And added this nice assert. >>> int JavaThread::java_suspend_self() { >>> ? assert(thread_state() == _thread_blocked, "wrong state for >>> java_suspend_self()"); >>> >>> When checking if a thread is safepoint/handshake safe there no need >>> to look at ext suspend flag anymore, since the thread is blocked. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8252414 >>> Code: >>> http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html >> >> Sorry for the delay. I had to go back and reread the >> "8221207: Redo JDK-8218446 - SuspendAtExit hangs" review thread. >> >> >> src/hotspot/share/runtime/handshake.cpp >> ???? No comments. >> >> src/hotspot/share/runtime/safepoint.cpp >> ???? No comments. >> >>> When checking if a thread is safepoint/handshake safe there no need to >>> look at ext suspend flag anymore, since the thread is blocked. >> >> I wouldn't quite phrase it that way. We consider a thread to >> be is_ext_suspended() when it's running native code also. In >> that case, it is _thread_in_native, but if it should happen >> to return from native then we'll catch it in >> JavaThread::java_suspend_self_with_safepoint_check() so it >> will become properly blocked. > > If a suspended thread is in native then we consider it > is_external_suspend() but not is_ext_suspended(). The latter only > occurs when the thread is back in the VM and self-suspends via > JavaThread::java_suspend_self() (at which point it is already > _thread_blocked). So Robbin's phrasing was quite accurate. You'd think I would remember that especially since I'm pretty sure that is_ext_suspended() is something that I wrote many, many years ago... sigh... Yup... found the delta from Teamware... $ sp -r1.321 src/share/vm/runtime/thread.hpp src/share/vm/runtime/SCCS/s.thread.hpp: D 1.321 01/10/17 20:38:49 dcubed 732 731??????? 00092/00022/01306 MRs: COMMENTS: 4510838 - Add _suspend_equivalent support for _thread_blocked state. Change _is_suspended to _suspend_done_type; this allows us to know which type of suspend is done first. Add SR_RequestTypes to abstract suspend and resume requests. Change various suspend() and resume() parameters to SR_RequestTypes. Add new query functions for suspend status. Update a lot of comments They say memory is the second thing to go... what's the first? :-) > Full disclosure: yes I had the same initial thought "or else it is in > native", but then went and checked the details. :) As I should have. Thanks for being pedantic! Dan > > Cheers, > David > >> Thumbs up! >> >> Dan >> >> >>> >>> Passes t1-5 >>> >>> Thanks, Robbin >> From daniel.daugherty at oracle.com Fri Aug 28 15:38:32 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 28 Aug 2020 11:38:32 -0400 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: References: Message-ID: <0409009e-d815-2ce3-618f-39ab83250a11@oracle.com> Hi Richard, This is the right place and the right people have been watching this thread, but we should take this up in a new thread and likely with a new bug ID... Dan On 8/28/20 9:33 AM, Reingruber, Richard wrote: > Hi Robbin, > > your change looks correct to me. Thanks! > > I see a related issue that allows a suspended thread to continue executing > java. Hope it's ok to ask about it here. > > Assume a thread T is resumed and then taken off CPU in JavaThread::java_suspend_self_with_safepoint_check() > just before restoring its prior state [1]. > > In this situation T gets suspended again (by executing VM_ThreadSuspend) > > Now T is put back on the CPU. There are code paths where T won't check its > suspend flags and return to executing java even though it is suspended. > > > Example: JavaCallWrapper > > Stack: > > JavaThread::java_suspend_self_with_safepoint_check() : void > JavaThread::handle_special_runtime_exit_condition(bool) : void > JavaCallWrapper::JavaCallWrapper(const methodHandle &, Handle, JavaValue *, Thread *) > > Example: Polling page exception > > Stack: > > JavaThread::handle_special_runtime_exit_condition(bool) : void > SafepointSynchronize::block(JavaThread *) : void > SafepointMechanism::block_or_handshake(JavaThread *) : void > SafepointMechanism::block_if_requested_slow(JavaThread *) : void > SafepointMechanism::block_if_requested(JavaThread *) : void > ThreadSafepointState::handle_polling_page_exception() : void > > Would you agree or have I missed something? > > Note that there is no problem when T is put back on CPU while a handshake is > executed on its behalf because after restoring its prior state T calls > SafepointMechanism::block_if_requested() in > JavaThread::java_suspend_self_with_safepoint_check(). > > Thanks, Richard. > > [1] T is taken off CPU before restoring its prior state. > https://github.com/openjdk/jdk/blob/30c2dbea95f817c93f84f0d8f7dc3b5c159b5e25/src/hotspot/share/runtime/thread.cpp#L2619 > > -----Original Message----- > From: hotspot-runtime-dev On Behalf Of Robbin Ehn > Sent: Donnerstag, 27. August 2020 09:21 > To: hotspot-runtime-dev at openjdk.java.net > Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe > > Hi all, please review. > > In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a > thread is always blocked when suspended. > > And added this nice assert. > int JavaThread::java_suspend_self() { > assert(thread_state() == _thread_blocked, "wrong state for > java_suspend_self()"); > > When checking if a thread is safepoint/handshake safe there no need to > look at ext suspend flag anymore, since the thread is blocked. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8252414 > Code: > http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html > > Passes t1-5 > > Thanks, Robbin From richard.reingruber at sap.com Fri Aug 28 15:53:50 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 28 Aug 2020 15:53:50 +0000 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: <0409009e-d815-2ce3-618f-39ab83250a11@oracle.com> References: <0409009e-d815-2ce3-618f-39ab83250a11@oracle.com> Message-ID: Hi Dan, thanks for commenting on this. I'll start a new thread (probably after the weekend). Thanks, Richard. -----Original Message----- From: Daniel D. Daugherty Sent: Freitag, 28. August 2020 17:39 To: Reingruber, Richard ; Robbin Ehn ; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe Hi Richard, This is the right place and the right people have been watching this thread, but we should take this up in a new thread and likely with a new bug ID... Dan On 8/28/20 9:33 AM, Reingruber, Richard wrote: > Hi Robbin, > > your change looks correct to me. Thanks! > > I see a related issue that allows a suspended thread to continue executing > java. Hope it's ok to ask about it here. > > Assume a thread T is resumed and then taken off CPU in JavaThread::java_suspend_self_with_safepoint_check() > just before restoring its prior state [1]. > > In this situation T gets suspended again (by executing VM_ThreadSuspend) > > Now T is put back on the CPU. There are code paths where T won't check its > suspend flags and return to executing java even though it is suspended. > > > Example: JavaCallWrapper > > Stack: > > JavaThread::java_suspend_self_with_safepoint_check() : void > JavaThread::handle_special_runtime_exit_condition(bool) : void > JavaCallWrapper::JavaCallWrapper(const methodHandle &, Handle, JavaValue *, Thread *) > > Example: Polling page exception > > Stack: > > JavaThread::handle_special_runtime_exit_condition(bool) : void > SafepointSynchronize::block(JavaThread *) : void > SafepointMechanism::block_or_handshake(JavaThread *) : void > SafepointMechanism::block_if_requested_slow(JavaThread *) : void > SafepointMechanism::block_if_requested(JavaThread *) : void > ThreadSafepointState::handle_polling_page_exception() : void > > Would you agree or have I missed something? > > Note that there is no problem when T is put back on CPU while a handshake is > executed on its behalf because after restoring its prior state T calls > SafepointMechanism::block_if_requested() in > JavaThread::java_suspend_self_with_safepoint_check(). > > Thanks, Richard. > > [1] T is taken off CPU before restoring its prior state. > https://github.com/openjdk/jdk/blob/30c2dbea95f817c93f84f0d8f7dc3b5c159b5e25/src/hotspot/share/runtime/thread.cpp#L2619 > > -----Original Message----- > From: hotspot-runtime-dev On Behalf Of Robbin Ehn > Sent: Donnerstag, 27. August 2020 09:21 > To: hotspot-runtime-dev at openjdk.java.net > Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe > > Hi all, please review. > > In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a > thread is always blocked when suspended. > > And added this nice assert. > int JavaThread::java_suspend_self() { > assert(thread_state() == _thread_blocked, "wrong state for > java_suspend_self()"); > > When checking if a thread is safepoint/handshake safe there no need to > look at ext suspend flag anymore, since the thread is blocked. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8252414 > Code: > http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html > > Passes t1-5 > > Thanks, Robbin From daniel.daugherty at oracle.com Fri Aug 28 15:57:30 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 28 Aug 2020 11:57:30 -0400 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: References: <0409009e-d815-2ce3-618f-39ab83250a11@oracle.com> Message-ID: Would you mind if I filed a bug on your behalf? Something like: ??? possible race in java_suspend_self_with_safepoint_check() Dan On 8/28/20 11:53 AM, Reingruber, Richard wrote: > Hi Dan, > > thanks for commenting on this. > > I'll start a new thread (probably after the weekend). > > Thanks, Richard. > > -----Original Message----- > From: Daniel D. Daugherty > Sent: Freitag, 28. August 2020 17:39 > To: Reingruber, Richard ; Robbin Ehn ; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe > > Hi Richard, > > This is the right place and the right people have been watching this > thread, but we should take this up in a new thread and likely with a > new bug ID... > > Dan > > > On 8/28/20 9:33 AM, Reingruber, Richard wrote: >> Hi Robbin, >> >> your change looks correct to me. Thanks! >> >> I see a related issue that allows a suspended thread to continue executing >> java. Hope it's ok to ask about it here. >> >> Assume a thread T is resumed and then taken off CPU in JavaThread::java_suspend_self_with_safepoint_check() >> just before restoring its prior state [1]. >> >> In this situation T gets suspended again (by executing VM_ThreadSuspend) >> >> Now T is put back on the CPU. There are code paths where T won't check its >> suspend flags and return to executing java even though it is suspended. >> >> >> Example: JavaCallWrapper >> >> Stack: >> >> JavaThread::java_suspend_self_with_safepoint_check() : void >> JavaThread::handle_special_runtime_exit_condition(bool) : void >> JavaCallWrapper::JavaCallWrapper(const methodHandle &, Handle, JavaValue *, Thread *) >> >> Example: Polling page exception >> >> Stack: >> >> JavaThread::handle_special_runtime_exit_condition(bool) : void >> SafepointSynchronize::block(JavaThread *) : void >> SafepointMechanism::block_or_handshake(JavaThread *) : void >> SafepointMechanism::block_if_requested_slow(JavaThread *) : void >> SafepointMechanism::block_if_requested(JavaThread *) : void >> ThreadSafepointState::handle_polling_page_exception() : void >> >> Would you agree or have I missed something? >> >> Note that there is no problem when T is put back on CPU while a handshake is >> executed on its behalf because after restoring its prior state T calls >> SafepointMechanism::block_if_requested() in >> JavaThread::java_suspend_self_with_safepoint_check(). >> >> Thanks, Richard. >> >> [1] T is taken off CPU before restoring its prior state. >> https://urldefense.com/v3/__https://github.com/openjdk/jdk/blob/30c2dbea95f817c93f84f0d8f7dc3b5c159b5e25/src/hotspot/share/runtime/thread.cpp*L2619__;Iw!!GqivPVa7Brio!N14B23JPPfP4dkCJeZPO3k0RlIRBIuKdiBY7lZZAHZeGHPnz00kp_R7ZxQO-4EwWeAdF$ >> >> -----Original Message----- >> From: hotspot-runtime-dev On Behalf Of Robbin Ehn >> Sent: Donnerstag, 27. August 2020 09:21 >> To: hotspot-runtime-dev at openjdk.java.net >> Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe >> >> Hi all, please review. >> >> In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a >> thread is always blocked when suspended. >> >> And added this nice assert. >> int JavaThread::java_suspend_self() { >> assert(thread_state() == _thread_blocked, "wrong state for >> java_suspend_self()"); >> >> When checking if a thread is safepoint/handshake safe there no need to >> look at ext suspend flag anymore, since the thread is blocked. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8252414 >> Code: >> http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html >> >> Passes t1-5 >> >> Thanks, Robbin From robbin.ehn at oracle.com Fri Aug 28 16:03:38 2020 From: robbin.ehn at oracle.com (Robbin Ehn) Date: Fri, 28 Aug 2020 18:03:38 +0200 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: References: Message-ID: <3be4308b-7fd7-afdb-0e73-593981ea31b9@oracle.com> Hi Richard, On 2020-08-28 15:33, Reingruber, Richard wrote: > Hi Robbin, > > your change looks correct to me. Thanks! Thanks! Sorry, I already push and cannot credit you. (maybe I broke the 24h...) Had a quick glance, yes it looks like we are missing a check on _suspend_flag (e.g. call to ~handle_special_runtime_exit_condition, causing this handling to be recursive). Should be producible with a sleep before setting the state back. Create a issue in jira! Thanks, Robbin > > I see a related issue that allows a suspended thread to continue executing > java. Hope it's ok to ask about it here. > > Assume a thread T is resumed and then taken off CPU in JavaThread::java_suspend_self_with_safepoint_check() > just before restoring its prior state [1]. > > In this situation T gets suspended again (by executing VM_ThreadSuspend) > > Now T is put back on the CPU. There are code paths where T won't check its > suspend flags and return to executing java even though it is suspended. > > > Example: JavaCallWrapper > > Stack: > > JavaThread::java_suspend_self_with_safepoint_check() : void > JavaThread::handle_special_runtime_exit_condition(bool) : void > JavaCallWrapper::JavaCallWrapper(const methodHandle &, Handle, JavaValue *, Thread *) > > Example: Polling page exception > > Stack: > > JavaThread::handle_special_runtime_exit_condition(bool) : void > SafepointSynchronize::block(JavaThread *) : void > SafepointMechanism::block_or_handshake(JavaThread *) : void > SafepointMechanism::block_if_requested_slow(JavaThread *) : void > SafepointMechanism::block_if_requested(JavaThread *) : void > ThreadSafepointState::handle_polling_page_exception() : void > > Would you agree or have I missed something? > > Note that there is no problem when T is put back on CPU while a handshake is > executed on its behalf because after restoring its prior state T calls > SafepointMechanism::block_if_requested() in > JavaThread::java_suspend_self_with_safepoint_check(). > > Thanks, Richard. > > [1] T is taken off CPU before restoring its prior state. > https://urldefense.com/v3/__https://github.com/openjdk/jdk/blob/30c2dbea95f817c93f84f0d8f7dc3b5c159b5e25/src/hotspot/share/runtime/thread.cpp*L2619__;Iw!!GqivPVa7Brio!JQs56n6YnkiVXt_nGgC8TVQQDa-YtrDVsgtxE4CMDkszZ4iQmghMsuIU0ojdRbpU$ > > -----Original Message----- > From: hotspot-runtime-dev On Behalf Of Robbin Ehn > Sent: Donnerstag, 27. August 2020 09:21 > To: hotspot-runtime-dev at openjdk.java.net > Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe > > Hi all, please review. > > In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a > thread is always blocked when suspended. > > And added this nice assert. > int JavaThread::java_suspend_self() { > assert(thread_state() == _thread_blocked, "wrong state for > java_suspend_self()"); > > When checking if a thread is safepoint/handshake safe there no need to > look at ext suspend flag anymore, since the thread is blocked. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8252414 > Code: > http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html > > Passes t1-5 > > Thanks, Robbin > From richard.reingruber at sap.com Fri Aug 28 17:18:53 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 28 Aug 2020 17:18:53 +0000 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: References: <0409009e-d815-2ce3-618f-39ab83250a11@oracle.com> Message-ID: Sounds good. Thanks for doing it. Richard. -----Original Message----- From: Daniel D. Daugherty Sent: Freitag, 28. August 2020 17:58 To: Reingruber, Richard ; Robbin Ehn ; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe Would you mind if I filed a bug on your behalf? Something like: ??? possible race in java_suspend_self_with_safepoint_check() Dan On 8/28/20 11:53 AM, Reingruber, Richard wrote: > Hi Dan, > > thanks for commenting on this. > > I'll start a new thread (probably after the weekend). > > Thanks, Richard. > > -----Original Message----- > From: Daniel D. Daugherty > Sent: Freitag, 28. August 2020 17:39 > To: Reingruber, Richard ; Robbin Ehn ; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe > > Hi Richard, > > This is the right place and the right people have been watching this > thread, but we should take this up in a new thread and likely with a > new bug ID... > > Dan > > > On 8/28/20 9:33 AM, Reingruber, Richard wrote: >> Hi Robbin, >> >> your change looks correct to me. Thanks! >> >> I see a related issue that allows a suspended thread to continue executing >> java. Hope it's ok to ask about it here. >> >> Assume a thread T is resumed and then taken off CPU in JavaThread::java_suspend_self_with_safepoint_check() >> just before restoring its prior state [1]. >> >> In this situation T gets suspended again (by executing VM_ThreadSuspend) >> >> Now T is put back on the CPU. There are code paths where T won't check its >> suspend flags and return to executing java even though it is suspended. >> >> >> Example: JavaCallWrapper >> >> Stack: >> >> JavaThread::java_suspend_self_with_safepoint_check() : void >> JavaThread::handle_special_runtime_exit_condition(bool) : void >> JavaCallWrapper::JavaCallWrapper(const methodHandle &, Handle, JavaValue *, Thread *) >> >> Example: Polling page exception >> >> Stack: >> >> JavaThread::handle_special_runtime_exit_condition(bool) : void >> SafepointSynchronize::block(JavaThread *) : void >> SafepointMechanism::block_or_handshake(JavaThread *) : void >> SafepointMechanism::block_if_requested_slow(JavaThread *) : void >> SafepointMechanism::block_if_requested(JavaThread *) : void >> ThreadSafepointState::handle_polling_page_exception() : void >> >> Would you agree or have I missed something? >> >> Note that there is no problem when T is put back on CPU while a handshake is >> executed on its behalf because after restoring its prior state T calls >> SafepointMechanism::block_if_requested() in >> JavaThread::java_suspend_self_with_safepoint_check(). >> >> Thanks, Richard. >> >> [1] T is taken off CPU before restoring its prior state. >> https://urldefense.com/v3/__https://github.com/openjdk/jdk/blob/30c2dbea95f817c93f84f0d8f7dc3b5c159b5e25/src/hotspot/share/runtime/thread.cpp*L2619__;Iw!!GqivPVa7Brio!N14B23JPPfP4dkCJeZPO3k0RlIRBIuKdiBY7lZZAHZeGHPnz00kp_R7ZxQO-4EwWeAdF$ >> >> -----Original Message----- >> From: hotspot-runtime-dev On Behalf Of Robbin Ehn >> Sent: Donnerstag, 27. August 2020 09:21 >> To: hotspot-runtime-dev at openjdk.java.net >> Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe >> >> Hi all, please review. >> >> In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a >> thread is always blocked when suspended. >> >> And added this nice assert. >> int JavaThread::java_suspend_self() { >> assert(thread_state() == _thread_blocked, "wrong state for >> java_suspend_self()"); >> >> When checking if a thread is safepoint/handshake safe there no need to >> look at ext suspend flag anymore, since the thread is blocked. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8252414 >> Code: >> http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html >> >> Passes t1-5 >> >> Thanks, Robbin From richard.reingruber at sap.com Fri Aug 28 17:27:05 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 28 Aug 2020 17:27:05 +0000 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: <3be4308b-7fd7-afdb-0e73-593981ea31b9@oracle.com> References: <3be4308b-7fd7-afdb-0e73-593981ea31b9@oracle.com> Message-ID: Hi Robbin, > Thanks! Sorry, I already push and cannot credit you. > (maybe I broke the 24h...) No problem really. > Had a quick glance, yes it looks like we are missing a check on > _suspend_flag (e.g. call to ~handle_special_runtime_exit_condition, > causing this handling to be recursive). > Should be producible with a sleep before setting the state back. Yes I think so. > Create a issue in jira! Will do on Monday if Dan hasn't. Thanks, Richard. -----Original Message----- From: Robbin Ehn Sent: Freitag, 28. August 2020 18:04 To: Reingruber, Richard ; hotspot-runtime-dev at openjdk.java.net Subject: Re: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe Hi Richard, On 2020-08-28 15:33, Reingruber, Richard wrote: > Hi Robbin, > > your change looks correct to me. Thanks! Thanks! Sorry, I already push and cannot credit you. (maybe I broke the 24h...) Had a quick glance, yes it looks like we are missing a check on _suspend_flag (e.g. call to ~handle_special_runtime_exit_condition, causing this handling to be recursive). Should be producible with a sleep before setting the state back. Create a issue in jira! Thanks, Robbin > > I see a related issue that allows a suspended thread to continue executing > java. Hope it's ok to ask about it here. > > Assume a thread T is resumed and then taken off CPU in JavaThread::java_suspend_self_with_safepoint_check() > just before restoring its prior state [1]. > > In this situation T gets suspended again (by executing VM_ThreadSuspend) > > Now T is put back on the CPU. There are code paths where T won't check its > suspend flags and return to executing java even though it is suspended. > > > Example: JavaCallWrapper > > Stack: > > JavaThread::java_suspend_self_with_safepoint_check() : void > JavaThread::handle_special_runtime_exit_condition(bool) : void > JavaCallWrapper::JavaCallWrapper(const methodHandle &, Handle, JavaValue *, Thread *) > > Example: Polling page exception > > Stack: > > JavaThread::handle_special_runtime_exit_condition(bool) : void > SafepointSynchronize::block(JavaThread *) : void > SafepointMechanism::block_or_handshake(JavaThread *) : void > SafepointMechanism::block_if_requested_slow(JavaThread *) : void > SafepointMechanism::block_if_requested(JavaThread *) : void > ThreadSafepointState::handle_polling_page_exception() : void > > Would you agree or have I missed something? > > Note that there is no problem when T is put back on CPU while a handshake is > executed on its behalf because after restoring its prior state T calls > SafepointMechanism::block_if_requested() in > JavaThread::java_suspend_self_with_safepoint_check(). > > Thanks, Richard. > > [1] T is taken off CPU before restoring its prior state. > https://urldefense.com/v3/__https://github.com/openjdk/jdk/blob/30c2dbea95f817c93f84f0d8f7dc3b5c159b5e25/src/hotspot/share/runtime/thread.cpp*L2619__;Iw!!GqivPVa7Brio!JQs56n6YnkiVXt_nGgC8TVQQDa-YtrDVsgtxE4CMDkszZ4iQmghMsuIU0ojdRbpU$ > > -----Original Message----- > From: hotspot-runtime-dev On Behalf Of Robbin Ehn > Sent: Donnerstag, 27. August 2020 09:21 > To: hotspot-runtime-dev at openjdk.java.net > Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe > > Hi all, please review. > > In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a > thread is always blocked when suspended. > > And added this nice assert. > int JavaThread::java_suspend_self() { > assert(thread_state() == _thread_blocked, "wrong state for > java_suspend_self()"); > > When checking if a thread is safepoint/handshake safe there no need to > look at ext suspend flag anymore, since the thread is blocked. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8252414 > Code: > http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html > > Passes t1-5 > > Thanks, Robbin > From daniel.daugherty at oracle.com Fri Aug 28 17:28:28 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 28 Aug 2020 13:28:28 -0400 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: References: <0409009e-d815-2ce3-618f-39ab83250a11@oracle.com> Message-ID: <800889d3-74d3-3701-cb6b-84559be6bb83@oracle.com> Done: ??? JDK-8252521 possible race in java_suspend_self_with_safepoint_check ??? https://bugs.openjdk.java.net/browse/JDK-8252521 Dan On 8/28/20 1:18 PM, Reingruber, Richard wrote: > Sounds good. Thanks for doing it. > > Richard. > > -----Original Message----- > From: Daniel D. Daugherty > Sent: Freitag, 28. August 2020 17:58 > To: Reingruber, Richard ; Robbin Ehn ; hotspot-runtime-dev at openjdk.java.net > Subject: Re: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe > > Would you mind if I filed a bug on your behalf? Something like: > > ??? possible race in java_suspend_self_with_safepoint_check() > > Dan > > On 8/28/20 11:53 AM, Reingruber, Richard wrote: >> Hi Dan, >> >> thanks for commenting on this. >> >> I'll start a new thread (probably after the weekend). >> >> Thanks, Richard. >> >> -----Original Message----- >> From: Daniel D. Daugherty >> Sent: Freitag, 28. August 2020 17:39 >> To: Reingruber, Richard ; Robbin Ehn ; hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe >> >> Hi Richard, >> >> This is the right place and the right people have been watching this >> thread, but we should take this up in a new thread and likely with a >> new bug ID... >> >> Dan >> >> >> On 8/28/20 9:33 AM, Reingruber, Richard wrote: >>> Hi Robbin, >>> >>> your change looks correct to me. Thanks! >>> >>> I see a related issue that allows a suspended thread to continue executing >>> java. Hope it's ok to ask about it here. >>> >>> Assume a thread T is resumed and then taken off CPU in JavaThread::java_suspend_self_with_safepoint_check() >>> just before restoring its prior state [1]. >>> >>> In this situation T gets suspended again (by executing VM_ThreadSuspend) >>> >>> Now T is put back on the CPU. There are code paths where T won't check its >>> suspend flags and return to executing java even though it is suspended. >>> >>> >>> Example: JavaCallWrapper >>> >>> Stack: >>> >>> JavaThread::java_suspend_self_with_safepoint_check() : void >>> JavaThread::handle_special_runtime_exit_condition(bool) : void >>> JavaCallWrapper::JavaCallWrapper(const methodHandle &, Handle, JavaValue *, Thread *) >>> >>> Example: Polling page exception >>> >>> Stack: >>> >>> JavaThread::handle_special_runtime_exit_condition(bool) : void >>> SafepointSynchronize::block(JavaThread *) : void >>> SafepointMechanism::block_or_handshake(JavaThread *) : void >>> SafepointMechanism::block_if_requested_slow(JavaThread *) : void >>> SafepointMechanism::block_if_requested(JavaThread *) : void >>> ThreadSafepointState::handle_polling_page_exception() : void >>> >>> Would you agree or have I missed something? >>> >>> Note that there is no problem when T is put back on CPU while a handshake is >>> executed on its behalf because after restoring its prior state T calls >>> SafepointMechanism::block_if_requested() in >>> JavaThread::java_suspend_self_with_safepoint_check(). >>> >>> Thanks, Richard. >>> >>> [1] T is taken off CPU before restoring its prior state. >>> https://urldefense.com/v3/__https://github.com/openjdk/jdk/blob/30c2dbea95f817c93f84f0d8f7dc3b5c159b5e25/src/hotspot/share/runtime/thread.cpp*L2619__;Iw!!GqivPVa7Brio!N14B23JPPfP4dkCJeZPO3k0RlIRBIuKdiBY7lZZAHZeGHPnz00kp_R7ZxQO-4EwWeAdF$ >>> >>> -----Original Message----- >>> From: hotspot-runtime-dev On Behalf Of Robbin Ehn >>> Sent: Donnerstag, 27. August 2020 09:21 >>> To: hotspot-runtime-dev at openjdk.java.net >>> Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe >>> >>> Hi all, please review. >>> >>> In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a >>> thread is always blocked when suspended. >>> >>> And added this nice assert. >>> int JavaThread::java_suspend_self() { >>> assert(thread_state() == _thread_blocked, "wrong state for >>> java_suspend_self()"); >>> >>> When checking if a thread is safepoint/handshake safe there no need to >>> look at ext suspend flag anymore, since the thread is blocked. >>> >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8252414 >>> Code: >>> http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html >>> >>> Passes t1-5 >>> >>> Thanks, Robbin From calvin.cheung at oracle.com Fri Aug 28 17:42:51 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Fri, 28 Aug 2020 10:42:51 -0700 Subject: RFR(S): 8251860: ClassLoaderData::loaded_classes_do fails with "assert(ZAddress::is_marked(addr)) failed: Should be marked" In-Reply-To: References: <9217b9d3-4be7-b4f4-7be3-e4473fe18cf9@oracle.com> Message-ID: <1a85f6c1-76b1-61a3-c188-aaa7413ed04e@oracle.com> Hi Coleen, Thanks for your review. I've re-implemented the fix with your suggestions. Here's an updated webrev: ??? http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.01/ On 8/26/20 3:24 PM, Coleen Phillimore wrote: > > Hi Calvin, > > I think you should change this to call > ClassLoaderDataGraph::loaded_cld_do(&collect_cld) and collect > ClassLoaderData in the GrowableArray, since you then walk through them > again to link the classes in the class loader data. With the ClassLoaderDataGraph::loaded_cld_do approach, some klasses in the _klasses list may already in the link state. So I have to account for that and not link those klasses again. It complicates the fix a little bit. > > Can linking one klass cause a new class to be added to a _klasses list > of a CLD that you've already walked?? It seems like you need the do() > loop in the current code wrapped around walking the clds, then link > the classes until no progress.? You can fast exit when you find a > klass linked, I believe, since the classes are added to the front of > the _klasses list. Yes, it is possible. The updated patch will check if the first klass in the _klass list of each CLD hasn't been linked, then it will loop back again to go through all the CLD's _klasses lists. > > Also with this change, you can remove locked_loaded_classes_do() > because it was added for CDS and apparently the comment is wrong. > > // This case can block but cannot do unloading (called from CDS) > void ClassLoaderDataGraph::unlocked_loaded_classes_do(KlassClosure* > klass_closure) { > ? for (ClassLoaderData* cld = _head; cld != NULL; cld = cld->next()) { > ??? cld->loaded_classes_do(klass_closure); > ? } > } I've removed the above. thanks, Calvin > > Thanks, > Coleen > > On 8/26/20 5:35 PM, calvin.cheung at oracle.com wrote: >> bug: https://bugs.openjdk.java.net/browse/JDK-8251860 >> >> webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.00/ >> >> Please refer to the bug report for an analysis of the crash. >> >> The proposed fix is to acquire the ClassLoaderDataGraph_lock before >> calling ClassLoaderDataGraph::loaded_classes_do(&link_closure). The >> link_closure will not do the actual linking but just to store the >> InstanceKlass'es in an array and increment the keep_alive counter of >> the class_loader_data of each InstanceKlass. After the call to >> ClassLoaderDataGraph::loaded_classes_do, each InstanceKlass in the >> array will be linked and the keep_alive counter of the >> class_loader_data will be decremented. >> >> Testing: >> ??? - tested about 40 times on the MacOS host where the crash was seen >> ?????? (before the fix, the crash was seen about once in 20 runs) >> ??? - Tier1 - 4. >> thanks, >> Calvin > From martin.doerr at sap.com Fri Aug 28 17:53:22 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 28 Aug 2020 17:53:22 +0000 Subject: Fatal errors when running JCK tests with JDK15/16 debug build Message-ID: Hi, we have seen the following fatal error more than 50 times since 2020-05-25 in various JCK tests vm/jvmti. fatal error: String conversion failure: [check] ExitLock destroyed --> [check] ExitLock exited (followed by garbage output) 8166358: Re-enable String verification in java_lang_String::create_from_str() was pushed at that date which introduced the call to fatal. Stack (example from linuxppc64le, but also observed on x86 and aarch64): V [libjvm.so+0xee242c] java_lang_String::create_from_str(char const*, Thread*) [clone .part.158]+0x51c V [libjvm.so+0xee2530] java_lang_String::create_oop_from_str(char const*, Thread*)+0x40 V [libjvm.so+0x1026a30] jni_NewStringUTF+0x1e0 C [libjckjvmti.so+0x3ce4c] logWrite+0x5c C [libjckjvmti.so+0x3cd20] lprintf+0x170 C [libjckjvmti.so+0x485b8] gast00104_agent_proc+0x254 V [libjvm.so+0x1218f0c] JvmtiAgentThread::call_start_function()+0x24c V [libjvm.so+0x193a8fc] JavaThread::thread_main_inner()+0x32c V [libjvm.so+0x19418a0] Thread::call_run()+0x160 V [libjvm.so+0x15c9d0c] thread_native_entry(Thread*)+0x18c C [libpthread.so.0+0x9b48] start_thread+0x108 (Problem could have been there before but without this fatal message.) The messages are generated by: tests/vm/jvmti/GetAllStackTraces/gast001/gast00104/gast00104.c This looks like a race condition. The message changes while the VM creates a String object from it. Has anybody seen this before? Is it a test problem? I'm not familiar with the lprintf calls in the test. Best regards, Martin From richard.reingruber at sap.com Fri Aug 28 18:02:36 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Fri, 28 Aug 2020 18:02:36 +0000 Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe In-Reply-To: <800889d3-74d3-3701-cb6b-84559be6bb83@oracle.com> References: <0409009e-d815-2ce3-618f-39ab83250a11@oracle.com> , <800889d3-74d3-3701-cb6b-84559be6bb83@oracle.com> Message-ID: Excellent! Cheers, Richard. > Am 28.08.2020 um 19:31 schrieb Daniel D. Daugherty : > > ?Done: > > JDK-8252521 possible race in java_suspend_self_with_safepoint_check > https://bugs.openjdk.java.net/browse/JDK-8252521 > > Dan > > >> On 8/28/20 1:18 PM, Reingruber, Richard wrote: >> Sounds good. Thanks for doing it. >> >> Richard. >> >> -----Original Message----- >> From: Daniel D. Daugherty >> Sent: Freitag, 28. August 2020 17:58 >> To: Reingruber, Richard ; Robbin Ehn ; hotspot-runtime-dev at openjdk.java.net >> Subject: Re: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe >> >> Would you mind if I filed a bug on your behalf? Something like: >> >> possible race in java_suspend_self_with_safepoint_check() >> >> Dan >> >>> On 8/28/20 11:53 AM, Reingruber, Richard wrote: >>> Hi Dan, >>> >>> thanks for commenting on this. >>> >>> I'll start a new thread (probably after the weekend). >>> >>> Thanks, Richard. >>> >>> -----Original Message----- >>> From: Daniel D. Daugherty >>> Sent: Freitag, 28. August 2020 17:39 >>> To: Reingruber, Richard ; Robbin Ehn ; hotspot-runtime-dev at openjdk.java.net >>> Subject: Re: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe >>> >>> Hi Richard, >>> >>> This is the right place and the right people have been watching this >>> thread, but we should take this up in a new thread and likely with a >>> new bug ID... >>> >>> Dan >>> >>> >>> On 8/28/20 9:33 AM, Reingruber, Richard wrote: >>>> Hi Robbin, >>>> >>>> your change looks correct to me. Thanks! >>>> >>>> I see a related issue that allows a suspended thread to continue executing >>>> java. Hope it's ok to ask about it here. >>>> >>>> Assume a thread T is resumed and then taken off CPU in JavaThread::java_suspend_self_with_safepoint_check() >>>> just before restoring its prior state [1]. >>>> >>>> In this situation T gets suspended again (by executing VM_ThreadSuspend) >>>> >>>> Now T is put back on the CPU. There are code paths where T won't check its >>>> suspend flags and return to executing java even though it is suspended. >>>> >>>> >>>> Example: JavaCallWrapper >>>> >>>> Stack: >>>> >>>> JavaThread::java_suspend_self_with_safepoint_check() : void >>>> JavaThread::handle_special_runtime_exit_condition(bool) : void >>>> JavaCallWrapper::JavaCallWrapper(const methodHandle &, Handle, JavaValue *, Thread *) >>>> >>>> Example: Polling page exception >>>> >>>> Stack: >>>> >>>> JavaThread::handle_special_runtime_exit_condition(bool) : void >>>> SafepointSynchronize::block(JavaThread *) : void >>>> SafepointMechanism::block_or_handshake(JavaThread *) : void >>>> SafepointMechanism::block_if_requested_slow(JavaThread *) : void >>>> SafepointMechanism::block_if_requested(JavaThread *) : void >>>> ThreadSafepointState::handle_polling_page_exception() : void >>>> >>>> Would you agree or have I missed something? >>>> >>>> Note that there is no problem when T is put back on CPU while a handshake is >>>> executed on its behalf because after restoring its prior state T calls >>>> SafepointMechanism::block_if_requested() in >>>> JavaThread::java_suspend_self_with_safepoint_check(). >>>> >>>> Thanks, Richard. >>>> >>>> [1] T is taken off CPU before restoring its prior state. >>>> https://urldefense.com/v3/__https://github.com/openjdk/jdk/blob/30c2dbea95f817c93f84f0d8f7dc3b5c159b5e25/src/hotspot/share/runtime/thread.cpp*L2619__;Iw!!GqivPVa7Brio!N14B23JPPfP4dkCJeZPO3k0RlIRBIuKdiBY7lZZAHZeGHPnz00kp_R7ZxQO-4EwWeAdF$ >>>> >>>> -----Original Message----- >>>> From: hotspot-runtime-dev On Behalf Of Robbin Ehn >>>> Sent: Donnerstag, 27. August 2020 09:21 >>>> To: hotspot-runtime-dev at openjdk.java.net >>>> Subject: RFR(s): 8252414: Redundant suspend check when determining if a java thread is safe >>>> >>>> Hi all, please review. >>>> >>>> In 8221207 - "Redo JDK-8218446 - SuspendAtExit hangs" we fixed so a >>>> thread is always blocked when suspended. >>>> >>>> And added this nice assert. >>>> int JavaThread::java_suspend_self() { >>>> assert(thread_state() == _thread_blocked, "wrong state for >>>> java_suspend_self()"); >>>> >>>> When checking if a thread is safepoint/handshake safe there no need to >>>> look at ext suspend flag anymore, since the thread is blocked. >>>> >>>> Bug: >>>> https://bugs.openjdk.java.net/browse/JDK-8252414 >>>> Code: >>>> http://cr.openjdk.java.net/~rehn/8252414/webrev/index.html >>>> >>>> Passes t1-5 >>>> >>>> Thanks, Robbin > From coleen.phillimore at oracle.com Fri Aug 28 19:18:09 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 28 Aug 2020 15:18:09 -0400 Subject: RFR (S) 8230052: MLVM findDeadlock test timed out In-Reply-To: <13b41959-00fb-a647-dc15-d424246e13f4@oracle.com> References: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> <13b41959-00fb-a647-dc15-d424246e13f4@oracle.com> Message-ID: Continuing the discussion in the bug, I've taken out the System.gc() and added -Xlog:gc+safepoint to see if these give any ideas why the test times out. See: bug link https://bugs.openjdk.java.net/browse/JDK-8230052 open webrev at http://cr.openjdk.java.net/~coleenp/2020/8230052.02/webrev thanks, Coleen On 8/28/20 8:22 AM, Patricio Chilano wrote: > Hi Coleen, > > On 8/27/20 7:53 PM, Coleen Phillimore wrote: >> Summary: Add a System.gc() and reduce iteration count for more >> stability. >> >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8230052.01/webrev >> bug link https://bugs.openjdk.java.net/browse/JDK-8230052 > Looks good to me. > If we see any new failure I think we can try to run the test with > extra VM logging enabled. > > > Thanks, > Patricio >> Ran test 100x with < 1minute run time. Should I add a timeout of 10 >> minutes also? >> >> Thanks, >> Coleen > From igor.ignatyev at oracle.com Fri Aug 28 19:23:16 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 28 Aug 2020 12:23:16 -0700 Subject: RFR (S) 8230052: MLVM findDeadlock test timed out In-Reply-To: References: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> <13b41959-00fb-a647-dc15-d424246e13f4@oracle.com> Message-ID: <12046E4E-A552-4979-AEFF-9CEF6DB5C39D@oracle.com> Hi Coleen, LGTM -- Igor > On Aug 28, 2020, at 12:18 PM, Coleen Phillimore wrote: > > > Continuing the discussion in the bug, I've taken out the System.gc() and added -Xlog:gc+safepoint to see if these give any ideas why the test times out. See: > > bug link https://bugs.openjdk.java.net/browse/JDK-8230052 > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8230052.02/webrev > > thanks, > Coleen > > > On 8/28/20 8:22 AM, Patricio Chilano wrote: >> Hi Coleen, >> >> On 8/27/20 7:53 PM, Coleen Phillimore wrote: >>> Summary: Add a System.gc() and reduce iteration count for more stability. >>> >>> open webrev at http://cr.openjdk.java.net/~coleenp/2020/8230052.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8230052 >> Looks good to me. >> If we see any new failure I think we can try to run the test with extra VM logging enabled. >> >> >> Thanks, >> Patricio >>> Ran test 100x with < 1minute run time. Should I add a timeout of 10 minutes also? >>> >>> Thanks, >>> Coleen >> > From coleen.phillimore at oracle.com Fri Aug 28 19:39:56 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 28 Aug 2020 15:39:56 -0400 Subject: RFR (S) 8230052: MLVM findDeadlock test timed out In-Reply-To: <12046E4E-A552-4979-AEFF-9CEF6DB5C39D@oracle.com> References: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> <13b41959-00fb-a647-dc15-d424246e13f4@oracle.com> <12046E4E-A552-4979-AEFF-9CEF6DB5C39D@oracle.com> Message-ID: Thanks Igor, and thanks for your help! Coleen On 8/28/20 3:23 PM, Igor Ignatyev wrote: > Hi Coleen, > > LGTM > > -- Igor > >> On Aug 28, 2020, at 12:18 PM, Coleen Phillimore wrote: >> >> >> Continuing the discussion in the bug, I've taken out the System.gc() and added -Xlog:gc+safepoint to see if these give any ideas why the test times out. See: >> >> bug link https://bugs.openjdk.java.net/browse/JDK-8230052 >> open webrev at http://cr.openjdk.java.net/~coleenp/2020/8230052.02/webrev >> >> thanks, >> Coleen >> >> >> On 8/28/20 8:22 AM, Patricio Chilano wrote: >>> Hi Coleen, >>> >>> On 8/27/20 7:53 PM, Coleen Phillimore wrote: >>>> Summary: Add a System.gc() and reduce iteration count for more stability. >>>> >>>> open webrev at http://cr.openjdk.java.net/~coleenp/2020/8230052.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8230052 >>> Looks good to me. >>> If we see any new failure I think we can try to run the test with extra VM logging enabled. >>> >>> >>> Thanks, >>> Patricio >>>> Ran test 100x with < 1minute run time. Should I add a timeout of 10 minutes also? >>>> >>>> Thanks, >>>> Coleen From patricio.chilano.mateo at oracle.com Fri Aug 28 19:44:10 2020 From: patricio.chilano.mateo at oracle.com (Patricio Chilano) Date: Fri, 28 Aug 2020 16:44:10 -0300 Subject: RFR (S) 8230052: MLVM findDeadlock test timed out In-Reply-To: References: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> <13b41959-00fb-a647-dc15-d424246e13f4@oracle.com> Message-ID: <63d423da-e51e-02fa-3e4b-87578a9b62a4@oracle.com> On 8/28/20 4:18 PM, Coleen Phillimore wrote: > > Continuing the discussion in the bug, I've taken out the System.gc() > and added -Xlog:gc+safepoint to see if these give any ideas why the > test times out. See: > > bug link https://bugs.openjdk.java.net/browse/JDK-8230052 > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8230052.02/webrev Looks good! Thanks, Patricio > thanks, > Coleen > > > On 8/28/20 8:22 AM, Patricio Chilano wrote: >> Hi Coleen, >> >> On 8/27/20 7:53 PM, Coleen Phillimore wrote: >>> Summary: Add a System.gc() and reduce iteration count for more >>> stability. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2020/8230052.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8230052 >> Looks good to me. >> If we see any new failure I think we can try to run the test with >> extra VM logging enabled. >> >> >> Thanks, >> Patricio >>> Ran test 100x with < 1minute run time. Should I add a timeout of 10 >>> minutes also? >>> >>> Thanks, >>> Coleen >> > From coleen.phillimore at oracle.com Fri Aug 28 20:02:09 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 28 Aug 2020 16:02:09 -0400 Subject: RFR (S) 8230052: MLVM findDeadlock test timed out In-Reply-To: <63d423da-e51e-02fa-3e4b-87578a9b62a4@oracle.com> References: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> <13b41959-00fb-a647-dc15-d424246e13f4@oracle.com> <63d423da-e51e-02fa-3e4b-87578a9b62a4@oracle.com> Message-ID: <3b0ed506-5b55-21b3-0b8b-3ba1be5e23c5@oracle.com> Thanks for help with debugging, Patricio! Coleen On 8/28/20 3:44 PM, Patricio Chilano wrote: > > On 8/28/20 4:18 PM, Coleen Phillimore wrote: >> >> Continuing the discussion in the bug, I've taken out the System.gc() >> and added -Xlog:gc+safepoint to see if these give any ideas why the >> test times out. See: >> >> bug link https://bugs.openjdk.java.net/browse/JDK-8230052 >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8230052.02/webrev > Looks good! > > Thanks, > Patricio >> thanks, >> Coleen >> >> >> On 8/28/20 8:22 AM, Patricio Chilano wrote: >>> Hi Coleen, >>> >>> On 8/27/20 7:53 PM, Coleen Phillimore wrote: >>>> Summary: Add a System.gc() and reduce iteration count for more >>>> stability. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/2020/8230052.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8230052 >>> Looks good to me. >>> If we see any new failure I think we can try to run the test with >>> extra VM logging enabled. >>> >>> >>> Thanks, >>> Patricio >>>> Ran test 100x with < 1minute run time. Should I add a timeout of 10 >>>> minutes also? >>>> >>>> Thanks, >>>> Coleen >>> >> > From gerard.ziemski at oracle.com Fri Aug 28 20:27:38 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Fri, 28 Aug 2020 15:27:38 -0500 Subject: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) In-Reply-To: References: <022e6096-a388-dc79-f959-00a2d933ae89@oracle.com> Message-ID: <1d5921d1-1cb4-b286-a95b-7e2d4b60ced0@oracle.com> hi Thomas, Thank you for your offer to test the changes. I know that you are currently busy with your metaspace changes, but when you do you have some time, then I have made a preliminary changes for AIX here to test:http://cr.openjdk.java.net/~gziemski/8252324_pre2 cheers On 8/27/20 4:48 PM, Thomas St?fe wrote: > Hi Gerard, Martin, > > I like the direction this patch is going, this is definitely a good > cleanup! Also thanks from me for not wanting to break AIX :) > > Wrt to signals AIX should not be that different from other Posix > platforms. Any differences probably stem from the fact that the AIX > port had a long time brewing in our proprietary code base before we > contributed it, with not much sync points with other?ports after the > initial fork. (Memory management in AIX is different for a number of > reasons, but signal stuff should be pretty standard). > > There is this small issue that we use SIGTRAP for implicit null checks > (@Martin: is this still a thing?) but strictly speaking this is a ppc > issue, not an AIX issue, so it hits Linux too (@Martin: there seem to > be small diffs between how sigtrap is handled in AIX and Linux, see > e.g. the use of UseSigTrap, is that by design?). But all that should > do is to add another signal to handle/unblock. > > I would say, give it your best shot and dry-code the changes. When in > doubt, just ask. We will test your patch and correct if necessary. > > --- > > Some more general remarks to your preliminary changes: > > I would really like it if we could start documenting APIs. I am always > astonished that this is so seldom done. Ironically, almost all > existing API comments in os_posix.hpp were contributed by SAP :-) > > Since you now move all APIs to a central location, and it's all fresh > in your head, maybe?this would be a good point to add comments to > them, describing behavior and input/output params. > > --- > > I quite liked the os::Posix namespace and think the signal functions > are well placed there. But I see what you want, you want to separate > the signal functions from the os namespace since that one requires > including?os.hpp? Unfortunately this also makes the diff more > complicated since a lot of code just moved. > > Side note, I wish we would start using real C++ namespaces, we could > make os a namespace and hence split it over as many header files as we > want. Maybe something for future cleanups. > > --- > > ucontext_(s|g)et_pc() have not much to do with signals, even though > they are only used for signal continuation currently. They seem > misplaced in PosixSignals. > > --- > > Thanks, Thomas > > > On Thu, Aug 27, 2020 at 10:33 PM Doerr, Martin > wrote: > > Hi Gerard, > > > I don?t want to straight break AIX port unexpectedly. > Thanks for that! We highly appreciate that you take care of it. > > It?s not surprising that adapting the AIX code is difficult. > > I?ve added Thomas to Cc. He?s surely the most experienced person > for these AIX specific things. > Unfortunately, he?s currently very busy because he?s contributing > his new metaspace implementation. > > We have to discuss how we can help. > > Best regards, > Martin > > > > From: gerard ziemski > > Sent: Donnerstag, 27. August 2020 19:19 > To: hotspot-runtime-dev at openjdk.java.net > > Cc: Schmidt, Lutz >; Lindenmaier, Goetz > >; > Doerr, Martin > > Subject: Attention AIX developers - factoring out POSIX signal > code (JDK-8252324) > > > hi all, > > I?m hoping I have included the right people in this email, but > everyone is welcome to provide feedback. > > I have a code for JDK-8252324, which factors out the common POSIX > signal related code out into common file, so that all platforms > can benefit from fixes that apply to all POSIX platforms (like > JDK-82514380) > > The preliminary webrev can be found here: > http://cr.openjdk.java.net/~gziemski/8252324_pre1 > > > I tried to port the changes to AIX myself, but got bogged down by > more "#ifdef AIX? than I was hoping for (possibly too many > differences to factor out the code for AIX, but frankly I don't > think so), so I stopped (though I probably was on the right track) > - to me it looks like AIX has some extra fixes and differences in > the signal code than BSD/Linux. > > My question is: given my inability to test on AIX, what can I do > to make it easier for AIX developers to adopt this fix? I will try > porting my changes again, but in the meantime if anyone has any > pointers or offer of other help, I?d appreciate it. I don?t want > to straight break AIX port unexpectedly. > > Reference: https://bugs.openjdk.java.net/browse/JDK-8252324 > > Some notes about the affected APIs: > > -------------------------------------------------------------------------------------------------- > The APIs which moved from os/bsd/os_bsd.cpp (similarly for linux) > to to os/posix/PosixSignals.cpp: > > //////////////////////////////////////////////////////////////////////////////// > // signal support > void os::Bsd::signal_sets_init() > sigset_t* os::Bsd::unblocked_signals() > sigset_t* os::Bsd::vm_signals() > void os::Bsd::hotspot_sigmask(Thread* thread) > //////////////////////////////////////////////////////////////////////////////// > // sun.misc.Signal support > static void UserHandler(int sig, void *siginfo, void *context) > void* os::user_handler() > void* os::signal(int signal_number, void* handler) > void os::signal_raise(int signal_number) > int os::sigexitnum_pd() > static void jdk_misc_signal_init() > void os::signal_notify(int sig) > static int check_pending_signals() > int os::signal_wait() > //////////////////////////////////////////////////////////////////////////////// > // suspend/resume support > static void resume_clear_context(OSThread *osthread) > static void suspend_save_context(OSThread *osthread, siginfo_t* > siginfo, ucontext_t* context) > static void SR_handler(int sig, siginfo_t* siginfo, ucontext_t* > context) > static int SR_initialize() > static int sr_notify(OSThread* osthread) > static bool do_suspend(OSThread* osthread) > static void do_resume(OSThread* osthread) > /////////////////////////////////////////////////////////////////////////////////// > // signal handling (except suspend/resume) > static void signalHandler(int sig, siginfo_t* info, void* uc) > struct sigaction* os::Bsd::get_chained_signal_action(int sig) > static bool call_chained_handler(struct sigaction *actp, int sig, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?siginfo_t *siginfo, void *context) > bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* > context) > int os::Bsd::get_our_sigflags(int sig) > void os::Bsd::set_our_sigflags(int sig, int flags) > void os::Bsd::set_signal_handler(int sig, bool set_installed) > void os::Bsd::install_signal_handlers() > static const char* get_signal_handler_name(address handler, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?char* buf, int buflen) > static void print_signal_handler(outputStream* st, int sig, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?char* buf, size_t buflen) > void os::run_periodic_checks() > void os::Bsd::check_signal_handler(int sig) > > ----------------------------------------------------------------------------- > The APIs which moved from os/posix/os_posix.cpp to > os/posix/PosixSignals.cpp: > > const char* os::Posix::get_signal_name(int sig, char* out, size_t > outlen) > int os::Posix::get_signal_number(const char* signal_name) > int os::get_signal_number(const char* signal_name) > bool os::Posix::is_valid_signal(int sig) > bool os::Posix::is_sig_ignored(int sig) > const char* os::exception_name(int sig, char* buf, size_t size) > const char* os::Posix::describe_signal_set_short(const sigset_t* > set, char* buffer, size_t buf_size) > void os::Posix::print_signal_set_short(outputStream* st, const > sigset_t* set) > const char* os::Posix::describe_sa_flags(int flags, char* buffer, > size_t size) > oid os::Posix::print_sa_flags(outputStream* st, int flags) > static bool get_signal_code_description(const siginfo_t* si, > enum_sigcode_desc_t* out) > void os::print_siginfo(outputStream* os, const void* si0) > bool os::signal_thread(Thread* thread, int sig, const char* reason) > int os::Posix::unblock_thread_signal_mask(const sigset_t *set) > address os::Posix::ucontext_get_pc(const ucontext_t* ctx) > void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) > struct sigaction* os::Posix::get_preinstalled_handler(int sig) > void os::Posix::save_preinstalled_handler(int sig, struct > sigaction& oldAct) > > > > DETAILS: > > -------------------------------------------------------- > Public APIs which are now internal static PosixSignals:: > > sigset_t* os::Bsd::vm_signals() > struct sigaction* os::Bsd::get_chained_signal_action(int sig) > int os::Bsd::get_our_sigflags(int sig) > void os::Bsd::set_our_sigflags(int sig, int flags) > void os::Bsd::set_signal_handler(int sig, bool set_installed) > void os::Bsd::check_signal_handler(int sig) > const char* os::Posix::get_signal_name(int sig, char* out, size_t > outlen) > bool os::Posix::is_valid_signal(int sig) > const char* os::Posix::describe_signal_set_short(const sigset_t* > set, char* buffer, size_t buf_size) > void os::Posix::print_signal_set_short(outputStream* st, const > sigset_t* set) > const char* os::Posix::describe_sa_flags(int flags, char* buffer, > size_t size) > oid os::Posix::print_sa_flags(outputStream* st, int flags) > static bool get_signal_code_description(const siginfo_t* si, > enum_sigcode_desc_t* out) > void os::Posix::save_preinstalled_handler(int sig, struct > sigaction& oldAct) > > ------------------------------------------------ > Public APIs which moved to public PosixSignals:: > > void os::Bsd::signal_sets_init() > void os::Bsd::hotspot_sigmask(Thread* thread) > bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* > context) > void os::Bsd::install_signal_handlers() > bool os::Posix::is_sig_ignored(int sig) > int os::Posix::unblock_thread_signal_mask(const sigset_t *set) > address os::Posix::ucontext_get_pc(const ucontext_t* ctx) > void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) > > ---------------------------------------------------- > Internal APIs which are now public in PosixSignals:: > > static void jdk_misc_signal_init() > static int SR_initialize() > static bool do_suspend(OSThread* osthread) > static void do_resume(OSThread* osthread) > static void print_signal_handler(outputStream* st, int sig, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?char* buf, size_t buflen) > > -------------------------- > New APIs in PosixSignals:: > > static bool are_signal_handlers_installed(); > > > cheers > From martin.doerr at sap.com Fri Aug 28 21:23:07 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Fri, 28 Aug 2020 21:23:07 +0000 Subject: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) In-Reply-To: <1d5921d1-1cb4-b286-a95b-7e2d4b60ced0@oracle.com> References: <022e6096-a388-dc79-f959-00a2d933ae89@oracle.com> <1d5921d1-1cb4-b286-a95b-7e2d4b60ced0@oracle.com> Message-ID: Hi Gerard, I just gave it a quick spin. Fix for os_aix_ppc.cpp is trivial (see below). signals_posix.cpp takes more effort. I can see that a lot of AIX specific code is for as400 PASE support which we never contributed to OpenJDK. Maybe we can remove it and use the regular posix code instead. @Thomas: What do you think? Link which works correctly (version below links to pre1): http://cr.openjdk.java.net/~gziemski/8252324_pre2/ Best regards, Martin diff -r 39c430bf9bb3 src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp --- a/src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp Fri Aug 28 22:59:25 2020 +0200 +++ b/src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp Fri Aug 28 23:13:25 2020 +0200 @@ -51,6 +51,7 @@ #include "runtime/stubRoutines.hpp" #include "runtime/thread.inline.hpp" #include "runtime/timer.hpp" +#include "signals_posix.hpp" #include "utilities/events.hpp" #include "utilities/vmError.hpp" #ifdef COMPILER1 @@ -225,7 +226,7 @@ JavaThread* thread = NULL; VMThread* vmthread = NULL; - if (os::Aix::signal_handlers_are_installed) { + if (os::signal_handlers_are_installed) { if (t != NULL) { if(t->is_Java_thread()) { thread = (JavaThread*)t; From: gerard ziemski Sent: Freitag, 28. August 2020 22:28 To: Thomas St?fe ; Doerr, Martin Cc: hotspot-runtime-dev at openjdk.java.net; Schmidt, Lutz ; Lindenmaier, Goetz ; Stuefe, Thomas Subject: Re: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) hi Thomas, Thank you for your offer to test the changes. I know that you are currently busy with your metaspace changes, but when you do you have some time, then I have made a preliminary changes for AIX here to test: http://cr.openjdk.java.net/~gziemski/8252324_pre2 cheers On 8/27/20 4:48 PM, Thomas St?fe wrote: Hi Gerard, Martin, I like the direction this patch is going, this is definitely a good cleanup! Also thanks from me for not wanting to break AIX :) Wrt to signals AIX should not be that different from other Posix platforms. Any differences probably stem from the fact that the AIX port had a long time brewing in our proprietary code base before we contributed it, with not much sync points with other ports after the initial fork. (Memory management in AIX is different for a number of reasons, but signal stuff should be pretty standard). There is this small issue that we use SIGTRAP for implicit null checks (@Martin: is this still a thing?) but strictly speaking this is a ppc issue, not an AIX issue, so it hits Linux too (@Martin: there seem to be small diffs between how sigtrap is handled in AIX and Linux, see e.g. the use of UseSigTrap, is that by design?). But all that should do is to add another signal to handle/unblock. I would say, give it your best shot and dry-code the changes. When in doubt, just ask. We will test your patch and correct if necessary. --- Some more general remarks to your preliminary changes: I would really like it if we could start documenting APIs. I am always astonished that this is so seldom done. Ironically, almost all existing API comments in os_posix.hpp were contributed by SAP :-) Since you now move all APIs to a central location, and it's all fresh in your head, maybe this would be a good point to add comments to them, describing behavior and input/output params. --- I quite liked the os::Posix namespace and think the signal functions are well placed there. But I see what you want, you want to separate the signal functions from the os namespace since that one requires including os.hpp? Unfortunately this also makes the diff more complicated since a lot of code just moved. Side note, I wish we would start using real C++ namespaces, we could make os a namespace and hence split it over as many header files as we want. Maybe something for future cleanups. --- ucontext_(s|g)et_pc() have not much to do with signals, even though they are only used for signal continuation currently. They seem misplaced in PosixSignals. --- Thanks, Thomas On Thu, Aug 27, 2020 at 10:33 PM Doerr, Martin > wrote: Hi Gerard, > I don?t want to straight break AIX port unexpectedly. Thanks for that! We highly appreciate that you take care of it. It?s not surprising that adapting the AIX code is difficult. I?ve added Thomas to Cc. He?s surely the most experienced person for these AIX specific things. Unfortunately, he?s currently very busy because he?s contributing his new metaspace implementation. We have to discuss how we can help. Best regards, Martin From: gerard ziemski > Sent: Donnerstag, 27. August 2020 19:19 To: hotspot-runtime-dev at openjdk.java.net Cc: Schmidt, Lutz >; Lindenmaier, Goetz >; Doerr, Martin > Subject: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) hi all, I?m hoping I have included the right people in this email, but everyone is welcome to provide feedback. I have a code for JDK-8252324, which factors out the common POSIX signal related code out into common file, so that all platforms can benefit from fixes that apply to all POSIX platforms (like JDK-82514380) The preliminary webrev can be found here: http://cr.openjdk.java.net/~gziemski/8252324_pre1 I tried to port the changes to AIX myself, but got bogged down by more "#ifdef AIX? than I was hoping for (possibly too many differences to factor out the code for AIX, but frankly I don't think so), so I stopped (though I probably was on the right track) - to me it looks like AIX has some extra fixes and differences in the signal code than BSD/Linux. My question is: given my inability to test on AIX, what can I do to make it easier for AIX developers to adopt this fix? I will try porting my changes again, but in the meantime if anyone has any pointers or offer of other help, I?d appreciate it. I don?t want to straight break AIX port unexpectedly. Reference: https://bugs.openjdk.java.net/browse/JDK-8252324 Some notes about the affected APIs: -------------------------------------------------------------------------------------------------- The APIs which moved from os/bsd/os_bsd.cpp (similarly for linux) to to os/posix/PosixSignals.cpp: //////////////////////////////////////////////////////////////////////////////// // signal support void os::Bsd::signal_sets_init() sigset_t* os::Bsd::unblocked_signals() sigset_t* os::Bsd::vm_signals() void os::Bsd::hotspot_sigmask(Thread* thread) //////////////////////////////////////////////////////////////////////////////// // sun.misc.Signal support static void UserHandler(int sig, void *siginfo, void *context) void* os::user_handler() void* os::signal(int signal_number, void* handler) void os::signal_raise(int signal_number) int os::sigexitnum_pd() static void jdk_misc_signal_init() void os::signal_notify(int sig) static int check_pending_signals() int os::signal_wait() //////////////////////////////////////////////////////////////////////////////// // suspend/resume support static void resume_clear_context(OSThread *osthread) static void suspend_save_context(OSThread *osthread, siginfo_t* siginfo, ucontext_t* context) static void SR_handler(int sig, siginfo_t* siginfo, ucontext_t* context) static int SR_initialize() static int sr_notify(OSThread* osthread) static bool do_suspend(OSThread* osthread) static void do_resume(OSThread* osthread) /////////////////////////////////////////////////////////////////////////////////// // signal handling (except suspend/resume) static void signalHandler(int sig, siginfo_t* info, void* uc) struct sigaction* os::Bsd::get_chained_signal_action(int sig) static bool call_chained_handler(struct sigaction *actp, int sig, siginfo_t *siginfo, void *context) bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* context) int os::Bsd::get_our_sigflags(int sig) void os::Bsd::set_our_sigflags(int sig, int flags) void os::Bsd::set_signal_handler(int sig, bool set_installed) void os::Bsd::install_signal_handlers() static const char* get_signal_handler_name(address handler, char* buf, int buflen) static void print_signal_handler(outputStream* st, int sig, char* buf, size_t buflen) void os::run_periodic_checks() void os::Bsd::check_signal_handler(int sig) ----------------------------------------------------------------------------- The APIs which moved from os/posix/os_posix.cpp to os/posix/PosixSignals.cpp: const char* os::Posix::get_signal_name(int sig, char* out, size_t outlen) int os::Posix::get_signal_number(const char* signal_name) int os::get_signal_number(const char* signal_name) bool os::Posix::is_valid_signal(int sig) bool os::Posix::is_sig_ignored(int sig) const char* os::exception_name(int sig, char* buf, size_t size) const char* os::Posix::describe_signal_set_short(const sigset_t* set, char* buffer, size_t buf_size) void os::Posix::print_signal_set_short(outputStream* st, const sigset_t* set) const char* os::Posix::describe_sa_flags(int flags, char* buffer, size_t size) oid os::Posix::print_sa_flags(outputStream* st, int flags) static bool get_signal_code_description(const siginfo_t* si, enum_sigcode_desc_t* out) void os::print_siginfo(outputStream* os, const void* si0) bool os::signal_thread(Thread* thread, int sig, const char* reason) int os::Posix::unblock_thread_signal_mask(const sigset_t *set) address os::Posix::ucontext_get_pc(const ucontext_t* ctx) void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) struct sigaction* os::Posix::get_preinstalled_handler(int sig) void os::Posix::save_preinstalled_handler(int sig, struct sigaction& oldAct) DETAILS: -------------------------------------------------------- Public APIs which are now internal static PosixSignals:: sigset_t* os::Bsd::vm_signals() struct sigaction* os::Bsd::get_chained_signal_action(int sig) int os::Bsd::get_our_sigflags(int sig) void os::Bsd::set_our_sigflags(int sig, int flags) void os::Bsd::set_signal_handler(int sig, bool set_installed) void os::Bsd::check_signal_handler(int sig) const char* os::Posix::get_signal_name(int sig, char* out, size_t outlen) bool os::Posix::is_valid_signal(int sig) const char* os::Posix::describe_signal_set_short(const sigset_t* set, char* buffer, size_t buf_size) void os::Posix::print_signal_set_short(outputStream* st, const sigset_t* set) const char* os::Posix::describe_sa_flags(int flags, char* buffer, size_t size) oid os::Posix::print_sa_flags(outputStream* st, int flags) static bool get_signal_code_description(const siginfo_t* si, enum_sigcode_desc_t* out) void os::Posix::save_preinstalled_handler(int sig, struct sigaction& oldAct) ------------------------------------------------ Public APIs which moved to public PosixSignals:: void os::Bsd::signal_sets_init() void os::Bsd::hotspot_sigmask(Thread* thread) bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* context) void os::Bsd::install_signal_handlers() bool os::Posix::is_sig_ignored(int sig) int os::Posix::unblock_thread_signal_mask(const sigset_t *set) address os::Posix::ucontext_get_pc(const ucontext_t* ctx) void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) ---------------------------------------------------- Internal APIs which are now public in PosixSignals:: static void jdk_misc_signal_init() static int SR_initialize() static bool do_suspend(OSThread* osthread) static void do_resume(OSThread* osthread) static void print_signal_handler(outputStream* st, int sig, char* buf, size_t buflen) -------------------------- New APIs in PosixSignals:: static bool are_signal_handlers_installed(); cheers From ioi.lam at oracle.com Fri Aug 28 22:31:12 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 28 Aug 2020 15:31:12 -0700 Subject: RFR (S) 8230052: MLVM findDeadlock test timed out In-Reply-To: References: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> <13b41959-00fb-a647-dc15-d424246e13f4@oracle.com> Message-ID: <24c09ce2-a39e-a014-c429-82fbbf4d6d89@oracle.com> Looks good to me. Thanks - Ioi On 8/28/20 12:18 PM, Coleen Phillimore wrote: > > Continuing the discussion in the bug, I've taken out the System.gc() > and added -Xlog:gc+safepoint to see if these give any ideas why the > test times out. See: > > bug link https://bugs.openjdk.java.net/browse/JDK-8230052 > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8230052.02/webrev > > thanks, > Coleen > > > On 8/28/20 8:22 AM, Patricio Chilano wrote: >> Hi Coleen, >> >> On 8/27/20 7:53 PM, Coleen Phillimore wrote: >>> Summary: Add a System.gc() and reduce iteration count for more >>> stability. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2020/8230052.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8230052 >> Looks good to me. >> If we see any new failure I think we can try to run the test with >> extra VM logging enabled. >> >> >> Thanks, >> Patricio >>> Ran test 100x with < 1minute run time. Should I add a timeout of 10 >>> minutes also? >>> >>> Thanks, >>> Coleen >> > From ioi.lam at oracle.com Fri Aug 28 22:54:14 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Fri, 28 Aug 2020 15:54:14 -0700 Subject: RFR(S): 8251860: ClassLoaderData::loaded_classes_do fails with "assert(ZAddress::is_marked(addr)) failed: Should be marked" In-Reply-To: <1a85f6c1-76b1-61a3-c188-aaa7413ed04e@oracle.com> References: <9217b9d3-4be7-b4f4-7be3-e4473fe18cf9@oracle.com> <1a85f6c1-76b1-61a3-c188-aaa7413ed04e@oracle.com> Message-ID: <0d0bdd73-7f9d-4309-a670-4a9d81608bdf@oracle.com> Hi Calvin, ?300 void ClassLoaderData::inc_keep_alive() { ?301?? if (has_class_mirror_holder()) { ?302???? if (!Arguments::is_dumping_archive()) { ?303?????? assert(_keep_alive > 0, "Invalid keep alive increment count"); ?304???? } ?305???? _keep_alive++; ?306?? } ?307 } Why is 302 needed? I think in CollectCLDClosure, you wouldn't collect any CLD whose _keep_alive count is zero. I think MetaspaceShared::link_and_cleanup_shared_classes() can be simplified so you don't need to scan the CLDs in two passes. 1393? while (true) { ????? ? bool has_linked = false; ??????? for (int i = 0; i < _loaded_cld->length(); i++) { ????????? ClassLoaderData* cld = _loaded_cld->at(i); ????????? for (Klass* klass = cld->klasses(); klass != NULL; klass = klass->next_link()) { ???????? ?? if (klass->is_instance_klass()) { ????????? ? ? InstanceKlass* ik = InstanceKlass::cast(ik); ?????????? ?? if (inking_required(ik)) { ??????????? ? ? has_linked = link_class_for_cds(ik, THREAD); ?? ? ? ?????? } ??????????? } ????????? } ??????? } ? ? ? ? if (!has_linked) { ?? ? ? ?? break; ?? ? ?? } ??????? // Class linking includes verification which may load more classes. ? ? ??? // Keep scanning until we have linked no more classes. ? ? ? } Also link_class_for_cds should return the value of MetaspaceShared::try_link_class(), so it has the same behavior as before. The rest of the changes looks good. Thanks - Ioi On 8/28/20 10:42 AM, calvin.cheung at oracle.com wrote: > Hi Coleen, > > Thanks for your review. > > I've re-implemented the fix with your suggestions. > > Here's an updated webrev: > > ??? http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.01/ > > On 8/26/20 3:24 PM, Coleen Phillimore wrote: >> >> Hi Calvin, >> >> I think you should change this to call >> ClassLoaderDataGraph::loaded_cld_do(&collect_cld) and collect >> ClassLoaderData in the GrowableArray, since you then walk through >> them again to link the classes in the class loader data. > With the ClassLoaderDataGraph::loaded_cld_do approach, some klasses in > the _klasses list may already in the link state. So I have to account > for that and not link those klasses again. It complicates the fix a > little bit. >> >> Can linking one klass cause a new class to be added to a _klasses >> list of a CLD that you've already walked?? It seems like you need the >> do() loop in the current code wrapped around walking the clds, then >> link the classes until no progress.? You can fast exit when you find >> a klass linked, I believe, since the classes are added to the front >> of the _klasses list. > Yes, it is possible. The updated patch will check if the first klass > in the _klass list of each CLD hasn't been linked, then it will loop > back again to go through all the CLD's _klasses lists. >> >> Also with this change, you can remove locked_loaded_classes_do() >> because it was added for CDS and apparently the comment is wrong. >> >> // This case can block but cannot do unloading (called from CDS) >> void ClassLoaderDataGraph::unlocked_loaded_classes_do(KlassClosure* >> klass_closure) { >> ? for (ClassLoaderData* cld = _head; cld != NULL; cld = cld->next()) { >> ??? cld->loaded_classes_do(klass_closure); >> ? } >> } > > I've removed the above. > > thanks, > > Calvin > >> >> Thanks, >> Coleen >> >> On 8/26/20 5:35 PM, calvin.cheung at oracle.com wrote: >>> bug: https://bugs.openjdk.java.net/browse/JDK-8251860 >>> >>> webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.00/ >>> >>> Please refer to the bug report for an analysis of the crash. >>> >>> The proposed fix is to acquire the ClassLoaderDataGraph_lock before >>> calling ClassLoaderDataGraph::loaded_classes_do(&link_closure). The >>> link_closure will not do the actual linking but just to store the >>> InstanceKlass'es in an array and increment the keep_alive counter of >>> the class_loader_data of each InstanceKlass. After the call to >>> ClassLoaderDataGraph::loaded_classes_do, each InstanceKlass in the >>> array will be linked and the keep_alive counter of the >>> class_loader_data will be decremented. >>> >>> Testing: >>> ??? - tested about 40 times on the MacOS host where the crash was seen >>> ?????? (before the fix, the crash was seen about once in 20 runs) >>> ??? - Tier1 - 4. >>> thanks, >>> Calvin >> From david.holmes at oracle.com Sat Aug 29 01:55:47 2020 From: david.holmes at oracle.com (David Holmes) Date: Sat, 29 Aug 2020 11:55:47 +1000 Subject: RFR (S) 8230052: MLVM findDeadlock test timed out In-Reply-To: References: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> <13b41959-00fb-a647-dc15-d424246e13f4@oracle.com> Message-ID: <51a0d6f8-5bab-3615-a5a0-2a52e8d6b58b@oracle.com> On 29/08/2020 5:18 am, Coleen Phillimore wrote: > > Continuing the discussion in the bug, I've taken out the System.gc() and > added -Xlog:gc+safepoint to see if these give any ideas why the test > times out. See: So no longer trying to fix it just trying to get more diagnostics. But how much logging info is that going to produce? Are we just going to hit the jtreg buffer limit? I would have gone for some coarser higher-level "logging" initially. It is going to be hard to correlate the log entries with what is happening at the test level without output from the test itself that will appear in-order with the system logging. So I'm not at all sure this logging will really help. But lets give it a try. David > bug link https://bugs.openjdk.java.net/browse/JDK-8230052 > open webrev at http://cr.openjdk.java.net/~coleenp/2020/8230052.02/webrev > > thanks, > Coleen > > > On 8/28/20 8:22 AM, Patricio Chilano wrote: >> Hi Coleen, >> >> On 8/27/20 7:53 PM, Coleen Phillimore wrote: >>> Summary: Add a System.gc() and reduce iteration count for more >>> stability. >>> >>> open webrev at >>> http://cr.openjdk.java.net/~coleenp/2020/8230052.01/webrev >>> bug link https://bugs.openjdk.java.net/browse/JDK-8230052 >> Looks good to me. >> If we see any new failure I think we can try to run the test with >> extra VM logging enabled. >> >> >> Thanks, >> Patricio >>> Ran test 100x with < 1minute run time. Should I add a timeout of 10 >>> minutes also? >>> >>> Thanks, >>> Coleen >> > From igor.ignatyev at oracle.com Sat Aug 29 03:52:19 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 28 Aug 2020 20:52:19 -0700 Subject: RFR(T) : 8252532 : use Utils.TEST_NATIVE_PATH instead of System.getProperty("test.nativepath") Message-ID: http://cr.openjdk.java.net/~iignatyev//8252532/webrev.00 > 145 lines changed: 28 ins; 22 del; 95 mod; Hi all, could you please review this trivial clean up which replaces System.getProperty("test.nativepath") w/ Utils.TEST_NATIVE_PATH where appropriate? while updating these files, I've also cleaned them up a bit, removed unneeded imports, added/removed spaces, etc testing: runtime, serviceability and vmTestbase/nsk/jvmti/ tests on {linux,windows,macos}-x64 JBS: https://bugs.openjdk.java.net/browse/JDK-8252532 webrev: http://cr.openjdk.java.net/~iignatyev//8252532/webrev.00 Thanks, -- Igor From thomas.stuefe at gmail.com Sat Aug 29 09:16:44 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Sat, 29 Aug 2020 11:16:44 +0200 Subject: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) In-Reply-To: References: <022e6096-a388-dc79-f959-00a2d933ae89@oracle.com> <1d5921d1-1cb4-b286-a95b-7e2d4b60ced0@oracle.com> Message-ID: On Fri, Aug 28, 2020 at 11:23 PM Doerr, Martin wrote: > Hi Gerard, > > > > I just gave it a quick spin. > > > > Fix for os_aix_ppc.cpp is trivial (see below). > > > > signals_posix.cpp takes more effort. I can see that a lot of AIX specific > code is for as400 PASE support which we never contributed to OpenJDK. > > Maybe we can remove it and use the regular posix code instead. > > @Thomas: What do you think? > > I see that we have a patch in AIX for the synchronous-error-signals-during-signal-handling-issue which would make sense for all platforms. I opened issue https://bugs.openjdk.java.net/browse/JDK-8252533 to track that. We already deal with this (see https://bugs.openjdk.java.net/browse/JDK-8065895, one of my first contributions btw :) but that first fix leaves some windows open which should be closed. I see Gerard already reviewed the original patch :) I'd propose to ignore this AIX-specific patch, just remove it for the time being and treat AIX like every other posix platform. Then, re-evaluate https://bugs.openjdk.java.net/browse/JDK-825253 after this cleanup as a separate issue. @Gerard: If you want to take over https://bugs.openjdk.java.net/browse/JDK-825253 this would be fine by me. Other than that I cannot find much AIX/pase specifics in the original code. @Martin: Have I missed something? Cheers, Thomas > > > Link which works correctly (version below links to pre1): > > http://cr.openjdk.java.net/~gziemski/8252324_pre2/ > > > > Best regards, > > Martin > > > > > > diff -r 39c430bf9bb3 src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp > > --- a/src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp Fri Aug 28 22:59:25 2020 > +0200 > > +++ b/src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp Fri Aug 28 23:13:25 2020 > +0200 > > @@ -51,6 +51,7 @@ > > #include "runtime/stubRoutines.hpp" > > #include "runtime/thread.inline.hpp" > > #include "runtime/timer.hpp" > > +#include "signals_posix.hpp" > > #include "utilities/events.hpp" > > #include "utilities/vmError.hpp" > > #ifdef COMPILER1 > > @@ -225,7 +226,7 @@ > > > > JavaThread* thread = NULL; > > VMThread* vmthread = NULL; > > - if (os::Aix::signal_handlers_are_installed) { > > + if (os::signal_handlers_are_installed) { > > if (t != NULL) { > > if(t->is_Java_thread()) { > > thread = (JavaThread*)t; > > > > > > > > > > *From:* gerard ziemski > *Sent:* Freitag, 28. August 2020 22:28 > *To:* Thomas St?fe ; Doerr, Martin < > martin.doerr at sap.com> > *Cc:* hotspot-runtime-dev at openjdk.java.net; Schmidt, Lutz < > lutz.schmidt at sap.com>; Lindenmaier, Goetz ; > Stuefe, Thomas > *Subject:* Re: Attention AIX developers - factoring out POSIX signal code > (JDK-8252324) > > > > hi Thomas, > > Thank you for your offer to test the changes. I know that you are > currently busy with your metaspace changes, but when you do you have some > time, then I have made a preliminary changes for AIX here to test: > http://cr.openjdk.java.net/~gziemski/8252324_pre2 > > > > cheers > > On 8/27/20 4:48 PM, Thomas St?fe wrote: > > Hi Gerard, Martin, > > > > I like the direction this patch is going, this is definitely a good > cleanup! Also thanks from me for not wanting to break AIX :) > > > > Wrt to signals AIX should not be that different from other Posix > platforms. Any differences probably stem from the fact that the AIX port > had a long time brewing in our proprietary code base before we contributed > it, with not much sync points with other ports after the initial fork. > (Memory management in AIX is different for a number of reasons, but signal > stuff should be pretty standard). > > > > There is this small issue that we use SIGTRAP for implicit null checks > (@Martin: is this still a thing?) but strictly speaking this is a ppc > issue, not an AIX issue, so it hits Linux too (@Martin: there seem to be > small diffs between how sigtrap is handled in AIX and Linux, see e.g. the > use of UseSigTrap, is that by design?). But all that should do is to add > another signal to handle/unblock. > > > > I would say, give it your best shot and dry-code the changes. When in > doubt, just ask. We will test your patch and correct if necessary. > > > > --- > > > > Some more general remarks to your preliminary changes: > > > > I would really like it if we could start documenting APIs. I am always > astonished that this is so seldom done. Ironically, almost all existing > API comments in os_posix.hpp were contributed by SAP :-) > > > > Since you now move all APIs to a central location, and it's all fresh in > your head, maybe this would be a good point to add comments to them, > describing behavior and input/output params. > > > > --- > > > > I quite liked the os::Posix namespace and think the signal functions are > well placed there. But I see what you want, you want to separate the signal > functions from the os namespace since that one requires including os.hpp? > Unfortunately this also makes the diff more complicated since a lot of code > just moved. > > > > Side note, I wish we would start using real C++ namespaces, we could make > os a namespace and hence split it over as many header files as we want. > Maybe something for future cleanups. > > > > --- > > > > ucontext_(s|g)et_pc() have not much to do with signals, even though they > are only used for signal continuation currently. They seem misplaced in > PosixSignals. > > > > --- > > > > Thanks, Thomas > > > > > > On Thu, Aug 27, 2020 at 10:33 PM Doerr, Martin > wrote: > > Hi Gerard, > > > I don?t want to straight break AIX port unexpectedly. > Thanks for that! We highly appreciate that you take care of it. > > It?s not surprising that adapting the AIX code is difficult. > > I?ve added Thomas to Cc. He?s surely the most experienced person for these > AIX specific things. > Unfortunately, he?s currently very busy because he?s contributing his new > metaspace implementation. > > We have to discuss how we can help. > > Best regards, > Martin > > > > From: gerard ziemski > Sent: Donnerstag, 27. August 2020 19:19 > To: hotspot-runtime-dev at openjdk.java.net > Cc: Schmidt, Lutz ; Lindenmaier, Goetz < > goetz.lindenmaier at sap.com>; Doerr, Martin > Subject: Attention AIX developers - factoring out POSIX signal code > (JDK-8252324) > > > hi all, > > I?m hoping I have included the right people in this email, but everyone is > welcome to provide feedback. > > I have a code for JDK-8252324, which factors out the common POSIX signal > related code out into common file, so that all platforms can benefit from > fixes that apply to all POSIX platforms (like JDK-82514380) > > The preliminary webrev can be found here: > http://cr.openjdk.java.net/~gziemski/8252324_pre1 > > > I tried to port the changes to AIX myself, but got bogged down by more > "#ifdef AIX? than I was hoping for (possibly too many differences to factor > out the code for AIX, but frankly I don't think so), so I stopped (though I > probably was on the right track) - to me it looks like AIX has some extra > fixes and differences in the signal code than BSD/Linux. > > My question is: given my inability to test on AIX, what can I do to make > it easier for AIX developers to adopt this fix? I will try porting my > changes again, but in the meantime if anyone has any pointers or offer of > other help, I?d appreciate it. I don?t want to straight break AIX port > unexpectedly. > > Reference: https://bugs.openjdk.java.net/browse/JDK-8252324 > > Some notes about the affected APIs: > > > -------------------------------------------------------------------------------------------------- > The APIs which moved from os/bsd/os_bsd.cpp (similarly for linux) to to > os/posix/PosixSignals.cpp: > > > //////////////////////////////////////////////////////////////////////////////// > // signal support > void os::Bsd::signal_sets_init() > sigset_t* os::Bsd::unblocked_signals() > sigset_t* os::Bsd::vm_signals() > void os::Bsd::hotspot_sigmask(Thread* thread) > > //////////////////////////////////////////////////////////////////////////////// > // sun.misc.Signal support > static void UserHandler(int sig, void *siginfo, void *context) > void* os::user_handler() > void* os::signal(int signal_number, void* handler) > void os::signal_raise(int signal_number) > int os::sigexitnum_pd() > static void jdk_misc_signal_init() > void os::signal_notify(int sig) > static int check_pending_signals() > int os::signal_wait() > > //////////////////////////////////////////////////////////////////////////////// > // suspend/resume support > static void resume_clear_context(OSThread *osthread) > static void suspend_save_context(OSThread *osthread, siginfo_t* siginfo, > ucontext_t* context) > static void SR_handler(int sig, siginfo_t* siginfo, ucontext_t* context) > static int SR_initialize() > static int sr_notify(OSThread* osthread) > static bool do_suspend(OSThread* osthread) > static void do_resume(OSThread* osthread) > > /////////////////////////////////////////////////////////////////////////////////// > // signal handling (except suspend/resume) > static void signalHandler(int sig, siginfo_t* info, void* uc) > struct sigaction* os::Bsd::get_chained_signal_action(int sig) > static bool call_chained_handler(struct sigaction *actp, int sig, > siginfo_t *siginfo, void *context) > bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* context) > int os::Bsd::get_our_sigflags(int sig) > void os::Bsd::set_our_sigflags(int sig, int flags) > void os::Bsd::set_signal_handler(int sig, bool set_installed) > void os::Bsd::install_signal_handlers() > static const char* get_signal_handler_name(address handler, > char* buf, int buflen) > static void print_signal_handler(outputStream* st, int sig, > char* buf, size_t buflen) > void os::run_periodic_checks() > void os::Bsd::check_signal_handler(int sig) > > > ----------------------------------------------------------------------------- > The APIs which moved from os/posix/os_posix.cpp to > os/posix/PosixSignals.cpp: > > const char* os::Posix::get_signal_name(int sig, char* out, size_t outlen) > int os::Posix::get_signal_number(const char* signal_name) > int os::get_signal_number(const char* signal_name) > bool os::Posix::is_valid_signal(int sig) > bool os::Posix::is_sig_ignored(int sig) > const char* os::exception_name(int sig, char* buf, size_t size) > const char* os::Posix::describe_signal_set_short(const sigset_t* set, > char* buffer, size_t buf_size) > void os::Posix::print_signal_set_short(outputStream* st, const sigset_t* > set) > const char* os::Posix::describe_sa_flags(int flags, char* buffer, size_t > size) > oid os::Posix::print_sa_flags(outputStream* st, int flags) > static bool get_signal_code_description(const siginfo_t* si, > enum_sigcode_desc_t* out) > void os::print_siginfo(outputStream* os, const void* si0) > bool os::signal_thread(Thread* thread, int sig, const char* reason) > int os::Posix::unblock_thread_signal_mask(const sigset_t *set) > address os::Posix::ucontext_get_pc(const ucontext_t* ctx) > void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) > struct sigaction* os::Posix::get_preinstalled_handler(int sig) > void os::Posix::save_preinstalled_handler(int sig, struct sigaction& > oldAct) > > > > DETAILS: > > -------------------------------------------------------- > Public APIs which are now internal static PosixSignals:: > > sigset_t* os::Bsd::vm_signals() > struct sigaction* os::Bsd::get_chained_signal_action(int sig) > int os::Bsd::get_our_sigflags(int sig) > void os::Bsd::set_our_sigflags(int sig, int flags) > void os::Bsd::set_signal_handler(int sig, bool set_installed) > void os::Bsd::check_signal_handler(int sig) > const char* os::Posix::get_signal_name(int sig, char* out, size_t outlen) > bool os::Posix::is_valid_signal(int sig) > const char* os::Posix::describe_signal_set_short(const sigset_t* set, > char* buffer, size_t buf_size) > void os::Posix::print_signal_set_short(outputStream* st, const sigset_t* > set) > const char* os::Posix::describe_sa_flags(int flags, char* buffer, size_t > size) > oid os::Posix::print_sa_flags(outputStream* st, int flags) > static bool get_signal_code_description(const siginfo_t* si, > enum_sigcode_desc_t* out) > void os::Posix::save_preinstalled_handler(int sig, struct sigaction& > oldAct) > > ------------------------------------------------ > Public APIs which moved to public PosixSignals:: > > void os::Bsd::signal_sets_init() > void os::Bsd::hotspot_sigmask(Thread* thread) > bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* context) > void os::Bsd::install_signal_handlers() > bool os::Posix::is_sig_ignored(int sig) > int os::Posix::unblock_thread_signal_mask(const sigset_t *set) > address os::Posix::ucontext_get_pc(const ucontext_t* ctx) > void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) > > ---------------------------------------------------- > Internal APIs which are now public in PosixSignals:: > > static void jdk_misc_signal_init() > static int SR_initialize() > static bool do_suspend(OSThread* osthread) > static void do_resume(OSThread* osthread) > static void print_signal_handler(outputStream* st, int sig, > char* buf, size_t buflen) > > -------------------------- > New APIs in PosixSignals:: > > static bool are_signal_handlers_installed(); > > > cheers > > > From daniel.daugherty at oracle.com Sun Aug 30 16:18:26 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Sun, 30 Aug 2020 12:18:26 -0400 Subject: URGENT RFR(T): 8252551: JDK-8250630 causes build error on Win* Message-ID: <45f93bc0-81db-7395-ea44-84208f39e9b6@oracle.com> Greetings, I have a trivial fix for the following build breaking bug: ??? JDK-8252551 JDK-8250630 causes build error on Win* ??? https://bugs.openjdk.java.net/browse/JDK-8252551 Here's the context diffs for the fix: $ hg diff diff -r d7707af10c98 src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c Sun Aug 30 15:48:16 2020 +0300 +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c Sun Aug 30 11:37:39 2020 -0400 @@ -716,7 +716,6 @@ ??????????????????????????????? char** actualAddress) ?{ ???? int err; -??? int pass; ???? struct addrinfo *addrInfo = NULL; ???? struct addrinfo *listenAddr = NULL; ???? struct addrinfo *ai = NULL; The fix is being tested by a Mach5 Tier3 job set. Thanks, in advance, for any comments, questions or suggestions. Dan From dms at samersoff.net Sun Aug 30 16:29:59 2020 From: dms at samersoff.net (Dmitry Samersoff) Date: Sun, 30 Aug 2020 19:29:59 +0300 Subject: URGENT RFR(T): 8252551: JDK-8250630 causes build error on Win* In-Reply-To: <45f93bc0-81db-7395-ea44-84208f39e9b6@oracle.com> References: <45f93bc0-81db-7395-ea44-84208f39e9b6@oracle.com> Message-ID: <2595895b-6f68-04ea-7bfd-32ba04d413b5@samersoff.net> Hello Dan, Looks good to me. Thank you for fixing it. -Dmitry On 8/30/2020 7:18 PM, Daniel D. Daugherty wrote: > Greetings, > > I have a trivial fix for the following build breaking bug: > > ??? JDK-8252551 JDK-8250630 causes build error on Win* > ??? https://bugs.openjdk.java.net/browse/JDK-8252551 > > Here's the context diffs for the fix: > > $ hg diff > diff -r d7707af10c98 > src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c > --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c Sun > Aug 30 15:48:16 2020 +0300 > +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c Sun > Aug 30 11:37:39 2020 -0400 > @@ -716,7 +716,6 @@ > ??????????????????????????????? char** actualAddress) > ?{ > ???? int err; > -??? int pass; > ???? struct addrinfo *addrInfo = NULL; > ???? struct addrinfo *listenAddr = NULL; > ???? struct addrinfo *ai = NULL; > > > The fix is being tested by a Mach5 Tier3 job set. > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > From daniel.daugherty at oracle.com Sun Aug 30 16:46:40 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Sun, 30 Aug 2020 12:46:40 -0400 Subject: URGENT RFR(T): 8252551: JDK-8250630 causes build error on Win* In-Reply-To: <2595895b-6f68-04ea-7bfd-32ba04d413b5@samersoff.net> References: <45f93bc0-81db-7395-ea44-84208f39e9b6@oracle.com> <2595895b-6f68-04ea-7bfd-32ba04d413b5@samersoff.net> Message-ID: <0dc3c267-f2b6-08ba-65b9-b7d81e658cc0@oracle.com> Thanks for the fast review! Pushed. Dan On 8/30/20 12:29 PM, Dmitry Samersoff wrote: > Hello Dan, > > Looks good to me. > > Thank you for fixing it. > > -Dmitry > > On 8/30/2020 7:18 PM, Daniel D. Daugherty wrote: >> Greetings, >> >> I have a trivial fix for the following build breaking bug: >> >> ???? JDK-8252551 JDK-8250630 causes build error on Win* >> ???? https://bugs.openjdk.java.net/browse/JDK-8252551 >> >> Here's the context diffs for the fix: >> >> $ hg diff >> diff -r d7707af10c98 >> src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >> --- a/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >> Sun Aug 30 15:48:16 2020 +0300 >> +++ b/src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c >> Sun Aug 30 11:37:39 2020 -0400 >> @@ -716,7 +716,6 @@ >> ???????????????????????????????? char** actualAddress) >> ??{ >> ????? int err; >> -??? int pass; >> ????? struct addrinfo *addrInfo = NULL; >> ????? struct addrinfo *listenAddr = NULL; >> ????? struct addrinfo *ai = NULL; >> >> >> The fix is being tested by a Mach5 Tier3 job set. >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> > From david.holmes at oracle.com Mon Aug 31 02:33:43 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 31 Aug 2020 12:33:43 +1000 Subject: Fatal errors when running JCK tests with JDK15/16 debug build In-Reply-To: References: Message-ID: Hi Martin, On 29/08/2020 3:53 am, Doerr, Martin wrote: > Hi, > > we have seen the following fatal error more than 50 times since > 2020-05-25 in various JCK tests vm/jvmti. > > fatal error: String conversion failure: [check] ExitLock destroyed > > --> ?? [check] ExitLock exited > > (followed by garbage output) > > 8166358: Re-enable String verification in > java_lang_String::create_from_str() > > was pushed at that date which introduced the call to fatal. > > Stack (example from linuxppc64le, but also observed on x86 and aarch64): > V? [libjvm.so+0xee242c]? java_lang_String::create_from_str(char const*, > Thread*) [clone .part.158]+0x51c > V? [libjvm.so+0xee2530]? java_lang_String::create_oop_from_str(char > const*, Thread*)+0x40 > V? [libjvm.so+0x1026a30]? jni_NewStringUTF+0x1e0 > C? [libjckjvmti.so+0x3ce4c]? logWrite+0x5c > C? [libjckjvmti.so+0x3cd20]? lprintf+0x170 > C? [libjckjvmti.so+0x485b8]? gast00104_agent_proc+0x254 > V? [libjvm.so+0x1218f0c]? JvmtiAgentThread::call_start_function()+0x24c > V? [libjvm.so+0x193a8fc]? JavaThread::thread_main_inner()+0x32c > V? [libjvm.so+0x19418a0]? Thread::call_run()+0x160 > V? [libjvm.so+0x15c9d0c]? thread_native_entry(Thread*)+0x18c > C? [libpthread.so.0+0x9b48]? start_thread+0x108 > > (Problem could have been there before but without this fatal message.) > > The messages are generated by: > > tests/vm/jvmti/GetAllStackTraces/gast001/gast00104/gast00104.c > > This looks like a race condition. The message changes while the VM > creates a String object from it. Has anybody seen this before? No but ... > Is it a test problem? I?m not familiar with the lprintf calls in the test. ... the lprintf is part of the JCK support library (support.c if you have access to sources) and it uses a static buffer for the log messages and so it not thread-safe. This test creates a thread and both it and the main thread call lprintf concurrently. So this is a JCK test/test-library bug that appears to be exposed by the changes made in 8166358. Cheers, David ----- > Best regards, > > Martin > From david.holmes at oracle.com Mon Aug 31 02:44:03 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 31 Aug 2020 12:44:03 +1000 Subject: RFR: 8234930: Use MAP_JIT when allocating pages for code cache on macOS In-Reply-To: <13480ffc-6afe-c59d-e671-88abf54d8179@azul.com> References: <13480ffc-6afe-c59d-e671-88abf54d8179@azul.com> Message-ID: Hi Anton, cc'ing hotspot-gc-dev as a number of GC files are touched, even though in a trivial way. It also broadens the scope for people who understand the detailed nuances of these mapping flags. Cheers, David On 28/08/2020 11:57 pm, Anton Kozlov wrote: > Hi, > > Could you please review a change for > > JDK-8234930: Use MAP_JIT when allocating pages for code cache on macOS > Bug: https://bugs.openjdk.java.net/browse/JDK-8234930 > Webrev: http://cr.openjdk.java.net/~akozlov/8234930/webrev.00/ > > On macOS, MAP_JIT cannot be used with MAP_FIXED[1]. So pd_reserve_memory have > to provide MAP_JIT for mmap(NULL, PROT_NONE), the function was made aware of > exec permissions. > > For executable and data regions, pd_commit_memory only unlocks the memory with > mprotect, this should make no difference compared with old code. > > For data regions, pd_uncommit_memory still uses a new overlapping anonymous > mmap which returns pages to the OS and immediately reflects this in diagnostic > tools like ps. For executable regions it would require MAP_FIXED|MAP_JIT, so > instead madvise(MADV_FREE)+mprotect(PROT_NONE) are used. They should also allow > OS to reclaim pages, but apparently this does not happen immediately. In > practice, it should not be a problem for executable regions, as codecache does > not shrink (if I haven't missed anything, by the implementation and in > principle). > > Tested: > * local tier1 > * jdk-submit > * codesign[2] with hardened runtime and allow-jit but without > allow-unsigned-executable-memory entitlements[3] produce a working bundle. > > Thanks, > Anton > > [1] https://github.com/apple/darwin-xnu/blob/master/bsd/kern/kern_mman.c#L227 > [2] > codesign \ > --sign - \ > --options runtime \ > --entitlements ents.plist \ > --timestamp \ > $J/bin/* $J/lib/server/*.dylib $J/lib/*.dylib > [3] > > > > > com.apple.security.cs.allow-jit > > com.apple.security.cs.disable-library-validation > > com.apple.security.cs.allow-dyld-environment-variables > > > > > > > From david.holmes at oracle.com Mon Aug 31 04:53:09 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 31 Aug 2020 14:53:09 +1000 Subject: RFR(T) : 8252532 : use Utils.TEST_NATIVE_PATH instead of System.getProperty("test.nativepath") In-Reply-To: References: Message-ID: <93076ed6-f112-dd27-15d3-13f67cdf5de0@oracle.com> Hi Igor, On 29/08/2020 1:52 pm, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8252532/webrev.00 >> 145 lines changed: 28 ins; 22 del; 95 mod; > > > Hi all, > > could you please review this trivial clean up which replaces System.getProperty("test.nativepath") w/ Utils.TEST_NATIVE_PATH where appropriate? > > while updating these files, I've also cleaned them up a bit, removed unneeded imports, added/removed spaces, etc > > testing: runtime, serviceability and vmTestbase/nsk/jvmti/ tests on {linux,windows,macos}-x64 > JBS: https://bugs.openjdk.java.net/browse/JDK-8252532 > webrev: http://cr.openjdk.java.net/~iignatyev//8252532/webrev.00 Generally seems fine (though the fact the patch file contained a series of changesets threw me initially!) test/hotspot/jtreg/runtime/signal/SigTestDriver.java // add test specific arguments w/o signame cmd.addAll(Arrays.asList(args) - .subList(1, args.length)); + .subList(1, args.length)); Your changed line doesn't have the right indent. Can this just be put on one line anyway: // add test specific arguments w/o signame cmd.addAll(Arrays.asList(args).subList(1, args.length)); that seems better to me as the fact there is only one argument seems clearer. Though for greater clarity perhaps: // add test specific arguments w/o signame var argList = Arrays.asList(args).subList(1, args.length); cmd.addAll(argList); -- + Arrays.stream(Utils.JAVA_OPTIONS.split(" "))) + .filter(s -> !s.isEmpty()) + .filter(s -> s.startsWith("-X")) + .flatMap(arg -> Stream.of("-vmopt", arg)) + .collect(Collectors.toList()); The preferred/common style for chained stream operations is to align the dots: Arrays.stream(Utils.JAVA_OPTIONS.split(" "))) .filter(s -> !s.isEmpty()) .filter(s -> s.startsWith("-X")) .flatMap(arg -> Stream.of("-vmopt", arg)) .collect(Collectors.toList()); --- test/lib/jdk/test/lib/process/ProcessTools.java - System.out.println("\t" + t + - " stack: (length = " + stack.length + ")"); + System.out.println("\t" + t + + " stack: (length = " + stack.length + ")"); The original code is more stylistically correct - when breaking arguments across lines the indent should align with the start of the arguments. Similarly here: + return String.format("--- ProcessLog ---%n" + + "cmd: %s%n" + + "exitvalue: %s%n" + + "stderr: %s%n" + + "stdout: %s%n", + getCommandLine(pb), exitValue, stderr, stdout); should be: + return String.format("--- ProcessLog ---%n" + + "cmd: %s%n" + + "exitvalue: %s%n" + + "stderr: %s%n" + + "stdout: %s%n", + getCommandLine(pb), exitValue, stderr, stdout); and here: + String executable = Paths.get(Utils.TEST_NATIVE_PATH, executableName) + .toAbsolutePath() + .toString(); indentation again. Thanks, David ----- > Thanks, > -- Igor > From thomas.schatzl at oracle.com Mon Aug 31 13:07:12 2020 From: thomas.schatzl at oracle.com (Thomas Schatzl) Date: Mon, 31 Aug 2020 15:07:12 +0200 Subject: RFR: 8251850: Simplify ResourceMark constructors using delegation In-Reply-To: <47D913FE-00BE-41D6-B2F1-0AC79151CBB2@oracle.com> References: <3D5CCF0C-4B3B-4C44-A262-0DA062E56BC1@oracle.com> <8B1940B1-31F1-45F5-A2BB-54DAD8406BA9@oracle.com> <05824ffe-96b6-1dea-663f-9e8b11ac2d3e@oracle.com> <77F6BD2A-DFE1-4676-AD31-55D33C658825@oracle.com> <47D913FE-00BE-41D6-B2F1-0AC79151CBB2@oracle.com> Message-ID: <6404749e-8ccf-1aea-8d7f-7d0daf29fa72@oracle.com> Hi, On 25.08.20 04:30, Kim Barrett wrote: >> On Aug 22, 2020, at 2:13 AM, Thomas St?fe wrote: >> Good. >> [...] > Good point. I've tweaked the code and the comment. > > New webrevs: > full: https://cr.openjdk.java.net/~kbarrett/8251850/open.03/ > incr: https://cr.openjdk.java.net/~kbarrett/8251850/open.03.inc/ > > Reran the ErrorHandler.java test locally. > >> I am fine with this change. I leave if up to you if you change the comment or not. >> >> Cheers, Thomas > > Thanks, but since I (lightly) touched the code again? > still good. Thomas(2) From lois.foltan at oracle.com Mon Aug 31 13:10:45 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Mon, 31 Aug 2020 09:10:45 -0400 Subject: RFR(S) 8252481 Remove excessive include of memTracker.hpp In-Reply-To: <1eda87bd-c398-ed0e-d6e4-63f2a1f88baf@oracle.com> References: <1eda87bd-c398-ed0e-d6e4-63f2a1f88baf@oracle.com> Message-ID: Looks good Ioi! Lois On 8/28/2020 1:21 AM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8252481 > http://cr.openjdk.java.net/~iklam/jdk16/8252481-excessive-memTracker-hpp.v01/ > > > memTracker.hpp is included by 847 out of 971 .o files in the HotSpot > build. > > Most of these are caused by these header files: > > jfrAllocation.hpp -- completely unnecessary > > allocation.hpp, allocation.inline.hpp -- the use of CURRENT_PC can be > avoid by calling this function instead: > > char* AllocateHeap(size_t size, > ?????????????????? MEMFLAGS flags, > ?????????????????? AllocFailType alloc_failmode /* = > AllocFailStrategy::EXIT_OOM*/) { > ? return AllocateHeap(size, flags, CALLER_PC); > } > > > ====================== > > Before: > > $ cd build/linux-x64 > $ find hotspot -name \*.o | wc -l > 971 > $ find hotspot -name \*.d | xargs grep -l memTracker.hpp | wc -l > 847 > $ find hotspot -name \*.d | xargs cat | wc -l > 528505 > > After: > > $ find hotspot -name \*.d | xargs grep -l memTracker.hpp | wc -l > 95 > $ find hotspot -name \*.d | xargs cat | wc -l > 512731 > > The total number of included header files has reduced for about 3% > > 512731 / 528505 = 0.970 > > ====================== > > Testing -- I am running mach5 tiers 1-4. From coleen.phillimore at oracle.com Mon Aug 31 13:16:56 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 31 Aug 2020 09:16:56 -0400 Subject: RFR (S) 8230052: MLVM findDeadlock test timed out In-Reply-To: <51a0d6f8-5bab-3615-a5a0-2a52e8d6b58b@oracle.com> References: <4fddcde2-60b1-b53f-a13c-1951e9cae1ab@oracle.com> <13b41959-00fb-a647-dc15-d424246e13f4@oracle.com> <51a0d6f8-5bab-3615-a5a0-2a52e8d6b58b@oracle.com> Message-ID: <7d01433e-a9f5-a18b-da1f-9fd1c842ad36@oracle.com> I've looked at the core file some more in the Serviceability Agent and honestly have no idea if this logging would be helpful, but seeing the interaction between GC and the safepoint to FindDeadlocks might be useful.? So I'll push this change for now.? I'm sure this bug will be back and somebody's going to have to figure out how to test this without ambiguous timeouts. Coleen On 8/28/20 9:55 PM, David Holmes wrote: > On 29/08/2020 5:18 am, Coleen Phillimore wrote: >> >> Continuing the discussion in the bug, I've taken out the System.gc() >> and added -Xlog:gc+safepoint to see if these give any ideas why the >> test times out. See: > > So no longer trying to fix it just trying to get more diagnostics. But > how much logging info is that going to produce? Are we just going to > hit the jtreg buffer limit? > > I would have gone for some coarser higher-level "logging" initially. > It is going to be hard to correlate the log entries with what is > happening at the test level without output from the test itself that > will appear in-order with the system logging. So I'm not at all sure > this logging will really help. > > But lets give it a try. > > David > >> bug link https://bugs.openjdk.java.net/browse/JDK-8230052 >> open webrev at >> http://cr.openjdk.java.net/~coleenp/2020/8230052.02/webrev >> >> thanks, >> Coleen >> >> >> On 8/28/20 8:22 AM, Patricio Chilano wrote: >>> Hi Coleen, >>> >>> On 8/27/20 7:53 PM, Coleen Phillimore wrote: >>>> Summary: Add a System.gc() and reduce iteration count for more >>>> stability. >>>> >>>> open webrev at >>>> http://cr.openjdk.java.net/~coleenp/2020/8230052.01/webrev >>>> bug link https://bugs.openjdk.java.net/browse/JDK-8230052 >>> Looks good to me. >>> If we see any new failure I think we can try to run the test with >>> extra VM logging enabled. >>> >>> >>> Thanks, >>> Patricio >>>> Ran test 100x with < 1minute run time. Should I add a timeout of 10 >>>> minutes also? >>>> >>>> Thanks, >>>> Coleen >>> >> From ioi.lam at oracle.com Mon Aug 31 15:31:36 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 31 Aug 2020 08:31:36 -0700 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: <234856fa-1eb1-8514-6786-bce6689afd16@oracle.com> References: <9e6b0043-65a1-dd97-a3d2-33679c8048d4@oracle.com> <234856fa-1eb1-8514-6786-bce6689afd16@oracle.com> Message-ID: Hi Lois, Thanks for the review. Your comments has led me to discover a couple of pretty serious issues, which hopefully I have fixed. Also the code is cleaner now, with much fewer control-flow changes than the last webrev. http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v03/ http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v03.delta/ Please see my comments in-line. On 8/25/20 7:58 AM, Lois Foltan wrote: > > Hi Ioi, > > Changes looks really good.? Comments interspersed below. > > Thanks, > Lois > > On 8/12/2020 6:06 PM, Ioi Lam wrote: >> Hi Lois, >> >> Thanks for the comments. I have an updated webrev >> >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/ >> >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02.delta/ >> >> >> Here are the general notes on the changes. Replies to your questions >> are in-line below. >> >> (1) Integrated updates in the Java code from Alan Bateman. Here are >> Alan's >> ? ? notes: >> >> ??? The archive of the boot layer is as before, except that archiving is >> ??? skipped if there are split packages or incubator modules. Incubating >> ??? modules aren't resolved by default so we shouldn't have them in the >> ??? boot layer by default anyway. >> >> ??? I've dropped the module finders from the boot layer archive. I've >> ??? left the IllegalAccessLogger.Builder in the acrhive for now (even >> ??? though it is not the boot layer). We should be able to remove that >> ??? once the JEP to disallow illegal access by default is in. >> >> ??? Related is that I don't like the archive for the module graph >> ??? (ArchivedModuleGraph, pre-dates this RFE) including the set of >> ??? packages to export/open for illegal access as they aren't part >> ??? of the module graph. I've left it for now but we can also remove >> ??? that once we disallow illegal access by default (as those sets >> ??? will be empty). >> >> ??? The archive of built-in class loaders is now in one object >> ??? jdk.internal.loader.ArchivedClassLoaders which I think will be a >> ??? bit more maintainable. I've also drop the ucp field from the >> ??? AppClassLoader as the changes to BuiltinClassLoader means is no >> ??? longer needs to duplicated. >> >> ??? There's one remaining issue in ModuleBootstrap.boot where it has fix >> ??? an app class loader value (ModuleLayer.CLV). Ideally the >> initialization >> ??? of the built-in class loaders would do this but we are kinda forced >> ??? to separate the archiving of the built-in class loaders from the >> boot >> ??? layer. I might look at this again some time. >> >> >> (2) Moved code from classLoaderData.cpp -> classLoaderSharedData.cpp >> >> (3) Reverted unnecessary changes in JavaClasses::compute_offset >> >> (4) Minor clean up to use QuickSort::sort() instead of qsort() >> >> (5) Moved the C-side of module initialization for platform/system >> ? ? loaders to inside java.lang.System::initPhase2(), so this happens >> ??? at the same time as without full module archiving. >> >> (6) Moved the use of Module_lock to so all modules in a class loader >> ??? are restored atomically. See ArchivedClassLoaderData::restore() >> >> ??? This fixed a bug where test/jdk/com/sun/jdi/ModulesTest.java >> ? ? would fail as it sees a partially restored set of modules. >> >> >> >> On 8/7/20 12:06 PM, Lois Foltan wrote: >>> Hi Ioi, >>> >>> Overall looks promising.? I have some review comments below, but not >>> a complete review.? I concentrated on the JVM side primarily how the >>> archived module graph is read in, how the ModuleEntry and >>> PackageEntry tables are created from the archive for the 3 builtin >>> loaders and potential timing issues during start up. >>> >>> On 7/22/2020 3:36 PM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8244778 >>>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/ >>>> >>>> >>>> Please review this patch that stores the full module graph in the CDS >>>> archive heap. This reduces the initialization time of the basic JVM by >>>> about 22%: >>>> >>>> $ perf stat -r 100 bin/java -version >>>> before: 98,219,329 instructions 0.03971 secs elapsed (+- 0.44%) >>>> after:? 55,835,815 instructions 0.03109 secs elapsed (+- 0.65%) >>>> >>>> [1] Start with ModuleBootstrap.java. The current implementation is >>>> ??? quite restrictive: the archived module graph is used only when no >>>> ??? module options are specified. >>>> >>>> ??? See ModuleBootstrap.mayUseArchivedBootLayer(). >>>> >>>> ??? We can probably support options such as main module and module >>>> path >>>> ??? in a future RFE. >>> >>> Yes, I noticed the restrictions.? The JBS issue discusses the >>> possibility of using the dumped module graph in the event that the >>> value of the options are the same.? I think for this implementation >>> to be viable and have a positive impact you should consider >>> comparing at least the options --add-modules, --add-exports and >>> --add-reads. >> >> I agree. I want to keep the changes minimal in this RFE, and will add >> more support for other use cases in follow-on RFEs. Instead of >> requiring these options to be unspecified, we can relax the >> restriction such that these options must be the same between archive >> dump time and run time. > > Sounds good! > >> >>> >>> >>>> >>>> [2] In the current JDK implementation, there is no single object >>>> ??? that represents "the module graph". Most of the information >>>> ??? is stored in the archive bootLayer object, but a few additional >>>> ??? restoration operations need to be performed: >>>> >>>> ??? + See ModuleBootstrap.getArchivedBootLayer() >>>> ??? + Some static fields need to be archived/restored in >>>> ????? Module.java, BuiltinClassLoader.java, ClassLoaders.java >>>> ????? and BootLoader.java >>>> >>>> [3] I ran into a complication with two loader instances of >>>> ??? PlatformClassLoader and AppClassLoader. They are stored in >>>> ??? multiple tables inside the module graph (e.g., >>>> ??? BuiltinClassLoader$LoadedModule) so I cannot easily recreate >>>> ??? them at runtime. >>>> >>>> ??? However, these two loaders contain information specific to the >>>> ??? dump time VM lifecycle (such as the classes that were loaded >>>> ??? during CDS dumping) that need to be scrubbed. I couldn't find an >>>> ??? elegant way of doing this, so I added a private >>>> "resetArchivedStates" >>>> ??? method to a few classes. They are called inside >>>> ??? HeapShared::reset_archived_object_states(). >>>> >>>> [4] Related native data structures (PackageEntry and ModuleEntry) >>>> ??? are also archived. Start with classLoaderData.cpp >>>> >>>> Passes mach5 tiers 1-4. I will test with additional tiers. >>>> >>>> Thanks >>>> - Ioi >>> >>> classfile/classLoader.cpp >>> - line #1644 pulling the javabase_moduleEntry() check outside of the >>> Module_lock maybe problematic if after you check it another >>> ? thread initializes in the time between the check and the obtaining >>> of the Module_lock on line #1645. >> >> Fixed. > > That looks good.? I think it is fine that you are checking if > java.base is defined via a call to javabase_moduleEntry() because you > are just trying to determine if a ModuleEntry should be created or > not.? In most cases though using ModuleEntryTable::javabase_defined() > is the correct way to ensure that both the ModuleEntry for java.base > has been created and that the ModuleEntry has been injected in the > corresponding java.lang.Module oop. > >> >>> >>> classfile/classLoader.hpp >>> - somewhere in ArchivedClassLoaderData there should be an assert to >>> make sure that the CLD, whose package entries and module entries are >>> being archived is not a "_has_class_mirror_holder" CLD for a weakly >>> defined hidden class.? Those dedicated CLDs should never have >>> package entries or module entries. >>> >> >> I added ArchivedClassLoaderData::assert_valid() >> >>> classfile/moduleEntry.cpp >>> - line #400, typo "conver" --> "convert" >>> - line #500, maybe sort if n is greater than 1 instead of 0 (same >>> comment for packageEntry.cpp line #270) >>> >> Fixed >> >>> classfile/systemDictionary.cpp >>> - It looks like the PackageEntry and ModuleEntry tables for the >>> system and platform class loaders are? added within >>> SystemDictionary::compute_java_loaders() which is called from >>> Thread::create_vm() but after the call in Thread::create_vm() to >>> call_initPhase2 which is when Universe::_module_initialized is set >>> to true.? Did I read this correctly?? If yes, then I think you have >>> to be sure that Universe::_module_initialized is not set until the >>> module graph for the 3 builtin loaders is loaded from the archive. >>> >> >> I moved the code to be called by ModuleBootstrap::boot() so it should >> now happen inside call_initPhase2. > > Yes, that looks good. > >> >>> memory/filemap.cpp >>> - line #237 typo "modue" --> "module" >>> >> >> Fixed >> >>> - Since the CDS support for capturing the module graph does not >>> archive unnamed modules, I assume >>> Modules::set_bootloader_unnamed_module() is still called.? Is this >>> true or does the unnamed module for the boot loader get archived? >> The unnamed module for the boot loader is not archived. >> >> Modules::set_bootloader_unnamed_module()? wasn't called in my last >> webrev. Thanks for catching it. >> >> I added a call to BootLoader.getUnnamedModule() in >> ModuleBootstrap::boot()? to trigger of BootLoader, which >> will call into the VM for Modules::set_bootloader_unnamed_module(). > > Looks good. > >> >> >> >>> Clarification/timing questions: >>> >> >> Here's an overall problem I am facing: >> >> The module graph is archived after the module system has fully >> started up. This means that for the boot loader, we only know the >> full set of modules/packages, but we don't know which part is the >> subset that was initialized during early VM bootstraping (e.g., when >> ModuleEntryTable::javabase_defined() == false). >> >> So the behavior is a bit different during the early bootstrap phase >> (all the way before we reach ModuleBootstrap::boot()): >> ClassLoaderData::the_null_class_loader_data()->modules() and >> ClassLoaderData::the_null_class_loader_data()->packages() are already >> populated before a single class is loaded. > > If this is the case then, at the point when a ModuleEntry is created > for java.base using the archived module graph, there should be a > assertion that java_lang_Class::_fixup_module_field_list is NULL. To > confirm no class has been loaded before java.base is defined. Maybe in > ClassLoaderDataShared::serialize where you restore the boot loader's > archived modules? Thanks for pointing this out. It turned out that in the last webrev (v02), I had a bug where the module of the primitive classes were not initialized. Now I have changed the initialization code to behave the same whether archived full module graph is used or not. The only differences are: [1] ClassLoader::create_javabase(): ModuleEntryTable::javabase_moduleEntry() may be inited by CDS. [2] When archived full module graph is used, ModuleEntryTable::patch_javabase_entries is called from Modules::define_archived_modules. (java_lang_Class::_fixup_module_field_list is used within this call.) I also added a new test case: cds/PrimitiveClassMirrors.java > >> This difference doesn't seem to make a practical difference. E.g., >> none of our code seems to assume that "before any classes in the >> java.util package is loaded, >> ClassLoaderData::the_null_class_loader_data()->packages() must not >> contain an entry for java.util". >> >> I think we have two choices when the archived module graph is used: >> >> [1] We require that the state of the module system must, at every >> step of VM initialization, be identical to that of a VM that doesn't >> use an archived module graph. >> >> [2] We make sure that the VM/JDK bootstrap code can tolerate the >> existence of module/packages that are added earlier than a VM that >> doesn't use an archived module graph. >> >> I tried doing a version of [1] and found that to be too difficult. >> [2] seems much simpler and is the approach I am using now. > > I think [2] is reasonable. > >> >>> oops/instanceKlass.cpp >>> - line #2545, comment "TO DO? -- point it to the archived >>> PackageEntry (JDK-8249262)" >>> are you thinking that since the module graph is read in ahead of >>> time that it can be set when an InstanceKlass is created? There is a >>> point before java.base is defined that InstanceKlass::set_package >>> takes into account that could be a timing issue. >>> >>> >> >> I think it will work as if another class in the same package has >> already been defined. >> >>> - There are some checks in modules.cpp that are valuable to have >>> during bootstrapping.? For example, packages that are encountered >>> during bootstrapping before java.base is defined are put into the >>> java.base module but then verified after java.base is defined via >>> verify_javabase_packages.? So at what point in the bootstrapping >>> process does the boot loader's archived module's become known? Could >>> classes have been defined to the VM before this point? Then their >>> packages will have to be verified to be sure that they are indeed >>> packages within java.base.? Consider looking at other checks that >>> occur within classfile/modules.cpp as well. >>> >> As mentioned above, calling verify_javabase_packages() at run time >> will fail, as we have loaded all packages for the boot loader, not >> just those for java.base. > > I think not calling verify_javabase_packages works because as you > stated above no classes have been loaded before java.base is defined > which is not the same situation as running without the archived module > graph. > > A couple of additional comments: > > - ModuleEntry's reads list and PackageEntry's exports list.? We had > hoped eventually to change these lists from being a c heap allocated > GrowableArray over to a ResourceHashTable for faster lookup.? Doing > that as a separate RFE might help CDS archiving not to have to archive > those lists as an Array? > CDS cannot archive ResourceHashTable either. We have CompactHashtable which can be archived, but it cannot be modified at run time. I think the export list can be modified at run time with java.lang.Module::addExports(), so we probably need to do it like: - if a Module was archived, check its CompactHashtable first - if not found, check the ResourceHashTable This would make the start-up a little faster (no more copying from the array lists into the hashtable, but would make the code more complicated. I need to investigate to see if it's worthwhile. > - moduleEntry.cpp and packageEntry.cpp: Both methods > "compare_module_by_name" and "compare_package_by_name" should have an > assert if the names are equal.? No ModuleEntryTable or > PackageEntryTable should ever have 2 same named modules or packages. > I tried adding: static int compare_package_by_name(PackageEntry* a, PackageEntry* b) { ? assert(a != b && a->name() != b->name(), "array should not contain duplicated entries"); ? return a->name()->fast_compare(b->name()); } but this caused an assert, because our QuickSort implementation would sometimes compare the same element! #7? 0x00007ffff659238b in QuickSort::partition (array=0x80043fcb0, ??? pivot=248, length=496, comparator=0x7ffff6590a5a ) ??? at /jdk2/gil/open/src/hotspot/share/utilities/quickSort.hpp:76 76??? ????? for ( ; comparator(array[left_index], pivot_val) < 0; ++left_index) { (gdb) p array[left_index] $1 = (PackageEntry *) 0x7ffff0439d00 (gdb) p pivot_val $2 = (PackageEntry *) 0x7ffff0439d00 (gdb) p pivot $3 = 248 (gdb) p left_index $4 = 248 So I ended up with: static int compare_package_by_name(PackageEntry* a, PackageEntry* b) { ? assert(a == b || a->name() != b->name(), "no duplicated names"); ? return a->name()->fast_compare(b->name()); } > - Another timing clarification question for me.? I assume that the > module graph is dumped post module initialization when Java has > completely defined the module graph to the JVM, is this correct? Yes > My concern is that there could be a window post module initialization > and pre archiving the module graph where another thread could define a > new module or add Module readability or add Package exportability.? So > that the module graph you are dumping is not the same module graph at > the end of module initialization.? Is this a concern? We don't allow arbitrary code to be executed during -Xshare:dump, so the module graph shouldn't be changed after module initialization has finished. I've added Modules::check_cds_restrictions() to check for this. I also added asserts to make sure that none of the classes used by the archived module graph can be modified by JVMTI. All these classes are loaded in the "early" phase of JVMTI, and we would disable CDS if a JVMTI agent is registered to modify classes in this phase, so we should be safe, but the asserts will ensure that. I updated the ReplaceCriticalClassesForSubgraphs.java test and added a new test RedefineClassesInModuleGraph.java. Thanks - Ioi > > Thanks, > Lois > >> >> Well, verify_javabase_packages() was called once and it succeeded, >> but that was during CDS dump time. So we know at least we have >> verified this once :-) >> >> Thanks >> - Ioi >> >>> I may have more review comments as I continue to look at this! >>> >>> Thanks, >>> Lois >>> >>> >>> >>> >>> >> > > > > From ioi.lam at oracle.com Mon Aug 31 15:41:17 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 31 Aug 2020 08:41:17 -0700 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: <3234fa66-5199-32dd-1f13-e15b61c7dc63@oracle.com> References: <3234fa66-5199-32dd-1f13-e15b61c7dc63@oracle.com> Message-ID: <5c9b2f21-42c4-1409-fb6f-49dc8f98d003@oracle.com> Hi Coleen, Thanks for your comments. Please see the new version: http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v03/ http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v03.delta/ Replies in-line below On 8/25/20 1:24 PM, Coleen Phillimore wrote: > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html > > > Are these #includes still needed? > These are left-overs. I removed them. > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/classLoaderDataShared.hpp.html > > > Looks like the include guards are wrong.? Should have pushed my > #pragma once change then nobody would have to worry about these. > Fixed. > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/classLoaderDataShared.cpp.html > > > ?136 void ClassLoaderDataShared::init_archived_tables() { > ?137?? assert(DumpSharedSpaces, "must be"); > ?138?? if (MetaspaceShared::use_full_module_graph()) { > > > Shouldn't these functions be called only if > MetaspaceShared::use_full_module_graph() instead of having them > guarded entirely by these if statements. > ie. assert(DumpSharedSpaces && > MetaspaceShared::use_full_module_graph(), "must be"); > I fixed as you suggested. > Thank you for moving the new code to classLoaderDataShared.cpp > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/packageEntry.cpp.frames.html > > > Can you organize this so that the archiving functions for PackageEntry > are after the PackageEntry:: functions, and the ones for > PackageEntryTable are after the PackageEntryTable:: functions? > > They'll still be sort of mixed up, but the CDS special code in the > middle isn't good.? And if we ever want to change the entries or the > tables themselves it'll be easier. > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/moduleEntry.cpp.frames.html > > > Same comment.? The code in both looks fine, even though it looks the > same. I don't know how to generalize it without being artificial and > not resembling either.? Separating the functions will help. The code of PackageEntry and PackageEntryTable are somewhat mixed up in packageEntry.cpp. I've tried to put my new code for PackageEntry after the first block of code for PackageEntry. Same for PackageEntryTable. If we want to reorder packageEntry.cpp, maybe we should do that in a separate RFE? (Same for moduleEntry.cpp). Thanks - Ioi > > On 8/12/20 6:11 PM, Ioi Lam wrote: >> Hi Coleen, >> >> Thanks for the comments. I have an updated webrev >> >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/ >> >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02.delta/ >> >> >> >> On 8/4/20 7:49 AM, Coleen Phillimore wrote: >>> This is sort of massive, so I'll only make a couple initial comments. >>> >>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html >>> >>> >>> All this new code in ClassLoaderData should go somewhere else, like >>> maybe classLoaderDataShared.hpp/cpp ?? I don't see how it needs to >>> be owned by CLD, except maybe the serialized function. >>> >>> >> Fixed >> >>> + if (loader_data) { >>> >>> >>> There are many checks that should be != NULL here. >>> >> >> Fixed and added comments >> >> >>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/javaClasses.cpp.udiff.html >>> >>> >>> I don't see why compute_offset/s() has been exported outside >>> javaClasses.cpp. >>> >> >> It was left-over code that shouldn't be there. I removed it. >> >> >>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/moduleEntry.cpp.udiff.html >>> >>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/packageEntry.cpp.udiff.html >>> >>> >>> These look like the same code to me.? Maybe they can be generalized >>> and put together in a moduleEntryShared.cpp/hpp ? >>> >> >> They look similar but also do different things. Any suggestion for >> how to generalize them? >> >> >>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/systemDictionary.cpp.udiff.html >>> >>> >>> Why do you register the loaders here? >>> >> >> It's to force the creation of the ClassLoaderData if a loader oop >> doesn't have one yet. Also, in the new version, I moved the code to >> Modules::define_archived_modules(). > > Yes, this is a better place. > > Thanks, > Coleen >> >> >> Thanks >> - Ioi >> >>> I don't know enough about the Java code to review that. >>> >>> I assume that this going to be rebased on other changes, so this is >>> only an incomplete review. >>> >>> Coleen >>> >>> >>> On 7/22/20 3:36 PM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8244778 >>>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/ >>>> >>>> >>>> Please review this patch that stores the full module graph in the CDS >>>> archive heap. This reduces the initialization time of the basic JVM by >>>> about 22%: >>>> >>>> $ perf stat -r 100 bin/java -version >>>> before: 98,219,329 instructions 0.03971 secs elapsed (+- 0.44%) >>>> after:? 55,835,815 instructions 0.03109 secs elapsed (+- 0.65%) >>>> >>>> [1] Start with ModuleBootstrap.java. The current implementation is >>>> ??? quite restrictive: the archived module graph is used only when no >>>> ??? module options are specified. >>>> >>>> ??? See ModuleBootstrap.mayUseArchivedBootLayer(). >>>> >>>> ??? We can probably support options such as main module and module >>>> path >>>> ??? in a future RFE. >>>> >>>> [2] In the current JDK implementation, there is no single object >>>> ??? that represents "the module graph". Most of the information >>>> ??? is stored in the archive bootLayer object, but a few additional >>>> ??? restoration operations need to be performed: >>>> >>>> ??? + See ModuleBootstrap.getArchivedBootLayer() >>>> ??? + Some static fields need to be archived/restored in >>>> ????? Module.java, BuiltinClassLoader.java, ClassLoaders.java >>>> ????? and BootLoader.java >>>> >>>> [3] I ran into a complication with two loader instances of >>>> ??? PlatformClassLoader and AppClassLoader. They are stored in >>>> ??? multiple tables inside the module graph (e.g., >>>> ??? BuiltinClassLoader$LoadedModule) so I cannot easily recreate >>>> ??? them at runtime. >>>> >>>> ??? However, these two loaders contain information specific to the >>>> ??? dump time VM lifecycle (such as the classes that were loaded >>>> ??? during CDS dumping) that need to be scrubbed. I couldn't find an >>>> ??? elegant way of doing this, so I added a private >>>> "resetArchivedStates" >>>> ??? method to a few classes. They are called inside >>>> ??? HeapShared::reset_archived_object_states(). >>>> >>>> [4] Related native data structures (PackageEntry and ModuleEntry) >>>> ??? are also archived. Start with classLoaderData.cpp >>>> >>>> Passes mach5 tiers 1-4. I will test with additional tiers. >>>> >>>> Thanks >>>> - Ioi >>> >> > From calvin.cheung at oracle.com Mon Aug 31 15:57:29 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Mon, 31 Aug 2020 08:57:29 -0700 Subject: RFR(S): 8251860: ClassLoaderData::loaded_classes_do fails with "assert(ZAddress::is_marked(addr)) failed: Should be marked" In-Reply-To: <0d0bdd73-7f9d-4309-a670-4a9d81608bdf@oracle.com> References: <9217b9d3-4be7-b4f4-7be3-e4473fe18cf9@oracle.com> <1a85f6c1-76b1-61a3-c188-aaa7413ed04e@oracle.com> <0d0bdd73-7f9d-4309-a670-4a9d81608bdf@oracle.com> Message-ID: <93a76872-34f9-1462-c341-515047671280@oracle.com> Hi Ioi, Thanks for your review! On 8/28/20 3:54 PM, Ioi Lam wrote: > Hi Calvin, > > ?300 void ClassLoaderData::inc_keep_alive() { > ?301?? if (has_class_mirror_holder()) { > ?302???? if (!Arguments::is_dumping_archive()) { > ?303?????? assert(_keep_alive > 0, "Invalid keep alive increment count"); > ?304???? } > ?305???? _keep_alive++; > ?306?? } > ?307 } > > Why is 302 needed? I think in CollectCLDClosure, you wouldn't collect > any CLD whose _keep_alive count is zero. The (!cld->is_unloading()) doesn't mean that the _keep_alive count is non-zero. We currently have a few tests which would hit the assert if line 302 is not there. One of them is the appcds/dynamicArchive/LinkClassTest. While the "Parent x = new Child()" is being verified during linking, the VerificationType::resolve_and_check_assignability will be called. This will eventually call the SystemDictionary::register_loader but it will not add a class_loader to the ClassLoaderDataGraph. Prior to this patch, the _keep_alive will be incremented at 2 places: 1) java_lang_Class::set_mirror_module_field ?? In this case, the _keep_alive will be incremented during vm startup. 2) ClassLoaderData::ClassLoaderData ?? _keep_alive((has_class_mirror_holder || h_class_loader.is_null()) ? 1 : 0), Neither of the above code path will be taken for the assignability check. > > I think MetaspaceShared::link_and_cleanup_shared_classes() can be > simplified so you don't need to scan the CLDs in two passes. > > 1393? while (true) { > ????? ? bool has_linked = false; > ??????? for (int i = 0; i < _loaded_cld->length(); i++) { > ????????? ClassLoaderData* cld = _loaded_cld->at(i); > ????????? for (Klass* klass = cld->klasses(); klass != NULL; klass = > klass->next_link()) { > ???????? ?? if (klass->is_instance_klass()) { > ????????? ? ? InstanceKlass* ik = InstanceKlass::cast(ik); > ?????????? ?? if (inking_required(ik)) { > ??????????? ? ? has_linked = link_class_for_cds(ik, THREAD); > ?? ? ? ?????? } > ??????????? } > ????????? } > ??????? } > > ? ? ? ? if (!has_linked) { > ?? ? ? ?? break; > ?? ? ?? } > ??????? // Class linking includes verification which may load more > classes. > ? ? ??? // Keep scanning until we have linked no more classes. > ? ? ? } > I think the has_linked assignment should be "has_linked |= link_class_for_cds(ik, THREAD)" ? The above single loop is cleaner but I think it is less efficient. For the simple case if there's no linking required after the first loop. It will need to do another full loop to check if all classes don't need to be linked in order to exit the loop. With the 2 passes approach, it only needs to inspect the first element of the _klasses list of each CLD to determine if we can exit the loop. I'm good with the single loop since CDS dump isn't too performance critical and it makes the code easier to read. > > Also link_class_for_cds should return the value of > MetaspaceShared::try_link_class(), so it has the same behavior as before. Fixed. Here's an updated webrev: ??? http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.02/ thanks, Calvin > > The rest of the changes looks good. > > Thanks > - Ioi > > On 8/28/20 10:42 AM, calvin.cheung at oracle.com wrote: >> Hi Coleen, >> >> Thanks for your review. >> >> I've re-implemented the fix with your suggestions. >> >> Here's an updated webrev: >> >> ??? http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.01/ >> >> On 8/26/20 3:24 PM, Coleen Phillimore wrote: >>> >>> Hi Calvin, >>> >>> I think you should change this to call >>> ClassLoaderDataGraph::loaded_cld_do(&collect_cld) and collect >>> ClassLoaderData in the GrowableArray, since you then walk through >>> them again to link the classes in the class loader data. >> With the ClassLoaderDataGraph::loaded_cld_do approach, some klasses >> in the _klasses list may already in the link state. So I have to >> account for that and not link those klasses again. It complicates the >> fix a little bit. >>> >>> Can linking one klass cause a new class to be added to a _klasses >>> list of a CLD that you've already walked?? It seems like you need >>> the do() loop in the current code wrapped around walking the clds, >>> then link the classes until no progress. You can fast exit when you >>> find a klass linked, I believe, since the classes are added to the >>> front of the _klasses list. >> Yes, it is possible. The updated patch will check if the first klass >> in the _klass list of each CLD hasn't been linked, then it will loop >> back again to go through all the CLD's _klasses lists. >>> >>> Also with this change, you can remove locked_loaded_classes_do() >>> because it was added for CDS and apparently the comment is wrong. >>> >>> // This case can block but cannot do unloading (called from CDS) >>> void ClassLoaderDataGraph::unlocked_loaded_classes_do(KlassClosure* >>> klass_closure) { >>> ? for (ClassLoaderData* cld = _head; cld != NULL; cld = cld->next()) { >>> ??? cld->loaded_classes_do(klass_closure); >>> ? } >>> } >> >> I've removed the above. >> >> thanks, >> >> Calvin >> >>> >>> Thanks, >>> Coleen >>> >>> On 8/26/20 5:35 PM, calvin.cheung at oracle.com wrote: >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8251860 >>>> >>>> webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.00/ >>>> >>>> Please refer to the bug report for an analysis of the crash. >>>> >>>> The proposed fix is to acquire the ClassLoaderDataGraph_lock before >>>> calling ClassLoaderDataGraph::loaded_classes_do(&link_closure). The >>>> link_closure will not do the actual linking but just to store the >>>> InstanceKlass'es in an array and increment the keep_alive counter >>>> of the class_loader_data of each InstanceKlass. After the call to >>>> ClassLoaderDataGraph::loaded_classes_do, each InstanceKlass in the >>>> array will be linked and the keep_alive counter of the >>>> class_loader_data will be decremented. >>>> >>>> Testing: >>>> ??? - tested about 40 times on the MacOS host where the crash was seen >>>> ?????? (before the fix, the crash was seen about once in 20 runs) >>>> ??? - Tier1 - 4. >>>> thanks, >>>> Calvin >>> > From richard.reingruber at sap.com Mon Aug 31 16:05:40 2020 From: richard.reingruber at sap.com (Reingruber, Richard) Date: Mon, 31 Aug 2020 16:05:40 +0000 Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace In-Reply-To: References: Message-ID: Hi Thomas, the goals of jep387 do make sense and yet even better: the implementation lives up to them as you showed with tests and benchmarks. Thanks for putting all the effort into it! Sorry for being such a slow reviewer. I haven't even completed a full pass yet but I think it is time to share my findings so far. Thanks, Richard. ====== You added includes of memory/metaspace.hpp to some files without additional modifications. How did you chose these files? To jvmciCompilerToVM.hpp you added an include of collectedHeap.hpp. Why did you do this? ====== src/hotspot/share/memory/metaspace.cpp 194 // ... and class spacelist. 195 VirtualSpaceList* vsl = VirtualSpaceList::vslist_nonclass(); 196 assert(vsl != NULL, "Sanity"); 197 vsl->verify(slow); In 195 VirtualSpaceList::vslist_class() should be called I suppose. You could reuse the local vsl as you did with the local cm. Any reason you assert vsl not NULL in 196 but not in the non-class case? 637 // CCS must be aligned to root chunk size, and be at least the size of one 638 // root chunk. 639 adjusted_ccs_size = align_up(adjusted_ccs_size, reserve_alignment()); 640 adjusted_ccs_size = MAX2(adjusted_ccs_size, reserve_alignment()); Line 640 is redundant, isn't it? @@ -1274,7 +798,7 @@ assert(loader_data != NULL, "Should never pass around a NULL loader_data. " "ClassLoaderData::the_null_class_loader_data() should have been used."); - MetadataType mdtype = (type == MetaspaceObj::ClassType) ? ClassType : NonClassType; + Metaspace::MetadataType mdtype = (type == MetaspaceObj::ClassType) ? Metaspace::ClassType : Metaspace::NonClassType; // Try to allocate metadata. MetaWord* result = loader_data->metaspace_non_null()->allocate(word_size, mdtype); This hunk is not needed. ====== src/hotspot/share/memory/metaspace/binlist.hpp 94 // block sizes this structure can keep are limited by [_min_block_size, _max_block_size) 95 const static size_t minimal_word_size = smallest_size; 96 const static size_t maximal_word_size = minimal_word_size + num_lists; _min_block_size/_max_block_size should be minimal_word_size/maximal_word_size. The upper limit 'maximal_word_size' should be inclusive IMHO: const static size_t maximal_word_size = minimal_word_size + num_lists - 1; That would better match the meaning of the variable name. Checks in l.148 and l.162 should be adapted in case you agree. 181 182 } else { 183 184 *p_real_word_size = 0; 185 return NULL; 186 187 } Lines 181, 183, 186 can be deleted. ====== src/hotspot/share/memory/metaspace/blocktree.hpp The file would benefit from a whitespace clean-up. 43 // We store node pointer information in these blocks when storing them. That 44 // imposes a minimum size to the managed memory blocks. 45 // See MetaspaceArene::get_raw_allocation_word_size(). s/MetaspaceArene::get_raw_allocation_word_size/metaspace::get_raw_word_size_for_requested_word_size/ I agree with the comment, but metaspace::get_raw_word_size_for_requested_word_size() does not seem to take this into account. 86 // blocks with the same size are put in a list with this node as head. 89 // word size of node. Note that size cannot be larger than max metaspace size, 115 // given a node n, add it to the list starting at head 123 // given a node list starting at head, remove one node from it and return it. You should begin a sentence consistently with a capital letter (you mostly do it). 123 // given a node list starting at head, remove one node from it and return it. 124 // List must contain at least one other node. 125 static node_t* remove_from_list(node_t* head) { 126 assert(head->next != NULL, "sanity"); 127 node_t* n = head->next; 128 if (n != NULL) { 129 head->next = n->next; 130 } 131 return n; 132 } Line 129 must be executed unconditionally. I'd prefer a more generic implementation that allows head->next to be NULL. Maybe even head == NULL. 134 // Given a node n and a node p, wire up n as left child of p. 135 static void set_left_child(node_t* p, node_t* c) { 143 // Given a node n and a node p, wire up n as right child of p. 144 static void set_right_child(node_t* p, node_t* c) { Comments should use same variable names as method definition. 215 // Given a node n and a node forebear, insert n under forebear 216 void insert(node_t* forebear, node_t* n) { 217 if (n->size == forebear->size) { 218 add_to_list(n, forebear); // parent stays NULL in this case. 219 } else { 220 if (n->size < forebear->size) { 221 if (forebear->left == NULL) { 222 set_left_child(forebear, n); 223 } else { 224 insert(forebear->left, n); 225 } 226 } else { 227 assert(n->size > forebear->size, "sanity"); 228 if (forebear->right == NULL) { 229 set_right_child(forebear, n); 230 if (_largest_size_added < n->size) { 231 _largest_size_added = n->size; 232 } 233 } else { 234 insert(forebear->right, n); 235 } 236 } 237 } 238 } This assertion in line 227 is redundant (cannot fail). Leo> There are at least two recursive calls of insert that could be Leo> tail-called instead (it would be somewhat harder to read, so I am not Leo> proposing it). I think they _are_ tail-recursions in the current form. Gcc eliminates them. I checked the release build with gdb: (disass /s metaspace::FreeBlocks::add_block) Recursive tail-calls can be easily replaced with loops. To be save I'd suggest to do that or at least add 'return' after each call with a comment that nothing must be added between the call and the return too keep it a tail-recursion. Maybe that's sufficient... on the other hand we don't know if every C++ compiler can eliminate the calls and stack overflows when debugging would be also irritating. 251 return find_closest_fit(n->right, s); 260 return find_closest_fit(n->left, s); More tail-recursion. Same as above. 257 assert(n->size > s, "Sanity"); Assertion is redundant. 262 // n is the best fit. 263 return n; In the following example it is not, is it? N1:40 / / N2:20 \ \ N3:30 find_closest_fit(N1, 30) will return N2 but N3 is the closest fit. I think you have to search the left tree for a better fit independently of the size of its root node. 293 if (n->left == NULL && n->right == NULL) { 294 replace_node_in_parent(n, NULL); 295 296 } else if (n->left == NULL && n->right != NULL) { 297 replace_node_in_parent(n, n->right); 298 299 } else if (n->left != NULL && n->right == NULL) { 300 replace_node_in_parent(n, n->left); 301 302 } else { Can be simplified to: if (n->left == NULL) { replace_node_in_parent(n, n->right); } else if (n->right == NULL) { replace_node_in_parent(n, n->left); } else { 341 // The right child of the successor (if there was one) replaces the successor at its parent's left child. Please add a line break. The comments and assertions in remove_node_from_tree() helped to understand the logic. Thanks! ====== src/hotspot/share/memory/metaspace/blocktree.cpp 40 // These asserts prints the tree, then asserts 41 #define assrt(cond, format, ...) \ 42 if (!(cond)) { \ 43 print_tree(tty); \ 44 assert(cond, format, __VA_ARGS__); \ 45 } 46 47 // This assert prints the tree, then stops (generic message) 48 #define assrt0(cond) \ 49 if (!(cond)) { \ 50 print_tree(tty); \ 51 assert(cond, "sanity"); \ 52 } Better wrap into do-while(0) (see definition of vmassert) 66 while (n2 != NULL) { 67 assrt0(n2->size == size); 68 vd->counter.add(n2->size); 69 if (prev_sib != NULL) { 70 assrt0(prev_sib->next == n2); 71 assrt0(prev_sib != n2); 72 } 73 prev_sib = n2; 74 n2 = n2->next; 75 } The assertion in line 70 cannot fail. 110 verify_node(n->left, left_limit, n->size, vd, lvl + 1); Recursive call that isn't a tail call. Prone to stack overflow. Well I guess you need a stack to traverse a tree. GrowableArray is a common choice if you want to eliminate this recursion. As it is only verification code you might as well leave it and interpret stack overflow as verification failure. 118 verify_node(n->right, n->size, right_limit, vd, lvl + 1); Tail-recursion can be easily eliminated. See comments on blocktree.hpp above. ====== src/hotspot/share/memory/metaspace/chunkManager.cpp The slow parameter in ChunkManager::verify*() is not used. ====== src/hotspot/share/memory/metaspace/counter.hpp 104 void decrement() { 105 #ifdef ASSERT 106 T old = Atomic::load_acquire(&_c); 107 assert(old >= 1, 108 "underflow (" UINT64_FORMAT "-1)", (uint64_t)old); 109 #endif 110 Atomic::dec(&_c); 111 } I think you could use Atomic::add() which returns the old value and make the assert atomic too: void decrement() { T old = Atomic::add(&_c, T(-1)); #ifdef ASSERT assert(old >= 1, "underflow (" UINT64_FORMAT "-1)", (uint64_t)old); #endif } Same for increment(), increment_by(), decrement_by(), ... ====== src/hotspot/share/memory/metaspace/metaspaceArena.cpp There's too much vertical white space, I'd think. metaspace::get_raw_allocation_word_size() is a duplicate of metaspace::get_raw_word_size_for_requested_word_size() metaspace::get_raw_allocation_word_size() is only referenced in comments and should be removed. ====== src/hotspot/share/memory/metaspace/metaspaceCommon.cpp 174 // Given a net allocation word size, return the raw word size we actually allocate. 175 // Note: externally visible for gtests. 176 //static 177 size_t get_raw_word_size_for_requested_word_size(size_t word_size) { There are call sites in other compilation units too. The lines 175, 176 can be removed I'd say. 181 // Deallocated metablocks are kept in a binlist which limits their minimal 182 // size to at least the size of a binlist item (2 words). 183 byte_size = MAX2(byte_size, FreeBlocks::minimal_word_size * BytesPerWord); byte_size should also depend on BlockTree::minimal_word_size I think. Something like if (worde_size > FreeBlocks::maximal_word_size) byte_size = MAX2(byte_size, BlockTree::minimal_word_size * BytesPerWord); FreeBlocks::maximal_word_size needs to be defined for this. -----Original Message----- From: hotspot-gc-dev On Behalf Of Thomas St?fe Sent: Mittwoch, 12. August 2020 20:00 To: Hotspot dev runtime ; Hotspot-Gc-Dev Subject: RFR: 8251158: Implementation of JEP 387: Elastic Metaspace Dear all, I would like to start the review for the implementation of JEP 387 "Elastic Metaspace". The relevant JBS items are: JEP: https://openjdk.java.net/jeps/387 Implementation Issue (pretty much only a placeholder currently): https://bugs.openjdk.java.net/browse/JDK-8251158 -- Reminder of why we do this: 1. The new metaspace saves memory. How much depends on context: if it is not a problem today it will continue to be none. But in cases where today we face large metaspace consumption we may get reductions, sometimes drastic ones. These reductions are caused by two facts: - after mass class unloading we release memory more promptly to the OS - even without class unloading chunk allocation is just plain smarter, which reduces the overhead per class loader. This is especially true for scenarios involving masses of small loaders which only load very few classes. As an example, see [1] where two VMs - one stock, one patched - run the same test program which creates tons of small loaders. The second run shows a much reduced memory footprint and increased elasticity. 2. The rewritten metaspace code base got cleaner and less complex and thus should be much easier to maintain and to extend. It also would be possible to easily reuse the allocator for different parts of the VM, since it is less tightly tied to the notion of just storing class metadata. -- In preparation of this review I prepared a guide [2], [3], which gives an architectural overview and should be the starting point for an interested Reviewer. The prototype has been tested extensively for quite some time in SAP's CI system. We regularly run JCK test, JTReg tests and a whole battery of SAP internal tests on 8 platforms. Tests are also currently ongoing at Oracle and Red Hat. -- The full webrev [4] is somewhat large. In order to make this more bearable I broke the patch up into three parts: 1) The core parts [5] This is the heart of the metaspace, mostly rewritten from scratch. I propose any Reviewer should not look at the diff but rather just examine the new implementation. One possible exception is metaspace.hpp, which is the outside interface to metaspace. There are several ways to get a feeling for this code (after reading at least the concept part of the provided guide [2]). The central, most "beefy" classes are: - VirtualSpaceNode - does all the work of reserving, committing, uncommitting memory - RootChunkArea - does the grunt work of chunk splitting and merging - ChunkManager - which takes care of the chunk freelists, of directing chunk splits and merges, of enlarging chunks in place - MetaspaceArena - which takes care of fine granular allocation on behalf of a CLD, and of managing deallocated blocks. One way to review could be bottom up: starting at VirtualSpaceNode/CommitMask/RootChunkArea(LUT), working your way up to ChunkManager and Metachunk; finally up to to MetaspaceArena while taking a side stroll to FreeBlocks/BinList/BlockTree. Another way would be to follow typical paths: Allocation path: Starting at MetaspaceArena::allocate() -> Metachunk::allocate() (note the committing-on-demand code path starting here) -> ChunkManager::get_chunk() -> VirtualSpaceList/Node->allocate_root_chunk(). Destruction: ~MetaspaceArena() -> ChunkManager::return_chunk() -> (merging chunks) -> (uncommitting chunks) Premature deallocation: -> MetaspaceArena::deallocate() -> see freeblock list handling 2) The tests [6] This part of the patch contains all the new tests. There are a lot; the test coverage of the new metaspace is very thorough. New gtests have been added under 'test/hotspot/gtest/metaspace'. New jtreg have been added under 'test/hotspot/jtreg/runtime/Metaspace/elastic' and under 'test/hotspot/jtreg/gtest/MetaspaceGtests.java'. 4) Miscellaneous diffs [7] This is the part of the patch which is neither considered core nor test. These changes are small, often uninteresting, and can be reviewed via diff. --- Out of scope for this patch is revamping outside metaspace statistics: monitoring of metaspace statistics is mostly left untouched, beyond the work needed to keep existing tests green. I wanted to keep those changes separate from the core changes for JEP387. They are tracked in JDK-8251392 [8] and JDK-8251342 [9], respectively. --- Code complexity: Numerically, lines of code went ever so slightly down with this patch: Before: (cloc hotspot/share/memory) ------------------------------------------------------------------------------- C++ 36 2533 3097 12735 C/C++ Header 54 1728 2867 6278 ------------------------------------------------------------------------------- SUM: 90 4261 5964 19013 After: ------------------------------------------------------------------------------- C++ 50 3048 3774 13127 C/C++ Header 63 2006 3575 5738 ------------------------------------------------------------------------------- SUM: 113 5054 7349 18865 But since the new implementation is more powerful than the old one - doing things like committing/uncommitting on demand, guarding allocated blocks etc - one could argue that the new solution does quite a lot more with slightly less code, which I think is a good result. --- Idle musing: it would simplify matters quite a bit if we were to unify class space and non-class metaspace. If we keep the current narrow Klass pointer encoding scheme, we would still need to keep the space they are stored in contiguous. But we could use the class space for everything, in effect dropping the "class" moniker from the name. It would have to be larger than it is currently, of course. Cons: - we would have to abandon the notion of "limitless metaspace" - but if we run with class space this has never been true anyway since the number of classes is limited by the size of the compressed class space. - virtual process size would go up since it now needs to be as large as the total projected metaspace size. - we may need to expand narrow Klass pointer encoding to include shifting, if 4G are not enough to hold all metaspace data. Pros: - this would simplify a lot of code. - this would save real (committed) memory, since we save quite a bit of overhead. - we could narrow-pointer-encode other metadata too, not only Klass pointers, should that ever be interesting ... but that is not part of this JEP. --- With this patch, we have a much stronger separation of concerns, and it should be easy to reuse the metaspace allocator in other hotspot areas as well. For instance, ResourceAreas and friends could be replaced by MetaspaceArenas. They do almost the same thing. And as we have seen in the past at SAP, the C-heap backed hotspot Arenas can be a pain since spikes in Arena usage lingers forever as process footprint (we even once rewrote parts of the arena code to use mmap, which is just on a more primitive level what Metaspace does). --- I know this is short notice, but there will be a call for interested people tomorrow at 11AM ET. In case any potential Reviewers are interested just drop me a short note. --- Thank you, Thomas [1] http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/reduction-small-loaders.pdf [2] http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.pdf [3] http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/guide/review-guide.html [4] http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-all/webrev/ [5] http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-core/webrev/ [6] http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-test/webrev/ [7] http://cr.openjdk.java.net/~stuefe/jep387/review/2020-08-11/webrev-misc/webrev/ [8] https://bugs.openjdk.java.net/browse/JDK-8251342 [9] https://bugs.openjdk.java.net/browse/JDK-8251392 From stefan.karlsson at oracle.com Mon Aug 31 16:46:35 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 31 Aug 2020 18:46:35 +0200 Subject: RFR: 8252589: Code duplication in ParallelSPCleanupTask Message-ID: Hi all, Please review this small patch to remove some code duplication in ParallelSPCleanupTask. https://cr.openjdk.java.net/~stefank/8252589/webrev.01 https://bugs.openjdk.java.net/browse/JDK-8252589 I noticed this while reviewing the patches for JEP 376. I think this makes it more apparent what the individual sub-tasks are doing. Thanks, StefanK From ioi.lam at oracle.com Mon Aug 31 16:49:05 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 31 Aug 2020 09:49:05 -0700 Subject: RFR(S) 8252481 Remove excessive include of memTracker.hpp In-Reply-To: References: <1eda87bd-c398-ed0e-d6e4-63f2a1f88baf@oracle.com> Message-ID: Hi Lois, Thanks for the review! - Ioi On 8/31/20 6:10 AM, Lois Foltan wrote: > Looks good Ioi! > Lois > > On 8/28/2020 1:21 AM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8252481 >> http://cr.openjdk.java.net/~iklam/jdk16/8252481-excessive-memTracker-hpp.v01/ >> >> >> memTracker.hpp is included by 847 out of 971 .o files in the HotSpot >> build. >> >> Most of these are caused by these header files: >> >> jfrAllocation.hpp -- completely unnecessary >> >> allocation.hpp, allocation.inline.hpp -- the use of CURRENT_PC can be >> avoid by calling this function instead: >> >> char* AllocateHeap(size_t size, >> ?????????????????? MEMFLAGS flags, >> ?????????????????? AllocFailType alloc_failmode /* = >> AllocFailStrategy::EXIT_OOM*/) { >> ? return AllocateHeap(size, flags, CALLER_PC); >> } >> >> >> ====================== >> >> Before: >> >> $ cd build/linux-x64 >> $ find hotspot -name \*.o | wc -l >> 971 >> $ find hotspot -name \*.d | xargs grep -l memTracker.hpp | wc -l >> 847 >> $ find hotspot -name \*.d | xargs cat | wc -l >> 528505 >> >> After: >> >> $ find hotspot -name \*.d | xargs grep -l memTracker.hpp | wc -l >> 95 >> $ find hotspot -name \*.d | xargs cat | wc -l >> 512731 >> >> The total number of included header files has reduced for about 3% >> >> 512731 / 528505 = 0.970 >> >> ====================== >> >> Testing -- I am running mach5 tiers 1-4. > From shade at redhat.com Mon Aug 31 16:50:06 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 31 Aug 2020 18:50:06 +0200 Subject: RFR (S) 8252592: Non-PCH build is broken after JDK-8251560 Message-ID: <0225f3e5-ef5f-8e3e-17bc-6b66c681921c@redhat.com> Bug: https://bugs.openjdk.java.net/browse/JDK-8252592 Fix is to add the appropriate #include where vm_exit_during_initialization is used. diff -r e10f558e1df5 src/hotspot/share/gc/shenandoah/mode/shenandoahIUMode.cpp --- a/src/hotspot/share/gc/shenandoah/mode/shenandoahIUMode.cpp Mon Aug 31 16:12:32 2020 +0100 +++ b/src/hotspot/share/gc/shenandoah/mode/shenandoahIUMode.cpp Mon Aug 31 18:47:57 2020 +0200 @@ -33,2 +33,3 @@ #include "logging/logTag.hpp" +#include "runtime/java.hpp" diff -r e10f558e1df5 src/hotspot/share/gc/shenandoah/mode/shenandoahSATBMode.cpp --- a/src/hotspot/share/gc/shenandoah/mode/shenandoahSATBMode.cpp Mon Aug 31 16:12:32 2020 +0100 +++ b/src/hotspot/share/gc/shenandoah/mode/shenandoahSATBMode.cpp Mon Aug 31 18:47:57 2020 +0200 @@ -33,2 +33,3 @@ #include "logging/logTag.hpp" +#include "runtime/java.hpp" diff -r e10f558e1df5 src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp --- a/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Mon Aug 31 16:12:32 2020 +0100 +++ b/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Mon Aug 31 18:47:57 2020 +0200 @@ -31,2 +31,3 @@ #include "gc/shenandoah/shenandoahHeapRegion.hpp" +#include "runtime/java.hpp" #include "utilities/defaultStream.hpp" diff -r e10f558e1df5 src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp --- a/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp Mon Aug 31 16:12:32 2020 +0100 +++ b/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp Mon Aug 31 18:47:57 2020 +0200 @@ -78,2 +78,3 @@ #include "runtime/interfaceSupport.inline.hpp" +#include "runtime/java.hpp" #include "runtime/orderAccess.hpp" Testing: local builds -- Thanks, -Aleksey From shade at redhat.com Mon Aug 31 16:51:37 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Mon, 31 Aug 2020 18:51:37 +0200 Subject: RFR: 8252589: Code duplication in ParallelSPCleanupTask In-Reply-To: References: Message-ID: <4e0cc0c3-faf4-4c81-31ef-4b923bec22dc@redhat.com> On 8/31/20 6:46 PM, Stefan Karlsson wrote: > Hi all, > > Please review this small patch to remove some code duplication in > ParallelSPCleanupTask. > > https://cr.openjdk.java.net/~stefank/8252589/webrev.01 > https://bugs.openjdk.java.net/browse/JDK-8252589 Looks fine. -- Thanks, -Aleksey From stefan.karlsson at oracle.com Mon Aug 31 16:52:24 2020 From: stefan.karlsson at oracle.com (Stefan Karlsson) Date: Mon, 31 Aug 2020 18:52:24 +0200 Subject: RFR: 8252589: Code duplication in ParallelSPCleanupTask In-Reply-To: <4e0cc0c3-faf4-4c81-31ef-4b923bec22dc@redhat.com> References: <4e0cc0c3-faf4-4c81-31ef-4b923bec22dc@redhat.com> Message-ID: <808112cc-4422-55cb-091f-2d39fc70fbcc@oracle.com> Thanks, Aleksey, StefanK On 2020-08-31 18:51, Aleksey Shipilev wrote: > On 8/31/20 6:46 PM, Stefan Karlsson wrote: >> Hi all, >> >> Please review this small patch to remove some code duplication in >> ParallelSPCleanupTask. >> >> https://cr.openjdk.java.net/~stefank/8252589/webrev.01 >> https://bugs.openjdk.java.net/browse/JDK-8252589 > > Looks fine. > From martin.doerr at sap.com Mon Aug 31 17:00:10 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 31 Aug 2020 17:00:10 +0000 Subject: Fatal errors when running JCK tests with JDK15/16 debug build In-Reply-To: References: Message-ID: Hi David, thanks for analyzing it. We need to exclude the test for now. Best regards, Martin > -----Original Message----- > From: David Holmes > Sent: Montag, 31. August 2020 04:34 > To: Doerr, Martin ; serviceability- > dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net > Subject: Re: Fatal errors when running JCK tests with JDK15/16 debug build > > Hi Martin, > > On 29/08/2020 3:53 am, Doerr, Martin wrote: > > Hi, > > > > we have seen the following fatal error more than 50 times since > > 2020-05-25 in various JCK tests vm/jvmti. > > > > fatal error: String conversion failure: [check] ExitLock destroyed > > > > --> ?? [check] ExitLock exited > > > > (followed by garbage output) > > > > 8166358: Re-enable String verification in > > java_lang_String::create_from_str() > > > > was pushed at that date which introduced the call to fatal. > > > > Stack (example from linuxppc64le, but also observed on x86 and aarch64): > > V? [libjvm.so+0xee242c]? java_lang_String::create_from_str(char const*, > > Thread*) [clone .part.158]+0x51c > > V? [libjvm.so+0xee2530]? java_lang_String::create_oop_from_str(char > > const*, Thread*)+0x40 > > V? [libjvm.so+0x1026a30]? jni_NewStringUTF+0x1e0 > > C? [libjckjvmti.so+0x3ce4c]? logWrite+0x5c > > C? [libjckjvmti.so+0x3cd20]? lprintf+0x170 > > C? [libjckjvmti.so+0x485b8]? gast00104_agent_proc+0x254 > > V? [libjvm.so+0x1218f0c]? JvmtiAgentThread::call_start_function()+0x24c > > V? [libjvm.so+0x193a8fc]? JavaThread::thread_main_inner()+0x32c > > V? [libjvm.so+0x19418a0]? Thread::call_run()+0x160 > > V? [libjvm.so+0x15c9d0c]? thread_native_entry(Thread*)+0x18c > > C? [libpthread.so.0+0x9b48]? start_thread+0x108 > > > > (Problem could have been there before but without this fatal message.) > > > > The messages are generated by: > > > > tests/vm/jvmti/GetAllStackTraces/gast001/gast00104/gast00104.c > > > > This looks like a race condition. The message changes while the VM > > creates a String object from it. Has anybody seen this before? > > No but ... > > > Is it a test problem? I'm not familiar with the lprintf calls in the test. > > ... the lprintf is part of the JCK support library (support.c if you > have access to sources) and it uses a static buffer for the log messages > and so it not thread-safe. This test creates a thread and both it and > the main thread call lprintf concurrently. > > So this is a JCK test/test-library bug that appears to be exposed by the > changes made in 8166358. > > Cheers, > David > ----- > > > Best regards, > > > > Martin > > From martin.doerr at sap.com Mon Aug 31 17:20:10 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Mon, 31 Aug 2020 17:20:10 +0000 Subject: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) In-Reply-To: References: <022e6096-a388-dc79-f959-00a2d933ae89@oracle.com> <1d5921d1-1cb4-b286-a95b-7e2d4b60ced0@oracle.com> Message-ID: Hi Thomas, I have found several checks for if (os::Aix::on_aix()) in the new signals_posix.cpp. That is code which checks if it?s running on regular AIX or on as400 PASE. @Gerard: Some background: We had originally built the original AIX code in a way that we can run in on as400 PASE, too (which is an incomplete POSIX layer for a non-POSIX OS, Thomas knows it better). We have avoided to link against certain libraries statically and only use limited features. The VM can check if it?s running on regular AIX instead of as400 PASE and it can link against additional libraries dynamically and use more features. However, jdk/jdk doesn?t support as400 PASE, so I?d be fine with using Gerard?s new POSIX code and removing all AIX specific stuff which was built to support as400 PASE. I believe the semaphore stuff works on AIX, so we don?t need ?#if !defined(AIX)?, right? Best regards, Martin From: Thomas St?fe Sent: Samstag, 29. August 2020 11:17 To: Doerr, Martin Cc: gerard ziemski ; hotspot-runtime-dev at openjdk.java.net; Schmidt, Lutz ; Lindenmaier, Goetz ; Stuefe, Thomas Subject: Re: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) On Fri, Aug 28, 2020 at 11:23 PM Doerr, Martin > wrote: Hi Gerard, I just gave it a quick spin. Fix for os_aix_ppc.cpp is trivial (see below). signals_posix.cpp takes more effort. I can see that a lot of AIX specific code is for as400 PASE support which we never contributed to OpenJDK. Maybe we can remove it and use the regular posix code instead. @Thomas: What do you think? I see that we have a patch in AIX for the synchronous-error-signals-during-signal-handling-issue which would make sense for all platforms. I opened issue https://bugs.openjdk.java.net/browse/JDK-8252533 to track that. We already deal with this (see https://bugs.openjdk.java.net/browse/JDK-8065895, one of my first contributions btw :) but that first fix leaves some windows open which should be closed. I see Gerard already reviewed the original patch :) I'd propose to ignore this AIX-specific patch, just remove it for the time being and treat AIX like every other posix platform. Then, re-evaluate https://bugs.openjdk.java.net/browse/JDK-825253 after this cleanup as a separate issue. @Gerard: If you want to take over https://bugs.openjdk.java.net/browse/JDK-825253 this would be fine by me. Other than that I cannot find much AIX/pase specifics in the original code. @Martin: Have I missed something? Cheers, Thomas Link which works correctly (version below links to pre1): http://cr.openjdk.java.net/~gziemski/8252324_pre2/ Best regards, Martin diff -r 39c430bf9bb3 src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp --- a/src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp Fri Aug 28 22:59:25 2020 +0200 +++ b/src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp Fri Aug 28 23:13:25 2020 +0200 @@ -51,6 +51,7 @@ #include "runtime/stubRoutines.hpp" #include "runtime/thread.inline.hpp" #include "runtime/timer.hpp" +#include "signals_posix.hpp" #include "utilities/events.hpp" #include "utilities/vmError.hpp" #ifdef COMPILER1 @@ -225,7 +226,7 @@ JavaThread* thread = NULL; VMThread* vmthread = NULL; - if (os::Aix::signal_handlers_are_installed) { + if (os::signal_handlers_are_installed) { if (t != NULL) { if(t->is_Java_thread()) { thread = (JavaThread*)t; From: gerard ziemski > Sent: Freitag, 28. August 2020 22:28 To: Thomas St?fe >; Doerr, Martin > Cc: hotspot-runtime-dev at openjdk.java.net; Schmidt, Lutz >; Lindenmaier, Goetz >; Stuefe, Thomas > Subject: Re: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) hi Thomas, Thank you for your offer to test the changes. I know that you are currently busy with your metaspace changes, but when you do you have some time, then I have made a preliminary changes for AIX here to test: http://cr.openjdk.java.net/~gziemski/8252324_pre2 cheers On 8/27/20 4:48 PM, Thomas St?fe wrote: Hi Gerard, Martin, I like the direction this patch is going, this is definitely a good cleanup! Also thanks from me for not wanting to break AIX :) Wrt to signals AIX should not be that different from other Posix platforms. Any differences probably stem from the fact that the AIX port had a long time brewing in our proprietary code base before we contributed it, with not much sync points with other ports after the initial fork. (Memory management in AIX is different for a number of reasons, but signal stuff should be pretty standard). There is this small issue that we use SIGTRAP for implicit null checks (@Martin: is this still a thing?) but strictly speaking this is a ppc issue, not an AIX issue, so it hits Linux too (@Martin: there seem to be small diffs between how sigtrap is handled in AIX and Linux, see e.g. the use of UseSigTrap, is that by design?). But all that should do is to add another signal to handle/unblock. I would say, give it your best shot and dry-code the changes. When in doubt, just ask. We will test your patch and correct if necessary. --- Some more general remarks to your preliminary changes: I would really like it if we could start documenting APIs. I am always astonished that this is so seldom done. Ironically, almost all existing API comments in os_posix.hpp were contributed by SAP :-) Since you now move all APIs to a central location, and it's all fresh in your head, maybe this would be a good point to add comments to them, describing behavior and input/output params. --- I quite liked the os::Posix namespace and think the signal functions are well placed there. But I see what you want, you want to separate the signal functions from the os namespace since that one requires including os.hpp? Unfortunately this also makes the diff more complicated since a lot of code just moved. Side note, I wish we would start using real C++ namespaces, we could make os a namespace and hence split it over as many header files as we want. Maybe something for future cleanups. --- ucontext_(s|g)et_pc() have not much to do with signals, even though they are only used for signal continuation currently. They seem misplaced in PosixSignals. --- Thanks, Thomas On Thu, Aug 27, 2020 at 10:33 PM Doerr, Martin > wrote: Hi Gerard, > I don?t want to straight break AIX port unexpectedly. Thanks for that! We highly appreciate that you take care of it. It?s not surprising that adapting the AIX code is difficult. I?ve added Thomas to Cc. He?s surely the most experienced person for these AIX specific things. Unfortunately, he?s currently very busy because he?s contributing his new metaspace implementation. We have to discuss how we can help. Best regards, Martin From: gerard ziemski > Sent: Donnerstag, 27. August 2020 19:19 To: hotspot-runtime-dev at openjdk.java.net Cc: Schmidt, Lutz >; Lindenmaier, Goetz >; Doerr, Martin > Subject: Attention AIX developers - factoring out POSIX signal code (JDK-8252324) hi all, I?m hoping I have included the right people in this email, but everyone is welcome to provide feedback. I have a code for JDK-8252324, which factors out the common POSIX signal related code out into common file, so that all platforms can benefit from fixes that apply to all POSIX platforms (like JDK-82514380) The preliminary webrev can be found here: http://cr.openjdk.java.net/~gziemski/8252324_pre1 I tried to port the changes to AIX myself, but got bogged down by more "#ifdef AIX? than I was hoping for (possibly too many differences to factor out the code for AIX, but frankly I don't think so), so I stopped (though I probably was on the right track) - to me it looks like AIX has some extra fixes and differences in the signal code than BSD/Linux. My question is: given my inability to test on AIX, what can I do to make it easier for AIX developers to adopt this fix? I will try porting my changes again, but in the meantime if anyone has any pointers or offer of other help, I?d appreciate it. I don?t want to straight break AIX port unexpectedly. Reference: https://bugs.openjdk.java.net/browse/JDK-8252324 Some notes about the affected APIs: -------------------------------------------------------------------------------------------------- The APIs which moved from os/bsd/os_bsd.cpp (similarly for linux) to to os/posix/PosixSignals.cpp: //////////////////////////////////////////////////////////////////////////////// // signal support void os::Bsd::signal_sets_init() sigset_t* os::Bsd::unblocked_signals() sigset_t* os::Bsd::vm_signals() void os::Bsd::hotspot_sigmask(Thread* thread) //////////////////////////////////////////////////////////////////////////////// // sun.misc.Signal support static void UserHandler(int sig, void *siginfo, void *context) void* os::user_handler() void* os::signal(int signal_number, void* handler) void os::signal_raise(int signal_number) int os::sigexitnum_pd() static void jdk_misc_signal_init() void os::signal_notify(int sig) static int check_pending_signals() int os::signal_wait() //////////////////////////////////////////////////////////////////////////////// // suspend/resume support static void resume_clear_context(OSThread *osthread) static void suspend_save_context(OSThread *osthread, siginfo_t* siginfo, ucontext_t* context) static void SR_handler(int sig, siginfo_t* siginfo, ucontext_t* context) static int SR_initialize() static int sr_notify(OSThread* osthread) static bool do_suspend(OSThread* osthread) static void do_resume(OSThread* osthread) /////////////////////////////////////////////////////////////////////////////////// // signal handling (except suspend/resume) static void signalHandler(int sig, siginfo_t* info, void* uc) struct sigaction* os::Bsd::get_chained_signal_action(int sig) static bool call_chained_handler(struct sigaction *actp, int sig, siginfo_t *siginfo, void *context) bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* context) int os::Bsd::get_our_sigflags(int sig) void os::Bsd::set_our_sigflags(int sig, int flags) void os::Bsd::set_signal_handler(int sig, bool set_installed) void os::Bsd::install_signal_handlers() static const char* get_signal_handler_name(address handler, char* buf, int buflen) static void print_signal_handler(outputStream* st, int sig, char* buf, size_t buflen) void os::run_periodic_checks() void os::Bsd::check_signal_handler(int sig) ----------------------------------------------------------------------------- The APIs which moved from os/posix/os_posix.cpp to os/posix/PosixSignals.cpp: const char* os::Posix::get_signal_name(int sig, char* out, size_t outlen) int os::Posix::get_signal_number(const char* signal_name) int os::get_signal_number(const char* signal_name) bool os::Posix::is_valid_signal(int sig) bool os::Posix::is_sig_ignored(int sig) const char* os::exception_name(int sig, char* buf, size_t size) const char* os::Posix::describe_signal_set_short(const sigset_t* set, char* buffer, size_t buf_size) void os::Posix::print_signal_set_short(outputStream* st, const sigset_t* set) const char* os::Posix::describe_sa_flags(int flags, char* buffer, size_t size) oid os::Posix::print_sa_flags(outputStream* st, int flags) static bool get_signal_code_description(const siginfo_t* si, enum_sigcode_desc_t* out) void os::print_siginfo(outputStream* os, const void* si0) bool os::signal_thread(Thread* thread, int sig, const char* reason) int os::Posix::unblock_thread_signal_mask(const sigset_t *set) address os::Posix::ucontext_get_pc(const ucontext_t* ctx) void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) struct sigaction* os::Posix::get_preinstalled_handler(int sig) void os::Posix::save_preinstalled_handler(int sig, struct sigaction& oldAct) DETAILS: -------------------------------------------------------- Public APIs which are now internal static PosixSignals:: sigset_t* os::Bsd::vm_signals() struct sigaction* os::Bsd::get_chained_signal_action(int sig) int os::Bsd::get_our_sigflags(int sig) void os::Bsd::set_our_sigflags(int sig, int flags) void os::Bsd::set_signal_handler(int sig, bool set_installed) void os::Bsd::check_signal_handler(int sig) const char* os::Posix::get_signal_name(int sig, char* out, size_t outlen) bool os::Posix::is_valid_signal(int sig) const char* os::Posix::describe_signal_set_short(const sigset_t* set, char* buffer, size_t buf_size) void os::Posix::print_signal_set_short(outputStream* st, const sigset_t* set) const char* os::Posix::describe_sa_flags(int flags, char* buffer, size_t size) oid os::Posix::print_sa_flags(outputStream* st, int flags) static bool get_signal_code_description(const siginfo_t* si, enum_sigcode_desc_t* out) void os::Posix::save_preinstalled_handler(int sig, struct sigaction& oldAct) ------------------------------------------------ Public APIs which moved to public PosixSignals:: void os::Bsd::signal_sets_init() void os::Bsd::hotspot_sigmask(Thread* thread) bool os::Bsd::chained_handler(int sig, siginfo_t* siginfo, void* context) void os::Bsd::install_signal_handlers() bool os::Posix::is_sig_ignored(int sig) int os::Posix::unblock_thread_signal_mask(const sigset_t *set) address os::Posix::ucontext_get_pc(const ucontext_t* ctx) void os::Posix::ucontext_set_pc(ucontext_t* ctx, address pc) ---------------------------------------------------- Internal APIs which are now public in PosixSignals:: static void jdk_misc_signal_init() static int SR_initialize() static bool do_suspend(OSThread* osthread) static void do_resume(OSThread* osthread) static void print_signal_handler(outputStream* st, int sig, char* buf, size_t buflen) -------------------------- New APIs in PosixSignals:: static bool are_signal_handlers_installed(); cheers From zgu at redhat.com Mon Aug 31 17:29:57 2020 From: zgu at redhat.com (Zhengyu Gu) Date: Mon, 31 Aug 2020 13:29:57 -0400 Subject: RFR (S) 8252592: Non-PCH build is broken after JDK-8251560 In-Reply-To: <0225f3e5-ef5f-8e3e-17bc-6b66c681921c@redhat.com> References: <0225f3e5-ef5f-8e3e-17bc-6b66c681921c@redhat.com> Message-ID: Looks good to me. -Zhengyu On 8/31/20 12:50 PM, Aleksey Shipilev wrote: > diff -r e10f558e1df5 > src/hotspot/share/gc/shenandoah/mode/shenandoahIUMode.cpp > --- a/src/hotspot/share/gc/shenandoah/mode/shenandoahIUMode.cpp Mon Aug > 31 16:12:32 2020 +0100 > +++ b/src/hotspot/share/gc/shenandoah/mode/shenandoahIUMode.cpp Mon Aug > 31 18:47:57 2020 +0200 > @@ -33,2 +33,3 @@ > ?#include "logging/logTag.hpp" > +#include "runtime/java.hpp" > > diff -r e10f558e1df5 > src/hotspot/share/gc/shenandoah/mode/shenandoahSATBMode.cpp > --- a/src/hotspot/share/gc/shenandoah/mode/shenandoahSATBMode.cpp > Mon Aug 31 16:12:32 2020 +0100 > +++ b/src/hotspot/share/gc/shenandoah/mode/shenandoahSATBMode.cpp > Mon Aug 31 18:47:57 2020 +0200 > @@ -33,2 +33,3 @@ > ?#include "logging/logTag.hpp" > +#include "runtime/java.hpp" > > diff -r e10f558e1df5 > src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp > --- a/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp?? Mon Aug > 31 16:12:32 2020 +0100 > +++ b/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp?? Mon Aug > 31 18:47:57 2020 +0200 > @@ -31,2 +31,3 @@ > ?#include "gc/shenandoah/shenandoahHeapRegion.hpp" > +#include "runtime/java.hpp" > ?#include "utilities/defaultStream.hpp" > diff -r e10f558e1df5 src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp > --- a/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp??????? Mon Aug > 31 16:12:32 2020 +0100 > +++ b/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp??????? Mon Aug > 31 18:47:57 2020 +0200 > @@ -78,2 +78,3 @@ > ?#include "runtime/interfaceSupport.inline.hpp" > +#include "runtime/java.hpp" > ?#include "runtime/orderAccess.hpp" From ioi.lam at oracle.com Mon Aug 31 18:13:16 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 31 Aug 2020 11:13:16 -0700 Subject: RFR (S) 8252592: Non-PCH build is broken after JDK-8251560 In-Reply-To: <0225f3e5-ef5f-8e3e-17bc-6b66c681921c@redhat.com> References: <0225f3e5-ef5f-8e3e-17bc-6b66c681921c@redhat.com> Message-ID: <874d911e-c583-2355-fcec-f51dc8307568@oracle.com> Looks good to me. Sorry about breaking the build for you :-( Thanks - Ioi On 8/31/20 9:50 AM, Aleksey Shipilev wrote: > Bug: > ? https://bugs.openjdk.java.net/browse/JDK-8252592 > > Fix is to add the appropriate #include where > vm_exit_during_initialization is used. > > diff -r e10f558e1df5 > src/hotspot/share/gc/shenandoah/mode/shenandoahIUMode.cpp > --- a/src/hotspot/share/gc/shenandoah/mode/shenandoahIUMode.cpp Mon > Aug 31 16:12:32 2020 +0100 > +++ b/src/hotspot/share/gc/shenandoah/mode/shenandoahIUMode.cpp Mon > Aug 31 18:47:57 2020 +0200 > @@ -33,2 +33,3 @@ > ?#include "logging/logTag.hpp" > +#include "runtime/java.hpp" > > diff -r e10f558e1df5 > src/hotspot/share/gc/shenandoah/mode/shenandoahSATBMode.cpp > --- a/src/hotspot/share/gc/shenandoah/mode/shenandoahSATBMode.cpp Mon > Aug 31 16:12:32 2020 +0100 > +++ b/src/hotspot/share/gc/shenandoah/mode/shenandoahSATBMode.cpp Mon > Aug 31 18:47:57 2020 +0200 > @@ -33,2 +33,3 @@ > ?#include "logging/logTag.hpp" > +#include "runtime/java.hpp" > > diff -r e10f558e1df5 > src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp > --- a/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Mon Aug > 31 16:12:32 2020 +0100 > +++ b/src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp Mon Aug > 31 18:47:57 2020 +0200 > @@ -31,2 +31,3 @@ > ?#include "gc/shenandoah/shenandoahHeapRegion.hpp" > +#include "runtime/java.hpp" > ?#include "utilities/defaultStream.hpp" > diff -r e10f558e1df5 src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp > --- a/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp Mon Aug 31 > 16:12:32 2020 +0100 > +++ b/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp Mon Aug 31 > 18:47:57 2020 +0200 > @@ -78,2 +78,3 @@ > ?#include "runtime/interfaceSupport.inline.hpp" > +#include "runtime/java.hpp" > ?#include "runtime/orderAccess.hpp" > > Testing: local builds > From ioi.lam at oracle.com Mon Aug 31 18:13:45 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 31 Aug 2020 11:13:45 -0700 Subject: RFR(M) 8252526 Remove excessive inclusion of jvmti.h and jvmtiExport.hpp Message-ID: https://bugs.openjdk.java.net/browse/JDK-8252526 http://cr.openjdk.java.net/~iklam/jdk16/8252526-fix-jvmti-hpp.v01/ (I marked this RFR as "M" because 63 files have changed, but most of the are just adding a missing #include "prims/jvmtiExport.hpp"). jvmti.h is included 905 times and jvmtiExport.hpp is included 776 times (by 971 hotspot .o files). Most of these are unnecessarily included by the following 3 popular header files: [1] javaClasses.hpp: ThreadStatus is rarely used, and should be moved ? ? to javaThreadStatus.hpp. I also converted the enum to an C++ 11 ? ? enum class for better type safety. (see also JDK-8247938). [2] os.hpp: No need to include jvm.h. Use forward declaration ? ? "typedef struct _jvmtiTimerInfo jvmtiTimerInfo;" instead. [3] thread.hpp: No need to include jvmExport.hpp. Use forward declaration ? ? for JvmtiSampledObjectAllocEventCollector and ? ? JvmtiVMObjectAllocEventCollector instead. The total number of includes have reduced from 252033 to 250001. Build time of slow-debug libjvm.so is reduced from 2:07 to 2:04 on my machine. Thanks - Ioi From calvin.cheung at oracle.com Mon Aug 31 18:35:18 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Mon, 31 Aug 2020 11:35:18 -0700 Subject: RFR(S) 8252481 Remove excessive include of memTracker.hpp In-Reply-To: <1eda87bd-c398-ed0e-d6e4-63f2a1f88baf@oracle.com> References: <1eda87bd-c398-ed0e-d6e4-63f2a1f88baf@oracle.com> Message-ID: Hi Ioi, Looks good. Just one nit in jfrAllocation.cpp: ? 26 #include "jfr/utilities/jfrAllocation.hpp" ? 27 #include "jfr/recorder/jfrRecorder.hpp" ? 28 #include "jfr/utilities/jfrAllocation.hpp" Line 28 is extra? thanks, Calvin On 8/27/20 10:21 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8252481 > http://cr.openjdk.java.net/~iklam/jdk16/8252481-excessive-memTracker-hpp.v01/ > > > memTracker.hpp is included by 847 out of 971 .o files in the HotSpot > build. > > Most of these are caused by these header files: > > jfrAllocation.hpp -- completely unnecessary > > allocation.hpp, allocation.inline.hpp -- the use of CURRENT_PC can be > avoid by calling this function instead: > > char* AllocateHeap(size_t size, > ?????????????????? MEMFLAGS flags, > ?????????????????? AllocFailType alloc_failmode /* = > AllocFailStrategy::EXIT_OOM*/) { > ? return AllocateHeap(size, flags, CALLER_PC); > } > > > ====================== > > Before: > > $ cd build/linux-x64 > $ find hotspot -name \*.o | wc -l > 971 > $ find hotspot -name \*.d | xargs grep -l memTracker.hpp | wc -l > 847 > $ find hotspot -name \*.d | xargs cat | wc -l > 528505 > > After: > > $ find hotspot -name \*.d | xargs grep -l memTracker.hpp | wc -l > 95 > $ find hotspot -name \*.d | xargs cat | wc -l > 512731 > > The total number of included header files has reduced for about 3% > > 512731 / 528505 = 0.970 > > ====================== > > Testing -- I am running mach5 tiers 1-4. From igor.ignatyev at oracle.com Mon Aug 31 19:02:47 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 31 Aug 2020 12:02:47 -0700 Subject: RFR(S) : 8252004 : remove usage of PropertyResolvingWrapper in vmTestbase/nsk/sysdict In-Reply-To: References: Message-ID: <76F1769B-C665-47B9-9D6F-88CC0AB974DB@oracle.com> ping? -- Igor > On Aug 21, 2020, at 10:46 AM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev/8252004/webrev.00/ >> 22 lines changed: 0 ins; 22 del; 0 mod; > > > Hi all, > > could you please review this small patch which removes usage of PropertyResolvingWrapper from nsk/sysdict tests and reenables allowSmartActionArgs? > > background from the main bug: >> CODETOOLS-7902352 added support of using ${property} in action directive, so PropertyResolvingWrapper isn't needed anymore and can be removed. > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8252004 > webrev: http://cr.openjdk.java.net/~iignatyev/8252004/webrev.00 > testing: :vmTestbase_nsk_sysdict > > Thanks, > -- Igor > From ioi.lam at oracle.com Mon Aug 31 20:21:03 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 31 Aug 2020 13:21:03 -0700 Subject: RFR(S) 8252481 Remove excessive include of memTracker.hpp In-Reply-To: References: <1eda87bd-c398-ed0e-d6e4-63f2a1f88baf@oracle.com> Message-ID: <1ff21ee7-4a9d-99ad-08e2-0abd21f40649@oracle.com> Hi Calvin, Thanks for the review. On 8/31/20 11:35 AM, calvin.cheung at oracle.com wrote: > Hi Ioi, > > Looks good. Just one nit in jfrAllocation.cpp: > > ? 26 #include "jfr/utilities/jfrAllocation.hpp" > ? 27 #include "jfr/recorder/jfrRecorder.hpp" > ? 28 #include "jfr/utilities/jfrAllocation.hpp" > Good catch. I added line 26 for debugging purpose. I'll remove it. Thanks - Ioi > Line 28 is extra? > > thanks, > Calvin > On 8/27/20 10:21 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8252481 >> http://cr.openjdk.java.net/~iklam/jdk16/8252481-excessive-memTracker-hpp.v01/ >> >> >> memTracker.hpp is included by 847 out of 971 .o files in the HotSpot >> build. >> >> Most of these are caused by these header files: >> >> jfrAllocation.hpp -- completely unnecessary >> >> allocation.hpp, allocation.inline.hpp -- the use of CURRENT_PC can be >> avoid by calling this function instead: >> >> char* AllocateHeap(size_t size, >> ?????????????????? MEMFLAGS flags, >> ?????????????????? AllocFailType alloc_failmode /* = >> AllocFailStrategy::EXIT_OOM*/) { >> ? return AllocateHeap(size, flags, CALLER_PC); >> } >> >> >> ====================== >> >> Before: >> >> $ cd build/linux-x64 >> $ find hotspot -name \*.o | wc -l >> 971 >> $ find hotspot -name \*.d | xargs grep -l memTracker.hpp | wc -l >> 847 >> $ find hotspot -name \*.d | xargs cat | wc -l >> 528505 >> >> After: >> >> $ find hotspot -name \*.d | xargs grep -l memTracker.hpp | wc -l >> 95 >> $ find hotspot -name \*.d | xargs cat | wc -l >> 512731 >> >> The total number of included header files has reduced for about 3% >> >> 512731 / 528505 = 0.970 >> >> ====================== >> >> Testing -- I am running mach5 tiers 1-4. From coleen.phillimore at oracle.com Mon Aug 31 20:23:50 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 31 Aug 2020 16:23:50 -0400 Subject: RFR: 8252589: Code duplication in ParallelSPCleanupTask In-Reply-To: References: Message-ID: <0ee9a1af-d095-9fd6-62be-1086ec4fc86c@oracle.com> Looks good to me.? Seems trivial and safe too. Coleen On 8/31/20 12:46 PM, Stefan Karlsson wrote: > Hi all, > > Please review this small patch to remove some code duplication in > ParallelSPCleanupTask. > > https://cr.openjdk.java.net/~stefank/8252589/webrev.01 > https://bugs.openjdk.java.net/browse/JDK-8252589 > > I noticed this while reviewing the patches for JEP 376. I think this > makes it more apparent what the individual sub-tasks are doing. > > Thanks, > StefanK From gerard.ziemski at oracle.com Mon Aug 31 21:12:22 2020 From: gerard.ziemski at oracle.com (gerard ziemski) Date: Mon, 31 Aug 2020 16:12:22 -0500 Subject: RFR(S) : 8252004 : remove usage of PropertyResolvingWrapper in vmTestbase/nsk/sysdict In-Reply-To: <76F1769B-C665-47B9-9D6F-88CC0AB974DB@oracle.com> References: <76F1769B-C665-47B9-9D6F-88CC0AB974DB@oracle.com> Message-ID: hi Igor, Looks fine as far a I can tell, however, other tests are waiting till JDK-8219408 to re-enable allowSmartActionnArgs - why don't we need to wait for that here as well? I noticed that there are other issues filed to address the same issue in other tests, of which there are many (i.e. JDK-8252002) - they will be following the same pattern as this one? cheers On 8/31/20 2:02 PM, Igor Ignatyev wrote: > ping? > > -- Igor > >> On Aug 21, 2020, at 10:46 AM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev/8252004/webrev.00/ >>> 22 lines changed: 0 ins; 22 del; 0 mod; >> >> Hi all, >> >> could you please review this small patch which removes usage of PropertyResolvingWrapper from nsk/sysdict tests and reenables allowSmartActionArgs? >> >> background from the main bug: >>> CODETOOLS-7902352 added support of using ${property} in action directive, so PropertyResolvingWrapper isn't needed anymore and can be removed. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8252004 >> webrev: http://cr.openjdk.java.net/~iignatyev/8252004/webrev.00 >> testing: :vmTestbase_nsk_sysdict >> >> Thanks, >> -- Igor >> From yumin.qi at oracle.com Mon Aug 31 21:32:26 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Mon, 31 Aug 2020 14:32:26 -0700 Subject: 8248337: sparc related code clean up after solaris removal Message-ID: HI, ? Please review for ? bug: https://bugs.openjdk.java.net/browse/JDK-8248337 ? webrev:http://cr.openjdk.java.net/~minqi/2020/8248337/webrev-01/ ? Summary: After Solaris supported files removed from repo, there are some remnants which needs cleaning up. Some comments are not correct, and some refer to wrong files. There is a flag seems only useful for Sparc: UseRDPCForConstantTableBase, which got removed in this patch . Also in postaloc.cpp, the delay slot seems is only for sparc too, but I am not sure about that. Most of the patch are in comment section. ? Tests passed tier1-4 ? Thanks ? Yumin From igor.ignatyev at oracle.com Mon Aug 31 21:51:08 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 31 Aug 2020 14:51:08 -0700 Subject: RFR(S) : 8252004 : remove usage of PropertyResolvingWrapper in vmTestbase/nsk/sysdict In-Reply-To: References: <76F1769B-C665-47B9-9D6F-88CC0AB974DB@oracle.com> Message-ID: <2BD817A5-F836-4974-9D00-20D0AA1D7ED9@oracle.com> Hi Gerard, thanks for your review. 8219408 is to update tests residing in test/jdk, hotspot tests are going to be updated by 8219140's sub-tasks, which include 8252004 and 8252002. I am not updating all hotspot tests by one patch, b/c in some cases, the tests need small modification to work w/o PropertyResolvingWrapper and splitting the work by test groups make it easier to both work on and review such patches. Cheers, -- Igor > On Aug 31, 2020, at 2:12 PM, gerard ziemski wrote: > > hi Igor, > > Looks fine as far a I can tell, however, other tests are waiting till JDK-8219408 to re-enable allowSmartActionnArgs - why don't we need to wait for that here as well? > > I noticed that there are other issues filed to address the same issue in other tests, of which there are many (i.e. JDK-8252002) - they will be following the same pattern as this one? > > > cheers > > > On 8/31/20 2:02 PM, Igor Ignatyev wrote: >> ping? >> >> -- Igor >> >>> On Aug 21, 2020, at 10:46 AM, Igor Ignatyev wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev/8252004/webrev.00/ >>>> 22 lines changed: 0 ins; 22 del; 0 mod; >>> >>> Hi all, >>> >>> could you please review this small patch which removes usage of PropertyResolvingWrapper from nsk/sysdict tests and reenables allowSmartActionArgs? >>> >>> background from the main bug: >>>> CODETOOLS-7902352 added support of using ${property} in action directive, so PropertyResolvingWrapper isn't needed anymore and can be removed. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8252004 >>> webrev: http://cr.openjdk.java.net/~iignatyev/8252004/webrev.00 >>> testing: :vmTestbase_nsk_sysdict >>> >>> Thanks, >>> -- Igor >>> > From coleen.phillimore at oracle.com Mon Aug 31 22:02:08 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 31 Aug 2020 18:02:08 -0400 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: <5c9b2f21-42c4-1409-fb6f-49dc8f98d003@oracle.com> References: <3234fa66-5199-32dd-1f13-e15b61c7dc63@oracle.com> <5c9b2f21-42c4-1409-fb6f-49dc8f98d003@oracle.com> Message-ID: The delta version looks good. On 8/31/20 11:41 AM, Ioi Lam wrote: > Hi Coleen, > > Thanks for your comments. Please see the new version: > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v03/ > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v03.delta/ > > > > Replies in-line below > > > On 8/25/20 1:24 PM, Coleen Phillimore wrote: >> >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html >> >> >> Are these #includes still needed? >> > These are left-overs. I removed them. > >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/classLoaderDataShared.hpp.html >> >> >> Looks like the include guards are wrong.? Should have pushed my >> #pragma once change then nobody would have to worry about these. >> > > Fixed. > >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/classLoaderDataShared.cpp.html >> >> >> ?136 void ClassLoaderDataShared::init_archived_tables() { >> ?137?? assert(DumpSharedSpaces, "must be"); >> ?138?? if (MetaspaceShared::use_full_module_graph()) { >> >> >> Shouldn't these functions be called only if >> MetaspaceShared::use_full_module_graph() instead of having them >> guarded entirely by these if statements. >> ie. assert(DumpSharedSpaces && >> MetaspaceShared::use_full_module_graph(), "must be"); >> > > I fixed as you suggested. > >> Thank you for moving the new code to classLoaderDataShared.cpp >> >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/packageEntry.cpp.frames.html >> >> >> Can you organize this so that the archiving functions for >> PackageEntry are after the PackageEntry:: functions, and the ones for >> PackageEntryTable are after the PackageEntryTable:: functions? >> >> They'll still be sort of mixed up, but the CDS special code in the >> middle isn't good.? And if we ever want to change the entries or the >> tables themselves it'll be easier. >> >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/src/hotspot/share/classfile/moduleEntry.cpp.frames.html >> >> >> Same comment.? The code in both looks fine, even though it looks the >> same. I don't know how to generalize it without being artificial and >> not resembling either.? Separating the functions will help. > > > The code of PackageEntry and PackageEntryTable are somewhat mixed up > in packageEntry.cpp. I've tried to put my new code for PackageEntry > after the first block of code for PackageEntry. Same for > PackageEntryTable. If we want to reorder packageEntry.cpp, maybe we > should do that in a separate RFE? > > (Same for moduleEntry.cpp). Yes, this is fine as another RFE. thanks, Coleen > > Thanks > - Ioi > > >> >> On 8/12/20 6:11 PM, Ioi Lam wrote: >>> Hi Coleen, >>> >>> Thanks for the comments. I have an updated webrev >>> >>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/ >>> >>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02.delta/ >>> >>> >>> >>> On 8/4/20 7:49 AM, Coleen Phillimore wrote: >>>> This is sort of massive, so I'll only make a couple initial comments. >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/classLoaderData.cpp.udiff.html >>>> >>>> >>>> All this new code in ClassLoaderData should go somewhere else, like >>>> maybe classLoaderDataShared.hpp/cpp ?? I don't see how it needs to >>>> be owned by CLD, except maybe the serialized function. >>>> >>>> >>> Fixed >>> >>>> + if (loader_data) { >>>> >>>> >>>> There are many checks that should be != NULL here. >>>> >>> >>> Fixed and added comments >>> >>> >>>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/javaClasses.cpp.udiff.html >>>> >>>> >>>> I don't see why compute_offset/s() has been exported outside >>>> javaClasses.cpp. >>>> >>> >>> It was left-over code that shouldn't be there. I removed it. >>> >>> >>>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/moduleEntry.cpp.udiff.html >>>> >>>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/packageEntry.cpp.udiff.html >>>> >>>> >>>> These look like the same code to me.? Maybe they can be generalized >>>> and put together in a moduleEntryShared.cpp/hpp ? >>>> >>> >>> They look similar but also do different things. Any suggestion for >>> how to generalize them? >>> >>> >>>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/src/hotspot/share/classfile/systemDictionary.cpp.udiff.html >>>> >>>> >>>> Why do you register the loaders here? >>>> >>> >>> It's to force the creation of the ClassLoaderData if a loader oop >>> doesn't have one yet. Also, in the new version, I moved the code to >>> Modules::define_archived_modules(). >> >> Yes, this is a better place. >> >> Thanks, >> Coleen >>> >>> >>> Thanks >>> - Ioi >>> >>>> I don't know enough about the Java code to review that. >>>> >>>> I assume that this going to be rebased on other changes, so this is >>>> only an incomplete review. >>>> >>>> Coleen >>>> >>>> >>>> On 7/22/20 3:36 PM, Ioi Lam wrote: >>>>> https://bugs.openjdk.java.net/browse/JDK-8244778 >>>>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/ >>>>> >>>>> >>>>> Please review this patch that stores the full module graph in the CDS >>>>> archive heap. This reduces the initialization time of the basic >>>>> JVM by >>>>> about 22%: >>>>> >>>>> $ perf stat -r 100 bin/java -version >>>>> before: 98,219,329 instructions 0.03971 secs elapsed (+- 0.44%) >>>>> after:? 55,835,815 instructions 0.03109 secs elapsed (+- 0.65%) >>>>> >>>>> [1] Start with ModuleBootstrap.java. The current implementation is >>>>> ??? quite restrictive: the archived module graph is used only when no >>>>> ??? module options are specified. >>>>> >>>>> ??? See ModuleBootstrap.mayUseArchivedBootLayer(). >>>>> >>>>> ??? We can probably support options such as main module and module >>>>> path >>>>> ??? in a future RFE. >>>>> >>>>> [2] In the current JDK implementation, there is no single object >>>>> ??? that represents "the module graph". Most of the information >>>>> ??? is stored in the archive bootLayer object, but a few additional >>>>> ??? restoration operations need to be performed: >>>>> >>>>> ??? + See ModuleBootstrap.getArchivedBootLayer() >>>>> ??? + Some static fields need to be archived/restored in >>>>> ????? Module.java, BuiltinClassLoader.java, ClassLoaders.java >>>>> ????? and BootLoader.java >>>>> >>>>> [3] I ran into a complication with two loader instances of >>>>> ??? PlatformClassLoader and AppClassLoader. They are stored in >>>>> ??? multiple tables inside the module graph (e.g., >>>>> ??? BuiltinClassLoader$LoadedModule) so I cannot easily recreate >>>>> ??? them at runtime. >>>>> >>>>> ??? However, these two loaders contain information specific to the >>>>> ??? dump time VM lifecycle (such as the classes that were loaded >>>>> ??? during CDS dumping) that need to be scrubbed. I couldn't find an >>>>> ??? elegant way of doing this, so I added a private >>>>> "resetArchivedStates" >>>>> ??? method to a few classes. They are called inside >>>>> ??? HeapShared::reset_archived_object_states(). >>>>> >>>>> [4] Related native data structures (PackageEntry and ModuleEntry) >>>>> ??? are also archived. Start with classLoaderData.cpp >>>>> >>>>> Passes mach5 tiers 1-4. I will test with additional tiers. >>>>> >>>>> Thanks >>>>> - Ioi >>>> >>> >> > From coleen.phillimore at oracle.com Mon Aug 31 22:22:42 2020 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 31 Aug 2020 18:22:42 -0400 Subject: RFR(S): 8251860: ClassLoaderData::loaded_classes_do fails with "assert(ZAddress::is_marked(addr)) failed: Should be marked" In-Reply-To: <93a76872-34f9-1462-c341-515047671280@oracle.com> References: <9217b9d3-4be7-b4f4-7be3-e4473fe18cf9@oracle.com> <1a85f6c1-76b1-61a3-c188-aaa7413ed04e@oracle.com> <0d0bdd73-7f9d-4309-a670-4a9d81608bdf@oracle.com> <93a76872-34f9-1462-c341-515047671280@oracle.com> Message-ID: <46baf7ab-b770-5871-472b-56ff7f58eff6@oracle.com> On 8/31/20 11:57 AM, calvin.cheung at oracle.com wrote: > Hi Ioi, > > Thanks for your review! > > On 8/28/20 3:54 PM, Ioi Lam wrote: >> Hi Calvin, >> >> ?300 void ClassLoaderData::inc_keep_alive() { >> ?301?? if (has_class_mirror_holder()) { >> ?302???? if (!Arguments::is_dumping_archive()) { >> ?303?????? assert(_keep_alive > 0, "Invalid keep alive increment >> count"); >> ?304???? } >> ?305???? _keep_alive++; >> ?306?? } >> ?307 } >> >> Why is 302 needed? I think in CollectCLDClosure, you wouldn't collect >> any CLD whose _keep_alive count is zero. > > The (!cld->is_unloading()) doesn't mean that the _keep_alive count is > non-zero. > > We currently have a few tests which would hit the assert if line 302 > is not there. One of them is the appcds/dynamicArchive/LinkClassTest. > While the "Parent x = new Child()" is being verified during linking, > the VerificationType::resolve_and_check_assignability will be called. > This will eventually call the SystemDictionary::register_loader but it > will not add a class_loader to the ClassLoaderDataGraph. Prior to this > patch, the _keep_alive will be incremented at 2 places: > > 1) java_lang_Class::set_mirror_module_field > > ?? In this case, the _keep_alive will be incremented during vm startup. > > 2) ClassLoaderData::ClassLoaderData > > ?? _keep_alive((has_class_mirror_holder || h_class_loader.is_null()) ? > 1 : 0), > > Neither of the above code path will be taken for the assignability check. > >> >> I think MetaspaceShared::link_and_cleanup_shared_classes() can be >> simplified so you don't need to scan the CLDs in two passes. >> >> 1393? while (true) { >> ????? ? bool has_linked = false; >> ??????? for (int i = 0; i < _loaded_cld->length(); i++) { >> ????????? ClassLoaderData* cld = _loaded_cld->at(i); >> ????????? for (Klass* klass = cld->klasses(); klass != NULL; klass = >> klass->next_link()) { >> ???????? ?? if (klass->is_instance_klass()) { >> ????????? ? ? InstanceKlass* ik = InstanceKlass::cast(ik); >> ?????????? ?? if (inking_required(ik)) { >> ??????????? ? ? has_linked = link_class_for_cds(ik, THREAD); >> ?? ? ? ?????? } >> ??????????? } >> ????????? } >> ??????? } >> >> ? ? ? ? if (!has_linked) { >> ?? ? ? ?? break; >> ?? ? ?? } >> ??????? // Class linking includes verification which may load more >> classes. >> ? ? ??? // Keep scanning until we have linked no more classes. >> ? ? ? } >> > I think the has_linked assignment should be "has_linked |= > link_class_for_cds(ik, THREAD)" ? > > The above single loop is cleaner but I think it is less efficient. For > the simple case if there's no linking required after the first loop. > It will need to do another full loop to check if all classes don't > need to be linked in order to exit the loop. With the 2 passes > approach, it only needs to inspect the first element of the _klasses > list of each CLD to determine if we can exit the loop. I'm good with > the single loop since CDS dump isn't too performance critical and it > makes the code easier to read. > >> >> Also link_class_for_cds should return the value of >> MetaspaceShared::try_link_class(), so it has the same behavior as >> before. > > Fixed. > > Here's an updated webrev: > > ??? http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.02/ This looks good! Coleen > > thanks, > Calvin >> >> The rest of the changes looks good. >> >> Thanks >> - Ioi >> >> On 8/28/20 10:42 AM, calvin.cheung at oracle.com wrote: >>> Hi Coleen, >>> >>> Thanks for your review. >>> >>> I've re-implemented the fix with your suggestions. >>> >>> Here's an updated webrev: >>> >>> http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.01/ >>> >>> On 8/26/20 3:24 PM, Coleen Phillimore wrote: >>>> >>>> Hi Calvin, >>>> >>>> I think you should change this to call >>>> ClassLoaderDataGraph::loaded_cld_do(&collect_cld) and collect >>>> ClassLoaderData in the GrowableArray, since you then walk through >>>> them again to link the classes in the class loader data. >>> With the ClassLoaderDataGraph::loaded_cld_do approach, some klasses >>> in the _klasses list may already in the link state. So I have to >>> account for that and not link those klasses again. It complicates >>> the fix a little bit. >>>> >>>> Can linking one klass cause a new class to be added to a _klasses >>>> list of a CLD that you've already walked?? It seems like you need >>>> the do() loop in the current code wrapped around walking the clds, >>>> then link the classes until no progress. You can fast exit when you >>>> find a klass linked, I believe, since the classes are added to the >>>> front of the _klasses list. >>> Yes, it is possible. The updated patch will check if the first klass >>> in the _klass list of each CLD hasn't been linked, then it will loop >>> back again to go through all the CLD's _klasses lists. >>>> >>>> Also with this change, you can remove locked_loaded_classes_do() >>>> because it was added for CDS and apparently the comment is wrong. >>>> >>>> // This case can block but cannot do unloading (called from CDS) >>>> void ClassLoaderDataGraph::unlocked_loaded_classes_do(KlassClosure* >>>> klass_closure) { >>>> ? for (ClassLoaderData* cld = _head; cld != NULL; cld = cld->next()) { >>>> ??? cld->loaded_classes_do(klass_closure); >>>> ? } >>>> } >>> >>> I've removed the above. >>> >>> thanks, >>> >>> Calvin >>> >>>> >>>> Thanks, >>>> Coleen >>>> >>>> On 8/26/20 5:35 PM, calvin.cheung at oracle.com wrote: >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8251860 >>>>> >>>>> webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.00/ >>>>> >>>>> Please refer to the bug report for an analysis of the crash. >>>>> >>>>> The proposed fix is to acquire the ClassLoaderDataGraph_lock >>>>> before calling >>>>> ClassLoaderDataGraph::loaded_classes_do(&link_closure). The >>>>> link_closure will not do the actual linking but just to store the >>>>> InstanceKlass'es in an array and increment the keep_alive counter >>>>> of the class_loader_data of each InstanceKlass. After the call to >>>>> ClassLoaderDataGraph::loaded_classes_do, each InstanceKlass in the >>>>> array will be linked and the keep_alive counter of the >>>>> class_loader_data will be decremented. >>>>> >>>>> Testing: >>>>> ??? - tested about 40 times on the MacOS host where the crash was >>>>> seen >>>>> ?????? (before the fix, the crash was seen about once in 20 runs) >>>>> ??? - Tier1 - 4. >>>>> thanks, >>>>> Calvin >>>> >> From david.holmes at oracle.com Mon Aug 31 22:37:11 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 1 Sep 2020 08:37:11 +1000 Subject: Fatal errors when running JCK tests with JDK15/16 debug build In-Reply-To: References: Message-ID: <80c5c750-2a02-9bd9-d0b4-628481c71264@oracle.com> On 1/09/2020 3:00 am, Doerr, Martin wrote: > Hi David, > > thanks for analyzing it. We need to exclude the test for now. Can you file a JCK bug? I can file one on our internal JCK Jira but I'm not sure what the right process is in this case. Thanks, David > Best regards, > Martin > > >> -----Original Message----- >> From: David Holmes >> Sent: Montag, 31. August 2020 04:34 >> To: Doerr, Martin ; serviceability- >> dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net >> Subject: Re: Fatal errors when running JCK tests with JDK15/16 debug build >> >> Hi Martin, >> >> On 29/08/2020 3:53 am, Doerr, Martin wrote: >>> Hi, >>> >>> we have seen the following fatal error more than 50 times since >>> 2020-05-25 in various JCK tests vm/jvmti. >>> >>> fatal error: String conversion failure: [check] ExitLock destroyed >>> >>> --> ?? [check] ExitLock exited >>> >>> (followed by garbage output) >>> >>> 8166358: Re-enable String verification in >>> java_lang_String::create_from_str() >>> >>> was pushed at that date which introduced the call to fatal. >>> >>> Stack (example from linuxppc64le, but also observed on x86 and aarch64): >>> V? [libjvm.so+0xee242c]? java_lang_String::create_from_str(char const*, >>> Thread*) [clone .part.158]+0x51c >>> V? [libjvm.so+0xee2530]? java_lang_String::create_oop_from_str(char >>> const*, Thread*)+0x40 >>> V? [libjvm.so+0x1026a30]? jni_NewStringUTF+0x1e0 >>> C? [libjckjvmti.so+0x3ce4c]? logWrite+0x5c >>> C? [libjckjvmti.so+0x3cd20]? lprintf+0x170 >>> C? [libjckjvmti.so+0x485b8]? gast00104_agent_proc+0x254 >>> V? [libjvm.so+0x1218f0c]? JvmtiAgentThread::call_start_function()+0x24c >>> V? [libjvm.so+0x193a8fc]? JavaThread::thread_main_inner()+0x32c >>> V? [libjvm.so+0x19418a0]? Thread::call_run()+0x160 >>> V? [libjvm.so+0x15c9d0c]? thread_native_entry(Thread*)+0x18c >>> C? [libpthread.so.0+0x9b48]? start_thread+0x108 >>> >>> (Problem could have been there before but without this fatal message.) >>> >>> The messages are generated by: >>> >>> tests/vm/jvmti/GetAllStackTraces/gast001/gast00104/gast00104.c >>> >>> This looks like a race condition. The message changes while the VM >>> creates a String object from it. Has anybody seen this before? >> >> No but ... >> >>> Is it a test problem? I'm not familiar with the lprintf calls in the test. >> >> ... the lprintf is part of the JCK support library (support.c if you >> have access to sources) and it uses a static buffer for the log messages >> and so it not thread-safe. This test creates a thread and both it and >> the main thread call lprintf concurrently. >> >> So this is a JCK test/test-library bug that appears to be exposed by the >> changes made in 8166358. >> >> Cheers, >> David >> ----- >> >>> Best regards, >>> >>> Martin >>> From david.holmes at oracle.com Mon Aug 31 22:55:58 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 1 Sep 2020 08:55:58 +1000 Subject: RFR: 8252589: Code duplication in ParallelSPCleanupTask In-Reply-To: References: Message-ID: Looks good! Nice simpification. Thanks, David On 1/09/2020 2:46 am, Stefan Karlsson wrote: > Hi all, > > Please review this small patch to remove some code duplication in > ParallelSPCleanupTask. > > https://cr.openjdk.java.net/~stefank/8252589/webrev.01 > https://bugs.openjdk.java.net/browse/JDK-8252589 > > I noticed this while reviewing the patches for JEP 376. I think this > makes it more apparent what the individual sub-tasks are doing. > > Thanks, > StefanK From ioi.lam at oracle.com Mon Aug 31 22:56:23 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 31 Aug 2020 15:56:23 -0700 Subject: RFR(S): 8251860: ClassLoaderData::loaded_classes_do fails with "assert(ZAddress::is_marked(addr)) failed: Should be marked" In-Reply-To: <93a76872-34f9-1462-c341-515047671280@oracle.com> References: <9217b9d3-4be7-b4f4-7be3-e4473fe18cf9@oracle.com> <1a85f6c1-76b1-61a3-c188-aaa7413ed04e@oracle.com> <0d0bdd73-7f9d-4309-a670-4a9d81608bdf@oracle.com> <93a76872-34f9-1462-c341-515047671280@oracle.com> Message-ID: <65d28a3b-9db2-8d55-eca5-d3b7cf1b0ec3@oracle.com> Hi Calvin, Thanks for the explanations. The new version looks good to me. Thanks - Ioi On 8/31/20 8:57 AM, calvin.cheung at oracle.com wrote: > Hi Ioi, > > Thanks for your review! > > On 8/28/20 3:54 PM, Ioi Lam wrote: >> Hi Calvin, >> >> ?300 void ClassLoaderData::inc_keep_alive() { >> ?301?? if (has_class_mirror_holder()) { >> ?302???? if (!Arguments::is_dumping_archive()) { >> ?303?????? assert(_keep_alive > 0, "Invalid keep alive increment >> count"); >> ?304???? } >> ?305???? _keep_alive++; >> ?306?? } >> ?307 } >> >> Why is 302 needed? I think in CollectCLDClosure, you wouldn't collect >> any CLD whose _keep_alive count is zero. > > The (!cld->is_unloading()) doesn't mean that the _keep_alive count is > non-zero. > > We currently have a few tests which would hit the assert if line 302 > is not there. One of them is the appcds/dynamicArchive/LinkClassTest. > While the "Parent x = new Child()" is being verified during linking, > the VerificationType::resolve_and_check_assignability will be called. > This will eventually call the SystemDictionary::register_loader but it > will not add a class_loader to the ClassLoaderDataGraph. Prior to this > patch, the _keep_alive will be incremented at 2 places: > > 1) java_lang_Class::set_mirror_module_field > > ?? In this case, the _keep_alive will be incremented during vm startup. > > 2) ClassLoaderData::ClassLoaderData > > ?? _keep_alive((has_class_mirror_holder || h_class_loader.is_null()) ? > 1 : 0), > > Neither of the above code path will be taken for the assignability check. > >> >> I think MetaspaceShared::link_and_cleanup_shared_classes() can be >> simplified so you don't need to scan the CLDs in two passes. >> >> 1393? while (true) { >> ????? ? bool has_linked = false; >> ??????? for (int i = 0; i < _loaded_cld->length(); i++) { >> ????????? ClassLoaderData* cld = _loaded_cld->at(i); >> ????????? for (Klass* klass = cld->klasses(); klass != NULL; klass = >> klass->next_link()) { >> ???????? ?? if (klass->is_instance_klass()) { >> ????????? ? ? InstanceKlass* ik = InstanceKlass::cast(ik); >> ?????????? ?? if (inking_required(ik)) { >> ??????????? ? ? has_linked = link_class_for_cds(ik, THREAD); >> ?? ? ? ?????? } >> ??????????? } >> ????????? } >> ??????? } >> >> ? ? ? ? if (!has_linked) { >> ?? ? ? ?? break; >> ?? ? ?? } >> ??????? // Class linking includes verification which may load more >> classes. >> ? ? ??? // Keep scanning until we have linked no more classes. >> ? ? ? } >> > I think the has_linked assignment should be "has_linked |= > link_class_for_cds(ik, THREAD)" ? > > The above single loop is cleaner but I think it is less efficient. For > the simple case if there's no linking required after the first loop. > It will need to do another full loop to check if all classes don't > need to be linked in order to exit the loop. With the 2 passes > approach, it only needs to inspect the first element of the _klasses > list of each CLD to determine if we can exit the loop. I'm good with > the single loop since CDS dump isn't too performance critical and it > makes the code easier to read. > >> >> Also link_class_for_cds should return the value of >> MetaspaceShared::try_link_class(), so it has the same behavior as >> before. > > Fixed. > > Here's an updated webrev: > > ??? http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.02/ > > thanks, > Calvin >> >> The rest of the changes looks good. >> >> Thanks >> - Ioi >> >> On 8/28/20 10:42 AM, calvin.cheung at oracle.com wrote: >>> Hi Coleen, >>> >>> Thanks for your review. >>> >>> I've re-implemented the fix with your suggestions. >>> >>> Here's an updated webrev: >>> >>> http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.01/ >>> >>> On 8/26/20 3:24 PM, Coleen Phillimore wrote: >>>> >>>> Hi Calvin, >>>> >>>> I think you should change this to call >>>> ClassLoaderDataGraph::loaded_cld_do(&collect_cld) and collect >>>> ClassLoaderData in the GrowableArray, since you then walk through >>>> them again to link the classes in the class loader data. >>> With the ClassLoaderDataGraph::loaded_cld_do approach, some klasses >>> in the _klasses list may already in the link state. So I have to >>> account for that and not link those klasses again. It complicates >>> the fix a little bit. >>>> >>>> Can linking one klass cause a new class to be added to a _klasses >>>> list of a CLD that you've already walked?? It seems like you need >>>> the do() loop in the current code wrapped around walking the clds, >>>> then link the classes until no progress. You can fast exit when you >>>> find a klass linked, I believe, since the classes are added to the >>>> front of the _klasses list. >>> Yes, it is possible. The updated patch will check if the first klass >>> in the _klass list of each CLD hasn't been linked, then it will loop >>> back again to go through all the CLD's _klasses lists. >>>> >>>> Also with this change, you can remove locked_loaded_classes_do() >>>> because it was added for CDS and apparently the comment is wrong. >>>> >>>> // This case can block but cannot do unloading (called from CDS) >>>> void ClassLoaderDataGraph::unlocked_loaded_classes_do(KlassClosure* >>>> klass_closure) { >>>> ? for (ClassLoaderData* cld = _head; cld != NULL; cld = cld->next()) { >>>> ??? cld->loaded_classes_do(klass_closure); >>>> ? } >>>> } >>> >>> I've removed the above. >>> >>> thanks, >>> >>> Calvin >>> >>>> >>>> Thanks, >>>> Coleen >>>> >>>> On 8/26/20 5:35 PM, calvin.cheung at oracle.com wrote: >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8251860 >>>>> >>>>> webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.00/ >>>>> >>>>> Please refer to the bug report for an analysis of the crash. >>>>> >>>>> The proposed fix is to acquire the ClassLoaderDataGraph_lock >>>>> before calling >>>>> ClassLoaderDataGraph::loaded_classes_do(&link_closure). The >>>>> link_closure will not do the actual linking but just to store the >>>>> InstanceKlass'es in an array and increment the keep_alive counter >>>>> of the class_loader_data of each InstanceKlass. After the call to >>>>> ClassLoaderDataGraph::loaded_classes_do, each InstanceKlass in the >>>>> array will be linked and the keep_alive counter of the >>>>> class_loader_data will be decremented. >>>>> >>>>> Testing: >>>>> ??? - tested about 40 times on the MacOS host where the crash was >>>>> seen >>>>> ?????? (before the fix, the crash was seen about once in 20 runs) >>>>> ??? - Tier1 - 4. >>>>> thanks, >>>>> Calvin >>>> >> From david.holmes at oracle.com Mon Aug 31 23:05:04 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 1 Sep 2020 09:05:04 +1000 Subject: RFR(M) 8252526 Remove excessive inclusion of jvmti.h and jvmtiExport.hpp In-Reply-To: References: Message-ID: <610aa37d-3f5e-cae7-197a-b3294b217aac@oracle.com> Hi Ioi, I haven't looked at the code changes ... On 1/09/2020 4:13 am, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8252526 > http://cr.openjdk.java.net/~iklam/jdk16/8252526-fix-jvmti-hpp.v01/ > > (I marked this RFR as "M" because 63 files have changed, but most of the > are > just adding a missing #include "prims/jvmtiExport.hpp"). > > jvmti.h is included 905 times and jvmtiExport.hpp is included 776 times > (by 971 hotspot .o files). Most of these are unnecessarily included by the > following 3 popular header files: > > [1] javaClasses.hpp: ThreadStatus is rarely used, and should be moved > ? ? to javaThreadStatus.hpp. I also converted the enum to an C++ 11 > ? ? enum class for better type safety. (see also JDK-8247938). > > [2] os.hpp: No need to include jvm.h. Use forward declaration > ? ? "typedef struct _jvmtiTimerInfo jvmtiTimerInfo;" instead. That does not seem reasonable to me. It is one thing to do a simple forward declaration of a class but this is an internal detail of JVMTI which os.hpp has no business knowing about. > [3] thread.hpp: No need to include jvmExport.hpp. Use forward declaration > ? ? for JvmtiSampledObjectAllocEventCollector and > ? ? JvmtiVMObjectAllocEventCollector instead. > > > The total number of includes have reduced from 252033 to 250001. Build > time of > slow-debug libjvm.so is reduced from 2:07 to 2:04 on my machine. I'm not sure why we really care. The build times are not that problematic that it warrants some of these header file hacks IMO. Are those figures with or without PCH? David ----- > > Thanks > - Ioi > From calvin.cheung at oracle.com Mon Aug 31 23:34:27 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Mon, 31 Aug 2020 16:34:27 -0700 Subject: RFR(S): 8251860: ClassLoaderData::loaded_classes_do fails with "assert(ZAddress::is_marked(addr)) failed: Should be marked" In-Reply-To: <46baf7ab-b770-5871-472b-56ff7f58eff6@oracle.com> References: <9217b9d3-4be7-b4f4-7be3-e4473fe18cf9@oracle.com> <1a85f6c1-76b1-61a3-c188-aaa7413ed04e@oracle.com> <0d0bdd73-7f9d-4309-a670-4a9d81608bdf@oracle.com> <93a76872-34f9-1462-c341-515047671280@oracle.com> <46baf7ab-b770-5871-472b-56ff7f58eff6@oracle.com> Message-ID: <3b422ab5-0249-17d7-123d-d01ef1af8269@oracle.com> Hi Coleen, Thanks for taking another look. Calvin On 8/31/20 3:22 PM, Coleen Phillimore wrote: > > > On 8/31/20 11:57 AM, calvin.cheung at oracle.com wrote: >> Hi Ioi, >> >> Thanks for your review! >> >> On 8/28/20 3:54 PM, Ioi Lam wrote: >>> Hi Calvin, >>> >>> ?300 void ClassLoaderData::inc_keep_alive() { >>> ?301?? if (has_class_mirror_holder()) { >>> ?302???? if (!Arguments::is_dumping_archive()) { >>> ?303?????? assert(_keep_alive > 0, "Invalid keep alive increment >>> count"); >>> ?304???? } >>> ?305???? _keep_alive++; >>> ?306?? } >>> ?307 } >>> >>> Why is 302 needed? I think in CollectCLDClosure, you wouldn't >>> collect any CLD whose _keep_alive count is zero. >> >> The (!cld->is_unloading()) doesn't mean that the _keep_alive count is >> non-zero. >> >> We currently have a few tests which would hit the assert if line 302 >> is not there. One of them is the appcds/dynamicArchive/LinkClassTest. >> While the "Parent x = new Child()" is being verified during linking, >> the VerificationType::resolve_and_check_assignability will be called. >> This will eventually call the SystemDictionary::register_loader but >> it will not add a class_loader to the ClassLoaderDataGraph. Prior to >> this patch, the _keep_alive will be incremented at 2 places: >> >> 1) java_lang_Class::set_mirror_module_field >> >> ?? In this case, the _keep_alive will be incremented during vm startup. >> >> 2) ClassLoaderData::ClassLoaderData >> >> ?? _keep_alive((has_class_mirror_holder || h_class_loader.is_null()) >> ? 1 : 0), >> >> Neither of the above code path will be taken for the assignability >> check. >> >>> >>> I think MetaspaceShared::link_and_cleanup_shared_classes() can be >>> simplified so you don't need to scan the CLDs in two passes. >>> >>> 1393? while (true) { >>> ????? ? bool has_linked = false; >>> ??????? for (int i = 0; i < _loaded_cld->length(); i++) { >>> ????????? ClassLoaderData* cld = _loaded_cld->at(i); >>> ????????? for (Klass* klass = cld->klasses(); klass != NULL; klass = >>> klass->next_link()) { >>> ???????? ?? if (klass->is_instance_klass()) { >>> ????????? ? ? InstanceKlass* ik = InstanceKlass::cast(ik); >>> ?????????? ?? if (inking_required(ik)) { >>> ??????????? ? ? has_linked = link_class_for_cds(ik, THREAD); >>> ?? ? ? ?????? } >>> ??????????? } >>> ????????? } >>> ??????? } >>> >>> ? ? ? ? if (!has_linked) { >>> ?? ? ? ?? break; >>> ?? ? ?? } >>> ??????? // Class linking includes verification which may load more >>> classes. >>> ? ? ??? // Keep scanning until we have linked no more classes. >>> ? ? ? } >>> >> I think the has_linked assignment should be "has_linked |= >> link_class_for_cds(ik, THREAD)" ? >> >> The above single loop is cleaner but I think it is less efficient. >> For the simple case if there's no linking required after the first >> loop. It will need to do another full loop to check if all classes >> don't need to be linked in order to exit the loop. With the 2 passes >> approach, it only needs to inspect the first element of the _klasses >> list of each CLD to determine if we can exit the loop. I'm good with >> the single loop since CDS dump isn't too performance critical and it >> makes the code easier to read. >> >>> >>> Also link_class_for_cds should return the value of >>> MetaspaceShared::try_link_class(), so it has the same behavior as >>> before. >> >> Fixed. >> >> Here's an updated webrev: >> >> ??? http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.02/ > > This looks good! > Coleen >> >> thanks, >> Calvin >>> >>> The rest of the changes looks good. >>> >>> Thanks >>> - Ioi >>> >>> On 8/28/20 10:42 AM, calvin.cheung at oracle.com wrote: >>>> Hi Coleen, >>>> >>>> Thanks for your review. >>>> >>>> I've re-implemented the fix with your suggestions. >>>> >>>> Here's an updated webrev: >>>> >>>> http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.01/ >>>> >>>> On 8/26/20 3:24 PM, Coleen Phillimore wrote: >>>>> >>>>> Hi Calvin, >>>>> >>>>> I think you should change this to call >>>>> ClassLoaderDataGraph::loaded_cld_do(&collect_cld) and collect >>>>> ClassLoaderData in the GrowableArray, since you then walk through >>>>> them again to link the classes in the class loader data. >>>> With the ClassLoaderDataGraph::loaded_cld_do approach, some klasses >>>> in the _klasses list may already in the link state. So I have to >>>> account for that and not link those klasses again. It complicates >>>> the fix a little bit. >>>>> >>>>> Can linking one klass cause a new class to be added to a _klasses >>>>> list of a CLD that you've already walked?? It seems like you need >>>>> the do() loop in the current code wrapped around walking the clds, >>>>> then link the classes until no progress. You can fast exit when >>>>> you find a klass linked, I believe, since the classes are added to >>>>> the front of the _klasses list. >>>> Yes, it is possible. The updated patch will check if the first >>>> klass in the _klass list of each CLD hasn't been linked, then it >>>> will loop back again to go through all the CLD's _klasses lists. >>>>> >>>>> Also with this change, you can remove locked_loaded_classes_do() >>>>> because it was added for CDS and apparently the comment is wrong. >>>>> >>>>> // This case can block but cannot do unloading (called from CDS) >>>>> void >>>>> ClassLoaderDataGraph::unlocked_loaded_classes_do(KlassClosure* >>>>> klass_closure) { >>>>> ? for (ClassLoaderData* cld = _head; cld != NULL; cld = >>>>> cld->next()) { >>>>> ??? cld->loaded_classes_do(klass_closure); >>>>> ? } >>>>> } >>>> >>>> I've removed the above. >>>> >>>> thanks, >>>> >>>> Calvin >>>> >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 8/26/20 5:35 PM, calvin.cheung at oracle.com wrote: >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8251860 >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.00/ >>>>>> >>>>>> Please refer to the bug report for an analysis of the crash. >>>>>> >>>>>> The proposed fix is to acquire the ClassLoaderDataGraph_lock >>>>>> before calling >>>>>> ClassLoaderDataGraph::loaded_classes_do(&link_closure). The >>>>>> link_closure will not do the actual linking but just to store the >>>>>> InstanceKlass'es in an array and increment the keep_alive counter >>>>>> of the class_loader_data of each InstanceKlass. After the call to >>>>>> ClassLoaderDataGraph::loaded_classes_do, each InstanceKlass in >>>>>> the array will be linked and the keep_alive counter of the >>>>>> class_loader_data will be decremented. >>>>>> >>>>>> Testing: >>>>>> ??? - tested about 40 times on the MacOS host where the crash was >>>>>> seen >>>>>> ?????? (before the fix, the crash was seen about once in 20 runs) >>>>>> ??? - Tier1 - 4. >>>>>> thanks, >>>>>> Calvin >>>>> >>> > From calvin.cheung at oracle.com Mon Aug 31 23:36:36 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Mon, 31 Aug 2020 16:36:36 -0700 Subject: RFR(S): 8251860: ClassLoaderData::loaded_classes_do fails with "assert(ZAddress::is_marked(addr)) failed: Should be marked" In-Reply-To: <65d28a3b-9db2-8d55-eca5-d3b7cf1b0ec3@oracle.com> References: <9217b9d3-4be7-b4f4-7be3-e4473fe18cf9@oracle.com> <1a85f6c1-76b1-61a3-c188-aaa7413ed04e@oracle.com> <0d0bdd73-7f9d-4309-a670-4a9d81608bdf@oracle.com> <93a76872-34f9-1462-c341-515047671280@oracle.com> <65d28a3b-9db2-8d55-eca5-d3b7cf1b0ec3@oracle.com> Message-ID: <43050672-9e96-082c-4e3f-72a9e641081d@oracle.com> Thanks, Ioi! On 8/31/20 3:56 PM, Ioi Lam wrote: > Hi Calvin, > > Thanks for the explanations. The new version looks good to me. > > Thanks > - Ioi > > On 8/31/20 8:57 AM, calvin.cheung at oracle.com wrote: >> Hi Ioi, >> >> Thanks for your review! >> >> On 8/28/20 3:54 PM, Ioi Lam wrote: >>> Hi Calvin, >>> >>> ?300 void ClassLoaderData::inc_keep_alive() { >>> ?301?? if (has_class_mirror_holder()) { >>> ?302???? if (!Arguments::is_dumping_archive()) { >>> ?303?????? assert(_keep_alive > 0, "Invalid keep alive increment >>> count"); >>> ?304???? } >>> ?305???? _keep_alive++; >>> ?306?? } >>> ?307 } >>> >>> Why is 302 needed? I think in CollectCLDClosure, you wouldn't >>> collect any CLD whose _keep_alive count is zero. >> >> The (!cld->is_unloading()) doesn't mean that the _keep_alive count is >> non-zero. >> >> We currently have a few tests which would hit the assert if line 302 >> is not there. One of them is the appcds/dynamicArchive/LinkClassTest. >> While the "Parent x = new Child()" is being verified during linking, >> the VerificationType::resolve_and_check_assignability will be called. >> This will eventually call the SystemDictionary::register_loader but >> it will not add a class_loader to the ClassLoaderDataGraph. Prior to >> this patch, the _keep_alive will be incremented at 2 places: >> >> 1) java_lang_Class::set_mirror_module_field >> >> ?? In this case, the _keep_alive will be incremented during vm startup. >> >> 2) ClassLoaderData::ClassLoaderData >> >> ?? _keep_alive((has_class_mirror_holder || h_class_loader.is_null()) >> ? 1 : 0), >> >> Neither of the above code path will be taken for the assignability >> check. >> >>> >>> I think MetaspaceShared::link_and_cleanup_shared_classes() can be >>> simplified so you don't need to scan the CLDs in two passes. >>> >>> 1393? while (true) { >>> ????? ? bool has_linked = false; >>> ??????? for (int i = 0; i < _loaded_cld->length(); i++) { >>> ????????? ClassLoaderData* cld = _loaded_cld->at(i); >>> ????????? for (Klass* klass = cld->klasses(); klass != NULL; klass = >>> klass->next_link()) { >>> ???????? ?? if (klass->is_instance_klass()) { >>> ????????? ? ? InstanceKlass* ik = InstanceKlass::cast(ik); >>> ?????????? ?? if (inking_required(ik)) { >>> ??????????? ? ? has_linked = link_class_for_cds(ik, THREAD); >>> ?? ? ? ?????? } >>> ??????????? } >>> ????????? } >>> ??????? } >>> >>> ? ? ? ? if (!has_linked) { >>> ?? ? ? ?? break; >>> ?? ? ?? } >>> ??????? // Class linking includes verification which may load more >>> classes. >>> ? ? ??? // Keep scanning until we have linked no more classes. >>> ? ? ? } >>> >> I think the has_linked assignment should be "has_linked |= >> link_class_for_cds(ik, THREAD)" ? >> >> The above single loop is cleaner but I think it is less efficient. >> For the simple case if there's no linking required after the first >> loop. It will need to do another full loop to check if all classes >> don't need to be linked in order to exit the loop. With the 2 passes >> approach, it only needs to inspect the first element of the _klasses >> list of each CLD to determine if we can exit the loop. I'm good with >> the single loop since CDS dump isn't too performance critical and it >> makes the code easier to read. >> >>> >>> Also link_class_for_cds should return the value of >>> MetaspaceShared::try_link_class(), so it has the same behavior as >>> before. >> >> Fixed. >> >> Here's an updated webrev: >> >> ??? http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.02/ >> >> thanks, >> Calvin >>> >>> The rest of the changes looks good. >>> >>> Thanks >>> - Ioi >>> >>> On 8/28/20 10:42 AM, calvin.cheung at oracle.com wrote: >>>> Hi Coleen, >>>> >>>> Thanks for your review. >>>> >>>> I've re-implemented the fix with your suggestions. >>>> >>>> Here's an updated webrev: >>>> >>>> http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.01/ >>>> >>>> On 8/26/20 3:24 PM, Coleen Phillimore wrote: >>>>> >>>>> Hi Calvin, >>>>> >>>>> I think you should change this to call >>>>> ClassLoaderDataGraph::loaded_cld_do(&collect_cld) and collect >>>>> ClassLoaderData in the GrowableArray, since you then walk through >>>>> them again to link the classes in the class loader data. >>>> With the ClassLoaderDataGraph::loaded_cld_do approach, some klasses >>>> in the _klasses list may already in the link state. So I have to >>>> account for that and not link those klasses again. It complicates >>>> the fix a little bit. >>>>> >>>>> Can linking one klass cause a new class to be added to a _klasses >>>>> list of a CLD that you've already walked?? It seems like you need >>>>> the do() loop in the current code wrapped around walking the clds, >>>>> then link the classes until no progress. You can fast exit when >>>>> you find a klass linked, I believe, since the classes are added to >>>>> the front of the _klasses list. >>>> Yes, it is possible. The updated patch will check if the first >>>> klass in the _klass list of each CLD hasn't been linked, then it >>>> will loop back again to go through all the CLD's _klasses lists. >>>>> >>>>> Also with this change, you can remove locked_loaded_classes_do() >>>>> because it was added for CDS and apparently the comment is wrong. >>>>> >>>>> // This case can block but cannot do unloading (called from CDS) >>>>> void >>>>> ClassLoaderDataGraph::unlocked_loaded_classes_do(KlassClosure* >>>>> klass_closure) { >>>>> ? for (ClassLoaderData* cld = _head; cld != NULL; cld = >>>>> cld->next()) { >>>>> ??? cld->loaded_classes_do(klass_closure); >>>>> ? } >>>>> } >>>> >>>> I've removed the above. >>>> >>>> thanks, >>>> >>>> Calvin >>>> >>>>> >>>>> Thanks, >>>>> Coleen >>>>> >>>>> On 8/26/20 5:35 PM, calvin.cheung at oracle.com wrote: >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8251860 >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~ccheung/jdk16/8251860/webrev.00/ >>>>>> >>>>>> Please refer to the bug report for an analysis of the crash. >>>>>> >>>>>> The proposed fix is to acquire the ClassLoaderDataGraph_lock >>>>>> before calling >>>>>> ClassLoaderDataGraph::loaded_classes_do(&link_closure). The >>>>>> link_closure will not do the actual linking but just to store the >>>>>> InstanceKlass'es in an array and increment the keep_alive counter >>>>>> of the class_loader_data of each InstanceKlass. After the call to >>>>>> ClassLoaderDataGraph::loaded_classes_do, each InstanceKlass in >>>>>> the array will be linked and the keep_alive counter of the >>>>>> class_loader_data will be decremented. >>>>>> >>>>>> Testing: >>>>>> ??? - tested about 40 times on the MacOS host where the crash was >>>>>> seen >>>>>> ?????? (before the fix, the crash was seen about once in 20 runs) >>>>>> ??? - Tier1 - 4. >>>>>> thanks, >>>>>> Calvin >>>>> >>> > From Bruno.Borges at microsoft.com Sun Aug 30 22:59:40 2020 From: Bruno.Borges at microsoft.com (Bruno Borges) Date: Sun, 30 Aug 2020 22:59:40 +0000 Subject: Regression in JDK15/16: CGroup v2 support Message-ID: <260D8760-CB66-43FE-8870-1AD7A2A6336E@microsoft.com> Hi all, Just wanted to check if anyone at Oracle had a chance to review bug 9066610 that was submitted last week. We would like to continue our discussion and a proposed fix for evaluation. ?On 2020-08-26, 12:06 AM, "David Holmes" wrote: Please note that the discuss at openjdk.java.net mailing list is not the appropriate place to discuss this kind of issue. hotspot-runtime-dev would be the correct place to discuss this. Thanks, David On 25/08/2020 7:29 pm, Bruno Borges wrote: > No worries. > > I am not aware of any other way for non-OpenJDK Authors to submit a bug to OpenJDK except through bugreport.java.com. If there is, happy to follow that for any future issue. > > bb. > > On 2020-08-25, 1:38 AM, "Severin Gehwolf" wrote: > > Hi Bruno, > > On Tue, 2020-08-25 at 08:05 +0000, Bruno Borges wrote: > > Hi Severin, > > > > Issue created: 9066610. > > Thanks. We might need somebody from Oracle to push this one through to > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.java.net%2F&data=02%7C01%7CBruno.Borges%40microsoft.com%7C60dffad37c904298679408d8498e85f3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637340223777936292&sdata=upvvC%2BHc49Il6L%2B6koJYhVj2hwJhPxDBlApIwKH6pN0%3D&reserved=0. 9XXXX bugs are AFAIK created by the web- > interface and need active triage to show up as JDK-8XXXX bugs. > > > Charlie has a fix to propose. > > Great! Looking forward to it. > > Thanks, > Severin > >